starred/fake-gcs-server

Fork 0

mirror of https://github.com/fsouza/fake-gcs-server.git synced 2026-04-25 21:55:56 +03:00

[GH-ISSUE #280] [go-example] object doesn't exist #2207

New issue

Closed

opened 2026-03-15 18:01:34 +03:00 by kerem · 9 comments

kerem commented

2026-03-15 18:01:34 +03:00

Owner

Originally created by @ncarenton on GitHub (Jun 19, 2020).
Original GitHub issue: https://github.com/fsouza/fake-gcs-server/issues/280

Thanks for this project!

I can not make the go-example work, it finds the bucket but not the object:

docker run -d --name fake-gcs-server -p 4443:4443 -v ${PWD}/examples/data:/data fsouza/fake-gcs-server
763d400668bf735e067fc61b0983a49e4393eb673ae9473379e961faf050a90b
cd examples/go
go build
./go-example 
buckets: [sample-bucket]
2020/06/19 19:31:47 storage: object doesn't exist

The strange thing is that curl is working fine so the object is actually there:

 curl --insecure https://127.0.0.1:4443/storage/v1/b/sample-bucket/o
{"kind":"storage#objects","items":[{"kind":"storage#object","name":"some_file.txt","id":"sample-bucket/some_file.txt","bucket":"sample-bucket","size":"33","timeCreated":"2020-06-19T17:41:33Z","timeDeleted":"0001-01-01T00:00:00Z","updated":"2020-06-19T17:41:33Z","generation":"0"}]}

It is also working fine with the python example:

 python python.py 
Bucket: sample-bucket

Blob: some_file.txt
b'Some amazing content to be loaded'

Originally created by @ncarenton on GitHub (Jun 19, 2020). Original GitHub issue: https://github.com/fsouza/fake-gcs-server/issues/280 Thanks for this project! I can not make the go-example work, it finds the bucket but not the object: ``` docker run -d --name fake-gcs-server -p 4443:4443 -v ${PWD}/examples/data:/data fsouza/fake-gcs-server 763d400668bf735e067fc61b0983a49e4393eb673ae9473379e961faf050a90b cd examples/go go build ./go-example buckets: [sample-bucket] 2020/06/19 19:31:47 storage: object doesn't exist ``` The strange thing is that curl is working fine so the object is actually there: ``` curl --insecure https://127.0.0.1:4443/storage/v1/b/sample-bucket/o {"kind":"storage#objects","items":[{"kind":"storage#object","name":"some_file.txt","id":"sample-bucket/some_file.txt","bucket":"sample-bucket","size":"33","timeCreated":"2020-06-19T17:41:33Z","timeDeleted":"0001-01-01T00:00:00Z","updated":"2020-06-19T17:41:33Z","generation":"0"}]} ``` It is also working fine with the python example: ``` python python.py Bucket: sample-bucket Blob: some_file.txt b'Some amazing content to be loaded' ```

kerem

2026-03-15 18:01:34 +03:00

closed this issue
added the
question
label

kerem commented

2026-03-15 18:01:50 +03:00

Author

Owner

@fsouza commented on GitHub (Jun 21, 2020):

Hey @ncarenton, thanks for reporting and apologies for the delayed response. Not sure if you noticed this comment in the example file: github.com/fsouza/fake-gcs-server@c822c3b926/examples/go/main.go (L5-L9)

(I feel like we can probably move that comment elsewhere in the file, it's easy to miss it as part of the package statement)

In order for the downloads to work in the way the Go example expects, the server needs to be started with some flags.

@fsouza commented on GitHub (Jun 21, 2020): Hey @ncarenton, thanks for reporting and apologies for the delayed response. Not sure if you noticed this comment in the example file: https://github.com/fsouza/fake-gcs-server/blob/c822c3b926f90755b38471be4b77d422d3623007/examples/go/main.go#L5-L9 (I feel like we can probably move that comment elsewhere in the file, it's easy to miss it as part of the package statement) In order for the downloads to work in the way the Go example expects, the server needs to be started with some flags.

kerem commented

2026-03-15 18:01:55 +03:00

Author

Owner

@ncarenton commented on GitHub (Jun 21, 2020):

Hi @fsouza, no worries for the short delay :)

I actually did miss this comment, it could indeed be moved to downloadFile() for example, for better visibility.

Starting my docker container with -public-host does fix my issue! Still I would like to ask if it is possible to use a regular GCS client with the fake server, meaning without these 2 arguments: option.WithEndpoint("https://storage.gcs.127.0.0.1.nip.io:4443/storage/v1/"), option.WithHTTPClient(httpClient) ? Is it possible to do that and set STORAGE_EMULATOR_HOST env var so we start the fake server without -public-host then ? I get the feeling it would be simpler.

Many thanks for your help

@ncarenton commented on GitHub (Jun 21, 2020): Hi @fsouza, no worries for the short delay :) I actually did miss this comment, it could indeed be moved to `downloadFile()` for example, for better visibility. Starting my docker container with `-public-host` does fix my issue! Still I would like to ask if it is possible to use a regular GCS client with the fake server, meaning without these 2 arguments: `option.WithEndpoint("https://storage.gcs.127.0.0.1.nip.io:4443/storage/v1/"), option.WithHTTPClient(httpClient)` ? Is it possible to do that and set `STORAGE_EMULATOR_HOST ` env var so we start the fake server without ` -public-host` then ? I get the feeling it would be simpler. Many thanks for your help

kerem commented

2026-03-15 18:02:00 +03:00

Author

Owner

@fsouza commented on GitHub (Jun 22, 2020):

I don't know if the Go sdk supports STORAGE_EMULATOR_HOST 🤔 Does it?

If the sdk supports that env var, we could support it in the emulator too, or figure out a way to bind to any host (not sure if it's possible because of conflicting paths).

@fsouza commented on GitHub (Jun 22, 2020): I don't know if the Go sdk supports `STORAGE_EMULATOR_HOST` 🤔 Does it? If the sdk supports that env var, we could support it in the emulator too, or figure out a way to bind to any host (not sure if it's possible because of conflicting paths).

kerem commented

2026-03-15 18:02:05 +03:00

Author

Owner

@ncarenton commented on GitHub (Jun 22, 2020):

I guess so: https://github.com/googleapis/google-cloud-go/blob/master/storage/storage.go#L104

@ncarenton commented on GitHub (Jun 22, 2020): I guess so: https://github.com/googleapis/google-cloud-go/blob/master/storage/storage.go#L104

kerem commented

2026-03-15 18:02:10 +03:00

Author

Owner

@fsouza commented on GitHub (Jun 22, 2020):

Nice, in that case we need to look for that variable and if it's defined, bind in the proper port and use HTTP instead of HTTPs.

@fsouza commented on GitHub (Jun 22, 2020): Nice, in that case we need to look for that variable and if it's defined, bind in the proper port and use HTTP instead of HTTPs.

kerem commented

2026-03-15 18:02:16 +03:00

Author

Owner

@yiyue115 commented on GitHub (Sep 30, 2020):

Hello 😃 Actually I faced a similar issue as well cuz I missed that comment 😛
Is there any update on this one?

@yiyue115 commented on GitHub (Sep 30, 2020): Hello 😃 Actually I faced a similar issue as well cuz I missed that comment 😛 Is there any update on this one?

kerem commented

2026-03-15 18:02:21 +03:00

Author

Owner

@OrangeFlag commented on GitHub (Dec 8, 2020):

Hello, in my case I have set the correct external-url and public-host for my docker container, but when I call fake-gcs from the local machine, I also get a 404

I used wireshark and realized that the requests differ in the Host field and response in Location field.
Wouldn't it be nice to reply based on the Host being sent and remove the external-url and public-host?
So it would be possible to call the service from any environment

Thanks for your work!

TLDR

docker-compose for fast-gcs

gcs:
    image: fsouza/fake-gcs-server
    command: -backend memory -scheme http -port 4443 -public-host gcs:4443 -external-url http://gcs:4443
    ports:
      - 4443:4443

Fast fix to request fake-gcs from localhost:

storage.NewClient(ctx,
		option.WithEndpoint("http://localhost:4443/storage/v1/"),
		option.WithoutAuthentication(),
		option.WithHTTPClient(&http.Client{
			Transport: &HostFixRoundTripper{&http.Transport{}},
		}))
//...
//...
type HostFixRoundTripper struct {
	Proxy http.RoundTripper
}

func (l HostFixRoundTripper) RoundTrip(request *http.Request) (*http.Response, error) {
	request.Host = "gcs:4443"
	res, err := l.Proxy.RoundTrip(request)
	if res != nil {
		location := res.Header.Get("Location")
		if len(location) != 0 {
			res.Header.Set("Location", strings.Replace(location, "gcs", "localhost", 1))
		}
	}
	return res, err
}

@OrangeFlag commented on GitHub (Dec 8, 2020): Hello, in my case I have set the correct external-url and public-host for my docker container, but when I call fake-gcs from the local machine, I also get a 404 I used wireshark and realized that the requests differ in the Host field and response in Location field. Wouldn't it be nice to reply based on the Host being sent and remove the external-url and public-host? So it would be possible to call the service from any environment Thanks for your work! TLDR docker-compose for fast-gcs ```docker-compose gcs: image: fsouza/fake-gcs-server command: -backend memory -scheme http -port 4443 -public-host gcs:4443 -external-url http://gcs:4443 ports: - 4443:4443 ``` Fast fix to request fake-gcs from localhost: ```golang storage.NewClient(ctx, option.WithEndpoint("http://localhost:4443/storage/v1/"), option.WithoutAuthentication(), option.WithHTTPClient(&http.Client{ Transport: &HostFixRoundTripper{&http.Transport{}}, })) //... //... type HostFixRoundTripper struct { Proxy http.RoundTripper } func (l HostFixRoundTripper) RoundTrip(request *http.Request) (*http.Response, error) { request.Host = "gcs:4443" res, err := l.Proxy.RoundTrip(request) if res != nil { location := res.Header.Get("Location") if len(location) != 0 { res.Header.Set("Location", strings.Replace(location, "gcs", "localhost", 1)) } } return res, err } ```

kerem commented

2026-03-15 18:02:26 +03:00

Author

Owner

@sfro commented on GitHub (Mar 16, 2021):

I'm not able to create objects no matter what public-host I put 🤔 logs show that the request isn't even reaching the server.

Using cloud.google.com/go/storage v1.14.0.

@sfro commented on GitHub (Mar 16, 2021): I'm not able to create objects no matter what public-host I put 🤔 logs show that the request isn't even reaching the server. Using cloud.google.com/go/storage v1.14.0.

kerem commented

2026-03-15 18:02:31 +03:00

Author

Owner

@fsouza commented on GitHub (Mar 18, 2021):

Hey @sfro, I'll try updating the example to include upload. thanks for reporting!

@fsouza commented on GitHub (Mar 18, 2021): Hey @sfro, I'll try updating the example to include upload. thanks for reporting!