starred/fake-gcs-server

Fork 0

mirror of https://github.com/fsouza/fake-gcs-server.git synced 2026-04-26 14:15:49 +03:00

[GH-ISSUE #237] Object.NewReader doesn't seem to be hitting the right endpoint #46

New issue

Closed

opened 2026-03-03 12:07:45 +03:00 by kerem · 6 comments

kerem commented

2026-03-03 12:07:45 +03:00

Owner

Originally created by @simondrake on GitHub (May 7, 2020).
Original GitHub issue: https://github.com/fsouza/fake-gcs-server/issues/237

Firstly, thanks for the project!

I'm currently writing some integration tests, using the docker image, and there seems to be something wrong with the mapping from Object().NewReader.

Using the Go example returns an unable to create new reader: storage: object doesn't exist error.

Looking at the Docker logs, it appears to be mapped to the wrong endpoint:

time="2020-05-07T14:14:05Z" level=info msg="- - [07/May/2020:14:14:05 +0000] \"GET /test-bucket/some_file.txt HTTP/1.1\" 404 19"

Note how the endpoint is /test-bucket/some_file.txt whereas other GET requests have /storage/v1/b/ prefixed

time="2020-05-07T14:14:05Z" level=info msg="- - [07/May/2020:14:14:05 +0000] \"GET /storage/v1/b/test-bucket/o/dir%2Fb?alt=json&prettyPrint=false&projection=full HTTP/1.1\" 200 412"

Originally created by @simondrake on GitHub (May 7, 2020). Original GitHub issue: https://github.com/fsouza/fake-gcs-server/issues/237 Firstly, thanks for the project! I'm currently writing some integration tests, using the docker image, and there seems to be something wrong with the mapping from `Object().NewReader`. Using the [Go example](https://github.com/fsouza/fake-gcs-server/blob/master/examples/go/main.go#L62-L69) returns an `unable to create new reader: storage: object doesn't exist` error. Looking at the Docker logs, it appears to be mapped to the wrong endpoint: ``` time="2020-05-07T14:14:05Z" level=info msg="- - [07/May/2020:14:14:05 +0000] \"GET /test-bucket/some_file.txt HTTP/1.1\" 404 19" ``` Note how the endpoint is `/test-bucket/some_file.txt` whereas other GET requests have `/storage/v1/b/` prefixed ``` time="2020-05-07T14:14:05Z" level=info msg="- - [07/May/2020:14:14:05 +0000] \"GET /storage/v1/b/test-bucket/o/dir%2Fb?alt=json&prettyPrint=false&projection=full HTTP/1.1\" 200 412" ```

kerem closed this issue

2026-03-03 12:07:45 +03:00

kerem commented

2026-03-03 12:07:46 +03:00

Author

Owner

@simondrake commented on GitHub (May 7, 2020):

More information

I ran the docker image as described in the README:

docker run -d --name fake-gcs-server -p 4443:4443 -v /Users/{USER}/examples/data:/data fsouza/fake-gcs-server

And then ran the main.go file, exactly as is:

╰─ go run main.go
buckets: [test-bucket]
2020/05/07 15:22:13 storage: object doesn't exist
exit status 1

@simondrake commented on GitHub (May 7, 2020): ## More information I ran the docker image as described in the README: ``` docker run -d --name fake-gcs-server -p 4443:4443 -v /Users/{USER}/examples/data:/data fsouza/fake-gcs-server ``` And then ran the `main.go` file, exactly as is: ``` ╰─ go run main.go buckets: [test-bucket] 2020/05/07 15:22:13 storage: object doesn't exist exit status 1 ```

kerem commented

2026-03-03 12:07:46 +03:00

Author

Owner

@fsouza commented on GitHub (May 7, 2020):

Hi @simondrake, thanks for opening this issue. For the example to work, you need to start the server with a -public-host: github.com/fsouza/fake-gcs-server@8a565dd691/ci/run-go-example.sh (L7)

Perhaps we can add a comment to the example mentioning it?

@fsouza commented on GitHub (May 7, 2020): Hi @simondrake, thanks for opening this issue. For the example to work, you need to start the server with a `-public-host`: https://github.com/fsouza/fake-gcs-server/blob/8a565dd69148c760107f4c203e1462975dcbfe19/ci/run-go-example.sh#L7 Perhaps we can add a comment to the example mentioning it?

kerem commented

2026-03-03 12:07:46 +03:00

Author

Owner

@simondrake commented on GitHub (May 7, 2020):

Ahh right, ok. Yeah that works, thanks 👍 I didn't actually want/need the data volume for my tests, so that was definitely the missing piece.

docker run -d --name fake-gcs-server -p 4443:4443 fsouza/fake-gcs-server -public-host storage.gcs.127.0.0.1.nip.io:4443

I'll raise a PR shortly to add that to the documentation.

@simondrake commented on GitHub (May 7, 2020): Ahh right, ok. Yeah that works, thanks 👍 I didn't actually want/need the data volume for my tests, so that was definitely the missing piece. ``` docker run -d --name fake-gcs-server -p 4443:4443 fsouza/fake-gcs-server -public-host storage.gcs.127.0.0.1.nip.io:4443 ``` I'll raise a PR shortly to add that to the documentation.

kerem commented

2026-03-03 12:07:46 +03:00

Author

Owner

@simondrake commented on GitHub (May 7, 2020):

To save me going through the code - can you give me a quick explanation of what that does, and why it's needed when using NewReader ?

@simondrake commented on GitHub (May 7, 2020): To save me going through the code - can you give me a quick explanation of what that does, and why it's needed when using `NewReader` ?

kerem commented

2026-03-03 12:07:46 +03:00

Author

Owner

@fsouza commented on GitHub (May 11, 2020):

Hi @simondrake, I apologize for not replying earlier. It seems that the go sdk uses an undocumented endpoint for downloading the file using the download host, which defaults to storage.googleapis.com. The server is actually configured to serve downloads with that host/url, but it only works within Go tests when you use a client obtained from the server.

For external access, you need to (1) teach the client to use a different download host (depends on the client) and (2) configure fake-gcs-server to serve the download handler in with that host (using the -public-host flag).

I pushed a commit with a note on the Go example about providing the flag.

@fsouza commented on GitHub (May 11, 2020): Hi @simondrake, I apologize for not replying earlier. It seems that the go sdk uses an undocumented endpoint for downloading the file using the download host, which defaults to `storage.googleapis.com`. The server is actually configured to serve downloads with that host/url, but it only works within Go tests when you use a client obtained from the server. For external access, you need to (1) teach the client to use a different download host (depends on the client) and (2) configure fake-gcs-server to serve the download handler in with that host (using the `-public-host` flag). I pushed a commit with a note on the Go example about providing the flag.

kerem commented

2026-03-03 12:07:47 +03:00

Author

Owner

@anz-rfc commented on GitHub (May 25, 2020):

See also earlier issue https://github.com/fsouza/fake-gcs-server/issues/201

@anz-rfc commented on GitHub (May 25, 2020): See also earlier issue https://github.com/fsouza/fake-gcs-server/issues/201

kerem referenced this issue

2026-03-03 12:10:07 +03:00

[PR #46] [CLOSED] cmd: add gcs-emulator #294

kerem referenced this issue

2026-03-15 17:49:24 +03:00

[GH-ISSUE #45] Add a dockerized server configuration #2160