[GH-ISSUE #1753] Unable to upload file to bucket: getting 404

kerem commented

2026-03-03 12:09:14 +03:00

Owner

Originally created by @mswami1 on GitHub (Sep 23, 2024).
Original GitHub issue: https://github.com/fsouza/fake-gcs-server/issues/1753

Error:

Writer.Close: googleapi: Error 404: Not Found, Not Found

docker service:

storage:
image: fsouza/fake-gcs-server
environment:
STORAGE_EMULATOR_HOST: localhost:4443
ports:
- '4443:4443'
command: -scheme http -port 4443 -public-host localhost:4443
volumes:
- .:/gcs_storage

client, err1 := storage.NewClient(context.TODO(), option.WithEndpoint("http://127.0.0.1:4443/storage/v1/"))

Originally created by @mswami1 on GitHub (Sep 23, 2024). Original GitHub issue: https://github.com/fsouza/fake-gcs-server/issues/1753 Error: Writer.Close: googleapi: Error 404: Not Found, Not Found docker service: storage: image: fsouza/fake-gcs-server environment: STORAGE_EMULATOR_HOST: localhost:4443 ports: - '4443:4443' command: -scheme http -port 4443 -public-host localhost:4443 volumes: - .:/gcs_storage client, err1 := storage.NewClient(context.TODO(), option.WithEndpoint("http://127.0.0.1:4443/storage/v1/"))

kerem added the

enhancement

label

2026-03-03 12:09:14 +03:00

kerem commented

2026-03-03 12:09:15 +03:00

Author

Owner

@KeithScheiwiller commented on GitHub (Dec 30, 2024):

Did you create a directory within /storage with the matching bucket name? I faced the same issue but doing so solved it. I didn't see it mentioned in the docs, but it seems it does not auto-create buckets.

@KeithScheiwiller commented on GitHub (Dec 30, 2024): Did you create a directory within `/storage` with the matching bucket name? I faced the same issue but doing so solved it. I didn't see it mentioned in the docs, but it seems it does not auto-create buckets.

kerem commented

2026-03-03 12:09:15 +03:00

Author

Owner

@fsouza commented on GitHub (Jan 3, 2025):

Yeah upload will fail if the bucket doesn't exist. I'll keep this open to improve the error message or at least log that the bucket doesn't exist.

@fsouza commented on GitHub (Jan 3, 2025): Yeah upload will fail if the bucket doesn't exist. I'll keep this open to improve the error message or at least log that the bucket doesn't exist.

kerem commented

2026-03-03 12:09:15 +03:00

Author

Owner

@daaain commented on GitHub (Feb 3, 2025):

Does the bucket need to be created with an API call after startup?

I tried mounting a volume with a directory matching the bucket name, but that didn't help.

Also, is /storage a separate thing from the -data parameter?

services:
  fake-gcs:
    container_name: fake-gcs
    image: fsouza/fake-gcs-server:latest
    entrypoint: /bin/fake-gcs-server
    command: -data /storage -scheme both -cors-headers x-goog-content-length-range -public-host fake-gcs
    ports:
      - "127.0.0.1:9023:4443"
      - "127.0.0.1:8000:8000"
    volumes:
      - ${PWD}/.docker/gcs-emulator:/storage

@daaain commented on GitHub (Feb 3, 2025): Does the bucket need to be created with an API call after startup? I tried mounting a volume with a directory matching the bucket name, but that didn't help. Also, is `/storage` a separate thing from the `-data` parameter? ```yaml services: fake-gcs: container_name: fake-gcs image: fsouza/fake-gcs-server:latest entrypoint: /bin/fake-gcs-server command: -data /storage -scheme both -cors-headers x-goog-content-length-range -public-host fake-gcs ports: - "127.0.0.1:9023:4443" - "127.0.0.1:8000:8000" volumes: - ${PWD}/.docker/gcs-emulator:/storage ```

kerem commented

2026-03-03 12:09:15 +03:00

Author

Owner

@daaain commented on GitHub (Feb 3, 2025):

To answer my own question, /data is what supposed to be a volume and /storage is the internal backend where *.bucketMetadata and *.metadata files get created, so if you set -data /storage and have a volume attached to /storage you'll get one level deeper *.metadata.metadata on every restart 😅

Also, attaching a subdirectory to /data does make the emulator treat it as an existing bucket. Also, files uploaded via the emulator only get written to /storage, but not to /data, so it's just an initial seed.

Would be really helpful if this was documented!

So I can upload to the bucket using the gsutil example, but I still get 404 when trying to use a signed URL generated by the Node SDK 🤷 the container logs it so the request gets through, it just returns 404

This is how the logs look like, the first POST is the successful gsutil upload, then a successful OPTION for CORS and then the 404 signed upload PUT:

2025-02-03 16:51:44 time=2025-02-03T16:51:44.277Z level=INFO msg="server started at http://0.0.0.0:8000"
2025-02-03 16:51:44 time=2025-02-03T16:51:44.277Z level=INFO msg="server started at https://0.0.0.0:4443"
2025-02-03 16:51:55 time=2025-02-03T16:51:55.580Z level=INFO msg="172.20.0.1 - - [03/Feb/2025:16:51:55 +0000] \"POST /upload/storage/v1/b/platform-onboarding-uploads/o?alt=json&fields=crc32c%2Cgeneration%2Cmd5Hash%2CcustomerEncryption%2Csize%2Cetag&uploadType=multipart HTTP/1.1\" 200 902\n"
2025-02-03 17:02:13 time=2025-02-03T17:02:13.744Z level=INFO msg="172.20.0.1 - - [03/Feb/2025:17:02:13 +0000] \"OPTIONS /platform-onboarding-uploads/bec97b0702b47525ada988ca6af9705b/cm6p0vpgj0002qzs7ticlxhdp/DeepSeek_R1.pdf?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=... HTTP/1.1\" 200 0\n"
2025-02-03 17:02:13 time=2025-02-03T17:02:13.747Z level=INFO msg="172.20.0.1 - - [03/Feb/2025:17:02:13 +0000] \"PUT /platform-onboarding-uploads/bec97b0702b47525ada988ca6af9705b/cm6p0vpgj0002qzs7ticlxhdp/DeepSeek_R1.pdf?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=... HTTP/1.1\" 404 119\n"

@daaain commented on GitHub (Feb 3, 2025): To answer my own question, `/data` is what supposed to be a volume and `/storage` is the internal backend where `*.bucketMetadata` and `*.metadata` files get created, so if you set `-data /storage` and have a volume attached to `/storage` you'll get one level deeper `*.metadata.metadata` on every restart 😅 Also, attaching a subdirectory to `/data` does make the emulator treat it as an existing bucket. Also, files uploaded via the emulator only get written to `/storage`, but not to `/data`, so it's just an initial seed. Would be really helpful if this was documented! So I can upload to the bucket using the [gsutil example](https://github.com/fsouza/fake-gcs-server/blob/main/examples/gsutil/gsutil-example.sh), but I still get 404 when trying to use a signed URL generated by the Node SDK 🤷 the container logs it so the request gets through, it just returns 404 This is how the logs look like, the first POST is the successful `gsutil` upload, then a successful OPTION for CORS and then the 404 signed upload PUT: ``` 2025-02-03 16:51:44 time=2025-02-03T16:51:44.277Z level=INFO msg="server started at http://0.0.0.0:8000" 2025-02-03 16:51:44 time=2025-02-03T16:51:44.277Z level=INFO msg="server started at https://0.0.0.0:4443" 2025-02-03 16:51:55 time=2025-02-03T16:51:55.580Z level=INFO msg="172.20.0.1 - - [03/Feb/2025:16:51:55 +0000] \"POST /upload/storage/v1/b/platform-onboarding-uploads/o?alt=json&fields=crc32c%2Cgeneration%2Cmd5Hash%2CcustomerEncryption%2Csize%2Cetag&uploadType=multipart HTTP/1.1\" 200 902\n" 2025-02-03 17:02:13 time=2025-02-03T17:02:13.744Z level=INFO msg="172.20.0.1 - - [03/Feb/2025:17:02:13 +0000] \"OPTIONS /platform-onboarding-uploads/bec97b0702b47525ada988ca6af9705b/cm6p0vpgj0002qzs7ticlxhdp/DeepSeek_R1.pdf?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=... HTTP/1.1\" 200 0\n" 2025-02-03 17:02:13 time=2025-02-03T17:02:13.747Z level=INFO msg="172.20.0.1 - - [03/Feb/2025:17:02:13 +0000] \"PUT /platform-onboarding-uploads/bec97b0702b47525ada988ca6af9705b/cm6p0vpgj0002qzs7ticlxhdp/DeepSeek_R1.pdf?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=... HTTP/1.1\" 404 119\n" ```

kerem commented

2026-03-03 12:09:15 +03:00

Author

Owner

@daaain commented on GitHub (Feb 3, 2025):

Got it working! Apparently it was a public-host mismatch, with this configuration the PUT signed URL did upload:

services:
  fake-gcs:
    container_name: fake-gcs
    image: fsouza/fake-gcs-server:latest
    command: -scheme both -cors-headers x-goog-content-length-range,content-length -public-host 127.0.0.1:8000
    ports:
      - "127.0.0.1:9023:4443"
      - "127.0.0.1:8000:8000"
    volumes:
      - ${PWD}/.docker/gcs-emulator:/data
  app:
    ...
    environment:
      STORAGE_EMULATOR_HOST: http://127.0.0.1:8000

@daaain commented on GitHub (Feb 3, 2025): Got it working! Apparently it was a `public-host` mismatch, with this configuration the PUT signed URL did upload: ```yaml services: fake-gcs: container_name: fake-gcs image: fsouza/fake-gcs-server:latest command: -scheme both -cors-headers x-goog-content-length-range,content-length -public-host 127.0.0.1:8000 ports: - "127.0.0.1:9023:4443" - "127.0.0.1:8000:8000" volumes: - ${PWD}/.docker/gcs-emulator:/data app: ... environment: STORAGE_EMULATOR_HOST: http://127.0.0.1:8000 ```

kerem commented

2026-03-03 12:09:15 +03:00

Author

Owner

@yassenb commented on GitHub (Apr 8, 2025):

I second documenting this. I was banging my head why no data was persisted even if I had a Docker volume mounted in /data. It's far from obvious there's another volume that should be persisted and mounted under /storage

@yassenb commented on GitHub (Apr 8, 2025): I second documenting this. I was banging my head why no data was persisted even if I had a Docker volume mounted in `/data`. It's far from obvious there's another volume that should be persisted and mounted under `/storage`

kerem commented

2026-03-03 12:09:15 +03:00

Author

Owner

@gsouf commented on GitHub (Oct 23, 2025):

I'm run running it behind a reverse proxy and I'm also getting the same 404 error.

I don't understand what's the expected value in public-host and why it would lead to the upload to fail if it's incorrect.

The doc says Optional URL for public host (default "storage.googleapis.com") but it's not really explaining how that works.

Could anyone help to clarify that?

@gsouf commented on GitHub (Oct 23, 2025): I'm run running it behind a reverse proxy and I'm also getting the same 404 error. I don't understand what's the expected value in `public-host` and why it would lead to the upload to fail if it's incorrect. The doc says `Optional URL for public host (default "storage.googleapis.com")` but it's not really explaining how that works. Could anyone help to clarify that?

kerem commented

2026-03-03 12:09:15 +03:00

Author

Owner

@gsouf commented on GitHub (Oct 23, 2025):

Ok I found out that when it's behind a reverse proxy, it has to be the private url to which the proxy forwards the request too, and not the public uri of the proxy server itself.

It seems that this public-host setting is a pain point for many people out there. Once it works it's ok, but most people, including myself, seemingly waste hours figuring out how to configure it properly

@gsouf commented on GitHub (Oct 23, 2025): Ok I found out that when it's behind a reverse proxy, it has to be the private url to which the proxy forwards the request too, and not the public uri of the proxy server itself. It seems that this public-host setting is a pain point for many people out there. Once it works it's ok, but most people, including myself, seemingly waste hours figuring out how to configure it properly

kerem referenced this issue

2026-03-03 12:29:59 +03:00

[PR #219] [MERGED] ci: re-enable node example #433

Rows
Columns

[GH-ISSUE #1753] Unable to upload file to bucket: getting 404 #219