[GH-ISSUE #791] Avoid setting "X-Goog-Hash" header in downloads when range is provided #129

New issue

Closed

opened 2026-03-03 12:08:34 +03:00 by kerem · 2 comments

kerem commented

2026-03-03 12:08:34 +03:00

Owner

Originally created by @mcrichton on GitHub (May 20, 2022).
Original GitHub issue: https://github.com/fsouza/fake-gcs-server/issues/791

Expected: the "X-Goog-Hash" is not set for ranged download requests (I cannot find any documentation saying this should be the case, but doing a range GET request against a blob in "real" GCS does not return an "X-Goog-Hash" header)

Actual: the "X-Goog-Hash" is set regardless of whether or not a ranged request is being made

This notably causes issues when using fake-gcs-server with the Google Cloud Python SDK, as the SDK finds an MD5 mismatch (since fake-gcs-server (correctly, according to Google's own docs 😔) returns the MD5 for the entire blob) and crashes

Originally created by @mcrichton on GitHub (May 20, 2022). Original GitHub issue: https://github.com/fsouza/fake-gcs-server/issues/791 Expected: the "X-Goog-Hash" is not set for ranged download requests (I cannot find any documentation saying this should be the case, but doing a range GET request against a blob in "real" GCS does not return an "X-Goog-Hash" header) Actual: the "X-Goog-Hash" is [set regardless of whether or not a ranged request is being made](https://github.com/fsouza/fake-gcs-server/blob/076577042e85a326642c0b811f5eebde9f8f44a2/fakestorage/object.go#L662) This notably causes issues when using `fake-gcs-server` with the Google Cloud Python SDK, as the SDK finds an MD5 mismatch (since `fake-gcs-server` (correctly, according to [Google's own docs](https://cloud.google.com/storage/docs/xml-api/get-object-download#request_headers) 😔) returns the MD5 for the entire blob) and crashes

kerem

2026-03-03 12:08:34 +03:00

closed this issue
added the
bug

compatibility-issue
labels

kerem commented

2026-03-03 12:08:34 +03:00

Author

Owner

@dnerdy commented on GitHub (Jun 1, 2022):

I just sent a range request to GCS, and it looks like x-goog-hash is always included.

This request:

curl -D - https://storage.googleapis.com/marksandstrom-test/test.txt

Outputs:

HTTP/2 200
x-guploader-uploadid: ADPycdsqZLIEmb1-MJqq8qkC1W1BYDZgLpVrIcH9hIIHuHF9s8iVWRfqrpmq6qmI51Lgo3pDYteEjZj9yG9v7VZDsyacdg
expires: Wed, 01 Jun 2022 22:24:35 GMT
date: Wed, 01 Jun 2022 21:24:35 GMT
cache-control: public, max-age=3600
last-modified: Mon, 30 May 2022 16:09:41 GMT
etag: "2d01d5d9c24034d54fe4fba0ede5182d"
x-goog-generation: 1653926981160112
x-goog-metageneration: 1
x-goog-stored-content-encoding: identity
x-goog-stored-content-length: 12
content-type: text/plain
x-goog-hash: crc32c=FrGT1A==
x-goog-hash: md5=LQHV2cJANNVP5Pug7eUYLQ==
x-goog-storage-class: STANDARD
accept-ranges: bytes
content-length: 12
server: UploadServer
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"

hello there

And this request:

curl -D - https://storage.googleapis.com/marksandstrom-test/test.txt -H"Range: bytes=0-4"

Returns:

HTTP/2 206
x-guploader-uploadid: ADPycdsZnnRssVGOBUtdMg247DqZjXbBHA4rJ0AKhPAFSTHGMm03FvIbYaUkICkqTOLitmNPHxA1qCP7PAVi7j8FxSY83FSMlcr5
expires: Wed, 01 Jun 2022 22:23:32 GMT
date: Wed, 01 Jun 2022 21:23:32 GMT
cache-control: public, max-age=3600
last-modified: Mon, 30 May 2022 16:09:41 GMT
etag: "2d01d5d9c24034d54fe4fba0ede5182d"
x-goog-generation: 1653926981160112
x-goog-metageneration: 1
x-goog-stored-content-encoding: identity
x-goog-stored-content-length: 12
content-type: text/plain
x-goog-hash: crc32c=FrGT1A==
x-goog-hash: md5=LQHV2cJANNVP5Pug7eUYLQ==
x-goog-storage-class: STANDARD
accept-ranges: bytes
content-range: bytes 0-4/12
content-length: 5
server: UploadServer
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"

hello

@dnerdy commented on GitHub (Jun 1, 2022): I just sent a range request to GCS, and it looks like x-goog-hash is always included. This request: ``` curl -D - https://storage.googleapis.com/marksandstrom-test/test.txt ``` Outputs: ``` HTTP/2 200 x-guploader-uploadid: ADPycdsqZLIEmb1-MJqq8qkC1W1BYDZgLpVrIcH9hIIHuHF9s8iVWRfqrpmq6qmI51Lgo3pDYteEjZj9yG9v7VZDsyacdg expires: Wed, 01 Jun 2022 22:24:35 GMT date: Wed, 01 Jun 2022 21:24:35 GMT cache-control: public, max-age=3600 last-modified: Mon, 30 May 2022 16:09:41 GMT etag: "2d01d5d9c24034d54fe4fba0ede5182d" x-goog-generation: 1653926981160112 x-goog-metageneration: 1 x-goog-stored-content-encoding: identity x-goog-stored-content-length: 12 content-type: text/plain x-goog-hash: crc32c=FrGT1A== x-goog-hash: md5=LQHV2cJANNVP5Pug7eUYLQ== x-goog-storage-class: STANDARD accept-ranges: bytes content-length: 12 server: UploadServer alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43" hello there ``` And this request: ``` curl -D - https://storage.googleapis.com/marksandstrom-test/test.txt -H"Range: bytes=0-4" ``` Returns: ``` HTTP/2 206 x-guploader-uploadid: ADPycdsZnnRssVGOBUtdMg247DqZjXbBHA4rJ0AKhPAFSTHGMm03FvIbYaUkICkqTOLitmNPHxA1qCP7PAVi7j8FxSY83FSMlcr5 expires: Wed, 01 Jun 2022 22:23:32 GMT date: Wed, 01 Jun 2022 21:23:32 GMT cache-control: public, max-age=3600 last-modified: Mon, 30 May 2022 16:09:41 GMT etag: "2d01d5d9c24034d54fe4fba0ede5182d" x-goog-generation: 1653926981160112 x-goog-metageneration: 1 x-goog-stored-content-encoding: identity x-goog-stored-content-length: 12 content-type: text/plain x-goog-hash: crc32c=FrGT1A== x-goog-hash: md5=LQHV2cJANNVP5Pug7eUYLQ== x-goog-storage-class: STANDARD accept-ranges: bytes content-range: bytes 0-4/12 content-length: 5 server: UploadServer alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43" hello ```

kerem commented

2026-03-03 12:08:35 +03:00

Author

Owner

@mcrichton commented on GitHub (Jun 2, 2022):

A mistake on my end, on re-checking. Thanks for verifying!

@mcrichton commented on GitHub (Jun 2, 2022): A mistake on my end, on re-checking. Thanks for verifying!

kerem referenced this issue

2026-03-03 12:29:40 +03:00

[PR #129] [MERGED] build(deps): bump @google-cloud/storage from 4.1.1 to 4.1.2 in /examples/node #365