[GH-ISSUE #1362] Signed url upload returns 200 response instead of 308 for headers = {"Content-Length": "0", "Content-Range": "bytes /"} #193

New issue

Open

opened 2026-03-03 12:09:03 +03:00 by kerem · 0 comments

kerem commented

2026-03-03 12:09:03 +03:00

Owner

Originally created by @j-antunes on GitHub (Oct 12, 2023).
Original GitHub issue: https://github.com/fsouza/fake-gcs-server/issues/1362

Hello!

I think for the resumable uploads there is a small difference between the emulated and real GCS for when we start an upload.

The below code snippet is an example of an initial request that we can do to get the current status of the upload. You usually do this to see if there is anything already uploaded.

import requests
headers = {"Content-Length": "0", "Content-Range": "bytes */*"}
response = requests.put(session_uri, headers=headers)

I would expect to receive a 308 response (upload incomplete), but instead I'm getting a 200 response.

Note that you can skip this step and just directly upload the file, but it would be nice that this would work as expected.

Here is an example to reproduce the issue


# Copyright 2019 Francisco Souza. All rights reserved.
# Use of this source code is governed by a BSD-style
# license that can be found in the LICENSE file.

# How to run this example
# 1 - Build the docker image by running the command "docker build -t fsouza/fake-gcs-server ."
# 2 - Start the docker container: "docker run -d --name fake-gcs-server -p 4443:4443 -v ${PWD}/examples/data:/data fsouza/fake-gcs-server -scheme http"
# 3 - Check if it's working by running: "curl http://0.0.0.0:4443/storage/v1/b"
# 4 - Create a python virtual enviroment (Ex: python -m .venv venv)
# 5 - Source the env (source .venv/bin/activate)
# 7 - Go to the following directory examples/python: (cd examples/python)
# 6 - Install requirements: "pip install -r requirements.txt" and "pip install -r requirements.in"
# 7 - Run this script

import tempfile

from google.auth.credentials import AnonymousCredentials
from google.cloud import storage
import requests

client = storage.Client(
    credentials=AnonymousCredentials(),
    project="test",
    # This endpoint assumes that you are using the default port 4443 from the container.
    # If you are using a different port you need to update this endpoint
    client_options={"api_endpoint": "http://localhost:4443"},
)

bucket = client.bucket("sample-bucket")
with tempfile.NamedTemporaryFile(suffix=".html", delete=False) as temp_file:
    temp_file.write(b"<html><body><h1>Hello, World!</h1></body></html>")

    blob = bucket.blob(temp_file.name)

    session_url = blob.create_resumable_upload_session(client=client)


headers = {"Content-Length": "0", "Content-Range": "bytes */*"}
response = requests.put(session_url, headers=headers)

print(response.json())

This is the response that I got

<Response [200]>
special variables
function variables
apparent_encoding:
'ascii'
connection:
<requests.adapters.HTTPAdapter object at 0x7f2d951ed6f0>
content:
b'{"kind":"storage#object","name":"/tmp/tmpmsqjyo4o.html","id":"sample-bucket//tmp/tmpmsqjyo4o.html","bucket":"sample-bucket","size":"0","contentType":"application/octet-stream","crc32c":"AAAAAA==","acl":[{"bucket":"sample-bucket","entity":"projectOwner-test-project","object":"/tmp/tmpmsqjyo4o.html","projectTeam":{},"role":"OWNER"}],"md5Hash":"1B2M2Y8AsgTpgAmY7PhCfg==","etag":"\\"1B2M2Y8AsgTpgAmY7PhCfg==\\"","timeCreated":"2023-10-12T17:03:59.99945Z","updated":"2023-10-12T17:03:59.999452Z","generation":"1697130239999453"}\n'
cookies:
<RequestsCookieJar[]>
elapsed:
datetime.timedelta(microseconds=2909)
encoding:
'utf-8'
headers:
{'Content-Type': 'application/json', 'Range': 'bytes=0-0', 'Date': 'Thu, 12 Oct 2023 17:04:00 GMT', 'Content-Length': '524'}
history:
[]
is_permanent_redirect:
False
is_redirect:
False
links:
{}
next:
None
ok:
True
raw:
<urllib3.response.HTTPResponse object at 0x7f2d951eec80>
reason:
'OK'
request:
<PreparedRequest [PUT]>
status_code:
200
text:
'{"kind":"storage#object","name":"/tmp/tmpmsqjyo4o.html","id":"sample-bucket//tmp/tmpmsqjyo4o.html","bucket":"sample-bucket","size":"0","contentType":"application/octet-stream","crc32c":"AAAAAA==","acl":[{"bucket":"sample-bucket","entity":"projectOwner-test-project","object":"/tmp/tmpmsqjyo4o.html","projectTeam":{},"role":"OWNER"}],"md5Hash":"1B2M2Y8AsgTpgAmY7PhCfg==","etag":"\\"1B2M2Y8AsgTpgAmY7PhCfg==\\"","timeCreated":"2023-10-12T17:03:59.99945Z","updated":"2023-10-12T17:03:59.999452Z","generation":"1697130239999453"}\n'
url:
'http://0.0.0.0:4443/upload/storage/v1/b/sample-bucket/o?uploadType=resumable&name=%2Ftmp%2Ftmpmsqjyo4o.html&upload_id=7e7cd5bec6cf35ee112a0f99e4b57f2b'
_content:
b'{"kind":"storage#object","name":"/tmp/tmpmsqjyo4o.html","id":"sample-bucket//tmp/tmpmsqjyo4o.html","bucket":"sample-bucket","size":"0","contentType":"application/octet-stream","crc32c":"AAAAAA==","acl":[{"bucket":"sample-bucket","entity":"projectOwner-test-project","object":"/tmp/tmpmsqjyo4o.html","projectTeam":{},"role":"OWNER"}],"md5Hash":"1B2M2Y8AsgTpgAmY7PhCfg==","etag":"\\"1B2M2Y8AsgTpgAmY7PhCfg==\\"","timeCreated":"2023-10-12T17:03:59.99945Z","updated":"2023-10-12T17:03:59.999452Z","generation":"1697130239999453"}\n'
_content_consumed:
True
_next:
None

Running the same example against the real GCS I get the following response:

<Response [308]>
special variables
function variables
apparent_encoding:
None
connection:
<requests.adapters.HTTPAdapter object at 0x7f396620d0f0>
content:
b''
cookies:
<RequestsCookieJar[]>
elapsed:
datetime.timedelta(microseconds=147664)
encoding:
'utf-8'
headers:
{'Content-Type': 'text/plain; charset=utf-8', 'X-GUploader-UploadID': '<REDACTED>', 'Content-Length': '0', 'Date': 'Thu, 12 Oct 2023 17:08:36 GMT', 'Server': 'UploadServer', 'Alt-Svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000'}
history:
[]
is_permanent_redirect:
False
is_redirect:
False
links:
{}
next:
None
ok:
True
raw:
<urllib3.response.HTTPResponse object at 0x7f396620c820>
reason:
'Resume Incomplete'
request:
<PreparedRequest [PUT]>
status_code:
308
text:
''
url:
'https://storage.googleapis.com/upload/storage/v1/b/<REDACTED>/o?uploadType=resumable&upload_id=<REDACTED>'
_content:
b''
_content_consumed:
True
_next:
None

Originally created by @j-antunes on GitHub (Oct 12, 2023). Original GitHub issue: https://github.com/fsouza/fake-gcs-server/issues/1362 Hello! I think for the resumable uploads there is a small difference between the emulated and real GCS for when we start an upload. The below code snippet is an example of an initial request that we can do to get the current status of the upload. You usually do this to see if there is anything already uploaded. ``` import requests headers = {"Content-Length": "0", "Content-Range": "bytes */*"} response = requests.put(session_uri, headers=headers) ``` I would expect to receive a 308 response (upload incomplete), but instead I'm getting a 200 response. Note that you can skip this step and just directly upload the file, but it would be nice that this would work as expected. Here is an example to reproduce the issue ``` # Copyright 2019 Francisco Souza. All rights reserved. # Use of this source code is governed by a BSD-style # license that can be found in the LICENSE file. # How to run this example # 1 - Build the docker image by running the command "docker build -t fsouza/fake-gcs-server ." # 2 - Start the docker container: "docker run -d --name fake-gcs-server -p 4443:4443 -v ${PWD}/examples/data:/data fsouza/fake-gcs-server -scheme http" # 3 - Check if it's working by running: "curl http://0.0.0.0:4443/storage/v1/b" # 4 - Create a python virtual enviroment (Ex: python -m .venv venv) # 5 - Source the env (source .venv/bin/activate) # 7 - Go to the following directory examples/python: (cd examples/python) # 6 - Install requirements: "pip install -r requirements.txt" and "pip install -r requirements.in" # 7 - Run this script import tempfile from google.auth.credentials import AnonymousCredentials from google.cloud import storage import requests client = storage.Client( credentials=AnonymousCredentials(), project="test", # This endpoint assumes that you are using the default port 4443 from the container. # If you are using a different port you need to update this endpoint client_options={"api_endpoint": "http://localhost:4443"}, ) bucket = client.bucket("sample-bucket") with tempfile.NamedTemporaryFile(suffix=".html", delete=False) as temp_file: temp_file.write(b"<html><body><h1>Hello, World!</h1></body></html>") blob = bucket.blob(temp_file.name) session_url = blob.create_resumable_upload_session(client=client) headers = {"Content-Length": "0", "Content-Range": "bytes */*"} response = requests.put(session_url, headers=headers) print(response.json()) ``` This is the response that I got ``` <Response [200]> special variables function variables apparent_encoding: 'ascii' connection: <requests.adapters.HTTPAdapter object at 0x7f2d951ed6f0> content: b'{"kind":"storage#object","name":"/tmp/tmpmsqjyo4o.html","id":"sample-bucket//tmp/tmpmsqjyo4o.html","bucket":"sample-bucket","size":"0","contentType":"application/octet-stream","crc32c":"AAAAAA==","acl":[{"bucket":"sample-bucket","entity":"projectOwner-test-project","object":"/tmp/tmpmsqjyo4o.html","projectTeam":{},"role":"OWNER"}],"md5Hash":"1B2M2Y8AsgTpgAmY7PhCfg==","etag":"\\"1B2M2Y8AsgTpgAmY7PhCfg==\\"","timeCreated":"2023-10-12T17:03:59.99945Z","updated":"2023-10-12T17:03:59.999452Z","generation":"1697130239999453"}\n' cookies: <RequestsCookieJar[]> elapsed: datetime.timedelta(microseconds=2909) encoding: 'utf-8' headers: {'Content-Type': 'application/json', 'Range': 'bytes=0-0', 'Date': 'Thu, 12 Oct 2023 17:04:00 GMT', 'Content-Length': '524'} history: [] is_permanent_redirect: False is_redirect: False links: {} next: None ok: True raw: <urllib3.response.HTTPResponse object at 0x7f2d951eec80> reason: 'OK' request: <PreparedRequest [PUT]> status_code: 200 text: '{"kind":"storage#object","name":"/tmp/tmpmsqjyo4o.html","id":"sample-bucket//tmp/tmpmsqjyo4o.html","bucket":"sample-bucket","size":"0","contentType":"application/octet-stream","crc32c":"AAAAAA==","acl":[{"bucket":"sample-bucket","entity":"projectOwner-test-project","object":"/tmp/tmpmsqjyo4o.html","projectTeam":{},"role":"OWNER"}],"md5Hash":"1B2M2Y8AsgTpgAmY7PhCfg==","etag":"\\"1B2M2Y8AsgTpgAmY7PhCfg==\\"","timeCreated":"2023-10-12T17:03:59.99945Z","updated":"2023-10-12T17:03:59.999452Z","generation":"1697130239999453"}\n' url: 'http://0.0.0.0:4443/upload/storage/v1/b/sample-bucket/o?uploadType=resumable&name=%2Ftmp%2Ftmpmsqjyo4o.html&upload_id=7e7cd5bec6cf35ee112a0f99e4b57f2b' _content: b'{"kind":"storage#object","name":"/tmp/tmpmsqjyo4o.html","id":"sample-bucket//tmp/tmpmsqjyo4o.html","bucket":"sample-bucket","size":"0","contentType":"application/octet-stream","crc32c":"AAAAAA==","acl":[{"bucket":"sample-bucket","entity":"projectOwner-test-project","object":"/tmp/tmpmsqjyo4o.html","projectTeam":{},"role":"OWNER"}],"md5Hash":"1B2M2Y8AsgTpgAmY7PhCfg==","etag":"\\"1B2M2Y8AsgTpgAmY7PhCfg==\\"","timeCreated":"2023-10-12T17:03:59.99945Z","updated":"2023-10-12T17:03:59.999452Z","generation":"1697130239999453"}\n' _content_consumed: True _next: None ``` Running the same example against the real GCS I get the following response: ``` <Response [308]> special variables function variables apparent_encoding: None connection: <requests.adapters.HTTPAdapter object at 0x7f396620d0f0> content: b'' cookies: <RequestsCookieJar[]> elapsed: datetime.timedelta(microseconds=147664) encoding: 'utf-8' headers: {'Content-Type': 'text/plain; charset=utf-8', 'X-GUploader-UploadID': '<REDACTED>', 'Content-Length': '0', 'Date': 'Thu, 12 Oct 2023 17:08:36 GMT', 'Server': 'UploadServer', 'Alt-Svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000'} history: [] is_permanent_redirect: False is_redirect: False links: {} next: None ok: True raw: <urllib3.response.HTTPResponse object at 0x7f396620c820> reason: 'Resume Incomplete' request: <PreparedRequest [PUT]> status_code: 308 text: '' url: 'https://storage.googleapis.com/upload/storage/v1/b/<REDACTED>/o?uploadType=resumable&upload_id=<REDACTED>' _content: b'' _content_consumed: True _next: None ```