[GH-ISSUE #1948] Unable to write object using "pafs.GcsFileSystem" #230

New issue

Open

opened 2026-03-03 12:09:19 +03:00 by kerem · 0 comments

kerem commented

2026-03-03 12:09:19 +03:00

Owner

Originally created by @dsonawane-conviva on GitHub (Apr 3, 2025).
Original GitHub issue: https://github.com/fsouza/fake-gcs-server/issues/1948

I am getting error while writing to GCS bucket on fake-gcs-server using "pafs.GcsFileSystem". Below code fails -

gcs = (
    DockerContainer("fsouza/fake-gcs-server:latest")
    .with_exposed_ports(4443)
    .with_command("-scheme http")
)

project_id = "test_project"
bucket_name = "test_bucket"
gcs.create_bucket(bucket_name, project_id)

_fs = pafs.GcsFileSystem(
    anonymous=True,
    endpoint_override=f"localhost:{gcs.port}",
    retry_time_limit=timedelta(seconds=10),
    project_id=project_id,
    scheme="http",
)

path_ = "test_bucket/test_data.parquet"
with _fs.open_output_stream(path_) as f:
    pq.write_table(table, f)

Error message -

OSError: google::cloud::Status(UNAVAILABLE: Retry policy exhausted, with a last message of Retry policy exhausted, with a last message of PerformWork() - CURL error [7]=Couldn't connect to server error_info={reason=, domain=, metadata={gcloud-cpp.retry.reason=retry-policy-exhausted, gcloud-cpp.retry.on-entry=false, gcloud-cpp.retry.function=UploadChunk, gcloud-cpp.retry.original-message=Retry policy exhausted, with a last message of PerformWork() - CURL error [7]=Couldn't connect to server}})

Note that -

I am creating bucket before writing to fake gcs server
I am able to write to actual gcs bucket with above code
with same _fs as above, I can read file from fake-gcs-server, that is, _fs.open_input_file(file_path) this works

Server logs -

time=2025-04-03T17:59:55.198Z level=INFO msg="server started at http://0.0.0.0:4443/"
time=2025-04-03T18:00:00.166Z level=INFO msg="172.17.0.1 - - [03/Apr/2025:18:00:00 +0000] \"POST /storage/v1/b?project=test_project HTTP/1.1\" 200 345\n"
time=2025-04-03T18:00:09.631Z level=INFO msg="172.17.0.1 - - [03/Apr/2025:18:00:09 +0000] \"POST /upload/storage/v1/b/test_bucket/o?uploadType=resumable&name=test_data.parquet HTTP/1.1\" 200 550\n"

Could be related to Issue-1882

Originally created by @dsonawane-conviva on GitHub (Apr 3, 2025). Original GitHub issue: https://github.com/fsouza/fake-gcs-server/issues/1948 I am getting error while writing to GCS bucket on fake-gcs-server using "pafs.GcsFileSystem". Below code fails - ``` gcs = ( DockerContainer("fsouza/fake-gcs-server:latest") .with_exposed_ports(4443) .with_command("-scheme http") ) project_id = "test_project" bucket_name = "test_bucket" gcs.create_bucket(bucket_name, project_id) _fs = pafs.GcsFileSystem( anonymous=True, endpoint_override=f"localhost:{gcs.port}", retry_time_limit=timedelta(seconds=10), project_id=project_id, scheme="http", ) path_ = "test_bucket/test_data.parquet" with _fs.open_output_stream(path_) as f: pq.write_table(table, f) ``` Error message - ``` OSError: google::cloud::Status(UNAVAILABLE: Retry policy exhausted, with a last message of Retry policy exhausted, with a last message of PerformWork() - CURL error [7]=Couldn't connect to server error_info={reason=, domain=, metadata={gcloud-cpp.retry.reason=retry-policy-exhausted, gcloud-cpp.retry.on-entry=false, gcloud-cpp.retry.function=UploadChunk, gcloud-cpp.retry.original-message=Retry policy exhausted, with a last message of PerformWork() - CURL error [7]=Couldn't connect to server}}) ``` Note that - 1. I am creating bucket before writing to fake gcs server 2. I am able to write to actual gcs bucket with above code 3. with same _fs as above, I can read file from fake-gcs-server, that is, _fs.open_input_file(file_path) this works Server logs - ``` time=2025-04-03T17:59:55.198Z level=INFO msg="server started at http://0.0.0.0:4443/" time=2025-04-03T18:00:00.166Z level=INFO msg="172.17.0.1 - - [03/Apr/2025:18:00:00 +0000] \"POST /storage/v1/b?project=test_project HTTP/1.1\" 200 345\n" time=2025-04-03T18:00:09.631Z level=INFO msg="172.17.0.1 - - [03/Apr/2025:18:00:09 +0000] \"POST /upload/storage/v1/b/test_bucket/o?uploadType=resumable&name=test_data.parquet HTTP/1.1\" 200 550\n" ``` Could be related to [Issue-1882](https://github.com/fsouza/fake-gcs-server/issues/1882)