[GH-ISSUE #381] Signed URLs?

kerem commented

2026-03-03 12:08:01 +03:00

Owner

Originally created by @steezeburger on GitHub (Dec 11, 2020).
Original GitHub issue: https://github.com/fsouza/fake-gcs-server/issues/381

Hey thanks for the docker image! I got everything running locally, but I'm curious, does this support generating signed upload and download URLs?

Originally created by @steezeburger on GitHub (Dec 11, 2020). Original GitHub issue: https://github.com/fsouza/fake-gcs-server/issues/381 Hey thanks for the docker image! I got everything running locally, but I'm curious, does this support generating signed upload and download URLs?

kerem closed this issue

2026-03-03 12:08:01 +03:00

kerem commented

2026-03-03 12:08:02 +03:00

Author

Owner

@kenniaa commented on GitHub (Dec 20, 2020):

It does in fact support signed URLs.

There should be no additional setup on the fake-gcs-server side outside of creating the bucket. If you run into CORS issues over unsupported headers, you can pass in the -cors-headers flag with a list of comma separated headers.

@kenniaa commented on GitHub (Dec 20, 2020): It does in fact support signed URLs. There should be no additional setup on the `fake-gcs-server` side outside of creating the bucket. If you run into CORS issues over unsupported headers, you can pass in the `-cors-headers` flag with a list of comma separated headers.

kerem commented

2026-03-03 12:08:02 +03:00

Author

Owner

@fsouza commented on GitHub (Dec 22, 2020):

@steezeburger signed urls are generated by the client and as far I understand they only have some extra querystring parameters which we ignore, so yes they should work. There's no validation of the signature though, so if you're using this to test that your client signs the URL correctly you may want to use a different strategy.

@fsouza commented on GitHub (Dec 22, 2020): @steezeburger signed urls are generated by the client and as far I understand they only have some extra querystring parameters which we ignore, so yes they should work. There's no validation of the signature though, so if you're using this to test that your client signs the URL correctly you may want to use a different strategy.

kerem commented

2026-03-03 12:08:02 +03:00

Author

Owner

@steezeburger commented on GitHub (Dec 22, 2020):

Thanks @kenniaa and @fsouza

I realized I was also needing a mock of the cloud storage SDK, not just the endpoints. This library is still super helpful though, I just misunderstood my needs and what this repo solves. Thanks again for the work and replies!

@steezeburger commented on GitHub (Dec 22, 2020): Thanks @kenniaa and @fsouza I realized I was also needing a mock of the cloud storage SDK, not just the endpoints. This library is still super helpful though, I just misunderstood my needs and what this repo solves. Thanks again for the work and replies!

kerem commented

2026-03-03 12:08:02 +03:00

Author

Owner

@fsouza commented on GitHub (Dec 22, 2020):

@steezeburger no problem, I'm sorry it took me so long to get to this (EOY productivity slump heh). Feel free to open new issues in the future if you have questions! :D

@fsouza commented on GitHub (Dec 22, 2020): @steezeburger no problem, I'm sorry it took me so long to get to this (EOY productivity slump heh). Feel free to open new issues in the future if you have questions! :D

kerem commented

2026-03-03 12:08:02 +03:00

Author

Owner

@vikramray-fc commented on GitHub (Sep 27, 2021):

Hi, thanks for this project. I was able to generate a download url http://0.0.0.0:4443/storage/v1/b/bucket/o/example.json?alt=media but I am not able to generate upload URL, any example would be helpful. Thanks!

@vikramray-fc commented on GitHub (Sep 27, 2021): Hi, thanks for this project. I was able to generate a download url `http://0.0.0.0:4443/storage/v1/b/bucket/o/example.json?alt=media` but I am not able to generate upload URL, any example would be helpful. Thanks!

kerem commented

2026-03-03 12:08:02 +03:00

Author

Owner

@fsouza commented on GitHub (Oct 1, 2021):

Hi, thanks for this project. I was able to generate a download url http://0.0.0.0:4443/storage/v1/b/bucket/o/example.json?alt=media but I am not able to generate upload URL, any example would be helpful. Thanks!

Hmm it should just work. Can you share a snippet of code that you expected to work and didn't work? Thanks!

@fsouza commented on GitHub (Oct 1, 2021): > Hi, thanks for this project. I was able to generate a download url `http://0.0.0.0:4443/storage/v1/b/bucket/o/example.json?alt=media` but I am not able to generate upload URL, any example would be helpful. Thanks! Hmm it should just work. Can you share a snippet of code that you expected to work and didn't work? Thanks!

kerem commented

2026-03-03 12:08:02 +03:00

Author

Owner

@vikramray-fc commented on GitHub (Dec 7, 2021):

Hi @fsouza, Sorry for late response.

I have generated a signed URL from the gcp code snippet and its working for cloud storage. (without emulator)

def generate_upload_signed_url(bucket_name, blob_name,content_type,expiration=15):
    storage_client = storage.Client()
    bucket = storage_client.bucket(bucket_name)
    blob = bucket.blob(blob_name)
    url = blob.generate_signed_url(
        version="v4",
        expiration=timedelta(minutes=expiration),
        method="PUT",
        content_type=content_type,
    )
    return url

While using emulator, I generated a signed URL with the above snippets, it returned -

https://storage.googleapis.com/fc/uploads/vikram.com/ojoijoijoi.csv?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=fc-team%40vikram1.iam.gserviceaccount.com%2F20211207%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20211207T091804Z&X-Goog-Expires=1800&X-Goog-SignedHeaders=content-type%3Bhost&X-Goog-Signature=88355ed841e88bfbdfedff41bd59ss287701ed298e7e20a29bd107f7e23fd5f54b960b2da753879661f1120f833db4ca1c362f5e515ca0dad89cc128f3640a9ead0c4dce9500b44469d74d6fde07f3dd2c563a891c450bdb899c40904cc4d71b718be0a48298a48eabbc5e6e8d31336ad0fa1aebf0588add5b7141e8140e0434d59879ed9656e7f4d0c7cdc70b9a1900574e4169688dcc508d7330b7ed12d4ddf8f2cbbdd74e948bb1ae9f1af52f48648127a89a5221c326d103cca1a1710f3e6f35ed5263110bd15379f6559404106e113ae1b82a37076390ef6c5255a69abacd7ec48bdd8df5ee5c44454c997c0365fd2ed16a4b83da341dfe10403916e9be9be

Which is not a emulator URL and also i tried replacing the https://storage.googleapis.com with http://127.0.0.1:4443 but it didnt worked.

Any help will be useful.

@vikramray-fc commented on GitHub (Dec 7, 2021): Hi @fsouza, Sorry for late response. I have generated a signed URL from the gcp code snippet and its working for cloud storage. (without emulator) ``` def generate_upload_signed_url(bucket_name, blob_name,content_type,expiration=15): storage_client = storage.Client() bucket = storage_client.bucket(bucket_name) blob = bucket.blob(blob_name) url = blob.generate_signed_url( version="v4", expiration=timedelta(minutes=expiration), method="PUT", content_type=content_type, ) return url ``` While using emulator, I generated a signed URL with the above snippets, it returned - `https://storage.googleapis.com/fc/uploads/vikram.com/ojoijoijoi.csv?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=fc-team%40vikram1.iam.gserviceaccount.com%2F20211207%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20211207T091804Z&X-Goog-Expires=1800&X-Goog-SignedHeaders=content-type%3Bhost&X-Goog-Signature=88355ed841e88bfbdfedff41bd59ss287701ed298e7e20a29bd107f7e23fd5f54b960b2da753879661f1120f833db4ca1c362f5e515ca0dad89cc128f3640a9ead0c4dce9500b44469d74d6fde07f3dd2c563a891c450bdb899c40904cc4d71b718be0a48298a48eabbc5e6e8d31336ad0fa1aebf0588add5b7141e8140e0434d59879ed9656e7f4d0c7cdc70b9a1900574e4169688dcc508d7330b7ed12d4ddf8f2cbbdd74e948bb1ae9f1af52f48648127a89a5221c326d103cca1a1710f3e6f35ed5263110bd15379f6559404106e113ae1b82a37076390ef6c5255a69abacd7ec48bdd8df5ee5c44454c997c0365fd2ed16a4b83da341dfe10403916e9be9be` Which is not a emulator URL and also i tried replacing the `https://storage.googleapis.com` with `http://127.0.0.1:4443` but it didnt worked. Any help will be useful.

kerem commented

2026-03-03 12:08:02 +03:00

Author

Owner

@thaont-0210 commented on GitHub (Feb 22, 2022):

@vikramray-fc
I think you have found your own solution, but I think my solution can help some one other.
In Go, SignedURLOptions have property is Style, we use style BucketBoundHostname as below.

opts := &storage.SignedURLOptions{
	GoogleAccessID: "thao@hihi.com",
	PrivateKey: dummyRsa(),
	Scheme:  storage.SigningSchemeV4,
	Method:  "GET",
	Expires: time.Now().Add(15 * time.Minute),
	Style: storage.BucketBoundHostname("localhost:8080/"+bucket),
	Insecure: true,
	ContentType: "text/csv",
}

u, _ := client.Bucket(bucket).SignedURL(object, opts)

u will be encoded, so, url will be http://localhost:8080%2Fbucket-hihi/abc.csv...... => we need decode that url

decodedValue, _ := url.QueryUnescape(u)

I use signed url for get method, I think put the same

@thaont-0210 commented on GitHub (Feb 22, 2022): @vikramray-fc I think you have found your own solution, but I think my solution can help some one other. In Go, `SignedURLOptions` have property is `Style`, we use style `BucketBoundHostname` as below. ```Go opts := &storage.SignedURLOptions{ GoogleAccessID: "thao@hihi.com", PrivateKey: dummyRsa(), Scheme: storage.SigningSchemeV4, Method: "GET", Expires: time.Now().Add(15 * time.Minute), Style: storage.BucketBoundHostname("localhost:8080/"+bucket), Insecure: true, ContentType: "text/csv", } u, _ := client.Bucket(bucket).SignedURL(object, opts) ``` `u` will be encoded, so, url will be `http://localhost:8080%2Fbucket-hihi/abc.csv......` => we need decode that url ```Go decodedValue, _ := url.QueryUnescape(u) ``` I use signed url for get method, I think put the same

kerem commented

2026-03-03 12:08:02 +03:00

Author

Owner

@mj3c commented on GitHub (Mar 16, 2022):

I did not manage to generate a signed URL for uploading using Python. Regular uploading/downloading of blobs work.

Here's what I tried:

import datetime
from google.cloud import storage

client = storage.Client()
bucket = client.bucket('test')
blob = bucket.blob('testblob')

blob.generate_signed_url(expiration=datetime.timedelta(minutes=15))

This is the error:

AttributeError: you need a private key to sign credentials.the credentials you are currently using <class 'google.auth.credentials.AnonymousCredentials'> just contains a token. see https://googleapis.dev/python/google-api-core/latest/auth.html#setting-up-a-service-account for more details.

And this is from the generate_signed_url docstring:

def generate_upload_signed_url_v4(bucket_name, blob_name):
    """Generates a v4 signed URL for uploading a blob using HTTP PUT.
    Note that this method requires a service account key file. You can not use
    this if you are using Application Default Credentials from Google Compute
    Engine or from the Google Cloud SDK.
    """

I tried creating a gcp.json file with all fake values and using that when initializing the Client but then got an error:

google.auth.exceptions.RefreshError: ('invalid_grant: Invalid grant: account not found', {'error': 'invalid_grant', 'error_description': 'Invalid grant: account not found'})

Any ideas on how to work around this?

@mj3c commented on GitHub (Mar 16, 2022): I did not manage to generate a signed URL for uploading using Python. Regular uploading/downloading of blobs work. Here's what I tried: ```python import datetime from google.cloud import storage client = storage.Client() bucket = client.bucket('test') blob = bucket.blob('testblob') blob.generate_signed_url(expiration=datetime.timedelta(minutes=15)) ``` This is the error: `AttributeError: you need a private key to sign credentials.the credentials you are currently using <class 'google.auth.credentials.AnonymousCredentials'> just contains a token. see https://googleapis.dev/python/google-api-core/latest/auth.html#setting-up-a-service-account for more details.` And this is from the `generate_signed_url` docstring: ```python def generate_upload_signed_url_v4(bucket_name, blob_name): """Generates a v4 signed URL for uploading a blob using HTTP PUT. Note that this method requires a service account key file. You can not use this if you are using Application Default Credentials from Google Compute Engine or from the Google Cloud SDK. """ ``` I tried creating a gcp.json file with all fake values and using that when initializing the `Client` but then got an error: `google.auth.exceptions.RefreshError: ('invalid_grant: Invalid grant: account not found', {'error': 'invalid_grant', 'error_description': 'Invalid grant: account not found'})` Any ideas on how to work around this?

kerem commented

2026-03-03 12:08:03 +03:00

Author

Owner

@fxn commented on GitHub (May 30, 2022):

you need a private key to sign credentials.the credentials you are currently using <class 'google.auth.credentials.AnonymousCredentials'> just contains a token

Facing the same issue.

@fxn commented on GitHub (May 30, 2022): > you need a private key to sign credentials.the credentials you are currently using <class 'google.auth.credentials.AnonymousCredentials'> just contains a token Facing the same issue.

kerem commented

2026-03-03 12:08:03 +03:00

Author

Owner

@fxn commented on GitHub (May 31, 2022):

Tried using this class

class TestCredentials(AnonymousCredentials, Signing):
    _private_key = serialization.load_pem_private_key(PRIVATE_KEY, password=None)
    _signer = RSASigner(_private_key)

    def sign_bytes(self, message):
        return self.signer().sign(message)

    def signer_email(self):
        return 'foo@example.com'

    def signer(self):
        return self._signer

For a dummy private key generated with openssl, and I get pass that error message, to now get a HTTP response of

b"<?xml version='1.0' encoding='UTF-8'?><Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method.</Message><StringToSign>GET\n\n\n1653984505\n/cdk/a.csv</StringToSign></Error>"

Can anybody help? This is just for a test suite.

@fxn commented on GitHub (May 31, 2022): Tried using this class ```python class TestCredentials(AnonymousCredentials, Signing): _private_key = serialization.load_pem_private_key(PRIVATE_KEY, password=None) _signer = RSASigner(_private_key) def sign_bytes(self, message): return self.signer().sign(message) def signer_email(self): return 'foo@example.com' def signer(self): return self._signer ``` For a dummy private key generated with `openssl`, and I get pass that error message, to now get a HTTP response of ``` b"<?xml version='1.0' encoding='UTF-8'?><Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method.</Message><StringToSign>GET\n\n\n1653984505\n/cdk/a.csv</StringToSign></Error>" ``` Can anybody help? This is just for a test suite.

kerem commented

2026-03-03 12:08:03 +03:00

Author

Owner

@papagian commented on GitHub (Feb 21, 2025):

Any ideas on how to work around this?

Recover from reauth related error worked for me!

@papagian commented on GitHub (Feb 21, 2025): > Any ideas on how to work around this? [Recover from reauth related error](https://support.google.com/a/answer/9368756#reauth_related_error) worked for me!

kerem referenced this issue

2026-03-03 12:31:07 +03:00

[PR #512] [MERGED] Add support for copyTo via rewriteTo #674

kerem referenced this issue

2026-03-15 17:49:29 +03:00

[GH-ISSUE #72] Copy file from bucket to bucket is not supported #2165

Rows
Columns

[GH-ISSUE #381] Signed URLs? #72