mirror of
https://github.com/fsouza/fake-gcs-server.git
synced 2026-04-26 06:05:54 +03:00
[GH-ISSUE #952] Signed URLs: Unable to authenticate with AnonymousCredentials #151
Labels
No labels
bug
compatibility-issue
docker
documentation
enhancement
help wanted
needs information
pull-request
question
stale
unfortunate
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/fake-gcs-server#151
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @Sorarinu on GitHub (Oct 15, 2022).
Original GitHub issue: https://github.com/fsouza/fake-gcs-server/issues/952
Hi, everyone!
When I generate Signed URLs using
service-account.jsonissued from GCP the authentication succeeds, but when I try to useAnonymousCredentials()as credentials it does not work.docker logs:
We would like to avoid getting GCS service-account.json in the local development environment as much as possible.
Is there a better way to do this?
@fsouza commented on GitHub (Oct 15, 2022):
Hey, are you saying that generating the URL fails or sending a request to the generated URL fails?
The reason I ask is because the server (fake or real) is not involved in generating URLs, it happens within the client.
@Sorarinu commented on GitHub (Oct 15, 2022):
It appears that the URL generation is failing.
As you pointed out, the URL generation and authentication are done by the google-cloud-storage client, so it was a mistake to inquire with fake-gcs-server 🙇
@fsouza commented on GitHub (Oct 15, 2022):
Yeah I think the client will simply not work with AnonymousCredentials. Perhaps you can have some process that generates a valid-but-fake service-account.json? IIRC, that's how I used to do it in a previous job.
@Sorarinu commented on GitHub (Oct 15, 2022):
I will try to generate valid-but-fake service-account.json and see if the URL generation succeeds.
@Sorarinu commented on GitHub (Oct 15, 2022):
I tried passing dummy information based on the format of valid service-account.json, and it succeeded in generating a Signed URL!
Thank you for your help!
@ddelange commented on GitHub (Dec 12, 2022):
@Sorarinu could you post your mock json here?
@ddelange commented on GitHub (Dec 12, 2022):
I'm currently stuck creating a valid fake json:
raises:
@fsouza commented on GitHub (Dec 12, 2022):
You can generate an actual file (with a service account) and revoke it, or use SSL to generate a private key.
@ddelange commented on GitHub (Dec 12, 2022):
yeah was looking into this but failing 😅 I'm writing tests for a storage backend I dont have access to (s3 guy here), so I'm already just really thankful for the existence of this repo! Will continue the fiddle
@ddelange commented on GitHub (Dec 13, 2022):
@fsouza I have a valid PEM for presigning URLs now, but the dummy token_uri from the service account used for JWT auth needs to point to a functional endpoint. does fake-gcs-server provide it?
will give a 404:
do you know how to get around it? I don't have creds to talk to googleapis.com (ref)
@fsouza commented on GitHub (Dec 13, 2022):
Hmm, fake-gcs-server doesn't have that endpoint because it only implements the Storage API. Curious on why you can't use AnonymousCredentials? Can you clarify the use case a bit more?
@ddelange commented on GitHub (Dec 13, 2022):
In https://github.com/stevearc/pypicloud/pull/320 I'm switching from mocked client classes to fake-gcs-server and the real client. Mainly because we missed a bug that would've been caught with the latter, and was missed because mocks weren't properly updated.
This test (testing that generating presigned url works) won't work with
AnonymousCredentials(this issue), so in test setUp I passed the PEM.But now
bucket.exists()(which happens during init ofpypicloud.storage.gcs.GoogleCloudStorage) won't work, because thetoken_uriis fake. That's the traceback in my last comment above. But probably this faketoken_uriwill break all other calls as well.So long story short: we have a test that mixes interacting with fake-gcs-server (only works with AnonymousCredentials) and generating presigned url (does not work with AnonymousCredentials).
@ddelange commented on GitHub (Dec 23, 2022):
Apart from patching
generate_presigned_urlwith a mock, is there another option you see?@fsouza commented on GitHub (Dec 28, 2022):
Hey, I'm really sorry for the delay, things got a but busy at the end of the year with the holidays and me starting a new job.
Gotcha, I think this is also related to how the Python SDK was designed. For example, in the Go SDK, you're allowed to invoke a function to generate a signed URL, without ever creating a client instance, because that's technically not required.
So in terms of moving forward, the options are:
generate_presigned_urllike you mentionedI'm OK with 3, as long as it's somewhat documented? And we can test it with a real SDK.
@ddelange commented on GitHub (Dec 28, 2022):
Thanks for the reply. I've patched the call that talks to this endpoint under the hood. Communicating with fake-gcs-server in the tests now works properly, despite passing a fake service account (with real PEM) to the client instead of
AnonymousCredentials.Thanks again!