starred/fake-gcs-server

Fork 0

mirror of https://github.com/fsouza/fake-gcs-server.git synced 2026-04-28 07:05:52 +03:00

[GH-ISSUE #941] make_public fails due to invalid "projectOwner" ACL entity #148

New issue

Closed

opened 2026-03-03 12:08:42 +03:00 by kerem · 4 comments

kerem commented

2026-03-03 12:08:42 +03:00

Owner

Originally created by @marcguidancenl on GitHub (Oct 5, 2022).
Original GitHub issue: https://github.com/fsouza/fake-gcs-server/issues/941

I'm just getting started on Google Cloud Storage, so please correct me if I'm doing something the wrong way. And, thanks for the emulator, it helps a lot.

The dev setup: fake-gcs-server is running in a Docker container, and I'm using Google's Python client. I've created buckets and files in buckets. The buckets are private by default (I think).

Now I want to make a single file public, using

bucket = client.get_bucket(bucket_name)
blob = bucket.blob(blob_name)
blob.make_public()

This gives me an exception that looks a bit like this:

  ... in make_public
    blob.acl.reload()
  File "/usr/local/lib/python3.10/site-packages/google/cloud/storage/acl.py", line 399, in reload
    self.add_entity(self.entity_from_dict(entry))
  File "/usr/local/lib/python3.10/site-packages/google/cloud/storage/acl.py", line 216, in entity_from_dict
    raise ValueError(f"Invalid dictionary: {entity_dict}")
ValueError: Invalid dictionary: {'bucket': 'cc3b6e77-6bf2-4532-bbd7-b8307a3e27c3', 'entity': 'projectOwner', 'object': 'some-image.svg', 'projectTeam': {}, 'role': 'OWNER'}

As far as I can tell, 'projectOwner' isn't a valid entity (https://cloud.google.com/storage/docs/access-control/lists#scopes). This value is returned from getObjectACL in upload.go, if there is no predefinedACL (I don't know Go either, just reading it to find some clues...).

The Google Python Client (https://github.com/googleapis/python-storage/blob/main/google/cloud/storage/acl.py) tries to create an ACL after the reload(), but it only recognizes 'allUsers', 'allAuthenticatedUsers', or entity-strings with a dash in them (elif "-" in entity).

The 'projectOwner' value triggers a ValueError:

    if not isinstance(entity, _ACLEntity):
        raise ValueError(f"Invalid dictionary: {entity_dict}")

If I look for the projectOwner value in the Google documentation, I can find it for the gsutil tool, but in relation to IAM, not ACL (these are related too, but different enough).

I'm not sure what getObjectACL should return if there is no predefinedACL. Perhaps an empty list? It looks like that's what blob.acl.clear() does.

My current workaround is to just disable make_public and make_private in the dev environment, and hope all is well when I switch to an actual GCS environment for testing. I would be really happy if I can get this working in my dev environment too.

Thanks, Marc.

Originally created by @marcguidancenl on GitHub (Oct 5, 2022). Original GitHub issue: https://github.com/fsouza/fake-gcs-server/issues/941 I'm just getting started on Google Cloud Storage, so please correct me if I'm doing something the wrong way. And, thanks for the emulator, it helps a lot. The dev setup: fake-gcs-server is running in a Docker container, and I'm using Google's Python client. I've created buckets and files in buckets. The buckets are private by default (I think). Now I want to make a single file public, using ``` bucket = client.get_bucket(bucket_name) blob = bucket.blob(blob_name) blob.make_public() ``` This gives me an exception that looks a bit like this: ``` ... in make_public blob.acl.reload() File "/usr/local/lib/python3.10/site-packages/google/cloud/storage/acl.py", line 399, in reload self.add_entity(self.entity_from_dict(entry)) File "/usr/local/lib/python3.10/site-packages/google/cloud/storage/acl.py", line 216, in entity_from_dict raise ValueError(f"Invalid dictionary: {entity_dict}") ValueError: Invalid dictionary: {'bucket': 'cc3b6e77-6bf2-4532-bbd7-b8307a3e27c3', 'entity': 'projectOwner', 'object': 'some-image.svg', 'projectTeam': {}, 'role': 'OWNER'} ``` As far as I can tell, 'projectOwner' isn't a valid entity (https://cloud.google.com/storage/docs/access-control/lists#scopes). This value is returned from getObjectACL in upload.go, if there is no predefinedACL (I don't know Go either, just reading it to find some clues...). The Google Python Client (https://github.com/googleapis/python-storage/blob/main/google/cloud/storage/acl.py) tries to create an ACL after the reload(), but it only recognizes 'allUsers', 'allAuthenticatedUsers', or entity-strings with a dash in them (`elif "-" in entity`). The 'projectOwner' value triggers a ValueError: ``` if not isinstance(entity, _ACLEntity): raise ValueError(f"Invalid dictionary: {entity_dict}") ``` If I look for the projectOwner value in the Google documentation, I can find it for the gsutil tool, but in relation to IAM, not ACL (these are related too, but different enough). I'm not sure what `getObjectACL` should return if there is no predefinedACL. Perhaps an empty list? It looks like that's what blob.acl.clear() does. --- My current workaround is to just disable make_public and make_private in the dev environment, and hope all is well when I switch to an actual GCS environment for testing. I would be really happy if I can get this working in my dev environment too. Thanks, Marc.

kerem

2026-03-03 12:08:42 +03:00

closed this issue
added the
bug
label

kerem commented

2026-03-03 12:08:44 +03:00

Author

Owner

@fsouza commented on GitHub (Oct 6, 2022):

@marcguidancenl thanks for reporting! Can you try #943?

@fsouza commented on GitHub (Oct 6, 2022): @marcguidancenl thanks for reporting! Can you try #943?

kerem commented

2026-03-03 12:08:44 +03:00

Author

Owner

@marcguidancenl commented on GitHub (Oct 6, 2022):

#943 did solve the issue, but while testing I discovered that I also got another ACL error when using make_public on data that is used to initialize the docker container.

this is the sample python code:

    os.environ["STORAGE_EMULATOR_HOST"] = "http://0.0.0.0:4443"

    client = storage.Client(
        credentials=AnonymousCredentials(),
        project="test-project",
    )

    # initial bucket/file for docker image
    bucket_name = "sample-bucket"
    blob_name = "some_file.txt"

    # test
    bucket = client.get_bucket(bucket_name)
    blob = bucket.blob(blob_name)
    blob.make_public()  #             <<<<< this errors on None not being iterable
    blob.make_private()  #             <<<<< this too

this gives TypeError: 'NoneType' object is not iterable in google/cloud/storage/acl.py", line 398
(as far as I can tell this is somewhere in response.go from aclListResponse to getAccessControlsListFromObject, where it somehow returns {"items":null} instead of {"items":[]} ??)

Doing the same thing with a bucket and blob created from code works now (needs #943), but I did notice that the ACL doesn't actually get changed with make_public and make_private. Sample code:

    os.environ["STORAGE_EMULATOR_HOST"] = "http://0.0.0.0:4443"

    client = storage.Client(
        credentials=AnonymousCredentials(),
        project="test-project",
    )

    # buckets/files to create and test
    upload_bucket_name = "test-bucket-with-globally-unique-name"
    upload_blob_name = "test-blob-upload.svg"
    upload_blob_file_path = "./image.svg"

    # initialize
    try:
        bucket = client.bucket(upload_bucket_name)
        bucket.storage_class = storage.constants.STANDARD_STORAGE_CLASS
        client.create_bucket(bucket, location="EU", retry=None)
    except:
        pass
    bucket = client.get_bucket(upload_bucket_name)
    blob = bucket.blob(upload_blob_name)
    blob.upload_from_filename(upload_blob_file_path, retry=None)

    # test
    bucket = client.get_bucket(upload_bucket_name)
    blob = bucket.blob(upload_blob_name)
    print(list(blob.acl))
    blob.make_public()
    print(list(blob.acl))  #            <<<<< didn't change ACL entities!
    blob.make_private()
    print(list(blob.acl))

this lists the ACL three times - initial ACL, public ACL, and private ACL:

[{'entity': 'projectOwner-test-project', 'role': 'OWNER'}]
[{'entity': 'projectOwner-test-project', 'role': 'OWNER'}]
[{'entity': 'projectOwner-test-project', 'role': 'OWNER'}]

Should this be two new, separate issues?

I don't really mind these issues, since I don't use the docker-image-initial-data and I don't check the ACLs, so I'm happy with the #943 fix :-)

@marcguidancenl commented on GitHub (Oct 6, 2022): #943 did solve the issue, but while testing I discovered that I also got another ACL error when using make_public on data that is used to initialize the docker container. this is the sample python code: ```python os.environ["STORAGE_EMULATOR_HOST"] = "http://0.0.0.0:4443" client = storage.Client( credentials=AnonymousCredentials(), project="test-project", ) # initial bucket/file for docker image bucket_name = "sample-bucket" blob_name = "some_file.txt" # test bucket = client.get_bucket(bucket_name) blob = bucket.blob(blob_name) blob.make_public() # <<<<< this errors on None not being iterable blob.make_private() # <<<<< this too ``` this gives `TypeError: 'NoneType' object is not iterable` in google/cloud/storage/acl.py", line 398 (as far as I can tell this is somewhere in `response.go` from aclListResponse to getAccessControlsListFromObject, where it somehow returns {"items":null} instead of {"items":[]} ??) Doing the same thing with a bucket and blob created from code works now (needs #943), but I did notice that the ACL doesn't actually get changed with make_public and make_private. Sample code: ```python os.environ["STORAGE_EMULATOR_HOST"] = "http://0.0.0.0:4443" client = storage.Client( credentials=AnonymousCredentials(), project="test-project", ) # buckets/files to create and test upload_bucket_name = "test-bucket-with-globally-unique-name" upload_blob_name = "test-blob-upload.svg" upload_blob_file_path = "./image.svg" # initialize try: bucket = client.bucket(upload_bucket_name) bucket.storage_class = storage.constants.STANDARD_STORAGE_CLASS client.create_bucket(bucket, location="EU", retry=None) except: pass bucket = client.get_bucket(upload_bucket_name) blob = bucket.blob(upload_blob_name) blob.upload_from_filename(upload_blob_file_path, retry=None) # test bucket = client.get_bucket(upload_bucket_name) blob = bucket.blob(upload_blob_name) print(list(blob.acl)) blob.make_public() print(list(blob.acl)) # <<<<< didn't change ACL entities! blob.make_private() print(list(blob.acl)) ``` this lists the ACL three times - initial ACL, public ACL, and private ACL: ``` [{'entity': 'projectOwner-test-project', 'role': 'OWNER'}] [{'entity': 'projectOwner-test-project', 'role': 'OWNER'}] [{'entity': 'projectOwner-test-project', 'role': 'OWNER'}] ``` --- Should this be two new, separate issues? I don't really mind these issues, since I don't use the docker-image-initial-data and I don't check the ACLs, so I'm happy with the #943 fix :-)

kerem commented

2026-03-03 12:08:44 +03:00

Author

Owner

@fsouza commented on GitHub (Oct 6, 2022):

Thank you for the detailed report. There's probably a lot of things to investigate around ACLs. Can you open separate GH issues for those issues?

@fsouza commented on GitHub (Oct 6, 2022): Thank you for the detailed report. There's probably a lot of things to investigate around ACLs. Can you open separate GH issues for those issues?

kerem commented

2026-03-03 12:08:44 +03:00

Author

Owner

@marcguidancenl commented on GitHub (Oct 6, 2022):

Sure, done :)

@marcguidancenl commented on GitHub (Oct 6, 2022): Sure, done :)

kerem referenced this issue

2026-03-03 12:29:44 +03:00

[PR #148] [MERGED] build(deps): bump google.golang.org/api from 0.14.0 to 0.15.0 in /examples/go #378