starred/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2026-04-25 13:26:00 +03:00
Code Issues 305 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #2063] CheckBucket fails with restrictive IAM policy and mounting a path #1043

New issue
Closed
opened 2026-03-04 01:50:54 +03:00 by kerem · 5 comments
kerem commented 2026-03-04 01:50:54 +03:00
Owner
Copy link

Originally created by @francisATgwn on GitHub (Nov 22, 2022).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2063

Version of s3fs being used (s3fs --version)

Amazon Simple Storage Service File System V1.91 (commit:d98fdf4) with OpenSSL

Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse)

2.9.9

Kernel information (uname -r)

5.15.0-1023-aws

GNU/Linux Distribution, if applicable (cat /etc/os-release)

PRETTY_NAME="Ubuntu 22.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.1 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

s3fs command line used, if applicable

sudo s3fs -f REDACTED:/path/to/redacted /mnt/foo -o ro,_netdev,passwd_file=REDACTED,allow_other,umask=0007,nodev,dbglevel=info,curldbg=body

s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)

2022-11-22T20:25:42.934Z [CRT] s3fs_logger.cpp:LowSetLogLevel(240): change debug level from [CRT] to [INF]
2022-11-22T20:25:42.934Z [INF]     s3fs.cpp:set_mountpoint_attribute(4136): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755)
2022-11-22T20:25:42.938Z [INF] curl.cpp:InitMimeType(428): Loaded mime information from /etc/mime.types
2022-11-22T20:25:42.938Z [INF] fdcache_stat.cpp:CheckCacheFileStatTopDir(78): The path to cache top dir is empty, thus not need to check permission.
2022-11-22T20:25:42.938Z [INF] threadpoolman.cpp:StopThreads(195): Any threads are running now, then nothing to do.
2022-11-22T20:25:42.938Z [INF]       threadpoolman.cpp:Worker(76): Start worker thread in ThreadPoolMan.
2022-11-22T20:25:42.938Z [INF]       threadpoolman.cpp:Worker(76): Start worker thread in ThreadPoolMan.
2022-11-22T20:25:42.938Z [INF]       threadpoolman.cpp:Worker(76): Start worker thread in ThreadPoolMan.
2022-11-22T20:25:42.942Z [INF]       threadpoolman.cpp:Worker(76): Start worker thread in ThreadPoolMan.
2022-11-22T20:25:42.942Z [CRT] s3fs_cred.cpp:VersionS3fsCredential(60): Check why built-in function was called, the external credential library must have VersionS3fsCredential function.
2022-11-22T20:25:42.943Z [INF] s3fs.cpp:s3fs_init(3902): init v1.91(commit:d98fdf4) with OpenSSL, credential-library(built-in)
2022-11-22T20:25:42.943Z [INF] s3fs.cpp:s3fs_check_service(4022): check services.
2022-11-22T20:25:42.943Z [INF]       curl.cpp:CheckBucket(3523): check a bucket.
2022-11-22T20:25:42.943Z [INF]       curl_util.cpp:prepare_url(257): URL is https://s3.amazonaws.com/REDACTED/
2022-11-22T20:25:42.943Z [INF]       curl_util.cpp:prepare_url(290): URL changed is https://REDACTED.s3.amazonaws.com/
2022-11-22T20:25:42.943Z [INF]       curl.cpp:insertV4Headers(2741): computing signature [GET] [/] [] []
2022-11-22T20:25:42.943Z [INF]       curl_util.cpp:url_to_host(334): url is https://s3.amazonaws.com
2022-11-22T20:25:42.944Z [INF]       threadpoolman.cpp:Worker(76): Start worker thread in ThreadPoolMan.
2022-11-22T20:25:42.947Z [CURL DBG] *   Trying 52.216.245.164:443...
2022-11-22T20:25:42.947Z [CURL DBG] * Connected to REDACTED.s3.amazonaws.com (52.216.245.164) port 443 (#0)
2022-11-22T20:25:43.039Z [CURL DBG] *  CAfile: /etc/ssl/certs/ca-certificates.crt
2022-11-22T20:25:43.039Z [CURL DBG] *  CApath: /etc/ssl/certs
2022-11-22T20:25:43.039Z [CURL DBG] * TLSv1.0 (OUT), TLS header, Certificate Status (22):
2022-11-22T20:25:43.039Z [CURL DBG] * TLSv1.3 (OUT), TLS handshake, Client hello (1):
2022-11-22T20:25:43.040Z [CURL DBG] * TLSv1.2 (IN), TLS header, Certificate Status (22):
2022-11-22T20:25:43.040Z [CURL DBG] * TLSv1.3 (IN), TLS handshake, Server hello (2):
2022-11-22T20:25:43.040Z [CURL DBG] * TLSv1.2 (IN), TLS header, Certificate Status (22):
2022-11-22T20:25:43.040Z [CURL DBG] * TLSv1.2 (IN), TLS handshake, Certificate (11):
2022-11-22T20:25:43.043Z [CURL DBG] * TLSv1.2 (IN), TLS header, Certificate Status (22):
2022-11-22T20:25:43.043Z [CURL DBG] * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
2022-11-22T20:25:43.043Z [CURL DBG] * TLSv1.2 (IN), TLS header, Certificate Status (22):
2022-11-22T20:25:43.043Z [CURL DBG] * TLSv1.2 (IN), TLS handshake, Server finished (14):
2022-11-22T20:25:43.043Z [CURL DBG] * TLSv1.2 (OUT), TLS header, Certificate Status (22):
2022-11-22T20:25:43.043Z [CURL DBG] * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
2022-11-22T20:25:43.044Z [CURL DBG] * TLSv1.2 (OUT), TLS header, Finished (20):
2022-11-22T20:25:43.044Z [CURL DBG] * TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
2022-11-22T20:25:43.044Z [CURL DBG] * TLSv1.2 (OUT), TLS header, Certificate Status (22):
2022-11-22T20:25:43.044Z [CURL DBG] * TLSv1.2 (OUT), TLS handshake, Finished (20):
2022-11-22T20:25:43.044Z [CURL DBG] * TLSv1.2 (IN), TLS header, Finished (20):
2022-11-22T20:25:43.044Z [CURL DBG] * TLSv1.2 (IN), TLS header, Certificate Status (22):
2022-11-22T20:25:43.044Z [CURL DBG] * TLSv1.2 (IN), TLS handshake, Finished (20):
2022-11-22T20:25:43.045Z [CURL DBG] * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
2022-11-22T20:25:43.045Z [CURL DBG] * Server certificate:
2022-11-22T20:25:43.045Z [CURL DBG] *  subject: CN=*.s3.amazonaws.com
2022-11-22T20:25:43.045Z [CURL DBG] *  start date: Sep 21 00:00:00 2022 GMT
2022-11-22T20:25:43.045Z [CURL DBG] *  expire date: Aug 26 23:59:59 2023 GMT
2022-11-22T20:25:43.045Z [CURL DBG] *  subjectAltName: host "REDACTED.s3.amazonaws.com" matched cert's "*.s3.amazonaws.com"
2022-11-22T20:25:43.045Z [CURL DBG] *  issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon
2022-11-22T20:25:43.045Z [CURL DBG] *  SSL certificate verify ok.
2022-11-22T20:25:43.045Z [CURL DBG] * TLSv1.2 (OUT), TLS header, Supplemental data (23):
2022-11-22T20:25:43.045Z [CURL DBG] > GET / HTTP/1.1
2022-11-22T20:25:43.045Z [CURL DBG] > Host: REDACTED.s3.amazonaws.com
2022-11-22T20:25:43.045Z [CURL DBG] > User-Agent: s3fs/1.91 (commit hash d98fdf4; OpenSSL)
2022-11-22T20:25:43.045Z [CURL DBG] > Accept: */*
2022-11-22T20:25:43.045Z [CURL DBG] > Authorization: AWS4-HMAC-SHA256 Credential=REDACTED/20221122/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=6927117d434483ccd33b2d2f17ae495985fab30ca1f82b394363b591b99131bc
2022-11-22T20:25:43.045Z [CURL DBG] > x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2022-11-22T20:25:43.045Z [CURL DBG] > x-amz-date: 20221122T202542Z
2022-11-22T20:25:43.045Z [CURL DBG] >
2022-11-22T20:25:43.075Z [CURL DBG] * TLSv1.2 (IN), TLS header, Supplemental data (23):
2022-11-22T20:25:43.075Z [CURL DBG] * Mark bundle as not supporting multiuse
2022-11-22T20:25:43.075Z [CURL DBG] < HTTP/1.1 403 Forbidden
2022-11-22T20:25:43.075Z [CURL DBG] < x-amz-bucket-region: us-east-1
2022-11-22T20:25:43.075Z [CURL DBG] < x-amz-request-id: 06XDAXAHDWPB5P4E
2022-11-22T20:25:43.075Z [CURL DBG] < x-amz-id-2: A94mVE2Cq/M1bCwvUG0EGN46rY2k8aBRga47mqeDCtVghdP+2sTqjoTB/RLZMcppQGp7wtJNM2U=
2022-11-22T20:25:43.075Z [CURL DBG] < Content-Type: application/xml
2022-11-22T20:25:43.075Z [CURL DBG] < Transfer-Encoding: chunked
2022-11-22T20:25:43.075Z [CURL DBG] < Date: Tue, 22 Nov 2022 20:25:42 GMT
2022-11-22T20:25:43.075Z [CURL DBG] < Server: AmazonS3

Details about issue

We have a setup that mounts a path under an S3 bucket. The IAM user in question is configured with the minimum necessary permissions, it is restricted to listing and reading objects only under its path prefix in the bucket. It does not have permission to ListBucket on the root of the bucket.

While upgrading from Ubuntu 20.04 to 22.04, s3fs upgraded from 1.86 to 1.90 and experienced a regression in functionality. It worked in 1.86 and fails in 1.90 without any configuration change on our part.

I ran git bisect which found that d3278f4886 is the commit that introduced the regression.

Here is the IAM policy in effect on s3fs:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::REDACTED"
            ],
            "Condition": {
                "StringLike": {
                    "s3:prefix": "path/to/redacted*"
                }
            }
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::REDACTED/path/to/redacted/*"
            ]
        }
    ]
}
Originally created by @francisATgwn on GitHub (Nov 22, 2022). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2063 #### Version of s3fs being used (s3fs --version) ``` Amazon Simple Storage Service File System V1.91 (commit:d98fdf4) with OpenSSL ``` #### Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse) `2.9.9` #### Kernel information (uname -r) `5.15.0-1023-aws` #### GNU/Linux Distribution, if applicable (cat /etc/os-release) ``` PRETTY_NAME="Ubuntu 22.04.1 LTS" NAME="Ubuntu" VERSION_ID="22.04" VERSION="22.04.1 LTS (Jammy Jellyfish)" VERSION_CODENAME=jammy ID=ubuntu ID_LIKE=debian HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" UBUNTU_CODENAME=jammy ``` #### s3fs command line used, if applicable ``` sudo s3fs -f REDACTED:/path/to/redacted /mnt/foo -o ro,_netdev,passwd_file=REDACTED,allow_other,umask=0007,nodev,dbglevel=info,curldbg=body ``` #### s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs) ``` 2022-11-22T20:25:42.934Z [CRT] s3fs_logger.cpp:LowSetLogLevel(240): change debug level from [CRT] to [INF] 2022-11-22T20:25:42.934Z [INF] s3fs.cpp:set_mountpoint_attribute(4136): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755) 2022-11-22T20:25:42.938Z [INF] curl.cpp:InitMimeType(428): Loaded mime information from /etc/mime.types 2022-11-22T20:25:42.938Z [INF] fdcache_stat.cpp:CheckCacheFileStatTopDir(78): The path to cache top dir is empty, thus not need to check permission. 2022-11-22T20:25:42.938Z [INF] threadpoolman.cpp:StopThreads(195): Any threads are running now, then nothing to do. 2022-11-22T20:25:42.938Z [INF] threadpoolman.cpp:Worker(76): Start worker thread in ThreadPoolMan. 2022-11-22T20:25:42.938Z [INF] threadpoolman.cpp:Worker(76): Start worker thread in ThreadPoolMan. 2022-11-22T20:25:42.938Z [INF] threadpoolman.cpp:Worker(76): Start worker thread in ThreadPoolMan. 2022-11-22T20:25:42.942Z [INF] threadpoolman.cpp:Worker(76): Start worker thread in ThreadPoolMan. 2022-11-22T20:25:42.942Z [CRT] s3fs_cred.cpp:VersionS3fsCredential(60): Check why built-in function was called, the external credential library must have VersionS3fsCredential function. 2022-11-22T20:25:42.943Z [INF] s3fs.cpp:s3fs_init(3902): init v1.91(commit:d98fdf4) with OpenSSL, credential-library(built-in) 2022-11-22T20:25:42.943Z [INF] s3fs.cpp:s3fs_check_service(4022): check services. 2022-11-22T20:25:42.943Z [INF] curl.cpp:CheckBucket(3523): check a bucket. 2022-11-22T20:25:42.943Z [INF] curl_util.cpp:prepare_url(257): URL is https://s3.amazonaws.com/REDACTED/ 2022-11-22T20:25:42.943Z [INF] curl_util.cpp:prepare_url(290): URL changed is https://REDACTED.s3.amazonaws.com/ 2022-11-22T20:25:42.943Z [INF] curl.cpp:insertV4Headers(2741): computing signature [GET] [/] [] [] 2022-11-22T20:25:42.943Z [INF] curl_util.cpp:url_to_host(334): url is https://s3.amazonaws.com 2022-11-22T20:25:42.944Z [INF] threadpoolman.cpp:Worker(76): Start worker thread in ThreadPoolMan. 2022-11-22T20:25:42.947Z [CURL DBG] * Trying 52.216.245.164:443... 2022-11-22T20:25:42.947Z [CURL DBG] * Connected to REDACTED.s3.amazonaws.com (52.216.245.164) port 443 (#0) 2022-11-22T20:25:43.039Z [CURL DBG] * CAfile: /etc/ssl/certs/ca-certificates.crt 2022-11-22T20:25:43.039Z [CURL DBG] * CApath: /etc/ssl/certs 2022-11-22T20:25:43.039Z [CURL DBG] * TLSv1.0 (OUT), TLS header, Certificate Status (22): 2022-11-22T20:25:43.039Z [CURL DBG] * TLSv1.3 (OUT), TLS handshake, Client hello (1): 2022-11-22T20:25:43.040Z [CURL DBG] * TLSv1.2 (IN), TLS header, Certificate Status (22): 2022-11-22T20:25:43.040Z [CURL DBG] * TLSv1.3 (IN), TLS handshake, Server hello (2): 2022-11-22T20:25:43.040Z [CURL DBG] * TLSv1.2 (IN), TLS header, Certificate Status (22): 2022-11-22T20:25:43.040Z [CURL DBG] * TLSv1.2 (IN), TLS handshake, Certificate (11): 2022-11-22T20:25:43.043Z [CURL DBG] * TLSv1.2 (IN), TLS header, Certificate Status (22): 2022-11-22T20:25:43.043Z [CURL DBG] * TLSv1.2 (IN), TLS handshake, Server key exchange (12): 2022-11-22T20:25:43.043Z [CURL DBG] * TLSv1.2 (IN), TLS header, Certificate Status (22): 2022-11-22T20:25:43.043Z [CURL DBG] * TLSv1.2 (IN), TLS handshake, Server finished (14): 2022-11-22T20:25:43.043Z [CURL DBG] * TLSv1.2 (OUT), TLS header, Certificate Status (22): 2022-11-22T20:25:43.043Z [CURL DBG] * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): 2022-11-22T20:25:43.044Z [CURL DBG] * TLSv1.2 (OUT), TLS header, Finished (20): 2022-11-22T20:25:43.044Z [CURL DBG] * TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1): 2022-11-22T20:25:43.044Z [CURL DBG] * TLSv1.2 (OUT), TLS header, Certificate Status (22): 2022-11-22T20:25:43.044Z [CURL DBG] * TLSv1.2 (OUT), TLS handshake, Finished (20): 2022-11-22T20:25:43.044Z [CURL DBG] * TLSv1.2 (IN), TLS header, Finished (20): 2022-11-22T20:25:43.044Z [CURL DBG] * TLSv1.2 (IN), TLS header, Certificate Status (22): 2022-11-22T20:25:43.044Z [CURL DBG] * TLSv1.2 (IN), TLS handshake, Finished (20): 2022-11-22T20:25:43.045Z [CURL DBG] * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256 2022-11-22T20:25:43.045Z [CURL DBG] * Server certificate: 2022-11-22T20:25:43.045Z [CURL DBG] * subject: CN=*.s3.amazonaws.com 2022-11-22T20:25:43.045Z [CURL DBG] * start date: Sep 21 00:00:00 2022 GMT 2022-11-22T20:25:43.045Z [CURL DBG] * expire date: Aug 26 23:59:59 2023 GMT 2022-11-22T20:25:43.045Z [CURL DBG] * subjectAltName: host "REDACTED.s3.amazonaws.com" matched cert's "*.s3.amazonaws.com" 2022-11-22T20:25:43.045Z [CURL DBG] * issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon 2022-11-22T20:25:43.045Z [CURL DBG] * SSL certificate verify ok. 2022-11-22T20:25:43.045Z [CURL DBG] * TLSv1.2 (OUT), TLS header, Supplemental data (23): 2022-11-22T20:25:43.045Z [CURL DBG] > GET / HTTP/1.1 2022-11-22T20:25:43.045Z [CURL DBG] > Host: REDACTED.s3.amazonaws.com 2022-11-22T20:25:43.045Z [CURL DBG] > User-Agent: s3fs/1.91 (commit hash d98fdf4; OpenSSL) 2022-11-22T20:25:43.045Z [CURL DBG] > Accept: */* 2022-11-22T20:25:43.045Z [CURL DBG] > Authorization: AWS4-HMAC-SHA256 Credential=REDACTED/20221122/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=6927117d434483ccd33b2d2f17ae495985fab30ca1f82b394363b591b99131bc 2022-11-22T20:25:43.045Z [CURL DBG] > x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 2022-11-22T20:25:43.045Z [CURL DBG] > x-amz-date: 20221122T202542Z 2022-11-22T20:25:43.045Z [CURL DBG] > 2022-11-22T20:25:43.075Z [CURL DBG] * TLSv1.2 (IN), TLS header, Supplemental data (23): 2022-11-22T20:25:43.075Z [CURL DBG] * Mark bundle as not supporting multiuse 2022-11-22T20:25:43.075Z [CURL DBG] < HTTP/1.1 403 Forbidden 2022-11-22T20:25:43.075Z [CURL DBG] < x-amz-bucket-region: us-east-1 2022-11-22T20:25:43.075Z [CURL DBG] < x-amz-request-id: 06XDAXAHDWPB5P4E 2022-11-22T20:25:43.075Z [CURL DBG] < x-amz-id-2: A94mVE2Cq/M1bCwvUG0EGN46rY2k8aBRga47mqeDCtVghdP+2sTqjoTB/RLZMcppQGp7wtJNM2U= 2022-11-22T20:25:43.075Z [CURL DBG] < Content-Type: application/xml 2022-11-22T20:25:43.075Z [CURL DBG] < Transfer-Encoding: chunked 2022-11-22T20:25:43.075Z [CURL DBG] < Date: Tue, 22 Nov 2022 20:25:42 GMT 2022-11-22T20:25:43.075Z [CURL DBG] < Server: AmazonS3 ``` ### Details about issue We have a setup that mounts a path under an S3 bucket. The IAM user in question is configured with the minimum necessary permissions, it is restricted to listing and reading objects only under its path prefix in the bucket. It does not have permission to ListBucket on the root of the bucket. While upgrading from Ubuntu 20.04 to 22.04, s3fs upgraded from 1.86 to 1.90 and experienced a regression in functionality. It worked in 1.86 and fails in 1.90 without any configuration change on our part. I ran git bisect which found that d3278f488676990560ed0d4066220ee4f00183ec is the commit that introduced the regression. Here is the IAM policy in effect on s3fs: ```json { "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": [ "s3:ListBucket" ], "Resource": [ "arn:aws:s3:::REDACTED" ], "Condition": { "StringLike": { "s3:prefix": "path/to/redacted*" } } }, { "Sid": "VisualEditor1", "Effect": "Allow", "Action": [ "s3:GetObject" ], "Resource": [ "arn:aws:s3:::REDACTED/path/to/redacted/*" ] } ] } ```
kerem closed this issue 2026-03-04 01:50:54 +03:00
kerem commented 2026-03-04 01:50:55 +03:00
Author
Owner
Copy link

@francisATgwn commented on GitHub (Nov 29, 2022):

To work around this issue until a proper fix is available, we have expanded the IAM permissions by adding the following statement to the policy I included in the issue description. This gives the s3fs user access to run ListBucket on the root of the S3 bucket in question.

{
    "Sid": "BucketRootLevelListAccess",
    "Effect": "Allow",
    "Action": "s3:ListBucket",
    "Resource": "arn:aws:s3:::REDACTED",
    "Condition": {
        "StringEquals": {
            "s3:prefix": [
                ""
            ],
            "s3:delimiter": [
                "/"
            ]
        }
    }
}

However, this was ALSO too restrictive for s3fs. This IAM policy statement does allow listing the root of the S3 bucket:

~ $ aws s3api list-objects-v2 --bucket REDACTED --delimiter / --prefix ""
{
    "CommonPrefixes": [
...

However, s3fs is still getting a 403, which likely means it is not including the delimiter and prefix parameters in the request.

This statement is now what we're using as the workaround.

{
    "Sid": "BucketRootLevelListAccess",
    "Effect": "Allow",
    "Action": "s3:ListBucket",
    "Resource": "arn:aws:s3:::REDACTED"
}
<!-- gh-comment-id:1331130635 --> @francisATgwn commented on GitHub (Nov 29, 2022): To work around this issue until a proper fix is available, we have expanded the IAM permissions by adding the following statement to the policy I included in the issue description. This gives the s3fs user access to run ListBucket on the root of the S3 bucket in question. ```json { "Sid": "BucketRootLevelListAccess", "Effect": "Allow", "Action": "s3:ListBucket", "Resource": "arn:aws:s3:::REDACTED", "Condition": { "StringEquals": { "s3:prefix": [ "" ], "s3:delimiter": [ "/" ] } } } ``` However, this was ALSO too restrictive for s3fs. This IAM policy statement does allow listing the root of the S3 bucket: ``` ~ $ aws s3api list-objects-v2 --bucket REDACTED --delimiter / --prefix "" { "CommonPrefixes": [ ... ``` However, s3fs is still getting a 403, which likely means it is not including the `delimiter` and `prefix` parameters in the request. This statement is now what we're using as the workaround. ```json { "Sid": "BucketRootLevelListAccess", "Effect": "Allow", "Action": "s3:ListBucket", "Resource": "arn:aws:s3:::REDACTED" } ```
kerem commented 2026-03-04 01:50:55 +03:00
Author
Owner
Copy link

@arontsang commented on GitHub (Jan 4, 2023):

+1 We also have this problem. But we cannot use this workaround.
Our company uses shared buckets and partition them to each group.

We can't give everyone root read access, nor can we provision a bucket per group (aws bucket limits).

<!-- gh-comment-id:1370506315 --> @arontsang commented on GitHub (Jan 4, 2023): +1 We also have this problem. But we cannot use this workaround. Our company uses shared buckets and partition them to each group. We can't give everyone root read access, nor can we provision a bucket per group (aws bucket limits).
kerem commented 2026-03-04 01:50:55 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Jan 5, 2023):

@francisATgwn @arontsang
I'm sorry for late to reply and thanks for the details of the problem.

I've created PR #2087 to confirm what the bug was and to make it work as before.
This PR code checks bucket access at first, and if that fails, checks including the directory path.
I think that this code should allow the check to pass as before.

I would appreciate it if you could try it with the code in this PR or please wait for merging.

<!-- gh-comment-id:1372128938 --> @ggtakec commented on GitHub (Jan 5, 2023): @francisATgwn @arontsang I'm sorry for late to reply and thanks for the details of the problem. I've created PR #2087 to confirm what the bug was and to make it work as before. This PR code checks bucket access at first, and if that fails, checks including the directory path. I think that this code should allow the check to pass as before. I would appreciate it if you could try it with the code in this PR or please wait for merging.
kerem commented 2026-03-04 01:50:55 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Jan 9, 2023):

#2087 has been merged. Please try it if you can.
This issue will be closed, but if the problem still exists, please reopen it or post a new issue.

<!-- gh-comment-id:1375743775 --> @ggtakec commented on GitHub (Jan 9, 2023): #2087 has been merged. Please try it if you can. This issue will be closed, but if the problem still exists, please reopen it or post a new issue.
kerem commented 2026-03-04 01:50:55 +03:00
Author
Owner
Copy link

@francisATgwn commented on GitHub (Jan 31, 2023):

Confirmed.

<!-- gh-comment-id:1410525632 --> @francisATgwn commented on GitHub (Jan 31, 2023): Confirmed.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
v1.97
v1.96
v1.95
v1.94
v1.93
v1.92
v1.91
v1.90
v1.89
v1.88
v1.87
v1.86
v1.85
v1.84
v1.83
v1.82
v1.81
v1.80
v1.79
v1.78
Pre-v1.78
v1.77
v1.76
v1.75
v1.74
Labels
Clear labels   
bug

   
bug

   
dataloss

   
duplicate

   
enhancement

   
feature request

   
help wanted

   
invalid

   
need info

   
performance

   
pull-request

Mirrored from GitHub Pull Request

  
question

   
question

   
testing
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#1043
No description provided.