mirror of
https://github.com/s3fs-fuse/s3fs-fuse.git
synced 2026-04-25 05:16:00 +03:00
[GH-ISSUE #1657] ls: reading directory '.': Operation not permitted #863
Labels
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/s3fs-fuse#863
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @martymcfly8891 on GitHub (May 11, 2021).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1657
Additional Information
Unable to list or touch anything in my mounted s3 via s3fs. The s3 bucket lives in account A and the ec2 lives in account B. i do not have this issue on any of the EC2s in account A but all ec2s in account B have this error:
ls: reading directory '.': Operation not permitted
the directories that s3 is mounted to is 777 and the ec2s have the same roles and an assumed role assigned to them.
Version of s3fs being used (s3fs --version)
Repository : Filesystem tools and FUSE-related packages (SLE_15_SP2)
Name : s3fs
Version : 1.89-33.2
Arch : x86_64
Vendor : obs://build.opensuse.org/filesystems
Support Level : unknown
Installed Size : 986.8 KiB
Installed : Yes
Status : up-to-date
Source package : s3fs-1.89-33.2.src
Summary : FUSE file system backed by Amazon S3 bucket
Description :
FUSE-based file system backed by Amazon S3. Mount a bucket as a local
file system read/write. Store files/folders natively and transparently
Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse)
Repository : Filesystem tools and FUSE-related packages (SLE_15_SP2)
Name : fuse
Version : 2.9.9-99.7
Arch : x86_64
Vendor : obs://build.opensuse.org/filesystems
Support Level : unknown
Installed Size : 330.3 KiB
Installed : Yes (automatically)
Status : up-to-date
Source package : fuse-2.9.9-99.7.src
Summary : Reference implementation of the "Filesystem in Userspace"
Description :
FUSE (Filesystem in Userspace) is an interface by the Linux kernel
for userspace programs to export a filesystem to the kernel.
Kernel information (uname -r)
5.3.18-24.49-default
GNU/Linux Distribution, if applicable (cat /etc/os-release)
NAME="SLES"
VERSION="15-SP2"
VERSION_ID="15.2"
PRETTY_NAME="SUSE Linux Enterprise Server 15 SP2"
ID="sles"
ID_LIKE="suse"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sles:15:sp2"
/etc/fstab entry, if applicable
nimblshareds3:/U/ /U/ fuse.s3fs _netdev,allow_other,rw 0 0
nimblshareds3:/S/ /S/ fuse.s3fs _netdev,allow_other,rw 0 0
nimblshareds3:/V/ /V/ fuse.s3fs _netdev,allow_other,rw 0 0
s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)
albolctx1:~ # s3fs nimblshareds3:/S/ /S/ -o iam_role=auto -o dbglevel=info -f -o curldbg
2021-05-11T18:25:32.768Z [CRT] s3fs_logger.cpp:LowSetLogLevel(219): change debug level from [CRT] to [INF]
2021-05-11T18:25:32.768Z [INF] s3fs.cpp:set_mountpoint_attribute(4020): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40777)
2021-05-11T18:25:32.769Z [INF] curl.cpp:InitMimeType(436): Loaded mime information from /etc/mime.types
2021-05-11T18:25:32.769Z [INF] fdcache_stat.cpp:CheckCacheFileStatTopDir(79): The path to cache top dir is empty, thus not need to check permission.
2021-05-11T18:25:32.769Z [INF] s3fs.cpp:s3fs_init(3331): init v1.89(commit:unknown) with OpenSSL
2021-05-11T18:25:32.769Z [INF] curl.cpp:LoadIAMRoleFromMetaData(2881): Get IAM Role name
2021-05-11T18:25:32.769Z [CURL DBG] * Uses proxy env variable no_proxy == 'localhost,127.0.0.1,monoclesys.xtm,monoclesys.corp,169.254.169.254,seamless.systems,solman.systems'
2021-05-11T18:25:32.769Z [CURL DBG] * Trying 169.254.169.254:80...
2021-05-11T18:25:32.769Z [CURL DBG] * TCP_NODELAY set
2021-05-11T18:25:32.770Z [CURL DBG] * Connected to 169.254.169.254 (169.254.169.254) port 80 (#0)
2021-05-11T18:25:32.770Z [CURL DBG] > GET /latest/meta-data/iam/security-credentials/ HTTP/1.1
2021-05-11T18:25:32.770Z [CURL DBG] > Host: 169.254.169.254
2021-05-11T18:25:32.770Z [CURL DBG] > User-Agent: s3fs/1.89 (commit hash unknown; OpenSSL)
2021-05-11T18:25:32.770Z [CURL DBG] > Accept: /
2021-05-11T18:25:32.770Z [CURL DBG] >
2021-05-11T18:25:32.770Z [CURL DBG] * Mark bundle as not supporting multiuse
2021-05-11T18:25:32.770Z [CURL DBG] < HTTP/1.1 200 OK
2021-05-11T18:25:32.770Z [CURL DBG] < Content-Type: text/plain
2021-05-11T18:25:32.770Z [CURL DBG] < Accept-Ranges: none
2021-05-11T18:25:32.770Z [CURL DBG] < Last-Modified: Tue, 11 May 2021 18:01:09 GMT
2021-05-11T18:25:32.770Z [CURL DBG] < Content-Length: 16
2021-05-11T18:25:32.770Z [CURL DBG] < Date: Tue, 11 May 2021 18:25:32 GMT
2021-05-11T18:25:32.770Z [CURL DBG] < Server: EC2ws
2021-05-11T18:25:32.770Z [CURL DBG] < Connection: close
2021-05-11T18:25:32.770Z [CURL DBG] <
2021-05-11T18:25:32.770Z [CURL DBG] * Closing connection 0
2021-05-11T18:25:32.770Z [INF] curl.cpp:RequestPerform(2287): HTTP response code 200
2021-05-11T18:25:32.770Z [INF] curl.cpp:SetIAMRoleFromMetaData(1773): IAM role name response = "s3fsmountingrole"
2021-05-11T18:25:32.770Z [INF] s3fs.cpp:s3fs_init(3348): loaded IAM role name = s3fsmountingrole
2021-05-11T18:25:32.770Z [INF] curl_handlerpool.cpp:ReturnHandler(110): Pool full: destroy the oldest handler
2021-05-11T18:25:32.770Z [INF] s3fs.cpp:s3fs_check_service(3447): check services.
2021-05-11T18:25:32.770Z [INF] curl.cpp:CheckIAMCredentialUpdate(1741): IAM Access Token refreshing...
2021-05-11T18:25:32.770Z [INF] curl.cpp:GetIAMCredentials(2784): [IAM role=s3fsmountingrole]
2021-05-11T18:25:32.770Z [CURL DBG] * Uses proxy env variable no_proxy == 'localhost,127.0.0.1,monoclesys.xtm,monoclesys.corp,169.254.169.254,seamless.systems,solman.systems'
2021-05-11T18:25:32.770Z [CURL DBG] * Hostname 169.254.169.254 was found in DNS cache
2021-05-11T18:25:32.770Z [CURL DBG] * Trying 169.254.169.254:80...
2021-05-11T18:25:32.770Z [CURL DBG] * TCP_NODELAY set
2021-05-11T18:25:32.770Z [CURL DBG] * Connected to 169.254.169.254 (169.254.169.254) port 80 (#1)
2021-05-11T18:25:32.770Z [CURL DBG] > PUT /latest/api/token HTTP/1.1
2021-05-11T18:25:32.770Z [CURL DBG] > Host: 169.254.169.254
2021-05-11T18:25:32.770Z [CURL DBG] > User-Agent: s3fs/1.89 (commit hash unknown; OpenSSL)
2021-05-11T18:25:32.770Z [CURL DBG] > Accept: /
2021-05-11T18:25:32.770Z [CURL DBG] > Transfer-Encoding: chunked
2021-05-11T18:25:32.770Z [CURL DBG] > X-aws-ec2-metadata-token-ttl-seconds: 21600
2021-05-11T18:25:32.770Z [CURL DBG] > Expect: 100-continue
2021-05-11T18:25:32.770Z [CURL DBG] >
2021-05-11T18:25:32.770Z [CURL DBG] * Mark bundle as not supporting multiuse
2021-05-11T18:25:32.770Z [CURL DBG] < HTTP/1.1 100 Continue
@wjes commented on GitHub (May 12, 2021):
I just had the very same error but it turned out it was my bucker's policy that lacked the
s3:ListBucketaction@martymcfly8891 commented on GitHub (May 12, 2021):
Thanks for the reply! i was thinking that as well so to rule out the roles and polices i added this policy to the role and still have the same issue.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"s3:GetLifecycleConfiguration",
"s3:GetBucketTagging",
"s3:GetInventoryConfiguration",
"s3:PutAnalyticsConfiguration",
"s3:PutAccelerateConfiguration",
"s3:ListBucketVersions",
"s3:GetBucketLogging",
"s3:CreateBucket",
"s3:ListBucket",
"s3:GetAccelerateConfiguration",
"s3:GetBucketPolicy",
"s3:PutEncryptionConfiguration",
"s3:GetEncryptionConfiguration",
"s3:GetBucketObjectLockConfiguration",
"s3:DeleteBucketWebsite",
"s3:GetIntelligentTieringConfiguration",
"s3:GetBucketRequestPayment",
"s3:DeleteBucketOwnershipControls",
"s3:PutLifecycleConfiguration",
"s3:GetMetricsConfiguration",
"s3:GetBucketOwnershipControls",
"s3:DeleteBucket",
"s3:PutBucketVersioning",
"s3:GetBucketPublicAccessBlock",
"s3:GetBucketPolicyStatus",
"s3:ListBucketMultipartUploads",
"s3:PutIntelligentTieringConfiguration",
"s3:GetBucketWebsite",
"s3:PutMetricsConfiguration",
"s3:PutBucketOwnershipControls",
"s3:PutReplicationConfiguration",
"s3:GetBucketVersioning",
"s3:PutBucketCORS",
"s3:GetBucketAcl",
"s3:GetBucketNotification",
"s3:GetReplicationConfiguration",
"s3:PutInventoryConfiguration",
"s3:PutBucketNotification",
"s3:PutBucketWebsite",
"s3:PutBucketRequestPayment",
"s3:PutBucketLogging",
"s3:GetBucketCORS",
"s3:GetAnalyticsConfiguration",
"s3:PutBucketObjectLockConfiguration",
"s3:GetBucketLocation"
],
"Resource": "arn:aws:s3:::mybucket/"
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"s3:GetObjectRetention",
"s3:DeleteObjectVersion",
"s3:GetObjectVersionTagging",
"s3:RestoreObject",
"s3:PutObjectLegalHold",
"s3:GetObjectLegalHold",
"s3:ListMultipartUploadParts",
"s3:ReplicateObject",
"s3:GetObjectVersionTorrent",
"s3:PutObject",
"s3:GetObjectAcl",
"s3:GetObject",
"s3:GetObjectTorrent",
"s3:AbortMultipartUpload",
"s3:PutObjectRetention",
"s3:GetObjectVersionAcl",
"s3:GetObjectTagging",
"s3:GetObjectVersionForReplication",
"s3:DeleteObject",
"s3:ReplicateDelete",
"s3:GetObjectVersion"
],
"Resource": "arn:aws:s3:::mybucket/"
},
{
"Sid": "VisualEditor2",
"Effect": "Allow",
"Action": [
"s3:ListStorageLensConfigurations",
"s3:ListAccessPointsForObjectLambda",
"s3:GetAccountPublicAccessBlock",
"s3:ListAllMyBuckets",
"s3:ListAccessPoints"
],
"Resource": "*"
}
]
}
@wjes commented on GitHub (May 12, 2021):
I'm not totally sure but just with this simple policy you should be able to list and get all the objects mounted with
s3fsMind the
Resourcesyntax, particularly forVisualEditor0(without the last slash/)@martymcfly8891 commented on GitHub (May 13, 2021):
Thank you I tried a simple policy as you have listed prior to using this current policy I have listed and had the same results hence testing the s3 admin one I listed above.
@somebodysomebodynov commented on GitHub (Mar 7, 2024):
Set up for EC2 instance (c6g.4xlarge) on
ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-arm64-server-20240207.1. Give it an IAM role with S3FullAccess.Mounted S3 bucket at EC2 instance using command
s3fs -o allow_other -o iam_role=<s3-full-access-iam-role> -o endpoint=eu-west-1 -o url="http://s3.amazonaws.com" <bucket-name> /home/ubuntu/s3disk -f -dBut there is same error:
@ggtakec commented on GitHub (Mar 10, 2024):
Try starting s3fs with
dbglevel=info(ordbg) orcurldbgand check its detailed operation log.The log may contain information that may give you a hint as to why the error occurred.
Thanks in advance for your assistance.