starred/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2026-04-25 05:16:00 +03:00
Code Issues 305 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #537] Issue with access Policy (or maybe my config error?) #304

New issue
Closed
opened 2026-03-04 01:44:13 +03:00 by kerem · 7 comments
kerem commented 2026-03-04 01:44:13 +03:00
Owner
Copy link

Originally created by @lyquix-owner on GitHub (Feb 23, 2017).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/537

Additional Information

The following information is very important in order to help us to help you. Omission of the following details may delay your support request or receive no attention at all.

  • Version of s3fs being used (s3fs --version)
    V1.80(commit:d40da2c)

  • Version of fuse being used (pkg-config --modversion fuse)
    2.9.2

  • System information (uname -a)
    Linux lyquix 4.8.3-x86_64-linode76 #1 SMP Thu Oct 20 19:05:39 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux

  • Distro (cat /etc/issue)
    Ubuntu 14.04.5 LTS \n \l

  • s3fs command line used (if applicable)
    s3fs lyquix-s3backup /s3backup -o passwd_file=/etc/passwd-s3fs

  • /etc/fstab entry (if applicable):
    lyquix-s3backup /s3backup fuse.s3fs _netdev,allow_other 0 0

  • s3fs syslog messages (grep s3fs /var/log/syslog, or s3fs outputs)
    if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages
    No error shown when mounting
    No errors shown in /var/log/syslog

Details about issue

I have one AWS account where I have multiple S3 buckets. I am trying to setup separate users that only have access to one bucket. I have created separate Policies for each user. When using the default "AmazonS3FullAccess" policy s3fs works perfectly:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": "*"
        }
    ]
}

However, when I specify the bucket, as follows:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::lyquix-s3backup"
        }
    ]
}

I get the error "Operation not permitted" when I try any read/write/delete file operation:

root@lyquix:~# echo "Hello world!" > /s3backup/test.txt
-bash: /s3backup/test.txt: Operation not permitted

and when I try a list I get "No such file or directory" errors:

root@lyquix:~# ls -la /s3backup/
ls: cannot access /s3backup/daily.sql.tar.gz: No such file or directory
ls: cannot access /s3backup/daily.tar.gz: No such file or directory
total 5
drwx------  1 root root    0 Dec 31  1969 .
drwxr-xr-x 23 root root 4096 Feb 22 17:41 ..
??????????  ? ?    ?       ?            ? daily.sql.tar.gz
??????????  ? ?    ?       ?            ? daily.tar.gz

When I test this policy in the Amazon Policy Simulator I get no errors and all operations are supposed to be allowed.

I am new to s3fs so maybe this is a dumb configuration error on my part.
Thank you for your help.

Originally created by @lyquix-owner on GitHub (Feb 23, 2017). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/537 #### Additional Information _The following information is very important in order to help us to help you. Omission of the following details may delay your support request or receive no attention at all._ - Version of s3fs being used (s3fs --version) V1.80(commit:d40da2c) - Version of fuse being used (pkg-config --modversion fuse) 2.9.2 - System information (uname -a) Linux lyquix 4.8.3-x86_64-linode76 #1 SMP Thu Oct 20 19:05:39 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux - Distro (cat /etc/issue) Ubuntu 14.04.5 LTS \n \l - s3fs command line used (if applicable) `s3fs lyquix-s3backup /s3backup -o passwd_file=/etc/passwd-s3fs` - /etc/fstab entry (if applicable): `lyquix-s3backup /s3backup fuse.s3fs _netdev,allow_other 0 0` - s3fs syslog messages (grep s3fs /var/log/syslog, or s3fs outputs) _if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages_ No error shown when mounting No errors shown in /var/log/syslog #### Details about issue I have one AWS account where I have multiple S3 buckets. I am trying to setup separate users that only have access to one bucket. I have created separate Policies for each user. When using the default "AmazonS3FullAccess" policy s3fs works perfectly: ``` { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "s3:*", "Resource": "*" } ] } ``` However, when I specify the bucket, as follows: ``` { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "s3:*", "Resource": "arn:aws:s3:::lyquix-s3backup" } ] } ``` I get the error "Operation not permitted" when I try any read/write/delete file operation: ``` root@lyquix:~# echo "Hello world!" > /s3backup/test.txt -bash: /s3backup/test.txt: Operation not permitted ``` and when I try a list I get "No such file or directory" errors: ``` root@lyquix:~# ls -la /s3backup/ ls: cannot access /s3backup/daily.sql.tar.gz: No such file or directory ls: cannot access /s3backup/daily.tar.gz: No such file or directory total 5 drwx------ 1 root root 0 Dec 31 1969 . drwxr-xr-x 23 root root 4096 Feb 22 17:41 .. ?????????? ? ? ? ? ? daily.sql.tar.gz ?????????? ? ? ? ? ? daily.tar.gz ``` When I test this policy in the Amazon Policy Simulator I get no errors and all operations are supposed to be allowed. I am new to s3fs so maybe this is a dumb configuration error on my part. Thank you for your help.
kerem closed this issue 2026-03-04 01:44:13 +03:00
kerem commented 2026-03-04 01:44:14 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Apr 2, 2017):

@lyquix-owner
It seems that your access policy does not have any problem.

First of all, we should know if this problem is a problem caused by s3fs or a policy setting etc.
For that reason, let s3fs do debug output(you can use dbglevel and curldbg option, please see man page) and do simple operation.
That will cause s3fs to output a lot of logs.
I think these logs help us to give some hints to slove this problem.

Thanks in advance for your help.

<!-- gh-comment-id:290967062 --> @ggtakec commented on GitHub (Apr 2, 2017): @lyquix-owner It seems that your access policy does not have any problem. First of all, we should know if this problem is a problem caused by s3fs or a policy setting etc. For that reason, let s3fs do debug output(you can use dbglevel and curldbg option, please see man page) and do simple operation. That will cause s3fs to output a lot of logs. I think these logs help us to give some hints to slove this problem. Thanks in advance for your help.
kerem commented 2026-03-04 01:44:14 +03:00
Author
Owner
Copy link

@lyquix-owner commented on GitHub (Apr 3, 2017):

@ggtakec, thank you for your response.

I have mounted the s3fs drive using the debug options provided. Please note that the name of my bucket has changed since last time, it is now lyquix-s3backup-rothman, and the policy file is the following:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::lyquix-s3backup-rothman"
        }
    ]
}

Here is a log from the command line:

root@rothman:/# s3fs lyquix-s3backup-rothman /s3backup -d -o passwd_file=/etc/passwd-s3fs -o curldbg -o dbglevel=info
root@rothman:/# ls -la /s3backup/
ls: cannot access /s3backup/daily.sql.tar.gz: No such file or directory
ls: cannot access /s3backup/daily.tar.gz: No such file or directory
ls: cannot access /s3backup/monthly.sql.tar.gz: No such file or directory
ls: cannot access /s3backup/monthly.tar.gz: No such file or directory
ls: cannot access /s3backup/weekly.sql.tar.gz: No such file or directory
ls: cannot access /s3backup/weekly.tar.gz: No such file or directory
total 5
drwx------  1 root root    0 Dec 31  1969 .
drwxr-xr-x 23 root root 4096 Apr  3 12:21 ..
??????????  ? ?    ?       ?            ? daily.sql.tar.gz
??????????  ? ?    ?       ?            ? daily.tar.gz
??????????  ? ?    ?       ?            ? monthly.sql.tar.gz
??????????  ? ?    ?       ?            ? monthly.tar.gz
??????????  ? ?    ?       ?            ? weekly.sql.tar.gz
??????????  ? ?    ?       ?            ? weekly.tar.gz
root@rothman:/# cp /var/lib/mysql/daily.sql.tar.gz /s3backup/daily.sql.tar.gz
cp: cannot create regular file ‘/s3backup/daily.sql.tar.gz’: Operation not permitted
root@rothman:~# cp /srv/www/daily.tar.gz /s3backup/daily.tar.gz
cp: cannot create regular file ‘/s3backup/daily.tar.gz’: Operation not permitted
root@rothman:~# umount /s3backup

You can find the extract from syslog here: https://gist.github.com/lyquix-owner/deecec57cd784e9789035a1465994062

Please note that I have redacted all instances of

Authorization header (access credential and signature)
x-amz-content-sha256
x-amz-id-2

<!-- gh-comment-id:291204279 --> @lyquix-owner commented on GitHub (Apr 3, 2017): @ggtakec, thank you for your response. I have mounted the s3fs drive using the debug options provided. Please note that the name of my bucket has changed since last time, it is now `lyquix-s3backup-rothman`, and the policy file is the following: ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "s3:*", "Resource": "arn:aws:s3:::lyquix-s3backup-rothman" } ] } ``` Here is a log from the command line: ``` root@rothman:/# s3fs lyquix-s3backup-rothman /s3backup -d -o passwd_file=/etc/passwd-s3fs -o curldbg -o dbglevel=info root@rothman:/# ls -la /s3backup/ ls: cannot access /s3backup/daily.sql.tar.gz: No such file or directory ls: cannot access /s3backup/daily.tar.gz: No such file or directory ls: cannot access /s3backup/monthly.sql.tar.gz: No such file or directory ls: cannot access /s3backup/monthly.tar.gz: No such file or directory ls: cannot access /s3backup/weekly.sql.tar.gz: No such file or directory ls: cannot access /s3backup/weekly.tar.gz: No such file or directory total 5 drwx------ 1 root root 0 Dec 31 1969 . drwxr-xr-x 23 root root 4096 Apr 3 12:21 .. ?????????? ? ? ? ? ? daily.sql.tar.gz ?????????? ? ? ? ? ? daily.tar.gz ?????????? ? ? ? ? ? monthly.sql.tar.gz ?????????? ? ? ? ? ? monthly.tar.gz ?????????? ? ? ? ? ? weekly.sql.tar.gz ?????????? ? ? ? ? ? weekly.tar.gz root@rothman:/# cp /var/lib/mysql/daily.sql.tar.gz /s3backup/daily.sql.tar.gz cp: cannot create regular file ‘/s3backup/daily.sql.tar.gz’: Operation not permitted root@rothman:~# cp /srv/www/daily.tar.gz /s3backup/daily.tar.gz cp: cannot create regular file ‘/s3backup/daily.tar.gz’: Operation not permitted root@rothman:~# umount /s3backup ``` You can find the extract from syslog here: https://gist.github.com/lyquix-owner/deecec57cd784e9789035a1465994062 Please note that I have redacted all instances of Authorization header (access credential and signature) x-amz-content-sha256 x-amz-id-2
kerem commented 2026-03-04 01:44:14 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Apr 9, 2017):

@lyquix-owner
It seems that "*.tar.gz" files which are not uploaded by s3fs do not have any attributes(mode/uid/gid).
s3fs needs file permission mode owner group(x-amz-meta-* headers present).
I think that these files are uploaded by another s3 tools(ex. s3cmd).

So if you can, you can run s3fs with uid/gid/umask option, these options set uid/gid/mode for these files.
Thus you can do "cp" command.

Please try it.
Regards,

<!-- gh-comment-id:292767328 --> @ggtakec commented on GitHub (Apr 9, 2017): @lyquix-owner It seems that "\*.tar.gz" files which are not uploaded by s3fs do not have any attributes(mode/uid/gid). s3fs needs file permission mode owner group(x-amz-meta-* headers present). I think that these files are uploaded by another s3 tools(ex. s3cmd). So if you can, you can run s3fs with uid/gid/umask option, these options set uid/gid/mode for these files. Thus you can do "cp" command. Please try it. Regards,
kerem commented 2026-03-04 01:44:14 +03:00
Author
Owner
Copy link

@sqlbot commented on GitHub (Apr 9, 2017):

I believe the error actually is in the policy.

"Resource": "arn:aws:s3:::lyquix-s3backup-rothman"

That's only permission on the bucket, not the objects in the bucket. Listing objects should work, but fetching metadata would not, and that may be a more likely explanation for the strange directory listing, since my assumption from the messages above is that this bucket's contents were created by s3fs, so the metadata should be there.

I believe a correct policy resource would look more like this:

"Resource": [ "arn:aws:s3:::lyquix-s3backup-rothman", "arn:aws:s3:::lyquix-s3backup-rothman/*" ]

<!-- gh-comment-id:292770253 --> @sqlbot commented on GitHub (Apr 9, 2017): I believe the error actually _is_ in the policy. `"Resource": "arn:aws:s3:::lyquix-s3backup-rothman"` That's only permission on the bucket, not the objects in the bucket. Listing objects should work, but fetching metadata would not, and that may be a more likely explanation for the strange directory listing, since my assumption from the messages above is that this bucket's contents were created by s3fs, so the metadata should be there. I believe a correct policy resource would look more like this: `"Resource": [ "arn:aws:s3:::lyquix-s3backup-rothman", "arn:aws:s3:::lyquix-s3backup-rothman/*" ]`
kerem commented 2026-03-04 01:44:14 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Apr 9, 2017):

@sqlbot Thanks you for your kindness.

@lyquix-owner Please try to change your policy like sample.
Thanks in advance for your assistance.

<!-- gh-comment-id:292775606 --> @ggtakec commented on GitHub (Apr 9, 2017): @sqlbot Thanks you for your kindness. @lyquix-owner Please try to change your policy like sample. Thanks in advance for your assistance.
kerem commented 2026-03-04 01:44:14 +03:00
Author
Owner
Copy link

@lyquix-owner commented on GitHub (Apr 10, 2017):

Thanks for your help @sqlbot @ggtakec !

I am happy to report that changing the policy as recommended solved the problem.

"Resource": [
              "arn:aws:s3:::lyquix-s3backup-rothman", 
              "arn:aws:s3:::lyquix-s3backup-rothman/*"
]
<!-- gh-comment-id:293022627 --> @lyquix-owner commented on GitHub (Apr 10, 2017): Thanks for your help @sqlbot @ggtakec ! I am happy to report that changing the policy as recommended solved the problem. ``` "Resource": [ "arn:aws:s3:::lyquix-s3backup-rothman", "arn:aws:s3:::lyquix-s3backup-rothman/*" ] ```
kerem commented 2026-03-04 01:44:14 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Apr 23, 2017):

@lyquix-owner I'm closing this issue, and tahnks for your report.
Regards,

<!-- gh-comment-id:296413904 --> @ggtakec commented on GitHub (Apr 23, 2017): @lyquix-owner I'm closing this issue, and tahnks for your report. Regards,
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
v1.97
v1.96
v1.95
v1.94
v1.93
v1.92
v1.91
v1.90
v1.89
v1.88
v1.87
v1.86
v1.85
v1.84
v1.83
v1.82
v1.81
v1.80
v1.79
v1.78
Pre-v1.78
v1.77
v1.76
v1.75
v1.74
Labels
Clear labels   
bug

   
bug

   
dataloss

   
duplicate

   
enhancement

   
feature request

   
help wanted

   
invalid

   
need info

   
performance

   
pull-request

Mirrored from GitHub Pull Request

  
question

   
question

   
testing
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#304
No description provided.