[GH-ISSUE #2033] Using S3FS with OpenSSH SFTP Chroot causes EACCES despite correct permissions

kerem commented

2026-03-04 01:50:45 +03:00

Owner

Originally created by @michaelsmoody on GitHub (Sep 16, 2022).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2033

Additional Information

The following information is very important in order to help us to help you. Omission of the following details may delay your support request or receive no attention at all.
Keep in mind that the commands we provide to retrieve information are oriented to GNU/Linux Distributions, so you could need to use others if you use s3fs on macOS or BSD

Version of s3fs being used (s3fs --version)

s3fs 1.91

Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse)

FUSE 2.9.2

Kernel information (uname -r)

5.10.135-122.509.amzn2.x86_64

GNU/Linux Distribution, if applicable (cat /etc/os-release)

Amazon Linux 2

s3fs command line used, if applicable

s3fs -o url=https://s3-us-gov-west-1.amazonaws.com -o endpoint=us-gov-west-1 -o allow_other -o mp_umask=022 -o dev -o suid -o use_xattr cust-s3-transfer-storage:/FOLDER /mnt/folder

/etc/fstab entry, if applicable

s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)

if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages

Details about issue

With an identical working setup on EFS (works perfectly with the following) in /etc/ssh/sshd_config:

Subsystem sftp internal-sftp -f LOCAL7 -l INFO
ForceCommand internal-sftp -d /mnt/folder -u 0007 -f LOCAL7 -l INFO
# Chroot the connection into the specified directory
ChrootDirectory /mnt/folder

And with relevant groups setup on the remote filesystem (EFS/NFS), S3FS fails to work the same way. We were investigating this as a possible "swap" for AWS Transfer Family (which can also use an S3 storage backend, but maintaining POSIX).

Logs in the chroot SFTP show:

error: chdir to /mnt/folder failed: No such file or directory

Additionally, and much more importantly!

We have many folders on the filesystem that belong to remove groups with 770 permission. This leverages remote groups (however, local groups were tested, and the problem remains, ensuring that the problem is S3FS). Essentially, if the directory is root:namedgroupname, it works properly on EFS (chroot), and users can properly traverse if they belong to that group, but on S3FS, if we enable SFTP with that S3FS as the backing store, they CANNOT traverse. Same groups, same files, same permissions (we actually copied the same files and folders using a cp -a as a test bed!), and we are unable to traverse into these directories if we belong to those groups via SFTP (permission denied). All things being equal, the only change was S3FS vs EFS as the backing store!

We've attempted to even set the groups again on the same directories manually, to no avail. We've leveraged strace and auditctl to investigate at a low level, and in all cases, the EACCES continues.

We are of course open to any and all suggestions, as this is a one-to-one swap, with the only change simply being the backing store, and no other settings changes. OpenSSH was unable to SEE files in the S3FS when moving to sftp-server rather than internal-sftp (the alternative implementation). And the chroot prevented quite a few things that should normally function as expected. SELinux was even disabled as a "just in case" also to no avail. Please assist if possible.

Originally created by @michaelsmoody on GitHub (Sep 16, 2022). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2033 ### Additional Information _The following information is very important in order to help us to help you. Omission of the following details may delay your support request or receive no attention at all._ _Keep in mind that the commands we provide to retrieve information are oriented to GNU/Linux Distributions, so you could need to use others if you use s3fs on macOS or BSD_ #### Version of s3fs being used (s3fs --version) s3fs 1.91 #### Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse) FUSE 2.9.2 #### Kernel information (uname -r) 5.10.135-122.509.amzn2.x86_64 #### GNU/Linux Distribution, if applicable (cat /etc/os-release) Amazon Linux 2 #### s3fs command line used, if applicable ``` s3fs -o url=https://s3-us-gov-west-1.amazonaws.com -o endpoint=us-gov-west-1 -o allow_other -o mp_umask=022 -o dev -o suid -o use_xattr cust-s3-transfer-storage:/FOLDER /mnt/folder ``` #### /etc/fstab entry, if applicable ``` ``` #### s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs) _if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages_ ``` ``` ### Details about issue With an identical working setup on EFS (works perfectly with the following) in `/etc/ssh/sshd_config`: ``` Subsystem sftp internal-sftp -f LOCAL7 -l INFO ForceCommand internal-sftp -d /mnt/folder -u 0007 -f LOCAL7 -l INFO # Chroot the connection into the specified directory ChrootDirectory /mnt/folder ``` And with relevant groups setup on the remote filesystem (EFS/NFS), S3FS fails to work the same way. We were investigating this as a possible "swap" for AWS Transfer Family (which can also use an S3 storage backend, but maintaining POSIX). Logs in the chroot SFTP show: `error: chdir to /mnt/folder failed: No such file or directory` Additionally, and much more importantly! We have many folders on the filesystem that belong to remove groups with 770 permission. This leverages remote groups (however, local groups were tested, and the problem remains, ensuring that the problem is S3FS). Essentially, if the directory is root:namedgroupname, it works properly on EFS (chroot), and users can properly traverse if they belong to that group, but on S3FS, if we enable SFTP with that S3FS as the backing store, they CANNOT traverse. Same groups, same files, same permissions (we actually copied the same files and folders using a `cp -a` as a test bed!), and we are unable to traverse into these directories if we belong to those groups via SFTP (permission denied). All things being equal, the only change was S3FS vs EFS as the backing store! We've attempted to even set the groups again on the same directories manually, to no avail. We've leveraged strace and auditctl to investigate at a low level, and in all cases, the EACCES continues. We are of course open to any and all suggestions, as this is a one-to-one swap, with the only change simply being the backing store, and no other settings changes. OpenSSH was unable to _**SEE**_ files in the S3FS when moving to sftp-server rather than internal-sftp (the alternative implementation). And the chroot prevented quite a few things that should normally function as expected. SELinux was even disabled as a "just in case" also to no avail. Please assist if possible.

kerem closed this issue

2026-03-04 01:50:46 +03:00

kerem commented

2026-03-04 01:50:46 +03:00

Author

Owner

@ggtakec commented on GitHub (Sep 24, 2022):

This issue appears to be the same as #775.
The cause of this is now known and will be fixed.
I'm planning to start fixing it, and if it's the same thing, can you confirm with that PR code (I'll upload it soon)?

@ggtakec commented on GitHub (Sep 24, 2022): This issue appears to be the same as #775. The cause of this is now known and will be fixed. I'm planning to start fixing it, and if it's the same thing, can you confirm with that PR code (I'll upload it soon)?

kerem commented

2026-03-04 01:50:46 +03:00

Author

Owner

@ggtakec commented on GitHub (Sep 25, 2022):

@michaelsmoody I also posted to #775, I implemented POSIX ACL compliant behavior in PR #2039.

@ggtakec commented on GitHub (Sep 25, 2022): @michaelsmoody I also posted to #775, I implemented POSIX ACL compliant behavior in PR #2039.

kerem commented

2026-03-04 01:50:46 +03:00

Author

Owner

@ggtakec commented on GitHub (Oct 22, 2022):

Sorry for the delay in replying.
Merge #2039.
I believe this merge has resolved the issue.
If the problem still persists, please repoen or post a new issue.

@ggtakec commented on GitHub (Oct 22, 2022): _Sorry for the delay in replying._ Merge #2039. I believe this merge has resolved the issue. If the problem still persists, please repoen or post a new issue.

kerem commented

2026-03-04 01:50:47 +03:00

Author

Owner

@michaelsmoody commented on GitHub (Nov 1, 2022):

@ggtakec Similar to #2032, I see the merge for #2039, and I'm curious what the release cycle looks like for this to become a packaged release?

While we can build it, I'm not sure if that's preferable? Please let us know.

@michaelsmoody commented on GitHub (Nov 1, 2022): @ggtakec Similar to #2032, I see the merge for #2039, and I'm curious what the release cycle looks like for this to become a packaged release? While we can build it, I'm not sure if that's preferable? Please let us know.

kerem commented

2026-03-04 01:50:47 +03:00

Author

Owner

@michaelsmoody commented on GitHub (Nov 10, 2022):

@gaul Forgive my pulling you in, as I see you're the one who creates release tags. What defines the release cycle? As I noted above ☝🏻 these relate to POSIX, mounting permissions, etc, and while we would like to use it, we want to make sure that we do so in a "stable" state, as preferred by the team here. While we could absolutely perform a git clone, we also prefer to use officially tagged releases for compliance reasons, wherever possible, but I'd like to be most clear that in no way that this should be take as "please rush this". Rather, "thank you", and if the answer is "we release when we release", that's of course entirely acceptable too!

@michaelsmoody commented on GitHub (Nov 10, 2022): @gaul Forgive my pulling you in, as I see you're the one who creates release tags. What defines the release cycle? As I noted above ☝🏻 these relate to POSIX, mounting permissions, etc, and while we would _like_ to use it, we want to make sure that we do so in a "stable" state, as preferred by the team here. While we could absolutely perform a `git clone`, we also prefer to use officially tagged releases for compliance reasons, wherever possible, but I'd like to be most clear that in no way that this should be take as "please rush this". Rather, "thank you", and if the answer is "we release when we release", that's of course entirely acceptable too!

kerem commented

2026-03-04 01:50:47 +03:00

Author

Owner

@ggtakec commented on GitHub (Nov 10, 2022):

@michaelsmoody
@gaul and I would like to do quarterly releases if possible, or semi-annual releases at the latest.
But this is not documented as a product.
This time around, we're making some big fixes and some bugs haven't been fixed yet, which slows down the release cycle.
We would like to make a new release soon, but I'm sorry, but I don't think it will be soon(but we would like to do it as soon as possible).

@ggtakec commented on GitHub (Nov 10, 2022): @michaelsmoody @gaul and I would like to do quarterly releases if possible, or semi-annual releases at the latest. _But this is not documented as a product._ This time around, we're making some big fixes and some bugs haven't been fixed yet, which slows down the release cycle. We would like to make a new release soon, but I'm sorry, but I don't think it will be soon(but we would like to do it as soon as possible).

kerem referenced this issue

2026-03-04 02:02:04 +03:00

[PR #1025] [MERGED] Update ctime during nocopyapi operations #1766

Rows
Columns

[GH-ISSUE #2033] Using S3FS with OpenSSH SFTP Chroot causes EACCES despite correct permissions #1025

Additional Information

Version of s3fs being used (s3fs --version)

Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse)

Kernel information (uname -r)

GNU/Linux Distribution, if applicable (cat /etc/os-release)

s3fs command line used, if applicable

/etc/fstab entry, if applicable

s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)

Details about issue