starred/s3fs-fuse

Fork 0

mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2026-04-25 13:26:00 +03:00

[GH-ISSUE #2190] Directories not shown with v1.92 using IAM Role #1114

New issue

Open

opened 2026-03-04 01:51:31 +03:00 by kerem · 6 comments

kerem commented

2026-03-04 01:51:31 +03:00

Owner

Originally created by @heironeous on GitHub (Jun 20, 2023).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2190

Additional Information

Version of s3fs being used (`s3fs --version`)

V1.92 (commit: 161be3f )

Version of fuse being used (`pkg-config --modversion fuse`, `rpm -qi fuse` or `dpkg -s fuse`)

2.9.9

Kernel information (`uname -r`)

6.1.29-50.88.amzn2023.x86_64

GNU/Linux Distribution, if applicable (`cat /etc/os-release`)

NAME="Amazon Linux"
VERSION="2023"
ID="amzn"
ID_LIKE="fedora"
VERSION_ID="2023"
PLATFORM_ID="platform:al2023"
PRETTY_NAME="Amazon Linux 2023"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2023"
HOME_URL="https://aws.amazon.com/linux/"
BUG_REPORT_URL="https://github.com/amazonlinux/amazon-linux-2023"
SUPPORT_END="2028-03-01"

How to run s3fs, if applicable

[x] command line
[] /etc/fstab

s3fs "<BUCKET_NAME_HERE>" "/mnt/backups" -o iam_role -o logfile=/tmp/s3fs.log

s3fs syslog messages (`grep s3fs /var/log/syslog`, `journalctl | grep s3fs`, or `s3fs outputs`)

2023-06-20T11:15:15.888Z [CRT] s3fs_cred.cpp:VersionS3fsCredential(60): Check why built-in function was called, the external credential library must have VersionS3fsCredential function.
2023-06-20T11:15:15.888Z [INF] s3fs.cpp:s3fs_init(4210): init v1.92(commit:unknown) with OpenSSL, credential-library(built-in)

I'll attach the whole debug logs after a round of sanitization

Details about issue

This issue happens with v1.92, but v1.91 is still working.

Here's how I can reproduce

Create new EC2 instance
Attach an IAM role that has S3 Admin
Confirm with awscli that it can access S3
Install S3FS
Mount an S3 bucket
ls the contents
Directories are NOT visible
Files that are in the root level of S3 bucket ARE visible

Since v1.91 is no longer available in many repos as RPM, I have compiled it from source.

No issues with compiling or installation, also the above process works correctly.

I am seeing some '400's in the logs, and I will be posting the whole log after I sanitize it.

Originally created by @heironeous on GitHub (Jun 20, 2023). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2190  ### Additional Information #### Version of s3fs being used (`s3fs --version`) `V1.92 (commit: 161be3f )` #### Version of fuse being used (`pkg-config --modversion fuse`, `rpm -qi fuse` or `dpkg -s fuse`) `2.9.9` #### Kernel information (`uname -r`) `6.1.29-50.88.amzn2023.x86_64` #### GNU/Linux Distribution, if applicable (`cat /etc/os-release`) ``` NAME="Amazon Linux" VERSION="2023" ID="amzn" ID_LIKE="fedora" VERSION_ID="2023" PLATFORM_ID="platform:al2023" PRETTY_NAME="Amazon Linux 2023" ANSI_COLOR="0;33" CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2023" HOME_URL="https://aws.amazon.com/linux/" BUG_REPORT_URL="https://github.com/amazonlinux/amazon-linux-2023" SUPPORT_END="2028-03-01" ``` #### How to run s3fs, if applicable  [x] command line [] /etc/fstab  ``` s3fs "<BUCKET_NAME_HERE>" "/mnt/backups" -o iam_role -o logfile=/tmp/s3fs.log ``` #### s3fs syslog messages (`grep s3fs /var/log/syslog`, `journalctl | grep s3fs`, or `s3fs outputs`)  ``` 2023-06-20T11:15:15.888Z [CRT] s3fs_cred.cpp:VersionS3fsCredential(60): Check why built-in function was called, the external credential library must have VersionS3fsCredential function. 2023-06-20T11:15:15.888Z [INF] s3fs.cpp:s3fs_init(4210): init v1.92(commit:unknown) with OpenSSL, credential-library(built-in) ``` I'll attach the whole debug logs after a round of sanitization ### Details about issue  This issue happens with v1.92, but v1.91 is still working. Here's how I can reproduce - Create new EC2 instance - Attach an IAM role that has S3 Admin - Confirm with awscli that it can access S3 - Install S3FS - Mount an S3 bucket - `ls` the contents - Directories are NOT visible - Files that are in the root level of S3 bucket ARE visible Since v1.91 is no longer available in many repos as RPM, I have compiled it from source. No issues with compiling or installation, also the above process works correctly. I am seeing some '400's in the logs, and I will be posting the whole log after I sanitize it.

kerem added the

need info

label

2026-03-04 01:51:31 +03:00

kerem commented

2026-03-04 01:51:32 +03:00

Author

Owner

@heironeous commented on GitHub (Jun 20, 2023):

Here's the sanitized log file

s3fs.log

@heironeous commented on GitHub (Jun 20, 2023): Here's the sanitized log file [s3fs.log](https://github.com/s3fs-fuse/s3fs-fuse/files/11800108/s3fs.log)

kerem commented

2026-03-04 01:51:32 +03:00

Author

Owner

@heironeous commented on GitHub (Jun 20, 2023):

Here's another run on a different machine (Amazon Linux 2 instead of Amazon Linux 2023), which also includes the 404 for the folder s3fs-with-folder.log

@heironeous commented on GitHub (Jun 20, 2023): Here's another run on a different machine (Amazon Linux 2 instead of Amazon Linux 2023), which also includes the 404 for the folder [s3fs-with-folder.log](https://github.com/s3fs-fuse/s3fs-fuse/files/11801791/s3fs-with-folder.log)

kerem commented

2026-03-04 01:51:32 +03:00

Author

Owner

@ggtakec commented on GitHub (Jun 26, 2023):

@heironeous
I checked the log you provided.
I haven't been able to reproduce this issue yet, so I have a few questions:

Please tell me what to do after starting s3fs.
Did the s3fs process continue to run after this operation? (or has it been forcibly terminated?)
In your log, I am concerned because there was a message about s3fs destroying at the end.
If it's crashing, it may have something to do with #2188.

Thanks in advance for your assistance.

@ggtakec commented on GitHub (Jun 26, 2023): @heironeous I checked the log you provided. I haven't been able to reproduce this issue yet, so I have a few questions: - Please tell me what to do after starting s3fs. - Did the s3fs process continue to run after this operation? (or has it been forcibly terminated?) _In your log, I am concerned because there was a message about s3fs destroying at the end._ _If it's crashing, it may have something to do with #2188._ Thanks in advance for your assistance.

kerem commented

2026-03-04 01:51:32 +03:00

Author

Owner

@ggtakec commented on GitHub (Jul 2, 2023):

I merged PR #2204 from @VVoidV.
The cause of this is not yet clear, but it is possible that the bug found in #2204 has an effect.
If you can, could you try the latest code(current master branch)?

@ggtakec commented on GitHub (Jul 2, 2023): I merged PR #2204 from @VVoidV. The cause of this is not yet clear, but it is possible that the bug found in #2204 has an effect. If you can, could you try the latest code(current master branch)?

kerem commented

2026-03-04 01:51:32 +03:00

Author

Owner

@heironeous commented on GitHub (Jul 30, 2023):

I apologize for the radio silence. For some reason, I'm not getting any e-mails from Github for notifications, I'll see about and fix that.

Please tell me what to do after starting s3fs.

I provided 'how to reproduce', a simple ls showed that root-level objects were visible, but folders or items within folders were not visible within the ls output. I'm aware that 'folders' do not exist on the S3 level, but they are visualized as folders within an S3FS mount.

Did the s3fs process continue to run after this operation? (or has it been forcibly terminated?)

The logs contain 'destroy' becuase I ran a umount at the end, to show how it behaves at while doing a umount. I did not document that initial ticket, so you're right to think that #2204 might be the culprit, but I don't think it would be in this case.

S3FS continued to work, but I couldn't see 'folders' as stated.

If you can, could you try the latest code(current master branch)?

Will do, and I'll return with the results. Thanks.

@heironeous commented on GitHub (Jul 30, 2023): I apologize for the radio silence. For some reason, I'm not getting any e-mails from Github for notifications, I'll see about and fix that. > Please tell me what to do after starting s3fs. I provided 'how to reproduce', a simple `ls` showed that root-level objects were visible, but folders or items within folders were not visible within the `ls` output. I'm aware that 'folders' do not exist on the S3 level, but they are visualized as folders within an S3FS mount. > Did the s3fs process continue to run after this operation? (or has it been forcibly terminated?) The logs contain 'destroy' becuase I ran a `umount` at the end, to show how it behaves at while doing a `umount`. I did not document that initial ticket, so you're right to think that #2204 might be the culprit, but I don't think it would be in this case. S3FS continued to work, but I couldn't see 'folders' as stated. > If you can, could you try the latest code(current master branch)? Will do, and I'll return with the results. Thanks.

kerem commented

2026-03-04 01:51:32 +03:00

Author

Owner

@gaul commented on GitHub (Sep 8, 2023):

Could you test with 1.93 which fixes the directory regression?

@gaul commented on GitHub (Sep 8, 2023): Could you test with 1.93 which fixes the directory regression?

kerem referenced this issue

2026-03-04 02:02:21 +03:00

[PR #1114] [MERGED] Consume return value from FdEntity::Flush #1826