[GH-ISSUE #2398] S3FS fails quietly on incorrect password #1181

Open
opened 2026-03-04 01:51:58 +03:00 by kerem · 6 comments
Owner

Originally created by @JeremyWesthead on GitHub (Jan 11, 2024).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2398

Additional Information

Version of s3fs being used (s3fs --version)

V1.90 (commit:unknown)

Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse or dpkg -s fuse)

No package 'fuse' found or similar, but S3FS works fine generally

Kernel information (uname -r)

6.2.0-39-generic

GNU/Linux Distribution, if applicable (cat /etc/os-release)

PRETTY_NAME="Ubuntu 22.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.3 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

How to run s3fs, if applicable

[X] command line
[] /etc/fstab

s3fs <bucket name> <local empty dir> -o passwd_file=<password file> -o url=<OCI endpoint> -o use_path_request_style -o nonempty

s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)

Jan 11 13:10:51 yrden s3fs[720477]: s3fs version 1.90(unknown) : s3fs -o passwd_file=<password file> -o url=<OCI endpoint> -o use_path_request_style -o nonempty <bucket name> <local empty dir>
Jan 11 13:10:51 yrden s3fs[720477]: Loaded mime information from /etc/mime.types
Jan 11 13:10:51 yrden s3fs[720482]: init v1.90(commit:unknown) with GnuTLS(gcrypt)
Jan 11 13:10:53 yrden s3fs[720482]: s3fs.cpp:s3fs_check_service(3575): Failed to connect by sigv4, so retry to connect by signature version 2.
Jan 11 13:10:53 yrden s3fs[720482]: s3fs.cpp:s3fs_check_service(3587): Bad Request(host=<OCI endpoint>) - result of checking service.

Details about issue

If the password file (<password file> above) contains a valid but incorrect password, s3fs gives no indication of failure to mount. This has been tested with OCI's buckets - not sure if platform specific.

Cases which cause sensible 'failure to mount' errors (non-0 exitcode and a stderr message):

  • Password file has permissions other than 600
  • Password file contains whitespace
  • Password file is not of correct format

Specific case this issue refers to which has an exitcode of 0 and no stderr message:

  • Password file has correct permissions and formatting, but is functionally an incorrect password

Password file contents of access_key:not_real_key should replicate this behaviour. With a valid password file contents, the same command correctly mounts. Such a fail to mount should tell the user that the mount has failed.

Originally created by @JeremyWesthead on GitHub (Jan 11, 2024). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2398 <!-- -------------------------------------------------------------------------- The following information is very important in order to help us to help you. Omission of the following details may delay your support request or receive no attention at all. Keep in mind that the commands we provide to retrieve information are oriented to GNU/Linux Distributions, so you could need to use others if you use s3fs on macOS or BSD. --------------------------------------------------------------------------- --> ### Additional Information #### Version of s3fs being used (`s3fs --version`) `V1.90 (commit:unknown)` #### Version of fuse being used (`pkg-config --modversion fuse`, `rpm -qi fuse` or `dpkg -s fuse`) `No package 'fuse' found` or similar, but S3FS works fine generally #### Kernel information (`uname -r`) 6.2.0-39-generic #### GNU/Linux Distribution, if applicable (`cat /etc/os-release`) PRETTY_NAME="Ubuntu 22.04.3 LTS" NAME="Ubuntu" VERSION_ID="22.04" VERSION="22.04.3 LTS (Jammy Jellyfish)" VERSION_CODENAME=jammy ID=ubuntu ID_LIKE=debian HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" UBUNTU_CODENAME=jammy #### How to run s3fs, if applicable <!-- Describe the s3fs "command line" or "/etc/fstab" entry used. --> [X] command line [] /etc/fstab <!-- Executed command line or /etc/fastab entry --> ``` s3fs <bucket name> <local empty dir> -o passwd_file=<password file> -o url=<OCI endpoint> -o use_path_request_style -o nonempty ``` #### s3fs syslog messages (`grep s3fs /var/log/syslog`, `journalctl | grep s3fs`, or `s3fs outputs`) <!-- if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages. --> ``` Jan 11 13:10:51 yrden s3fs[720477]: s3fs version 1.90(unknown) : s3fs -o passwd_file=<password file> -o url=<OCI endpoint> -o use_path_request_style -o nonempty <bucket name> <local empty dir> Jan 11 13:10:51 yrden s3fs[720477]: Loaded mime information from /etc/mime.types Jan 11 13:10:51 yrden s3fs[720482]: init v1.90(commit:unknown) with GnuTLS(gcrypt) Jan 11 13:10:53 yrden s3fs[720482]: s3fs.cpp:s3fs_check_service(3575): Failed to connect by sigv4, so retry to connect by signature version 2. Jan 11 13:10:53 yrden s3fs[720482]: s3fs.cpp:s3fs_check_service(3587): Bad Request(host=<OCI endpoint>) - result of checking service. ``` ### Details about issue <!-- Please describe the content of the issue in detail. --> If the password file (`<password file>` above) contains a valid but incorrect password, s3fs gives no indication of failure to mount. This has been tested with OCI's buckets - not sure if platform specific. Cases which cause sensible 'failure to mount' errors (non-0 exitcode and a stderr message): * Password file has permissions other than `600` * Password file contains whitespace * Password file is not of correct format Specific case this issue refers to which has an exitcode of 0 and no stderr message: * Password file has correct permissions and formatting, but is functionally an incorrect password Password file contents of `access_key:not_real_key` should replicate this behaviour. With a valid password file contents, the same command correctly mounts. Such a fail to mount should tell the user that the mount has failed.
Author
Owner

@ggtakec commented on GitHub (Feb 12, 2024):

@JeremyWesthead
I tried it on AWS S3, but in this case I receive a 403 error and s3fs exits with Invalid Credentials.
I checked the code for v1.90 and your error message, and you're probably receiving the 400 error.
This is probably why the Bad Request error is displayed.

Since the error message in your case cannot be parsed, create a PR #2410 and modify it to output the error message included in the response.

<!-- gh-comment-id:1938054979 --> @ggtakec commented on GitHub (Feb 12, 2024): @JeremyWesthead I tried it on AWS S3, but in this case I receive a `403` error and s3fs exits with `Invalid Credentials`. I checked the code for v1.90 and your error message, and you're probably receiving the `400` error. This is probably why the `Bad Request` error is displayed. Since the error message in your case cannot be parsed, create a PR #2410 and modify it to output the error message included in the response.
Author
Owner

@brodly commented on GitHub (Mar 4, 2024):

I'm finding that s3fs fails silently on many types of errors.

Running this command
mount --target /mnt/newBucket/ --source my-bucket --type fuse.s3fs -o profile=test

will yield these logs if bucket is wrong or creds are wrong:
s3fs.cpp:s3fs_check_service(3593): bucket or key not found(host=https://s3.amazonaws.com) - result of checking service.
s3fs.cpp:s3fs_check_service(3590): invalid credentials(host=https://s3.amazonaws.com) - result of checking service.

s3fs returns exit code 0 and signals to system to mount, in my case these were the logs

systemd[3563169]: mnt-newBucket.mount: Succeeded.
systemd[1]: mnt-newBucket.mount: Succeeded.

not sure if this is intended, seems strange esp for automated mounting solutions. any thoughts?

<!-- gh-comment-id:1977647936 --> @brodly commented on GitHub (Mar 4, 2024): I'm finding that s3fs fails silently on many types of errors. Running this command ` mount --target /mnt/newBucket/ --source my-bucket --type fuse.s3fs -o profile=test` will yield these logs if bucket is wrong or creds are wrong: `s3fs.cpp:s3fs_check_service(3593): bucket or key not found(host=https://s3.amazonaws.com) - result of checking service.` `s3fs.cpp:s3fs_check_service(3590): invalid credentials(host=https://s3.amazonaws.com) - result of checking service.` s3fs returns exit code 0 and signals to system to mount, in my case these were the logs ``` systemd[3563169]: mnt-newBucket.mount: Succeeded. systemd[1]: mnt-newBucket.mount: Succeeded. ``` not sure if this is intended, seems strange esp for automated mounting solutions. any thoughts?
Author
Owner

@ggtakec commented on GitHub (Mar 6, 2024):

@brodly Thanks for the details.
I was able to confirm that the exit code of the mount command is 0 if it fails to start via the mount command(a case of wrong credential, etc).
(then s3fs has failed to run and cannot be mounted.)

We need to check why the mount command is not inheriting the s3fs exit code.

<!-- gh-comment-id:1981116660 --> @ggtakec commented on GitHub (Mar 6, 2024): @brodly Thanks for the details. I was able to confirm that the exit code of the mount command is 0 if it fails to start via the mount command(a case of wrong credential, etc). _(then s3fs has failed to run and cannot be mounted.)_ We need to check why the mount command is not inheriting the s3fs exit code.
Author
Owner

@brodly commented on GitHub (Mar 6, 2024):

thanks @ggtakec. I also noticed this behavior when using the s3fs bin directly too. It reported no logs in stdout and just returned a 0. Saw the same logs as I posted above in journalctl and running with a -f.

Since my use case is automated, had to build a little utility to check /proc/mounts to see if we actually mounted the bucket before continuing on.

FYI I have been using v1.89 but was able to also test on 1.90. Havent tested on versions beyond that.

<!-- gh-comment-id:1981208521 --> @brodly commented on GitHub (Mar 6, 2024): thanks @ggtakec. I also noticed this behavior when using the `s3fs` bin directly too. It reported no logs in stdout and just returned a 0. Saw the same logs as I posted above in journalctl and running with a `-f`. Since my use case is automated, had to build a little utility to check /proc/mounts to see if we actually mounted the bucket before continuing on. FYI I have been using v1.89 but was able to also test on 1.90. Havent tested on versions beyond that.
Author
Owner

@ggtakec commented on GitHub (Mar 6, 2024):

Thank you for your reply.
I checked the current master code and confirmed that exit code=0 at that time.
Since the s3fs process is not exiting with 0, we need to know if there is some other way to convey the exit code to the mount command.

<!-- gh-comment-id:1981226531 --> @ggtakec commented on GitHub (Mar 6, 2024): Thank you for your reply. I checked the current master code and confirmed that exit code=0 at that time. Since the s3fs process is not exiting with 0, we need to know if there is some other way to convey the exit code to the mount command.
Author
Owner

@ggtakec commented on GitHub (Mar 10, 2024):

I looked into this a little more, but the result remains the same that s3fs itself exits with an error code (1), but the mount command always returns 0.
I checked to see if I could manipulate this value using libfuse's functions , but I could not find such I/F exists.
If it fails, there is not no entry in mtab either, so we should check instead of using the result of the mount command as you do now.

If anyone knows the cause of this, I look forward to helping.

<!-- gh-comment-id:1987040196 --> @ggtakec commented on GitHub (Mar 10, 2024): I looked into this a little more, but the result remains the same that s3fs itself exits with an error code (1), but the mount command always returns 0. I checked to see if I could manipulate this value using libfuse's functions , but I could not find such I/F exists. If it fails, there is not no entry in mtab either, so we should check instead of using the result of the mount command as you do now. If anyone knows the cause of this, I look forward to helping.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#1181
No description provided.