[GH-ISSUE #2504] SIGv4 , IAM, and credentials issue. New bucket in ap-southeast-2 with url and endpoint specified. #1221

New issue

Closed

opened 2026-03-04 01:52:19 +03:00 by kerem · 2 comments

kerem commented

2026-03-04 01:52:19 +03:00

Owner

Originally created by @sxwebster on GitHub (Jul 25, 2024).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2504

Additional Information

Version of s3fs being used (`s3fs --version`)

1.93

Version of fuse being used (`pkg-config --modversion fuse`, `rpm -qi fuse` or `dpkg -s fuse`)

3.14.0

Kernel information (`uname -r`)

6.8.0-1012-aws

GNU/Linux Distribution, if applicable (`cat /etc/os-release`)

Ubuntu 24.04 & 22.04

How to run s3fs, if applicable

Established via a service. my-backups is defined as an environment variable and the service file is re-written during deployment to include it. This part works.

    [Unit]
    Description=Mount S3 Bucket and create symlinks
    After=network-online.target
    Wants=network-online.target

    [Service]
    Type=simple
    RemainAfterExit=yes
    ExecStart=/usr/bin/s3fs ${my-backups} /home/ubuntu/s3fs_mount -o url=https://s3.ap-southeast-2.amazonaws.com -o iam_role=auto 
    ExecStop=/bin/umount /home/ubuntu/s3fs_mount
    Restart=always
    RestartSec=10

    [Install]
    WantedBy=multi-user.target

I have also tried variations of the options, including the full and subset of the following:

ExecStart=/usr/bin/s3fs my-backups /home/ubuntu/s3fs_mount -o url=http://s3.ap-southeast-2.amazonaws.com -o endpoint=ap-southeast-2 -o iam_role=my-ec2-iam-policys -o nonempty -o allow_other

I would very much prefer not to be so granular with my policy name and leave it to auto if possible, since it's already attached to the instance.

s3fs syslog messages (`grep s3fs /var/log/syslog`, `journalctl | grep s3fs`, or `s3fs outputs`)

Jul 25 02:53:06 ip-10-0-21-44 systemd[1]: Starting s3fs-mount.service - Mount S3 Bucket and create symlinks...
░░ Subject: A start job for unit s3fs-mount.service has begun execution
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░ 
░░ A start job for unit s3fs-mount.service has begun execution.
░░ 
░░ The job identifier is 73701.
Jul 25 02:53:06 ip-10-0-21-44 s3fs[8605]: s3fs version 1.93(unknown) : s3fs -o url=http://s3.ap-southeast-2.amazonaws.com -o endpoint=ap-southeast-2 -o iam_role=qfd-spark-ec2-iam-policys -o nonempty -o allow_other qfd-spark-backups /home/ubuntu/s3fs_mount
Jul 25 02:53:06 ip-10-0-21-44 s3fs[8605]: Loaded mime information from /etc/mime.types
Jul 25 02:53:06 ip-10-0-21-44 s3fs[8609]: init v1.93(commit:unknown) with GnuTLS(gcrypt), credential-library(built-in)
Jul 25 02:53:06 ip-10-0-21-44 s3fs[8609]: s3fs.cpp:s3fs_check_service(4508): Failed to connect by sigv4, so retry to connect by signature version 2. But you should to review url and endpoint option.
Jul 25 02:53:06 ip-10-0-21-44 s3fs[8609]: s3fs.cpp:s3fs_check_service(4522): Failed to check bucket and directory for mount point : Invalid Credentials(host=http://s3.ap-southeast-2.amazonaws.com)
Jul 25 02:53:06 ip-10-0-21-44 systemd[1]: Started s3fs-mount.service - Mount S3 Bucket and create symlinks.
░░ Subject: A start job for unit s3fs-mount.service has finished successfully
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░ 
░░ A start job for unit s3fs-mount.service has finished successfully.
░░ 
░░ The job identifier is 73701.

Details about issue

All advice I've been able to find so far suggests using the url and endpoint flags. Based upon the error messages, they seem to understand that I'm providing these flags. SIGv4 seems to be the first issue, followed by credentials. I do not wish to supply a set of credentials since this is a production system that's web facing, and supplying plaintext credentials is never good policy.

Originally created by @sxwebster on GitHub (Jul 25, 2024). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2504  ### Additional Information #### Version of s3fs being used (`s3fs --version`) 1.93 #### Version of fuse being used (`pkg-config --modversion fuse`, `rpm -qi fuse` or `dpkg -s fuse`) 3.14.0 #### Kernel information (`uname -r`) 6.8.0-1012-aws #### GNU/Linux Distribution, if applicable (`cat /etc/os-release`) Ubuntu 24.04 & 22.04 #### How to run s3fs, if applicable Established via a service. my-backups is defined as an environment variable and the service file is re-written during deployment to include it. This part works. ``` [Unit] Description=Mount S3 Bucket and create symlinks After=network-online.target Wants=network-online.target [Service] Type=simple RemainAfterExit=yes ExecStart=/usr/bin/s3fs ${my-backups} /home/ubuntu/s3fs_mount -o url=https://s3.ap-southeast-2.amazonaws.com -o iam_role=auto ExecStop=/bin/umount /home/ubuntu/s3fs_mount Restart=always RestartSec=10 [Install] WantedBy=multi-user.target ``` I have also tried variations of the options, including the full and subset of the following: ``` ExecStart=/usr/bin/s3fs my-backups /home/ubuntu/s3fs_mount -o url=http://s3.ap-southeast-2.amazonaws.com -o endpoint=ap-southeast-2 -o iam_role=my-ec2-iam-policys -o nonempty -o allow_other ``` I would very much prefer not to be so granular with my policy name and leave it to auto if possible, since it's already attached to the instance. #### s3fs syslog messages (`grep s3fs /var/log/syslog`, `journalctl | grep s3fs`, or `s3fs outputs`)  ``` Jul 25 02:53:06 ip-10-0-21-44 systemd[1]: Starting s3fs-mount.service - Mount S3 Bucket and create symlinks... ░░ Subject: A start job for unit s3fs-mount.service has begun execution ░░ Defined-By: systemd ░░ Support: http://www.ubuntu.com/support ░░ ░░ A start job for unit s3fs-mount.service has begun execution. ░░ ░░ The job identifier is 73701. Jul 25 02:53:06 ip-10-0-21-44 s3fs[8605]: s3fs version 1.93(unknown) : s3fs -o url=http://s3.ap-southeast-2.amazonaws.com -o endpoint=ap-southeast-2 -o iam_role=qfd-spark-ec2-iam-policys -o nonempty -o allow_other qfd-spark-backups /home/ubuntu/s3fs_mount Jul 25 02:53:06 ip-10-0-21-44 s3fs[8605]: Loaded mime information from /etc/mime.types Jul 25 02:53:06 ip-10-0-21-44 s3fs[8609]: init v1.93(commit:unknown) with GnuTLS(gcrypt), credential-library(built-in) Jul 25 02:53:06 ip-10-0-21-44 s3fs[8609]: s3fs.cpp:s3fs_check_service(4508): Failed to connect by sigv4, so retry to connect by signature version 2. But you should to review url and endpoint option. Jul 25 02:53:06 ip-10-0-21-44 s3fs[8609]: s3fs.cpp:s3fs_check_service(4522): Failed to check bucket and directory for mount point : Invalid Credentials(host=http://s3.ap-southeast-2.amazonaws.com) Jul 25 02:53:06 ip-10-0-21-44 systemd[1]: Started s3fs-mount.service - Mount S3 Bucket and create symlinks. ░░ Subject: A start job for unit s3fs-mount.service has finished successfully ░░ Defined-By: systemd ░░ Support: http://www.ubuntu.com/support ░░ ░░ A start job for unit s3fs-mount.service has finished successfully. ░░ ░░ The job identifier is 73701. ``` ### Details about issue  All advice I've been able to find so far suggests using the url and endpoint flags. Based upon the error messages, they seem to understand that I'm providing these flags. SIGv4 seems to be the first issue, followed by credentials. I do not wish to supply a set of credentials since this is a production system that's web facing, and supplying plaintext credentials is never good policy.

kerem closed this issue

2026-03-04 01:52:19 +03:00

kerem commented

2026-03-04 01:52:20 +03:00

Author

Owner

@sxwebster commented on GitHub (Jul 25, 2024):

I made a mistake with the bucket name in my policy. Once this was fixed, everything worked.

@sxwebster commented on GitHub (Jul 25, 2024): I made a mistake with the bucket name in my policy. Once this was fixed, everything worked.

kerem commented

2026-03-04 01:52:20 +03:00

Author

Owner

@sxwebster commented on GitHub (Jul 25, 2024):

I should also say, I needed to specify the role name and not leave it as auto, even though the iam policy is attached to the instance.

@sxwebster commented on GitHub (Jul 25, 2024): I should also say, I needed to specify the role name and not leave it as auto, even though the iam policy is attached to the instance.