credentials and dynamic update credentials #1170

New issue

Open

opened 2026-03-04 01:51:53 +03:00 by kerem · 2 comments

kerem commented

2026-03-04 01:51:53 +03:00

Owner

Originally created by @Scisaga on GitHub (Nov 25, 2023).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2378

Additional Information

Version of s3fs being used (`s3fs --version`)

V1.9.3

Version of fuse being used (`pkg-config --modversion fuse`, `rpm -qi fuse` or `dpkg -s fuse`)

2.9.9-3

Kernel information (`uname -r`)

5.4.0-155-generic

GNU/Linux Distribution, if applicable (`cat /etc/os-release`)

How to run s3fs, if applicable

s3fs "$bucket:/$prefix" "$mount_point/$bucket" -o use_session_token -o url=$url -o use_path_request_style -o compat_dir -o allow_other -o dbglevel=info -o curldbg

Details about issue

I'm creating a docker container using s3fs mount several buckets to load dataset for my model trainer, for some safety reasons, each container has it's one specific sts role. Just before container creatioin, i can get aws_access_key_id, aws_secret_access_key, aws_session_token from ceph sts endpoint and save into ~/.aws/credentials, buckets can be mounted as expected. But eventually token expires, i must restart all s3fs process once a hour to load new credentials, which is is not very appropriate and may lead IO exception when python script running at the right moment.

Can you please provide any mechanism to reload the new credentials without restarting s3fs process? Thank you

Originally created by @Scisaga on GitHub (Nov 25, 2023). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2378 ### Additional Information #### Version of s3fs being used (`s3fs --version`) V1.9.3 #### Version of fuse being used (`pkg-config --modversion fuse`, `rpm -qi fuse` or `dpkg -s fuse`) 2.9.9-3 #### Kernel information (`uname -r`) 5.4.0-155-generic #### GNU/Linux Distribution, if applicable (`cat /etc/os-release`)  #### How to run s3fs, if applicable s3fs "$bucket:/$prefix" "$mount_point/$bucket" -o use_session_token -o url=$url -o use_path_request_style -o compat_dir -o allow_other -o dbglevel=info -o curldbg ### Details about issue I'm creating a docker container using s3fs mount several buckets to load dataset for my model trainer, for some safety reasons, each container has it's one specific sts role. Just before container creatioin, i can get aws_access_key_id, aws_secret_access_key, aws_session_token from ceph sts endpoint and save into ~/.aws/credentials, buckets can be mounted as expected. But eventually token expires, i must restart all s3fs process once a hour to load new credentials, which is is not very appropriate and may lead IO exception when python script running at the right moment. Can you please provide any mechanism to reload the new credentials without restarting s3fs process? Thank you

kerem added the

enhancement

label

2026-03-04 01:51:53 +03:00

kerem commented

2026-03-04 01:51:54 +03:00

Author

Owner

@Scisaga commented on GitHub (Nov 25, 2023):

related issue: #1290

@Scisaga commented on GitHub (Nov 25, 2023): related issue: #1290

kerem commented

2026-03-04 01:51:54 +03:00

Author

Owner

@ggtakec commented on GitHub (Feb 12, 2024):

@Scisaga
The current s3fs does not check for updates to the credential file. (In other words, it does not have a reload function)
Regarding this feature, the question of whether or not to support STS will also be an issue to consider.
It may be possible to modify it to detect file updates, but it will require a little more thought.

If the credential can be updated from outside of s3fs, you may be able to use the following DSO.
https://github.com/ggtakec/s3fs-fuse-awscred-lib
Please try it and contact the issue if there is any prevention.

@ggtakec commented on GitHub (Feb 12, 2024): @Scisaga The current s3fs does not check for updates to the credential file. (In other words, it does not have a reload function) Regarding this feature, the question of whether or not to support STS will also be an issue to consider. It may be possible to modify it to detect file updates, but it will require a little more thought. If the credential can be updated from outside of s3fs, you may be able to use the following DSO. https://github.com/ggtakec/s3fs-fuse-awscred-lib Please try it and contact the issue if there is any prevention.

kerem referenced this issue

2026-03-04 02:02:32 +03:00

[PR #1170] [CLOSED] just test ensure_diskfree param #1862