starred/s3fs-fuse

Fork 0

mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2026-04-25 13:26:00 +03:00

[GH-ISSUE #1122] profile option is not sufficient to authenticate #594

New issue

Open

opened 2026-03-04 01:47:02 +03:00 by kerem · 6 comments

kerem commented

2026-03-04 01:47:02 +03:00

Owner

Originally created by @ooverandout on GitHub (Aug 7, 2019).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1122

sf3s: 1.85
Linux Mint 19.2

Hi! I am trying to run s3fs the same way as I am running aws cli with profile name. However, while profile name is sufficient with aws cli to trigger my MFA authorization and proceed with command, s3fs seems just to not see inside the bucket. Also, profile option seems to use my profile setup from credentials, instead of ~/.aws/config:

Cannot do it at all with MFA (most urgent):

user  ➤  s3fs -o profile="my_credentials" -o iam_role="my_role" bucket:/path /home/user/test
fuse: failed to exec fusermount: No such file or directory

Would like to do it with profile from ~/.aws/config

user  ➤  s3fs -o profile="my_profile" bucket:/path /home/user/test
s3fs: Could not find profile: my_profile in file: /home/user/.aws/credentials

Originally created by @ooverandout on GitHub (Aug 7, 2019). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1122 sf3s: 1.85 Linux Mint 19.2 ----- Hi! I am trying to run s3fs the same way as I am running aws cli with profile name. However, while profile name is sufficient with aws cli to trigger my MFA authorization and proceed with command, s3fs seems just to not see inside the bucket. Also, profile option seems to use my profile setup from credentials, instead of ~/.aws/config: * Cannot do it at all with MFA (most urgent): ``` user ➤ s3fs -o profile="my_credentials" -o iam_role="my_role" bucket:/path /home/user/test fuse: failed to exec fusermount: No such file or directory ``` * Would like to do it with profile from ~/.aws/config ``` user ➤ s3fs -o profile="my_profile" bucket:/path /home/user/test s3fs: Could not find profile: my_profile in file: /home/user/.aws/credentials ```

kerem added the

enhancement

help wanted

labels

2026-03-04 01:47:02 +03:00

kerem commented

2026-03-04 01:47:03 +03:00

Author

Owner

@gaul commented on GitHub (Aug 7, 2019):

Try using s3fs 1.85. 1.82 does not include the -o profile option.

@gaul commented on GitHub (Aug 7, 2019): Try using s3fs 1.85. 1.82 does not include the `-o profile` option.

kerem commented

2026-03-04 01:47:03 +03:00

Author

Owner

@ooverandout commented on GitHub (Aug 7, 2019):

Hi ! I use 1.85, please read my post once more (I have updated it some Time ago)

@ooverandout commented on GitHub (Aug 7, 2019): Hi ! I use 1.85, please read my post once more (I have updated it some Time ago)

kerem commented

2026-03-04 01:47:03 +03:00

Author

Owner

@OJFord commented on GitHub (May 29, 2020):

I have the same issue, with --version:

Amazon Simple Storage Service File System V1.86 (commit:unknown) with OpenSSL

To be clear, OP & I are using an aws/config (not a credentials) file, and in my case it specifies an external process to execute in order to retrieve the credentials:

[profile s3fs]
credential_process=/bin/somecmd

so there is no file ~/.aws/credentials.

https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sourcing-external.html

Additionally, the error pointing at ~/.aws (expanded) suggests that it's ignoring the AWS_CONFIG_FILE environment variable, which in my case points at $XDG_CONFIG_HOME/aws, which is different to that default from the error message.

https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html

Edit: Actually, I might be wrong about the last bit, since I see AWS_CONFIG_FILE points directly at the .../aws/config, and there's a separate AWS_SHARED_CREDENTIALS_FILE, which of course I haven't set because I don't have one, so that would be correct to be looking in the default credentials location, if only I hadn't specified a credential_process in my AWS_CONFIG_FILE.

@OJFord commented on GitHub (May 29, 2020): I have the same issue, with `--version`: > Amazon Simple Storage Service File System V1.86 (commit:unknown) with OpenSSL To be clear, OP & I are using an aws/_config_ (*not* a credentials) file, and in my case it specifies an external process to execute in order to retrieve the credentials: ```ini [profile s3fs] credential_process=/bin/somecmd ``` so there is no file `~/.aws/credentials`. https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sourcing-external.html ~Additionally, the error pointing at `~/.aws` (expanded) suggests that it's ignoring the `AWS_CONFIG_FILE` environment variable, which in my case points at `$XDG_CONFIG_HOME/aws`, which is different to that default from the error message.~ https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html Edit: Actually, I might be wrong about the last bit, since I see `AWS_CONFIG_FILE` points directly at the `.../aws/config`, and there's a separate `AWS_SHARED_CREDENTIALS_FILE`, which of course I haven't set because I don't have one, so that would be correct to be looking in the default credentials location, if only I hadn't specified a `credential_process` in my `AWS_CONFIG_FILE`.

kerem commented

2026-03-04 01:47:03 +03:00

Author

Owner

@gaul commented on GitHub (Jul 25, 2021):

This might not be difficult to implement:

https://docs.aws.amazon.com/sdkref/latest/guide/setting-global-credential_process.html

Essentially s3fs just runs the process on startup and periodically renews it based on Expiration.

@gaul commented on GitHub (Jul 25, 2021): This might not be difficult to implement: https://docs.aws.amazon.com/sdkref/latest/guide/setting-global-credential_process.html Essentially s3fs just runs the process on startup and periodically renews it based on `Expiration`.

kerem commented

2026-03-04 01:47:03 +03:00

Author

Owner

@OJFord commented on GitHub (Jul 25, 2021):

@gaul I assume this is only an issue because it's not just delegated to the SDK? (I haven't used the C++ one, but as I understand they're all generated from the same; in Go, Python, and Rust I've just made whatever call without being explicit about authentication, and it works as expected looking for credentials according to precedence in the docs.) Is there a reason it can't be?

Edit: Ah actually, I haven't done much more than search 'Aws.h', but it looks like the SDK might not be used in s3fs at all?

Edit 2: 🤦🏻 now I see #1068 'consider using AWS SDK' linked above. Sorry.

@OJFord commented on GitHub (Jul 25, 2021): @gaul I assume this is only an issue because it's not just delegated to the SDK? (I haven't used the C++ one, but as I understand they're all generated from the same; in Go, Python, and Rust I've just made whatever call without being explicit about authentication, and it works as expected looking for credentials according to precedence in the docs.) Is there a reason it can't be? Edit: Ah actually, I haven't done much more than search 'Aws.h', but it looks like [the SDK](https://docs.aws.amazon.com/sdk-for-cpp/v1/developer-guide/welcome.html) might not be used in s3fs at all? Edit 2: 🤦🏻 _now_ I see #1068 'consider using AWS SDK' linked above. Sorry.

kerem commented

2026-03-04 01:47:03 +03:00

Author

Owner

@gaul commented on GitHub (Jul 29, 2021):

Unfortunately s3fs does not use the AWS SDK. The historical reason for this is that s3fs was written in 2007 and the SDK in 2015. But using the SDK requires a fair amount of rewriting, imposes some environment restrictions, and most (all?) distributions don't have the library which creates headaches for our packagers.

@gaul commented on GitHub (Jul 29, 2021): Unfortunately s3fs does not use the AWS SDK. The historical reason for this is that s3fs was written in 2007 and the SDK in 2015. But using the SDK requires a fair amount of rewriting, imposes some environment restrictions, and most (all?) distributions don't have the library which creates headaches for our packagers.

kerem referenced this issue

2026-03-04 02:01:04 +03:00

[PR #594] [MERGED] Check bucket at public bucket and add nocopyapi option automatically #1564