starred/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2026-04-25 13:26:00 +03:00
Code Issues 305 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #721] Invalid credentials (working in s3cmd) #410

New issue
Closed
opened 2026-03-04 01:45:17 +03:00 by kerem · 17 comments
kerem commented 2026-03-04 01:45:17 +03:00
Owner
Copy link

Originally created by @AdvancedStyle on GitHub (Feb 22, 2018).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/721

I'm using the same IAM credentials on the same machine with s3cmd and it is working with normal access, but when using s3fs get invalid credentials message.

IAM Policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation",
                "s3:ListAllMyBuckets"
            ],
            "Resource": "arn:aws:s3:::mybucketnamexxxxx"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": [
                "arn:aws:s3:::mybucketnamexxxxx",
                "arn:aws:s3:::mybucketnamexxxxx/*"
            ]
        }
    ]
}

Version of s3fs being used (s3fs --version)

Amazon Simple Storage Service File System V1.83(commit:e1dafe7) with OpenSSL
Copyright (C) 2010 Randy Rizun rrizun@gmail.com
License GPL2: GNU GPL version 2 http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Version of fuse being used (pkg-config --modversion fuse)

2.9.4

System information (uname -r)

4.4.0-1049-aws

Distro (cat /etc/issue)

Ubuntu 16.04.3 LTS

s3fs command line used (if applicable)

sudo s3fs -o allow_other -f -o dbglevel=info -o endpoint=ap-southeast-1 -o passwd_file=/etc/passwd-s3fs -o use_cache=/tmp_bucket mybucketnamexxxxx /mailu

s3fs syslog messages (grep s3fs /var/log/syslog, or s3fs outputs)

if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages

[CRT] s3fs.cpp:set_s3fs_log_level(271): change debug level from [CRT] to [INF]
[INF]     s3fs.cpp:set_mountpoint_attribute(4206): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40777)
[INF] s3fs.cpp:s3fs_init(3371): init v1.83(commit:e1dafe7) with OpenSSL
[INF] s3fs.cpp:s3fs_check_service(3747): check services.
[INF]       curl.cpp:CheckBucket(3068): check a bucket.
[INF]       curl.cpp:prepare_url(4253): URL is https://s3.amazonaws.com/mybucketnamexxxxx/
[INF]       curl.cpp:prepare_url(4285): URL changed is https://mybucketnamexxxxx.s3.amazonaws.com/
[INF]       curl.cpp:insertV4Headers(2400): computing signature [GET] [/] [] []
[INF]       curl.cpp:url_to_host(101): url is https://s3.amazonaws.com
[INF]       curl.cpp:RequestPerform(2068): HTTP response code 403 was returned, returning EPERM
[ERR] curl.cpp:CheckBucket(3096): Check bucket failed, S3 response: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>5275E4C03A402F30</RequestId><HostId>DQfAKocwKztFonOFHx5nvdnHnbGBT4ejLMsGa38eL15cx9YEp435i3fB3EjJiGRfTaW3Lsg+ldc=</HostId></Error>
[WAN] s3fs.cpp:s3fs_check_service(3788): Could not connect, so retry to connect by signature version 2.
[INF]       curl.cpp:CheckBucket(3068): check a bucket.
[INF]       curl.cpp:prepare_url(4253): URL is https://s3.amazonaws.com/mybucketnamexxxxx/
[INF]       curl.cpp:prepare_url(4285): URL changed is https://mybucketnamexxxxx.s3.amazonaws.com/
[INF]       curl.cpp:RequestPerform(2068): HTTP response code 403 was returned, returning EPERM
[ERR] curl.cpp:CheckBucket(3096): Check bucket failed, S3 response: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>3BD27DFF3BDE0BAB</RequestId><HostId>B2D74fQR0pN0Hp2N/LwRKssUO5DnYKHkrq84SVTfRBUupvQemQeR4ppHzIr46mmWkT6Iu92Vq/c=</HostId></Error>
[CRT] s3fs.cpp:s3fs_check_service(3803): invalid credentials(host=https://s3.amazonaws.com) - result of checking service.
[ERR] s3fs.cpp:s3fs_exit_fuseloop(3361): Exiting FUSE event loop due to errors

[INF] s3fs.cpp:s3fs_destroy(3434): destroy
[WAN] s3fs.cpp:s3fs_destroy(3438): Could not release curl library.
Originally created by @AdvancedStyle on GitHub (Feb 22, 2018). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/721 I'm using the same IAM credentials on the same machine with s3cmd and it is working with normal access, but when using s3fs get invalid credentials message. ### IAM Policy ``` { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:ListBucket", "s3:GetBucketLocation", "s3:ListAllMyBuckets" ], "Resource": "arn:aws:s3:::mybucketnamexxxxx" }, { "Effect": "Allow", "Action": [ "s3:*" ], "Resource": [ "arn:aws:s3:::mybucketnamexxxxx", "arn:aws:s3:::mybucketnamexxxxx/*" ] } ] } ``` #### Version of s3fs being used (s3fs --version) Amazon Simple Storage Service File System V1.83(commit:e1dafe7) with OpenSSL Copyright (C) 2010 Randy Rizun <rrizun@gmail.com> License GPL2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. #### Version of fuse being used (pkg-config --modversion fuse) 2.9.4 #### System information (uname -r) 4.4.0-1049-aws #### Distro (cat /etc/issue) Ubuntu 16.04.3 LTS #### s3fs command line used (if applicable) ``` sudo s3fs -o allow_other -f -o dbglevel=info -o endpoint=ap-southeast-1 -o passwd_file=/etc/passwd-s3fs -o use_cache=/tmp_bucket mybucketnamexxxxx /mailu ``` #### s3fs syslog messages (grep s3fs /var/log/syslog, or s3fs outputs) _if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages_ ``` [CRT] s3fs.cpp:set_s3fs_log_level(271): change debug level from [CRT] to [INF] [INF] s3fs.cpp:set_mountpoint_attribute(4206): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40777) [INF] s3fs.cpp:s3fs_init(3371): init v1.83(commit:e1dafe7) with OpenSSL [INF] s3fs.cpp:s3fs_check_service(3747): check services. [INF] curl.cpp:CheckBucket(3068): check a bucket. [INF] curl.cpp:prepare_url(4253): URL is https://s3.amazonaws.com/mybucketnamexxxxx/ [INF] curl.cpp:prepare_url(4285): URL changed is https://mybucketnamexxxxx.s3.amazonaws.com/ [INF] curl.cpp:insertV4Headers(2400): computing signature [GET] [/] [] [] [INF] curl.cpp:url_to_host(101): url is https://s3.amazonaws.com [INF] curl.cpp:RequestPerform(2068): HTTP response code 403 was returned, returning EPERM [ERR] curl.cpp:CheckBucket(3096): Check bucket failed, S3 response: <?xml version="1.0" encoding="UTF-8"?> <Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>5275E4C03A402F30</RequestId><HostId>DQfAKocwKztFonOFHx5nvdnHnbGBT4ejLMsGa38eL15cx9YEp435i3fB3EjJiGRfTaW3Lsg+ldc=</HostId></Error> [WAN] s3fs.cpp:s3fs_check_service(3788): Could not connect, so retry to connect by signature version 2. [INF] curl.cpp:CheckBucket(3068): check a bucket. [INF] curl.cpp:prepare_url(4253): URL is https://s3.amazonaws.com/mybucketnamexxxxx/ [INF] curl.cpp:prepare_url(4285): URL changed is https://mybucketnamexxxxx.s3.amazonaws.com/ [INF] curl.cpp:RequestPerform(2068): HTTP response code 403 was returned, returning EPERM [ERR] curl.cpp:CheckBucket(3096): Check bucket failed, S3 response: <?xml version="1.0" encoding="UTF-8"?> <Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>3BD27DFF3BDE0BAB</RequestId><HostId>B2D74fQR0pN0Hp2N/LwRKssUO5DnYKHkrq84SVTfRBUupvQemQeR4ppHzIr46mmWkT6Iu92Vq/c=</HostId></Error> [CRT] s3fs.cpp:s3fs_check_service(3803): invalid credentials(host=https://s3.amazonaws.com) - result of checking service. [ERR] s3fs.cpp:s3fs_exit_fuseloop(3361): Exiting FUSE event loop due to errors [INF] s3fs.cpp:s3fs_destroy(3434): destroy [WAN] s3fs.cpp:s3fs_destroy(3438): Could not release curl library. ```
kerem closed this issue 2026-03-04 01:45:17 +03:00
kerem commented 2026-03-04 01:45:18 +03:00
Author
Owner
Copy link

@undiabler commented on GitHub (Apr 18, 2018):

Same problem. Any ideas how to fix this?

<!-- gh-comment-id:382350939 --> @undiabler commented on GitHub (Apr 18, 2018): Same problem. Any ideas how to fix this?
kerem commented 2026-03-04 01:45:18 +03:00
Author
Owner
Copy link

@undiabler commented on GitHub (Apr 18, 2018):

Looks like we loose auth data following redirects.
Adding url param like -o url="https://s3-eu-west-1.amazonaws.com" fixes the problem.

<!-- gh-comment-id:382353844 --> @undiabler commented on GitHub (Apr 18, 2018): Looks like we loose auth data following redirects. Adding url param like `-o url="https://s3-eu-west-1.amazonaws.com"` fixes the problem.
kerem commented 2026-03-04 01:45:18 +03:00
Author
Owner
Copy link

@tomfanning commented on GitHub (Oct 30, 2018):

Looks like we loose auth data following redirects.
Adding url param like -o url="https://s3-eu-west-1.amazonaws.com" fixes the problem.

Confirmed this fix works for me. Region eu-west-2, ubuntu 14.04

<!-- gh-comment-id:434246905 --> @tomfanning commented on GitHub (Oct 30, 2018): > Looks like we loose auth data following redirects. > Adding url param like `-o url="https://s3-eu-west-1.amazonaws.com"` fixes the problem. Confirmed this fix works for me. Region eu-west-2, ubuntu 14.04
kerem commented 2026-03-04 01:45:18 +03:00
Author
Owner
Copy link

@Rich43 commented on GitHub (Mar 22, 2019):

Confirmed this fixes the issue for me too, git cloned latest source.

<!-- gh-comment-id:475807596 --> @Rich43 commented on GitHub (Mar 22, 2019): Confirmed this fixes the issue for me too, git cloned latest source.
kerem commented 2026-03-04 01:45:18 +03:00
Author
Owner
Copy link

@shlha commented on GitHub (Jun 17, 2019):

Not fixing for me. My bucket is in eu-west-2, any other suggestions please?

<!-- gh-comment-id:502884706 --> @shlha commented on GitHub (Jun 17, 2019): Not fixing for me. My bucket is in eu-west-2, any other suggestions please?
kerem commented 2026-03-04 01:45:18 +03:00
Author
Owner
Copy link

@shlha commented on GitHub (Jun 17, 2019):

Buckets get mounted but i get this error: ls: reading directory .: Input/output error

<!-- gh-comment-id:502884932 --> @shlha commented on GitHub (Jun 17, 2019): Buckets get mounted but i get this error: ls: reading directory .: Input/output error
kerem commented 2026-03-04 01:45:18 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Jul 11, 2019):

Could you mount with debug options -f -d -o curldbg and share the output?

<!-- gh-comment-id:510292926 --> @gaul commented on GitHub (Jul 11, 2019): Could you mount with debug options `-f -d -o curldbg` and share the output?
kerem commented 2026-03-04 01:45:18 +03:00
Author
Owner
Copy link

@ortonomy commented on GitHub (Jan 15, 2020):

This is still an issue in 2020! Hours of searching later, and I find this issue! Is there no way we can fix this?

<!-- gh-comment-id:574559878 --> @ortonomy commented on GitHub (Jan 15, 2020): This is still an issue in 2020! Hours of searching later, and I find this issue! Is there no way we can fix this?
kerem commented 2026-03-04 01:45:19 +03:00
Author
Owner
Copy link

@krzysztof-magosa commented on GitHub (Jan 27, 2020):

It's weird but it works without specifying URL for some buckets while on others it fails. For a time being I started generating URL in Terraform with region EC2 sits in but IMO it's not elegant.

<!-- gh-comment-id:578865875 --> @krzysztof-magosa commented on GitHub (Jan 27, 2020): It's weird but it works without specifying URL for some buckets while on others it fails. For a time being I started generating URL in Terraform with region EC2 sits in but IMO it's not elegant.
kerem commented 2026-03-04 01:45:19 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Feb 2, 2020):

Closing due to inactivity. Please reopen if the issue persists and you can attach the debug logs. Issues do not magically fix themselves; without timely and precise feedback from users there is nothing we can do.

<!-- gh-comment-id:581119221 --> @gaul commented on GitHub (Feb 2, 2020): Closing due to inactivity. Please reopen if the issue persists and you can attach the debug logs. Issues do not magically fix themselves; without timely and precise feedback from users there is nothing we can do.
kerem commented 2026-03-04 01:45:19 +03:00
Author
Owner
Copy link

@ortonomy commented on GitHub (Feb 2, 2020):

@gaul -- specifying the URL works. What I meant was that you can't just use a region outside the default (US Ohio) without specifying the endpoint. Should it be able to handle the redirects?

<!-- gh-comment-id:581122242 --> @ortonomy commented on GitHub (Feb 2, 2020): @gaul -- specifying the URL works. What I meant was that you can't just use a region outside the default (US Ohio) without specifying the endpoint. Should it be able to handle the redirects?
kerem commented 2026-03-04 01:45:19 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Feb 2, 2020):

Can you test with the latest version 1.85 which adds some support for 301/307? Otherwise please open a separate issue.

<!-- gh-comment-id:581134468 --> @gaul commented on GitHub (Feb 2, 2020): Can you test with the latest version 1.85 which adds some support for 301/307? Otherwise please open a separate issue.
kerem commented 2026-03-04 01:45:19 +03:00
Author
Owner
Copy link

@krzysztof-magosa commented on GitHub (Feb 3, 2020):

In my case it was lack of s3:GetBucketLocation permission. After adding it, s3fuse started mounting buckets without touching URL.

<!-- gh-comment-id:581497299 --> @krzysztof-magosa commented on GitHub (Feb 3, 2020): In my case it was lack of `s3:GetBucketLocation` permission. After adding it, s3fuse started mounting buckets without touching URL.
kerem commented 2026-03-04 01:45:19 +03:00
Author
Owner
Copy link

@webristle commented on GitHub (Jun 12, 2020):

Looks like we loose auth data following redirects.
Adding url param like -o url="https://s3-eu-west-1.amazonaws.com" fixes the problem.

Great!!! thank you so much

<!-- gh-comment-id:643398910 --> @webristle commented on GitHub (Jun 12, 2020): > Looks like we loose auth data following redirects. > Adding url param like `-o url="https://s3-eu-west-1.amazonaws.com"` fixes the problem. Great!!! thank you so much
kerem commented 2026-03-04 01:45:19 +03:00
Author
Owner
Copy link

@Peter2121 commented on GitHub (Aug 23, 2020):

Still does not work for me.
All permissions are granted, the bucked is correctly located, but 'Access denied' trying to mount it.

<!-- gh-comment-id:678816178 --> @Peter2121 commented on GitHub (Aug 23, 2020): Still does not work for me. All permissions are granted, the bucked is correctly located, but 'Access denied' trying to mount it.
kerem commented 2026-03-04 01:45:19 +03:00
Author
Owner
Copy link

@juliogonzalez commented on GitHub (Aug 23, 2020):

Please open a new issue, provide all the information requested on the issue template, and the debug output as well (as requested at (https://github.com/s3fs-fuse/s3fs-fuse/issues/721#issuecomment-510292926)

The bug was reported against 1.83, and the latest version is now 1.87, so this require a fresh issue.

Also, it could be an IAM problem as pointed by @krzysztof-magosa at https://github.com/s3fs-fuse/s3fs-fuse/issues/721#issuecomment-581497299, so verify it is not your case.

<!-- gh-comment-id:678821090 --> @juliogonzalez commented on GitHub (Aug 23, 2020): Please open a new issue, provide all the information requested on the issue template, and the debug output as well (as requested at (https://github.com/s3fs-fuse/s3fs-fuse/issues/721#issuecomment-510292926) The bug was reported against 1.83, and the latest version is now 1.87, so this require a fresh issue. Also, it could be an IAM problem as pointed by @krzysztof-magosa at https://github.com/s3fs-fuse/s3fs-fuse/issues/721#issuecomment-581497299, so verify it is not your case.
kerem commented 2026-03-04 01:45:19 +03:00
Author
Owner
Copy link

@WoZ commented on GitHub (Jul 18, 2022):

If anyone read this on 2022, check --endpoint option.

github.com/s3fs-fuse/s3fs-fuse@2723e1049e/doc/man/s3fs.1.in (L235-L241)

<!-- gh-comment-id:1186811277 --> @WoZ commented on GitHub (Jul 18, 2022): If anyone read this on 2022, check `--endpoint` option. https://github.com/s3fs-fuse/s3fs-fuse/blob/2723e1049ed65b8efc5af97993cf761ef9673436/doc/man/s3fs.1.in#L235-L241
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
v1.97
v1.96
v1.95
v1.94
v1.93
v1.92
v1.91
v1.90
v1.89
v1.88
v1.87
v1.86
v1.85
v1.84
v1.83
v1.82
v1.81
v1.80
v1.79
v1.78
Pre-v1.78
v1.77
v1.76
v1.75
v1.74
Labels
Clear labels   
bug

   
bug

   
dataloss

   
duplicate

   
enhancement

   
feature request

   
help wanted

   
invalid

   
need info

   
performance

   
pull-request

Mirrored from GitHub Pull Request

  
question

   
question

   
testing
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#410
No description provided.