starred/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2026-04-25 13:26:00 +03:00
Code Issues 305 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #803] S3FS Only connects with -f flag #463

New issue
Closed
opened 2026-03-04 01:45:50 +03:00 by kerem · 6 comments
kerem commented 2026-03-04 01:45:50 +03:00
Owner
Copy link

Originally created by @zoranSpare on GitHub (Jul 17, 2018).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/803

I have installed s3fs on an EC instance of ubuntu with the following details:
Version: V1.84(commit:06032aa) with OpenSSL
Ubuntu: Ubuntu 16.04.4 LTS

If I run s3fs with the -f flag to bring it to the foreground all is well:
sudo s3fs restoreawsbucket -o use_cache=/tmp -o allow_other -o passwd_file=/etc/passwd-s3fs /restorebucket -f -d

I get the following output:

CRT] s3fs.cpp:set_s3fs_log_level(273): change debug level from [CRT] to [INF] 
[INF]     s3fs.cpp:set_mountpoint_attribute(4180): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755)
[INF] s3fs.cpp:s3fs_init(3379): init v1.84(commit:06032aa) with OpenSSL
[INF] s3fs.cpp:s3fs_check_service(3721): check services.
[INF]       curl.cpp:CheckBucket(3076): check a bucket.
[INF]       curl.cpp:prepare_url(4265): URL is https://s3.amazonaws.com/rexxxbucket/
[INF]       curl.cpp:prepare_url(4297): URL changed is https://rxxxbucket.s3.amazonaws.com/
[INF]       curl.cpp:insertV4Headers(2408): computing signature [GET] [/] [] []
[INF]       curl.cpp:url_to_host(101): url is https://s3.amazonaws.com
[ERR] curl.cpp:RequestPerform(2074): HTTP response code 400, returning EIO. Body Text: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'ap-southeast-2'</Message><Region>ap-southeast-2</Region><RequestId>xxx</RequestId><HostId>xxxx</HostId></Error>
[ERR] curl.cpp:CheckBucket(3104): Check bucket failed, S3 response: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'ap-southeast-2'</Message><Region>ap-southeast-2</Region><RequestId>xxx</RequestId><HostId>xxx</HostId></Error>
[CRT] s3fs.cpp:s3fs_check_service(3742): Could not connect wrong region us-east-1, so retry to connect region ap-southeast-2.
[INF]       curl.cpp:CheckBucket(3076): check a bucket.
[INF]       curl.cpp:prepare_url(4265): URL is https://s3-ap-southeast-2.amazonaws.com/rxxxxbucket/
[INF]       curl.cpp:prepare_url(4297): URL changed is https://rxxxbucket.s3-ap-southeast-2.amazonaws.com/
[INF]       curl.cpp:insertV4Headers(2408): computing signature [GET] [/] [] []
[INF]       curl.cpp:url_to_host(101): url is https://s3-ap-southeast-2.amazonaws.com
[INF]       curl.cpp:RequestPerform(2062): HTTP response code 200

But if I run it without -f flag it won't mount as follows:

init v1.84(commit:06032aa) with OpenSSL
check services.
check a bucket.
URL is https://s3.amazonaws.com/rxxxbucket/
URL changed is https://rxxxbucket.s3.amazonaws.com/
computing signature [GET] [/] [] []
url is https://s3.amazonaws.com
curl.cpp:RequestPerform(2078): HTTP response code 403, returning EPERM. Body Text: <?xml version="1.0" encoding="UTF-8"?>#012<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>xxxx</RequestId><HostId>xxxx</HostId></Error>
curl.cpp:CheckBucket(3104): Check bucket failed, S3 response: <?xml version="1.0" encoding="UTF-8"?>#012<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>xxxx</RequestId><HostId>xxx</HostId></Error>
s3fs.cpp:s3fs_check_service(3762): Could not connect, so retry to connect by signature version 2.
check a bucket.
URL is https://s3.amazonaws.com/rxxxbucket/
URL changed is https://rxxxxbucket.s3.amazonaws.com/
curl.cpp:RequestPerform(2078): HTTP response code 403, returning EPERM. Body Text: <?xml version="1.0" encoding="UTF-8"?>#012<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>xxxx</RequestId><HostId>xxxx</HostId></Error>
curl.cpp:CheckBucket(3104): Check bucket failed, S3 response: <?xml version="1.0" encoding="UTF-8"?>#012<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>xxx</RequestId><HostId>xxx</HostId></Error>
s3fs.cpp:s3fs_check_service(3777): invalid credentials(host=https://s3.amazonaws.com) - result of checking service.
s3fs.cpp:s3fs_exit_fuseloop(3369): Exiting FUSE event loop due to errors
destroy

I can't any options to get around this unless I'm missing something obvious. And, it that case apologies.

Z.

Originally created by @zoranSpare on GitHub (Jul 17, 2018). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/803 I have installed s3fs on an EC instance of ubuntu with the following details: Version: V1.84(commit:06032aa) with OpenSSL Ubuntu: Ubuntu 16.04.4 LTS If I run s3fs with the -f flag to bring it to the foreground all is well: `sudo s3fs restoreawsbucket -o use_cache=/tmp -o allow_other -o passwd_file=/etc/passwd-s3fs /restorebucket -f -d` I get the following output: ``` CRT] s3fs.cpp:set_s3fs_log_level(273): change debug level from [CRT] to [INF] [INF] s3fs.cpp:set_mountpoint_attribute(4180): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755) [INF] s3fs.cpp:s3fs_init(3379): init v1.84(commit:06032aa) with OpenSSL [INF] s3fs.cpp:s3fs_check_service(3721): check services. [INF] curl.cpp:CheckBucket(3076): check a bucket. [INF] curl.cpp:prepare_url(4265): URL is https://s3.amazonaws.com/rexxxbucket/ [INF] curl.cpp:prepare_url(4297): URL changed is https://rxxxbucket.s3.amazonaws.com/ [INF] curl.cpp:insertV4Headers(2408): computing signature [GET] [/] [] [] [INF] curl.cpp:url_to_host(101): url is https://s3.amazonaws.com [ERR] curl.cpp:RequestPerform(2074): HTTP response code 400, returning EIO. Body Text: <?xml version="1.0" encoding="UTF-8"?> <Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'ap-southeast-2'</Message><Region>ap-southeast-2</Region><RequestId>xxx</RequestId><HostId>xxxx</HostId></Error> [ERR] curl.cpp:CheckBucket(3104): Check bucket failed, S3 response: <?xml version="1.0" encoding="UTF-8"?> <Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'ap-southeast-2'</Message><Region>ap-southeast-2</Region><RequestId>xxx</RequestId><HostId>xxx</HostId></Error> [CRT] s3fs.cpp:s3fs_check_service(3742): Could not connect wrong region us-east-1, so retry to connect region ap-southeast-2. [INF] curl.cpp:CheckBucket(3076): check a bucket. [INF] curl.cpp:prepare_url(4265): URL is https://s3-ap-southeast-2.amazonaws.com/rxxxxbucket/ [INF] curl.cpp:prepare_url(4297): URL changed is https://rxxxbucket.s3-ap-southeast-2.amazonaws.com/ [INF] curl.cpp:insertV4Headers(2408): computing signature [GET] [/] [] [] [INF] curl.cpp:url_to_host(101): url is https://s3-ap-southeast-2.amazonaws.com [INF] curl.cpp:RequestPerform(2062): HTTP response code 200 ``` But if I run it without -f flag it won't mount as follows: ``` init v1.84(commit:06032aa) with OpenSSL check services. check a bucket. URL is https://s3.amazonaws.com/rxxxbucket/ URL changed is https://rxxxbucket.s3.amazonaws.com/ computing signature [GET] [/] [] [] url is https://s3.amazonaws.com curl.cpp:RequestPerform(2078): HTTP response code 403, returning EPERM. Body Text: <?xml version="1.0" encoding="UTF-8"?>#012<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>xxxx</RequestId><HostId>xxxx</HostId></Error> curl.cpp:CheckBucket(3104): Check bucket failed, S3 response: <?xml version="1.0" encoding="UTF-8"?>#012<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>xxxx</RequestId><HostId>xxx</HostId></Error> s3fs.cpp:s3fs_check_service(3762): Could not connect, so retry to connect by signature version 2. check a bucket. URL is https://s3.amazonaws.com/rxxxbucket/ URL changed is https://rxxxxbucket.s3.amazonaws.com/ curl.cpp:RequestPerform(2078): HTTP response code 403, returning EPERM. Body Text: <?xml version="1.0" encoding="UTF-8"?>#012<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>xxxx</RequestId><HostId>xxxx</HostId></Error> curl.cpp:CheckBucket(3104): Check bucket failed, S3 response: <?xml version="1.0" encoding="UTF-8"?>#012<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>xxx</RequestId><HostId>xxx</HostId></Error> s3fs.cpp:s3fs_check_service(3777): invalid credentials(host=https://s3.amazonaws.com) - result of checking service. s3fs.cpp:s3fs_exit_fuseloop(3369): Exiting FUSE event loop due to errors destroy ``` I can't any options to get around this unless I'm missing something obvious. And, it that case apologies. Z.
kerem closed this issue 2026-03-04 01:45:50 +03:00
kerem commented 2026-03-04 01:45:51 +03:00
Author
Owner
Copy link

@Protopopys commented on GitHub (Aug 14, 2018):

I have the same error
Version: V1.84(commit:06032aa) with OpenSSL
Ubuntu: Ubuntu 16.04.4 LTS

s3fs[18156]: curl.cpp:RequestPerform(2074): HTTP response code 400, returning EIO. Body Text: Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed.
Aug 14 15:28:52 s1 s3fs[18156]: curl.cpp:MultiPerform(3934): thread failed - rc(-5)
Aug 14 15:29:01 s1 s3fs[18156]: curl.cpp:RequestPerform(2074): HTTP response code 400, returning EIO. Body Text: Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed.

Command for mount:

s3fs  -o passwd_file=$PASS_FILE $BUCKET_NAME:$BUCKET_FOLDER $MOUNT_TO  -o use_cache=$CACHE_FOLDER -o allow_other,retries=10,connect_timeout=30,readwrite_timeout=30,use_path_request_style,parallel_count=10,dbglevel="info"
<!-- gh-comment-id:412843308 --> @Protopopys commented on GitHub (Aug 14, 2018): I have the same error Version: V1.84(commit:06032aa) with OpenSSL Ubuntu: Ubuntu 16.04.4 LTS ``` s3fs[18156]: curl.cpp:RequestPerform(2074): HTTP response code 400, returning EIO. Body Text: Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed. Aug 14 15:28:52 s1 s3fs[18156]: curl.cpp:MultiPerform(3934): thread failed - rc(-5) Aug 14 15:29:01 s1 s3fs[18156]: curl.cpp:RequestPerform(2074): HTTP response code 400, returning EIO. Body Text: Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed. ``` Command for mount: ``` s3fs -o passwd_file=$PASS_FILE $BUCKET_NAME:$BUCKET_FOLDER $MOUNT_TO -o use_cache=$CACHE_FOLDER -o allow_other,retries=10,connect_timeout=30,readwrite_timeout=30,use_path_request_style,parallel_count=10,dbglevel="info" ```
kerem commented 2026-03-04 01:45:51 +03:00
Author
Owner
Copy link

@zoranSpare commented on GitHub (Aug 14, 2018):

I found it resolved itself the following day. I didn’t make any changes just left it to finish up other stuff.

Z.

From: Kirill Protopopov notifications@github.com
Reply-To: s3fs-fuse/s3fs-fuse reply@reply.github.com
Date: Tuesday, 14 August 2018 at 9:39 pm
To: s3fs-fuse/s3fs-fuse s3fs-fuse@noreply.github.com
Cc: Zoran zoran@sparecreative.com, Author author@noreply.github.com
Subject: Re: [s3fs-fuse/s3fs-fuse] S3FS Only connects with -f flag (#803)

I have the same error
Version: V1.84(commit:06032aa) with OpenSSL
Ubuntu: Ubuntu 16.04.4 LTS
s3fs[18156]: curl.cpp:RequestPerform(2074): HTTP response code 400, returning EIO. Body Text: Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed.
Aug 14 15:28:52 s1 s3fs[18156]: curl.cpp:MultiPerform(3934): thread failed - rc(-5)
Aug 14 15:29:01 s1 s3fs[18156]: curl.cpp:RequestPerform(2074): HTTP response code 400, returning EIO. Body Text: Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed.
Command for mount:
s3fs  -o passwd_file=$PASS_FILE $BUCKET_NAME:$BUCKET_FOLDER $MOUNT_TO  -o use_cache=$CACHE_FOLDER -o allow_other,retries=10,connect_timeout=30,readwrite_timeout=30,use_path_request_style,parallel_count=10,dbglevel="info"
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

<!-- gh-comment-id:412848428 --> @zoranSpare commented on GitHub (Aug 14, 2018): I found it resolved itself the following day. I didn’t make any changes just left it to finish up other stuff. Z. From: Kirill Protopopov <notifications@github.com> Reply-To: s3fs-fuse/s3fs-fuse <reply@reply.github.com> Date: Tuesday, 14 August 2018 at 9:39 pm To: s3fs-fuse/s3fs-fuse <s3fs-fuse@noreply.github.com> Cc: Zoran <zoran@sparecreative.com>, Author <author@noreply.github.com> Subject: Re: [s3fs-fuse/s3fs-fuse] S3FS Only connects with -f flag (#803) I have the same error Version: V1.84(commit:06032aa) with OpenSSL Ubuntu: Ubuntu 16.04.4 LTS s3fs[18156]: curl.cpp:RequestPerform(2074): HTTP response code 400, returning EIO. Body Text: Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed. Aug 14 15:28:52 s1 s3fs[18156]: curl.cpp:MultiPerform(3934): thread failed - rc(-5) Aug 14 15:29:01 s1 s3fs[18156]: curl.cpp:RequestPerform(2074): HTTP response code 400, returning EIO. Body Text: Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed. Command for mount: s3fs  -o passwd_file=$PASS_FILE $BUCKET_NAME:$BUCKET_FOLDER $MOUNT_TO  -o use_cache=$CACHE_FOLDER -o allow_other,retries=10,connect_timeout=30,readwrite_timeout=30,use_path_request_style,parallel_count=10,dbglevel="info" — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.
kerem commented 2026-03-04 01:45:51 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Apr 9, 2019):

Do you still encounter this issue? 1.85 has some additional endpoint handling which may resolve this issue.

<!-- gh-comment-id:481183510 --> @gaul commented on GitHub (Apr 9, 2019): Do you still encounter this issue? 1.85 has some additional endpoint handling which may resolve this issue.
kerem commented 2026-03-04 01:45:51 +03:00
Author
Owner
Copy link

@robclancy commented on GitHub (Apr 15, 2019):

Connecting is super random for me. If I keep trying eventually it will work sometimes. Or with -f I can get it to work after around 3 tries. It gives 403 or 200 seemingly randomly.
EDIT: I just kept retrying and it took 28 times then it worked. I feel like it's more an issue with s3 than this.

<!-- gh-comment-id:483073883 --> @robclancy commented on GitHub (Apr 15, 2019): Connecting is super random for me. If I keep trying eventually it will work sometimes. Or with `-f` I can get it to work after around 3 tries. It gives 403 or 200 seemingly randomly. EDIT: I just kept retrying and it took 28 times then it worked. I feel like it's more an issue with s3 than this.
kerem commented 2026-03-04 01:45:51 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Apr 15, 2019):

Can you share log messages from your failed and successful runs? Please specify -d -o curldbg to increase log levels.

<!-- gh-comment-id:483111405 --> @gaul commented on GitHub (Apr 15, 2019): Can you share log messages from your failed and successful runs? Please specify `-d -o curldbg` to increase log levels.
kerem commented 2026-03-04 01:45:51 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Feb 3, 2020):

Closing due to inactivity. Please reopen if symptoms persist.

<!-- gh-comment-id:581293039 --> @gaul commented on GitHub (Feb 3, 2020): Closing due to inactivity. Please reopen if symptoms persist.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
v1.97
v1.96
v1.95
v1.94
v1.93
v1.92
v1.91
v1.90
v1.89
v1.88
v1.87
v1.86
v1.85
v1.84
v1.83
v1.82
v1.81
v1.80
v1.79
v1.78
Pre-v1.78
v1.77
v1.76
v1.75
v1.74
Labels
Clear labels   
bug

   
bug

   
dataloss

   
duplicate

   
enhancement

   
feature request

   
help wanted

   
invalid

   
need info

   
performance

   
pull-request

Mirrored from GitHub Pull Request

  
question

   
question

   
testing
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#463
No description provided.