[GH-ISSUE #1968] 1.90 on Ubuntu 22.04 fails to mount. 1.89 is fine #995

Open
opened 2026-03-04 01:50:30 +03:00 by kerem · 2 comments
Owner

Originally created by @kylegordon on GitHub (Jun 24, 2022).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1968

Version of s3fs being used (s3fs --version)

version: 1.90 (from Ubuntu 22.04 package repo)

Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse)

Version: 2.9.9-5ubuntu2

Kernel information (uname -r)

5.15.0-37-generic

GNU/Linux Distribution, if applicable (cat /etc/os-release)

PRETTY_NAME="Ubuntu 22.04 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

s3fs command line used, if applicable

s3fs -f deja-dup-auto-akiaii34jkwyomiaprqq mnt/s3 -o profile=personal -o endpoint=eu-west-1 -o curldbg ; echo $?

/etc/fstab entry, if applicable

n/a

s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)

└─(11:21:22)──> s3fs -f deja-dup-auto-akiaii34jkwyomiaprqq mnt/s3 -o profile=personal -o endpoint=eu-west-1 -o curldbg ; echo $?                                                                                              ──(Fri,Jun24)─┘
2022-06-24T10:22:48.932Z [INF] s3fs version 1.90(unknown) : s3fs -f -o profile=personal -o endpoint=eu-west-1 -o curldbg deja-dup-auto-akiaii34jkwyomiaprqq mnt/s3
2022-06-24T10:22:48.936Z [INF] curl.cpp:InitMimeType(434): Loaded mime information from /etc/mime.types
2022-06-24T10:22:48.939Z [INF] s3fs.cpp:s3fs_init(3382): init v1.90(commit:unknown) with GnuTLS(gcrypt)
2022-06-24T10:22:48.968Z [CURL DBG] *   Trying 52.218.98.162:443...
2022-06-24T10:22:48.997Z [CURL DBG] * Connected to deja-dup-auto-akiaii34jkwyomiaprqq.s3.amazonaws.com (52.218.98.162) port 443 (#0)
2022-06-24T10:22:49.036Z [CURL DBG] * found 384 certificates in /etc/ssl/certs
2022-06-24T10:22:49.036Z [CURL DBG] * GnuTLS ciphers: NORMAL:-ARCFOUR-128:-CTYPE-ALL:+CTYPE-X509:-VERS-SSL3.0
2022-06-24T10:22:49.096Z [CURL DBG] * SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
2022-06-24T10:22:49.102Z [CURL DBG] *   server certificate verification OK
2022-06-24T10:22:49.102Z [CURL DBG] *   server certificate status verification SKIPPED
2022-06-24T10:22:49.102Z [CURL DBG] *   common name: *.s3.amazonaws.com (matched)
2022-06-24T10:22:49.102Z [CURL DBG] *   server certificate expiration date OK
2022-06-24T10:22:49.102Z [CURL DBG] *   server certificate activation date OK
2022-06-24T10:22:49.102Z [CURL DBG] *   certificate public key: RSA
2022-06-24T10:22:49.102Z [CURL DBG] *   certificate version: #3
2022-06-24T10:22:49.102Z [CURL DBG] *   subject: CN=*.s3.amazonaws.com
2022-06-24T10:22:49.102Z [CURL DBG] *   start date: Wed, 15 Dec 2021 00:00:00 GMT
2022-06-24T10:22:49.102Z [CURL DBG] *   expire date: Sat, 03 Dec 2022 23:59:59 GMT
2022-06-24T10:22:49.102Z [CURL DBG] *   issuer: C=US,O=Amazon,OU=Server CA 1B,CN=Amazon
2022-06-24T10:22:49.103Z [CURL DBG] > GET / HTTP/1.1
2022-06-24T10:22:49.103Z [CURL DBG] > Host: deja-dup-auto-akiaii34jkwyomiaprqq.s3.amazonaws.com
2022-06-24T10:22:49.103Z [CURL DBG] > User-Agent: s3fs/1.90 (commit hash unknown; GnuTLS(gcrypt))
2022-06-24T10:22:49.103Z [CURL DBG] > Accept: */*
2022-06-24T10:22:49.103Z [CURL DBG] > Authorization: AWS4-HMAC-SHA256 Credential=AKIAJOZLVJUXURG34QGQ/20220624/eu-west-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=c1d8988ec18cc97a870c0f2bccce0c1ee77f08fa83edbde0c884f02584526925
2022-06-24T10:22:49.103Z [CURL DBG] > x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2022-06-24T10:22:49.103Z [CURL DBG] > x-amz-date: 20220624T102248Z
2022-06-24T10:22:49.103Z [CURL DBG] > 
2022-06-24T10:22:49.158Z [CURL DBG] * Mark bundle as not supporting multiuse
2022-06-24T10:22:49.158Z [CURL DBG] < HTTP/1.1 403 Forbidden
2022-06-24T10:22:49.158Z [CURL DBG] < x-amz-bucket-region: eu-west-1
2022-06-24T10:22:49.158Z [CURL DBG] < x-amz-request-id: E2CH22QD1126E1G6
2022-06-24T10:22:49.158Z [CURL DBG] < x-amz-id-2: 1oDwAnY0nPPIK6nlk0shBAUI0wJ4RdD/JI1ReKhXf8nlSmeSvtR5dLhd0pvBCCzpZz0zJ+38pic=
2022-06-24T10:22:49.158Z [CURL DBG] < Content-Type: application/xml
2022-06-24T10:22:49.158Z [CURL DBG] < Transfer-Encoding: chunked
2022-06-24T10:22:49.158Z [CURL DBG] < Date: Fri, 24 Jun 2022 10:22:58 GMT
2022-06-24T10:22:49.158Z [CURL DBG] < Server: AmazonS3
2022-06-24T10:22:49.158Z [CURL DBG] < 
2022-06-24T10:22:49.158Z [CURL DBG] * Connection #0 to host deja-dup-auto-akiaii34jkwyomiaprqq.s3.amazonaws.com left intact
2022-06-24T10:22:49.158Z [CRT] s3fs.cpp:s3fs_check_service(3575): Failed to connect by sigv4, so retry to connect by signature version 2.
2022-06-24T10:22:49.158Z [CURL DBG] * Found bundle for host deja-dup-auto-akiaii34jkwyomiaprqq.s3.amazonaws.com: 0x7ff590001350 [serially]
2022-06-24T10:22:49.158Z [CURL DBG] * Can not multiplex, even if we wanted to!
2022-06-24T10:22:49.158Z [CURL DBG] * Re-using existing connection! (#0) with host deja-dup-auto-akiaii34jkwyomiaprqq.s3.amazonaws.com
2022-06-24T10:22:49.158Z [CURL DBG] * Connected to deja-dup-auto-akiaii34jkwyomiaprqq.s3.amazonaws.com (52.218.98.162) port 443 (#0)
2022-06-24T10:22:49.159Z [CURL DBG] > GET / HTTP/1.1
2022-06-24T10:22:49.159Z [CURL DBG] > Host: deja-dup-auto-akiaii34jkwyomiaprqq.s3.amazonaws.com
2022-06-24T10:22:49.159Z [CURL DBG] > User-Agent: s3fs/1.90 (commit hash unknown; GnuTLS(gcrypt))
2022-06-24T10:22:49.159Z [CURL DBG] > Accept: */*
2022-06-24T10:22:49.159Z [CURL DBG] > Authorization: AWS AKIAJOZLVJUXURG34QGQ:JaktqyL1PBYtBjIALMELWwFDpUU=
2022-06-24T10:22:49.159Z [CURL DBG] > Date: Fri, 24 Jun 2022 10:22:49 GMT
2022-06-24T10:22:49.159Z [CURL DBG] > 
2022-06-24T10:22:49.201Z [CURL DBG] * Mark bundle as not supporting multiuse
2022-06-24T10:22:49.201Z [CURL DBG] < HTTP/1.1 403 Forbidden
2022-06-24T10:22:49.201Z [CURL DBG] < x-amz-bucket-region: eu-west-1
2022-06-24T10:22:49.201Z [CURL DBG] < x-amz-request-id: E2CM77NH498XNGFY
2022-06-24T10:22:49.201Z [CURL DBG] < x-amz-id-2: H5aBTfvnVbQ4yMYQNBrI+3Y5PgWmcGoKKKIclnNxWvKzQsm3KyF6sJJ+a4u29sdORWtkeFoyE4g=
2022-06-24T10:22:49.201Z [CURL DBG] < Content-Type: application/xml
2022-06-24T10:22:49.201Z [CURL DBG] < Transfer-Encoding: chunked
2022-06-24T10:22:49.201Z [CURL DBG] < Date: Fri, 24 Jun 2022 10:22:58 GMT
2022-06-24T10:22:49.201Z [CURL DBG] < Server: AmazonS3
2022-06-24T10:22:49.201Z [CURL DBG] < 
2022-06-24T10:22:49.201Z [CURL DBG] * Connection #0 to host deja-dup-auto-akiaii34jkwyomiaprqq.s3.amazonaws.com left intact
2022-06-24T10:22:49.201Z [CRT] s3fs.cpp:s3fs_check_service(3590): invalid credentials(host=https://s3.amazonaws.com) - result of checking service.
0
┌─(~)─(kyleg@PC1227:pts/3)─┐
└─(11:22:49)──> aws --profile personal s3 ls s3://deja-dup-auto-akiaii34jkwyomiaprqq                                                                                                                                          ──(Fri,Jun24)─┘
                           PRE PC1227/
                           PRE wmlp1000-aws/
                           PRE wmlp1000-lodge/
                           PRE wmlp1000/

Details about issue

When using s3fs 1.90 from Ubuntu 22.04 it fails as above, yet the same AWS access keys work just fine for AWS CLI. It even fails and exits with code 0 (!)

Downgrading to 1.89 from the Ubuntu 21.10 repository results in s3fs working as expected again.

Originally created by @kylegordon on GitHub (Jun 24, 2022). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1968 #### Version of s3fs being used (s3fs --version) version: 1.90 (from Ubuntu 22.04 package repo) #### Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse) Version: 2.9.9-5ubuntu2 #### Kernel information (uname -r) 5.15.0-37-generic #### GNU/Linux Distribution, if applicable (cat /etc/os-release) PRETTY_NAME="Ubuntu 22.04 LTS" NAME="Ubuntu" VERSION_ID="22.04" VERSION="22.04 LTS (Jammy Jellyfish)" VERSION_CODENAME=jammy ID=ubuntu ID_LIKE=debian HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" UBUNTU_CODENAME=jammy #### s3fs command line used, if applicable ``` s3fs -f deja-dup-auto-akiaii34jkwyomiaprqq mnt/s3 -o profile=personal -o endpoint=eu-west-1 -o curldbg ; echo $? ``` #### /etc/fstab entry, if applicable ``` n/a ``` #### s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs) ``` └─(11:21:22)──> s3fs -f deja-dup-auto-akiaii34jkwyomiaprqq mnt/s3 -o profile=personal -o endpoint=eu-west-1 -o curldbg ; echo $? ──(Fri,Jun24)─┘ 2022-06-24T10:22:48.932Z [INF] s3fs version 1.90(unknown) : s3fs -f -o profile=personal -o endpoint=eu-west-1 -o curldbg deja-dup-auto-akiaii34jkwyomiaprqq mnt/s3 2022-06-24T10:22:48.936Z [INF] curl.cpp:InitMimeType(434): Loaded mime information from /etc/mime.types 2022-06-24T10:22:48.939Z [INF] s3fs.cpp:s3fs_init(3382): init v1.90(commit:unknown) with GnuTLS(gcrypt) 2022-06-24T10:22:48.968Z [CURL DBG] * Trying 52.218.98.162:443... 2022-06-24T10:22:48.997Z [CURL DBG] * Connected to deja-dup-auto-akiaii34jkwyomiaprqq.s3.amazonaws.com (52.218.98.162) port 443 (#0) 2022-06-24T10:22:49.036Z [CURL DBG] * found 384 certificates in /etc/ssl/certs 2022-06-24T10:22:49.036Z [CURL DBG] * GnuTLS ciphers: NORMAL:-ARCFOUR-128:-CTYPE-ALL:+CTYPE-X509:-VERS-SSL3.0 2022-06-24T10:22:49.096Z [CURL DBG] * SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256 2022-06-24T10:22:49.102Z [CURL DBG] * server certificate verification OK 2022-06-24T10:22:49.102Z [CURL DBG] * server certificate status verification SKIPPED 2022-06-24T10:22:49.102Z [CURL DBG] * common name: *.s3.amazonaws.com (matched) 2022-06-24T10:22:49.102Z [CURL DBG] * server certificate expiration date OK 2022-06-24T10:22:49.102Z [CURL DBG] * server certificate activation date OK 2022-06-24T10:22:49.102Z [CURL DBG] * certificate public key: RSA 2022-06-24T10:22:49.102Z [CURL DBG] * certificate version: #3 2022-06-24T10:22:49.102Z [CURL DBG] * subject: CN=*.s3.amazonaws.com 2022-06-24T10:22:49.102Z [CURL DBG] * start date: Wed, 15 Dec 2021 00:00:00 GMT 2022-06-24T10:22:49.102Z [CURL DBG] * expire date: Sat, 03 Dec 2022 23:59:59 GMT 2022-06-24T10:22:49.102Z [CURL DBG] * issuer: C=US,O=Amazon,OU=Server CA 1B,CN=Amazon 2022-06-24T10:22:49.103Z [CURL DBG] > GET / HTTP/1.1 2022-06-24T10:22:49.103Z [CURL DBG] > Host: deja-dup-auto-akiaii34jkwyomiaprqq.s3.amazonaws.com 2022-06-24T10:22:49.103Z [CURL DBG] > User-Agent: s3fs/1.90 (commit hash unknown; GnuTLS(gcrypt)) 2022-06-24T10:22:49.103Z [CURL DBG] > Accept: */* 2022-06-24T10:22:49.103Z [CURL DBG] > Authorization: AWS4-HMAC-SHA256 Credential=AKIAJOZLVJUXURG34QGQ/20220624/eu-west-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=c1d8988ec18cc97a870c0f2bccce0c1ee77f08fa83edbde0c884f02584526925 2022-06-24T10:22:49.103Z [CURL DBG] > x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 2022-06-24T10:22:49.103Z [CURL DBG] > x-amz-date: 20220624T102248Z 2022-06-24T10:22:49.103Z [CURL DBG] > 2022-06-24T10:22:49.158Z [CURL DBG] * Mark bundle as not supporting multiuse 2022-06-24T10:22:49.158Z [CURL DBG] < HTTP/1.1 403 Forbidden 2022-06-24T10:22:49.158Z [CURL DBG] < x-amz-bucket-region: eu-west-1 2022-06-24T10:22:49.158Z [CURL DBG] < x-amz-request-id: E2CH22QD1126E1G6 2022-06-24T10:22:49.158Z [CURL DBG] < x-amz-id-2: 1oDwAnY0nPPIK6nlk0shBAUI0wJ4RdD/JI1ReKhXf8nlSmeSvtR5dLhd0pvBCCzpZz0zJ+38pic= 2022-06-24T10:22:49.158Z [CURL DBG] < Content-Type: application/xml 2022-06-24T10:22:49.158Z [CURL DBG] < Transfer-Encoding: chunked 2022-06-24T10:22:49.158Z [CURL DBG] < Date: Fri, 24 Jun 2022 10:22:58 GMT 2022-06-24T10:22:49.158Z [CURL DBG] < Server: AmazonS3 2022-06-24T10:22:49.158Z [CURL DBG] < 2022-06-24T10:22:49.158Z [CURL DBG] * Connection #0 to host deja-dup-auto-akiaii34jkwyomiaprqq.s3.amazonaws.com left intact 2022-06-24T10:22:49.158Z [CRT] s3fs.cpp:s3fs_check_service(3575): Failed to connect by sigv4, so retry to connect by signature version 2. 2022-06-24T10:22:49.158Z [CURL DBG] * Found bundle for host deja-dup-auto-akiaii34jkwyomiaprqq.s3.amazonaws.com: 0x7ff590001350 [serially] 2022-06-24T10:22:49.158Z [CURL DBG] * Can not multiplex, even if we wanted to! 2022-06-24T10:22:49.158Z [CURL DBG] * Re-using existing connection! (#0) with host deja-dup-auto-akiaii34jkwyomiaprqq.s3.amazonaws.com 2022-06-24T10:22:49.158Z [CURL DBG] * Connected to deja-dup-auto-akiaii34jkwyomiaprqq.s3.amazonaws.com (52.218.98.162) port 443 (#0) 2022-06-24T10:22:49.159Z [CURL DBG] > GET / HTTP/1.1 2022-06-24T10:22:49.159Z [CURL DBG] > Host: deja-dup-auto-akiaii34jkwyomiaprqq.s3.amazonaws.com 2022-06-24T10:22:49.159Z [CURL DBG] > User-Agent: s3fs/1.90 (commit hash unknown; GnuTLS(gcrypt)) 2022-06-24T10:22:49.159Z [CURL DBG] > Accept: */* 2022-06-24T10:22:49.159Z [CURL DBG] > Authorization: AWS AKIAJOZLVJUXURG34QGQ:JaktqyL1PBYtBjIALMELWwFDpUU= 2022-06-24T10:22:49.159Z [CURL DBG] > Date: Fri, 24 Jun 2022 10:22:49 GMT 2022-06-24T10:22:49.159Z [CURL DBG] > 2022-06-24T10:22:49.201Z [CURL DBG] * Mark bundle as not supporting multiuse 2022-06-24T10:22:49.201Z [CURL DBG] < HTTP/1.1 403 Forbidden 2022-06-24T10:22:49.201Z [CURL DBG] < x-amz-bucket-region: eu-west-1 2022-06-24T10:22:49.201Z [CURL DBG] < x-amz-request-id: E2CM77NH498XNGFY 2022-06-24T10:22:49.201Z [CURL DBG] < x-amz-id-2: H5aBTfvnVbQ4yMYQNBrI+3Y5PgWmcGoKKKIclnNxWvKzQsm3KyF6sJJ+a4u29sdORWtkeFoyE4g= 2022-06-24T10:22:49.201Z [CURL DBG] < Content-Type: application/xml 2022-06-24T10:22:49.201Z [CURL DBG] < Transfer-Encoding: chunked 2022-06-24T10:22:49.201Z [CURL DBG] < Date: Fri, 24 Jun 2022 10:22:58 GMT 2022-06-24T10:22:49.201Z [CURL DBG] < Server: AmazonS3 2022-06-24T10:22:49.201Z [CURL DBG] < 2022-06-24T10:22:49.201Z [CURL DBG] * Connection #0 to host deja-dup-auto-akiaii34jkwyomiaprqq.s3.amazonaws.com left intact 2022-06-24T10:22:49.201Z [CRT] s3fs.cpp:s3fs_check_service(3590): invalid credentials(host=https://s3.amazonaws.com) - result of checking service. 0 ┌─(~)─(kyleg@PC1227:pts/3)─┐ └─(11:22:49)──> aws --profile personal s3 ls s3://deja-dup-auto-akiaii34jkwyomiaprqq ──(Fri,Jun24)─┘ PRE PC1227/ PRE wmlp1000-aws/ PRE wmlp1000-lodge/ PRE wmlp1000/ ``` ### Details about issue When using s3fs 1.90 from Ubuntu 22.04 it fails as above, yet the same AWS access keys work just fine for AWS CLI. It even fails and exits with code 0 (!) Downgrading to 1.89 from the Ubuntu 21.10 repository results in s3fs working as expected again.
Author
Owner

@gaul commented on GitHub (Sep 8, 2023):

Please test with the latest version 1.93.

<!-- gh-comment-id:1710985642 --> @gaul commented on GitHub (Sep 8, 2023): Please test with the latest version 1.93.
Author
Owner

@odinxux commented on GitHub (Nov 5, 2023):

same problem
some buckets mount ok other failed
odin-ThinkPad-X1-Carbon-6th# 2023-11-05T11:01:20.862934+02:00 odin-ThinkPad-X1-Carbon-6th s3fs[20305]: s3fs version 1.90(unknown) : s3fs -o rw,nosuid,allow_other,use_path_request_style,url=https://s3.amazonaws.com,umask=000,dev curvetechai-global /mnt/global 2023-11-05T11:01:20.866015+02:00 odin-ThinkPad-X1-Carbon-6th s3fs[20305]: Loaded mime information from /etc/mime.types 2023-11-05T11:01:20.869221+02:00 odin-ThinkPad-X1-Carbon-6th s3fs[20306]: init v1.90(commit:unknown) with GnuTLS(gcrypt) 2023-11-05T11:01:22.144250+02:00 odin-ThinkPad-X1-Carbon-6th systemd[1]: mnt-global.mount: Deactivated successfully.
also version 1.93 not available on Ubuntu

<!-- gh-comment-id:1793679113 --> @odinxux commented on GitHub (Nov 5, 2023): same problem some buckets mount ok other failed `odin-ThinkPad-X1-Carbon-6th# 2023-11-05T11:01:20.862934+02:00 odin-ThinkPad-X1-Carbon-6th s3fs[20305]: s3fs version 1.90(unknown) : s3fs -o rw,nosuid,allow_other,use_path_request_style,url=https://s3.amazonaws.com,umask=000,dev curvetechai-global /mnt/global 2023-11-05T11:01:20.866015+02:00 odin-ThinkPad-X1-Carbon-6th s3fs[20305]: Loaded mime information from /etc/mime.types 2023-11-05T11:01:20.869221+02:00 odin-ThinkPad-X1-Carbon-6th s3fs[20306]: init v1.90(commit:unknown) with GnuTLS(gcrypt) 2023-11-05T11:01:22.144250+02:00 odin-ThinkPad-X1-Carbon-6th systemd[1]: mnt-global.mount: Deactivated successfully. ` also version 1.93 not available on Ubuntu
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#995
No description provided.