[GH-ISSUE #2217] s3fs: unable to access MOUNTPOINT {bucketname}:/directory-path/ #1124

Closed
opened 2026-03-04 01:51:34 +03:00 by kerem · 4 comments
Owner

Originally created by @haytham-mostafa on GitHub (Jul 13, 2023).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2217

Additional Information

Version of s3fs being used (s3fs --version)

Amazon Simple Storage Service File System V1.92 (commit:unknown) with OpenSSL
Copyright (C) 2010 Randy Rizun <rrizun@gmail.com>
License GPL2: GNU GPL version 2 <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Kernel information (uname -r)

3.10.0-1160.92.1.el7.x86_64

GNU/Linux Distribution, if applicable (cat /etc/os-release)

NAME="Red Hat Enterprise Linux Server"
VERSION="7.9 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.9"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.9 (Maipo)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.9:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.9
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.9"

s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)

Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: >
Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: < HTTP/1.1 404 Not Found
Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: < x-amz-request-id: 6R52BYC6GK28ZASJ
Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: < x-amz-id-2: +sYMpldZQXEBCpRdr66odnSGPUfQhWfLZlCq8nIZob8mSCEKKCbZdMuQqQ4y4O+5kNLWs9v/rsU=
Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: < Content-Type: application/xml
Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: < Transfer-Encoding: chunked
Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: < Date: Thu, 13 Jul 2023 15:11:29 GMT
Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: < Server: AmazonS3
Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: <
Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: * Connection #4 to host iotbigdatarawfilesft-lit.s3.amazonaws.com left intact
Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]:      HTTP response code 404 was returned, returning ENOENT
Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: curl.cpp:CheckBucket(3741): Check bucket failed, S3 response: <?xml version="1.0" encoding="UTF-8"?>#012<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>InputFiles/OCOMC-mnt/</Key><RequestId>6R52BYC6GK28ZASJ</RequestId><HostId>+sYMpldZQXEBCpRdr66odnSGPUfQhWfLZlCq8nIZob8mSCEKKCbZdMuQqQ4y4O+5kNLWs9v/rsU=</HostId></Error>
Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: s3fs.cpp:s3fs_check_service(4443): Failed to check bucket and directory for mount point : Bucket or directory(/InputFiles/OCOMC-mnt) not found(host=https://s3.amazonaws.com) - You may need to specify the compat_dir option.
Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: s3fs.cpp:s3fs_exit_fuseloop(4200): Exiting FUSE event loop due to errors
Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: destroy
Jul 13 15:11:30 ip-10-201-34-100 kernel: s3fs[147241]: segfault at 7f2de21d19d0 ip 00007f2de61dcf81 sp 00007fffd1587a10 error 4 in libpthread-2.17.so[7f2de61d4000+17000]
Jul 13 15:19:47 ip-10-201-34-100 s3fs[148455]: s3fs version 1.92(unknown) : s3fs -o iam_role=auto -o url=https://s3.amazonaws.com -o endpoint=eu-central-1 -o dbglevel=info -o curldbg -o allow_other -o use_cache=/tmp -o mp_umask=002 -o multireq_max=5 iotbigdatarawfilesft-lit:/InputFiles/OCOMC-mnt/ /opt/SP/users/oracle/data/input/
Jul 13 15:19:47 ip-10-201-34-100 s3fs[148455]: s3fs_logger.cpp:LowSetLogLevel(239): change debug level from [CRT] to [INF]
Jul 13 15:19:47 ip-10-201-34-100 s3fs[148455]:    PROC(uid=1002, gid=1013) - MountPoint(uid=1002, gid=1013, mode=40755)
Jul 13 15:19:47 ip-10-201-34-100 s3fs[148455]: Loaded mime information from /etc/mime.types
Jul 13 15:19:47 ip-10-201-34-100 s3fs[148455]: Any threads are running now, then nothing to do.
Jul 13 15:19:47 ip-10-201-34-100 s3fs[148455]:      Start worker thread in ThreadPoolMan.
Jul 13 15:19:47 ip-10-201-34-100 s3fs[148455]:      Start worker thread in ThreadPoolMan.
Jul 13 15:19:47 ip-10-201-34-100 s3fs[148455]:      Start worker thread in ThreadPoolMan.
Jul 13 15:19:47 ip-10-201-34-100 s3fs[148455]:      Start worker thread in ThreadPoolMan.
Jul 13 15:19:47 ip-10-201-34-100 s3fs[148455]:      Start worker thread in ThreadPoolMan.
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148517]: s3fs version 1.92(unknown) : s3fs -o iam_role=auto -o url=https://s3.amazonaws.com -o endpoint=eu-central-1 -o dbglevel=info -o curldbg -o allow_other -o use_cache=/tmp -o mp_umask=002 -o multireq_max=5 iotbigdatarawfilesft-lit:/InputFiles/OCOMC-mnt/ /opt/SP/users/oracle/data/input/
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148517]: s3fs_logger.cpp:LowSetLogLevel(239): change debug level from [CRT] to [INF]
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148517]:    PROC(uid=1002, gid=1013) - MountPoint(uid=1002, gid=1013, mode=40755)
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148517]: Loaded mime information from /etc/mime.types
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148517]: Any threads are running now, then nothing to do.
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148517]:      Start worker thread in ThreadPoolMan.
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148517]:      Start worker thread in ThreadPoolMan.
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148517]:      Start worker thread in ThreadPoolMan.
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148517]:      Start worker thread in ThreadPoolMan.
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: s3fs_cred.cpp:VersionS3fsCredential(60): Check why built-in function was called, the external credential library must have VersionS3fsCredential function.
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: init v1.92(commit:unknown) with OpenSSL, credential-library(built-in)
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: curl.cpp:ResetHandle(1974): The CURLOPT_SSL_ENABLE_ALPN option could not be unset. S3 server does not support ALPN, then this option should be disabled to maximize performance. you need to use libcurl 7.36.0 or later.
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: curl.cpp:ResetHandle(1977): The S3FS_CURLOPT_KEEP_SENDING_ON_ERROR option could not be set. For maximize performance you need to enable this option and you should use libcurl 7.51.0 or later.
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * About to connect() to 169.254.169.254 port 80 (#0)
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: *   Trying 169.254.169.254...
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Connected to 169.254.169.254 (169.254.169.254) port 80 (#0)
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > PUT /latest/api/token HTTP/1.1
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > User-Agent: s3fs/1.92 (commit hash unknown; OpenSSL)
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Host: 169.254.169.254
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Accept: */*
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > X-aws-ec2-metadata-token-ttl-seconds: 21600
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Content-Length: 0
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: >
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < HTTP/1.1 200 OK
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < X-Aws-Ec2-Metadata-Token-Ttl-Seconds: 21600
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Content-Length: 56
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Date: Thu, 13 Jul 2023 15:20:10 GMT
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Server: EC2ws
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Connection: close
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Content-Type: text/plain
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: <
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Closing connection 0
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]:      HTTP response code 200
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]:      Setting AWS IMDSv2 API token to AQAEAKujJ6DQobuYrz8VKiJPYnHovJQza9nQiVoIB48jpA9Ymk9w7A==
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]:      Get IAM Role name
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * About to connect() to 169.254.169.254 port 80 (#1)
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: *   Trying 169.254.169.254...
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Connected to 169.254.169.254 (169.254.169.254) port 80 (#1)
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > GET /latest/meta-data/iam/security-credentials/ HTTP/1.1
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > User-Agent: s3fs/1.92 (commit hash unknown; OpenSSL)
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Host: 169.254.169.254
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Accept: */*
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > X-aws-ec2-metadata-token: AQAEAKujJ6DQobuYrz8VKiJPYnHovJQza9nQiVoIB48jpA9Ymk9w7A==
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: >
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < HTTP/1.1 200 OK
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < X-Aws-Ec2-Metadata-Token-Ttl-Seconds: 21600
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Content-Type: text/plain
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Accept-Ranges: none
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Last-Modified: Thu, 13 Jul 2023 14:31:04 GMT
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Content-Length: 19
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Date: Thu, 13 Jul 2023 15:20:10 GMT
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Server: EC2ws
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Connection: close
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: <
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Closing connection 1
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]:      HTTP response code 200
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]:      IAM role name response = "mediation-ocomc-lit"
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: loaded IAM role name = mediation-ocomc-lit
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: check services.
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: IAM Access Token refreshing...
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]:      [IAM role=mediation-ocomc-lit]
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * About to connect() to 169.254.169.254 port 80 (#2)
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: *   Trying 169.254.169.254...
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Connected to 169.254.169.254 (169.254.169.254) port 80 (#2)
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > PUT /latest/api/token HTTP/1.1
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > User-Agent: s3fs/1.92 (commit hash unknown; OpenSSL)
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Host: 169.254.169.254
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Accept: */*
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > X-aws-ec2-metadata-token-ttl-seconds: 21600
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Content-Length: 0
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: >
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < HTTP/1.1 200 OK
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < X-Aws-Ec2-Metadata-Token-Ttl-Seconds: 21600
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Content-Length: 56
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Date: Thu, 13 Jul 2023 15:20:10 GMT
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Server: EC2ws
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Connection: close
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Content-Type: text/plain
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: <
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Closing connection 2
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]:      HTTP response code 200
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]:      Setting AWS IMDSv2 API token to AQAEAKujJ6BDtIo7a9bDouzIPGXlTsLLfSJ1RTOwsSKtthG7FftqVQ==
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * About to connect() to 169.254.169.254 port 80 (#3)
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: *   Trying 169.254.169.254...
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Connected to 169.254.169.254 (169.254.169.254) port 80 (#3)
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > GET /latest/meta-data/iam/security-credentials/mediation-ocomc-lit HTTP/1.1
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > User-Agent: s3fs/1.92 (commit hash unknown; OpenSSL)
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Host: 169.254.169.254
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Accept: */*
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > X-aws-ec2-metadata-token: AQAEAKujJ6BDtIo7a9bDouzIPGXlTsLLfSJ1RTOwsSKtthG7FftqVQ==
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: >
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < HTTP/1.1 200 OK
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < X-Aws-Ec2-Metadata-Token-Ttl-Seconds: 21600
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Content-Type: text/plain
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Accept-Ranges: none
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Last-Modified: Thu, 13 Jul 2023 14:31:04 GMT
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Content-Length: 1602
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Date: Thu, 13 Jul 2023 15:20:10 GMT
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Server: EC2ws
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Connection: close
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: <
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Closing connection 3
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]:      HTTP response code 200
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]:      IAM credential response = "{#012  "Code" : "Success",#012  "LastUpdated" : "2023-07-13T14:31:42Z",#012  "Type" : "AWS-HMAC",#012  "AccessKeyId" : "ASIA6GDLTVMB5SFOECR5",#012  "SecretAccessKey" : "kaylEXYONj3Zg89f6KN4+cbNmfuMK3J5/FSLozQA",#012  "Token" : "IQoJb3JpZ2luX2VjEHcaDGV1LWNlbnRyYWwtMSJGMEQCIHY/Fb8yk/UxAw4LoWl9Vbs3ZNA1EV4sd69x7fJZt/jHAiBug+sm9HYR/33zHx+SXunYh/kHnlrIJv2WePL+KkFaBSrPBQjw//////////8BEAIaDDk3NTE4MzI2MDQxOSIM6BA3p0S01D68ejxbKqMFYKhaBWa3FfzDVC/rmRYkdTWj8/4jcEtXTxVJ4TvsB7QhHncadfExb4TNohtDWNSehhw9HSXzv0VFqKVnYBICg+/1NYvHjlNrB5TE7W6mNMNc1HuU3ZL1Ke6DvX76hnnoIr8bC4DqZ15oRJwQ832k2QuciJlSE7370fcXpj9yqzIdqcYzfrslzj2JRBIEgSstPyMC3PoAVJ/BjqK51iVkpL25dKOqigcWUWp9k3kVKPy3ohXezds8VGOwCAvGwyG1LCTKIHzHJ08KZ//E6llnO+F5aKb1/0DaYNW3NbnYEpLxc7XZQ5FtkrCXuaCy3aWTza79k0q8ACz+88SdIH+G71dQcd2FC1fo/VLp7V2lSBjpuEiP23ck8s+a+eHI2agHsjfrSYDxF+B/6jrrC3caCfydh/NAaueziPp4hPwUsUj5KkDMGhonTNQRPqH/huwLnAY93Bt3Fb6SZhRpG6xP798HSA3PrXrJPZOMZ3UcT7nf1VzB6slks+bW37/+7zq2vNJt/au9ZVyNg/7EJ5XV8SCYdlX320KDtksHd+oC/uRudcGo/hbpw6e6F2RkVcbw5aHvJFM0/YvApzI5Ph1eGdpbUMK3hqPoPA4Wek0KXjOmssegHbELsw269z/IMBlMFtL6N+Uiot6JhkGhwiSyWudJaizO2EkYGsQ3ytRuOphWzTVNfSDnnM2XySr5eK9/O4FBPLlgiFBPwKnj3A0MkOWn6u2nKCvfU6DriIdMLbmj3KNUgzso80RUzGgqMwpn0z9kcA6BOOOl6miPDR1xT851vEENasehOGJDfaKmpsIkc2PoeoUZzya5QQirTTYPHHyheo1d5rWjMZ1qqbSwBOfYfutsPwa5hsgnlCeoj5CkFVcvuNboQwCdDSe4dPpYavERMKiVwKUGOrIBz8v6q1vCkERkbGOFMAb0tWYWG0gGKrltFD032k5oFc38Pc42fNSR8c8WWMlVjGpJx4nz3wa6v6XFyJ5VIyZgERy4E/LiU7lfQBbrpwMgOyYSFZPqYBLnJJeH0h6ds7a1clSTdZ4hiB6in76Ne6bVDE4xU8pp//D63gzN4Ka5FLYaodb8RCWxQ4616v1m/Z7VxkBN5ojnvsh5TzUD9aAkHidl9n19TT8buz2DcOyu6Ks7WQ==",#012  "Expiration" : "2023-07-13T20:34:10Z"#012}"
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: IAM Access Token refreshed
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]:      check a bucket path(/InputFiles/OCOMC-mnt/).
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]:      URL is https://s3.amazonaws.com/iotbigdatarawfilesft-lit/InputFiles/OCOMC-mnt/
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]:      URL changed is https://iotbigdatarawfilesft-lit.s3.amazonaws.com/InputFiles/OCOMC-mnt/
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]:      computing signature [GET] [/InputFiles/OCOMC-mnt/] [] []
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]:      url is https://s3.amazonaws.com
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * About to connect() to iotbigdatarawfilesft-lit.s3.amazonaws.com port 443 (#4)
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: *   Trying 52.219.72.125...
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Connected to iotbigdatarawfilesft-lit.s3.amazonaws.com (52.219.72.125) port 443 (#4)
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Initializing NSS with certpath: sql:/etc/pki/nssdb
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: *   CAfile: /etc/pki/tls/certs/ca-bundle.crt#012  CApath: none
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Server certificate:
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: *     subject: CN=*.s3.amazonaws.com
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: *     start date: Mar 21 00:00:00 2023 GMT
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: *     expire date: Dec 19 23:59:59 2023 GMT
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: *     common name: *.s3.amazonaws.com
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: *     issuer: CN=Amazon RSA 2048 M01,O=Amazon,C=US
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > GET /InputFiles/OCOMC-mnt/ HTTP/1.1
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > User-Agent: s3fs/1.92 (commit hash unknown; OpenSSL)
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Accept: */*
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Authorization: AWS4-HMAC-SHA256 Credential=ASIA6GDLTVMB5SFOECR5/20230713/eu-central-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=f2522f9dc3e54fe5fdc97fd51b65f308664b0da452fe0f414724baf5222a8f47
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > host: iotbigdatarawfilesft-lit.s3.amazonaws.com
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > x-amz-date: 20230713T152010Z
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > x-amz-security-token: IQoJb3JpZ2luX2VjEHcaDGV1LWNlbnRyYWwtMSJGMEQCIHY/Fb8yk/UxAw4LoWl9Vbs3ZNA1EV4sd69x7fJZt/jHAiBug+sm9HYR/33zHx+SXunYh/kHnlrIJv2WePL+KkFaBSrPBQjw//////////8BEAIaDDk3NTE4MzI2MDQxOSIM6BA3p0S01D68ejxbKqMFYKhaBWa3FfzDVC/rmRYkdTWj8/4jcEtXTxVJ4TvsB7QhHncadfExb4TNohtDWNSehhw9HSXzv0VFqKVnYBICg+/1NYvHjlNrB5TE7W6mNMNc1HuU3ZL1Ke6DvX76hnnoIr8bC4DqZ15oRJwQ832k2QuciJlSE7370fcXpj9yqzIdqcYzfrslzj2JRBIEgSstPyMC3PoAVJ/BjqK51iVkpL25dKOqigcWUWp9k3kVKPy3ohXezds8VGOwCAvGwyG1LCTKIHzHJ08KZ//E6llnO+F5aKb1/0DaYNW3NbnYEpLxc7XZQ5FtkrCXuaCy3aWTza79k0q8ACz+88SdIH+G71dQcd2FC1fo/VLp7V2lSBjpuEiP23ck8s+a+eHI2agHsjfrSYDxF+B/6jrrC3caCfydh/NAaueziPp4hPwUsUj5KkDMGhonTNQRPqH/huwLnAY93Bt3Fb6SZhRpG6xP798HSA3PrXrJPZOMZ3UcT7nf1VzB6slks+bW37/+7zq2vNJt/au9ZVyNg/7EJ5XV8SCYdlX320KDtksHd+oC/uRudcGo/hbpw6e6F2RkVcbw5aHvJFM0/YvApzI5Ph1eGdpbUMK3hqPoPA4Wek0KXjOmssegHbELsw269z/IMBlMFtL6N+Uiot6JhkGhwiSyWudJaizO2EkYGsQ3ytRuOphWzTVNfSDnnM2XySr5eK9/O4FBPLlgiFBPwKnj3A0MkOWn6u2nKCvfU6DriIdMLbmj3KNUgzso80RUzGgqMwpn0z9kcA6BOOOl6miPDR1xT851vEENasehOGJDfaKmpsIkc2PoeoUZzya5QQirTTYPHHyheo1d5rWjMZ1qqbSwBOfYfutsPwa5hsgnlCeoj5CkFVcvuNboQwCdDSe4dPpYavERMKiVwKUGOrIBz8v6q1vCkERkbGOFMAb0tWYWG0gGKrltFD032k5oFc38Pc42fNSR8c8WWMlVjGpJx4nz3wa6v6XFyJ5VIyZgERy4E/LiU7lfQBbrpwMgOyYSFZPqYBLnJJeH0h6ds7a1clSTdZ4hiB6in76Ne6bVDE4xU8pp//D63gzN4Ka5FLYaodb8RCWxQ4616v1m/Z7VxkBN5ojnvsh5TzUD9aAkHidl9n19TT8buz2DcOyu6Ks7WQ==
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: >
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < HTTP/1.1 404 Not Found
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < x-amz-request-id: 1E2ZR6WMF5ZDB80Y
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < x-amz-id-2: srOoAvgVjmsPlUoF1SIGvmnzzkmfzzVnlGnRa6N12Iyy+/Du5K9AeUq9UOaTKgtq6voyjMOfoEY=
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Content-Type: application/xml
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Transfer-Encoding: chunked
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Date: Thu, 13 Jul 2023 15:20:10 GMT
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Server: AmazonS3
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: <
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Connection #4 to host iotbigdatarawfilesft-lit.s3.amazonaws.com left intact
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]:      HTTP response code 404 was returned, returning ENOENT
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: curl.cpp:CheckBucket(3741): Check bucket failed, S3 response: <?xml version="1.0" encoding="UTF-8"?>#012<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>InputFiles/OCOMC-mnt/</Key><RequestId>1E2ZR6WMF5ZDB80Y</RequestId><HostId>srOoAvgVjmsPlUoF1SIGvmnzzkmfzzVnlGnRa6N12Iyy+/Du5K9AeUq9UOaTKgtq6voyjMOfoEY=</HostId></Error>
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: s3fs.cpp:s3fs_check_service(4443): Failed to check bucket and directory for mount point : Bucket or directory(/InputFiles/OCOMC-mnt) not found(host=https://s3.amazonaws.com) - You may need to specify the compat_dir option.
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: s3fs.cpp:s3fs_exit_fuseloop(4200): Exiting FUSE event loop due to errors
Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: destroy
Jul 13 15:20:10 ip-10-201-34-100 kernel: s3fs[148525]: segfault at 7f40732479d0 ip 00007f4077252f81 sp 00007ffc5ac05090 error 4 in libpthread-2.17.so[7f407724a000+17000]

Details about issue

I am trying the following command to mount the s3 bucket but with no success:
s3fs -o iam_role=auto -o url=https://s3.amazonaws.com -o endpoint=eu-central-1 -o dbglevel=info -o curldbg -o allow_other -o use_cache=/tmp -o mp_umask=002 -o multireq_max=5 iotbigdatarawfilesft-lit:/InputFiles/OCOMC-mnt/ /opt/SP/users/oracle/data/input/

I have tried a lot of fixes but with no success, actually I don't know what's the issue.
Please advise

Thanks & BR,
Haytham

Originally created by @haytham-mostafa on GitHub (Jul 13, 2023). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2217 <!-- -------------------------------------------------------------------------- The following information is very important in order to help us to help you. Omission of the following details may delay your support request or receive no attention at all. Keep in mind that the commands we provide to retrieve information are oriented to GNU/Linux Distributions, so you could need to use others if you use s3fs on macOS or BSD. --------------------------------------------------------------------------- --> ### Additional Information #### Version of s3fs being used (`s3fs --version`) ``` Amazon Simple Storage Service File System V1.92 (commit:unknown) with OpenSSL Copyright (C) 2010 Randy Rizun <rrizun@gmail.com> License GPL2: GNU GPL version 2 <https://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. ``` #### Kernel information (`uname -r`) `3.10.0-1160.92.1.el7.x86_64` #### GNU/Linux Distribution, if applicable (`cat /etc/os-release`) ``` NAME="Red Hat Enterprise Linux Server" VERSION="7.9 (Maipo)" ID="rhel" ID_LIKE="fedora" VARIANT="Server" VARIANT_ID="server" VERSION_ID="7.9" PRETTY_NAME="Red Hat Enterprise Linux Server 7.9 (Maipo)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:redhat:enterprise_linux:7.9:GA:server" HOME_URL="https://www.redhat.com/" BUG_REPORT_URL="https://bugzilla.redhat.com/" REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7" REDHAT_BUGZILLA_PRODUCT_VERSION=7.9 REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" REDHAT_SUPPORT_PRODUCT_VERSION="7.9" ``` #### s3fs syslog messages (`grep s3fs /var/log/syslog`, `journalctl | grep s3fs`, or `s3fs outputs`) ``` Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: > Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: < HTTP/1.1 404 Not Found Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: < x-amz-request-id: 6R52BYC6GK28ZASJ Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: < x-amz-id-2: +sYMpldZQXEBCpRdr66odnSGPUfQhWfLZlCq8nIZob8mSCEKKCbZdMuQqQ4y4O+5kNLWs9v/rsU= Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: < Content-Type: application/xml Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: < Transfer-Encoding: chunked Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: < Date: Thu, 13 Jul 2023 15:11:29 GMT Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: < Server: AmazonS3 Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: < Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: * Connection #4 to host iotbigdatarawfilesft-lit.s3.amazonaws.com left intact Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: HTTP response code 404 was returned, returning ENOENT Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: curl.cpp:CheckBucket(3741): Check bucket failed, S3 response: <?xml version="1.0" encoding="UTF-8"?>#012<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>InputFiles/OCOMC-mnt/</Key><RequestId>6R52BYC6GK28ZASJ</RequestId><HostId>+sYMpldZQXEBCpRdr66odnSGPUfQhWfLZlCq8nIZob8mSCEKKCbZdMuQqQ4y4O+5kNLWs9v/rsU=</HostId></Error> Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: s3fs.cpp:s3fs_check_service(4443): Failed to check bucket and directory for mount point : Bucket or directory(/InputFiles/OCOMC-mnt) not found(host=https://s3.amazonaws.com) - You may need to specify the compat_dir option. Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: s3fs.cpp:s3fs_exit_fuseloop(4200): Exiting FUSE event loop due to errors Jul 13 15:11:30 ip-10-201-34-100 s3fs[147241]: destroy Jul 13 15:11:30 ip-10-201-34-100 kernel: s3fs[147241]: segfault at 7f2de21d19d0 ip 00007f2de61dcf81 sp 00007fffd1587a10 error 4 in libpthread-2.17.so[7f2de61d4000+17000] Jul 13 15:19:47 ip-10-201-34-100 s3fs[148455]: s3fs version 1.92(unknown) : s3fs -o iam_role=auto -o url=https://s3.amazonaws.com -o endpoint=eu-central-1 -o dbglevel=info -o curldbg -o allow_other -o use_cache=/tmp -o mp_umask=002 -o multireq_max=5 iotbigdatarawfilesft-lit:/InputFiles/OCOMC-mnt/ /opt/SP/users/oracle/data/input/ Jul 13 15:19:47 ip-10-201-34-100 s3fs[148455]: s3fs_logger.cpp:LowSetLogLevel(239): change debug level from [CRT] to [INF] Jul 13 15:19:47 ip-10-201-34-100 s3fs[148455]: PROC(uid=1002, gid=1013) - MountPoint(uid=1002, gid=1013, mode=40755) Jul 13 15:19:47 ip-10-201-34-100 s3fs[148455]: Loaded mime information from /etc/mime.types Jul 13 15:19:47 ip-10-201-34-100 s3fs[148455]: Any threads are running now, then nothing to do. Jul 13 15:19:47 ip-10-201-34-100 s3fs[148455]: Start worker thread in ThreadPoolMan. Jul 13 15:19:47 ip-10-201-34-100 s3fs[148455]: Start worker thread in ThreadPoolMan. Jul 13 15:19:47 ip-10-201-34-100 s3fs[148455]: Start worker thread in ThreadPoolMan. Jul 13 15:19:47 ip-10-201-34-100 s3fs[148455]: Start worker thread in ThreadPoolMan. Jul 13 15:19:47 ip-10-201-34-100 s3fs[148455]: Start worker thread in ThreadPoolMan. Jul 13 15:20:10 ip-10-201-34-100 s3fs[148517]: s3fs version 1.92(unknown) : s3fs -o iam_role=auto -o url=https://s3.amazonaws.com -o endpoint=eu-central-1 -o dbglevel=info -o curldbg -o allow_other -o use_cache=/tmp -o mp_umask=002 -o multireq_max=5 iotbigdatarawfilesft-lit:/InputFiles/OCOMC-mnt/ /opt/SP/users/oracle/data/input/ Jul 13 15:20:10 ip-10-201-34-100 s3fs[148517]: s3fs_logger.cpp:LowSetLogLevel(239): change debug level from [CRT] to [INF] Jul 13 15:20:10 ip-10-201-34-100 s3fs[148517]: PROC(uid=1002, gid=1013) - MountPoint(uid=1002, gid=1013, mode=40755) Jul 13 15:20:10 ip-10-201-34-100 s3fs[148517]: Loaded mime information from /etc/mime.types Jul 13 15:20:10 ip-10-201-34-100 s3fs[148517]: Any threads are running now, then nothing to do. Jul 13 15:20:10 ip-10-201-34-100 s3fs[148517]: Start worker thread in ThreadPoolMan. Jul 13 15:20:10 ip-10-201-34-100 s3fs[148517]: Start worker thread in ThreadPoolMan. Jul 13 15:20:10 ip-10-201-34-100 s3fs[148517]: Start worker thread in ThreadPoolMan. Jul 13 15:20:10 ip-10-201-34-100 s3fs[148517]: Start worker thread in ThreadPoolMan. Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: s3fs_cred.cpp:VersionS3fsCredential(60): Check why built-in function was called, the external credential library must have VersionS3fsCredential function. Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: init v1.92(commit:unknown) with OpenSSL, credential-library(built-in) Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: curl.cpp:ResetHandle(1974): The CURLOPT_SSL_ENABLE_ALPN option could not be unset. S3 server does not support ALPN, then this option should be disabled to maximize performance. you need to use libcurl 7.36.0 or later. Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: curl.cpp:ResetHandle(1977): The S3FS_CURLOPT_KEEP_SENDING_ON_ERROR option could not be set. For maximize performance you need to enable this option and you should use libcurl 7.51.0 or later. Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * About to connect() to 169.254.169.254 port 80 (#0) Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Trying 169.254.169.254... Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Connected to 169.254.169.254 (169.254.169.254) port 80 (#0) Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > PUT /latest/api/token HTTP/1.1 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > User-Agent: s3fs/1.92 (commit hash unknown; OpenSSL) Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Host: 169.254.169.254 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Accept: */* Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > X-aws-ec2-metadata-token-ttl-seconds: 21600 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Content-Length: 0 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < HTTP/1.1 200 OK Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < X-Aws-Ec2-Metadata-Token-Ttl-Seconds: 21600 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Content-Length: 56 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Date: Thu, 13 Jul 2023 15:20:10 GMT Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Server: EC2ws Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Connection: close Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Content-Type: text/plain Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Closing connection 0 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: HTTP response code 200 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: Setting AWS IMDSv2 API token to AQAEAKujJ6DQobuYrz8VKiJPYnHovJQza9nQiVoIB48jpA9Ymk9w7A== Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: Get IAM Role name Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * About to connect() to 169.254.169.254 port 80 (#1) Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Trying 169.254.169.254... Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Connected to 169.254.169.254 (169.254.169.254) port 80 (#1) Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > GET /latest/meta-data/iam/security-credentials/ HTTP/1.1 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > User-Agent: s3fs/1.92 (commit hash unknown; OpenSSL) Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Host: 169.254.169.254 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Accept: */* Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > X-aws-ec2-metadata-token: AQAEAKujJ6DQobuYrz8VKiJPYnHovJQza9nQiVoIB48jpA9Ymk9w7A== Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < HTTP/1.1 200 OK Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < X-Aws-Ec2-Metadata-Token-Ttl-Seconds: 21600 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Content-Type: text/plain Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Accept-Ranges: none Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Last-Modified: Thu, 13 Jul 2023 14:31:04 GMT Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Content-Length: 19 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Date: Thu, 13 Jul 2023 15:20:10 GMT Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Server: EC2ws Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Connection: close Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Closing connection 1 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: HTTP response code 200 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: IAM role name response = "mediation-ocomc-lit" Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: loaded IAM role name = mediation-ocomc-lit Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: check services. Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: IAM Access Token refreshing... Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: [IAM role=mediation-ocomc-lit] Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * About to connect() to 169.254.169.254 port 80 (#2) Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Trying 169.254.169.254... Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Connected to 169.254.169.254 (169.254.169.254) port 80 (#2) Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > PUT /latest/api/token HTTP/1.1 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > User-Agent: s3fs/1.92 (commit hash unknown; OpenSSL) Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Host: 169.254.169.254 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Accept: */* Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > X-aws-ec2-metadata-token-ttl-seconds: 21600 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Content-Length: 0 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < HTTP/1.1 200 OK Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < X-Aws-Ec2-Metadata-Token-Ttl-Seconds: 21600 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Content-Length: 56 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Date: Thu, 13 Jul 2023 15:20:10 GMT Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Server: EC2ws Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Connection: close Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Content-Type: text/plain Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Closing connection 2 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: HTTP response code 200 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: Setting AWS IMDSv2 API token to AQAEAKujJ6BDtIo7a9bDouzIPGXlTsLLfSJ1RTOwsSKtthG7FftqVQ== Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * About to connect() to 169.254.169.254 port 80 (#3) Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Trying 169.254.169.254... Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Connected to 169.254.169.254 (169.254.169.254) port 80 (#3) Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > GET /latest/meta-data/iam/security-credentials/mediation-ocomc-lit HTTP/1.1 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > User-Agent: s3fs/1.92 (commit hash unknown; OpenSSL) Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Host: 169.254.169.254 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Accept: */* Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > X-aws-ec2-metadata-token: AQAEAKujJ6BDtIo7a9bDouzIPGXlTsLLfSJ1RTOwsSKtthG7FftqVQ== Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < HTTP/1.1 200 OK Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < X-Aws-Ec2-Metadata-Token-Ttl-Seconds: 21600 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Content-Type: text/plain Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Accept-Ranges: none Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Last-Modified: Thu, 13 Jul 2023 14:31:04 GMT Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Content-Length: 1602 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Date: Thu, 13 Jul 2023 15:20:10 GMT Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Server: EC2ws Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Connection: close Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Closing connection 3 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: HTTP response code 200 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: IAM credential response = "{#012 "Code" : "Success",#012 "LastUpdated" : "2023-07-13T14:31:42Z",#012 "Type" : "AWS-HMAC",#012 "AccessKeyId" : "ASIA6GDLTVMB5SFOECR5",#012 "SecretAccessKey" : "kaylEXYONj3Zg89f6KN4+cbNmfuMK3J5/FSLozQA",#012 "Token" : "IQoJb3JpZ2luX2VjEHcaDGV1LWNlbnRyYWwtMSJGMEQCIHY/Fb8yk/UxAw4LoWl9Vbs3ZNA1EV4sd69x7fJZt/jHAiBug+sm9HYR/33zHx+SXunYh/kHnlrIJv2WePL+KkFaBSrPBQjw//////////8BEAIaDDk3NTE4MzI2MDQxOSIM6BA3p0S01D68ejxbKqMFYKhaBWa3FfzDVC/rmRYkdTWj8/4jcEtXTxVJ4TvsB7QhHncadfExb4TNohtDWNSehhw9HSXzv0VFqKVnYBICg+/1NYvHjlNrB5TE7W6mNMNc1HuU3ZL1Ke6DvX76hnnoIr8bC4DqZ15oRJwQ832k2QuciJlSE7370fcXpj9yqzIdqcYzfrslzj2JRBIEgSstPyMC3PoAVJ/BjqK51iVkpL25dKOqigcWUWp9k3kVKPy3ohXezds8VGOwCAvGwyG1LCTKIHzHJ08KZ//E6llnO+F5aKb1/0DaYNW3NbnYEpLxc7XZQ5FtkrCXuaCy3aWTza79k0q8ACz+88SdIH+G71dQcd2FC1fo/VLp7V2lSBjpuEiP23ck8s+a+eHI2agHsjfrSYDxF+B/6jrrC3caCfydh/NAaueziPp4hPwUsUj5KkDMGhonTNQRPqH/huwLnAY93Bt3Fb6SZhRpG6xP798HSA3PrXrJPZOMZ3UcT7nf1VzB6slks+bW37/+7zq2vNJt/au9ZVyNg/7EJ5XV8SCYdlX320KDtksHd+oC/uRudcGo/hbpw6e6F2RkVcbw5aHvJFM0/YvApzI5Ph1eGdpbUMK3hqPoPA4Wek0KXjOmssegHbELsw269z/IMBlMFtL6N+Uiot6JhkGhwiSyWudJaizO2EkYGsQ3ytRuOphWzTVNfSDnnM2XySr5eK9/O4FBPLlgiFBPwKnj3A0MkOWn6u2nKCvfU6DriIdMLbmj3KNUgzso80RUzGgqMwpn0z9kcA6BOOOl6miPDR1xT851vEENasehOGJDfaKmpsIkc2PoeoUZzya5QQirTTYPHHyheo1d5rWjMZ1qqbSwBOfYfutsPwa5hsgnlCeoj5CkFVcvuNboQwCdDSe4dPpYavERMKiVwKUGOrIBz8v6q1vCkERkbGOFMAb0tWYWG0gGKrltFD032k5oFc38Pc42fNSR8c8WWMlVjGpJx4nz3wa6v6XFyJ5VIyZgERy4E/LiU7lfQBbrpwMgOyYSFZPqYBLnJJeH0h6ds7a1clSTdZ4hiB6in76Ne6bVDE4xU8pp//D63gzN4Ka5FLYaodb8RCWxQ4616v1m/Z7VxkBN5ojnvsh5TzUD9aAkHidl9n19TT8buz2DcOyu6Ks7WQ==",#012 "Expiration" : "2023-07-13T20:34:10Z"#012}" Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: IAM Access Token refreshed Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: check a bucket path(/InputFiles/OCOMC-mnt/). Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: URL is https://s3.amazonaws.com/iotbigdatarawfilesft-lit/InputFiles/OCOMC-mnt/ Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: URL changed is https://iotbigdatarawfilesft-lit.s3.amazonaws.com/InputFiles/OCOMC-mnt/ Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: computing signature [GET] [/InputFiles/OCOMC-mnt/] [] [] Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: url is https://s3.amazonaws.com Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * About to connect() to iotbigdatarawfilesft-lit.s3.amazonaws.com port 443 (#4) Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Trying 52.219.72.125... Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Connected to iotbigdatarawfilesft-lit.s3.amazonaws.com (52.219.72.125) port 443 (#4) Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Initializing NSS with certpath: sql:/etc/pki/nssdb Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * CAfile: /etc/pki/tls/certs/ca-bundle.crt#012 CApath: none Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Server certificate: Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * subject: CN=*.s3.amazonaws.com Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * start date: Mar 21 00:00:00 2023 GMT Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * expire date: Dec 19 23:59:59 2023 GMT Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * common name: *.s3.amazonaws.com Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * issuer: CN=Amazon RSA 2048 M01,O=Amazon,C=US Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > GET /InputFiles/OCOMC-mnt/ HTTP/1.1 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > User-Agent: s3fs/1.92 (commit hash unknown; OpenSSL) Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Accept: */* Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Authorization: AWS4-HMAC-SHA256 Credential=ASIA6GDLTVMB5SFOECR5/20230713/eu-central-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=f2522f9dc3e54fe5fdc97fd51b65f308664b0da452fe0f414724baf5222a8f47 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > host: iotbigdatarawfilesft-lit.s3.amazonaws.com Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > x-amz-date: 20230713T152010Z Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > x-amz-security-token: IQoJb3JpZ2luX2VjEHcaDGV1LWNlbnRyYWwtMSJGMEQCIHY/Fb8yk/UxAw4LoWl9Vbs3ZNA1EV4sd69x7fJZt/jHAiBug+sm9HYR/33zHx+SXunYh/kHnlrIJv2WePL+KkFaBSrPBQjw//////////8BEAIaDDk3NTE4MzI2MDQxOSIM6BA3p0S01D68ejxbKqMFYKhaBWa3FfzDVC/rmRYkdTWj8/4jcEtXTxVJ4TvsB7QhHncadfExb4TNohtDWNSehhw9HSXzv0VFqKVnYBICg+/1NYvHjlNrB5TE7W6mNMNc1HuU3ZL1Ke6DvX76hnnoIr8bC4DqZ15oRJwQ832k2QuciJlSE7370fcXpj9yqzIdqcYzfrslzj2JRBIEgSstPyMC3PoAVJ/BjqK51iVkpL25dKOqigcWUWp9k3kVKPy3ohXezds8VGOwCAvGwyG1LCTKIHzHJ08KZ//E6llnO+F5aKb1/0DaYNW3NbnYEpLxc7XZQ5FtkrCXuaCy3aWTza79k0q8ACz+88SdIH+G71dQcd2FC1fo/VLp7V2lSBjpuEiP23ck8s+a+eHI2agHsjfrSYDxF+B/6jrrC3caCfydh/NAaueziPp4hPwUsUj5KkDMGhonTNQRPqH/huwLnAY93Bt3Fb6SZhRpG6xP798HSA3PrXrJPZOMZ3UcT7nf1VzB6slks+bW37/+7zq2vNJt/au9ZVyNg/7EJ5XV8SCYdlX320KDtksHd+oC/uRudcGo/hbpw6e6F2RkVcbw5aHvJFM0/YvApzI5Ph1eGdpbUMK3hqPoPA4Wek0KXjOmssegHbELsw269z/IMBlMFtL6N+Uiot6JhkGhwiSyWudJaizO2EkYGsQ3ytRuOphWzTVNfSDnnM2XySr5eK9/O4FBPLlgiFBPwKnj3A0MkOWn6u2nKCvfU6DriIdMLbmj3KNUgzso80RUzGgqMwpn0z9kcA6BOOOl6miPDR1xT851vEENasehOGJDfaKmpsIkc2PoeoUZzya5QQirTTYPHHyheo1d5rWjMZ1qqbSwBOfYfutsPwa5hsgnlCeoj5CkFVcvuNboQwCdDSe4dPpYavERMKiVwKUGOrIBz8v6q1vCkERkbGOFMAb0tWYWG0gGKrltFD032k5oFc38Pc42fNSR8c8WWMlVjGpJx4nz3wa6v6XFyJ5VIyZgERy4E/LiU7lfQBbrpwMgOyYSFZPqYBLnJJeH0h6ds7a1clSTdZ4hiB6in76Ne6bVDE4xU8pp//D63gzN4Ka5FLYaodb8RCWxQ4616v1m/Z7VxkBN5ojnvsh5TzUD9aAkHidl9n19TT8buz2DcOyu6Ks7WQ== Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: > Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < HTTP/1.1 404 Not Found Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < x-amz-request-id: 1E2ZR6WMF5ZDB80Y Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < x-amz-id-2: srOoAvgVjmsPlUoF1SIGvmnzzkmfzzVnlGnRa6N12Iyy+/Du5K9AeUq9UOaTKgtq6voyjMOfoEY= Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Content-Type: application/xml Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Transfer-Encoding: chunked Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Date: Thu, 13 Jul 2023 15:20:10 GMT Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Server: AmazonS3 Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: < Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: * Connection #4 to host iotbigdatarawfilesft-lit.s3.amazonaws.com left intact Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: HTTP response code 404 was returned, returning ENOENT Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: curl.cpp:CheckBucket(3741): Check bucket failed, S3 response: <?xml version="1.0" encoding="UTF-8"?>#012<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>InputFiles/OCOMC-mnt/</Key><RequestId>1E2ZR6WMF5ZDB80Y</RequestId><HostId>srOoAvgVjmsPlUoF1SIGvmnzzkmfzzVnlGnRa6N12Iyy+/Du5K9AeUq9UOaTKgtq6voyjMOfoEY=</HostId></Error> Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: s3fs.cpp:s3fs_check_service(4443): Failed to check bucket and directory for mount point : Bucket or directory(/InputFiles/OCOMC-mnt) not found(host=https://s3.amazonaws.com) - You may need to specify the compat_dir option. Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: s3fs.cpp:s3fs_exit_fuseloop(4200): Exiting FUSE event loop due to errors Jul 13 15:20:10 ip-10-201-34-100 s3fs[148525]: destroy Jul 13 15:20:10 ip-10-201-34-100 kernel: s3fs[148525]: segfault at 7f40732479d0 ip 00007f4077252f81 sp 00007ffc5ac05090 error 4 in libpthread-2.17.so[7f407724a000+17000] ``` ### Details about issue I am trying the following command to mount the s3 bucket but with no success: `s3fs -o iam_role=auto -o url=https://s3.amazonaws.com -o endpoint=eu-central-1 -o dbglevel=info -o curldbg -o allow_other -o use_cache=/tmp -o mp_umask=002 -o multireq_max=5 iotbigdatarawfilesft-lit:/InputFiles/OCOMC-mnt/ /opt/SP/users/oracle/data/input/` I have tried a lot of fixes but with no success, actually I don't know what's the issue. Please advise Thanks & BR, Haytham
kerem closed this issue 2026-03-04 01:51:35 +03:00
Author
Owner

@jdoepp commented on GitHub (Jul 18, 2023):

I am having the same issue. s3fs runs without displaying any errors, but doesn't mount the volume. Using the -o debug argument I get the following messages:

FUSE library version: 2.9.2
nullpath_ok: 0
nopath: 0
utime_omit_ok: 1
unique: 1, opcode: INIT (26), nodeid: 0, insize: 56, pid: 0
INIT: 7.26
flags=0x001ffffb
max_readahead=0x00020000
   INIT: 7.19
   flags=0x00000011
   max_readahead=0x00020000
   max_write=0x00020000
   max_background=0
   congestion_threshold=0
   unique: 1, success, outsize: 40

When I run dmesg, and see the following:

[ 25.054175] s3fs[5095]: segfault at 7f6a0232d9d0 ip 00007f6a0663f6ae sp 00007ffd81b715a0 error 4 in libpthread-2.26.so[7f6a06637000+18000]

s3fs was recently updated on the system, but glibc-devel has not been and is still at v. 2.26.63.

<!-- gh-comment-id:1640913017 --> @jdoepp commented on GitHub (Jul 18, 2023): I am having the same issue. s3fs runs without displaying any errors, but doesn't mount the volume. Using the -o debug argument I get the following messages: ``` FUSE library version: 2.9.2 nullpath_ok: 0 nopath: 0 utime_omit_ok: 1 unique: 1, opcode: INIT (26), nodeid: 0, insize: 56, pid: 0 INIT: 7.26 flags=0x001ffffb max_readahead=0x00020000 INIT: 7.19 flags=0x00000011 max_readahead=0x00020000 max_write=0x00020000 max_background=0 congestion_threshold=0 unique: 1, success, outsize: 40 ``` When I run dmesg, and see the following: `[ 25.054175] s3fs[5095]: segfault at 7f6a0232d9d0 ip 00007f6a0663f6ae sp 00007ffd81b715a0 error 4 in libpthread-2.26.so[7f6a06637000+18000]` s3fs was recently updated on the system, but glibc-devel has not been and is still at v. 2.26.63.
Author
Owner

@jdoepp commented on GitHub (Jul 18, 2023):

My issue was solved. Used -f -o curldbg to check the curl debug. There was an issue accessing the directory. Solved by adding -o compat_dir.

<!-- gh-comment-id:1640964369 --> @jdoepp commented on GitHub (Jul 18, 2023): My issue was solved. Used `-f -o curldbg` to check the curl debug. There was an issue accessing the directory. Solved by adding `-o compat_dir`.
Author
Owner

@HeGaoYuan commented on GitHub (Aug 16, 2023):

Watching

<!-- gh-comment-id:1680556672 --> @HeGaoYuan commented on GitHub (Aug 16, 2023): Watching
Author
Owner

@gaul commented on GitHub (Sep 8, 2023):

Please test with 1.93 and reopen if your symptoms persist.

<!-- gh-comment-id:1710971834 --> @gaul commented on GitHub (Sep 8, 2023): Please test with 1.93 and reopen if your symptoms persist.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#1124
No description provided.