mirror of
https://github.com/s3fs-fuse/s3fs-fuse.git
synced 2026-04-25 05:16:00 +03:00
[GH-ISSUE #1687] Can't mount bucket with path when path wasn't created by s3fs #879
Labels
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/s3fs-fuse#879
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @DiemanXT on GitHub (Jun 18, 2021).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1687
Additional Information
I am trying to mount specific path within an s3 bucket. Path I am trying to mount wasn't created by s3fs. In Q&A I found paragraph that says that if a path wasn't created by s3fs it might be not possible to mount. I just want to understand if there is anything I can do to workaround it or this is a dead end.
Version of s3fs being used (s3fs --version)
[root@prog3 ~]# s3fs --version
Amazon Simple Storage Service File System V1.89 (commit:unknown) with OpenSSL
Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse)
[root@prog3 ~]# rpm -qi fuse-devel
Name : fuse-devel
Version : 2.9.2
Release : 11.el7
Architecture: x86_64
Install Date: Fri 18 Jun 2021 12:10:14 PM UTC
Group : Development/Libraries
Size : 126359
License : LGPLv2+
Signature : RSA/SHA256, Mon 06 Aug 2018 01:13:08 PM UTC, Key ID 199e2f91fd431d51
Source RPM : fuse-2.9.2-11.el7.src.rpm
Build Date : Tue 24 Jul 2018 02:58:28 PM UTC
Build Host : x86-039.build.eng.bos.redhat.com
Relocations : (not relocatable)
Packager : Red Hat, Inc. http://bugzilla.redhat.com/bugzilla
Vendor : Red Hat, Inc.
URL : https://github.com/libfuse/libfuse
Summary : File System in Userspace (FUSE) devel files
Description :
With FUSE it is possible to implement a fully functional filesystem in a
userspace program. This package contains development files (headers,
pgk-config) to develop FUSE based applications/filesystems.
Kernel information (uname -r)
[root@prog3 ~]# uname -r
3.10.0-1127.19.1.el7.x86_64
GNU/Linux Distribution, if applicable (cat /etc/os-release)
[root@prog3 ~]# cat /etc/os-release
NAME="Red Hat Enterprise Linux Server"
VERSION="7.8 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.8"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.8 (Maipo)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.8:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.8
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.8"
s3fs command line used, if applicable
s3fs xxx-user:/dsup /xxx-user -o use_session_token -o endpoint='eu-west-1' -o url='https://s3.eu-west-1.amazonaws.com' -o allow_other -o dbglevel=info -f -o curldbg
/etc/fstab entry, if applicable
N/A
s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)
[root@prog3 ~]# s3fs bucket-eu-west-1-787511913082-user:/dsup /bucket-eu-west-1-787511913082-user -o use_session_token -o endpoint='eu-west-1' -o url='https://s3.eu-west-1.amazonaws.com' -o allow_other -o dbglevel=info -f -o curldbg
2021-06-18T16:29:07.635Z [INF] s3fs version 1.89(unknown) : s3fs -o use_session_token -o endpoint=eu-west-1 -o url=https://s3.eu-west-1.amazonaws.com -o allow_other -o dbglevel=info -f -o curldbg bucket-eu-west-1-787511913082-user:/dsup /bucket-eu-west-1-787511913082-user
2021-06-18T16:29:07.637Z [CRT] s3fs_logger.cpp:LowSetLogLevel(240): change debug level from [CRT] to [INF]
2021-06-18T16:29:07.637Z [INF] s3fs.cpp:set_mountpoint_attribute(4084): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40777)
s3fs: MOUNTPOINT directory /bucket-eu-west-1-787511913082-user is not empty. if you are sure this is safe, can use the 'nonempty' mount option.
[root@prog3 ~]# umount /bucket-eu-west-1-787511913082-user/
[root@prog3 ~]# s3fs xxx-user:/dsup /bucket-eu-west-1-787511913082-user -o use_session_token -o endpoint='eu-west-1' -o url='https://s3.eu-west-1.amazonaws.com' -o allow_other -o dbglevel=info -f -o curldbg
2021-06-18T16:29:16.058Z [INF] s3fs version 1.89(unknown) : s3fs -o use_session_token -o endpoint=eu-west-1 -o url=https://s3.eu-west-1.amazonaws.com -o allow_other -o dbglevel=info -f -o curldbg xxx-user:/dsup /bucket-eu-west-1-787511913082-user
2021-06-18T16:29:16.060Z [CRT] s3fs_logger.cpp:LowSetLogLevel(240): change debug level from [CRT] to [INF]
2021-06-18T16:29:16.060Z [INF] s3fs.cpp:set_mountpoint_attribute(4084): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40777)
2021-06-18T16:29:16.060Z [INF] s3fs_util.cpp:compare_sysname(359): system name is Linux
2021-06-18T16:29:16.060Z [WAN] curl.cpp:InitMimeType(404): Could not find mime.types files, you have to create file(/etc/mime.types) or specify mime option for existing mime.types file.
2021-06-18T16:29:16.060Z [WAN] s3fs.cpp:main(5002): Missing MIME types prevents setting Content-Type on uploaded objects.
2021-06-18T16:29:16.062Z [INF] fdcache_stat.cpp:CheckCacheFileStatTopDir(79): The path to cache top dir is empty, thus not need to check permission.
2021-06-18T16:29:16.063Z [INF] s3fs.cpp:s3fs_init(3373): init v1.89(commit:unknown) with OpenSSL
2021-06-18T16:29:16.063Z [INF] s3fs.cpp:s3fs_check_service(3507): check services.
2021-06-18T16:29:16.063Z [INF] curl.cpp:CheckBucket(3373): check a bucket.
2021-06-18T16:29:16.063Z [WAN] curl.cpp:ResetHandle(1939): The CURLOPT_SSL_ENABLE_ALPN option could not be unset. S3 server does not support ALPN, then this option should be disabled to maximize performance. you need to use libcurl 7.36.0 or later.
2021-06-18T16:29:16.063Z [WAN] curl.cpp:ResetHandle(1942): The S3FS_CURLOPT_KEEP_SENDING_ON_ERROR option could not be set. For maximize performance you need to enable this option and you should use libcurl 7.51.0 or later.
2021-06-18T16:29:16.063Z [INF] curl_util.cpp:prepare_url(254): URL is https://s3.eu-west-1.amazonaws.com/xxx-user/dsup/
2021-06-18T16:29:16.063Z [INF] curl_util.cpp:prepare_url(287): URL changed is https://bucket-eu-west-1-787511913082-user.s3.eu-west-1.amazonaws.com/dsup/
2021-06-18T16:29:16.063Z [INF] curl.cpp:insertV4Headers(2669): computing signature [GET] [/dsup/] [] []
2021-06-18T16:29:16.063Z [INF] curl_util.cpp:url_to_host(331): url is https://s3.eu-west-1.amazonaws.com
2021-06-18T16:29:16.068Z [CURL DBG] * About to connect() to bucket-eu-west-1-787511913082-user.s3.eu-west-1.amazonaws.com port 443 (#0)
2021-06-18T16:29:16.068Z [CURL DBG] * Trying 52.218.116.2...
2021-06-18T16:29:16.068Z [CURL DBG] * Connected to bucket-eu-west-1-787511913082-user.s3.eu-west-1.amazonaws.com (52.218.116.2) port 443 (#0)
2021-06-18T16:29:16.068Z [CURL DBG] * Initializing NSS with certpath: sql:/etc/pki/nssdb
2021-06-18T16:29:16.185Z [CURL DBG] * CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
2021-06-18T16:29:16.191Z [CURL DBG] * SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
2021-06-18T16:29:16.191Z [CURL DBG] * Server certificate:
2021-06-18T16:29:16.191Z [CURL DBG] * subject: CN=*.s3-eu-west-1.amazonaws.com,O="Amazon.com, Inc.",L=Seattle,ST=Washington,C=US
2021-06-18T16:29:16.191Z [CURL DBG] * start date: Aug 04 00:00:00 2020 GMT
2021-06-18T16:29:16.191Z [CURL DBG] * expire date: Aug 09 12:00:00 2021 GMT
2021-06-18T16:29:16.191Z [CURL DBG] * common name: *.s3-eu-west-1.amazonaws.com
2021-06-18T16:29:16.191Z [CURL DBG] * issuer: CN=DigiCert Baltimore CA-2 G2,OU=www.digicert.com,O=DigiCert Inc,C=US
2021-06-18T16:29:16.191Z [CURL DBG] > GET /dsup/ HTTP/1.1
2021-06-18T16:29:16.191Z [CURL DBG] > User-Agent: s3fs/1.89 (commit hash unknown; OpenSSL)
2021-06-18T16:29:16.191Z [CURL DBG] > Accept: /
2021-06-18T16:29:16.191Z [CURL DBG] > Authorization: AWS4-HMAC-SHA256 Credential=ASIA3OW26FJ5HRNCLH4Q/20210618/eu-west-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=d6f3e0dad58bab24d4616413e066cdc20917ce3d616e98dedd069475cd5dcc3c
2021-06-18T16:29:16.191Z [CURL DBG] > host: bucket-eu-west-1-787511913082-user.s3.eu-west-1.amazonaws.com
2021-06-18T16:29:16.191Z [CURL DBG] > x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2021-06-18T16:29:16.191Z [CURL DBG] > x-amz-date: 20210618T162916Z
2021-06-18T16:29:16.191Z [CURL DBG] > x-amz-security-token: FwoGZXIvYXdzEMH//////////wEaDIkj79DagIG046UP/SKnAYoaxB4RkKGwJVvhrP5xXcp6s3WPVFyJ07MH0z8kYrsPgnI6vIYacXl0ZQASQX2C0C8ai3NtRas1P90yraZpDlzvk40NuLSYs5OFSfhsdS9Bpc0G1JtLr1dzIUdX0Gafcg0xlALZvPyZ5/fiIoqFM4YLO9tuL8g7Lmz52lk+ze8906qgBiYCvmiI42kcUJegeTQWeKoEZA7OV4doBWRBXf+0UPdCTIYSKIKGs4YGMi3jNsJAGsnsmrG/jEEasWyasN35fGpJVhtutE5/TPUIOrcGB3IB46ppG5yqfGM=
2021-06-18T16:29:16.191Z [CURL DBG] >
2021-06-18T16:29:16.224Z [CURL DBG] < HTTP/1.1 404 Not Found
2021-06-18T16:29:16.224Z [CURL DBG] < x-amz-request-id: 9SMNC9EMXDAW1MB3
2021-06-18T16:29:16.224Z [CURL DBG] < x-amz-id-2: 4mR4ZfI8LjbVfed62o7nz2byFXKxcevKaqmaAV/5JCsTXDsWl8y1KdNG2C4HWwV5+8DJ1+nFlGo=
2021-06-18T16:29:16.224Z [CURL DBG] < Content-Type: application/xml
2021-06-18T16:29:16.224Z [CURL DBG] < Transfer-Encoding: chunked
2021-06-18T16:29:16.224Z [CURL DBG] < Date: Fri, 18 Jun 2021 16:29:15 GMT
2021-06-18T16:29:16.224Z [CURL DBG] < Server: AmazonS3
2021-06-18T16:29:16.224Z [CURL DBG] <
2021-06-18T16:29:16.224Z [CURL DBG] * Connection #0 to host xxx-user.s3.eu-west-1.amazonaws.com left intact
2021-06-18T16:29:16.224Z [INF] curl.cpp:RequestPerform(2359): HTTP response code 404 was returned, returning ENOENT
2021-06-18T16:29:16.224Z [ERR] curl.cpp:CheckBucket(3405): Check bucket failed, S3 response:
NoSuchKeyThe specified key does not exist.dsup/9SMNC9EMXDAW1MB34mR4ZfI8LjbVfed62o7nz2byFXKxcevKaqmaAV/5JCsTXDsWl8y1KdNG2C4HWwV5+8DJ1+nFlGo=2021-06-18T16:29:16.224Z [CRT] s3fs.cpp:s3fs_check_service(3584): bucket or key not found(host=https://s3.eu-west-1.amazonaws.com) - result of checking service.
2021-06-18T16:29:16.224Z [INF] curl_handlerpool.cpp:ReturnHandler(110): Pool full: destroy the oldest handler
2021-06-18T16:29:16.224Z [ERR] s3fs.cpp:s3fs_exit_fuseloop(3363): Exiting FUSE event loop due to errors
2021-06-18T16:29:16.233Z [INF] s3fs.cpp:s3fs_destroy(3431): destroy
Details about issue
@ggtakec commented on GitHub (Jun 20, 2021):
@DiemanXT Thanks for your issue about s3fs.
How did you create the
path(ex./dsup) under the bucket?If you are using AWS S3, you need to create it as a
folder.For s3fs, the object corresponding to the
/dsuppath (in your example) should be/dsup/for the directory name.If you are using AWS S3, try to specify the path created as a folder on the console etc.
Thanks in advance for your assistance.
@gaul commented on GitHub (Jul 25, 2021):
Related StackOverflow question:
https://stackoverflow.com/questions/64584917/s3fs-bucket-name-contains-illegal-character/64585933#64585933