starred/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2026-04-25 13:26:00 +03:00
Code Issues 305 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #1573] Moving from goofys to s3fs , not as easy as I expected. Can you provide some guidance? #823

New issue
Open
opened 2026-03-04 01:49:06 +03:00 by kerem · 9 comments
kerem commented 2026-03-04 01:49:06 +03:00
Owner
Copy link

Originally created by @joanmarcriera on GitHub (Feb 15, 2021).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1573

Additional Information

The following information is very important in order to help us to help you. Omission of the following details may delay your support request or receive no attention at all.
Keep in mind that the commands we provide to retrieve information are oriented to GNU/Linux Distributions, so you could need to use others if you use s3fs on macOS or BSD

Version of s3fs being used (s3fs --version)

# s3fs --version
Amazon Simple Storage Service File System V1.88 (commit:unknown) with OpenSSL

Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse)

Name        : fuse
Version     : 2.9.7
Release     : 12.el8
Architecture: x86_64
Install Date: Tue 28 Jan 2020 12:15:05 PM GMT
Group       : Unspecified
Size        : 210844
License     : GPL+
Signature   : RSA/SHA256, Mon 01 Jul 2019 10:37:42 PM BST, Key ID 05b555b38483c65d
Source RPM  : fuse-2.9.7-12.el8.src.rpm
Build Date  : Sat 11 May 2019 03:07:38 PM BST
Build Host  : x86-02.mbox.centos.org
Relocations : (not relocatable)
Packager    : CentOS Buildsys <bugs@centos.org>
Vendor      : CentOS
URL         : http://fuse.sf.net
Summary     : File System in Userspace (FUSE) v2 utilities
Description :
With FUSE it is possible to implement a fully functional filesystem in a
userspace program. This package contains the FUSE v2 userspace tools to
mount a FUSE filesystem.

Kernel information (uname -r)

4.18.0-147.3.1.el8_1.x86_64
[root@redacted test]# lsb_release -a
LSB Version:    :core-4.1-amd64:core-4.1-noarch
Distributor ID: CentOS
Description:    CentOS Linux release 8.2.2004 (Core)
Release:        8.2.2004
Codename:       Core
[root@redacted test]#

GNU/Linux Distribution, if applicable (cat /etc/os-release)

VERSION="8 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="CentOS Linux 8 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:8"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-8"
CENTOS_MANTISBT_PROJECT_VERSION="8"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="8"

s3fs command line used, if applicable

[root@redacted test]# s3fs internal-public ~/test/internal-public/ -o url=https://redacted.uk -o public_bucket=1 -o use_path_request_style -o ro -o nomultipart  -o nomixupload -o noxmlns -o norenameapi  -o no_check_certificate -o nosscache -o nodnscache -f -d
[CRT] s3fs_logger.cpp:LowSetLogLevel(201): change debug level from [CRT] to [INF]
[INF]     s3fs.cpp:set_mountpoint_attribute(3989): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755)
[INF] curl.cpp:InitMimeType(438): Loaded mime information from /etc/mime.types
[WAN] s3fs.cpp:main(4902): Setting max_dirty_data to -1 when nomixupload is enabled
[INF] fdcache_stat.cpp:CheckCacheFileStatTopDir(79): The path to cache top dir is empty, thus not need to check permission.
[INF] s3fs.cpp:s3fs_init(3300): init v1.88(commit:unknown) with OpenSSL
[INF] s3fs.cpp:s3fs_check_service(3416): check services.
[INF]       curl.cpp:CheckBucket(3318): check a bucket.
[INF]       curl_util.cpp:prepare_url(250): URL is https://redacted.uk/internal-public/
[INF]       curl_util.cpp:prepare_url(283): URL changed is https://redacted.uk/internal-public/
[INF]       curl.cpp:insertV4Headers(2598): computing signature [GET] [/] [] []
[INF]       curl_util.cpp:url_to_host(327): url is https://redacted.uk
[ERR] curl.cpp:RequestPerform(2316): HTTP response code 401, returning EIO. Body Text:
[ERR] curl.cpp:CheckBucket(3344): Check bucket failed, S3 response:
[CRT] s3fs.cpp:s3fs_check_service(3493): unable to connect(host=https://redacted.uk) - result of checking service.
[INF] curl_handlerpool.cpp:ReturnHandler(110): Pool full: destroy the oldest handler
[ERR] s3fs.cpp:s3fs_exit_fuseloop(3290): Exiting FUSE event loop due to errors

[INF] s3fs.cpp:s3fs_destroy(3358): destroy

Works with goofys : 

[root@redacted test]# goofys --stat-cache-ttl 1s --type-cache-ttl 1s --endpoint https://redacted.uk internal-public ~/test/internal-public/
[root@redacted test]# ls ~/test/internal-public/
path  test
[root@redacted test]#

/etc/fstab entry, if applicable

nope

s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)

if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages

Added with the commands.

Details about issue

We have our own S3 implementation, it works with goofys and does not work with s3fs. We understand that we have done something that is not fully compatible but I have tried all the options in s3fs and at this point I'd like to ask if there is any obvious mistake you can see from the current log.

Goofys seams unmaintained, so we are looking to become compatible with s3fs if that is possible.

Thanks for your time, great code.

Originally created by @joanmarcriera on GitHub (Feb 15, 2021). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1573 ### Additional Information _The following information is very important in order to help us to help you. Omission of the following details may delay your support request or receive no attention at all._ _Keep in mind that the commands we provide to retrieve information are oriented to GNU/Linux Distributions, so you could need to use others if you use s3fs on macOS or BSD_ #### Version of s3fs being used (s3fs --version) ``` # s3fs --version Amazon Simple Storage Service File System V1.88 (commit:unknown) with OpenSSL ``` #### Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse) ```st]# rpm -qi fuse Name : fuse Version : 2.9.7 Release : 12.el8 Architecture: x86_64 Install Date: Tue 28 Jan 2020 12:15:05 PM GMT Group : Unspecified Size : 210844 License : GPL+ Signature : RSA/SHA256, Mon 01 Jul 2019 10:37:42 PM BST, Key ID 05b555b38483c65d Source RPM : fuse-2.9.7-12.el8.src.rpm Build Date : Sat 11 May 2019 03:07:38 PM BST Build Host : x86-02.mbox.centos.org Relocations : (not relocatable) Packager : CentOS Buildsys <bugs@centos.org> Vendor : CentOS URL : http://fuse.sf.net Summary : File System in Userspace (FUSE) v2 utilities Description : With FUSE it is possible to implement a fully functional filesystem in a userspace program. This package contains the FUSE v2 userspace tools to mount a FUSE filesystem. ``` #### Kernel information (uname -r) ```st]# uname -r 4.18.0-147.3.1.el8_1.x86_64 [root@redacted test]# lsb_release -a LSB Version: :core-4.1-amd64:core-4.1-noarch Distributor ID: CentOS Description: CentOS Linux release 8.2.2004 (Core) Release: 8.2.2004 Codename: Core [root@redacted test]# ``` #### GNU/Linux Distribution, if applicable (cat /etc/os-release) ```NAME="CentOS Linux" VERSION="8 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="8" PLATFORM_ID="platform:el8" PRETTY_NAME="CentOS Linux 8 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:8" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/" CENTOS_MANTISBT_PROJECT="CentOS-8" CENTOS_MANTISBT_PROJECT_VERSION="8" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="8" ``` #### s3fs command line used, if applicable ``` [root@redacted test]# s3fs internal-public ~/test/internal-public/ -o url=https://redacted.uk -o public_bucket=1 -o use_path_request_style -o ro -o nomultipart -o nomixupload -o noxmlns -o norenameapi -o no_check_certificate -o nosscache -o nodnscache -f -d [CRT] s3fs_logger.cpp:LowSetLogLevel(201): change debug level from [CRT] to [INF] [INF] s3fs.cpp:set_mountpoint_attribute(3989): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755) [INF] curl.cpp:InitMimeType(438): Loaded mime information from /etc/mime.types [WAN] s3fs.cpp:main(4902): Setting max_dirty_data to -1 when nomixupload is enabled [INF] fdcache_stat.cpp:CheckCacheFileStatTopDir(79): The path to cache top dir is empty, thus not need to check permission. [INF] s3fs.cpp:s3fs_init(3300): init v1.88(commit:unknown) with OpenSSL [INF] s3fs.cpp:s3fs_check_service(3416): check services. [INF] curl.cpp:CheckBucket(3318): check a bucket. [INF] curl_util.cpp:prepare_url(250): URL is https://redacted.uk/internal-public/ [INF] curl_util.cpp:prepare_url(283): URL changed is https://redacted.uk/internal-public/ [INF] curl.cpp:insertV4Headers(2598): computing signature [GET] [/] [] [] [INF] curl_util.cpp:url_to_host(327): url is https://redacted.uk [ERR] curl.cpp:RequestPerform(2316): HTTP response code 401, returning EIO. Body Text: [ERR] curl.cpp:CheckBucket(3344): Check bucket failed, S3 response: [CRT] s3fs.cpp:s3fs_check_service(3493): unable to connect(host=https://redacted.uk) - result of checking service. [INF] curl_handlerpool.cpp:ReturnHandler(110): Pool full: destroy the oldest handler [ERR] s3fs.cpp:s3fs_exit_fuseloop(3290): Exiting FUSE event loop due to errors [INF] s3fs.cpp:s3fs_destroy(3358): destroy Works with goofys : [root@redacted test]# goofys --stat-cache-ttl 1s --type-cache-ttl 1s --endpoint https://redacted.uk internal-public ~/test/internal-public/ [root@redacted test]# ls ~/test/internal-public/ path test [root@redacted test]# ``` #### /etc/fstab entry, if applicable ``` nope ``` #### s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs) _if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages_ ``` Added with the commands. ``` ### Details about issue We have our own S3 implementation, it works with goofys and does not work with s3fs. We understand that we have done something that is not fully compatible but I have tried all the options in s3fs and at this point I'd like to ask if there is any obvious mistake you can see from the current log. Goofys seams unmaintained, so we are looking to become compatible with s3fs if that is possible. Thanks for your time, great code.
kerem commented 2026-03-04 01:49:07 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Feb 16, 2021):

The key lines are:

[ERR] curl.cpp:RequestPerform(2316): HTTP response code 401, returning EIO. Body Text:
[ERR] curl.cpp:CheckBucket(3344): Check bucket failed, S3 response:

It is weird that your custom S3 implementation does not return an error code here. Checking the service here just issues a ListObjects query -- perhaps your S3 implementation only supports ListObjectsV2?

<!-- gh-comment-id:779805636 --> @gaul commented on GitHub (Feb 16, 2021): The key lines are: ``` [ERR] curl.cpp:RequestPerform(2316): HTTP response code 401, returning EIO. Body Text: [ERR] curl.cpp:CheckBucket(3344): Check bucket failed, S3 response: ``` It is weird that your custom S3 implementation does not return an error code here. Checking the service here just issues a `ListObjects` query -- perhaps your S3 implementation only supports `ListObjectsV2`?
kerem commented 2026-03-04 01:49:07 +03:00
Author
Owner
Copy link

@joanmarcriera commented on GitHub (Feb 18, 2021):

Thank you @gaul , that is exactly the issue here.
We will be working on having a more canonical implementation and that will probably be in our roadmap.

Thank you.

<!-- gh-comment-id:781488453 --> @joanmarcriera commented on GitHub (Feb 18, 2021): Thank you @gaul , that is exactly the issue here. We will be working on having a more canonical implementation and that will probably be in our roadmap. Thank you.
kerem commented 2026-03-04 01:49:07 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Feb 22, 2021):

Reopening since s3fs should allow optionally issuing ListObjectsV2.

<!-- gh-comment-id:782986770 --> @gaul commented on GitHub (Feb 22, 2021): Reopening since s3fs should allow optionally issuing `ListObjectsV2`.
kerem commented 2026-03-04 01:49:07 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Feb 23, 2021):

@joanmarcriera Could you test with the latest master and use -o listobjectsv2 and report back? You might also want to ask your S3 vendor to return HTTP 501 NotImplemented when encountering the older API.

<!-- gh-comment-id:783793596 --> @gaul commented on GitHub (Feb 23, 2021): @joanmarcriera Could you test with the latest master and use `-o listobjectsv2` and report back? You might also want to ask your S3 vendor to return HTTP 501 `NotImplemented` when encountering the older API.
kerem commented 2026-03-04 01:49:07 +03:00
Author
Owner
Copy link

@joanmarcriera commented on GitHub (Feb 26, 2021):

Hi @gaul ,

If I missed something just point it out and I'll amend it.

[root@hostnameREDACTED001 s3fs-fuse]# ./src/s3fs --version
Amazon Simple Storage Service File System V1.89 (commit:8c58ba8) with OpenSSL
Copyright (C) 2010 Randy Rizun <rrizun@gmail.com>
License GPL2: GNU GPL version 2 <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

[root@hostnameREDACTED001 s3fs-fuse]# ./src/s3fs internal-public ~/test/internal-public/ -o url=https://hostREDACTED.uk -o public_bucket=1 -o use_path_request_style -o ro -o nomultipart  -o nomixupload -o noxmlns -o norenameapi  -o no_check_certificate -o nosscache -o nodnscache -f -d
2021-02-26T11:00:59.337Z [CRT] s3fs_logger.cpp:LowSetLogLevel(219): change debug level from [CRT] to [INF]
2021-02-26T11:00:59.337Z [INF]     s3fs.cpp:set_mountpoint_attribute(4020): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755)
2021-02-26T11:00:59.339Z [INF] curl.cpp:InitMimeType(436): Loaded mime information from /etc/mime.types
2021-02-26T11:00:59.339Z [WAN] s3fs.cpp:main(4961): Setting max_dirty_data to -1 when nomixupload is enabled
2021-02-26T11:00:59.339Z [INF] fdcache_stat.cpp:CheckCacheFileStatTopDir(79): The path to cache top dir is empty, thus not need to check permission.
2021-02-26T11:00:59.343Z [INF] s3fs.cpp:s3fs_init(3331): init v1.89(commit:8c58ba8) with OpenSSL
2021-02-26T11:00:59.343Z [INF] s3fs.cpp:s3fs_check_service(3447): check services.
2021-02-26T11:00:59.343Z [INF]       curl.cpp:CheckBucket(3364): check a bucket.
2021-02-26T11:00:59.343Z [INF]       curl_util.cpp:prepare_url(250): URL is https://hostREDACTED.uk/internal-public/
2021-02-26T11:00:59.343Z [INF]       curl_util.cpp:prepare_url(283): URL changed is https://hostREDACTED.uk/internal-public/
2021-02-26T11:00:59.343Z [INF]       curl.cpp:insertV4Headers(2640): computing signature [GET] [/] [] []
2021-02-26T11:00:59.343Z [INF]       curl_util.cpp:url_to_host(327): url is https://hostREDACTED.uk
2021-02-26T11:00:59.365Z [ERR] curl.cpp:RequestPerform(2353): HTTP response code 401, returning EIO. Body Text:
2021-02-26T11:00:59.365Z [ERR] curl.cpp:CheckBucket(3396): Check bucket failed, S3 response:
2021-02-26T11:00:59.365Z [CRT] s3fs.cpp:s3fs_check_service(3524): unable to connect(host=https://hostREDACTED.uk) - result of checking service.
2021-02-26T11:00:59.365Z [INF] curl_handlerpool.cpp:ReturnHandler(110): Pool full: destroy the oldest handler
2021-02-26T11:00:59.365Z [ERR] s3fs.cpp:s3fs_exit_fuseloop(3321): Exiting FUSE event loop due to errors

2021-02-26T11:00:59.369Z [INF] s3fs.cpp:s3fs_destroy(3389): destroy


[root@hostnameREDACTED001 s3fs-fuse]# ./src/s3fs internal-public ~/test/internal-public/ -o url=https://hostREDACTED.uk -o public_bucket=1 -o use_path_request_style -o ro -o nomultipart  -o nomixupload -o noxmlns -o norenameapi  -o no_check_certificate -o nosscache -o nodnscache -o listobjectsv2 -f -d
2021-02-26T11:01:10.799Z [CRT] s3fs_logger.cpp:LowSetLogLevel(219): change debug level from [CRT] to [INF]
2021-02-26T11:01:10.799Z [INF]     s3fs.cpp:set_mountpoint_attribute(4020): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755)
2021-02-26T11:01:10.801Z [INF] curl.cpp:InitMimeType(436): Loaded mime information from /etc/mime.types
2021-02-26T11:01:10.801Z [WAN] s3fs.cpp:main(4961): Setting max_dirty_data to -1 when nomixupload is enabled
2021-02-26T11:01:10.801Z [INF] fdcache_stat.cpp:CheckCacheFileStatTopDir(79): The path to cache top dir is empty, thus not need to check permission.
2021-02-26T11:01:10.803Z [INF] s3fs.cpp:s3fs_init(3331): init v1.89(commit:8c58ba8) with OpenSSL
2021-02-26T11:01:10.804Z [INF] s3fs.cpp:s3fs_check_service(3447): check services.
2021-02-26T11:01:10.804Z [INF]       curl.cpp:CheckBucket(3364): check a bucket.
2021-02-26T11:01:10.804Z [INF]       curl_util.cpp:prepare_url(250): URL is https://hostREDACTED.uk/internal-public/?list-type=2
2021-02-26T11:01:10.804Z [INF]       curl_util.cpp:prepare_url(283): URL changed is https://hostREDACTED.uk/internal-public/?list-type=2
2021-02-26T11:01:10.804Z [INF]       curl.cpp:insertV4Headers(2640): computing signature [GET] [/] [list-type=2] []
2021-02-26T11:01:10.804Z [INF]       curl_util.cpp:url_to_host(327): url is https://hostREDACTED.uk
2021-02-26T11:01:10.825Z [ERR] curl.cpp:RequestPerform(2353): HTTP response code 401, returning EIO. Body Text:
2021-02-26T11:01:10.825Z [ERR] curl.cpp:CheckBucket(3396): Check bucket failed, S3 response:
2021-02-26T11:01:10.825Z [CRT] s3fs.cpp:s3fs_check_service(3524): unable to connect(host=https://hostREDACTED.uk) - result of checking service.
2021-02-26T11:01:10.825Z [INF] curl_handlerpool.cpp:ReturnHandler(110): Pool full: destroy the oldest handler
2021-02-26T11:01:10.825Z [ERR] s3fs.cpp:s3fs_exit_fuseloop(3321): Exiting FUSE event loop due to errors

2021-02-26T11:01:10.831Z [INF] s3fs.cpp:s3fs_destroy(3389): destroy


[root@hostnameREDACTED001 s3fs-fuse]# goofys --stat-cache-ttl 1s --type-cache-ttl 1s --endpoint https://hostREDACTED.uk internal-public ~/test/internal-public/
[root@hostnameREDACTED001 s3fs-fuse]# ls ~/test/internal-public/
path  test
[root@hostnameREDACTED001 s3fs-fuse]# umount ~/test/internal-public/
[root@hostnameREDACTED001 s3fs-fuse]#
<!-- gh-comment-id:786578987 --> @joanmarcriera commented on GitHub (Feb 26, 2021): Hi @gaul , If I missed something just point it out and I'll amend it. ``` [root@hostnameREDACTED001 s3fs-fuse]# ./src/s3fs --version Amazon Simple Storage Service File System V1.89 (commit:8c58ba8) with OpenSSL Copyright (C) 2010 Randy Rizun <rrizun@gmail.com> License GPL2: GNU GPL version 2 <https://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. ``` ``` [root@hostnameREDACTED001 s3fs-fuse]# ./src/s3fs internal-public ~/test/internal-public/ -o url=https://hostREDACTED.uk -o public_bucket=1 -o use_path_request_style -o ro -o nomultipart -o nomixupload -o noxmlns -o norenameapi -o no_check_certificate -o nosscache -o nodnscache -f -d 2021-02-26T11:00:59.337Z [CRT] s3fs_logger.cpp:LowSetLogLevel(219): change debug level from [CRT] to [INF] 2021-02-26T11:00:59.337Z [INF] s3fs.cpp:set_mountpoint_attribute(4020): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755) 2021-02-26T11:00:59.339Z [INF] curl.cpp:InitMimeType(436): Loaded mime information from /etc/mime.types 2021-02-26T11:00:59.339Z [WAN] s3fs.cpp:main(4961): Setting max_dirty_data to -1 when nomixupload is enabled 2021-02-26T11:00:59.339Z [INF] fdcache_stat.cpp:CheckCacheFileStatTopDir(79): The path to cache top dir is empty, thus not need to check permission. 2021-02-26T11:00:59.343Z [INF] s3fs.cpp:s3fs_init(3331): init v1.89(commit:8c58ba8) with OpenSSL 2021-02-26T11:00:59.343Z [INF] s3fs.cpp:s3fs_check_service(3447): check services. 2021-02-26T11:00:59.343Z [INF] curl.cpp:CheckBucket(3364): check a bucket. 2021-02-26T11:00:59.343Z [INF] curl_util.cpp:prepare_url(250): URL is https://hostREDACTED.uk/internal-public/ 2021-02-26T11:00:59.343Z [INF] curl_util.cpp:prepare_url(283): URL changed is https://hostREDACTED.uk/internal-public/ 2021-02-26T11:00:59.343Z [INF] curl.cpp:insertV4Headers(2640): computing signature [GET] [/] [] [] 2021-02-26T11:00:59.343Z [INF] curl_util.cpp:url_to_host(327): url is https://hostREDACTED.uk 2021-02-26T11:00:59.365Z [ERR] curl.cpp:RequestPerform(2353): HTTP response code 401, returning EIO. Body Text: 2021-02-26T11:00:59.365Z [ERR] curl.cpp:CheckBucket(3396): Check bucket failed, S3 response: 2021-02-26T11:00:59.365Z [CRT] s3fs.cpp:s3fs_check_service(3524): unable to connect(host=https://hostREDACTED.uk) - result of checking service. 2021-02-26T11:00:59.365Z [INF] curl_handlerpool.cpp:ReturnHandler(110): Pool full: destroy the oldest handler 2021-02-26T11:00:59.365Z [ERR] s3fs.cpp:s3fs_exit_fuseloop(3321): Exiting FUSE event loop due to errors 2021-02-26T11:00:59.369Z [INF] s3fs.cpp:s3fs_destroy(3389): destroy ``` ``` [root@hostnameREDACTED001 s3fs-fuse]# ./src/s3fs internal-public ~/test/internal-public/ -o url=https://hostREDACTED.uk -o public_bucket=1 -o use_path_request_style -o ro -o nomultipart -o nomixupload -o noxmlns -o norenameapi -o no_check_certificate -o nosscache -o nodnscache -o listobjectsv2 -f -d 2021-02-26T11:01:10.799Z [CRT] s3fs_logger.cpp:LowSetLogLevel(219): change debug level from [CRT] to [INF] 2021-02-26T11:01:10.799Z [INF] s3fs.cpp:set_mountpoint_attribute(4020): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755) 2021-02-26T11:01:10.801Z [INF] curl.cpp:InitMimeType(436): Loaded mime information from /etc/mime.types 2021-02-26T11:01:10.801Z [WAN] s3fs.cpp:main(4961): Setting max_dirty_data to -1 when nomixupload is enabled 2021-02-26T11:01:10.801Z [INF] fdcache_stat.cpp:CheckCacheFileStatTopDir(79): The path to cache top dir is empty, thus not need to check permission. 2021-02-26T11:01:10.803Z [INF] s3fs.cpp:s3fs_init(3331): init v1.89(commit:8c58ba8) with OpenSSL 2021-02-26T11:01:10.804Z [INF] s3fs.cpp:s3fs_check_service(3447): check services. 2021-02-26T11:01:10.804Z [INF] curl.cpp:CheckBucket(3364): check a bucket. 2021-02-26T11:01:10.804Z [INF] curl_util.cpp:prepare_url(250): URL is https://hostREDACTED.uk/internal-public/?list-type=2 2021-02-26T11:01:10.804Z [INF] curl_util.cpp:prepare_url(283): URL changed is https://hostREDACTED.uk/internal-public/?list-type=2 2021-02-26T11:01:10.804Z [INF] curl.cpp:insertV4Headers(2640): computing signature [GET] [/] [list-type=2] [] 2021-02-26T11:01:10.804Z [INF] curl_util.cpp:url_to_host(327): url is https://hostREDACTED.uk 2021-02-26T11:01:10.825Z [ERR] curl.cpp:RequestPerform(2353): HTTP response code 401, returning EIO. Body Text: 2021-02-26T11:01:10.825Z [ERR] curl.cpp:CheckBucket(3396): Check bucket failed, S3 response: 2021-02-26T11:01:10.825Z [CRT] s3fs.cpp:s3fs_check_service(3524): unable to connect(host=https://hostREDACTED.uk) - result of checking service. 2021-02-26T11:01:10.825Z [INF] curl_handlerpool.cpp:ReturnHandler(110): Pool full: destroy the oldest handler 2021-02-26T11:01:10.825Z [ERR] s3fs.cpp:s3fs_exit_fuseloop(3321): Exiting FUSE event loop due to errors 2021-02-26T11:01:10.831Z [INF] s3fs.cpp:s3fs_destroy(3389): destroy ``` ``` [root@hostnameREDACTED001 s3fs-fuse]# goofys --stat-cache-ttl 1s --type-cache-ttl 1s --endpoint https://hostREDACTED.uk internal-public ~/test/internal-public/ [root@hostnameREDACTED001 s3fs-fuse]# ls ~/test/internal-public/ path test [root@hostnameREDACTED001 s3fs-fuse]# umount ~/test/internal-public/ [root@hostnameREDACTED001 s3fs-fuse]# ```
kerem commented 2026-03-04 01:49:07 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Feb 26, 2021):

Can you run with -o curldbg=body? This should show which RPC fails.

<!-- gh-comment-id:786613684 --> @gaul commented on GitHub (Feb 26, 2021): Can you run with `-o curldbg=body`? This should show which RPC fails.
kerem commented 2026-03-04 01:49:07 +03:00
Author
Owner
Copy link

@joanmarcriera commented on GitHub (Feb 26, 2021):

Here it goes :

root@REDACTED s3fs-fuse]# ./src/s3fs --version
Amazon Simple Storage Service File System V1.89 (commit:8c58ba8) with OpenSSL
Copyright (C) 2010 Randy Rizun <rrizun@gmail.com>
License GPL2: GNU GPL version 2 <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
[root@READCTED s3fs-fuse]#

 ]# ./src/s3fs internal-public ~/test/internal-public/ -o url=https://hostname-REDACTED.uk -o public_bucket=1 -o use_path_request_style -o ro -o nomultipart  -o nomixupload -o noxmlns -o norenameapi  -o no_check_certificate -o nosscache -o nodnscache -o listobjectsv2 -o curldbg=body -f -d  

2021-02-26T16:57:38.076Z [CRT] s3fs_logger.cpp:LowSetLogLevel(219): change debug level from [CRT] to [INF]
2021-02-26T16:57:38.076Z [INF]     s3fs.cpp:set_mountpoint_attribute(4020): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755)
2021-02-26T16:57:38.077Z [INF] curl.cpp:InitMimeType(436): Loaded mime information from /etc/mime.types
2021-02-26T16:57:38.077Z [WAN] s3fs.cpp:main(4961): Setting max_dirty_data to -1 when nomixupload is enabled
2021-02-26T16:57:38.077Z [INF] fdcache_stat.cpp:CheckCacheFileStatTopDir(79): The path to cache top dir is empty, thus not need to check permission.
2021-02-26T16:57:38.080Z [INF] s3fs.cpp:s3fs_init(3331): init v1.89(commit:8c58ba8) with OpenSSL
2021-02-26T16:57:38.080Z [INF] s3fs.cpp:s3fs_check_service(3447): check services.
2021-02-26T16:57:38.080Z [INF]       curl.cpp:CheckBucket(3364): check a bucket.
2021-02-26T16:57:38.080Z [INF]       curl_util.cpp:prepare_url(250): URL is https://hostREDACTED.uk/internal-public/?list-type=2
2021-02-26T16:57:38.080Z [INF]       curl_util.cpp:prepare_url(283): URL changed is https://hostREDACTED.uk/internal-public/?list-type=2
2021-02-26T16:57:38.080Z [INF]       curl.cpp:insertV4Headers(2640): computing signature [GET] [/] [list-type=2] []
2021-02-26T16:57:38.080Z [INF]       curl_util.cpp:url_to_host(327): url is https://hostREDACTED.uk
2021-02-26T16:57:38.084Z [CURL DBG] *   Trying 10.7.137.196...
2021-02-26T16:57:38.084Z [CURL DBG] * TCP_NODELAY set
2021-02-26T16:57:38.084Z [CURL DBG] * Connected to hostREDACTED.uk (10.7.137.196) port 443 (#0)
2021-02-26T16:57:38.092Z [CURL DBG] * successfully set certificate verify locations:
2021-02-26T16:57:38.092Z [CURL DBG] *   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
2021-02-26T16:57:38.092Z [CURL DBG] * TLSv1.3 (OUT), TLS handshake, Client hello (1):
2021-02-26T16:57:38.099Z [CURL DBG] * TLSv1.3 (IN), TLS handshake, Server hello (2):
2021-02-26T16:57:38.099Z [CURL DBG] * NPN, negotiated HTTP1.1
2021-02-26T16:57:38.099Z [CURL DBG] * TLSv1.2 (IN), TLS handshake, Certificate (11):
2021-02-26T16:57:38.100Z [CURL DBG] * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
2021-02-26T16:57:38.100Z [CURL DBG] * TLSv1.2 (IN), TLS handshake, Server finished (14):
2021-02-26T16:57:38.100Z [CURL DBG] * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
2021-02-26T16:57:38.100Z [CURL DBG] * TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
2021-02-26T16:57:38.100Z [CURL DBG] * TLSv1.2 (OUT), TLS handshake, Next protocol (67):
2021-02-26T16:57:38.100Z [CURL DBG] * TLSv1.2 (OUT), TLS handshake, Finished (20):
2021-02-26T16:57:38.103Z [CURL DBG] * TLSv1.2 (IN), TLS handshake, Finished (20):
2021-02-26T16:57:38.103Z [CURL DBG] * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
2021-02-26T16:57:38.103Z [CURL DBG] * Server certificate:
2021-02-26T16:57:38.103Z [CURL DBG] *  subject: jurisdictionC=GB; businessCategory=REDACTED; CN=hostREDACTED.uk
2021-02-26T16:57:38.103Z [CURL DBG] *  start date: Dec 11 11:32:30 2020 GMT
2021-02-26T16:57:38.103Z [CURL DBG] *  expire date: Dec 11 11:42:00 2021 GMT
2021-02-26T16:57:38.103Z [CURL DBG] *  subjectAltName: host "hostREDACTED.uk" matched cert's "hostREDACTED.uk"
2021-02-26T16:57:38.103Z [CURL DBG] *  issuer: C=NL; O=QuoVadis Trustlink B.V.; CN=QuoVadis Europe EV SSL CA G1
2021-02-26T16:57:38.103Z [CURL DBG] *  SSL certificate verify ok.
2021-02-26T16:57:38.103Z [CURL DBG] > GET /internal-public/?list-type=2 HTTP/1.1
2021-02-26T16:57:38.103Z [CURL DBG] > Host: hostREDACTED.uk
2021-02-26T16:57:38.103Z [CURL DBG] > User-Agent: s3fs/1.89 (commit hash 8c58ba8; OpenSSL)
2021-02-26T16:57:38.103Z [CURL DBG] > Accept: */*
2021-02-26T16:57:38.103Z [CURL DBG] > x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2021-02-26T16:57:38.103Z [CURL DBG] > x-amz-date: 20210226T165738Z
2021-02-26T16:57:38.103Z [CURL DBG] >
2021-02-26T16:57:38.110Z [CURL DBG] < HTTP/1.1 401 Unauthorized
2021-02-26T16:57:38.110Z [CURL DBG] < Cache-Control: no-cache, no-store, max-age=0, must-revalidate
2021-02-26T16:57:38.110Z [CURL DBG] < Content-Length: 0
2021-02-26T16:57:38.110Z [CURL DBG] < Date: Fri, 26 Feb 2021 16:57:38 GMT
2021-02-26T16:57:38.110Z [CURL DBG] < Expires: 0
2021-02-26T16:57:38.110Z [CURL DBG] < Pragma: no-cache
2021-02-26T16:57:38.110Z [CURL DBG] < Server: AmazonS3
2021-02-26T16:57:38.110Z [CURL DBG] < X-Amz-Bucket-Region: us-east-1
2021-02-26T16:57:38.110Z [CURL DBG] < X-Content-Type-Options: nosniff
2021-02-26T16:57:38.110Z [CURL DBG] < X-Frame-Options: DENY
2021-02-26T16:57:38.110Z [CURL DBG] < X-Xss-Protection: 1; mode=block
2021-02-26T16:57:38.110Z [CURL DBG] <
2021-02-26T16:57:38.110Z [CURL DBG] * Connection #0 to host hostREDACTED.uk left intact
2021-02-26T16:57:38.110Z [ERR] curl.cpp:RequestPerform(2353): HTTP response code 401, returning EIO. Body Text:
2021-02-26T16:57:38.110Z [ERR] curl.cpp:CheckBucket(3396): Check bucket failed, S3 response:
2021-02-26T16:57:38.110Z [CRT] s3fs.cpp:s3fs_check_service(3524): unable to connect(host=https://hostREDACTED.uk) - result of checking service.
2021-02-26T16:57:38.110Z [INF] curl_handlerpool.cpp:ReturnHandler(110): Pool full: destroy the oldest handler
2021-02-26T16:57:38.110Z [ERR] s3fs.cpp:s3fs_exit_fuseloop(3321): Exiting FUSE event loop due to errors

2021-02-26T16:57:38.114Z [INF] s3fs.cpp:s3fs_destroy(3389): destroy
[root@hostname s3fs-fuse]#

I'm always using the same bucket to check this, and is a public bucket.

Thank you very much for looking into this.

Best regards
Marc

<!-- gh-comment-id:786771291 --> @joanmarcriera commented on GitHub (Feb 26, 2021): Here it goes : ``` root@REDACTED s3fs-fuse]# ./src/s3fs --version Amazon Simple Storage Service File System V1.89 (commit:8c58ba8) with OpenSSL Copyright (C) 2010 Randy Rizun <rrizun@gmail.com> License GPL2: GNU GPL version 2 <https://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. [root@READCTED s3fs-fuse]# ``` ``` ]# ./src/s3fs internal-public ~/test/internal-public/ -o url=https://hostname-REDACTED.uk -o public_bucket=1 -o use_path_request_style -o ro -o nomultipart -o nomixupload -o noxmlns -o norenameapi -o no_check_certificate -o nosscache -o nodnscache -o listobjectsv2 -o curldbg=body -f -d ``` ``` 2021-02-26T16:57:38.076Z [CRT] s3fs_logger.cpp:LowSetLogLevel(219): change debug level from [CRT] to [INF] 2021-02-26T16:57:38.076Z [INF] s3fs.cpp:set_mountpoint_attribute(4020): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755) 2021-02-26T16:57:38.077Z [INF] curl.cpp:InitMimeType(436): Loaded mime information from /etc/mime.types 2021-02-26T16:57:38.077Z [WAN] s3fs.cpp:main(4961): Setting max_dirty_data to -1 when nomixupload is enabled 2021-02-26T16:57:38.077Z [INF] fdcache_stat.cpp:CheckCacheFileStatTopDir(79): The path to cache top dir is empty, thus not need to check permission. 2021-02-26T16:57:38.080Z [INF] s3fs.cpp:s3fs_init(3331): init v1.89(commit:8c58ba8) with OpenSSL 2021-02-26T16:57:38.080Z [INF] s3fs.cpp:s3fs_check_service(3447): check services. 2021-02-26T16:57:38.080Z [INF] curl.cpp:CheckBucket(3364): check a bucket. 2021-02-26T16:57:38.080Z [INF] curl_util.cpp:prepare_url(250): URL is https://hostREDACTED.uk/internal-public/?list-type=2 2021-02-26T16:57:38.080Z [INF] curl_util.cpp:prepare_url(283): URL changed is https://hostREDACTED.uk/internal-public/?list-type=2 2021-02-26T16:57:38.080Z [INF] curl.cpp:insertV4Headers(2640): computing signature [GET] [/] [list-type=2] [] 2021-02-26T16:57:38.080Z [INF] curl_util.cpp:url_to_host(327): url is https://hostREDACTED.uk 2021-02-26T16:57:38.084Z [CURL DBG] * Trying 10.7.137.196... 2021-02-26T16:57:38.084Z [CURL DBG] * TCP_NODELAY set 2021-02-26T16:57:38.084Z [CURL DBG] * Connected to hostREDACTED.uk (10.7.137.196) port 443 (#0) 2021-02-26T16:57:38.092Z [CURL DBG] * successfully set certificate verify locations: 2021-02-26T16:57:38.092Z [CURL DBG] * CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: none 2021-02-26T16:57:38.092Z [CURL DBG] * TLSv1.3 (OUT), TLS handshake, Client hello (1): 2021-02-26T16:57:38.099Z [CURL DBG] * TLSv1.3 (IN), TLS handshake, Server hello (2): 2021-02-26T16:57:38.099Z [CURL DBG] * NPN, negotiated HTTP1.1 2021-02-26T16:57:38.099Z [CURL DBG] * TLSv1.2 (IN), TLS handshake, Certificate (11): 2021-02-26T16:57:38.100Z [CURL DBG] * TLSv1.2 (IN), TLS handshake, Server key exchange (12): 2021-02-26T16:57:38.100Z [CURL DBG] * TLSv1.2 (IN), TLS handshake, Server finished (14): 2021-02-26T16:57:38.100Z [CURL DBG] * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): 2021-02-26T16:57:38.100Z [CURL DBG] * TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1): 2021-02-26T16:57:38.100Z [CURL DBG] * TLSv1.2 (OUT), TLS handshake, Next protocol (67): 2021-02-26T16:57:38.100Z [CURL DBG] * TLSv1.2 (OUT), TLS handshake, Finished (20): 2021-02-26T16:57:38.103Z [CURL DBG] * TLSv1.2 (IN), TLS handshake, Finished (20): 2021-02-26T16:57:38.103Z [CURL DBG] * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256 2021-02-26T16:57:38.103Z [CURL DBG] * Server certificate: 2021-02-26T16:57:38.103Z [CURL DBG] * subject: jurisdictionC=GB; businessCategory=REDACTED; CN=hostREDACTED.uk 2021-02-26T16:57:38.103Z [CURL DBG] * start date: Dec 11 11:32:30 2020 GMT 2021-02-26T16:57:38.103Z [CURL DBG] * expire date: Dec 11 11:42:00 2021 GMT 2021-02-26T16:57:38.103Z [CURL DBG] * subjectAltName: host "hostREDACTED.uk" matched cert's "hostREDACTED.uk" 2021-02-26T16:57:38.103Z [CURL DBG] * issuer: C=NL; O=QuoVadis Trustlink B.V.; CN=QuoVadis Europe EV SSL CA G1 2021-02-26T16:57:38.103Z [CURL DBG] * SSL certificate verify ok. 2021-02-26T16:57:38.103Z [CURL DBG] > GET /internal-public/?list-type=2 HTTP/1.1 2021-02-26T16:57:38.103Z [CURL DBG] > Host: hostREDACTED.uk 2021-02-26T16:57:38.103Z [CURL DBG] > User-Agent: s3fs/1.89 (commit hash 8c58ba8; OpenSSL) 2021-02-26T16:57:38.103Z [CURL DBG] > Accept: */* 2021-02-26T16:57:38.103Z [CURL DBG] > x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 2021-02-26T16:57:38.103Z [CURL DBG] > x-amz-date: 20210226T165738Z 2021-02-26T16:57:38.103Z [CURL DBG] > 2021-02-26T16:57:38.110Z [CURL DBG] < HTTP/1.1 401 Unauthorized 2021-02-26T16:57:38.110Z [CURL DBG] < Cache-Control: no-cache, no-store, max-age=0, must-revalidate 2021-02-26T16:57:38.110Z [CURL DBG] < Content-Length: 0 2021-02-26T16:57:38.110Z [CURL DBG] < Date: Fri, 26 Feb 2021 16:57:38 GMT 2021-02-26T16:57:38.110Z [CURL DBG] < Expires: 0 2021-02-26T16:57:38.110Z [CURL DBG] < Pragma: no-cache 2021-02-26T16:57:38.110Z [CURL DBG] < Server: AmazonS3 2021-02-26T16:57:38.110Z [CURL DBG] < X-Amz-Bucket-Region: us-east-1 2021-02-26T16:57:38.110Z [CURL DBG] < X-Content-Type-Options: nosniff 2021-02-26T16:57:38.110Z [CURL DBG] < X-Frame-Options: DENY 2021-02-26T16:57:38.110Z [CURL DBG] < X-Xss-Protection: 1; mode=block 2021-02-26T16:57:38.110Z [CURL DBG] < 2021-02-26T16:57:38.110Z [CURL DBG] * Connection #0 to host hostREDACTED.uk left intact 2021-02-26T16:57:38.110Z [ERR] curl.cpp:RequestPerform(2353): HTTP response code 401, returning EIO. Body Text: 2021-02-26T16:57:38.110Z [ERR] curl.cpp:CheckBucket(3396): Check bucket failed, S3 response: 2021-02-26T16:57:38.110Z [CRT] s3fs.cpp:s3fs_check_service(3524): unable to connect(host=https://hostREDACTED.uk) - result of checking service. 2021-02-26T16:57:38.110Z [INF] curl_handlerpool.cpp:ReturnHandler(110): Pool full: destroy the oldest handler 2021-02-26T16:57:38.110Z [ERR] s3fs.cpp:s3fs_exit_fuseloop(3321): Exiting FUSE event loop due to errors 2021-02-26T16:57:38.114Z [INF] s3fs.cpp:s3fs_destroy(3389): destroy [root@hostname s3fs-fuse]# ``` I'm always using the same bucket to check this, and is a public bucket. Thank you very much for looking into this. Best regards Marc
kerem commented 2026-03-04 01:49:07 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Feb 27, 2021):

Can you run goofys with --debug_s3 to see what it does? I'm not sure if it has the exact same check bucket logic.

<!-- gh-comment-id:786992681 --> @gaul commented on GitHub (Feb 27, 2021): Can you run goofys with `--debug_s3` to see what it does? I'm not sure if it has the exact same check bucket logic.
kerem commented 2026-03-04 01:49:07 +03:00
Author
Owner
Copy link

@joanmarcriera commented on GitHub (Feb 28, 2021):

I hope this helps.

[root@REDACTED ~]# goofys -f --debug_s3 --stat-cache-ttl 1s --type-cache-ttl 1s --endpoint https://REDACTED.uk internal-public ~/test/internal-public/
2021/02/28 15:32:05.886873 s3.DEBUG HEAD https://REDACTED.uk/internal-public = 200 [us-east-1]
2021/02/28 15:32:05.886937 s3.INFO anonymous bucket detected
2021/02/28 15:32:05.887178 s3.DEBUG DEBUG: Request s3/HeadObject Details:
---[ REQUEST POST-SIGN ]-----------------------------
HEAD /internal-public/i2o9x4ktertbgrx2k3iel80gje98gde1 HTTP/1.1
Host: REDACTED.uk
User-Agent: aws-sdk-go/1.17.13 (go1.13.1; linux; amd64)


-----------------------------------------------------
2021/02/28 15:32:05.904612 s3.DEBUG DEBUG: Response s3/HeadObject Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 404 Not Found
Connection: close
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: application/json;charset=UTF-8
Date: Sun, 28 Feb 2021 15:32:05 GMT
Expires: 0
Pragma: no-cache
Set-Cookie: JSESSIONID=9A40591C9571470E8B9EA0E927FD646E; Path=/; HttpOnly
X-Content-Type-Options: nosniff
X-Fire-Request: 61c6ca24-47fd-4f31-8b22-302406c95041
X-Frame-Options: DENY
X-Xss-Protection: 1; mode=block


-----------------------------------------------------
2021/02/28 15:32:05.904693 s3.DEBUG DEBUG: Validate Response s3/HeadObject failed, not retrying, error NotFound: Not Found
        status code: 404, request id: , host id:
2021/02/28 15:32:05.905038 s3.DEBUG DEBUG: Request s3/ListMultipartUploads Details:
---[ REQUEST POST-SIGN ]-----------------------------
GET /internal-public?uploads= HTTP/1.1
Host: REDACTED.uk
User-Agent: aws-sdk-go/1.17.13 (go1.13.1; linux; amd64)
Accept-Encoding: identity


-----------------------------------------------------
2021/02/28 15:32:05.905144 main.INFO File system has been successfully mounted.
2021/02/28 15:32:05.910587 s3.DEBUG DEBUG: Response s3/ListMultipartUploads Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 404 Not Found
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Date: Sun, 28 Feb 2021 15:32:05 GMT
Expires: 0
Pragma: no-cache
Set-Cookie: JSESSIONID=A6C7CF614400B7169FD7FBBDC523305F; Path=/; HttpOnly
X-Content-Type-Options: nosniff
X-Fire-Request: 9a2e5906-eda3-4b1e-beed-9e56c42e0519
X-Frame-Options: DENY
X-Xss-Protection: 1; mode=block
Content-Length: 0


-----------------------------------------------------
2021/02/28 15:32:05.910664 s3.DEBUG DEBUG: Validate Response s3/ListMultipartUploads failed, not retrying, error NotFound: Not Found
        status code: 404, request id: , host id:

^Z
[1]+  Stopped                 goofys -f --debug_s3 --stat-cache-ttl 1s --type-cache-ttl 1s --endpoint https://REDACTEDuk internal-public ~/test/internal-public/
[root@REDACTED ~]# bg
[1]+ goofys -f --debug_s3 --stat-cache-ttl 1s --type-cache-ttl 1s --endpoint https://REDACTED.uk internal-public ~/test/internal-public/ &
[root@REDACTED ~]# ls ~/test/internal-public/
2021/02/28 15:32:21.458451 s3.DEBUG DEBUG: Request s3/ListObjectsV2 Details:
---[ REQUEST POST-SIGN ]-----------------------------
GET /internal-public?delimiter=%2F&list-type=2&prefix= HTTP/1.1
Host: REDACTED.uk
User-Agent: aws-sdk-go/1.17.13 (go1.13.1; linux; amd64)
Accept-Encoding: identity


-----------------------------------------------------
2021/02/28 15:32:21.469582 s3.DEBUG DEBUG: Response s3/ListObjectsV2 Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Content-Length: 352
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: application/xml;charset=UTF-8
Date: Sun, 28 Feb 2021 15:32:20 GMT
Expires: 0
Pragma: no-cache
Server: Fire-S3-Gateway
Set-Cookie: JSESSIONID=85EC907E0CE8CF421BFD8776DF58F635; Path=/; HttpOnly
X-Amz-Bucket-Region: us-east-1
X-Content-Type-Options: nosniff
X-Fire-Request: 87c64d48-fa2e-48a9-9984-3092f0220ac3
X-Frame-Options: DENY
X-Xss-Protection: 1; mode=block


-----------------------------------------------------
2021/02/28 15:32:21.469683 s3.DEBUG &{[path/ test/] [] 0xc000062a48 false : }
path  test
[root@REDACTED ~]# umount ~/test/internal-public/
2021/02/28 15:32:26.851748 main.INFO Successfully exiting.
[root@REDACTED ~]#
<!-- gh-comment-id:787471007 --> @joanmarcriera commented on GitHub (Feb 28, 2021): I hope this helps. ``` [root@REDACTED ~]# goofys -f --debug_s3 --stat-cache-ttl 1s --type-cache-ttl 1s --endpoint https://REDACTED.uk internal-public ~/test/internal-public/ 2021/02/28 15:32:05.886873 s3.DEBUG HEAD https://REDACTED.uk/internal-public = 200 [us-east-1] 2021/02/28 15:32:05.886937 s3.INFO anonymous bucket detected 2021/02/28 15:32:05.887178 s3.DEBUG DEBUG: Request s3/HeadObject Details: ---[ REQUEST POST-SIGN ]----------------------------- HEAD /internal-public/i2o9x4ktertbgrx2k3iel80gje98gde1 HTTP/1.1 Host: REDACTED.uk User-Agent: aws-sdk-go/1.17.13 (go1.13.1; linux; amd64) ----------------------------------------------------- 2021/02/28 15:32:05.904612 s3.DEBUG DEBUG: Response s3/HeadObject Details: ---[ RESPONSE ]-------------------------------------- HTTP/1.1 404 Not Found Connection: close Cache-Control: no-cache, no-store, max-age=0, must-revalidate Content-Type: application/json;charset=UTF-8 Date: Sun, 28 Feb 2021 15:32:05 GMT Expires: 0 Pragma: no-cache Set-Cookie: JSESSIONID=9A40591C9571470E8B9EA0E927FD646E; Path=/; HttpOnly X-Content-Type-Options: nosniff X-Fire-Request: 61c6ca24-47fd-4f31-8b22-302406c95041 X-Frame-Options: DENY X-Xss-Protection: 1; mode=block ----------------------------------------------------- 2021/02/28 15:32:05.904693 s3.DEBUG DEBUG: Validate Response s3/HeadObject failed, not retrying, error NotFound: Not Found status code: 404, request id: , host id: 2021/02/28 15:32:05.905038 s3.DEBUG DEBUG: Request s3/ListMultipartUploads Details: ---[ REQUEST POST-SIGN ]----------------------------- GET /internal-public?uploads= HTTP/1.1 Host: REDACTED.uk User-Agent: aws-sdk-go/1.17.13 (go1.13.1; linux; amd64) Accept-Encoding: identity ----------------------------------------------------- 2021/02/28 15:32:05.905144 main.INFO File system has been successfully mounted. 2021/02/28 15:32:05.910587 s3.DEBUG DEBUG: Response s3/ListMultipartUploads Details: ---[ RESPONSE ]-------------------------------------- HTTP/1.1 404 Not Found Cache-Control: no-cache, no-store, max-age=0, must-revalidate Date: Sun, 28 Feb 2021 15:32:05 GMT Expires: 0 Pragma: no-cache Set-Cookie: JSESSIONID=A6C7CF614400B7169FD7FBBDC523305F; Path=/; HttpOnly X-Content-Type-Options: nosniff X-Fire-Request: 9a2e5906-eda3-4b1e-beed-9e56c42e0519 X-Frame-Options: DENY X-Xss-Protection: 1; mode=block Content-Length: 0 ----------------------------------------------------- 2021/02/28 15:32:05.910664 s3.DEBUG DEBUG: Validate Response s3/ListMultipartUploads failed, not retrying, error NotFound: Not Found status code: 404, request id: , host id: ^Z [1]+ Stopped goofys -f --debug_s3 --stat-cache-ttl 1s --type-cache-ttl 1s --endpoint https://REDACTEDuk internal-public ~/test/internal-public/ [root@REDACTED ~]# bg [1]+ goofys -f --debug_s3 --stat-cache-ttl 1s --type-cache-ttl 1s --endpoint https://REDACTED.uk internal-public ~/test/internal-public/ & [root@REDACTED ~]# ls ~/test/internal-public/ 2021/02/28 15:32:21.458451 s3.DEBUG DEBUG: Request s3/ListObjectsV2 Details: ---[ REQUEST POST-SIGN ]----------------------------- GET /internal-public?delimiter=%2F&list-type=2&prefix= HTTP/1.1 Host: REDACTED.uk User-Agent: aws-sdk-go/1.17.13 (go1.13.1; linux; amd64) Accept-Encoding: identity ----------------------------------------------------- 2021/02/28 15:32:21.469582 s3.DEBUG DEBUG: Response s3/ListObjectsV2 Details: ---[ RESPONSE ]-------------------------------------- HTTP/1.1 200 OK Content-Length: 352 Cache-Control: no-cache, no-store, max-age=0, must-revalidate Content-Type: application/xml;charset=UTF-8 Date: Sun, 28 Feb 2021 15:32:20 GMT Expires: 0 Pragma: no-cache Server: Fire-S3-Gateway Set-Cookie: JSESSIONID=85EC907E0CE8CF421BFD8776DF58F635; Path=/; HttpOnly X-Amz-Bucket-Region: us-east-1 X-Content-Type-Options: nosniff X-Fire-Request: 87c64d48-fa2e-48a9-9984-3092f0220ac3 X-Frame-Options: DENY X-Xss-Protection: 1; mode=block ----------------------------------------------------- 2021/02/28 15:32:21.469683 s3.DEBUG &{[path/ test/] [] 0xc000062a48 false : } path test [root@REDACTED ~]# umount ~/test/internal-public/ 2021/02/28 15:32:26.851748 main.INFO Successfully exiting. [root@REDACTED ~]# ```
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
v1.97
v1.96
v1.95
v1.94
v1.93
v1.92
v1.91
v1.90
v1.89
v1.88
v1.87
v1.86
v1.85
v1.84
v1.83
v1.82
v1.81
v1.80
v1.79
v1.78
Pre-v1.78
v1.77
v1.76
v1.75
v1.74
Labels
Clear labels   
bug

   
bug

   
dataloss

   
duplicate

   
enhancement

   
feature request

   
help wanted

   
invalid

   
need info

   
performance

   
pull-request

Mirrored from GitHub Pull Request

  
question

   
question

   
testing
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#823
No description provided.