[GH-ISSUE #1251] list_bucket error extremely often #670

New issue

Closed

opened 2026-03-04 01:47:44 +03:00 by kerem · 1 comment

kerem commented

2026-03-04 01:47:44 +03:00

Owner

Originally created by @dfslezak on GitHub (Mar 17, 2020).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1251

I am getting list_bucket error extremely often. Any ideas?

Version of s3fs being used (s3fs --version)

1.86

Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse)

Version: 2.9.7-1ubuntu1

GNU/Linux Distribution, if applicable (cat /etc/os-release)

Ubuntu 18.04.3

s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)

> GET /?delimiter=/&max-keys=2&prefix=mydir/ HTTP/1.1
Mar 17 02:36:50 ip-10-2-20-87 s3fs[15049]: > host: <<<<<myhost>>>>>>>>>
Mar 17 02:36:50 ip-10-2-20-87 s3fs[15049]: > User-Agent: s3fs/1.86 (commit hash b72f4b4; OpenSSL)
Mar 17 02:36:50 ip-10-2-20-87 s3fs[15049]: > Accept: */*
Mar 17 02:36:50 ip-10-2-20-87 s3fs[15049]: > Authorization: AWS4-HMAC-SHA256 Credential=ASIAZVTXMPK5AMPJVLUO/20200317/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=85fd824113035e3978544f342a721d3c05b04d21c1ce3f85e0384e6fa18ff3da
Mar 17 02:36:50 ip-10-2-20-87 s3fs[15049]: > x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Mar 17 02:36:50 ip-10-2-20-87 s3fs[15049]: > x-amz-date: 20200317T023650Z
Mar 17 02:36:50 ip-10-2-20-87 s3fs[15049]: > x-amz-security-token: IQoJb3JpZ2luX2VjELL//////////wEaCXVzLWVhc3QtMSJHMEUCICvlA5ZWtn4yPe3eCWjCjHZ8pKisvx9FaqrDPt6e7MW0AiEA3107KjSkLIN4bn9aqxg7nZyYMPcI6mH6cAt3MPCFTigqvQMIm///////////ARABGgw2NjQ4OTY1Njc5OTQiDCn4y7GQLnkp7lubzSqRA7Dz75k9I5DA8UPdQ3T866c7fXnsxJhe3dYX5rCZLC9VN66e9eDBF7PNhscRD52tv2IqkBNm5AJHjjdpSKvG2VsazFK4TNd2rvTCi4nBA53FESAwsb3uiXC2EZ624sQx0YKgNVMcmPhcyho12GPp5aRqDldQBsRf68eEE+/4eW1lG+cVJAsvEEVvEvd9A5L/IpkHv3PdctR4Spo21U3jmxGC0BM8dYAJwAG4MjSF6Sms8DRKVboqSvZ2cA0JtdmJON5aPTo1hdolQ8ORbABS4xnXNXJEkWZp2mewBT5d5HYvn7vk0+ByUdzGQ/HOsdm1y0FfALZ7XSW3Jy+9Zkl5Sd4UEs+jVHm5vYzs8at/utHqeFDHL4PWep4B09RiHtVh1BviBBCbSgOwh8DOgubyAhHLbgbdX1j1YBwGM9CesATB//nyVaXActBOQDM4j4NJMUF5BevrdELGhNkvnr6ZnbN42yQ5q6R/0IrxzY++eZiXb+8hipxWZ70MeMKeMp2DYpHOnPRmbxJpeMUhqFcX1Hf1MLzMwPMFOusBik50citegOoe7Zv+8si7ivcvKCCud5wi8MTvvt0eqEV6lZZbPRPWRvsbgzhjao1rZM3J+sOUQdGRVvyX29kNGgqFWst3g8hmjoym4AZpf08zCIGnMkNSJPhs+28xyv3dIwsuL9ujzIV+fPQene49lcRkwyYRrCHW3taeJDtcXNh20o9Rw1WssSsrg7lfUjVsUIsC1bLVuCACfnUdsYHWeL2rWdVcoMeNvkrlQwCE6qqe4d18KqYR9JsPZSPmcnxoZ1PZ7z0otpuoMijszPAld7VWm3TxruWmtMR4E2xCkwPkgAQk8Ilmzteuuw==
Mar 17 02:36:50 ip-10-2-20-87 s3fs[15049]: >
Mar 17 02:36:50 ip-10-2-20-87 s3fs[15049]: curl.cpp:RequestPerform(2459): HTTP response code 500, returning EIO. Body Text: <?xml version="1.0" encoding="UTF-8"?>#012<Error><Code>InternalError</Code><Message>We encountered an internal error. Please try again.</Message><RequestId>37D33746F84A73A0</RequestId><HostId>iDZHQ3YurlPO3Yv7InuS6ZSqgOvEef7yqOK5Jslq7BgB8bqT4h+lSlFpKfTxAdZLkaJUQ4tZf5s=</HostId></Error>
Mar 17 02:36:50 ip-10-2-20-87 s3fs[15049]: s3fs.cpp:list_bucket(2632): ListBucketRequest returns with error.
Mar 17 02:36:50 ip-10-2-20-87 s3fs[15049]: s3fs.cpp:directory_empty(1178): list_bucket returns error.

Originally created by @dfslezak on GitHub (Mar 17, 2020). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1251 I am getting list_bucket error extremely often. Any ideas? #### Version of s3fs being used (s3fs --version) 1.86 #### Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse) Version: 2.9.7-1ubuntu1 #### GNU/Linux Distribution, if applicable (cat /etc/os-release) Ubuntu 18.04.3 #### s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs) ``` > GET /?delimiter=/&max-keys=2&prefix=mydir/ HTTP/1.1 Mar 17 02:36:50 ip-10-2-20-87 s3fs[15049]: > host: <<<<<myhost>>>>>>>>> Mar 17 02:36:50 ip-10-2-20-87 s3fs[15049]: > User-Agent: s3fs/1.86 (commit hash b72f4b4; OpenSSL) Mar 17 02:36:50 ip-10-2-20-87 s3fs[15049]: > Accept: */* Mar 17 02:36:50 ip-10-2-20-87 s3fs[15049]: > Authorization: AWS4-HMAC-SHA256 Credential=ASIAZVTXMPK5AMPJVLUO/20200317/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=85fd824113035e3978544f342a721d3c05b04d21c1ce3f85e0384e6fa18ff3da Mar 17 02:36:50 ip-10-2-20-87 s3fs[15049]: > x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 Mar 17 02:36:50 ip-10-2-20-87 s3fs[15049]: > x-amz-date: 20200317T023650Z Mar 17 02:36:50 ip-10-2-20-87 s3fs[15049]: > x-amz-security-token: IQoJb3JpZ2luX2VjELL//////////wEaCXVzLWVhc3QtMSJHMEUCICvlA5ZWtn4yPe3eCWjCjHZ8pKisvx9FaqrDPt6e7MW0AiEA3107KjSkLIN4bn9aqxg7nZyYMPcI6mH6cAt3MPCFTigqvQMIm///////////ARABGgw2NjQ4OTY1Njc5OTQiDCn4y7GQLnkp7lubzSqRA7Dz75k9I5DA8UPdQ3T866c7fXnsxJhe3dYX5rCZLC9VN66e9eDBF7PNhscRD52tv2IqkBNm5AJHjjdpSKvG2VsazFK4TNd2rvTCi4nBA53FESAwsb3uiXC2EZ624sQx0YKgNVMcmPhcyho12GPp5aRqDldQBsRf68eEE+/4eW1lG+cVJAsvEEVvEvd9A5L/IpkHv3PdctR4Spo21U3jmxGC0BM8dYAJwAG4MjSF6Sms8DRKVboqSvZ2cA0JtdmJON5aPTo1hdolQ8ORbABS4xnXNXJEkWZp2mewBT5d5HYvn7vk0+ByUdzGQ/HOsdm1y0FfALZ7XSW3Jy+9Zkl5Sd4UEs+jVHm5vYzs8at/utHqeFDHL4PWep4B09RiHtVh1BviBBCbSgOwh8DOgubyAhHLbgbdX1j1YBwGM9CesATB//nyVaXActBOQDM4j4NJMUF5BevrdELGhNkvnr6ZnbN42yQ5q6R/0IrxzY++eZiXb+8hipxWZ70MeMKeMp2DYpHOnPRmbxJpeMUhqFcX1Hf1MLzMwPMFOusBik50citegOoe7Zv+8si7ivcvKCCud5wi8MTvvt0eqEV6lZZbPRPWRvsbgzhjao1rZM3J+sOUQdGRVvyX29kNGgqFWst3g8hmjoym4AZpf08zCIGnMkNSJPhs+28xyv3dIwsuL9ujzIV+fPQene49lcRkwyYRrCHW3taeJDtcXNh20o9Rw1WssSsrg7lfUjVsUIsC1bLVuCACfnUdsYHWeL2rWdVcoMeNvkrlQwCE6qqe4d18KqYR9JsPZSPmcnxoZ1PZ7z0otpuoMijszPAld7VWm3TxruWmtMR4E2xCkwPkgAQk8Ilmzteuuw== Mar 17 02:36:50 ip-10-2-20-87 s3fs[15049]: > Mar 17 02:36:50 ip-10-2-20-87 s3fs[15049]: curl.cpp:RequestPerform(2459): HTTP response code 500, returning EIO. Body Text: <?xml version="1.0" encoding="UTF-8"?>#012<Error><Code>InternalError</Code><Message>We encountered an internal error. Please try again.</Message><RequestId>37D33746F84A73A0</RequestId><HostId>iDZHQ3YurlPO3Yv7InuS6ZSqgOvEef7yqOK5Jslq7BgB8bqT4h+lSlFpKfTxAdZLkaJUQ4tZf5s=</HostId></Error> Mar 17 02:36:50 ip-10-2-20-87 s3fs[15049]: s3fs.cpp:list_bucket(2632): ListBucketRequest returns with error. Mar 17 02:36:50 ip-10-2-20-87 s3fs[15049]: s3fs.cpp:directory_empty(1178): list_bucket returns error. ```

kerem closed this issue

2026-03-04 01:47:45 +03:00

kerem commented

2026-03-04 01:47:45 +03:00

Author

Owner

@gaul commented on GitHub (Mar 25, 2020):

What kind of S3 server do you use? List operations tend to be more expensive than others so it is possible the server is overloaded.

@gaul commented on GitHub (Mar 25, 2020): What kind of S3 server do you use? List operations tend to be more expensive than others so it is possible the server is overloaded.

kerem referenced this issue

2026-03-04 02:01:13 +03:00

[PR #670] [MERGED] Fixed a bug in S3fsCurl::LocateBundle #1595