[GH-ISSUE #616] 50x slowdown compared to s3cmd #350

New issue

Closed

opened 2026-03-04 01:44:38 +03:00 by kerem · 3 comments

kerem commented

2026-03-04 01:44:38 +03:00

Owner

Originally created by @tachim on GitHub (Jun 6, 2017).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/616

Additional Information

The following information is very important in order to help us to help you. Omission of the following details may delay your support request or receive no attention at all.

Version of s3fs being used (s3fs --version)

Amazon Simple Storage Service File System V1.79(commit:unknown) with GnuTLS(gcrypt)

Version of fuse being used (pkg-config --modversion fuse)
ii fuse 2.9.4-1ubuntu3 amd64 Filesystem in Userspace
System information (uname -a)
Linux DL2 4.4.0-66-generic #87-Ubuntu SMP Fri Mar 3 15:29:05 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Distro (cat /etc/issue)
Ubuntu 16.04 LTS
s3fs command line used (if applicable)

s3fs <our_bucket_redacted> /s3 -o rw,use_cache=/data3/s3cache,endpoint=us-west-2,storage_class=reduced_redundancy,allow_other,dev,suid

/etc/fstab entry (if applicable):

s3fs#<our_bucket_redacted> /s3 fuse use_cache=/data3/s3cache,endpoint=us-west-2,storage_class=reduced_redundancy,allow_other 0 0

s3fs syslog messages (grep s3fs /var/log/syslog, or s3fs outputs)
if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages

Details about issue

When we access a file in S3FS to do an operation like md5sum, the download speed observed with bmon is 50x slower than the speed we would get downloading the file with s3cmd (500KB/s vs 50MB/s). Any ideas if there's some configuration change that can match that perf?

Originally created by @tachim on GitHub (Jun 6, 2017). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/616 #### Additional Information _The following information is very important in order to help us to help you. Omission of the following details may delay your support request or receive no attention at all._ - Version of s3fs being used (s3fs --version) `Amazon Simple Storage Service File System V1.79(commit:unknown) with GnuTLS(gcrypt)` - Version of fuse being used (pkg-config --modversion fuse) `ii fuse 2.9.4-1ubuntu3 amd64 Filesystem in Userspace` - System information (uname -a) `Linux DL2 4.4.0-66-generic #87-Ubuntu SMP Fri Mar 3 15:29:05 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux` - Distro (cat /etc/issue) `Ubuntu 16.04 LTS` - s3fs command line used (if applicable) ``` s3fs <our_bucket_redacted> /s3 -o rw,use_cache=/data3/s3cache,endpoint=us-west-2,storage_class=reduced_redundancy,allow_other,dev,suid ``` - /etc/fstab entry (if applicable): ``` s3fs#<our_bucket_redacted> /s3 fuse use_cache=/data3/s3cache,endpoint=us-west-2,storage_class=reduced_redundancy,allow_other 0 0 ``` - s3fs syslog messages (grep s3fs /var/log/syslog, or s3fs outputs) _if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages_ ``` ``` #### Details about issue When we access a file in S3FS to do an operation like md5sum, the download speed observed with bmon is 50x slower than the speed we would get downloading the file with s3cmd (500KB/s vs 50MB/s). Any ideas if there's some configuration change that can match that perf?

kerem closed this issue

2026-03-04 01:44:38 +03:00

kerem commented

2026-03-04 01:44:39 +03:00

Author

Owner

@danielpflood commented on GitHub (Oct 5, 2017):

I would like to know also

@danielpflood commented on GitHub (Oct 5, 2017): I would like to know also

kerem commented

2026-03-04 01:44:39 +03:00

Author

Owner

@gaul commented on GitHub (Jan 24, 2019):

Could you test again against master? It includes some optimizations that may help.

@gaul commented on GitHub (Jan 24, 2019): Could you test again against master? It includes some optimizations that may help.

kerem commented

2026-03-04 01:44:39 +03:00

Author

Owner

@gaul commented on GitHub (Jun 25, 2019):

Closing due to inactivity.

@gaul commented on GitHub (Jun 25, 2019): Closing due to inactivity.

kerem referenced this issue

2026-03-04 01:54:08 +03:00

[PR #350] [MERGED] Changed cache out logic for stat - #340 #1472