[GH-ISSUE #2051] Improve performance of readdir #1037

New issue

Open

opened 2026-03-04 01:50:51 +03:00 by kerem · 1 comment

kerem commented

2026-03-04 01:50:51 +03:00

Owner

Originally created by @huntersman on GitHub (Nov 4, 2022).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2051

Version of s3fs being used (s3fs --version)

1.91

Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse)

2.9.2

Kernel information (uname -r)

5.4.211-1.el7.elrepo.x86_64

GNU/Linux Distribution, if applicable (cat /etc/os-release)

CentOS 7

s3fs command line used, if applicable

s3fs bucket /mount/bucket -o passwd_file=${HOME}/.passwd-s3fs -o url=http://ip -o use_path_request_style -o noxmlns -o dbglevel=error -o default_acl=public-read -o logfile=/var/log/s3fs.log -o allow_other -o multireq_max=1000

Details about issue

It takes about 30 seconds for ls 100 thousands files, maybe it could be faster.

Originally created by @huntersman on GitHub (Nov 4, 2022). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2051 #### Version of s3fs being used (s3fs --version) 1.91 #### Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse) 2.9.2 #### Kernel information (uname -r) 5.4.211-1.el7.elrepo.x86_64 #### GNU/Linux Distribution, if applicable (cat /etc/os-release) CentOS 7 #### s3fs command line used, if applicable ``` s3fs bucket /mount/bucket -o passwd_file=${HOME}/.passwd-s3fs -o url=http://ip -o use_path_request_style -o noxmlns -o dbglevel=error -o default_acl=public-read -o logfile=/var/log/s3fs.log -o allow_other -o multireq_max=1000 ``` ### Details about issue It takes about 30 seconds for `ls` 100 thousands files, maybe it could be faster.

kerem added the

performance

label

2026-03-04 01:50:51 +03:00

kerem commented

2026-03-04 01:50:52 +03:00

Author

Owner

@gaul commented on GitHub (May 30, 2023):

s3fs 1.92 reduces readdir S3 operations and thus performance by 33%. However something like #1482 is necessary for better performance.

@gaul commented on GitHub (May 30, 2023): s3fs 1.92 reduces `readdir` S3 operations and thus performance by 33%. However something like #1482 is necessary for better performance.