[GH-ISSUE #1482] Option to not use head requests #779

New issue

Open

opened 2026-03-04 01:48:42 +03:00 by kerem · 3 comments

kerem commented

2026-03-04 01:48:42 +03:00

Owner

Originally created by @warmconnect on GitHub (Nov 21, 2020).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1482

Additional Information

Version of s3fs being used (s3fs --version)

V1.87

Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse)

2.9.2

Kernel information (uname -r)

4.14.181-142.260.amzn2.x86_64

GNU/Linux Distribution, if applicable (cat /etc/os-release)

cat /etc/os-release

NAME="Amazon Linux"
VERSION="2"
ID="amzn"
ID_LIKE="centos rhel fedora"
VERSION_ID="2"
PRETTY_NAME="Amazon Linux 2"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3⭕amazon:amazon_linux:2"
HOME_URL="https://amazonlinux.com/"

Details about issue

Feature request: the per object head request slows s3fs considerably and is also not compatible with native s3 tools. Can we have an option to not issue the head requests? I.E. For read requests pretend the filesystem was populated by a native s3 tool like aws s3 sync in which case the x-amz headers will not be present. For write requests s3fs should not bother to write the headers.

Alternatively, could you point out where we should look to block s3fs from writing and reading custom headers and what issues we could run into?

Originally created by @warmconnect on GitHub (Nov 21, 2020). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1482 ### Additional Information #### Version of s3fs being used (s3fs --version) V1.87 #### Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse) 2.9.2 #### Kernel information (uname -r) 4.14.181-142.260.amzn2.x86_64 #### GNU/Linux Distribution, if applicable (cat /etc/os-release) # cat /etc/os-release NAME="Amazon Linux" VERSION="2" ID="amzn" ID_LIKE="centos rhel fedora" VERSION_ID="2" PRETTY_NAME="Amazon Linux 2" ANSI_COLOR="0;33" CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2" HOME_URL="https://amazonlinux.com/" ### Details about issue Feature request: the per object head request slows s3fs considerably and is also not compatible with native s3 tools. Can we have an option to not issue the head requests? I.E. For read requests pretend the filesystem was populated by a native s3 tool like aws s3 sync in which case the x-amz headers will not be present. For write requests s3fs should not bother to write the headers. Alternatively, could you point out where we should look to block s3fs from writing and reading custom headers and what issues we could run into?

kerem added the

performance

label

2026-03-04 01:48:42 +03:00

kerem commented

2026-03-04 01:48:43 +03:00

Author

Owner

@gaul commented on GitHub (Nov 21, 2020):

Agreed that this could be a good optional feature. readdir_multi_head issues parallel calls to S3fsCurl::PreHeadRequest. Instead of issuing these, it could fake out some of the metadata, e.g., atime, permissions, gid, etc. This option would be similar to goofys if you want to look at its implementation. You can also check out the slides and video of a talk I gave about these kinds of trade-offs.

@gaul commented on GitHub (Nov 21, 2020): Agreed that this could be a good optional feature. `readdir_multi_head` issues parallel calls to `S3fsCurl::PreHeadRequest`. Instead of issuing these, it could fake out some of the metadata, e.g., atime, permissions, gid, etc. This option would be similar to [goofys](https://github.com/kahing/goofys) if you want to look at its implementation. You can also check out the [slides](http://gaul.org/talks/s3fs-tradeoffs/) and [video](https://www.youtube.com/watch?v=9tHx74047yc&t=18407) of a talk I gave about these kinds of trade-offs.

kerem commented

2026-03-04 01:48:43 +03:00

Author

Owner

@basldfalksjdf commented on GitHub (Jan 9, 2024):

Is this feature request stale or are there plans on adding it?

@basldfalksjdf commented on GitHub (Jan 9, 2024): Is this feature request stale or are there plans on adding it?

kerem commented

2026-03-04 01:48:44 +03:00

Author

Owner

@ggtakec commented on GitHub (Feb 12, 2024):

@basldfalksjdf I'm aware of this issue and have ideas for fixing it, so we plan to improve it in near future. (I apologize for not starting yet)

@ggtakec commented on GitHub (Feb 12, 2024): @basldfalksjdf I'm aware of this issue and have ideas for fixing it, so we plan to improve it in near future. (I apologize for not starting yet)