[PR #1995] [MERGED] Improved to avoid unnecessary head request #2293

New issue

Closed

opened 2026-03-04 02:04:46 +03:00 by kerem · 0 comments

kerem commented

2026-03-04 02:04:46 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/s3fs-fuse/s3fs-fuse/pull/1995
Author: @ggtakec
Created: 7/22/2022
Status: ✅ Merged
Merged: 7/27/2022
Merged by: @gaul

Base: master ← Head: improve_multihead

📝 Commits (1)

37ad2a5 Improved to avoid unnecessary head request

📊 Changes

6 files changed (+80 additions, -37 deletions)

View changed files

📝 src/cache.h (+4 -0)
📝 src/curl.cpp (+12 -6)
📝 src/curl.h (+3 -3)
📝 src/curl_multi.cpp (+9 -2)
📝 src/curl_multi.h (+4 -2)
📝 src/s3fs.cpp (+48 -24)

📄 Description

Relevant Issue (if applicable)

n/a

Details

Fixed the readdir_multi_head function called from s3fs_readdir because it is inefficient.

This fixes an inefficiency if the stat cache was cached out during the processing of readdir_multi_head.

readdir_multi_head uses a filler(FUSE param) to return a list of files and directories stat returned by list_bucket.
At this time, even though the files and directories returned by list_bucket existed in the stat cache, s3fs was always sending a HEAD request.
Also, HEAD requests are processed in parallel, and those results are stored in the stat cache.
And readdir_multi_head reads the value from the stat cache after all request finished.
When the number of stat caches is small and stat cache of those result is out, s3fs requests HEADs which are unnecessary processing.

So readdir_multi_head passes its value to the filler as is, if it exists in the stat cache.
Also, the result of the HEAD request is passed to the filler without going through stat cache.(but it will be cached to stat)

This will reduce unnecessary HEAD requests and improve performance. (depending on the stat cache conditions)

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/s3fs-fuse/s3fs-fuse/pull/1995 **Author:** [@ggtakec](https://github.com/ggtakec) **Created:** 7/22/2022 **Status:** ✅ Merged **Merged:** 7/27/2022 **Merged by:** [@gaul](https://github.com/gaul) **Base:** `master` ← **Head:** `improve_multihead` --- ### 📝 Commits (1) - [`37ad2a5`](https://github.com/s3fs-fuse/s3fs-fuse/commit/37ad2a58e4b7c6ca227a92404eb6a80bb330db0f) Improved to avoid unnecessary head request ### 📊 Changes **6 files changed** (+80 additions, -37 deletions) <details> <summary>View changed files</summary> 📝 `src/cache.h` (+4 -0) 📝 `src/curl.cpp` (+12 -6) 📝 `src/curl.h` (+3 -3) 📝 `src/curl_multi.cpp` (+9 -2) 📝 `src/curl_multi.h` (+4 -2) 📝 `src/s3fs.cpp` (+48 -24) </details> ### 📄 Description ### Relevant Issue (if applicable) n/a ### Details Fixed the `readdir_multi_head` function called from `s3fs_readdir` because it is inefficient. This fixes an inefficiency if the stat cache was cached out during the processing of `readdir_multi_head`. `readdir_multi_head` uses a `filler`(FUSE param) to return a list of files and directories stat returned by `list_bucket`. At this time, even though the files and directories returned by `list_bucket` existed in the stat cache, s3fs was always sending a HEAD request. Also, HEAD requests are processed in parallel, and those results are stored in the stat cache. And `readdir_multi_head` reads the value from the stat cache after all request finished. When the number of stat caches is small and stat cache of those result is out, s3fs requests HEADs which are unnecessary processing. So `readdir_multi_head` passes its value to the filler as is, if it exists in the stat cache. Also, the result of the HEAD request is passed to the filler without going through stat cache.(but it will be cached to stat) This will reduce unnecessary HEAD requests and improve performance. (depending on the stat cache conditions) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>