[PR #2731] [MERGED] Improved stat cache to include ListBucket results #2752

New issue

Closed

opened 2026-03-04 02:07:07 +03:00 by kerem · 0 comments

kerem commented

2026-03-04 02:07:07 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/s3fs-fuse/s3fs-fuse/pull/2731
Author: @ggtakec
Created: 9/7/2025
Status: ✅ Merged
Merged: 11/8/2025
Merged by: @gaul

Base: master ← Head: improve/7_s3obj_cache

📝 Commits (1)

ad09e94 Improved stat cache to include ListBucket results

📊 Changes

7 files changed (+349 additions, -35 deletions)

View changed files

📝 src/cache.cpp (+51 -0)
📝 src/cache.h (+2 -0)
📝 src/cache_node.cpp (+178 -20)
📝 src/cache_node.h (+10 -0)
📝 src/s3fs.cpp (+39 -15)
📝 src/s3objlist.cpp (+65 -0)
📝 src/s3objlist.h (+4 -0)

📄 Description

Relevant Issue (if applicable)

#2730

Details

The S3ObjList resulting from temporary parsing of ListBucket results can now be cached in StatCache.
This reduces ListBucket requests.

For reference, a comparison of the number of requests between the base code in #2730 and this one is shown below.
(Compare the results of running small-integration-test.sh.)

Branch	Result
#2730 code	Number of head object requests: 4294 Number of put object requests: 616 Number of get object requests: 162 Number of delete object requests: 575 Number of list bucket requests: 2059 Number of initiate MPU requests: 104 Number of complete MPU requests: 104 Number of abort MPU requests: 0 Number of upload MPU part requests: 141 Number of copy MPU part requests: 76
this PR	Number of head object requests: 4286 Number of put object requests: 620 Number of get object requests: 153 Number of delete object requests: 575 Number of list bucket requests: 1749 Number of initiate MPU requests: 98 Number of complete MPU requests: 98 Number of abort MPU requests: 0 Number of upload MPU part requests: 133 Number of copy MPU part requests: 63

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/s3fs-fuse/s3fs-fuse/pull/2731 **Author:** [@ggtakec](https://github.com/ggtakec) **Created:** 9/7/2025 **Status:** ✅ Merged **Merged:** 11/8/2025 **Merged by:** [@gaul](https://github.com/gaul) **Base:** `master` ← **Head:** `improve/7_s3obj_cache` --- ### 📝 Commits (1) - [`ad09e94`](https://github.com/s3fs-fuse/s3fs-fuse/commit/ad09e941b086ccdce05e0e3fd2badd7fa276eec1) Improved stat cache to include ListBucket results ### 📊 Changes **7 files changed** (+349 additions, -35 deletions) <details> <summary>View changed files</summary> 📝 `src/cache.cpp` (+51 -0) 📝 `src/cache.h` (+2 -0) 📝 `src/cache_node.cpp` (+178 -20) 📝 `src/cache_node.h` (+10 -0) 📝 `src/s3fs.cpp` (+39 -15) 📝 `src/s3objlist.cpp` (+65 -0) 📝 `src/s3objlist.h` (+4 -0) </details> ### 📄 Description ### Relevant Issue (if applicable) #2730 ### Details The S3ObjList resulting from temporary parsing of ListBucket results can now be cached in StatCache. This reduces ListBucket requests. For reference, a comparison of the number of requests between the base code in #2730 and this one is shown below. (Compare the results of running `small-integration-test.sh`.) | Branch | Result | | ---------- | ------- | | #2730 code | Number of head object requests: 4294 Number of put object requests: 616 Number of get object requests: 162 Number of delete object requests: 575 Number of list bucket requests: 2059 Number of initiate MPU requests: 104 Number of complete MPU requests: 104 Number of abort MPU requests: 0 Number of upload MPU part requests: 141 Number of copy MPU part requests: 76 | | this PR | Number of head object requests: 4286 Number of put object requests: 620 Number of get object requests: 153 Number of delete object requests: 575 Number of list bucket requests: 1749 Number of initiate MPU requests: 98 Number of complete MPU requests: 98 Number of abort MPU requests: 0 Number of upload MPU part requests: 133 Number of copy MPU part requests: 63 | --- 🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.