[PR #2623] [MERGED] change the way to get existing fdentity to optimize concurrent IO performance #2674

New issue

Closed

opened 2026-03-04 02:06:43 +03:00 by kerem · 0 comments

kerem commented

2026-03-04 02:06:43 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/s3fs-fuse/s3fs-fuse/pull/2623
Author: @fangqianan
Created: 12/18/2024
Status: ✅ Merged
Merged: 2/28/2025
Merged by: @ggtakec

Base: master ← Head: fangqianan_lock

📝 Commits (5)

d47ad2a change the way to get existing fdentity
1e1432d fix compiling err for ver > 1.91
1342daf Merge branch 'master' into fangqianan_lock
804f95c use GetROPath instead of GetPath
fa0c9fb compare FdEntity::ro_path first in FdEntity::GetFdEntityHasLock()

📊 Changes

2 files changed (+34 additions, -23 deletions)

View changed files

📝 src/fdcache.cpp (+32 -22)
📝 src/fdcache_entity.cpp (+2 -1)

📄 Description

Relevant Issue (if applicable)

https://github.com/s3fs-fuse/s3fs-fuse/issues/2617

Details

Change the way to get existing fdentity to better conccurent IO's performance.
Instead of traversing the fent map and only compare the pseudo fd, which needs to hold every fdentity 's lock, I look for the fdentity with path at first and then verify the pseudo fd, which only needs to hold the right fdentity's lock.

I started 20 processes to read/write 500 files of 1MB in total concurrently. For ver 1.91, the RW performance is twice of that of the old code. And for ver 1.95, the performance is 20% better after this fix.

P.S. v1.95 performs worse than v1.91 (just half) according to my test (without this fix). This is another topic though. I will try to figure out the reason too and maybe you can also have a look into it.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/s3fs-fuse/s3fs-fuse/pull/2623 **Author:** [@fangqianan](https://github.com/fangqianan) **Created:** 12/18/2024 **Status:** ✅ Merged **Merged:** 2/28/2025 **Merged by:** [@ggtakec](https://github.com/ggtakec) **Base:** `master` ← **Head:** `fangqianan_lock` --- ### 📝 Commits (5) - [`d47ad2a`](https://github.com/s3fs-fuse/s3fs-fuse/commit/d47ad2a0573661becac9e5ae9a8b82e22e769dc6) change the way to get existing fdentity - [`1e1432d`](https://github.com/s3fs-fuse/s3fs-fuse/commit/1e1432dfccfc16752ea55b6858d48b806bb862c0) fix compiling err for ver > 1.91 - [`1342daf`](https://github.com/s3fs-fuse/s3fs-fuse/commit/1342daf9dc40f63826dd8407452bc67ca4ae6a34) Merge branch 'master' into fangqianan_lock - [`804f95c`](https://github.com/s3fs-fuse/s3fs-fuse/commit/804f95c59dd62501909cfcdda8ce7d6f3f1bbb61) use GetROPath instead of GetPath - [`fa0c9fb`](https://github.com/s3fs-fuse/s3fs-fuse/commit/fa0c9fb0886e9efbae1c33dd7738d53310932ec1) compare FdEntity::ro_path first in FdEntity::GetFdEntityHasLock() ### 📊 Changes **2 files changed** (+34 additions, -23 deletions) <details> <summary>View changed files</summary> 📝 `src/fdcache.cpp` (+32 -22) 📝 `src/fdcache_entity.cpp` (+2 -1) </details> ### 📄 Description  ### Relevant Issue (if applicable) https://github.com/s3fs-fuse/s3fs-fuse/issues/2617 ### Details Change the way to get existing fdentity to better conccurent IO's performance. Instead of traversing the fent map and only compare the pseudo fd, which needs to hold every fdentity 's lock, I look for the fdentity with path at first and then verify the pseudo fd, which only needs to hold the right fdentity's lock. I started 20 processes to read/write 500 files of 1MB in total concurrently. For ver 1.91, the RW performance is twice of that of the old code. And for ver 1.95, the performance is 20% better after this fix. P.S. v1.95 performs worse than v1.91 (just half) according to my test (without this fix). This is another topic though. I will try to figure out the reason too and maybe you can also have a look into it. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>