[PR #2028] [MERGED] Improved fdcache_page performance #2314

New issue

Closed

opened 2026-03-04 02:04:53 +03:00 by kerem · 0 comments

kerem commented

2026-03-04 02:04:53 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/s3fs-fuse/s3fs-fuse/pull/2028
Author: @ggtakec
Created: 9/4/2022
Status: ✅ Merged
Merged: 9/25/2022
Merged by: @gaul

Base: master ← Head: fix_fdcache_page

📝 Commits (1)

0605102 Improved fdcache_page performance

📊 Changes

1 file changed (+86 additions, -55 deletions)

View changed files

📝 src/fdcache_page.cpp (+86 -55)

📄 Description

Relevant Issue (if applicable)

#2019 #2027

Details

The following issues were reported.

High CPU load when running s3fs (#2027)
Extremely poor performance in fio test with multi number of job (#2019)

Both of these seem to be caused by the processing by the page list of fdcache operation in the PageList class.
There was a problem with page list compress and data extraction logic, resulting in poor performance.

Below are the values when measured with gprof:

 %   cumulative   self              self     total
time   seconds   seconds    calls  us/call  us/call  name
71.51      1.33     1.33    49576    26.83    26.83  raw_compress_fdpage_list(std::__cxx11::list<fdpage, std::allocator<fdpage> > const&, bool, bool, bool, bool) [clone .constprop.56]
13.44      1.58     0.25    12672    19.73    19.73  PageList::BytesModified() const
10.22      1.77     0.19    13184    14.41    14.41  PageList::GetTotalUnloadedPageSize(long, long, long) const
 3.76      1.84     0.07    49567     1.41    28.24  PageList::Compress()
 0.54      1.85     0.01    24964     0.40    28.22  PageList::SetPageLoadedStatus(long, long, PageList::page_status, bool)
...
...

Reviewed related functions including raw_compress_fdpage_list in problem.(Reduced useless copies, etc.)
I have confirmed that this fix does not cause a high load on the CPU.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/s3fs-fuse/s3fs-fuse/pull/2028 **Author:** [@ggtakec](https://github.com/ggtakec) **Created:** 9/4/2022 **Status:** ✅ Merged **Merged:** 9/25/2022 **Merged by:** [@gaul](https://github.com/gaul) **Base:** `master` ← **Head:** `fix_fdcache_page` --- ### 📝 Commits (1) - [`0605102`](https://github.com/s3fs-fuse/s3fs-fuse/commit/060510250bc5c235ad2eda1930491253157c918d) Improved fdcache_page performance ### 📊 Changes **1 file changed** (+86 additions, -55 deletions) <details> <summary>View changed files</summary> 📝 `src/fdcache_page.cpp` (+86 -55) </details> ### 📄 Description ### Relevant Issue (if applicable) #2019 #2027 ### Details The following issues were reported. - High CPU load when running s3fs (#2027) - Extremely poor performance in `fio` test with multi number of job (#2019) Both of these seem to be caused by the processing by the `page list` of `fdcache` operation in the `PageList` class. There was a problem with `page list` compress and data extraction logic, resulting in poor performance. Below are the values when measured with `gprof`: ``` % cumulative self self total time seconds seconds calls us/call us/call name 71.51 1.33 1.33 49576 26.83 26.83 raw_compress_fdpage_list(std::__cxx11::list<fdpage, std::allocator<fdpage> > const&, bool, bool, bool, bool) [clone .constprop.56] 13.44 1.58 0.25 12672 19.73 19.73 PageList::BytesModified() const 10.22 1.77 0.19 13184 14.41 14.41 PageList::GetTotalUnloadedPageSize(long, long, long) const 3.76 1.84 0.07 49567 1.41 28.24 PageList::Compress() 0.54 1.85 0.01 24964 0.40 28.22 PageList::SetPageLoadedStatus(long, long, PageList::page_status, bool) ... ... ``` Reviewed related functions including `raw_compress_fdpage_list` in problem.(Reduced useless copies, etc.) I have confirmed that this fix does not cause a high load on the CPU. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>