starred/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2026-04-25 05:16:00 +03:00
Code Issues 305 Projects Releases 5 Packages Wiki Activity

[PR #2209] [MERGED] Fixed data race in threads found thread sanitizer #2390

New issue
Closed
opened 2026-03-04 02:05:17 +03:00 by kerem · 0 comments
kerem commented 2026-03-04 02:05:17 +03:00
Owner
Copy link

📋 Pull Request Information

Original PR: https://github.com/s3fs-fuse/s3fs-fuse/pull/2209
Author: @ggtakec
Created: 7/9/2023
Status: ✅ Merged
Merged: 7/9/2023
Merged by: @gaul

Base: master ← Head: fix_data_race

📝 Commits (1)

  • c8b9d4a Fixed data race in threads found thread sanitizer

📊 Changes

3 files changed (+4 additions, -5 deletions)

View changed files

📝 src/curl.cpp (+2 -2)
📝 src/fdcache_entity.cpp (+2 -2)
📝 src/s3fs.cpp (+0 -1)

📄 Description

Relevant Issue (if applicable)

n/a

Details

The following error in the ThreadSinitizer test was detected, so it was fixed.

WARNING: ThreadSanitizer: data race (pid=22052)
  Read of size 4 at 0x7b4c000119a8 by thread T9 (mutexes: write M0):
    #0 FdEntity::IsDirtyNewFile() const /__w/s3fs-fuse/s3fs-fuse/src/fdcache_entity.cpp:2546 (s3fs+0x5dc836) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7)
    #1 s3fs_release(char const*, fuse_file_info*) /__w/s3fs-fuse/s3fs-fuse/src/s3fs.cpp:3023 (s3fs+0x54aae9) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7)
    #2 fuse_fs_release ??:? (libfuse.so.2+0xd345) (BuildId: 4db3553f25db655fdcad5600bb328da5dc18736b)

  Previous write of size 4 at 0x7b4c000119a8 by thread T44 (mutexes: write M1):
    #0 FdEntity::UploadPending(int, AutoLock::Type) /__w/s3fs-fuse/s3fs-fuse/src/fdcache_entity.cpp:2450 (s3fs+0x5d983b) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7)
    #1 s3fs_release(char const*, fuse_file_info*) /__w/s3fs-fuse/s3fs-fuse/src/s3fs.cpp:3026 (s3fs+0x54ab2b) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7)
    #2 fuse_fs_release ??:? (libfuse.so.2+0xd345) (BuildId: 4db3553f25db655fdcad5600bb328da5dc18736b)

  Location is heap block of size 448 at 0x7b4c00011800 allocated by thread T1:
    #0 operator new(unsigned long) ??:? (s3fs+0x53747b) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7)
    #1 FdManager::Open(int&, char const*, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, header_nocase_cmp, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >*, long, timespec const&, int, bool, bool, bool, AutoLock::Type) /__w/s3fs-fuse/s3fs-fuse/src/fdcache.cpp:593 (s3fs+0x5c7df3) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7)
    #2 AutoFdEntity::Open(char const*, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, header_nocase_cmp, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >*, long, timespec const&, int, bool, bool, bool, AutoLock::Type) /__w/s3fs-fuse/s3fs-fuse/src/fdcache_auto.cpp:103 (s3fs+0x5e9413) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7)
    #3 s3fs_create(char const*, unsigned int, fuse_file_info*) /__w/s3fs-fuse/s3fs-fuse/src/s3fs.cpp:1135 (s3fs+0x54c193) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7)
    #4 fuse_fs_create ??:? (libfuse.so.2+0xddff) (BuildId: 4db3553f25db655fdcad5600bb328da5dc18736b)

  Mutex M0 (0x7b4c00011920) created at:
    #0 pthread_mutex_init ??:? (s3fs+0x4b0354) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7)
    #1 FdEntity /__w/s3fs-fuse/s3fs-fuse/src/fdcache_entity.cpp:124 (s3fs+0x5ce9c4) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7)
    #2 FdManager::Open(int&, char const*, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, header_nocase_cmp, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >*, long, timespec const&, int, bool, bool, bool, AutoLock::Type) /__w/s3fs-fuse/s3fs-fuse/src/fdcache.cpp:593 (s3fs+0x5c7e27) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7)
    #3 AutoFdEntity::Open(char const*, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, header_nocase_cmp, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >*, long, timespec const&, int, bool, bool, bool, AutoLock::Type) /__w/s3fs-fuse/s3fs-fuse/src/fdcache_auto.cpp:103 (s3fs+0x5e9413) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7)
    #4 s3fs_create(char const*, unsigned int, fuse_file_info*) /__w/s3fs-fuse/s3fs-fuse/src/s3fs.cpp:1135 (s3fs+0x54c193) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7)
    #5 fuse_fs_create ??:? (libfuse.so.2+0xddff) (BuildId: 4db3553f25db655fdcad5600bb328da5dc18736b)

  Mutex M1 (0x7b4c00011800) created at:
    #0 pthread_mutex_init ??:? (s3fs+0x4b0354) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7)
    #1 FdEntity /__w/s3fs-fuse/s3fs-fuse/src/fdcache_entity.cpp:120 (s3fs+0x5ce967) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7)
    #2 FdManager::Open(int&, char const*, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, header_nocase_cmp, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >*, long, timespec const&, int, bool, bool, bool, AutoLock::Type) /__w/s3fs-fuse/s3fs-fuse/src/fdcache.cpp:593 (s3fs+0x5c7e27) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7)
    #3 AutoFdEntity::Open(char const*, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, header_nocase_cmp, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >*, long, timespec const&, int, bool, bool, bool, AutoLock::Type) /__w/s3fs-fuse/s3fs-fuse/src/fdcache_auto.cpp:103 (s3fs+0x5e9413) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7)
    #4 s3fs_create(char const*, unsigned int, fuse_file_info*) /__w/s3fs-fuse/s3fs-fuse/src/s3fs.cpp:1135 (s3fs+0x54c193) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7)
    #5 fuse_fs_create ??:? (libfuse.so.2+0xddff) (BuildId: 4db3553f25db655fdcad5600bb328da5dc18736b)

  Thread T9 (tid=22089, running) created by thread T1 at:
    #0 pthread_create ??:? (s3fs+0x4ae98f) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7)
    #1 <null> <null> (libfuse.so.2+0xc4e0) (BuildId: 4db3553f25db655fdcad5600bb328da5dc18736b)

  Thread T44 (tid=23204, running) created by thread T38 at:
    #0 pthread_create ??:? (s3fs+0x4ae98f) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7)
    #1 <null> <null> (libfuse.so.2+0xc4e0) (BuildId: 4db3553f25db655fdcad5600bb328da5dc18736b)

  Thread T1 (tid=22054, running) created by main thread at:
    #0 pthread_create ??:? (s3fs+0x4ae98f) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7)
    #1 <null> <null> (libfuse.so.2+0xc4e0) (BuildId: 4db3553f25db655fdcad5600bb328da5dc18736b)
    #2 __libc_start_call_main ??:? (libc.so.6+0x27b49) (BuildId: 245240a31888ad5c11bbc55b18e02d87388f59a9)

SUMMARY: ThreadSanitizer: data race /__w/s3fs-fuse/s3fs-fuse/src/fdcache_entity.cpp:2546 in FdEntity::IsDirtyNewFile() const

This is because some methods that deal with the pending_status variable use different locking variables than others.
These methods used fdent_data_lock and should have used fdent_lock.
(FdEntity::UploadPending calls the Flush method, so fdent_data_lock doesn't work)

Also, the comments in s3fs.cpp were confirmed by this fix and have been removed.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/s3fs-fuse/s3fs-fuse/pull/2209 **Author:** [@ggtakec](https://github.com/ggtakec) **Created:** 7/9/2023 **Status:** ✅ Merged **Merged:** 7/9/2023 **Merged by:** [@gaul](https://github.com/gaul) **Base:** `master` ← **Head:** `fix_data_race` --- ### 📝 Commits (1) - [`c8b9d4a`](https://github.com/s3fs-fuse/s3fs-fuse/commit/c8b9d4a0c29494116c15ebe23d94c97d40a4e2dd) Fixed data race in threads found thread sanitizer ### 📊 Changes **3 files changed** (+4 additions, -5 deletions) <details> <summary>View changed files</summary> 📝 `src/curl.cpp` (+2 -2) 📝 `src/fdcache_entity.cpp` (+2 -2) 📝 `src/s3fs.cpp` (+0 -1) </details> ### 📄 Description ### Relevant Issue (if applicable) n/a ### Details The following error in the ThreadSinitizer test was detected, so it was fixed. ``` WARNING: ThreadSanitizer: data race (pid=22052) Read of size 4 at 0x7b4c000119a8 by thread T9 (mutexes: write M0): #0 FdEntity::IsDirtyNewFile() const /__w/s3fs-fuse/s3fs-fuse/src/fdcache_entity.cpp:2546 (s3fs+0x5dc836) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7) #1 s3fs_release(char const*, fuse_file_info*) /__w/s3fs-fuse/s3fs-fuse/src/s3fs.cpp:3023 (s3fs+0x54aae9) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7) #2 fuse_fs_release ??:? (libfuse.so.2+0xd345) (BuildId: 4db3553f25db655fdcad5600bb328da5dc18736b) Previous write of size 4 at 0x7b4c000119a8 by thread T44 (mutexes: write M1): #0 FdEntity::UploadPending(int, AutoLock::Type) /__w/s3fs-fuse/s3fs-fuse/src/fdcache_entity.cpp:2450 (s3fs+0x5d983b) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7) #1 s3fs_release(char const*, fuse_file_info*) /__w/s3fs-fuse/s3fs-fuse/src/s3fs.cpp:3026 (s3fs+0x54ab2b) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7) #2 fuse_fs_release ??:? (libfuse.so.2+0xd345) (BuildId: 4db3553f25db655fdcad5600bb328da5dc18736b) Location is heap block of size 448 at 0x7b4c00011800 allocated by thread T1: #0 operator new(unsigned long) ??:? (s3fs+0x53747b) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7) #1 FdManager::Open(int&, char const*, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, header_nocase_cmp, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >*, long, timespec const&, int, bool, bool, bool, AutoLock::Type) /__w/s3fs-fuse/s3fs-fuse/src/fdcache.cpp:593 (s3fs+0x5c7df3) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7) #2 AutoFdEntity::Open(char const*, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, header_nocase_cmp, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >*, long, timespec const&, int, bool, bool, bool, AutoLock::Type) /__w/s3fs-fuse/s3fs-fuse/src/fdcache_auto.cpp:103 (s3fs+0x5e9413) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7) #3 s3fs_create(char const*, unsigned int, fuse_file_info*) /__w/s3fs-fuse/s3fs-fuse/src/s3fs.cpp:1135 (s3fs+0x54c193) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7) #4 fuse_fs_create ??:? (libfuse.so.2+0xddff) (BuildId: 4db3553f25db655fdcad5600bb328da5dc18736b) Mutex M0 (0x7b4c00011920) created at: #0 pthread_mutex_init ??:? (s3fs+0x4b0354) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7) #1 FdEntity /__w/s3fs-fuse/s3fs-fuse/src/fdcache_entity.cpp:124 (s3fs+0x5ce9c4) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7) #2 FdManager::Open(int&, char const*, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, header_nocase_cmp, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >*, long, timespec const&, int, bool, bool, bool, AutoLock::Type) /__w/s3fs-fuse/s3fs-fuse/src/fdcache.cpp:593 (s3fs+0x5c7e27) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7) #3 AutoFdEntity::Open(char const*, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, header_nocase_cmp, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >*, long, timespec const&, int, bool, bool, bool, AutoLock::Type) /__w/s3fs-fuse/s3fs-fuse/src/fdcache_auto.cpp:103 (s3fs+0x5e9413) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7) #4 s3fs_create(char const*, unsigned int, fuse_file_info*) /__w/s3fs-fuse/s3fs-fuse/src/s3fs.cpp:1135 (s3fs+0x54c193) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7) #5 fuse_fs_create ??:? (libfuse.so.2+0xddff) (BuildId: 4db3553f25db655fdcad5600bb328da5dc18736b) Mutex M1 (0x7b4c00011800) created at: #0 pthread_mutex_init ??:? (s3fs+0x4b0354) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7) #1 FdEntity /__w/s3fs-fuse/s3fs-fuse/src/fdcache_entity.cpp:120 (s3fs+0x5ce967) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7) #2 FdManager::Open(int&, char const*, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, header_nocase_cmp, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >*, long, timespec const&, int, bool, bool, bool, AutoLock::Type) /__w/s3fs-fuse/s3fs-fuse/src/fdcache.cpp:593 (s3fs+0x5c7e27) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7) #3 AutoFdEntity::Open(char const*, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, header_nocase_cmp, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >*, long, timespec const&, int, bool, bool, bool, AutoLock::Type) /__w/s3fs-fuse/s3fs-fuse/src/fdcache_auto.cpp:103 (s3fs+0x5e9413) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7) #4 s3fs_create(char const*, unsigned int, fuse_file_info*) /__w/s3fs-fuse/s3fs-fuse/src/s3fs.cpp:1135 (s3fs+0x54c193) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7) #5 fuse_fs_create ??:? (libfuse.so.2+0xddff) (BuildId: 4db3553f25db655fdcad5600bb328da5dc18736b) Thread T9 (tid=22089, running) created by thread T1 at: #0 pthread_create ??:? (s3fs+0x4ae98f) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7) #1 <null> <null> (libfuse.so.2+0xc4e0) (BuildId: 4db3553f25db655fdcad5600bb328da5dc18736b) Thread T44 (tid=23204, running) created by thread T38 at: #0 pthread_create ??:? (s3fs+0x4ae98f) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7) #1 <null> <null> (libfuse.so.2+0xc4e0) (BuildId: 4db3553f25db655fdcad5600bb328da5dc18736b) Thread T1 (tid=22054, running) created by main thread at: #0 pthread_create ??:? (s3fs+0x4ae98f) (BuildId: ab83e9890914b4c6d66a0fd5ea48fdf43707afe7) #1 <null> <null> (libfuse.so.2+0xc4e0) (BuildId: 4db3553f25db655fdcad5600bb328da5dc18736b) #2 __libc_start_call_main ??:? (libc.so.6+0x27b49) (BuildId: 245240a31888ad5c11bbc55b18e02d87388f59a9) SUMMARY: ThreadSanitizer: data race /__w/s3fs-fuse/s3fs-fuse/src/fdcache_entity.cpp:2546 in FdEntity::IsDirtyNewFile() const ``` This is because some methods that deal with the `pending_status` variable use different locking variables than others. These methods used `fdent_data_lock` and should have used `fdent_lock`. (`FdEntity::UploadPending` calls the `Flush` method, so `fdent_data_lock` doesn't work) Also, the comments in `s3fs.cpp` were confirmed by this fix and have been removed. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
kerem 2026-03-04 02:05:17 +03:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
v1.97
v1.96
v1.95
v1.94
v1.93
v1.92
v1.91
v1.90
v1.89
v1.88
v1.87
v1.86
v1.85
v1.84
v1.83
v1.82
v1.81
v1.80
v1.79
v1.78
Pre-v1.78
v1.77
v1.76
v1.75
v1.74
Labels
Clear labels   
bug

   
bug

   
dataloss

   
duplicate

   
enhancement

   
feature request

   
help wanted

   
invalid

   
need info

   
performance

   
pull-request

Mirrored from GitHub Pull Request

  
question

   
question

   
testing
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#2390
No description provided.