starred/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2026-04-25 13:26:00 +03:00
Code Issues 305 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #1819] s3fs blocked - most threads hang in AutoLock::AutoLock() -> __lll_lock_wait() #933

New issue
Closed
opened 2026-03-04 01:50:02 +03:00 by kerem · 2 comments
kerem commented 2026-03-04 01:50:02 +03:00
Owner
Copy link

Originally created by @CarstenGrohmann on GitHub (Dec 17, 2021).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1819

Dear S3FS Developer,

Running the current master (5de92e9 - "Bump CI to Fedora 35 (#1806)") causes a deadlock.

In this setup 4 parallel ZIP processes stores files in a bucket mounted with s3fs. The process got stuck with the following backtrace. The system log doesn't show any information. A separate debug log was not active.

The error occurred just once and I don't have the ability to compile with Clang to use the ThreadSanitizer provide more information

Do you have an idea about the root cause of this issue?

Thank you,
Carsten

Short backtrace;

(gdb) thread apply all bt

Thread 8 (Thread 0x7f8d4c140880 (LWP 32639)):
#0  0x00007f8d4a879adb in do_futex_wait.constprop.1 () from /lib64/libpthread.so.0
#1  0x00007f8d4a879b6f in __new_sem_wait_slow.constprop.0 () from /lib64/libpthread.so.0
#2  0x00007f8d4a879c0b in sem_wait@@GLIBC_2.2.5 () from /lib64/libpthread.so.0
#3  0x00007f8d4bd0f5c8 in fuse_session_loop_mt () from /lib64/libfuse.so.2
#4  0x00007f8d4bd14ce7 in fuse_loop_mt () from /lib64/libfuse.so.2
#5  0x00007f8d4bd17875 in fuse_main_common () from /lib64/libfuse.so.2
#6  0x0000000000409d19 in main (argc=<optimized out>, argv=<optimized out>) at s3fs.cpp:5289

Thread 7 (Thread 0x7f8d35ffb700 (LWP 4782)):
#0  0x00007f8d4a87a4ed in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007f8d4a875dcb in _L_lock_883 () from /lib64/libpthread.so.0
#2  0x00007f8d4a875c98 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x00000000004972ee in AutoLock::AutoLock (this=0x7f8d35ffa7e0, pmutex=<optimized out>, type=<optimized out>) at autolock.cpp:47
#4  0x0000000000474401 in FdManager::Open (this=this@entry=0x6c43c0 <FdManager::singleton>, fd=@0x7f8d35ffa968: -1, path=path@entry=0x7f8d2400f2a0 "/Data", pmeta=pmeta@entry=0x0, size=size@entry=-1, time=time@entry=-1,
    flags=flags@entry=0, force_tmpfile=force_tmpfile@entry=false, is_create=is_create@entry=false, type=type@entry=AutoLock::NONE) at fdcache.cpp:532
#5  0x0000000000475025 in FdManager::OpenExistFdEntity (this=0x6c43c0 <FdManager::singleton>, path=path@entry=0x7f8d2400f2a0 "/Data", fd=@0x7f8d35ffa968: -1, flags=flags@entry=0) at fdcache.cpp:631
#6  0x000000000048fb76 in AutoFdEntity::OpenExistFdEntity (this=this@entry=0x7f8d35ffa960, path=path@entry=0x7f8d2400f2a0 "/Data", flags=flags@entry=0) at fdcache_auto.cpp:130
#7  0x0000000000416561 in s3fs_getattr (_path=<optimized out>, stbuf=0x7f8d35ffab00) at s3fs.cpp:792
#8  0x00007f8d4bd079c8 in lookup_path () from /lib64/libfuse.so.2
#9  0x00007f8d4bd07bfd in fuse_lib_lookup () from /lib64/libfuse.so.2
#10 0x00007f8d4bd12b6b in fuse_ll_process_buf () from /lib64/libfuse.so.2
#11 0x00007f8d4bd0f401 in fuse_do_work () from /lib64/libfuse.so.2
#12 0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0
#13 0x00007f8d4a599ead in clone () from /lib64/libc.so.6

Thread 6 (Thread 0x7f8d367fc700 (LWP 1251)):
#0  0x00007f8d4a87a6fd in read () from /lib64/libpthread.so.0
#1  0x00007f8d4bd0ed62 in fuse_kern_chan_receive () from /lib64/libfuse.so.2
#2  0x00007f8d4bd0fd59 in fuse_ll_receive_buf () from /lib64/libfuse.so.2
#3  0x00007f8d4bd0f37e in fuse_do_work () from /lib64/libfuse.so.2
#4  0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0
#5  0x00007f8d4a599ead in clone () from /lib64/libc.so.6

Thread 5 (Thread 0x7f8d377fe700 (LWP 1014)):
#0  0x00007f8d4a87a4ed in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007f8d4a875dcb in _L_lock_883 () from /lib64/libpthread.so.0
#2  0x00007f8d4a875c98 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x00000000004972ee in AutoLock::AutoLock (this=0x7f8d377fd520, pmutex=<optimized out>, type=<optimized out>) at autolock.cpp:47
#4  0x00000000004738b3 in FdManager::ChangeEntityToTempPath (this=0x6c43c0 <FdManager::singleton>, ent=ent@entry=0x7f8d3802d5d0, path=0x7f8d380293c8 "/Data/<reducted>/ziqGxycK") at fdcache.cpp:712
#5  0x0000000000478629 in FdEntity::NoCacheLoadAndPost (this=this@entry=0x7f8d3802d5d0, pseudo_obj=pseudo_obj@entry=0x7f8d38003700, start=start@entry=0, size=size@entry=5074290069) at fdcache_entity.cpp:1084
#6  0x000000000047baf8 in FdEntity::WriteMixMultipart (this=this@entry=0x7f8d3802d5d0, pseudo_obj=pseudo_obj@entry=0x7f8d38003700,
    bytes=bytes@entry=0x7f8d4c095060 "G\365\n9mպܴØS\251Q\207\312ȱ\030F\332qT\243\352\321\031\340\020\f+h\210Õ±\224\022\061h(\255D\212\343\022FÔb\234\216\067\351\225 -\225bCB\t U\366\017HG\360\021%\233A\343(.3\023\264Z\241\305u\264É\217\350\335\022\221dD\202R(?L\366\201\322A\034\025\364~\023\034\"A\220\347\344\375\310\360\360\360:\021%\006\021lP\376v\277N$\003\300u&\006\227\241\244\\bbA\302\066\064\"\036\206\315\na\337\277\023\240\351xol\027\367Oo\333s\361Õ\037~\372\362w\177\373\354\344\356\217>\274\360\376\253G\247\276\277\371/\237\237>\275\177\317\366\245\351|7"..., start=start@entry=5074290069, size=size@entry=16384) at fdcache_entity.cpp:1996
#7  0x000000000047e987 in FdEntity::Write (this=this@entry=0x7f8d3802d5d0, fd=<optimized out>,
    bytes=bytes@entry=0x7f8d4c095060 "G\365\n9mպܴØS\251Q\207\312ȱ\030F\332qT\243\352\321\031\340\020\f+h\210Õ±\224\022\061h(\255D\212\343\022FÔb\234\216\067\351\225 -\225bCB\t U\366\017HG\360\021%\233A\343(.3\023\264Z\241\305u\264É\217\350\335\022\221dD\202R(?L\366\201\322A\034\025\364~\023\034\"A\220\347\344\375\310\360\360\360:\021%\006\021lP\376v\277N$\003\300u&\006\227\241\244\\bbA\302\066\064\"\036\206\315\na\337\277\023\240\351xol\027\367Oo\333s\361Õ\037~\372\362w\177\373\354\344\356\217>\274\360\376\253G\247\276\277\371/\237\237>\275\177\317\366\245\351|7"..., start=start@entry=5074290069, size=size@entry=16384) at fdcache_entity.cpp:1797
#8  0x000000000040c674 in s3fs_write (_path=<optimized out>,
    buf=0x7f8d4c095060 "G\365\n9mպܴØS\251Q\207\312ȱ\030F\332qT\243\352\321\031\340\020\f+h\210Õ±\224\022\061h(\255D\212\343\022FÔb\234\216\067\351\225 -\225bCB\t U\366\017HG\360\021%\233A\343(.3\023\264Z\241\305u\264É\217\350\335\022\221dD\202R(?L\366\201\322A\034\025\364~\023\034\"A\220\347\344\375\310\360\360\360:\021%\006\021lP\376v\277N$\003\300u&\006\227\241\244\\bbA\302\066\064\"\036\206\315\na\337\277\023\240\351xol\027\367Oo\333s\361Õ\037~\372\362w\177\373\354\344\356\217>\274\360\376\253G\247\276\277\371/\237\237>\275\177\317\366\245\351|7"..., size=16384, offset=5074290069, fi=0x7f8d377fdc30) at s3fs.cpp:2362
#9  0x00007f8d4bd09a7c in fuse_fs_write_buf () from /lib64/libfuse.so.2
#10 0x00007f8d4bd09bf8 in fuse_lib_write_buf () from /lib64/libfuse.so.2
#11 0x00007f8d4bd12cf8 in fuse_ll_process_buf () from /lib64/libfuse.so.2
#12 0x00007f8d4bd0f401 in fuse_do_work () from /lib64/libfuse.so.2
#13 0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0
#14 0x00007f8d4a599ead in clone () from /lib64/libc.so.6

Thread 4 (Thread 0x7f8d37fff700 (LWP 1009)):
#0  0x00007f8d4a87a4ed in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007f8d4a875dcb in _L_lock_883 () from /lib64/libpthread.so.0
#2  0x00007f8d4a875c98 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x00000000004972ee in AutoLock::AutoLock (this=0x7f8d37ffe910, pmutex=<optimized out>, type=<optimized out>) at autolock.cpp:47
#4  0x000000000047d726 in FdEntity::FindPseudoFd (this=0x7f8d38028c50, fd=fd@entry=5, lock_already_held=lock_already_held@entry=false) at fdcache_entity.cpp:356
#5  0x0000000000470143 in FdManager::GetExistFdEntity (this=0x6c43c0 <FdManager::singleton>, path=path@entry=0x7f8d3003cac0 "/Data/<reducted>/ziLCsXRu", existfd=5) at fdcache.cpp:617
#6  0x000000000048fb45 in AutoFdEntity::GetExistFdEntity (this=this@entry=0x7f8d37ffea40, path=path@entry=0x7f8d3003cac0 "/Data/<reducted>/ziLCsXRu", existfd=<optimized out>) at fdcache_auto.cpp:120
#7  0x000000000040c650 in s3fs_write (_path=<optimized out>,
    buf=0x7f8d4c0b7060 "\025g\264\272^?~\311!\240TÊÝ./\204\305\373\313\344\067\353\035s\354~\022T6g\353\253\355D\212\256\fy\263\223[\236\277X\261\372\331\032\244gXJ8\177\020\214JO\205X\232Z\211\376\246\062'\005HG[\v]\002\364bÖ\342;\351\201#Þ¯\377\326\a\245Ûµz\264>1\300\273\327Ù»\274<\202\227{jÚ±\265[\311.J\355\060\177\352\063r\344\321\303i\261\210\304\063G5\254T\210\225\203s\025$\026\253\204\347K\340\331ݲ\201az\373;\377î°¶\247\274Z\271:\311\325\066IM\016\302Z\337\335\346\201\373\365ã\032\320\315E\352â
\200r6xl\325HHXaP", <incomplete sequence \331>..., size=16384, offset=4912243697, fi=0x7f8d37ffec30) at s3fs.cpp:2357
#8  0x00007f8d4bd09a7c in fuse_fs_write_buf () from /lib64/libfuse.so.2
#9  0x00007f8d4bd09bf8 in fuse_lib_write_buf () from /lib64/libfuse.so.2
#10 0x00007f8d4bd12cf8 in fuse_ll_process_buf () from /lib64/libfuse.so.2
#11 0x00007f8d4bd0f401 in fuse_do_work () from /lib64/libfuse.so.2
#12 0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0
#13 0x00007f8d4a599ead in clone () from /lib64/libc.so.6

Thread 3 (Thread 0x7f8d45629700 (LWP 609)):
#0  0x00007f8d4a87a4ed in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007f8d4a875dcb in _L_lock_883 () from /lib64/libpthread.so.0
#2  0x00007f8d4a875c98 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x00000000004972ee in AutoLock::AutoLock (this=0x7f8d45628520, pmutex=<optimized out>, type=<optimized out>) at autolock.cpp:47
#4  0x00000000004738b3 in FdManager::ChangeEntityToTempPath (this=0x6c43c0 <FdManager::singleton>, ent=ent@entry=0x7f8d38028c50, path=0x7f8d38002df8 "/Data/<reducted>/zix3flNd") at fdcache.cpp:712
#5  0x0000000000478629 in FdEntity::NoCacheLoadAndPost (this=this@entry=0x7f8d38028c50, pseudo_obj=pseudo_obj@entry=0x7f8d3802d510, start=start@entry=0, size=size@entry=976145383) at fdcache_entity.cpp:1084
#6  0x000000000047baf8 in FdEntity::WriteMixMultipart (this=this@entry=0x7f8d38028c50, pseudo_obj=pseudo_obj@entry=0x7f8d3802d510,
    bytes=bytes@entry=0x7f8d4c0d9060 "\255\244\325\330<5m\025\177\200h\273z'\213\341\363\234\251çµ¢\024\246%\377\350\325X\367W\t\016\241\177\344\350F\334T\272G\226\006\253\271\247\v-Sı2\352\203\062\a\241\244\347\247\334#\243\357Væ¹s>\201\277\070=\313h\214\035×¥\367\005\031\206\005\226Õ*\260Ô\033\211h\315o\001\260ULk^\203", start=start@entry=976145383, size=size@entry=16384) at fdcache_entity.cpp:1996
#7  0x000000000047e987 in FdEntity::Write (this=this@entry=0x7f8d38028c50, fd=<optimized out>,
    bytes=bytes@entry=0x7f8d4c0d9060 "\255\244\325\330<5m\025\177\200h\273z'\213\341\363\234\251çµ¢\024\246%\377\350\325X\367W\t\016\241\177\344\350F\334T\272G\226\006\253\271\247\v-Sı2\352\203\062\a\241\244\347\247\334#\243\357Væ¹s>\201\277\070=\313h\214\035×¥\367\005\031\206\005\226Õ*\260Ô\033\211h\315o\001\260ULk^\203", start=start@entry=976145383, size=size@entry=16384) at fdcache_entity.cpp:1797
#8  0x000000000040c674 in s3fs_write (_path=<optimized out>,
    buf=0x7f8d4c0d9060 "\255\244\325\330<5m\025\177\200h\273z'\213\341\363\234\251çµ¢\024\246%\377\350\325X\367W\t\016\241\177\344\350F\334T\272G\226\006\253\271\247\v-Sı2\352\203\062\a\241\244\347\247\334#\243\357Væ¹s>\201\277\070=\313h\214\035×¥\367\005\031\206\005\226Õ*\260Ô\033\211h\315o\001\260ULk^\203", size=16384, offset=976145383, fi=0x7f8d45628c30) at s3fs.cpp:2362
#9  0x00007f8d4bd09a7c in fuse_fs_write_buf () from /lib64/libfuse.so.2
#10 0x00007f8d4bd09bf8 in fuse_lib_write_buf () from /lib64/libfuse.so.2
#11 0x00007f8d4bd12cf8 in fuse_ll_process_buf () from /lib64/libfuse.so.2
#12 0x00007f8d4bd0f401 in fuse_do_work () from /lib64/libfuse.so.2
#13 0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0
#14 0x00007f8d4a599ead in clone () from /lib64/libc.so.6

Thread 2 (Thread 0x7f8d45e2a700 (LWP 32642)):
#0  0x00007f8d4a87a6fd in read () from /lib64/libpthread.so.0
#1  0x00007f8d4bd0ed62 in fuse_kern_chan_receive () from /lib64/libfuse.so.2
#2  0x00007f8d4bd0fd59 in fuse_ll_receive_buf () from /lib64/libfuse.so.2
#3  0x00007f8d4bd0f37e in fuse_do_work () from /lib64/libfuse.so.2
#4  0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0
#5  0x00007f8d4a599ead in clone () from /lib64/libc.so.6

Thread 1 (Thread 0x7f8d4662b700 (LWP 32640)):
#0  0x00007f8d4a87a4ed in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007f8d4a875dcb in _L_lock_883 () from /lib64/libpthread.so.0
#2  0x00007f8d4a875c98 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x00000000004972ee in AutoLock::AutoLock (this=0x7f8d4662a830, pmutex=<optimized out>, type=<optimized out>) at autolock.cpp:47
#4  0x0000000000474401 in FdManager::Open (this=this@entry=0x6c43c0 <FdManager::singleton>, fd=@0x7f8d4662a9b8: -1, path=path@entry=0x7f8d400389d0 "/", pmeta=pmeta@entry=0x0, size=size@entry=-1, time=time@entry=-1,
    flags=flags@entry=0, force_tmpfile=force_tmpfile@entry=false, is_create=is_create@entry=false, type=type@entry=AutoLock::NONE) at fdcache.cpp:532
#5  0x0000000000475025 in FdManager::OpenExistFdEntity (this=0x6c43c0 <FdManager::singleton>, path=path@entry=0x7f8d400389d0 "/", fd=@0x7f8d4662a9b8: -1, flags=flags@entry=0) at fdcache.cpp:631
#6  0x000000000048fb76 in AutoFdEntity::OpenExistFdEntity (this=this@entry=0x7f8d4662a9b0, path=path@entry=0x7f8d400389d0 "/", flags=flags@entry=0) at fdcache_auto.cpp:130
#7  0x0000000000416561 in s3fs_getattr (_path=<optimized out>, stbuf=0x7f8d4662aad0) at s3fs.cpp:792
#8  0x00007f8d4bd0801d in fuse_lib_getattr () from /lib64/libfuse.so.2
#9  0x00007f8d4bd12445 in do_getattr () from /lib64/libfuse.so.2
#10 0x00007f8d4bd12b6b in fuse_ll_process_buf () from /lib64/libfuse.so.2
#11 0x00007f8d4bd0f401 in fuse_do_work () from /lib64/libfuse.so.2
#12 0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0
#13 0x00007f8d4a599ead in clone () from /lib64/libc.so.6

Full backtrace:

# gdb --eval-command="thread apply all bt full" bin/s3fs core.32639
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-114.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /root/root/bin/s3fs...done.
[New LWP 32640]
[New LWP 32642]
[New LWP 609]
[New LWP 1009]
[New LWP 1014]
[New LWP 1251]
[New LWP 4782]
[New LWP 32639]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/root/root/bin/s3fs'.
#0  0x00007f8d4a87a4ed in __lll_lock_wait () from /lib64/libpthread.so.0

Thread 8 (Thread 0x7f8d4c140880 (LWP 32639)):
#0  0x00007f8d4a879adb in do_futex_wait.constprop.1 () from /lib64/libpthread.so.0
No symbol table info available.
#1  0x00007f8d4a879b6f in __new_sem_wait_slow.constprop.0 () from /lib64/libpthread.so.0
No symbol table info available.
#2  0x00007f8d4a879c0b in sem_wait@@GLIBC_2.2.5 () from /lib64/libpthread.so.0
No symbol table info available.
#3  0x00007f8d4bd0f5c8 in fuse_session_loop_mt () from /lib64/libfuse.so.2
No symbol table info available.
#4  0x00007f8d4bd14ce7 in fuse_loop_mt () from /lib64/libfuse.so.2
No symbol table info available.
#5  0x00007f8d4bd17875 in fuse_main_common () from /lib64/libfuse.so.2
No symbol table info available.
#6  0x0000000000409d19 in main (argc=<optimized out>, argv=<optimized out>) at s3fs.cpp:5289
        option_index = 0
        long_opts = {{name = 0x499a74 "help", has_arg = 0, flag = 0x0, val = 104}, {name = 0x499970 "version", has_arg = 0, flag = 0x0, val = 0}, {name = 0x4995bb "debug", has_arg = 0, flag = 0x0, val = 100}, {
            name = 0x499a79 "incomplete-mpu-list", has_arg = 0, flag = 0x0, val = 117}, {name = 0x499a8d "incomplete-mpu-abort", has_arg = 2, flag = 0x0, val = 97}, {name = 0x0, has_arg = 0, flag = 0x0, val = 0}}
        found = <optimized out>
        fuse_res = <optimized out>
        __func__ = "main"
        ch = <optimized out>
        s3fs_oper = {getattr = 0x4163d0 <s3fs_getattr(char const*, stat*)>, readlink = 0x419df0 <s3fs_readlink(char const*, char*, size_t)>, getdir = 0x0, mknod = 0x4223f0 <s3fs_mknod(char const*, mode_t, dev_t)>,
          mkdir = 0x41e140 <s3fs_mkdir(char const*, mode_t)>, unlink = 0x414590 <s3fs_unlink(char const*)>, rmdir = 0x41a610 <s3fs_rmdir(char const*)>, symlink = 0x427930 <s3fs_symlink(char const*, char const*)>,
          rename = 0x425110 <s3fs_rename(char const*, char const*)>, link = 0x40ab90 <s3fs_link(char const*, char const*)>, chmod = 0x41fe10 <s3fs_chmod(char const*, mode_t)>,
          chown = 0x41f1d0 <s3fs_chown(char const*, uid_t, gid_t)>, truncate = 0x426ba0 <s3fs_truncate(char const*, off_t)>, utime = 0x0, open = 0x418c30 <s3fs_open(char const*, fuse_file_info*)>,
          read = 0x40ccc0 <s3fs_read(char const*, char*, size_t, off_t, fuse_file_info*)>, write = 0x40c4a0 <s3fs_write(char const*, char const*, size_t, off_t, fuse_file_info*)>,
          statfs = 0x40a640 <s3fs_statfs(char const*, statvfs*)>, flush = 0x4160d0 <s3fs_flush(char const*, fuse_file_info*)>, release = 0x40bf20 <s3fs_release(char const*, fuse_file_info*)>,
          fsync = 0x40aff0 <s3fs_fsync(char const*, int, fuse_file_info*)>, setxattr = 0x0, getxattr = 0x0, listxattr = 0x0, removexattr = 0x0, opendir = 0x415e70 <s3fs_opendir(char const*, fuse_file_info*)>,
          readdir = 0x414a70 <s3fs_readdir(char const*, void*, fuse_fill_dir_t, off_t, fuse_file_info*)>, releasedir = 0x0, fsyncdir = 0x0, init = 0x416d00 <s3fs_init(fuse_conn_info*)>, destroy = 0x40b2b0 <s3fs_destroy(void*)>,
          access = 0x414800 <s3fs_access(char const*, int)>, create = 0x4227d0 <s3fs_create(char const*, mode_t, fuse_file_info*)>, ftruncate = 0x0, fgetattr = 0x0, lock = 0x0,
          utimens = 0x41e400 <s3fs_utimens(char const*, timespec const*)>, bmap = 0x0, flag_nullpath_ok = 0, flag_nopath = 0, flag_utime_omit_ok = 0, flag_reserved = 0, ioctl = 0x0, poll = 0x0, write_buf = 0x0, read_buf = 0x0,
          flock = 0x0, fallocate = 0x0}
        incomp_abort_time = 86400
        singletonLog = {static NEST_MAX = 4, static nest_spaces = {0x49996f "", 0x4a67ea "  ", 0x4a67e8 "    ", 0x4a67e6 "      "}, static LOGFILEENV = 0x4a6892 <S3fsLog::LOGFILEENV> "S3FS_LOGFILE",
          static MSGTIMESTAMP = <same as static member of an already seen type>, static pSingleton = 0x7ffe6efc551a, static debug_level = S3fsLog::LEVEL_WARN, static logfp = 0x0, static plogfile = 0x0, static time_stamp = true}
        custom_args = {argc = 4, argv = 0x20e5b20, allocated = 1}

Thread 7 (Thread 0x7f8d35ffb700 (LWP 4782)):
#0  0x00007f8d4a87a4ed in __lll_lock_wait () from /lib64/libpthread.so.0
No symbol table info available.
#1  0x00007f8d4a875dcb in _L_lock_883 () from /lib64/libpthread.so.0
No symbol table info available.
#2  0x00007f8d4a875c98 in pthread_mutex_lock () from /lib64/libpthread.so.0
No symbol table info available.
#3  0x00000000004972ee in AutoLock::AutoLock (this=0x7f8d35ffa7e0, pmutex=<optimized out>, type=<optimized out>) at autolock.cpp:47
        result = <optimized out>
        __func__ = "AutoLock"
#4  0x0000000000474401 in FdManager::Open (this=this@entry=0x6c43c0 <FdManager::singleton>, fd=@0x7f8d35ffa968: -1, path=path@entry=0x7f8d2400f2a0 "/Data", pmeta=pmeta@entry=0x0, size=size@entry=-1, time=time@entry=-1,
Missing separate debuginfos, use: debuginfo-install cyrus-sasl-lib-2.1.26-23.el7.x86_64 fuse-libs-2.9.2-11.el7.x86_64 glibc-2.17-260.el7_6.3.x86_64 keyutils-libs-1.5.8-3.el7.x86_64 krb5-libs-1.15.1-37.el7_6.x86_64 libcom_err-1.42.9-13.el7.x86_64 libcurl-7.29.0-51.el7.x86_64 libgcc-4.8.5-36.el7_6.1.x86_64 libidn-1.28-4.el7.x86_64 libselinux-2.5-14.1.el7.x86_64 libssh2-1.4.3-12.el7.x86_64 libstdc++-4.8.5-36.el7_6.1.x86_64 libxml2-2.9.1-6.el7_2.3.x86_64 nspr-4.19.0-1.el7_5.x86_64 nss-3.36.0-7.1.el7_6.x86_64 nss-softokn-freebl-3.36.0-5.el7_5.x86_64 nss-util-3.36.0-1.1.el7_6.x86_64 openldap-2.4.44-21.el7_6.x86_64 openssl-libs-1.0.2k-16.el7_6.1.x86_64 pcre-8.32-17.el7.x86_64 xz-libs-5.2.2-1.el7.x86_64 zlib-1.2.7-18.el7.x86_64
    flags=flags@entry=0, force_tmpfile=force_tmpfile@entry=false, is_create=is_create@entry=false, type=type@entry=AutoLock::NONE) at fdcache.cpp:532
        __func__ = "Open"
        auto_lock = {auto_mutex = 0x6c4380 <FdManager::fd_manager_lock>, is_lock_acquired = false}
        iter = <optimized out>
        ent = <optimized out>
#5  0x0000000000475025 in FdManager::OpenExistFdEntity (this=0x6c43c0 <FdManager::singleton>, path=path@entry=0x7f8d2400f2a0 "/Data", fd=@0x7f8d35ffa968: -1, flags=flags@entry=0) at fdcache.cpp:631
        __func__ = "OpenExistFdEntity"
        ent = <optimized out>
#6  0x000000000048fb76 in AutoFdEntity::OpenExistFdEntity (this=this@entry=0x7f8d35ffa960, path=path@entry=0x7f8d2400f2a0 "/Data", flags=flags@entry=0) at fdcache_auto.cpp:130
No locals.
#7  0x0000000000416561 in s3fs_getattr (_path=<optimized out>, stbuf=0x7f8d35ffab00) at s3fs.cpp:792
        autoent = {pFdEntity = 0x0, pseudo_fd = -1}
        ent = <optimized out>
        path_buf = ""
        path = 0x7f8d2400f2a0 "/Data"
        result = <optimized out>
        __func__ = "s3fs_getattr"
#8  0x00007f8d4bd079c8 in lookup_path () from /lib64/libfuse.so.2
No symbol table info available.
#9  0x00007f8d4bd07bfd in fuse_lib_lookup () from /lib64/libfuse.so.2
No symbol table info available.
#10 0x00007f8d4bd12b6b in fuse_ll_process_buf () from /lib64/libfuse.so.2
No symbol table info available.
#11 0x00007f8d4bd0f401 in fuse_do_work () from /lib64/libfuse.so.2
No symbol table info available.
#12 0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#13 0x00007f8d4a599ead in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 6 (Thread 0x7f8d367fc700 (LWP 1251)):
#0  0x00007f8d4a87a6fd in read () from /lib64/libpthread.so.0
No symbol table info available.
#1  0x00007f8d4bd0ed62 in fuse_kern_chan_receive () from /lib64/libfuse.so.2
No symbol table info available.
#2  0x00007f8d4bd0fd59 in fuse_ll_receive_buf () from /lib64/libfuse.so.2
No symbol table info available.
#3  0x00007f8d4bd0f37e in fuse_do_work () from /lib64/libfuse.so.2
No symbol table info available.
#4  0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007f8d4a599ead in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 5 (Thread 0x7f8d377fe700 (LWP 1014)):
#0  0x00007f8d4a87a4ed in __lll_lock_wait () from /lib64/libpthread.so.0
No symbol table info available.
#1  0x00007f8d4a875dcb in _L_lock_883 () from /lib64/libpthread.so.0
No symbol table info available.
#2  0x00007f8d4a875c98 in pthread_mutex_lock () from /lib64/libpthread.so.0
No symbol table info available.
#3  0x00000000004972ee in AutoLock::AutoLock (this=0x7f8d377fd520, pmutex=<optimized out>, type=<optimized out>) at autolock.cpp:47
        result = <optimized out>
        __func__ = "AutoLock"
#4  0x00000000004738b3 in FdManager::ChangeEntityToTempPath (this=0x6c43c0 <FdManager::singleton>, ent=ent@entry=0x7f8d3802d5d0, path=0x7f8d380293c8 "/Data/<reducted>/ziqGxycK") at fdcache.cpp:712
        auto_lock = {auto_mutex = 0x6c4380 <FdManager::fd_manager_lock>, is_lock_acquired = false}
#5  0x0000000000478629 in FdEntity::NoCacheLoadAndPost (this=this@entry=0x7f8d3802d5d0, pseudo_obj=pseudo_obj@entry=0x7f8d38003700, start=start@entry=0, size=size@entry=5074290069) at fdcache_entity.cpp:1084
        result = 0
        __func__ = "NoCacheLoadAndPost"
        ptmpfp = <optimized out>
        tmpfd = <optimized out>
#6  0x000000000047baf8 in FdEntity::WriteMixMultipart (this=this@entry=0x7f8d3802d5d0, pseudo_obj=pseudo_obj@entry=0x7f8d38003700,
    bytes=bytes@entry=0x7f8d4c095060 "G\365\n9mպܴØS\251Q\207\312ȱ\030F\332qT\243\352\321\031\340\020\f+h\210Õ±\224\022\061h(\255D\212\343\022FÔb\234\216\067\351\225 -\225bCB\t U\366\017HG\360\021%\233A\343(.3\023\264Z\241\305u\264É\217\350\335\022\221dD\202R(?L\366\201\322A\034\025\364~\023\034\"A\220\347\344\375\310\360\360\360:\021%\006\021lP\376v\277N$\003\300u&\006\227\241\244\\bbA\302\066\064\"\036\206\315\na\337\277\023\240\351xol\027\367Oo\333s\361Õ\037~\372\362w\177\373\354\344\356\217>\274\360\376\253G\247\276\277\371/\237\237>\275\177\317\366\245\351|7"..., start=start@entry=5074290069, size=size@entry=16384) at fdcache_entity.cpp:1996
        restsize = <optimized out>
        __func__ = "WriteMixMultipart"
        result = 0
        wsize = <optimized out>
#7  0x000000000047e987 in FdEntity::Write (this=this@entry=0x7f8d3802d5d0, fd=<optimized out>,
    bytes=bytes@entry=0x7f8d4c095060 "G\365\n9mպܴØS\251Q\207\312ȱ\030F\332qT\243\352\321\031\340\020\f+h\210Õ±\224\022\061h(\255D\212\343\022FÔb\234\216\067\351\225 -\225bCB\t U\366\017HG\360\021%\233A\343(.3\023\264Z\241\305u\264É\217\350\335\022\221dD\202R(?L\366\201\322A\034\025\364~\023\034\"A\220\347\344\375\310\360\360\360:\021%\006\021lP\376v\277N$\003\300u&\006\227\241\244\\bbA\302\066\064\"\036\206\315\na\337\277\023\240\351xol\027\367Oo\333s\361Õ\037~\372\362w\177\373\354\344\356\217>\274\360\376\253G\247\276\277\371/\237\237>\275\177\317\366\245\351|7"..., start=start@entry=5074290069, size=size@entry=16384) at fdcache_entity.cpp:1797
        __func__ = "Write"
        pseudo_obj = 0x7f8d38003700
        auto_lock = {auto_mutex = 0x7f8d3802d5d0, is_lock_acquired = true}
        auto_lock2 = {auto_mutex = 0x7f8d3802d688, is_lock_acquired = true}
#8  0x000000000040c674 in s3fs_write (_path=<optimized out>,
    buf=0x7f8d4c095060 "G\365\n9mպܴØS\251Q\207\312ȱ\030F\332qT\243\352\321\031\340\020\f+h\210Õ±\224\022\061h(\255D\212\343\022FÔb\234\216\067\351\225 -\225bCB\t U\366\017HG\360\021%\233A\343(.3\023\264Z\241\305u\264É\217\350\335\022\221dD\202R(?L\366\201\322A\034\025\364~\023\034\"A\220\347\344\375\310\360\360\360:\021%\006\021lP\376v\277N$\003\300u&\006\227\241\244\\bbA\302\066\064\"\036\206\315\na\337\277\023\240\351xol\027\367Oo\333s\361Õ\037~\372\362w\177\373\354\344\356\217>\274\360\376\253G\247\276\277\371/\237\237>\275\177\317\366\245\351|7"..., size=16384, offset=5074290069, fi=0x7f8d377fdc30) at s3fs.cpp:2362
        __func__ = "s3fs_write"
        ent = 0x7f8d3802d5d0
        path_buf = ""
        path = 0x7f8d28036110 "/Data/<reducted>/ziqGxycK"
        res = <optimized out>
        autoent = {pFdEntity = 0x0, pseudo_fd = -1}
#9  0x00007f8d4bd09a7c in fuse_fs_write_buf () from /lib64/libfuse.so.2
No symbol table info available.
#10 0x00007f8d4bd09bf8 in fuse_lib_write_buf () from /lib64/libfuse.so.2
No symbol table info available.
#11 0x00007f8d4bd12cf8 in fuse_ll_process_buf () from /lib64/libfuse.so.2
No symbol table info available.
#12 0x00007f8d4bd0f401 in fuse_do_work () from /lib64/libfuse.so.2
No symbol table info available.
#13 0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#14 0x00007f8d4a599ead in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 4 (Thread 0x7f8d37fff700 (LWP 1009)):
#0  0x00007f8d4a87a4ed in __lll_lock_wait () from /lib64/libpthread.so.0
No symbol table info available.
#1  0x00007f8d4a875dcb in _L_lock_883 () from /lib64/libpthread.so.0
No symbol table info available.
#2  0x00007f8d4a875c98 in pthread_mutex_lock () from /lib64/libpthread.so.0
No symbol table info available.
#3  0x00000000004972ee in AutoLock::AutoLock (this=0x7f8d37ffe910, pmutex=<optimized out>, type=<optimized out>) at autolock.cpp:47
        result = <optimized out>
        __func__ = "AutoLock"
#4  0x000000000047d726 in FdEntity::FindPseudoFd (this=0x7f8d38028c50, fd=fd@entry=5, lock_already_held=lock_already_held@entry=false) at fdcache_entity.cpp:356
        auto_lock = {auto_mutex = 0x7f8d38028c50, is_lock_acquired = true}
#5  0x0000000000470143 in FdManager::GetExistFdEntity (this=0x6c43c0 <FdManager::singleton>, path=path@entry=0x7f8d3003cac0 "/Data/<reducted>/ziLCsXRu", existfd=5) at fdcache.cpp:617
        iter = {first = " __S3FS_UNEXISTED_PATH_17180b0b__ / /Data/<reducted>/zix3flNd", second = }
        __func__ = "GetExistFdEntity"
        auto_lock = {auto_mutex = 0x6c4380 <FdManager::fd_manager_lock>, is_lock_acquired = true}
#6  0x000000000048fb45 in AutoFdEntity::GetExistFdEntity (this=this@entry=0x7f8d37ffea40, path=path@entry=0x7f8d3003cac0 "/Data/<reducted>/ziLCsXRu", existfd=<optimized out>) at fdcache_auto.cpp:120
No locals.
#7  0x000000000040c650 in s3fs_write (_path=<optimized out>,
    buf=0x7f8d4c0b7060 "\025g\264\272^?~\311!\240TÊÝ./\204\305\373\313\344\067\353\035s\354~\022T6g\353\253\355D\212\256\fy\263\223[\236\277X\261\372\331\032\244gXJ8\177\020\214JO\205X\232Z\211\376\246\062'\005HG[\v]\002\364bÖ\342\351\201#Þ¯\377\326\a\245Ûµz\264>1\300\273\327Ù»\274<\202\227{jÚ±\265[\311.J\355\060\177\352\063r\344\321\303i\261\210\304\063G5\254T\210\225\203s\025$\026\253\204\347K\340\331ݲ\201az\373;\377î°¶\247\274Z\271:\311\325\066IM\016\302Z\337\335\346\201\373\365ã\032\320\315E\352â
\200r6xl\325HHXaP", <incomplete sequence \331>..., size=16384, offset=4912243697, fi=0x7f8d37ffec30) at s3fs.cpp:2357
        __func__ = "s3fs_write"
        ent = <optimized out>
        path_buf = ""
        path = 0x7f8d3003cac0 "/Data/<reducted>/ziLCsXRu"
        res = <optimized out>
        autoent = {pFdEntity = 0x0, pseudo_fd = -1}
#8  0x00007f8d4bd09a7c in fuse_fs_write_buf () from /lib64/libfuse.so.2
No symbol table info available.
#9  0x00007f8d4bd09bf8 in fuse_lib_write_buf () from /lib64/libfuse.so.2
No symbol table info available.
#10 0x00007f8d4bd12cf8 in fuse_ll_process_buf () from /lib64/libfuse.so.2
No symbol table info available.
#11 0x00007f8d4bd0f401 in fuse_do_work () from /lib64/libfuse.so.2
No symbol table info available.
#12 0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#13 0x00007f8d4a599ead in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 3 (Thread 0x7f8d45629700 (LWP 609)):
#0  0x00007f8d4a87a4ed in __lll_lock_wait () from /lib64/libpthread.so.0
No symbol table info available.
#1  0x00007f8d4a875dcb in _L_lock_883 () from /lib64/libpthread.so.0
No symbol table info available.
#2  0x00007f8d4a875c98 in pthread_mutex_lock () from /lib64/libpthread.so.0
No symbol table info available.
#3  0x00000000004972ee in AutoLock::AutoLock (this=0x7f8d45628520, pmutex=<optimized out>, type=<optimized out>) at autolock.cpp:47
        result = <optimized out>
        __func__ = "AutoLock"
#4  0x00000000004738b3 in FdManager::ChangeEntityToTempPath (this=0x6c43c0 <FdManager::singleton>, ent=ent@entry=0x7f8d38028c50, path=0x7f8d38002df8 "/Data/<reducted>/zix3flNd") at fdcache.cpp:712
        auto_lock = {auto_mutex = 0x6c4380 <FdManager::fd_manager_lock>, is_lock_acquired = false}
#5  0x0000000000478629 in FdEntity::NoCacheLoadAndPost (this=this@entry=0x7f8d38028c50, pseudo_obj=pseudo_obj@entry=0x7f8d3802d510, start=start@entry=0, size=size@entry=976145383) at fdcache_entity.cpp:1084
        result = 0
        __func__ = "NoCacheLoadAndPost"
        ptmpfp = <optimized out>
        tmpfd = <optimized out>
#6  0x000000000047baf8 in FdEntity::WriteMixMultipart (this=this@entry=0x7f8d38028c50, pseudo_obj=pseudo_obj@entry=0x7f8d3802d510,
    bytes=bytes@entry=0x7f8d4c0d9060 "\255\244\325\330<5m\025\177\200h\273z'\213\341\363\234\251çµ¢\024\246%\377\350\325X\367W\t\016\241\177\344\350F\334T\272G\226\006\253\271\247\v-Sı2\352\203\062\a\241\244\347\247\334#\243\357Væ¹s>\201\277\070=\313h\214\035×¥\367\005\031\206\005\226Õ*\260Ô\033\211h\315o\001\260ULk^\203", start=start@entry=976145383, size=size@entry=16384) at fdcache_entity.cpp:1996
        restsize = <optimized out>
        __func__ = "WriteMixMultipart"
        result = 0
        wsize = <optimized out>
#7  0x000000000047e987 in FdEntity::Write (this=this@entry=0x7f8d38028c50, fd=<optimized out>,
    bytes=bytes@entry=0x7f8d4c0d9060 "\255\244\325\330<5m\025\177\200h\273z'\213\341\363\234\251çµ¢\024\246%\377\350\325X\367W\t\016\241\177\344\350F\334T\272G\226\006\253\271\247\v-Sı2\352\203\062\a\241\244\347\247\334#\243\357Væ¹s>\201\277\070=\313h\214\035×¥\367\005\031\206\005\226Õ*\260Ô\033\211h\315o\001\260ULk^\203", start=start@entry=976145383, size=size@entry=16384) at fdcache_entity.cpp:1797
        __func__ = "Write"
        pseudo_obj = 0x7f8d3802d510
        auto_lock = {auto_mutex = 0x7f8d38028c50, is_lock_acquired = true}
        auto_lock2 = {auto_mutex = 0x7f8d38028d08, is_lock_acquired = true}
#8  0x000000000040c674 in s3fs_write (_path=<optimized out>,
    buf=0x7f8d4c0d9060 "\255\244\325\330<5m\025\177\200h\273z'\213\341\363\234\251çµ¢\024\246%\377\350\325X\367W\t\016\241\177\344\350F\334T\272G\226\006\253\271\247\v-Sı2\352\203\062\a\241\244\347\247\334#\243\357Væ¹s>\201\277\070=\313h\214\035×¥\367\005\031\206\005\226Õ*\260Ô\033\211h\315o\001\260ULk^\203", size=16384, offset=976145383, fi=0x7f8d45628c30) at s3fs.cpp:2362
        __func__ = "s3fs_write"
        ent = 0x7f8d38028c50
        path_buf = ""
        path = 0x7f8d3c008f10 "/Data/<reducted>/zix3flNd"
        res = <optimized out>
        autoent = {pFdEntity = 0x0, pseudo_fd = -1}
#9  0x00007f8d4bd09a7c in fuse_fs_write_buf () from /lib64/libfuse.so.2
No symbol table info available.
#10 0x00007f8d4bd09bf8 in fuse_lib_write_buf () from /lib64/libfuse.so.2
No symbol table info available.
#11 0x00007f8d4bd12cf8 in fuse_ll_process_buf () from /lib64/libfuse.so.2
No symbol table info available.
#12 0x00007f8d4bd0f401 in fuse_do_work () from /lib64/libfuse.so.2
No symbol table info available.
#13 0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#14 0x00007f8d4a599ead in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 2 (Thread 0x7f8d45e2a700 (LWP 32642)):
#0  0x00007f8d4a87a6fd in read () from /lib64/libpthread.so.0
No symbol table info available.
#1  0x00007f8d4bd0ed62 in fuse_kern_chan_receive () from /lib64/libfuse.so.2
No symbol table info available.
#2  0x00007f8d4bd0fd59 in fuse_ll_receive_buf () from /lib64/libfuse.so.2
No symbol table info available.
#3  0x00007f8d4bd0f37e in fuse_do_work () from /lib64/libfuse.so.2
No symbol table info available.
#4  0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x00007f8d4a599ead in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 1 (Thread 0x7f8d4662b700 (LWP 32640)):
#0  0x00007f8d4a87a4ed in __lll_lock_wait () from /lib64/libpthread.so.0
No symbol table info available.
#1  0x00007f8d4a875dcb in _L_lock_883 () from /lib64/libpthread.so.0
No symbol table info available.
#2  0x00007f8d4a875c98 in pthread_mutex_lock () from /lib64/libpthread.so.0
No symbol table info available.
#3  0x00000000004972ee in AutoLock::AutoLock (this=0x7f8d4662a830, pmutex=<optimized out>, type=<optimized out>) at autolock.cpp:47
        result = <optimized out>
        __func__ = "AutoLock"
#4  0x0000000000474401 in FdManager::Open (this=this@entry=0x6c43c0 <FdManager::singleton>, fd=@0x7f8d4662a9b8: -1, path=path@entry=0x7f8d400389d0 "/", pmeta=pmeta@entry=0x0, size=size@entry=-1, time=time@entry=-1,
    flags=flags@entry=0, force_tmpfile=force_tmpfile@entry=false, is_create=is_create@entry=false, type=type@entry=AutoLock::NONE) at fdcache.cpp:532
        __func__ = "Open"
        auto_lock = {auto_mutex = 0x6c4380 <FdManager::fd_manager_lock>, is_lock_acquired = 144}
        iter = <optimized out>
        ent = <optimized out>
#5  0x0000000000475025 in FdManager::OpenExistFdEntity (this=0x6c43c0 <FdManager::singleton>, path=path@entry=0x7f8d400389d0 "/", fd=@0x7f8d4662a9b8: -1, flags=flags@entry=0) at fdcache.cpp:631
        __func__ = "OpenExistFdEntity"
        ent = <optimized out>
#6  0x000000000048fb76 in AutoFdEntity::OpenExistFdEntity (this=this@entry=0x7f8d4662a9b0, path=path@entry=0x7f8d400389d0 "/", flags=flags@entry=0) at fdcache_auto.cpp:130
No locals.
#7  0x0000000000416561 in s3fs_getattr (_path=<optimized out>, stbuf=0x7f8d4662aad0) at s3fs.cpp:792
        autoent = {pFdEntity = 0x0, pseudo_fd = -1}
        ent = <optimized out>
        path_buf = ""
        path = 0x7f8d400389d0 "/"
        result = <optimized out>
        __func__ = "s3fs_getattr"
#8  0x00007f8d4bd0801d in fuse_lib_getattr () from /lib64/libfuse.so.2
No symbol table info available.
#9  0x00007f8d4bd12445 in do_getattr () from /lib64/libfuse.so.2
No symbol table info available.
#10 0x00007f8d4bd12b6b in fuse_ll_process_buf () from /lib64/libfuse.so.2
No symbol table info available.
#11 0x00007f8d4bd0f401 in fuse_do_work () from /lib64/libfuse.so.2
No symbol table info available.
#12 0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#13 0x00007f8d4a599ead in clone () from /lib64/libc.so.6
No symbol table info available.
(gdb) q
Originally created by @CarstenGrohmann on GitHub (Dec 17, 2021). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1819 Dear S3FS Developer, Running the current master (5de92e9 - "Bump CI to Fedora 35 (#1806)") causes a deadlock. In this setup 4 parallel ZIP processes stores files in a bucket mounted with s3fs. The process got stuck with the following backtrace. The system log doesn't show any information. A separate debug log was not active. The error occurred just once and I don't have the ability to compile with Clang to use the ThreadSanitizer provide more information Do you have an idea about the root cause of this issue? Thank you, Carsten Short backtrace; ``` (gdb) thread apply all bt Thread 8 (Thread 0x7f8d4c140880 (LWP 32639)): #0 0x00007f8d4a879adb in do_futex_wait.constprop.1 () from /lib64/libpthread.so.0 #1 0x00007f8d4a879b6f in __new_sem_wait_slow.constprop.0 () from /lib64/libpthread.so.0 #2 0x00007f8d4a879c0b in sem_wait@@GLIBC_2.2.5 () from /lib64/libpthread.so.0 #3 0x00007f8d4bd0f5c8 in fuse_session_loop_mt () from /lib64/libfuse.so.2 #4 0x00007f8d4bd14ce7 in fuse_loop_mt () from /lib64/libfuse.so.2 #5 0x00007f8d4bd17875 in fuse_main_common () from /lib64/libfuse.so.2 #6 0x0000000000409d19 in main (argc=<optimized out>, argv=<optimized out>) at s3fs.cpp:5289 Thread 7 (Thread 0x7f8d35ffb700 (LWP 4782)): #0 0x00007f8d4a87a4ed in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x00007f8d4a875dcb in _L_lock_883 () from /lib64/libpthread.so.0 #2 0x00007f8d4a875c98 in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x00000000004972ee in AutoLock::AutoLock (this=0x7f8d35ffa7e0, pmutex=<optimized out>, type=<optimized out>) at autolock.cpp:47 #4 0x0000000000474401 in FdManager::Open (this=this@entry=0x6c43c0 <FdManager::singleton>, fd=@0x7f8d35ffa968: -1, path=path@entry=0x7f8d2400f2a0 "/Data", pmeta=pmeta@entry=0x0, size=size@entry=-1, time=time@entry=-1, flags=flags@entry=0, force_tmpfile=force_tmpfile@entry=false, is_create=is_create@entry=false, type=type@entry=AutoLock::NONE) at fdcache.cpp:532 #5 0x0000000000475025 in FdManager::OpenExistFdEntity (this=0x6c43c0 <FdManager::singleton>, path=path@entry=0x7f8d2400f2a0 "/Data", fd=@0x7f8d35ffa968: -1, flags=flags@entry=0) at fdcache.cpp:631 #6 0x000000000048fb76 in AutoFdEntity::OpenExistFdEntity (this=this@entry=0x7f8d35ffa960, path=path@entry=0x7f8d2400f2a0 "/Data", flags=flags@entry=0) at fdcache_auto.cpp:130 #7 0x0000000000416561 in s3fs_getattr (_path=<optimized out>, stbuf=0x7f8d35ffab00) at s3fs.cpp:792 #8 0x00007f8d4bd079c8 in lookup_path () from /lib64/libfuse.so.2 #9 0x00007f8d4bd07bfd in fuse_lib_lookup () from /lib64/libfuse.so.2 #10 0x00007f8d4bd12b6b in fuse_ll_process_buf () from /lib64/libfuse.so.2 #11 0x00007f8d4bd0f401 in fuse_do_work () from /lib64/libfuse.so.2 #12 0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0 #13 0x00007f8d4a599ead in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x7f8d367fc700 (LWP 1251)): #0 0x00007f8d4a87a6fd in read () from /lib64/libpthread.so.0 #1 0x00007f8d4bd0ed62 in fuse_kern_chan_receive () from /lib64/libfuse.so.2 #2 0x00007f8d4bd0fd59 in fuse_ll_receive_buf () from /lib64/libfuse.so.2 #3 0x00007f8d4bd0f37e in fuse_do_work () from /lib64/libfuse.so.2 #4 0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f8d4a599ead in clone () from /lib64/libc.so.6 Thread 5 (Thread 0x7f8d377fe700 (LWP 1014)): #0 0x00007f8d4a87a4ed in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x00007f8d4a875dcb in _L_lock_883 () from /lib64/libpthread.so.0 #2 0x00007f8d4a875c98 in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x00000000004972ee in AutoLock::AutoLock (this=0x7f8d377fd520, pmutex=<optimized out>, type=<optimized out>) at autolock.cpp:47 #4 0x00000000004738b3 in FdManager::ChangeEntityToTempPath (this=0x6c43c0 <FdManager::singleton>, ent=ent@entry=0x7f8d3802d5d0, path=0x7f8d380293c8 "/Data/<reducted>/ziqGxycK") at fdcache.cpp:712 #5 0x0000000000478629 in FdEntity::NoCacheLoadAndPost (this=this@entry=0x7f8d3802d5d0, pseudo_obj=pseudo_obj@entry=0x7f8d38003700, start=start@entry=0, size=size@entry=5074290069) at fdcache_entity.cpp:1084 #6 0x000000000047baf8 in FdEntity::WriteMixMultipart (this=this@entry=0x7f8d3802d5d0, pseudo_obj=pseudo_obj@entry=0x7f8d38003700, bytes=bytes@entry=0x7f8d4c095060 "G\365\n9mպܴØS\251Q\207\312ȱ\030F\332qT\243\352\321\031\340\020\f+h\210Õ±\224\022\061h(\255D\212\343\022FÔb\234\216\067\351\225 -\225bCB\t U\366\017HG\360\021%\233A\343(.3\023\264Z\241\305u\264É\217\350\335\022\221dD\202R(?L\366\201\322A\034\025\364~\023\034\"A\220\347\344\375\310\360\360\360:\021%\006\021lP\376v\277N$\003\300u&\006\227\241\244\\bbA\302\066\064\"\036\206\315\na\337\277\023\240\351xol\027\367Oo\333s\361Õ\037~\372\362w\177\373\354\344\356\217>\274\360\376\253G\247\276\277\371/\237\237>\275\177\317\366\245\351|7"..., start=start@entry=5074290069, size=size@entry=16384) at fdcache_entity.cpp:1996 #7 0x000000000047e987 in FdEntity::Write (this=this@entry=0x7f8d3802d5d0, fd=<optimized out>, bytes=bytes@entry=0x7f8d4c095060 "G\365\n9mպܴØS\251Q\207\312ȱ\030F\332qT\243\352\321\031\340\020\f+h\210Õ±\224\022\061h(\255D\212\343\022FÔb\234\216\067\351\225 -\225bCB\t U\366\017HG\360\021%\233A\343(.3\023\264Z\241\305u\264É\217\350\335\022\221dD\202R(?L\366\201\322A\034\025\364~\023\034\"A\220\347\344\375\310\360\360\360:\021%\006\021lP\376v\277N$\003\300u&\006\227\241\244\\bbA\302\066\064\"\036\206\315\na\337\277\023\240\351xol\027\367Oo\333s\361Õ\037~\372\362w\177\373\354\344\356\217>\274\360\376\253G\247\276\277\371/\237\237>\275\177\317\366\245\351|7"..., start=start@entry=5074290069, size=size@entry=16384) at fdcache_entity.cpp:1797 #8 0x000000000040c674 in s3fs_write (_path=<optimized out>, buf=0x7f8d4c095060 "G\365\n9mպܴØS\251Q\207\312ȱ\030F\332qT\243\352\321\031\340\020\f+h\210Õ±\224\022\061h(\255D\212\343\022FÔb\234\216\067\351\225 -\225bCB\t U\366\017HG\360\021%\233A\343(.3\023\264Z\241\305u\264É\217\350\335\022\221dD\202R(?L\366\201\322A\034\025\364~\023\034\"A\220\347\344\375\310\360\360\360:\021%\006\021lP\376v\277N$\003\300u&\006\227\241\244\\bbA\302\066\064\"\036\206\315\na\337\277\023\240\351xol\027\367Oo\333s\361Õ\037~\372\362w\177\373\354\344\356\217>\274\360\376\253G\247\276\277\371/\237\237>\275\177\317\366\245\351|7"..., size=16384, offset=5074290069, fi=0x7f8d377fdc30) at s3fs.cpp:2362 #9 0x00007f8d4bd09a7c in fuse_fs_write_buf () from /lib64/libfuse.so.2 #10 0x00007f8d4bd09bf8 in fuse_lib_write_buf () from /lib64/libfuse.so.2 #11 0x00007f8d4bd12cf8 in fuse_ll_process_buf () from /lib64/libfuse.so.2 #12 0x00007f8d4bd0f401 in fuse_do_work () from /lib64/libfuse.so.2 #13 0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0 #14 0x00007f8d4a599ead in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x7f8d37fff700 (LWP 1009)): #0 0x00007f8d4a87a4ed in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x00007f8d4a875dcb in _L_lock_883 () from /lib64/libpthread.so.0 #2 0x00007f8d4a875c98 in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x00000000004972ee in AutoLock::AutoLock (this=0x7f8d37ffe910, pmutex=<optimized out>, type=<optimized out>) at autolock.cpp:47 #4 0x000000000047d726 in FdEntity::FindPseudoFd (this=0x7f8d38028c50, fd=fd@entry=5, lock_already_held=lock_already_held@entry=false) at fdcache_entity.cpp:356 #5 0x0000000000470143 in FdManager::GetExistFdEntity (this=0x6c43c0 <FdManager::singleton>, path=path@entry=0x7f8d3003cac0 "/Data/<reducted>/ziLCsXRu", existfd=5) at fdcache.cpp:617 #6 0x000000000048fb45 in AutoFdEntity::GetExistFdEntity (this=this@entry=0x7f8d37ffea40, path=path@entry=0x7f8d3003cac0 "/Data/<reducted>/ziLCsXRu", existfd=<optimized out>) at fdcache_auto.cpp:120 #7 0x000000000040c650 in s3fs_write (_path=<optimized out>, buf=0x7f8d4c0b7060 "\025g\264\272^?~\311!\240TÊÝ./\204\305\373\313\344\067\353\035s\354~\022T6g\353\253\355D\212\256\fy\263\223[\236\277X\261\372\331\032\244gXJ8\177\020\214JO\205X\232Z\211\376\246\062'\005HG[\v]\002\364bÖ\342;\351\201#Þ¯\377\326\a\245Ûµz\264>1\300\273\327Ù»\274<\202\227{jÚ±\265[\311.J\355\060\177\352\063r\344\321\303i\261\210\304\063G5\254T\210\225\203s\025$\026\253\204\347K\340\331ݲ\201az\373;\377î°¶\247\274Z\271:\311\325\066IM\016\302Z\337\335\346\201\373\365ã\032\320\315E\352â \200r6xl\325HHXaP", <incomplete sequence \331>..., size=16384, offset=4912243697, fi=0x7f8d37ffec30) at s3fs.cpp:2357 #8 0x00007f8d4bd09a7c in fuse_fs_write_buf () from /lib64/libfuse.so.2 #9 0x00007f8d4bd09bf8 in fuse_lib_write_buf () from /lib64/libfuse.so.2 #10 0x00007f8d4bd12cf8 in fuse_ll_process_buf () from /lib64/libfuse.so.2 #11 0x00007f8d4bd0f401 in fuse_do_work () from /lib64/libfuse.so.2 #12 0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0 #13 0x00007f8d4a599ead in clone () from /lib64/libc.so.6 Thread 3 (Thread 0x7f8d45629700 (LWP 609)): #0 0x00007f8d4a87a4ed in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x00007f8d4a875dcb in _L_lock_883 () from /lib64/libpthread.so.0 #2 0x00007f8d4a875c98 in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x00000000004972ee in AutoLock::AutoLock (this=0x7f8d45628520, pmutex=<optimized out>, type=<optimized out>) at autolock.cpp:47 #4 0x00000000004738b3 in FdManager::ChangeEntityToTempPath (this=0x6c43c0 <FdManager::singleton>, ent=ent@entry=0x7f8d38028c50, path=0x7f8d38002df8 "/Data/<reducted>/zix3flNd") at fdcache.cpp:712 #5 0x0000000000478629 in FdEntity::NoCacheLoadAndPost (this=this@entry=0x7f8d38028c50, pseudo_obj=pseudo_obj@entry=0x7f8d3802d510, start=start@entry=0, size=size@entry=976145383) at fdcache_entity.cpp:1084 #6 0x000000000047baf8 in FdEntity::WriteMixMultipart (this=this@entry=0x7f8d38028c50, pseudo_obj=pseudo_obj@entry=0x7f8d3802d510, bytes=bytes@entry=0x7f8d4c0d9060 "\255\244\325\330<5m\025\177\200h\273z'\213\341\363\234\251çµ¢\024\246%\377\350\325X\367W\t\016\241\177\344\350F\334T\272G\226\006\253\271\247\v-Sı2\352\203\062\a\241\244\347\247\334#\243\357Væ¹s>\201\277\070=\313h\214\035×¥\367\005\031\206\005\226Õ*\260Ô\033\211h\315o\001\260ULk^\203", start=start@entry=976145383, size=size@entry=16384) at fdcache_entity.cpp:1996 #7 0x000000000047e987 in FdEntity::Write (this=this@entry=0x7f8d38028c50, fd=<optimized out>, bytes=bytes@entry=0x7f8d4c0d9060 "\255\244\325\330<5m\025\177\200h\273z'\213\341\363\234\251çµ¢\024\246%\377\350\325X\367W\t\016\241\177\344\350F\334T\272G\226\006\253\271\247\v-Sı2\352\203\062\a\241\244\347\247\334#\243\357Væ¹s>\201\277\070=\313h\214\035×¥\367\005\031\206\005\226Õ*\260Ô\033\211h\315o\001\260ULk^\203", start=start@entry=976145383, size=size@entry=16384) at fdcache_entity.cpp:1797 #8 0x000000000040c674 in s3fs_write (_path=<optimized out>, buf=0x7f8d4c0d9060 "\255\244\325\330<5m\025\177\200h\273z'\213\341\363\234\251çµ¢\024\246%\377\350\325X\367W\t\016\241\177\344\350F\334T\272G\226\006\253\271\247\v-Sı2\352\203\062\a\241\244\347\247\334#\243\357Væ¹s>\201\277\070=\313h\214\035×¥\367\005\031\206\005\226Õ*\260Ô\033\211h\315o\001\260ULk^\203", size=16384, offset=976145383, fi=0x7f8d45628c30) at s3fs.cpp:2362 #9 0x00007f8d4bd09a7c in fuse_fs_write_buf () from /lib64/libfuse.so.2 #10 0x00007f8d4bd09bf8 in fuse_lib_write_buf () from /lib64/libfuse.so.2 #11 0x00007f8d4bd12cf8 in fuse_ll_process_buf () from /lib64/libfuse.so.2 #12 0x00007f8d4bd0f401 in fuse_do_work () from /lib64/libfuse.so.2 #13 0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0 #14 0x00007f8d4a599ead in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x7f8d45e2a700 (LWP 32642)): #0 0x00007f8d4a87a6fd in read () from /lib64/libpthread.so.0 #1 0x00007f8d4bd0ed62 in fuse_kern_chan_receive () from /lib64/libfuse.so.2 #2 0x00007f8d4bd0fd59 in fuse_ll_receive_buf () from /lib64/libfuse.so.2 #3 0x00007f8d4bd0f37e in fuse_do_work () from /lib64/libfuse.so.2 #4 0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f8d4a599ead in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x7f8d4662b700 (LWP 32640)): #0 0x00007f8d4a87a4ed in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x00007f8d4a875dcb in _L_lock_883 () from /lib64/libpthread.so.0 #2 0x00007f8d4a875c98 in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x00000000004972ee in AutoLock::AutoLock (this=0x7f8d4662a830, pmutex=<optimized out>, type=<optimized out>) at autolock.cpp:47 #4 0x0000000000474401 in FdManager::Open (this=this@entry=0x6c43c0 <FdManager::singleton>, fd=@0x7f8d4662a9b8: -1, path=path@entry=0x7f8d400389d0 "/", pmeta=pmeta@entry=0x0, size=size@entry=-1, time=time@entry=-1, flags=flags@entry=0, force_tmpfile=force_tmpfile@entry=false, is_create=is_create@entry=false, type=type@entry=AutoLock::NONE) at fdcache.cpp:532 #5 0x0000000000475025 in FdManager::OpenExistFdEntity (this=0x6c43c0 <FdManager::singleton>, path=path@entry=0x7f8d400389d0 "/", fd=@0x7f8d4662a9b8: -1, flags=flags@entry=0) at fdcache.cpp:631 #6 0x000000000048fb76 in AutoFdEntity::OpenExistFdEntity (this=this@entry=0x7f8d4662a9b0, path=path@entry=0x7f8d400389d0 "/", flags=flags@entry=0) at fdcache_auto.cpp:130 #7 0x0000000000416561 in s3fs_getattr (_path=<optimized out>, stbuf=0x7f8d4662aad0) at s3fs.cpp:792 #8 0x00007f8d4bd0801d in fuse_lib_getattr () from /lib64/libfuse.so.2 #9 0x00007f8d4bd12445 in do_getattr () from /lib64/libfuse.so.2 #10 0x00007f8d4bd12b6b in fuse_ll_process_buf () from /lib64/libfuse.so.2 #11 0x00007f8d4bd0f401 in fuse_do_work () from /lib64/libfuse.so.2 #12 0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0 #13 0x00007f8d4a599ead in clone () from /lib64/libc.so.6 ``` Full backtrace: ``` # gdb --eval-command="thread apply all bt full" bin/s3fs core.32639 GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-114.el7 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /root/root/bin/s3fs...done. [New LWP 32640] [New LWP 32642] [New LWP 609] [New LWP 1009] [New LWP 1014] [New LWP 1251] [New LWP 4782] [New LWP 32639] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `/root/root/bin/s3fs'. #0 0x00007f8d4a87a4ed in __lll_lock_wait () from /lib64/libpthread.so.0 Thread 8 (Thread 0x7f8d4c140880 (LWP 32639)): #0 0x00007f8d4a879adb in do_futex_wait.constprop.1 () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007f8d4a879b6f in __new_sem_wait_slow.constprop.0 () from /lib64/libpthread.so.0 No symbol table info available. #2 0x00007f8d4a879c0b in sem_wait@@GLIBC_2.2.5 () from /lib64/libpthread.so.0 No symbol table info available. #3 0x00007f8d4bd0f5c8 in fuse_session_loop_mt () from /lib64/libfuse.so.2 No symbol table info available. #4 0x00007f8d4bd14ce7 in fuse_loop_mt () from /lib64/libfuse.so.2 No symbol table info available. #5 0x00007f8d4bd17875 in fuse_main_common () from /lib64/libfuse.so.2 No symbol table info available. #6 0x0000000000409d19 in main (argc=<optimized out>, argv=<optimized out>) at s3fs.cpp:5289 option_index = 0 long_opts = {{name = 0x499a74 "help", has_arg = 0, flag = 0x0, val = 104}, {name = 0x499970 "version", has_arg = 0, flag = 0x0, val = 0}, {name = 0x4995bb "debug", has_arg = 0, flag = 0x0, val = 100}, { name = 0x499a79 "incomplete-mpu-list", has_arg = 0, flag = 0x0, val = 117}, {name = 0x499a8d "incomplete-mpu-abort", has_arg = 2, flag = 0x0, val = 97}, {name = 0x0, has_arg = 0, flag = 0x0, val = 0}} found = <optimized out> fuse_res = <optimized out> __func__ = "main" ch = <optimized out> s3fs_oper = {getattr = 0x4163d0 <s3fs_getattr(char const*, stat*)>, readlink = 0x419df0 <s3fs_readlink(char const*, char*, size_t)>, getdir = 0x0, mknod = 0x4223f0 <s3fs_mknod(char const*, mode_t, dev_t)>, mkdir = 0x41e140 <s3fs_mkdir(char const*, mode_t)>, unlink = 0x414590 <s3fs_unlink(char const*)>, rmdir = 0x41a610 <s3fs_rmdir(char const*)>, symlink = 0x427930 <s3fs_symlink(char const*, char const*)>, rename = 0x425110 <s3fs_rename(char const*, char const*)>, link = 0x40ab90 <s3fs_link(char const*, char const*)>, chmod = 0x41fe10 <s3fs_chmod(char const*, mode_t)>, chown = 0x41f1d0 <s3fs_chown(char const*, uid_t, gid_t)>, truncate = 0x426ba0 <s3fs_truncate(char const*, off_t)>, utime = 0x0, open = 0x418c30 <s3fs_open(char const*, fuse_file_info*)>, read = 0x40ccc0 <s3fs_read(char const*, char*, size_t, off_t, fuse_file_info*)>, write = 0x40c4a0 <s3fs_write(char const*, char const*, size_t, off_t, fuse_file_info*)>, statfs = 0x40a640 <s3fs_statfs(char const*, statvfs*)>, flush = 0x4160d0 <s3fs_flush(char const*, fuse_file_info*)>, release = 0x40bf20 <s3fs_release(char const*, fuse_file_info*)>, fsync = 0x40aff0 <s3fs_fsync(char const*, int, fuse_file_info*)>, setxattr = 0x0, getxattr = 0x0, listxattr = 0x0, removexattr = 0x0, opendir = 0x415e70 <s3fs_opendir(char const*, fuse_file_info*)>, readdir = 0x414a70 <s3fs_readdir(char const*, void*, fuse_fill_dir_t, off_t, fuse_file_info*)>, releasedir = 0x0, fsyncdir = 0x0, init = 0x416d00 <s3fs_init(fuse_conn_info*)>, destroy = 0x40b2b0 <s3fs_destroy(void*)>, access = 0x414800 <s3fs_access(char const*, int)>, create = 0x4227d0 <s3fs_create(char const*, mode_t, fuse_file_info*)>, ftruncate = 0x0, fgetattr = 0x0, lock = 0x0, utimens = 0x41e400 <s3fs_utimens(char const*, timespec const*)>, bmap = 0x0, flag_nullpath_ok = 0, flag_nopath = 0, flag_utime_omit_ok = 0, flag_reserved = 0, ioctl = 0x0, poll = 0x0, write_buf = 0x0, read_buf = 0x0, flock = 0x0, fallocate = 0x0} incomp_abort_time = 86400 singletonLog = {static NEST_MAX = 4, static nest_spaces = {0x49996f "", 0x4a67ea " ", 0x4a67e8 " ", 0x4a67e6 " "}, static LOGFILEENV = 0x4a6892 <S3fsLog::LOGFILEENV> "S3FS_LOGFILE", static MSGTIMESTAMP = <same as static member of an already seen type>, static pSingleton = 0x7ffe6efc551a, static debug_level = S3fsLog::LEVEL_WARN, static logfp = 0x0, static plogfile = 0x0, static time_stamp = true} custom_args = {argc = 4, argv = 0x20e5b20, allocated = 1} Thread 7 (Thread 0x7f8d35ffb700 (LWP 4782)): #0 0x00007f8d4a87a4ed in __lll_lock_wait () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007f8d4a875dcb in _L_lock_883 () from /lib64/libpthread.so.0 No symbol table info available. #2 0x00007f8d4a875c98 in pthread_mutex_lock () from /lib64/libpthread.so.0 No symbol table info available. #3 0x00000000004972ee in AutoLock::AutoLock (this=0x7f8d35ffa7e0, pmutex=<optimized out>, type=<optimized out>) at autolock.cpp:47 result = <optimized out> __func__ = "AutoLock" #4 0x0000000000474401 in FdManager::Open (this=this@entry=0x6c43c0 <FdManager::singleton>, fd=@0x7f8d35ffa968: -1, path=path@entry=0x7f8d2400f2a0 "/Data", pmeta=pmeta@entry=0x0, size=size@entry=-1, time=time@entry=-1, Missing separate debuginfos, use: debuginfo-install cyrus-sasl-lib-2.1.26-23.el7.x86_64 fuse-libs-2.9.2-11.el7.x86_64 glibc-2.17-260.el7_6.3.x86_64 keyutils-libs-1.5.8-3.el7.x86_64 krb5-libs-1.15.1-37.el7_6.x86_64 libcom_err-1.42.9-13.el7.x86_64 libcurl-7.29.0-51.el7.x86_64 libgcc-4.8.5-36.el7_6.1.x86_64 libidn-1.28-4.el7.x86_64 libselinux-2.5-14.1.el7.x86_64 libssh2-1.4.3-12.el7.x86_64 libstdc++-4.8.5-36.el7_6.1.x86_64 libxml2-2.9.1-6.el7_2.3.x86_64 nspr-4.19.0-1.el7_5.x86_64 nss-3.36.0-7.1.el7_6.x86_64 nss-softokn-freebl-3.36.0-5.el7_5.x86_64 nss-util-3.36.0-1.1.el7_6.x86_64 openldap-2.4.44-21.el7_6.x86_64 openssl-libs-1.0.2k-16.el7_6.1.x86_64 pcre-8.32-17.el7.x86_64 xz-libs-5.2.2-1.el7.x86_64 zlib-1.2.7-18.el7.x86_64 flags=flags@entry=0, force_tmpfile=force_tmpfile@entry=false, is_create=is_create@entry=false, type=type@entry=AutoLock::NONE) at fdcache.cpp:532 __func__ = "Open" auto_lock = {auto_mutex = 0x6c4380 <FdManager::fd_manager_lock>, is_lock_acquired = false} iter = <optimized out> ent = <optimized out> #5 0x0000000000475025 in FdManager::OpenExistFdEntity (this=0x6c43c0 <FdManager::singleton>, path=path@entry=0x7f8d2400f2a0 "/Data", fd=@0x7f8d35ffa968: -1, flags=flags@entry=0) at fdcache.cpp:631 __func__ = "OpenExistFdEntity" ent = <optimized out> #6 0x000000000048fb76 in AutoFdEntity::OpenExistFdEntity (this=this@entry=0x7f8d35ffa960, path=path@entry=0x7f8d2400f2a0 "/Data", flags=flags@entry=0) at fdcache_auto.cpp:130 No locals. #7 0x0000000000416561 in s3fs_getattr (_path=<optimized out>, stbuf=0x7f8d35ffab00) at s3fs.cpp:792 autoent = {pFdEntity = 0x0, pseudo_fd = -1} ent = <optimized out> path_buf = "" path = 0x7f8d2400f2a0 "/Data" result = <optimized out> __func__ = "s3fs_getattr" #8 0x00007f8d4bd079c8 in lookup_path () from /lib64/libfuse.so.2 No symbol table info available. #9 0x00007f8d4bd07bfd in fuse_lib_lookup () from /lib64/libfuse.so.2 No symbol table info available. #10 0x00007f8d4bd12b6b in fuse_ll_process_buf () from /lib64/libfuse.so.2 No symbol table info available. #11 0x00007f8d4bd0f401 in fuse_do_work () from /lib64/libfuse.so.2 No symbol table info available. #12 0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #13 0x00007f8d4a599ead in clone () from /lib64/libc.so.6 No symbol table info available. Thread 6 (Thread 0x7f8d367fc700 (LWP 1251)): #0 0x00007f8d4a87a6fd in read () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007f8d4bd0ed62 in fuse_kern_chan_receive () from /lib64/libfuse.so.2 No symbol table info available. #2 0x00007f8d4bd0fd59 in fuse_ll_receive_buf () from /lib64/libfuse.so.2 No symbol table info available. #3 0x00007f8d4bd0f37e in fuse_do_work () from /lib64/libfuse.so.2 No symbol table info available. #4 0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #5 0x00007f8d4a599ead in clone () from /lib64/libc.so.6 No symbol table info available. Thread 5 (Thread 0x7f8d377fe700 (LWP 1014)): #0 0x00007f8d4a87a4ed in __lll_lock_wait () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007f8d4a875dcb in _L_lock_883 () from /lib64/libpthread.so.0 No symbol table info available. #2 0x00007f8d4a875c98 in pthread_mutex_lock () from /lib64/libpthread.so.0 No symbol table info available. #3 0x00000000004972ee in AutoLock::AutoLock (this=0x7f8d377fd520, pmutex=<optimized out>, type=<optimized out>) at autolock.cpp:47 result = <optimized out> __func__ = "AutoLock" #4 0x00000000004738b3 in FdManager::ChangeEntityToTempPath (this=0x6c43c0 <FdManager::singleton>, ent=ent@entry=0x7f8d3802d5d0, path=0x7f8d380293c8 "/Data/<reducted>/ziqGxycK") at fdcache.cpp:712 auto_lock = {auto_mutex = 0x6c4380 <FdManager::fd_manager_lock>, is_lock_acquired = false} #5 0x0000000000478629 in FdEntity::NoCacheLoadAndPost (this=this@entry=0x7f8d3802d5d0, pseudo_obj=pseudo_obj@entry=0x7f8d38003700, start=start@entry=0, size=size@entry=5074290069) at fdcache_entity.cpp:1084 result = 0 __func__ = "NoCacheLoadAndPost" ptmpfp = <optimized out> tmpfd = <optimized out> #6 0x000000000047baf8 in FdEntity::WriteMixMultipart (this=this@entry=0x7f8d3802d5d0, pseudo_obj=pseudo_obj@entry=0x7f8d38003700, bytes=bytes@entry=0x7f8d4c095060 "G\365\n9mպܴØS\251Q\207\312ȱ\030F\332qT\243\352\321\031\340\020\f+h\210Õ±\224\022\061h(\255D\212\343\022FÔb\234\216\067\351\225 -\225bCB\t U\366\017HG\360\021%\233A\343(.3\023\264Z\241\305u\264É\217\350\335\022\221dD\202R(?L\366\201\322A\034\025\364~\023\034\"A\220\347\344\375\310\360\360\360:\021%\006\021lP\376v\277N$\003\300u&\006\227\241\244\\bbA\302\066\064\"\036\206\315\na\337\277\023\240\351xol\027\367Oo\333s\361Õ\037~\372\362w\177\373\354\344\356\217>\274\360\376\253G\247\276\277\371/\237\237>\275\177\317\366\245\351|7"..., start=start@entry=5074290069, size=size@entry=16384) at fdcache_entity.cpp:1996 restsize = <optimized out> __func__ = "WriteMixMultipart" result = 0 wsize = <optimized out> #7 0x000000000047e987 in FdEntity::Write (this=this@entry=0x7f8d3802d5d0, fd=<optimized out>, bytes=bytes@entry=0x7f8d4c095060 "G\365\n9mպܴØS\251Q\207\312ȱ\030F\332qT\243\352\321\031\340\020\f+h\210Õ±\224\022\061h(\255D\212\343\022FÔb\234\216\067\351\225 -\225bCB\t U\366\017HG\360\021%\233A\343(.3\023\264Z\241\305u\264É\217\350\335\022\221dD\202R(?L\366\201\322A\034\025\364~\023\034\"A\220\347\344\375\310\360\360\360:\021%\006\021lP\376v\277N$\003\300u&\006\227\241\244\\bbA\302\066\064\"\036\206\315\na\337\277\023\240\351xol\027\367Oo\333s\361Õ\037~\372\362w\177\373\354\344\356\217>\274\360\376\253G\247\276\277\371/\237\237>\275\177\317\366\245\351|7"..., start=start@entry=5074290069, size=size@entry=16384) at fdcache_entity.cpp:1797 __func__ = "Write" pseudo_obj = 0x7f8d38003700 auto_lock = {auto_mutex = 0x7f8d3802d5d0, is_lock_acquired = true} auto_lock2 = {auto_mutex = 0x7f8d3802d688, is_lock_acquired = true} #8 0x000000000040c674 in s3fs_write (_path=<optimized out>, buf=0x7f8d4c095060 "G\365\n9mպܴØS\251Q\207\312ȱ\030F\332qT\243\352\321\031\340\020\f+h\210Õ±\224\022\061h(\255D\212\343\022FÔb\234\216\067\351\225 -\225bCB\t U\366\017HG\360\021%\233A\343(.3\023\264Z\241\305u\264É\217\350\335\022\221dD\202R(?L\366\201\322A\034\025\364~\023\034\"A\220\347\344\375\310\360\360\360:\021%\006\021lP\376v\277N$\003\300u&\006\227\241\244\\bbA\302\066\064\"\036\206\315\na\337\277\023\240\351xol\027\367Oo\333s\361Õ\037~\372\362w\177\373\354\344\356\217>\274\360\376\253G\247\276\277\371/\237\237>\275\177\317\366\245\351|7"..., size=16384, offset=5074290069, fi=0x7f8d377fdc30) at s3fs.cpp:2362 __func__ = "s3fs_write" ent = 0x7f8d3802d5d0 path_buf = "" path = 0x7f8d28036110 "/Data/<reducted>/ziqGxycK" res = <optimized out> autoent = {pFdEntity = 0x0, pseudo_fd = -1} #9 0x00007f8d4bd09a7c in fuse_fs_write_buf () from /lib64/libfuse.so.2 No symbol table info available. #10 0x00007f8d4bd09bf8 in fuse_lib_write_buf () from /lib64/libfuse.so.2 No symbol table info available. #11 0x00007f8d4bd12cf8 in fuse_ll_process_buf () from /lib64/libfuse.so.2 No symbol table info available. #12 0x00007f8d4bd0f401 in fuse_do_work () from /lib64/libfuse.so.2 No symbol table info available. #13 0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #14 0x00007f8d4a599ead in clone () from /lib64/libc.so.6 No symbol table info available. Thread 4 (Thread 0x7f8d37fff700 (LWP 1009)): #0 0x00007f8d4a87a4ed in __lll_lock_wait () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007f8d4a875dcb in _L_lock_883 () from /lib64/libpthread.so.0 No symbol table info available. #2 0x00007f8d4a875c98 in pthread_mutex_lock () from /lib64/libpthread.so.0 No symbol table info available. #3 0x00000000004972ee in AutoLock::AutoLock (this=0x7f8d37ffe910, pmutex=<optimized out>, type=<optimized out>) at autolock.cpp:47 result = <optimized out> __func__ = "AutoLock" #4 0x000000000047d726 in FdEntity::FindPseudoFd (this=0x7f8d38028c50, fd=fd@entry=5, lock_already_held=lock_already_held@entry=false) at fdcache_entity.cpp:356 auto_lock = {auto_mutex = 0x7f8d38028c50, is_lock_acquired = true} #5 0x0000000000470143 in FdManager::GetExistFdEntity (this=0x6c43c0 <FdManager::singleton>, path=path@entry=0x7f8d3003cac0 "/Data/<reducted>/ziLCsXRu", existfd=5) at fdcache.cpp:617 iter = {first = " __S3FS_UNEXISTED_PATH_17180b0b__ / /Data/<reducted>/zix3flNd", second = } __func__ = "GetExistFdEntity" auto_lock = {auto_mutex = 0x6c4380 <FdManager::fd_manager_lock>, is_lock_acquired = true} #6 0x000000000048fb45 in AutoFdEntity::GetExistFdEntity (this=this@entry=0x7f8d37ffea40, path=path@entry=0x7f8d3003cac0 "/Data/<reducted>/ziLCsXRu", existfd=<optimized out>) at fdcache_auto.cpp:120 No locals. #7 0x000000000040c650 in s3fs_write (_path=<optimized out>, buf=0x7f8d4c0b7060 "\025g\264\272^?~\311!\240TÊÝ./\204\305\373\313\344\067\353\035s\354~\022T6g\353\253\355D\212\256\fy\263\223[\236\277X\261\372\331\032\244gXJ8\177\020\214JO\205X\232Z\211\376\246\062'\005HG[\v]\002\364bÖ\342\351\201#Þ¯\377\326\a\245Ûµz\264>1\300\273\327Ù»\274<\202\227{jÚ±\265[\311.J\355\060\177\352\063r\344\321\303i\261\210\304\063G5\254T\210\225\203s\025$\026\253\204\347K\340\331ݲ\201az\373;\377î°¶\247\274Z\271:\311\325\066IM\016\302Z\337\335\346\201\373\365ã\032\320\315E\352â \200r6xl\325HHXaP", <incomplete sequence \331>..., size=16384, offset=4912243697, fi=0x7f8d37ffec30) at s3fs.cpp:2357 __func__ = "s3fs_write" ent = <optimized out> path_buf = "" path = 0x7f8d3003cac0 "/Data/<reducted>/ziLCsXRu" res = <optimized out> autoent = {pFdEntity = 0x0, pseudo_fd = -1} #8 0x00007f8d4bd09a7c in fuse_fs_write_buf () from /lib64/libfuse.so.2 No symbol table info available. #9 0x00007f8d4bd09bf8 in fuse_lib_write_buf () from /lib64/libfuse.so.2 No symbol table info available. #10 0x00007f8d4bd12cf8 in fuse_ll_process_buf () from /lib64/libfuse.so.2 No symbol table info available. #11 0x00007f8d4bd0f401 in fuse_do_work () from /lib64/libfuse.so.2 No symbol table info available. #12 0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #13 0x00007f8d4a599ead in clone () from /lib64/libc.so.6 No symbol table info available. Thread 3 (Thread 0x7f8d45629700 (LWP 609)): #0 0x00007f8d4a87a4ed in __lll_lock_wait () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007f8d4a875dcb in _L_lock_883 () from /lib64/libpthread.so.0 No symbol table info available. #2 0x00007f8d4a875c98 in pthread_mutex_lock () from /lib64/libpthread.so.0 No symbol table info available. #3 0x00000000004972ee in AutoLock::AutoLock (this=0x7f8d45628520, pmutex=<optimized out>, type=<optimized out>) at autolock.cpp:47 result = <optimized out> __func__ = "AutoLock" #4 0x00000000004738b3 in FdManager::ChangeEntityToTempPath (this=0x6c43c0 <FdManager::singleton>, ent=ent@entry=0x7f8d38028c50, path=0x7f8d38002df8 "/Data/<reducted>/zix3flNd") at fdcache.cpp:712 auto_lock = {auto_mutex = 0x6c4380 <FdManager::fd_manager_lock>, is_lock_acquired = false} #5 0x0000000000478629 in FdEntity::NoCacheLoadAndPost (this=this@entry=0x7f8d38028c50, pseudo_obj=pseudo_obj@entry=0x7f8d3802d510, start=start@entry=0, size=size@entry=976145383) at fdcache_entity.cpp:1084 result = 0 __func__ = "NoCacheLoadAndPost" ptmpfp = <optimized out> tmpfd = <optimized out> #6 0x000000000047baf8 in FdEntity::WriteMixMultipart (this=this@entry=0x7f8d38028c50, pseudo_obj=pseudo_obj@entry=0x7f8d3802d510, bytes=bytes@entry=0x7f8d4c0d9060 "\255\244\325\330<5m\025\177\200h\273z'\213\341\363\234\251çµ¢\024\246%\377\350\325X\367W\t\016\241\177\344\350F\334T\272G\226\006\253\271\247\v-Sı2\352\203\062\a\241\244\347\247\334#\243\357Væ¹s>\201\277\070=\313h\214\035×¥\367\005\031\206\005\226Õ*\260Ô\033\211h\315o\001\260ULk^\203", start=start@entry=976145383, size=size@entry=16384) at fdcache_entity.cpp:1996 restsize = <optimized out> __func__ = "WriteMixMultipart" result = 0 wsize = <optimized out> #7 0x000000000047e987 in FdEntity::Write (this=this@entry=0x7f8d38028c50, fd=<optimized out>, bytes=bytes@entry=0x7f8d4c0d9060 "\255\244\325\330<5m\025\177\200h\273z'\213\341\363\234\251çµ¢\024\246%\377\350\325X\367W\t\016\241\177\344\350F\334T\272G\226\006\253\271\247\v-Sı2\352\203\062\a\241\244\347\247\334#\243\357Væ¹s>\201\277\070=\313h\214\035×¥\367\005\031\206\005\226Õ*\260Ô\033\211h\315o\001\260ULk^\203", start=start@entry=976145383, size=size@entry=16384) at fdcache_entity.cpp:1797 __func__ = "Write" pseudo_obj = 0x7f8d3802d510 auto_lock = {auto_mutex = 0x7f8d38028c50, is_lock_acquired = true} auto_lock2 = {auto_mutex = 0x7f8d38028d08, is_lock_acquired = true} #8 0x000000000040c674 in s3fs_write (_path=<optimized out>, buf=0x7f8d4c0d9060 "\255\244\325\330<5m\025\177\200h\273z'\213\341\363\234\251çµ¢\024\246%\377\350\325X\367W\t\016\241\177\344\350F\334T\272G\226\006\253\271\247\v-Sı2\352\203\062\a\241\244\347\247\334#\243\357Væ¹s>\201\277\070=\313h\214\035×¥\367\005\031\206\005\226Õ*\260Ô\033\211h\315o\001\260ULk^\203", size=16384, offset=976145383, fi=0x7f8d45628c30) at s3fs.cpp:2362 __func__ = "s3fs_write" ent = 0x7f8d38028c50 path_buf = "" path = 0x7f8d3c008f10 "/Data/<reducted>/zix3flNd" res = <optimized out> autoent = {pFdEntity = 0x0, pseudo_fd = -1} #9 0x00007f8d4bd09a7c in fuse_fs_write_buf () from /lib64/libfuse.so.2 No symbol table info available. #10 0x00007f8d4bd09bf8 in fuse_lib_write_buf () from /lib64/libfuse.so.2 No symbol table info available. #11 0x00007f8d4bd12cf8 in fuse_ll_process_buf () from /lib64/libfuse.so.2 No symbol table info available. #12 0x00007f8d4bd0f401 in fuse_do_work () from /lib64/libfuse.so.2 No symbol table info available. #13 0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #14 0x00007f8d4a599ead in clone () from /lib64/libc.so.6 No symbol table info available. Thread 2 (Thread 0x7f8d45e2a700 (LWP 32642)): #0 0x00007f8d4a87a6fd in read () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007f8d4bd0ed62 in fuse_kern_chan_receive () from /lib64/libfuse.so.2 No symbol table info available. #2 0x00007f8d4bd0fd59 in fuse_ll_receive_buf () from /lib64/libfuse.so.2 No symbol table info available. #3 0x00007f8d4bd0f37e in fuse_do_work () from /lib64/libfuse.so.2 No symbol table info available. #4 0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #5 0x00007f8d4a599ead in clone () from /lib64/libc.so.6 No symbol table info available. Thread 1 (Thread 0x7f8d4662b700 (LWP 32640)): #0 0x00007f8d4a87a4ed in __lll_lock_wait () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007f8d4a875dcb in _L_lock_883 () from /lib64/libpthread.so.0 No symbol table info available. #2 0x00007f8d4a875c98 in pthread_mutex_lock () from /lib64/libpthread.so.0 No symbol table info available. #3 0x00000000004972ee in AutoLock::AutoLock (this=0x7f8d4662a830, pmutex=<optimized out>, type=<optimized out>) at autolock.cpp:47 result = <optimized out> __func__ = "AutoLock" #4 0x0000000000474401 in FdManager::Open (this=this@entry=0x6c43c0 <FdManager::singleton>, fd=@0x7f8d4662a9b8: -1, path=path@entry=0x7f8d400389d0 "/", pmeta=pmeta@entry=0x0, size=size@entry=-1, time=time@entry=-1, flags=flags@entry=0, force_tmpfile=force_tmpfile@entry=false, is_create=is_create@entry=false, type=type@entry=AutoLock::NONE) at fdcache.cpp:532 __func__ = "Open" auto_lock = {auto_mutex = 0x6c4380 <FdManager::fd_manager_lock>, is_lock_acquired = 144} iter = <optimized out> ent = <optimized out> #5 0x0000000000475025 in FdManager::OpenExistFdEntity (this=0x6c43c0 <FdManager::singleton>, path=path@entry=0x7f8d400389d0 "/", fd=@0x7f8d4662a9b8: -1, flags=flags@entry=0) at fdcache.cpp:631 __func__ = "OpenExistFdEntity" ent = <optimized out> #6 0x000000000048fb76 in AutoFdEntity::OpenExistFdEntity (this=this@entry=0x7f8d4662a9b0, path=path@entry=0x7f8d400389d0 "/", flags=flags@entry=0) at fdcache_auto.cpp:130 No locals. #7 0x0000000000416561 in s3fs_getattr (_path=<optimized out>, stbuf=0x7f8d4662aad0) at s3fs.cpp:792 autoent = {pFdEntity = 0x0, pseudo_fd = -1} ent = <optimized out> path_buf = "" path = 0x7f8d400389d0 "/" result = <optimized out> __func__ = "s3fs_getattr" #8 0x00007f8d4bd0801d in fuse_lib_getattr () from /lib64/libfuse.so.2 No symbol table info available. #9 0x00007f8d4bd12445 in do_getattr () from /lib64/libfuse.so.2 No symbol table info available. #10 0x00007f8d4bd12b6b in fuse_ll_process_buf () from /lib64/libfuse.so.2 No symbol table info available. #11 0x00007f8d4bd0f401 in fuse_do_work () from /lib64/libfuse.so.2 No symbol table info available. #12 0x00007f8d4a873dd5 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #13 0x00007f8d4a599ead in clone () from /lib64/libc.so.6 No symbol table info available. (gdb) q ```
kerem closed this issue 2026-03-04 01:50:02 +03:00
kerem commented 2026-03-04 01:50:03 +03:00
Author
Owner
Copy link

@CarstenGrohmann commented on GitHub (Feb 15, 2022):

All s3fs instances have been running stable for a few weeks. Therefore, I am closing this topic and reopening it with new information if needed.

<!-- gh-comment-id:1040505244 --> @CarstenGrohmann commented on GitHub (Feb 15, 2022): All s3fs instances have been running stable for a few weeks. Therefore, I am closing this topic and reopening it with new information if needed.
kerem commented 2026-03-04 01:50:03 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Jun 12, 2022):

Sorry for the late reply -- you might try ThreadSanitizer deadlock detection next time:

https://github.com/google/sanitizers/wiki/ThreadSanitizerDeadlockDetector

Generally s3fs could use some more stress testing scripts if you want to contribute something!

<!-- gh-comment-id:1153169523 --> @gaul commented on GitHub (Jun 12, 2022): Sorry for the late reply -- you might try ThreadSanitizer deadlock detection next time: https://github.com/google/sanitizers/wiki/ThreadSanitizerDeadlockDetector Generally s3fs could use some more stress testing scripts if you want to contribute something!
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
v1.97
v1.96
v1.95
v1.94
v1.93
v1.92
v1.91
v1.90
v1.89
v1.88
v1.87
v1.86
v1.85
v1.84
v1.83
v1.82
v1.81
v1.80
v1.79
v1.78
Pre-v1.78
v1.77
v1.76
v1.75
v1.74
Labels
Clear labels   
bug

   
bug

   
dataloss

   
duplicate

   
enhancement

   
feature request

   
help wanted

   
invalid

   
need info

   
performance

   
pull-request

Mirrored from GitHub Pull Request

  
question

   
question

   
testing
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#933
No description provided.