[GH-ISSUE #2759] s3fs coredumped with a dirty mount point left behind #1289

Closed
opened 2026-03-04 01:52:50 +03:00 by kerem · 1 comment
Owner

Originally created by @myeyre on GitHub (Nov 27, 2025).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2759

Additional Information

Version of s3fs being used (s3fs --version)

Amazon Simple Storage Service File System V1.90 (commit:unknown) with GnuTLS(gcrypt)

Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse or dpkg -s fuse)

2.9.9-5ubuntu3

Provider (AWS, OVH, Hetzner, iDrive E2, ...)

JuiceFS S3 Gateway

Kernel information (uname -r)

4.19.0-240.23.29.e18_2.bclinux.x86 64

GNU/Linux Distribution, if applicable (cat /etc/os-release)

I'm running s3fs inside a container, the container os is:
PRETTY_NAME="Ubuntu 22.04.4 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.4 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

the host os is bclinux, a rhel compat distro in china:
NAME="BigCloud Enterprise Linux"
VERSION="8.2 (Core)"
ID="bclinux"
ID_LIKE="rhel fedora"
VERSION_ID="8.2"
PLATFORM_ID="platform:an8"
PRETTY_NAME="BigCloud Enterprise Linux 8.2 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:bclinux:bclinux:8"
HOME_URL="https://mirrors.bclinux.org/"
BUG_REPORT_URL="https://bugs.bclinux.org/"

BCLINUX_BUGZILLA_PRODUCT="BigCloud Enterprise Linux 8 (Core)"
BCLINUX_BUGZILLA_PRODUCT_VERSION=8.2
BCLINUX_SUPPORT_PRODUCT="BigCloud Enterprise Linux 8 (Core)"
BCLINUX_SUPPORT_PRODUCT_VERSION=8.2

How to run s3fs, if applicable

[x] command line
[] /etc/fstab

s3fs rawdata:/pipeline/tmpdata /mnt/pipeline/tmpdata  -o passwd_file=passwd_file  -o url=http://xxxx:10086 -o use_path_request_style -o parallel_count=100 -o ro

s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)

n/a

Details about issue

I'm running s3fs inside a container, s3fs coredumped with a dirty mount point left behind. The mountpoint then shows d????????? permissions and we are unable to access it.
gdb provides following output:

root@b01db183305c:~# gdb /usr/bin/s3fs core.s3fs.0.40dacfc503c2491a91595ccb24aac3ac.22890.1763402775000000
GNU gdb (Ubuntu 12.1-0ubuntu1~22.04.2) 12.1
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/s3fs...
Reading symbols from /usr/lib/debug/.build-id/7d/6e190a35e92705215395c80db6b3fffd0932db.debug...
[New LWP 37756]
[New LWP 38791]
[New LWP 38691]
[New LWP 38692]
[New LWP 34766]
[New LWP 13]
[New LWP 39090]
[New LWP 37538]
[New LWP 37028]
[New LWP 38120]
[New LWP 38121]
[New LWP 38690]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `s3fs rawdata /pipeline/tmpdata /mnt/pipeline/tmpdata -o passwd_file=/root/.pass'.
Program terminated with signal SIGABRT, Aborted.
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140103728993856) at ./nptl/pthread_kill.c:44
44	./nptl/pthread_kill.c: No such file or directory.
[Current thread is 1 (Thread 0x7f6c70ff9640 (LWP 37756))]
(gdb) bt
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140103728993856) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=140103728993856) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=140103728993856, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3  0x00007f6cc97d7476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x00007f6cc97bd7f3 in __GI_abort () at ./stdlib/abort.c:79
#5  0x00007f6cc9a60b9e in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00007f6cc9a6c20c in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#7  0x00007f6cc9a6c277 in std::terminate() () from /lib/x86_64-linux-gnu/libstdc++.so.6
#8  0x00007f6cc9a6c4d8 in __cxa_throw () from /lib/x86_64-linux-gnu/libstdc++.so.6
#9  0x00007f6cc9a63449 in std::__throw_length_error(char const*) () from /lib/x86_64-linux-gnu/libstdc++.so.6
#10 0x00007f6cc9b09c69 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long) () from /lib/x86_64-linux-gnu/libstdc++.so.6
#11 0x00007f6cc9b0a0be in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate(unsigned long, unsigned long, char const*, unsigned long) ()
   from /lib/x86_64-linux-gnu/libstdc++.so.6
#12 0x00007f6cc9b0b25e in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_replace(unsigned long, unsigned long, char const*, unsigned long) ()
   from /lib/x86_64-linux-gnu/libstdc++.so.6
#13 0x0000565386f8e7e0 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::assign (__s=0x565386fe79b1 "",
    this=0x565387011ae0 <GetXmlNsUrl(_xmlDoc*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)::strNs>) at /usr/include/c++/11/bits/basic_string.h:1455
#14 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::operator= (__s=<optimized out>, this=<optimized out>, this=<optimized out>, __s=<optimized out>)
    at /usr/include/c++/11/bits/basic_string.h:689
#15 GetXmlNsUrl (doc=doc@entry=0x7f69dc007a10, nsurl="") at ./src/s3fs_xml.cpp:49
#16 0x0000565386f8ea4d in get_base_exp (doc=0x7f69dc007a10, exp=0x565386fe7a7b "Prefix") at ./src/s3fs_xml.cpp:79
#17 0x0000565386f7c67b in get_prefix (doc=0x7f69dc007a10) at ./src/s3fs_xml.cpp:109
#18 append_objects_from_xml (head=..., doc=0x7f69dc007a10, path=0x7f69dc0069a0 "/691ad24e85fd6d55395b75f4") at ./src/s3fs_xml.cpp:414
#19 list_bucket (path=0x7f69dc0069a0 "/691ad24e85fd6d55395b75f4", head=..., delimiter=<optimized out>, check_content_only=true) at ./src/s3fs.cpp:2751
#20 0x0000565386f5fe33 in directory_empty (path=<optimized out>) at ./src/s3fs.cpp:1099
#21 0x0000565386f61054 in get_object_attribute (path=path@entry=0x7f6af800b570 "/691ad24e85fd6d55395b75f4", pstbuf=pstbuf@entry=0x7f6c70ff8600, pmeta=pmeta@entry=0x0, overcheck=overcheck@entry=true,
    pisforce=0x7f6c70ff80df, pisforce@entry=0x0, add_no_truncate_cache=add_no_truncate_cache@entry=false) at /usr/include/c++/11/bits/basic_string.h:194
#22 0x0000565386f61e14 in check_object_access (path=0x7f6af800b570 "/691ad24e85fd6d55395b75f4", mask=mask@entry=1, pstbuf=pstbuf@entry=0x0) at ./src/s3fs.cpp:541
#23 0x0000565386f62399 in check_parent_object_access (path=path@entry=0x7f69dc00b500 "/691ad24e85fd6d55395b75f4/319/data/A.Bite.of.China.S03E01.2018.HD1080P.X264.AAC.Mandarin.CHS.MF.mp4",
    mask=mask@entry=1) at /usr/include/c++/11/bits/basic_string.h:194
#24 0x0000565386f628bc in s3fs_getattr (_path=<optimized out>, stbuf=0x7f6c70ff8930) at ./src/s3fs.cpp:781
#25 0x00007f6cca1a4969 in ?? () from /lib/x86_64-linux-gnu/libfuse.so.2
#26 0x00007f6cca1b0308 in ?? () from /lib/x86_64-linux-gnu/libfuse.so.2
#27 0x00007f6cca1b069c in ?? () from /lib/x86_64-linux-gnu/libfuse.so.2
#28 0x00007f6cca1ad21d in ?? () from /lib/x86_64-linux-gnu/libfuse.so.2
#29 0x00007f6cc9829ac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#30 0x00007f6cc98baa04 in iopl () at ../sysdeps/unix/syscall-template.S:122
#31 0x0000000000000000 in ?? ()
(gdb)
Originally created by @myeyre on GitHub (Nov 27, 2025). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2759 <!-- -------------------------------------------------------------------------- The following information is very important in order to help us to help you. Omission of the following details may delay your support request or receive no attention at all. --------------------------------------------------------------------------- --> ### Additional Information #### Version of s3fs being used (`s3fs --version`) <!-- example: V1.91 (commit:b19262a) --> Amazon Simple Storage Service File System V1.90 (commit:unknown) with GnuTLS(gcrypt) #### Version of fuse being used (`pkg-config --modversion fuse`, `rpm -qi fuse` or `dpkg -s fuse`) <!-- example: 2.9.2 --> 2.9.9-5ubuntu3 #### Provider (`AWS`, `OVH`, `Hetzner`, `iDrive E2`, ...) <!-- example: AWS --> JuiceFS S3 Gateway #### Kernel information (`uname -r`) <!-- example: 5.10.96-90.460.amzn2.x86_64 --> 4.19.0-240.23.29.e18_2.bclinux.x86 64 #### GNU/Linux Distribution, if applicable (`cat /etc/os-release`) <!-- command result --> I'm running s3fs inside a container, the container os is: PRETTY_NAME="Ubuntu 22.04.4 LTS" NAME="Ubuntu" VERSION_ID="22.04" VERSION="22.04.4 LTS (Jammy Jellyfish)" VERSION_CODENAME=jammy ID=ubuntu ID_LIKE=debian HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" UBUNTU_CODENAME=jammy the host os is bclinux, a rhel compat distro in china: NAME="BigCloud Enterprise Linux" VERSION="8.2 (Core)" ID="bclinux" ID_LIKE="rhel fedora" VERSION_ID="8.2" PLATFORM_ID="platform:an8" PRETTY_NAME="BigCloud Enterprise Linux 8.2 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:bclinux:bclinux:8" HOME_URL="https://mirrors.bclinux.org/" BUG_REPORT_URL="https://bugs.bclinux.org/" BCLINUX_BUGZILLA_PRODUCT="BigCloud Enterprise Linux 8 (Core)" BCLINUX_BUGZILLA_PRODUCT_VERSION=8.2 BCLINUX_SUPPORT_PRODUCT="BigCloud Enterprise Linux 8 (Core)" BCLINUX_SUPPORT_PRODUCT_VERSION=8.2 #### How to run s3fs, if applicable <!-- Describe the s3fs "command line" or "/etc/fstab" entry used. --> [x] command line [] /etc/fstab <!-- Executed command line or /etc/fastab entry --> ``` s3fs rawdata:/pipeline/tmpdata /mnt/pipeline/tmpdata -o passwd_file=passwd_file -o url=http://xxxx:10086 -o use_path_request_style -o parallel_count=100 -o ro ``` #### s3fs syslog messages (`grep s3fs /var/log/syslog`, `journalctl | grep s3fs`, or `s3fs outputs`) <!-- if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages. --> ``` n/a ``` ### Details about issue <!-- Please describe the content of the issue in detail. --> I'm running s3fs inside a container, s3fs coredumped with a dirty mount point left behind. The mountpoint then shows d????????? permissions and we are unable to access it. gdb provides following output: ``` root@b01db183305c:~# gdb /usr/bin/s3fs core.s3fs.0.40dacfc503c2491a91595ccb24aac3ac.22890.1763402775000000 GNU gdb (Ubuntu 12.1-0ubuntu1~22.04.2) 12.1 Copyright (C) 2022 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <https://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/bin/s3fs... Reading symbols from /usr/lib/debug/.build-id/7d/6e190a35e92705215395c80db6b3fffd0932db.debug... [New LWP 37756] [New LWP 38791] [New LWP 38691] [New LWP 38692] [New LWP 34766] [New LWP 13] [New LWP 39090] [New LWP 37538] [New LWP 37028] [New LWP 38120] [New LWP 38121] [New LWP 38690] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `s3fs rawdata /pipeline/tmpdata /mnt/pipeline/tmpdata -o passwd_file=/root/.pass'. Program terminated with signal SIGABRT, Aborted. #0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=140103728993856) at ./nptl/pthread_kill.c:44 44 ./nptl/pthread_kill.c: No such file or directory. [Current thread is 1 (Thread 0x7f6c70ff9640 (LWP 37756))] (gdb) bt #0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=140103728993856) at ./nptl/pthread_kill.c:44 #1 __pthread_kill_internal (signo=6, threadid=140103728993856) at ./nptl/pthread_kill.c:78 #2 __GI___pthread_kill (threadid=140103728993856, signo=signo@entry=6) at ./nptl/pthread_kill.c:89 #3 0x00007f6cc97d7476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 #4 0x00007f6cc97bd7f3 in __GI_abort () at ./stdlib/abort.c:79 #5 0x00007f6cc9a60b9e in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6 #6 0x00007f6cc9a6c20c in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6 #7 0x00007f6cc9a6c277 in std::terminate() () from /lib/x86_64-linux-gnu/libstdc++.so.6 #8 0x00007f6cc9a6c4d8 in __cxa_throw () from /lib/x86_64-linux-gnu/libstdc++.so.6 #9 0x00007f6cc9a63449 in std::__throw_length_error(char const*) () from /lib/x86_64-linux-gnu/libstdc++.so.6 #10 0x00007f6cc9b09c69 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long) () from /lib/x86_64-linux-gnu/libstdc++.so.6 #11 0x00007f6cc9b0a0be in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate(unsigned long, unsigned long, char const*, unsigned long) () from /lib/x86_64-linux-gnu/libstdc++.so.6 #12 0x00007f6cc9b0b25e in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_replace(unsigned long, unsigned long, char const*, unsigned long) () from /lib/x86_64-linux-gnu/libstdc++.so.6 #13 0x0000565386f8e7e0 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::assign (__s=0x565386fe79b1 "", this=0x565387011ae0 <GetXmlNsUrl(_xmlDoc*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)::strNs>) at /usr/include/c++/11/bits/basic_string.h:1455 #14 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::operator= (__s=<optimized out>, this=<optimized out>, this=<optimized out>, __s=<optimized out>) at /usr/include/c++/11/bits/basic_string.h:689 #15 GetXmlNsUrl (doc=doc@entry=0x7f69dc007a10, nsurl="") at ./src/s3fs_xml.cpp:49 #16 0x0000565386f8ea4d in get_base_exp (doc=0x7f69dc007a10, exp=0x565386fe7a7b "Prefix") at ./src/s3fs_xml.cpp:79 #17 0x0000565386f7c67b in get_prefix (doc=0x7f69dc007a10) at ./src/s3fs_xml.cpp:109 #18 append_objects_from_xml (head=..., doc=0x7f69dc007a10, path=0x7f69dc0069a0 "/691ad24e85fd6d55395b75f4") at ./src/s3fs_xml.cpp:414 #19 list_bucket (path=0x7f69dc0069a0 "/691ad24e85fd6d55395b75f4", head=..., delimiter=<optimized out>, check_content_only=true) at ./src/s3fs.cpp:2751 #20 0x0000565386f5fe33 in directory_empty (path=<optimized out>) at ./src/s3fs.cpp:1099 #21 0x0000565386f61054 in get_object_attribute (path=path@entry=0x7f6af800b570 "/691ad24e85fd6d55395b75f4", pstbuf=pstbuf@entry=0x7f6c70ff8600, pmeta=pmeta@entry=0x0, overcheck=overcheck@entry=true, pisforce=0x7f6c70ff80df, pisforce@entry=0x0, add_no_truncate_cache=add_no_truncate_cache@entry=false) at /usr/include/c++/11/bits/basic_string.h:194 #22 0x0000565386f61e14 in check_object_access (path=0x7f6af800b570 "/691ad24e85fd6d55395b75f4", mask=mask@entry=1, pstbuf=pstbuf@entry=0x0) at ./src/s3fs.cpp:541 #23 0x0000565386f62399 in check_parent_object_access (path=path@entry=0x7f69dc00b500 "/691ad24e85fd6d55395b75f4/319/data/A.Bite.of.China.S03E01.2018.HD1080P.X264.AAC.Mandarin.CHS.MF.mp4", mask=mask@entry=1) at /usr/include/c++/11/bits/basic_string.h:194 #24 0x0000565386f628bc in s3fs_getattr (_path=<optimized out>, stbuf=0x7f6c70ff8930) at ./src/s3fs.cpp:781 #25 0x00007f6cca1a4969 in ?? () from /lib/x86_64-linux-gnu/libfuse.so.2 #26 0x00007f6cca1b0308 in ?? () from /lib/x86_64-linux-gnu/libfuse.so.2 #27 0x00007f6cca1b069c in ?? () from /lib/x86_64-linux-gnu/libfuse.so.2 #28 0x00007f6cca1ad21d in ?? () from /lib/x86_64-linux-gnu/libfuse.so.2 #29 0x00007f6cc9829ac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442 #30 0x00007f6cc98baa04 in iopl () at ../sysdeps/unix/syscall-template.S:122 #31 0x0000000000000000 in ?? () (gdb) ```
kerem closed this issue 2026-03-04 01:52:50 +03:00
Author
Owner

@gaul commented on GitHub (Nov 28, 2025):

s3fs 1.90 is five years old -- please retest with the latest version 1.96.

<!-- gh-comment-id:3587976537 --> @gaul commented on GitHub (Nov 28, 2025): s3fs 1.90 is five years old -- please retest with the latest version 1.96.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#1289
No description provided.