starred/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2026-04-25 05:16:00 +03:00
Code Issues 305 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #2203] StreamUpload is not working #1118

New issue
Closed
opened 2026-03-04 01:51:32 +03:00 by kerem · 5 comments
kerem commented 2026-03-04 01:51:32 +03:00
Owner
Copy link

Originally created by @VVoidV on GitHub (Jul 1, 2023).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2203

Additional Information

Version of s3fs being used (s3fs --version)

Amazon Simple Storage Service File System V1.92 (commit:unknown) with OpenSSL

Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse or dpkg -s fuse)

2.9.9

Kernel information (uname -r)

6.3.7-200.fc38.x86_64

GNU/Linux Distribution, if applicable (cat /etc/os-release)

NAME="Fedora Linux"
VERSION="38 (Workstation Edition)"
ID=fedora
VERSION_ID=38
VERSION_CODENAME=""
PLATFORM_ID="platform:f38"
PRETTY_NAME="Fedora Linux 38 (Workstation Edition)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:38"
DEFAULT_HOSTNAME="fedora"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f38/system-administrators-guide/"
SUPPORT_URL="https://ask.fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=38
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=38
SUPPORT_END=2024-05-14
VARIANT="Workstation Edition"
VARIANT_ID=workstation

How to run s3fs, if applicable

[x] command line
[] /etc/fstab

s3fs -o streamupload,dbglevel=info,use_cache=/var/s3fs_cache,logfile=/tmp/s3fs.log,use_path_request_style,allow_other,del_cache,connect_timeout=300,readwrite_timeout=120,url=http://192.168.204.132:4566 bucket /exports/test/

s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)

Details about issue

streamupload is not working.
use dd to write a 100MB file, dd while hung forever.

[root@void test]# dd if=/dev/urandom of=/exports/test/100 bs=1M count=100
dd: closing output file '/exports/test/100': Software caused connection abort

fuse thread is waiting for streamupload, but theadpool is gone ( Threre is only two of fuse thread and the main thread)

(gdb) bt
#0  0x00007f2122cab1d9 in __futex_abstimed_wait_common64 (private=<optimized out>, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x7f21140029e0) at futex-internal.c:57
#1  __futex_abstimed_wait_common (futex_word=futex_word@entry=0x7f21140029e0, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=<optimized out>, cancel=cancel@entry=true) at futex-internal.c:87
#2  0x00007f2122cab25f in __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x7f21140029e0, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=<optimized out>) at futex-internal.c:139
#3  0x00007f2122cb69bf in do_futex_wait (sem=sem@entry=0x7f21140029e0, abstime=0x0, clockid=0) at /usr/src/debug/glibc-2.37-4.fc38.x86_64/nptl/sem_waitcommon.c:111
#4  0x00007f2122cb6a58 in __new_sem_wait_slow64 (sem=0x7f21140029e0, abstime=0x0, clockid=0) at /usr/src/debug/glibc-2.37-4.fc38.x86_64/nptl/sem_waitcommon.c:183
#5  0x000055625f250012 in PseudoFdInfo::WaitAllThreadsExit() ()
#6  0x000055625f24e435 in FdEntity::RowFlushStreamMultipart(PseudoFdInfo*, char const*) ()
#7  0x000055625f24beb0 in FdEntity::RowFlush(int, char const*, AutoLock::Type, bool) ()
#8  0x000055625f2132ca in s3fs_flush(char const*, fuse_file_info*) [clone .lto_priv.0] ()
#9  0x00007f212385c1d2 in fuse_flush_common () from /lib64/libfuse.so.2
#10 0x00007f212385f728 in fuse_lib_flush () from /lib64/libfuse.so.2
#11 0x00007f2123860794 in do_flush.lto_priv () from /lib64/libfuse.so.2
#12 0x00007f212386cc89 in fuse_ll_process_buf () from /lib64/libfuse.so.2
#13 0x00007f212385ce55 in fuse_do_work () from /lib64/libfuse.so.2
#14 0x00007f2122cae907 in start_thread (arg=<optimized out>) at pthread_create.c:444
#15 0x00007f2122d34870 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
(gdb) info thre
  Id   Target Id                                 Frame 
  1    Thread 0x7f2122589880 (LWP 660438) "s3fs" 0x00007f2122cab1d9 in __futex_abstimed_wait_common64 (private=<optimized out>, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x7ffed7d960f0) at futex-internal.c:57
* 2    Thread 0x7f211bfff6c0 (LWP 660439) "s3fs" 0x00007f2122cab1d9 in __futex_abstimed_wait_common64 (private=<optimized out>, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x7f21140029e0) at futex-internal.c:57
  3    Thread 0x7f2120d836c0 (LWP 660440) "s3fs" 0x00007f2122d230fa in __GI___libc_read (nbytes=135168, buf=0x7f212053f010, fd=5) at ../sysdeps/unix/sysv/linux/read.c:26
Originally created by @VVoidV on GitHub (Jul 1, 2023). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2203 <!-- -------------------------------------------------------------------------- The following information is very important in order to help us to help you. Omission of the following details may delay your support request or receive no attention at all. Keep in mind that the commands we provide to retrieve information are oriented to GNU/Linux Distributions, so you could need to use others if you use s3fs on macOS or BSD. --------------------------------------------------------------------------- --> ### Additional Information #### Version of s3fs being used (`s3fs --version`) Amazon Simple Storage Service File System V1.92 (commit:unknown) with OpenSSL #### Version of fuse being used (`pkg-config --modversion fuse`, `rpm -qi fuse` or `dpkg -s fuse`) 2.9.9 #### Kernel information (`uname -r`) 6.3.7-200.fc38.x86_64 #### GNU/Linux Distribution, if applicable (`cat /etc/os-release`) NAME="Fedora Linux" VERSION="38 (Workstation Edition)" ID=fedora VERSION_ID=38 VERSION_CODENAME="" PLATFORM_ID="platform:f38" PRETTY_NAME="Fedora Linux 38 (Workstation Edition)" ANSI_COLOR="0;38;2;60;110;180" LOGO=fedora-logo-icon CPE_NAME="cpe:/o:fedoraproject:fedora:38" DEFAULT_HOSTNAME="fedora" HOME_URL="https://fedoraproject.org/" DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f38/system-administrators-guide/" SUPPORT_URL="https://ask.fedoraproject.org/" BUG_REPORT_URL="https://bugzilla.redhat.com/" REDHAT_BUGZILLA_PRODUCT="Fedora" REDHAT_BUGZILLA_PRODUCT_VERSION=38 REDHAT_SUPPORT_PRODUCT="Fedora" REDHAT_SUPPORT_PRODUCT_VERSION=38 SUPPORT_END=2024-05-14 VARIANT="Workstation Edition" VARIANT_ID=workstation #### How to run s3fs, if applicable <!-- Describe the s3fs "command line" or "/etc/fstab" entry used. --> [x] command line [] /etc/fstab <!-- Executed command line or /etc/fastab entry --> ``` s3fs -o streamupload,dbglevel=info,use_cache=/var/s3fs_cache,logfile=/tmp/s3fs.log,use_path_request_style,allow_other,del_cache,connect_timeout=300,readwrite_timeout=120,url=http://192.168.204.132:4566 bucket /exports/test/ ``` #### s3fs syslog messages (`grep s3fs /var/log/syslog`, `journalctl | grep s3fs`, or `s3fs outputs`) <!-- if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages. --> ``` ``` ### Details about issue <!-- Please describe the content of the issue in detail. --> streamupload is not working. use **dd** to write a 100MB file, dd while hung forever. ``` [root@void test]# dd if=/dev/urandom of=/exports/test/100 bs=1M count=100 dd: closing output file '/exports/test/100': Software caused connection abort ``` fuse thread is waiting for streamupload, but theadpool is gone ( Threre is only two of fuse thread and the main thread) ``` (gdb) bt #0 0x00007f2122cab1d9 in __futex_abstimed_wait_common64 (private=<optimized out>, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x7f21140029e0) at futex-internal.c:57 #1 __futex_abstimed_wait_common (futex_word=futex_word@entry=0x7f21140029e0, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=<optimized out>, cancel=cancel@entry=true) at futex-internal.c:87 #2 0x00007f2122cab25f in __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x7f21140029e0, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=<optimized out>) at futex-internal.c:139 #3 0x00007f2122cb69bf in do_futex_wait (sem=sem@entry=0x7f21140029e0, abstime=0x0, clockid=0) at /usr/src/debug/glibc-2.37-4.fc38.x86_64/nptl/sem_waitcommon.c:111 #4 0x00007f2122cb6a58 in __new_sem_wait_slow64 (sem=0x7f21140029e0, abstime=0x0, clockid=0) at /usr/src/debug/glibc-2.37-4.fc38.x86_64/nptl/sem_waitcommon.c:183 #5 0x000055625f250012 in PseudoFdInfo::WaitAllThreadsExit() () #6 0x000055625f24e435 in FdEntity::RowFlushStreamMultipart(PseudoFdInfo*, char const*) () #7 0x000055625f24beb0 in FdEntity::RowFlush(int, char const*, AutoLock::Type, bool) () #8 0x000055625f2132ca in s3fs_flush(char const*, fuse_file_info*) [clone .lto_priv.0] () #9 0x00007f212385c1d2 in fuse_flush_common () from /lib64/libfuse.so.2 #10 0x00007f212385f728 in fuse_lib_flush () from /lib64/libfuse.so.2 #11 0x00007f2123860794 in do_flush.lto_priv () from /lib64/libfuse.so.2 #12 0x00007f212386cc89 in fuse_ll_process_buf () from /lib64/libfuse.so.2 #13 0x00007f212385ce55 in fuse_do_work () from /lib64/libfuse.so.2 #14 0x00007f2122cae907 in start_thread (arg=<optimized out>) at pthread_create.c:444 #15 0x00007f2122d34870 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81 (gdb) info thre Id Target Id Frame 1 Thread 0x7f2122589880 (LWP 660438) "s3fs" 0x00007f2122cab1d9 in __futex_abstimed_wait_common64 (private=<optimized out>, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x7ffed7d960f0) at futex-internal.c:57 * 2 Thread 0x7f211bfff6c0 (LWP 660439) "s3fs" 0x00007f2122cab1d9 in __futex_abstimed_wait_common64 (private=<optimized out>, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x7f21140029e0) at futex-internal.c:57 3 Thread 0x7f2120d836c0 (LWP 660440) "s3fs" 0x00007f2122d230fa in __GI___libc_read (nbytes=135168, buf=0x7f212053f010, fd=5) at ../sysdeps/unix/sysv/linux/read.c:26 ```
kerem closed this issue 2026-03-04 01:51:32 +03:00
kerem commented 2026-03-04 01:51:33 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Jul 2, 2023):

@VVoidV Thanks for your kindness.
And I merged your #2204 PR code.

<!-- gh-comment-id:1616269323 --> @ggtakec commented on GitHub (Jul 2, 2023): @VVoidV Thanks for your kindness. And I merged your #2204 PR code.
kerem commented 2026-03-04 01:51:33 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Jul 2, 2023):

@gaul
It seems that this cause can be solved by PR #2204.
I'm requesting to the person who reported it in the (probably) related issue to see if they can test new master code.
If this cause also affects other issues, this issue becomes a critical one.
After their confirmation, I would like to propose to put out a release version urgently, so please give me your opinion.

<!-- gh-comment-id:1616286732 --> @ggtakec commented on GitHub (Jul 2, 2023): @gaul It seems that this cause can be solved by PR #2204. I'm requesting to the person who reported it in the (probably) related issue to see if they can test new master code. If this cause also affects other issues, this issue becomes a critical one. After their confirmation, I would like to propose to put out a release version urgently, so please give me your opinion.
kerem commented 2026-03-04 01:51:33 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Jul 2, 2023):

@ggtakec Let's figure out the folder issue which I can reproduce locally then we can run a new release.

<!-- gh-comment-id:1616592177 --> @gaul commented on GitHub (Jul 2, 2023): @ggtakec Let's figure out the folder issue which I can reproduce locally then we can run a new release.
kerem commented 2026-03-04 01:51:33 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Jul 2, 2023):

@gaul I'm sorry, I'm still a little confused.
(I'm wondering if we can release it without solving the following issues)
This critical issue has been resolved, but there is still the issue of not seeing folders.
I misunderstood that it was related to this matter, but it seems to be another cause.

<!-- gh-comment-id:1616599379 --> @ggtakec commented on GitHub (Jul 2, 2023): @gaul I'm sorry, I'm still a little confused. _(I'm wondering if we can release it without solving the following issues)_ This critical issue has been resolved, but there is still the issue of not seeing folders. I misunderstood that it was related to this matter, but it seems to be another cause.
kerem commented 2026-03-04 01:51:33 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Jul 17, 2023):

@gaul #2212 solved this, so I think this issue can be closed.

<!-- gh-comment-id:1637283144 --> @ggtakec commented on GitHub (Jul 17, 2023): @gaul #2212 solved this, so I think this issue can be closed.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
v1.97
v1.96
v1.95
v1.94
v1.93
v1.92
v1.91
v1.90
v1.89
v1.88
v1.87
v1.86
v1.85
v1.84
v1.83
v1.82
v1.81
v1.80
v1.79
v1.78
Pre-v1.78
v1.77
v1.76
v1.75
v1.74
Labels
Clear labels   
bug

   
bug

   
dataloss

   
duplicate

   
enhancement

   
feature request

   
help wanted

   
invalid

   
need info

   
performance

   
pull-request

Mirrored from GitHub Pull Request

  
question

   
question

   
testing
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#1118
No description provided.