starred/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2026-04-25 21:35:58 +03:00
Code Issues 305 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #2106] v1.91 slow and delay in generating xml files #1071

New issue
Open
opened 2026-03-04 01:51:08 +03:00 by kerem · 4 comments
kerem commented 2026-03-04 01:51:08 +03:00
Owner
Copy link

Originally created by @vcrego on GitHub (Feb 1, 2023).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2106

Additional Information

Version of s3fs being used (s3fs --version)

V1.91

Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse or dpkg -s fuse)

2.9.2

Kernel information (uname -r)

4.14.301-224.520.amzn2.x86_64

GNU/Linux Distribution, if applicable (cat /etc/os-release)

NAME="Amazon Linux"
VERSION="2"

How to run s3fs, if applicable

[] /etc/fstab

s3fs#s3-bucket-name /mnt/s3-general fuse allow_other,umask=0007,uid=1001,gid=1001,stat_cache_expire=10,use_cache=/tmp,iam_role=auto,nonempty 0 0

s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)

2023-02-01T15:39:45.479Z [INF] s3fs.cpp:s3fs_open(2255): [path=/sftp/Development/Interfaces/Diamond/diamond_interface_output/CPPM_01_accounts.xml][flags=0x8000]
2023-02-01T15:39:45.479Z [INF] fdcache.cpp:GetFdEntity(485): [path=/sftp/Development/Interfaces/Diamond/diamond_interface_output/CPPM_01_accounts.xml][pseudo_fd=-1]
2023-02-01T15:39:45.480Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/sftp/Development/Interfaces/Diamond/diamond_interface_output/CPPM_01_accounts.xml]
2023-02-01T15:39:45.480Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/sftp/Development/Interfaces/Diamond/diamond_interface_output/CPPM_01_accounts.xml]
2023-02-01T15:39:45.481Z [INF] s3fs.cpp:s3fs_flush(2409): [path=/sftp/Development/Interfaces/Diamond/diamond_interface_output/CPPM_01_accounts.xml][pseudo_fd=3]
2023-02-01T15:39:45.481Z [INF] fdcache_entity.cpp:RowFlush(1347): [tpath=][path=/sftp/Development/Interfaces/Diamond/diamond_interface_output/CPPM_01_accounts.xml][pseudo_fd=3][physical_fd=41]
2023-02-01T15:39:45.481Z [INF] cache.cpp:DelStat(591): delete stat cache entry[path=/sftp/Development/Interfaces/Diamond/diamond_interface_output/CPPM_01_accounts.xml]
2023-02-01T15:39:45.481Z [INF] s3fs.cpp:s3fs_release(2467): [path=/sftp/Development/Interfaces/Diamond/diamond_interface_output/CPPM_01_accounts.xml][pseudo_fd=3]
2023-02-01T15:39:45.481Z [INF] fdcache.cpp:GetFdEntity(485): [path=/sftp/Development/Interfaces/Diamond/diamond_interface_output/CPPM_01_accounts.xml][pseudo_fd=3]

Details about issue

We have upgraded from version 1.86 to v1.91 on amazon linux. With this upgrade using same s3fs options as given in the fstab entry above. Issues are:

  • The log above shows file "CPPM_01_accounts.xml" if tries to generate and we see file on s3 delayed, that is after an hours or more, some time no file. I can see file under /tmp but that is not complete.
  • Many times error: Transport endpoint is not connected
  • Responds slow if a directory on s3 got more than 1000 files, compare of previous s3fs version. Had to clear the file to make it work.

Please help how to improve with this new version, trying to get same performance like v 1.86. If there any options to be changed please advise. Thanks.

Originally created by @vcrego on GitHub (Feb 1, 2023). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2106 <!-- -------------------------------------------------------------------------- The following information is very important in order to help us to help you. Omission of the following details may delay your support request or receive no attention at all. Keep in mind that the commands we provide to retrieve information are oriented to GNU/Linux Distributions, so you could need to use others if you use s3fs on macOS or BSD. --------------------------------------------------------------------------- --> ### Additional Information #### Version of s3fs being used (`s3fs --version`) V1.91 #### Version of fuse being used (`pkg-config --modversion fuse`, `rpm -qi fuse` or `dpkg -s fuse`) 2.9.2 #### Kernel information (`uname -r`) 4.14.301-224.520.amzn2.x86_64 #### GNU/Linux Distribution, if applicable (`cat /etc/os-release`) NAME="Amazon Linux" VERSION="2" #### How to run s3fs, if applicable [] /etc/fstab <!-- Executed command line or /etc/fastab entry --> s3fs#s3-bucket-name /mnt/s3-general fuse allow_other,umask=0007,uid=1001,gid=1001,stat_cache_expire=10,use_cache=/tmp,iam_role=auto,nonempty 0 0 #### s3fs syslog messages (`grep s3fs /var/log/syslog`, `journalctl | grep s3fs`, or `s3fs outputs`) 2023-02-01T15:39:45.479Z [INF] s3fs.cpp:s3fs_open(2255): [path=/sftp/Development/Interfaces/Diamond/diamond_interface_output/CPPM_01_accounts.xml][flags=0x8000] 2023-02-01T15:39:45.479Z [INF] fdcache.cpp:GetFdEntity(485): [path=/sftp/Development/Interfaces/Diamond/diamond_interface_output/CPPM_01_accounts.xml][pseudo_fd=-1] 2023-02-01T15:39:45.480Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/sftp/Development/Interfaces/Diamond/diamond_interface_output/CPPM_01_accounts.xml] 2023-02-01T15:39:45.480Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/sftp/Development/Interfaces/Diamond/diamond_interface_output/CPPM_01_accounts.xml] 2023-02-01T15:39:45.481Z [INF] s3fs.cpp:s3fs_flush(2409): [path=/sftp/Development/Interfaces/Diamond/diamond_interface_output/CPPM_01_accounts.xml][pseudo_fd=3] 2023-02-01T15:39:45.481Z [INF] fdcache_entity.cpp:RowFlush(1347): [tpath=][path=/sftp/Development/Interfaces/Diamond/diamond_interface_output/CPPM_01_accounts.xml][pseudo_fd=3][physical_fd=41] 2023-02-01T15:39:45.481Z [INF] cache.cpp:DelStat(591): delete stat cache entry[path=/sftp/Development/Interfaces/Diamond/diamond_interface_output/CPPM_01_accounts.xml] 2023-02-01T15:39:45.481Z [INF] s3fs.cpp:s3fs_release(2467): [path=/sftp/Development/Interfaces/Diamond/diamond_interface_output/CPPM_01_accounts.xml][pseudo_fd=3] 2023-02-01T15:39:45.481Z [INF] fdcache.cpp:GetFdEntity(485): [path=/sftp/Development/Interfaces/Diamond/diamond_interface_output/CPPM_01_accounts.xml][pseudo_fd=3] ### Details about issue We have upgraded from version 1.86 to v1.91 on amazon linux. With this upgrade using same s3fs options as given in the fstab entry above. Issues are: - The log above shows file "CPPM_01_accounts.xml" if tries to generate and we see file on s3 delayed, that is after an hours or more, some time no file. I can see file under /tmp but that is not complete. - Many times error: Transport endpoint is not connected - Responds slow if a directory on s3 got more than 1000 files, compare of previous s3fs version. Had to clear the file to make it work. Please help how to improve with this new version, trying to get same performance like v 1.86. If there any options to be changed please advise. Thanks.
kerem commented 2026-03-04 01:51:09 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Feb 8, 2023):

@vcrego
In the log you gave me, I didn't find any lines indicating file delays.
If possible, add the dbglevel=info (and curldbg) option, and if there is a log obtained, it may be possible to analyze it.
If you are using the ls command etc. in a directory with a large number of files, increasing stat_cache_expire like the default value(900) may speed it up.

In addition, although it is unknown whether it is related to this report, the default of the object(file/directory) compatibility option has been changed since V1.91.
If your 1000 files were created from a client other than s3fs, you may need to give the compat_dir and complement_stat options.

<!-- gh-comment-id:1422711646 --> @ggtakec commented on GitHub (Feb 8, 2023): @vcrego In the log you gave me, I didn't find any lines indicating file delays. If possible, add the `dbglevel=info` (and `curldbg`) option, and if there is a log obtained, it may be possible to analyze it. If you are using the `ls` command etc. in a directory with a large number of files, increasing `stat_cache_expire` like the default value(900) may speed it up. In addition, although it is unknown whether it is related to this report, the default of the object(file/directory) compatibility option has been changed since V1.91. If your 1000 files were created from a client other than s3fs, you may need to give the `compat_dir` and `complement_stat` options.
kerem commented 2026-03-04 01:51:09 +03:00
Author
Owner
Copy link

@vcrego commented on GitHub (Feb 28, 2023):

@ggtakec Thanks for the response. We actually had to revert to earlier version which is stable to handle large set of data. The options I tried with v1.91 are below, however those definitely were not helping and were casing issue:

  • for clients having more than 999 files in a directory.
  • other issues we have see increasing of mount failures at server launch.
  • last slowness while access files, the example here the application on ec2 have java program generates files and try to access or write into file on s3, in most of the cases it use to fail with file not found error.

Please help with recommended options with this new v1.91 to get smooth performance just like v1.86, so that I can try and get back with mode details. Thanks.

fstab mount options tried:
allow_other,umask=0007,uid=1001,gid=1001,stat_cache_expire=10,use_cache=/tmp,iam_role=auto,nonempty 0 0

<!-- gh-comment-id:1447802891 --> @vcrego commented on GitHub (Feb 28, 2023): @ggtakec Thanks for the response. We actually had to revert to earlier version which is stable to handle large set of data. The options I tried with v1.91 are below, however those definitely were not helping and were casing issue: - for clients having more than 999 files in a directory. - other issues we have see increasing of mount failures at server launch. - last slowness while access files, the example here the application on ec2 have java program generates files and try to access or write into file on s3, in most of the cases it use to fail with file not found error. Please help with recommended options with this new v1.91 to get smooth performance just like v1.86, so that I can try and get back with mode details. Thanks. fstab mount options tried: _allow_other,umask=0007,uid=1001,gid=1001,stat_cache_expire=10,use_cache=/tmp,iam_role=auto,nonempty 0 0_
kerem commented 2026-03-04 01:51:09 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Mar 9, 2023):

@vcrego
Is your directory(on S3) which has many files created by s3fs?
If created with other S3 client tools, the compat_dir and complement_stat options may be required.

Also, could you try the code on the master branch now?
Some bugs have been fixed and we would like to know the result of the master branch as well.

<!-- gh-comment-id:1462055523 --> @ggtakec commented on GitHub (Mar 9, 2023): @vcrego Is your directory(on S3) which has many files created by s3fs? If created with other S3 client tools, the `compat_dir` and `complement_stat` options may be required. Also, could you try the code on the master branch now? Some bugs have been fixed and we would like to know the result of the master branch as well.
kerem commented 2026-03-04 01:51:09 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Mar 9, 2023):

You can also try git bisect which might help us understand which commit caused this regression.

<!-- gh-comment-id:1462218704 --> @gaul commented on GitHub (Mar 9, 2023): You can also try `git bisect` which might help us understand which commit caused this regression.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
v1.97
v1.96
v1.95
v1.94
v1.93
v1.92
v1.91
v1.90
v1.89
v1.88
v1.87
v1.86
v1.85
v1.84
v1.83
v1.82
v1.81
v1.80
v1.79
v1.78
Pre-v1.78
v1.77
v1.76
v1.75
v1.74
Labels
Clear labels   
bug

   
bug

   
dataloss

   
duplicate

   
enhancement

   
feature request

   
help wanted

   
invalid

   
need info

   
performance

   
pull-request

Mirrored from GitHub Pull Request

  
question

   
question

   
testing
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#1071
No description provided.