mirror of
https://github.com/s3fs-fuse/s3fs-fuse.git
synced 2026-04-25 13:26:00 +03:00
[GH-ISSUE #1607] S3fs calls S3 aggressively 1k calls / minute #846
Labels
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/s3fs-fuse#846
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @JeremieDoctrine on GitHub (Mar 18, 2021).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1607
Version of s3fs being used (s3fs --version)
1.89
Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse)
2.9.2
Kernel information (uname -r)
4.14.88-88.76.amzn2.x86_64
GNU/Linux Distribution, if applicable (cat /etc/os-release)
/etc/fstab entry, if applicable
s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)
if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages
Details about issue
Hi,
My bucket has something like 6 files in 2 directories, but every 5 seconds s3fs makes many calls to s3 which are super expensive. About 1k call per minute. I've edited the
updatedb.confwith the following:But it does not reduce the number of calls.
@JeremieDoctrine commented on GitHub (Mar 18, 2021):
Looks like there is a process in Elasticsearch scanning
/etc/elasticsearch/sharedevery five seconds. If I stop Elasticsearch I don't have any call anymore.Is there any setting to tell s3fs to stop calling s3 when doing a scan on a directory?
@gaul commented on GitHub (Mar 21, 2021):
I am not sure what you expect here -- a filesystem serves data when a process requests it. It is up to the application not to request that data. You can either configure Elasticsearch correctly or use some kind of unix permissions to prohibit access.
@gaul commented on GitHub (Apr 25, 2021):
Closing since there is nothing actionable on the s3fs side.