mirror of
https://github.com/s3fs-fuse/s3fs-fuse.git
synced 2026-04-26 05:45:57 +03:00
[GH-ISSUE #1496] CPU utilization reaching 100% - frequent production outage #784
Labels
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/s3fs-fuse#784
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @raj-aws on GitHub (Dec 13, 2020).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1496
Hi,
My s3fs version is below, I have two problems, Can someone please help me at the earliest to fix this
Problem 1:-
s3fs process is using 100% CPU usage for every 2 days, once we kill the job, after two days we again see s3fs consuming 100% CPU utilization
Problem 2:-
my use_cache is getting filled up very quickly, My total bucket size is 22 TB, But we are using only one folder actively which has around 100K objects
Both these problems are bringing production outage frequently, requesting please reply at the earliest
Version of s3fs being used (s3fs --version)
Amazon Simple Storage Service File System V1.87 (commit:38e1eaa) with OpenSSL
Copyright (C) 2010 Randy Rizun rrizun@gmail.com
License GPL2: GNU GPL version 2 https://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse)
example: 2.9.4
Kernel information (uname -r)
_command result: uname -r_3.10.0-1160.2.2.el7.x86_64
GNU/Linux Distribution, if applicable (cat /etc/os-release)
command result: cat /etc/os-release
NAME="Red Hat Enterprise Linux Server"
VERSION="7.9 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.9"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.9 (Maipo)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.9:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.9
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.9"
s3fs command line used, if applicable
/etc/fstab entry, if applicable
s3fs#XXXXXvault /XXXXXXX-vaults fuse passwd_file=/root/.passwd-s3fs,use_cache=/root/cache,allow_other,umask=0022,gid=3001,uid=5001 0 0
s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)
if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages
Details about issue
@gaul commented on GitHub (Dec 13, 2020):
You might want to see if you have
locatedbrunning:https://github.com/s3fs-fuse/s3fs-fuse/wiki/FAQ#q-why-does-my-cpu-spike-at-certain-times-of-the-day
Otherwise you can try enabling debug logging via
-f -dor attach a profiler.@raj-aws commented on GitHub (Dec 13, 2020):
Hi Gaul,
Thanks for the response, I added the path in updatedb.
Will watch the system for 2 days and will keep you updated.
Regards
Raj
@raj-aws commented on GitHub (Dec 16, 2020):
Hi Gaul,
The above setting has taken care of CPU utilization, but today we again had a production issue for /root filesystem getting filled with 100% usage.
Can you please help with a solution for it.
Regards
Raj
@gaul commented on GitHub (Dec 16, 2020):
I am glad that this setting helped. Please search open issues because I believe there is a similar one.