mirror of
https://github.com/s3fs-fuse/s3fs-fuse.git
synced 2026-04-25 13:26:00 +03:00
[GH-ISSUE #1342] Memory use #720
Labels
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/s3fs-fuse#720
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @idlemind on GitHub (Jul 26, 2020).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1342
Version of s3fs being used (s3fs --version)
Amazon Simple Storage Service File System V1.86 (commit:e0a38ad) with OpenSSL
Kernel information (uname -r)
4.15.0-111-generic
GNU/Linux Distribution, if applicable (cat /etc/os-release)
NAME="Ubuntu"
VERSION="18.04.4 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.4 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic
s3fs command line used, if applicable
s3fs -d -o endpoint="eu-west-2" -o allow_other -o umask=0002 -o passwd_file=/etc/passwd-s3fs-live /
s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)
if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages
Details about issue
Service appears to be using lots of server resource to the point other resources are being blocked. Can we find why so much memory is being consumed?
@idlemind commented on GitHub (Jul 26, 2020):
Memory use:
65.6758 MB php-fpm:
71.5039 MB s3fs
115.211 MB /usr/sbin/apache2
115.637 MB /usr/sbin/apache2
116.605 MB /usr/sbin/apache2
130.098 MB /usr/sbin/apache2
131.445 MB /usr/sbin/apache2
131.68 MB /usr/sbin/apache2
139.613 MB /usr/sbin/mysqld
2528.91 MB s3fs
@idlemind commented on GitHub (Jul 28, 2020):
I'm also seeing a flood of these messages in the syslog during high ram usage:
399999 Jul 28 08:57:33 live s3fs[32209]: [tpath=/files/55cddcb2be884][bpath=55cddcb2be884][save=/files/55cddcb2be884][sseckeypos=-1]
400000 Jul 28 08:57:33 live s3fs[32209]: URL is https://s3.amazonaws.com/xxxxxxxxxxxx/files/55cddcb2be884
400001 Jul 28 08:57:33 live s3fs[32209]: URL changed is https://xxxxxxxxxxxx.s3.amazonaws.com/files/55cddcb2be884
400002 Jul 28 08:57:33 live s3fs[32209]: [tpath=/files/55cddd154a603][bpath=55cddd154a603][save=/files/55cddd154a603][sseckeypos=-1]
400003 Jul 28 08:57:33 live s3fs[32209]: URL is https://s3.amazonaws.com/xxxxxxxxxxxx/files/55cddd154a603
400004 Jul 28 08:57:33 live s3fs[32209]: URL changed is https://xxxxxxxxxxxx.s3.amazonaws.com/files/55cddd154a603
400005 Jul 28 08:57:33 live s3fs[32209]: [tpath=/files/55cddd154a87e][bpath=55cddd154a87e][save=/files/55cddd154a87e][sseckeypos=-1]
400006 Jul 28 08:57:33 live s3fs[32209]: URL is https://s3.amazonaws.com/xxxxxxxxxxxx/files/55cddd154a87e
400007 Jul 28 08:57:33 live s3fs[32209]: URL changed is https://xxxxxxxxxxxx.s3.amazonaws.com/files/55cddd154a87e
400008 Jul 28 08:57:33 live s3fs[32209]: [tpath=/files/55cddd3d68cda][bpath=55cddd3d68cda][save=/files/55cddd3d68cda][sseckeypos=-1]
400009 Jul 28 08:57:33 live s3fs[32209]: URL is https://s3.amazonaws.com/xxxxxxxxxxxx/files/55cddd3d68cda
400010 Jul 28 08:57:33 live s3fs[32209]: URL changed is https://xxxxxxxxxxxx.s3.amazonaws.com/files/55cddd3d68cda
400011 Jul 28 08:57:33 live s3fs[32209]: [tpath=/files/55cddd3d68ff7][bpath=55cddd3d68ff7][save=/files/55cddd3d68ff7][sseckeypos=-1]
400012 Jul 28 08:57:33 live s3fs[32209]: URL is https://s3.amazonaws.com/xxxxxxxxxxxx/files/55cddd3d68ff7
400013 Jul 28 08:57:33 live s3fs[32209]: URL changed is https://xxxxxxxxxxxx.s3.amazonaws.com/files/55cddd3d68ff7
400014 Jul 28 08:57:33 live s3fs[32209]: [tpath=/files/55cddd3d692c2][bpath=55cddd3d692c2][save=/files/55cddd3d692c2][sseckeypos=-1]
400015 Jul 28 08:57:33 live s3fs[32209]: URL is https://s3.amazonaws.com/xxxxxxxxxxxx/files/55cddd3d692c2
400016 Jul 28 08:57:33 live s3fs[32209]: URL changed is https://xxxxxxxxxxxx.s3.amazonaws.com/files/55cddd3d692c2
400017 Jul 28 08:57:33 live s3fs[32209]: [tpath=/files/55cdde115305b][bpath=55cdde115305b][save=/files/55cdde115305b][sseckeypos=-1]
@ggtakec commented on GitHub (Aug 2, 2020):
@idlemind
Do you know how to operate the files(dirs) under the mount point(s3fs) when this happens?
Also, are multiple s3fs processes running?
@gaul commented on GitHub (Nov 15, 2020):
Please reopen if symptoms persist.