[GH-ISSUE #2035] High Memory Usage #1026

New issue

Open

opened 2026-03-04 01:50:45 +03:00 by kerem · 3 comments

kerem commented

2026-03-04 01:50:45 +03:00

Owner

Originally created by @SanjeetKr7 on GitHub (Sep 23, 2022).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2035

Details about issue

As per the load model applied (2 sftp-clients will fetch files in every 5 mins) we have observed a Linear increase in the memory. Once the memory reaches the limit, it becomes constant and after some time it memory drops.

resources:
  limits:
    cpu: 500m
    memory: 1Gi
  requests:
    cpu: 10m
    memory: 100Mi

Version of s3fs being used (s3fs --version)

sh-4.4# s3fs --version
Amazon Simple Storage Service File System V1.91 (commit:unknown) with OpenSSL
Copyright (C) 2010 Randy Rizun rrizun@gmail.com
License GPL2: GNU GPL version 2 <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse)

sh-4.4# rpm -qi fuse
Name : fuse
Version : 2.9.7
Release : 15.el8
Architecture: x86_64
Install Date: Fri Jul 22 20:33:52 2022
Group : Unspecified
Size : 208300
License : GPL+
Signature : RSA/SHA256, Fri Feb 25 18:38:22 2022, Key ID 199e2f91fd431d51
Source RPM : fuse-2.9.7-15.el8.src.rpm
Build Date : Thu Feb 24 17:57:08 2022
Build Host : x86-vm-55.build.eng.bos.redhat.com
Relocations : (not relocatable)
Packager : Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>
Vendor : Red Hat, Inc.
URL : http://fuse.sf.net/
Summary : File System in Userspace (FUSE) v2 utilities
Description :
With FUSE it is possible to implement a fully functional filesystem in a
userspace program. This package contains the FUSE v2 userspace tools to
mount a FUSE filesystem.

Kernel information (uname -r)

sh-4.4# uname -r
4.18.0-305.49.1.el8_4.x86_64

GNU/Linux Distribution, if applicable (cat /etc/os-release)

[admin@slab743prov ~]$ cat /etc/os-release
NAME="Red Hat Enterprise Linux"
VERSION="8.4 (Ootpa)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="8.4"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
HOME_URL="https://www.redhat.com/"
DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.4"

s3fs command line used, if applicable

/etc/fstab entry, if applicable

s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)

if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages

Originally created by @SanjeetKr7 on GitHub (Sep 23, 2022). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2035 ### Details about issue As per the load model applied (2 sftp-clients will fetch files in every 5 mins) we have observed a **Linear increase in the memory**. Once the memory reaches the limit, it becomes constant and after some time it memory drops. ``` resources: limits: cpu: 500m memory: 1Gi requests: cpu: 10m memory: 100Mi ``` ![17a1b695-ff39-4be5-854b-4167d7b9e64c](https://user-images.githubusercontent.com/36546704/191977061-8f9ade27-4298-4e3c-bcbf-e68ab8a9135f.jpg) #### Version of s3fs being used (s3fs --version) sh-4.4# s3fs --version Amazon Simple Storage Service File System V1.91 (commit:unknown) with OpenSSL Copyright (C) 2010 Randy Rizun [rrizun@gmail.com](mailto:rrizun@gmail.com) License GPL2: GNU GPL version 2 <[https://gnu.org/licenses/gpl.html>](https://gnu.org/licenses/gpl.html%3E) This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. #### Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse) sh-4.4# rpm -qi fuse Name : fuse Version : 2.9.7 Release : 15.el8 Architecture: x86_64 Install Date: Fri Jul 22 20:33:52 2022 Group : Unspecified Size : 208300 License : GPL+ Signature : RSA/SHA256, Fri Feb 25 18:38:22 2022, Key ID 199e2f91fd431d51 Source RPM : fuse-2.9.7-15.el8.src.rpm Build Date : Thu Feb 24 17:57:08 2022 Build Host : x86-vm-55.build.eng.bos.redhat.com Relocations : (not relocatable) Packager : Red Hat, Inc. <[http://bugzilla.redhat.com/bugzilla>](http://bugzilla.redhat.com/bugzilla%3E) Vendor : Red Hat, Inc. URL : http://fuse.sf.net/ Summary : File System in Userspace (FUSE) v2 utilities Description : With FUSE it is possible to implement a fully functional filesystem in a userspace program. This package contains the FUSE v2 userspace tools to mount a FUSE filesystem. #### Kernel information (uname -r) sh-4.4# uname -r 4.18.0-305.49.1.el8_4.x86_64 #### GNU/Linux Distribution, if applicable (cat /etc/os-release) [admin@slab743prov ~]$ cat /etc/os-release NAME="Red Hat Enterprise Linux" VERSION="8.4 (Ootpa)" ID="rhel" ID_LIKE="fedora" VERSION_ID="8.4" PLATFORM_ID="platform:el8" PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" HOME_URL="[https://www.redhat.com/"](https://www.redhat.com/%22) DOCUMENTATION_URL="[https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"](https://access.redhat.com/documentation/red_hat_enterprise_linux/8/%22) BUG_REPORT_URL="[https://bugzilla.redhat.com/"](https://bugzilla.redhat.com/%22) REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" REDHAT_SUPPORT_PRODUCT_VERSION="8.4" #### s3fs command line used, if applicable ``` ``` #### /etc/fstab entry, if applicable ``` ``` #### s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs) _if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages_ ``` ```

kerem commented

2026-03-04 01:50:46 +03:00

Author

Owner

@SanjeetKr7 commented on GitHub (Sep 24, 2022):

@ggtakec
Similar to #2027 of high CPU usage, we are also getting high memory usage issue in s3fs-fuse V1.91
will this PR #2028 also help to improve memory performance?

@SanjeetKr7 commented on GitHub (Sep 24, 2022): @ggtakec Similar to #2027 of high CPU usage, we are also getting high memory usage issue in s3fs-fuse `V1.91` will this PR #2028 also help to improve memory performance?

kerem commented

2026-03-04 01:50:46 +03:00

Author

Owner

@ggtakec commented on GitHub (Sep 24, 2022):

I think #2028 doesn't affect memory usage much.
Does memory size decrease after s3fs activity(executes some instructions)? Or during execution?

It's not certain, but it's possible that the number of parallel executions is high.
What is the value of the multireq_max option set to? (Or is it unspecified?)

@ggtakec commented on GitHub (Sep 24, 2022): I think #2028 doesn't affect memory usage much. Does memory size decrease after s3fs activity(executes some instructions)? Or during execution? It's not certain, but it's possible that the number of parallel executions is high. What is the value of the multireq_max option set to? (Or is it unspecified?)

kerem commented

2026-03-04 01:50:46 +03:00

Author

Owner

@SanjeetKr7 commented on GitHub (Sep 26, 2022):

@ggtakec
multireq_max is unspecified

Default s3fs cmd that we use

    s3fs "${s3Bucket}" "${mountPath}" \
    -o ro \
    -o noatime \
    -o passwd_file="${s3cred},use_path_request_style,url=${s3Endpoint}" \
    -o dbglevel="err" \
    -o logfile="${logtmp}" \
    -o readwrite_timeout="${s3fsReadTimeout}"

Environment -> SFTP clients accessing the s3 storage files via the s3fs-fuse v1.91
When the sftp client is stopped at 15:15 then it is clearly seen that the CPU usage is reduced to zero, however there is not much change in the memory usage

@SanjeetKr7 commented on GitHub (Sep 26, 2022): @ggtakec `multireq_max` is unspecified Default s3fs cmd that we use ``` s3fs "${s3Bucket}" "${mountPath}" \ -o ro \ -o noatime \ -o passwd_file="${s3cred},use_path_request_style,url=${s3Endpoint}" \ -o dbglevel="err" \ -o logfile="${logtmp}" \ -o readwrite_timeout="${s3fsReadTimeout}" ``` Environment -> SFTP clients accessing the s3 storage files via the s3fs-fuse v1.91 When the sftp client is stopped at 15:15 then it is clearly seen that the CPU usage is reduced to zero, however there is not much change in the memory usage ![MicrosoftTeams-image (1)](https://user-images.githubusercontent.com/36546704/192263620-d529a73b-cc68-4327-9cf2-1de629ebb099.png)