mirror of
https://github.com/s3fs-fuse/s3fs-fuse.git
synced 2026-04-25 13:26:00 +03:00
[GH-ISSUE #748] Out of Memory error when running s3fs for couple of days. #431
Labels
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/s3fs-fuse#431
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @H6 on GitHub (Apr 14, 2018).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/748
Hi, i am experiencing an out of memory error when running s3fs for couple of days. Seems to depend on the amount of data uploaded.. but the disk seems not to be full.
Version of s3fs being used (s3fs --version)
1.8.3Version of fuse being used (pkg-config --modversion fuse)
2.9.4System information (uname -r)
4.9.20-11.31.amzn1.x86_64Distro (cat /etc/issue)
s3fs command line used (if applicable)
Logs
@ggtakec commented on GitHub (May 2, 2018):
@H6
Please accept my apologies for the late response.
I have confirmed that there is probably a problem with the memory in s3fs with NSS(libnss).
However, I am still checking out all versions.
Please tell me the exact version of s3fs you are using.
(It is the result of executing "s3fs --version")
In order to investigate and solve this problem, I also want to know your s3fs version.
Thanks in advance for your assistance.
@H6 commented on GitHub (May 3, 2018):
Hi @ggtakec , thank for the reply. I wrote the version already above. It's
1.8.3. Thanks.@ggtakec commented on GitHub (May 6, 2018):
@H6 I'm looking for a memory leak using valgrind etc, but we have not found a certain one yet.
As a precaution measure, I merged # XXX, but this will not affect this problem.
Still, it will take time to solve this problem, but I will continue to investigate.
@ggtakec commented on GitHub (May 7, 2018):
@H6 Probably I found the cause of the memory leak.
(I hope #340 is as same as this cause)0
I will fix and test the code, and will merge it on git, so please wait a few days.
Thanks in advance for your assistance.
@ggtakec commented on GitHub (May 13, 2018):
@H6 I'm sorry about that I have not yet solved this Issue.
I tried to correct the contents that seems to be the leak cause, but just that correction is not enough, and it will take more time to investigate.
Please wait for a while.
@pankajbat commented on GitHub (May 23, 2018):
We are having the same issue. Should we switch to an older version?
@ggtakec commented on GitHub (May 27, 2018):
@H6 I'm sorry for my late reply
I checked, reproduced and corrected the memory leak about v1.83.
And I merged #768 PR into the master branch.
If you can, it is helpful if you build the latest code of the master branch and try it.
If both s3fs and curl are linked with the NSS library, the possibility that the memory used inside CURL is increasing remains.
In this case, set YES to the environment variable NSS_SDB_USE_CACHE.
Here, I have not confirmed about this, but there are cases where memory usage increases in CURL+NSS.
In order to avoid this memory increasing, try "export NSS_SDB_USE_CACHE=YES".
@pankajbat commented on GitHub (Jun 17, 2018):
Takeshi,
Thanks for your help on this issue! I have installed the latest version
with the config change that you recommended and so far it seems to have no
memory leak issues (3 days). I will continue to monitor it for a few more
days and report back, if I encounter any issues.
Thanks again,
Pankaj Batra
On Sun, May 27, 2018 at 7:21 AM, Takeshi Nakatani notifications@github.com
wrote:
--
Thanks,
Pankaj Batra
@silviomoreto commented on GitHub (Jun 20, 2018):
Hi @ggtakec ! Thanks fixing that! When are you planning to create a release containing the fix?
@ggtakec commented on GitHub (Jul 8, 2018):
I'm sorry for my late reply.
Just now, I release v1.84 with tagging, then it will create new package soon.
After packaging new version, please try to use it.
Thanks in advance for your assistance.
@mattzuba commented on GitHub (Oct 8, 2018):
@ggtakec - where should one specify the NSS_SDB_USE_CACHE environment variable? I'm encountering this memory leak issue on 1.84 without having this variable set.
@nondualit commented on GitHub (Mar 12, 2019):
Hi there,
I'm having this memory leak. I just downloaded the newest version. But the oldest was from last week. Is there a problem again? or some dependencies with this new UBUNTU version?
4.15.0-1033-aws (kernel)
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.2 LTS"
Error:
Mar 12 09:51:09 nondualit_aws kernel: [249822.806272] Out of memory: Kill process 26871 (s3fs) score 284 or sacrifice child
Mar 12 09:51:09 nondualit_aws kernel: [249822.822296] Killed process 26871 (s3fs) total-vm:831084kB, anon-rss:294936kB, file-rss:0kB, shmem-rss:0kB
Mar 12 09:52:42 nondualit_aws kernel: [ 0.000000] Linux version 4.15.0-1033-aws (buildd@lcy01-amd64-019) (gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)) #35-U
buntu SMP Wed Feb 6 13:29:46 UTC 2019 (Ubuntu 4.15.0-1033.35-aws 4.15.18)
Mar 12 09:52:42 nondualit_aws kernel: [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-4.15.0-1033-aws root=UUID=fc9a41df-6d71-4f4f-a487-e5999bd67182 ro
console=tty1 console=ttyS0 nvme.io_timeout=4294967295
@gaul commented on GitHub (Mar 12, 2019):
@nondualit Please open a new issue with the s3fs version, your symptoms, and steps to reproduce out of memory conditions. It may help to use a memory profiler like Valgrind massif to help us understand the root cause.
@nondualit commented on GitHub (Mar 12, 2019):
I will!