starred/s3fs-fuse

Fork 0

mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2026-04-25 13:26:00 +03:00

[GH-ISSUE #2128] Download and delete a file at the same time will eventually not release the cache space #1084

New issue

Closed

opened 2026-03-04 01:51:16 +03:00 by kerem · 5 comments

kerem commented

2026-03-04 01:51:16 +03:00

Owner

Originally created by @huntersman on GitHub (Mar 14, 2023).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2128

Additional Information

Version of s3fs being used (`s3fs --version`)

V1.91

Version of fuse being used (`pkg-config --modversion fuse`, `rpm -qi fuse` or `dpkg -s fuse`)

2.9.2

Kernel information (`uname -r`)

5.4.213-1.el7.elrepo.x86_64

GNU/Linux Distribution, if applicable (`cat /etc/os-release`)

CentOS 7

How to run s3fs, if applicable

s3fs demo /root/demo -o passwd_file=${HOME}/.passwd-s3fs -o url=http://ip -o use_path_request_style -o noxmlns -o dbglevel=error -o default_acl=public-read -o logfile=/var/log/s3fs.log -o allow_other -o multireq_max=500 -o nocopyapi -o use_cache="/root/s3fs"

Details about issue

Use a SFTP tool to download a file from s3fs (For example, a 2GB file).
When it's downloading, delete the file. rm -f /root/demo/2GB
.fuse_hidden file is generated by Linux.
After downloading finished, .fuse_hidden is deleted automatically, but the cache space is still 2 GB.

Originally created by @huntersman on GitHub (Mar 14, 2023). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2128  ### Additional Information #### Version of s3fs being used (`s3fs --version`)  V1.91 #### Version of fuse being used (`pkg-config --modversion fuse`, `rpm -qi fuse` or `dpkg -s fuse`)  2.9.2 #### Kernel information (`uname -r`)  5.4.213-1.el7.elrepo.x86_64 #### GNU/Linux Distribution, if applicable (`cat /etc/os-release`)  CentOS 7 #### How to run s3fs, if applicable  `s3fs demo /root/demo -o passwd_file=${HOME}/.passwd-s3fs -o url=http://ip -o use_path_request_style -o noxmlns -o dbglevel=error -o default_acl=public-read -o logfile=/var/log/s3fs.log -o allow_other -o multireq_max=500 -o nocopyapi -o use_cache="/root/s3fs"` ### Details about issue  1. Use a SFTP tool to download a file from s3fs (For example, a 2GB file). 2. When it's downloading, delete the file. `rm -f /root/demo/2GB` 3. `.fuse_hidden` file is generated by Linux. 4. After downloading finished, `.fuse_hidden` is deleted automatically, but the cache space is still 2 GB.

kerem closed this issue

2026-03-04 01:51:16 +03:00

kerem commented

2026-03-04 01:51:17 +03:00

Author

Owner

@ggtakec commented on GitHub (Mar 16, 2023):

@huntersman Thanks for PR.
But I would like to know more information about "the cache space is still 2 GB."
Does this mean there are 2GB files left under the /root/s3fs directory?
There is no direct relationship between the .fuse_hidden file and s3fs' own cache file, so I'm wondering.

@ggtakec commented on GitHub (Mar 16, 2023): @huntersman Thanks for PR. But I would like to know more information about `"the cache space is still 2 GB."` Does this mean there are 2GB files left under the `/root/s3fs` directory? There is no direct relationship between the `.fuse_hidden` file and s3fs' own cache file, so I'm wondering.

kerem commented

2026-03-04 01:51:17 +03:00

Author

Owner

@huntersman commented on GitHub (Mar 17, 2023):

There is no 2GB file in cache directory, neither in .demo.stat and .demo.mirror. By the way, rm -rf /root/s3fs does not free the space, I guess the file is still using by s3fs. The result of df -h shows that the space is not freed. And I notice that there is an error in the log.

[ERR] s3fs.cpp:s3fs_release(2963):could not find pseudo_fd(3) for path(/.fuse_hidden0000000500000002)

After I kill s3fs, the space will be freed.

kill -9 s3fsPid

@huntersman commented on GitHub (Mar 17, 2023): There is no 2GB file in cache directory, neither in `.demo.stat` and `.demo.mirror`. By the way, `rm -rf /root/s3fs` does not free the space, I guess the file is still using by s3fs. The result of `df -h` shows that the space is not freed. And I notice that there is an error in the log. ```shell [ERR] s3fs.cpp:s3fs_release(2963):could not find pseudo_fd(3) for path(/.fuse_hidden0000000500000002) ``` After I kill s3fs, the space will be freed. ```shell kill -9 s3fsPid ```

kerem commented

2026-03-04 01:51:17 +03:00

Author

Owner

@ggtakec commented on GitHub (Mar 19, 2023):

@huntersman
I understood the cause of this problem.

FUSE temporarily changes the file name to /.fuse_hiddenXXXYYY if the open file is deleted by another process etc.
When the open file is closed, this temporarily renamed file is also deleted.

Above case, s3fs did not rename the cache for large files, but deleted them.(but, renamed except for large files)
This may have been leaked in past fixes.

I created #2135. (It will be closed because the fix is different than the #2130 you submitted.)

If possible, please try #2135 code.
Thanks in advance for your assistance.

@ggtakec commented on GitHub (Mar 19, 2023): @huntersman I understood the cause of this problem. FUSE temporarily changes the file name to `/.fuse_hiddenXXXYYY` if the open file is deleted by another process etc. When the open file is closed, this temporarily renamed file is also deleted. Above case, s3fs did not rename the cache for large files, but deleted them.(but, renamed except for large files) This may have been leaked in past fixes. I created #2135. (It will be closed because the fix is different than the #2130 you submitted.) If possible, please try #2135 code. Thanks in advance for your assistance.

kerem commented

2026-03-04 01:51:17 +03:00

Author

Owner

@huntersman commented on GitHub (Mar 20, 2023):

@ggtakec Thank you very much, I test it and it worked. I will close the issue.

@huntersman commented on GitHub (Mar 20, 2023): @ggtakec Thank you very much, I test it and it worked. I will close the issue.

kerem commented

2026-03-04 01:51:17 +03:00

Author

Owner

@ggtakec commented on GitHub (Mar 21, 2023):

@huntersman Thanks for checking that code.
PR #2135 will be merged soon.

@ggtakec commented on GitHub (Mar 21, 2023): @huntersman Thanks for checking that code. PR #2135 will be merged soon.