mirror of
https://github.com/s3fs-fuse/s3fs-fuse.git
synced 2026-04-25 13:26:00 +03:00
[GH-ISSUE #1016] Trouble Disabling Cache Once Set #558
Labels
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/s3fs-fuse#558
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @jamiejackson on GitHub (Apr 18, 2019).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1016
Version of s3fs being used (s3fs --version)
Amazon Simple Storage Service File System V1.85(commit:unknown) with OpenSSL
Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse)
Name : fuse
Version : 2.9.2
Release : 11.el7
Architecture: x86_64
Kernel information (uname -r)
3.10.0-957.1.3.el7.x86_64
GNU/Linux Distribution, if applicable (cat /etc/os-release)
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"
CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"
s3fs command line used, if applicable
/etc/fstab entry, if applicable
s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)
if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages
Details about issue
What are the minimum steps needed to go from using cache (I'd had
use_cache=/tmp) to disabling it? I tried:use_cacheoption altogether and havinguse_cache=""sudo umount /mnt/s3/files-devsudo rm -rf /tmp/files-dev.mysite.comsudo mount /mnt/s3/files-devThen I accessed a file in the mount point, and
/tmp/files-dev.mysite.comreappeared with the file in it.I'm having trouble disabling it.
@jamiejackson commented on GitHub (Apr 19, 2019):
FYI, a reboot seemed to take care of it, but it would be nice to know the surgical, vs. sledgehammer, approach.
@ggtakec commented on GitHub (Apr 22, 2019):
@jamiejackson It was easy, but I also tried. However, it could not be reproduced.
(In addition, to disable use_cache, please do not specify use_cache option.)
It doesn't matter, but I think that the following directories exist, so it is recommended to delete them together.
/tmp/.files-dev.mysite.com.mirror
/tmp/.files-dev.mysite.com.stat
But I have not been reproduced it yet, so please let me know if there is any additional information.
Thanks in advance for your help.
@jamiejackson commented on GitHub (Apr 22, 2019):
Hi @ggtakec, I did try, in turn, both removing the
use_cacheoption and setting it to empty string, but neither seemed to disable it with just a re-mount.As I mentioned, it did work after rebooting the server, but that's not always practical.
@rhysmeister commented on GitHub (Apr 24, 2019):
I'm having similar issues...
fuse 2.9.3
s3fs 1.85
Redhat Ent 6.7
If I set use_cache="" it creates a directory on / called "" (yep double quotes) and uses that as cache. I have to remove it with rm -Rf \"\" (that makes me nervous). I've also messed around with the ensure_diskfree variable but this doesn't seem to help (it just bails when it hits the limit). Rebooting doesn't seem to help.
My mounts are set as follows...
s3fs#XXXXXXXX /mnt/XXXXXXXX fuse url=https://customendpoint.com,_netdev,allow_other,max_stat_cache_size=0,ensure_diskfree=500,multipart_size=10 0 0Is there a way to force multipart upload?
@jamiejackson commented on GitHub (May 2, 2019):
@rhysmeister , I just came across the behavior you described. The cache is tenacious!
FWIW, I had
use_cache='', so I ended up with a cache directory of/''/<mybucket>:-/Now I'm back the situation where I can't get rid of the cache (without rebooting), because one affected server has used
use_cache=''during this boot, and I'll probably have to remove the option and reboot to get rid of the caching.@rhysmeister , the only way I've been able to stop the cache is to remove the
use_cacheoption completely then reboot. Have you done exactly that yet?@jamiejackson commented on GitHub (May 2, 2019):
Maintainers: In addition to whatever investigation you might be doing, could you please modify the documentation to state that one must remove the option completely to disable the cache? Otherwise, other folks will continue to get bitten in the interim.
@jamiejackson commented on GitHub (May 2, 2019):
@ggtakec: FYI, I might be able to get around to enabling those debug settings and doing some testing at some point, but I won't be able to get to it right away.
@ggtakec commented on GitHub (May 4, 2019):
@jamiejackson
Did this problem occur when
""or''is given to use_cache option?I was able to confirm the same symptom as this problem by passing
use_cache=\"\".(In practice, normally(not root user) cache files could not be created directly under
/due to permission issues)Could you not pass double_quotes or single quotes for use_cache?
(If you run s3fs from shell, It is necessary to escape quotes.)
I think that it is possible to strengthen the check of options of s3fs, but there is also the issue of how much to check.
@jamiejackson commented on GitHub (May 6, 2019):
@rhysmeister (with
"") and I (with'') both had the problem of s3fs creating a directory in the root named as quotes.There's nothing keeping us from doing so, but it causes the problem we've already described. (It doesn't disable cache as the documentation claims it should.)
@rhysmeister commented on GitHub (May 6, 2019):
@jamiejackson I couldn't turn off the cache. It insisted on using /tmp. I threw some disk space at it.
@zjtheone commented on GitHub (May 26, 2019):
I'm having similar issues...,Has anyone solved this issue? Thanks