[GH-ISSUE #406] Problem access files in s3 uploaded through the S3 console #215

New issue

Closed

opened 2026-03-04 01:43:19 +03:00 by kerem · 4 comments

kerem commented

2026-03-04 01:43:19 +03:00

Owner

Originally created by @andrewrutter on GitHub (Apr 29, 2016).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/406

I am trying to use s3fs to handle files being uploaded by a different process to an s3 bucket - currently they are being manually uploaaded. I can see files listed in that mounted directory and the files are showing correctly timestamped and with the correct size. But trying to open the files either in code or using vi or nano just shows garbage (nulls) such as ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@

The files are public and if I curl the s3 link then I do see content so it is available and there do not seem to be permission issues over http.

I am able to create a new file in this directory and the content is saving fine and I can view it in the S3 explorer.

Originally created by @andrewrutter on GitHub (Apr 29, 2016). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/406 I am trying to use s3fs to handle files being uploaded by a different process to an s3 bucket - currently they are being manually uploaaded. I can see files listed in that mounted directory and the files are showing correctly timestamped and with the correct size. But trying to open the files either in code or using vi or nano just shows garbage (nulls) such as ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ The files are public and if I curl the s3 link then I do see content so it is available and there do not seem to be permission issues over http. I am able to create a new file in this directory and the content is saving fine and I can view it in the S3 explorer.

kerem closed this issue

2026-03-04 01:43:19 +03:00

kerem commented

2026-03-04 01:43:20 +03:00

Author

Owner

@gaul commented on GitHub (Apr 29, 2016):

Can you provide a minimized test case, e.g., s3cmd put file; s3fs cat file? Also which version of s3fs are you using?

@gaul commented on GitHub (Apr 29, 2016): Can you provide a minimized test case, e.g., s3cmd put file; s3fs cat file? Also which version of s3fs are you using?

kerem commented

2026-03-04 01:43:20 +03:00

Author

Owner

@ggtakec commented on GitHub (May 14, 2016):

@andrewrutter What s3fs version do you use?
After v1.79 we fixed some bugs about local cache which is used by s3fs.
So that you should use latest codes in master branch if you use v1.79 or older than v1.79.

If you use latest codes, please try to do that.

remove all cache data in cache directory(which directory is specified by use_cache option)
you can find buckat name directory under use_cache path.
remove all stat cache file
you can find ..stat file under use_cache path.
start to run s3fs

If you overwrite the file(which exists) on S3 from S3 console when you have mounted the bucket in s3fs, this problem might occur.
In particular, in case you overwrite a smaller file.
I think that there is a possibility that occurs because differ with the file size in stat cache and file cache.
At that time, please try to set 0 for max_stat_cache_size option(= no stat cache).

Thanks in advance for your assistance.

@ggtakec commented on GitHub (May 14, 2016): @andrewrutter What s3fs version do you use? After v1.79 we fixed some bugs about local cache which is used by s3fs. So that you should use latest codes in master branch if you use v1.79 or older than v1.79. If you use latest codes, please try to do that. 1) remove all cache data in cache directory(which directory is specified by use_cache option) you can find buckat name directory under use_cache path. 2) remove all stat cache file you can find .<buckat name>.stat file under use_cache path. 3) start to run s3fs If you overwrite the file(which exists) on S3 from S3 console when you have mounted the bucket in s3fs, this problem might occur. In particular, in case you overwrite a smaller file. I think that there is a possibility that occurs because differ with the file size in stat cache and file cache. At that time, please try to set 0 for max_stat_cache_size option(= no stat cache). Thanks in advance for your assistance.

kerem commented

2026-03-04 01:43:20 +03:00

Author

Owner

@andrewrutter commented on GitHub (May 15, 2016):

I disabled cache completely and went with very basic options which resolved the problem. Cache is not an issue in our current use case since we are using S3 as a transactional store. It is entirely possible that the change you mention above would have resolved our issue.

@andrewrutter commented on GitHub (May 15, 2016): I disabled cache completely and went with very basic options which resolved the problem. Cache is not an issue in our current use case since we are using S3 as a transactional store. It is entirely possible that the change you mention above would have resolved our issue.

kerem commented

2026-03-04 01:43:20 +03:00

Author

Owner

@ggtakec commented on GitHub (May 29, 2016):

Hi, @andrewrutter
When you upload a object(file), s3fs makes temporarily local file to save it.
And if you set enable for s3fs cache, this temporary file is used as a permanently cached.

There was a bug about s3fs cache created, and we had fixed it after the v1.79 release.
I think that it might have been affected by this bug when you upload a file in s3fs that has a problem.
Now, if the problem is in hiding, I think that it was the influence of this bug.

I'll close this issue, but if your problem recurs, please reopen this issue.

Thanks in advance for your assistance.

@ggtakec commented on GitHub (May 29, 2016): Hi, @andrewrutter When you upload a object(file), s3fs makes temporarily local file to save it. And if you set enable for s3fs cache, this temporary file is used as a permanently cached. There was a bug about s3fs cache created, and we had fixed it after the v1.79 release. I think that it might have been affected by this bug when you upload a file in s3fs that has a problem. Now, if the problem is in hiding, I think that it was the influence of this bug. I'll close this issue, but if your problem recurs, please reopen this issue. Thanks in advance for your assistance.

kerem referenced this issue

2026-03-04 01:53:46 +03:00

[PR #215] [MERGED] Fix mem leak in openssl_auth.cpp:s3fs_sha256hexsum #1403