[GH-ISSUE #1346] The moved file length become zero when disk space is insufficient #721

New issue

Closed

opened 2026-03-04 01:48:12 +03:00 by kerem · 1 comment

kerem commented

2026-03-04 01:48:12 +03:00

Owner

Originally created by @liuyongqing on GitHub (Jul 31, 2020).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1346

Version: S3FS 1.8.6
Os：CentOS
Description:
When the disk space is insufficient, we use mv command move one file to s3fs mount point, the final data exist in s3 is zero length object, we can use -oensure_diskfree to easily reproduce it

Analysis:
the following is what happened in disk space sufficient case:

the process create a zero byte object in s3fs
the process write some data to local disk space, due to disk space is insufficient, triggerd init upload and upload several part
the process utimesat function to set file time, which will call put object copy api and generate another zero byte object according to step 1 generated zero object
the process call close function, which will complete upload part
finally according to s3 timing, the zero object in step 3 will remain.

according to above analysis, a non expect zero object exist in s3，in other case, the data is corrupt.

Solution:

If disk space is insuffient, we return error, let user free more disk space for use, the short of this solution is：utimesat is not effective in new file, and can't upload several big file in same time：https://github.com/s3fs-fuse/s3fs-fuse/pull/1345/commits
change chmod、chown and utimesat and other s3fs api, if the path's FdEntity exist and uploadid exist, only add metadata to cache

Originally created by @liuyongqing on GitHub (Jul 31, 2020). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1346 Version: S3FS 1.8.6 Os：CentOS Description: When the disk space is insufficient, we use mv command move one file to s3fs mount point, the final data exist in s3 is zero length object, we can use -oensure_diskfree to easily reproduce it ![image](https://user-images.githubusercontent.com/5205797/89029037-14f2d080-d360-11ea-9e60-6cb6f0408a13.png) Analysis: the following is what happened in disk space sufficient case: 1. the process create a zero byte object in s3fs 2. the process write some data to local disk space, due to disk space is insufficient, triggerd init upload and upload several part 3. the process utimesat function to set file time, which will call put object copy api and generate another zero byte object according to step 1 generated zero object 4. the process call close function, which will complete upload part 5. finally according to s3 timing, the zero object in step 3 will remain. according to above analysis, a non expect zero object exist in s3，in other case, the data is corrupt. ![image](https://user-images.githubusercontent.com/5205797/89030548-48832a00-d363-11ea-94a1-dfd6d5f0f5a5.png) Solution: 1. If disk space is insuffient, we return error, let user free more disk space for use, the short of this solution is：utimesat is not effective in new file, and can't upload several big file in same time：https://github.com/s3fs-fuse/s3fs-fuse/pull/1345/commits 2. change chmod、chown and utimesat and other s3fs api, if the path's FdEntity exist and uploadid exist, only add metadata to cache