[PR #1345] [CLOSED] repair write file corruption when the disk space is insufficient #1927

New issue

Closed

opened 2026-03-04 02:02:50 +03:00 by kerem · 0 comments

kerem commented

2026-03-04 02:02:50 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/s3fs-fuse/s3fs-fuse/pull/1345
Author: @liuyongqing
Created: 7/31/2020
Status: ❌ Closed

Base: master ← Head: master

📝 Commits (1)

1d36003 repair write file corruption when the disk space is insufficient

📊 Changes

1 file changed (+15 additions, -30 deletions)

View changed files

📝 src/fdcache.cpp (+15 -30)

📄 Description

Relevant Issue (if applicable)

https://github.com/s3fs-fuse/s3fs-fuse/issues/1346

Version: S3FS 1.8.6
Os：CentOS
Description:
When the disk space is insufficient, we use mv command move one file to s3fs mount point, the final data exist in s3 is zero length object, we can use -oensure_diskfree to easily reproduce it

Analysis:
the following is what happened in disk space sufficient case:

the process create a zero byte object in s3fs
the process write some data to local disk space, due to disk space is insufficient, triggerd init upload and upload several part
the process utimesat function to set file time, which will call put object copy api and generate another zero byte object according to step 1 generated zero object
the process call close function, which will complete upload part
finally according to s3 timing, the zero object in step 3 will remain.

according to above analysis, a non expect zero object exist in s3，in other case, the data is corrupt.

Solution:

If disk space is insuffient, we return error, let user free more disk space for use, the short of this solution is：utimesat is not effective in new file, and can't upload several big file in same time：https://github.com/s3fs-fuse/s3fs-fuse/pull/1345/commits
change chmod、chown and utimesat and other s3fs api, if the path's FdEntity exist and uploadid exist, only add metadata to cache

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/s3fs-fuse/s3fs-fuse/pull/1345 **Author:** [@liuyongqing](https://github.com/liuyongqing) **Created:** 7/31/2020 **Status:** ❌ Closed **Base:** `master` ← **Head:** `master` --- ### 📝 Commits (1) - [`1d36003`](https://github.com/s3fs-fuse/s3fs-fuse/commit/1d36003140478816871db7540ab0fa21959ca4c7) repair write file corruption when the disk space is insufficient ### 📊 Changes **1 file changed** (+15 additions, -30 deletions) <details> <summary>View changed files</summary> 📝 `src/fdcache.cpp` (+15 -30) </details> ### 📄 Description ### Relevant Issue (if applicable) https://github.com/s3fs-fuse/s3fs-fuse/issues/1346 Version: S3FS 1.8.6 Os：CentOS Description: When the disk space is insufficient, we use mv command move one file to s3fs mount point, the final data exist in s3 is zero length object, we can use -oensure_diskfree to easily reproduce it ![image](https://user-images.githubusercontent.com/5205797/89029037-14f2d080-d360-11ea-9e60-6cb6f0408a13.png) Analysis: the following is what happened in disk space sufficient case: 1. the process create a zero byte object in s3fs 2. the process write some data to local disk space, due to disk space is insufficient, triggerd init upload and upload several part 3. the process utimesat function to set file time, which will call put object copy api and generate another zero byte object according to step 1 generated zero object 4. the process call close function, which will complete upload part 5. finally according to s3 timing, the zero object in step 3 will remain. according to above analysis, a non expect zero object exist in s3，in other case, the data is corrupt. ![image](https://user-images.githubusercontent.com/5205797/89030548-48832a00-d363-11ea-94a1-dfd6d5f0f5a5.png) Solution: 1. If disk space is insuffient, we return error, let user free more disk space for use, the short of this solution is：utimesat is not effective in new file, and can't upload several big file in same time：https://github.com/s3fs-fuse/s3fs-fuse/pull/1345/commits 2. change chmod、chown and utimesat and other s3fs api, if the path's FdEntity exist and uploadid exist, only add metadata to cache --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>