[PR #1098] [MERGED] Refixed to use copy api in multipart upload #1812

New issue

Closed

opened 2026-03-04 02:02:17 +03:00 by kerem · 0 comments

kerem commented

2026-03-04 02:02:17 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/s3fs-fuse/s3fs-fuse/pull/1098
Author: @ggtakec
Created: 7/15/2019
Status: ✅ Merged
Merged: 9/26/2019
Merged by: @gaul

Base: master ← Head: mix_upload

📝 Commits (2)

72d87cd Fixed to use copy api in multipart upload
3c0df7d Merge branch 'master' into mix_upload

📊 Changes

8 files changed (+619 additions, -20 deletions)

View changed files

📝 doc/man/s3fs.1 (+5 -0)
📝 src/curl.cpp (+168 -2)
📝 src/curl.h (+5 -0)
📝 src/fdcache.cpp (+326 -17)
📝 src/fdcache.h (+15 -1)
📝 src/s3fs.cpp (+9 -0)
📝 src/s3fs_util.cpp (+7 -0)
📝 test/integration-test-main.sh (+84 -0)

📄 Description

Relevant Issue (if applicable)

#607 #1027

Details

Fixes

Modified to use Upload part and Copy part(api) when multipart upload of object.

This will allow you to download and upload the smallest sized parts if you upload after writing a part of the object.
(In accordance with the minimum part size 5MB of multipart upload, useless download and upload will not be performed.)

As much as possible, s3fs will improve performance by uploading only the updated area.

The nomixupload option has been added to keep the previous behavior backward compatible.
When this option is specified, s3fs operates as before.
Also, when the nocopyapi, norenameapi and nomultipart options are specified, the same operation as before is performed.

Besides this, test cases for this modified are added. (and the two typo corrections.)

Notes(for canceled PR #1027 )

This PR is based on the canceled PR #1027 and rebases on the master that has advanced.
The issues pointed out in #1027 have been corrected.
However, there was a following bug in #1027.
The bug is not only uploading the wrote(updated) range but uploading all the downloaded range.
In this PR, only the minimum range written to the file(each part of multipart upload needs 5MB or more) is uploaded.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/s3fs-fuse/s3fs-fuse/pull/1098 **Author:** [@ggtakec](https://github.com/ggtakec) **Created:** 7/15/2019 **Status:** ✅ Merged **Merged:** 9/26/2019 **Merged by:** [@gaul](https://github.com/gaul) **Base:** `master` ← **Head:** `mix_upload` --- ### 📝 Commits (2) - [`72d87cd`](https://github.com/s3fs-fuse/s3fs-fuse/commit/72d87cd354ace1bb73f29de1eefadc7bf83d6e0c) Fixed to use copy api in multipart upload - [`3c0df7d`](https://github.com/s3fs-fuse/s3fs-fuse/commit/3c0df7dddc3a9be704503f0c9068bc4358df4569) Merge branch 'master' into mix_upload ### 📊 Changes **8 files changed** (+619 additions, -20 deletions) <details> <summary>View changed files</summary> 📝 `doc/man/s3fs.1` (+5 -0) 📝 `src/curl.cpp` (+168 -2) 📝 `src/curl.h` (+5 -0) 📝 `src/fdcache.cpp` (+326 -17) 📝 `src/fdcache.h` (+15 -1) 📝 `src/s3fs.cpp` (+9 -0) 📝 `src/s3fs_util.cpp` (+7 -0) 📝 `test/integration-test-main.sh` (+84 -0) </details> ### 📄 Description ## Relevant Issue (if applicable) #607 #1027 ## Details ### Fixes Modified to use Upload part and Copy part(api) when multipart upload of object. This will allow you to download and upload the smallest sized parts if you upload after writing a part of the object. (In accordance with the minimum part size 5MB of multipart upload, useless download and upload will not be performed.) As much as possible, s3fs will improve performance by uploading only the updated area. The nomixupload option has been added to keep the previous behavior backward compatible. When this option is specified, s3fs operates as before. Also, when the nocopyapi, norenameapi and nomultipart options are specified, the same operation as before is performed. Besides this, test cases for this modified are added. (and the two typo corrections.) ### Notes(for canceled PR #1027 ) This PR is based on the canceled PR #1027 and rebases on the master that has advanced. The issues pointed out in #1027 have been corrected. However, there was a following bug in #1027. The bug is not only uploading the wrote(updated) range but uploading all the downloaded range. In this PR, only the minimum range written to the file(each part of multipart upload needs 5MB or more) is uploaded. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>