[GH-ISSUE #1093] Multipart upload not working at all #590

New issue

Closed

opened 2026-03-04 01:47:00 +03:00 by kerem · 1 comment

kerem commented

2026-03-04 01:47:00 +03:00

Owner

Originally created by @larsch on GitHub (Jul 14, 2019).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1093

Additional Information

Version of s3fs being used (s3fs --version)

Amazon Simple Storage Service File System V1.85(commit:unknown) with OpenSSL

Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse)

% pacman -Qi fuse
Name            : fuse2
Version         : 2.9.9-1

s3fs command line used, if applicable

s3f  -o ensure_diskfree=1500 -o multipart_size=8 -o dbglevel=debug -o curldbg -f bucket /path/to/bucket -o passwd_file=...,url=...

s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)

if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages

[INF]       fdcache.cpp:RowFlush(1462): [tpath=][path=/bugfile][fd=8]
[INF]       curl.cpp:MultipartUploadRequest(3909): [upload_id=ODZk...jY2Mx][tpath=/bugfile][fd=8][offset=0][size=8519680]
[INF]       curl.cpp:UploadMultipartPostRequest(3667): [tpath=/bugfile][start=0][size=8519680][part=4]
[INF]       curl.cpp:UploadMultipartPostSetup(3607): [tpath=/bugfile][start=0][size=8519680][part=4]
[INF]       curl.cpp:prepare_url(4504): URL is https://anon/bugfile?partNumber=4&uploadId=ODZ...2Mx
[INF]       curl.cpp:prepare_url(4536): URL changed is https://anon/bugfile?partNumber=4&uploadId=OD...2Mx
[INF]       curl.cpp:insertV4Headers(2587): computing signature [PUT] [/bugfile] [partNumber=4&uploadId=ODZk...TRjY2Mx] [6532...999]
[INF]       curl.cpp:url_to_host(102): url is https://anon
[DBG] curl.cpp:RequestPerform(2212): connecting to URL 
[ERR] curl.cpp:RequestPerform(2376): ###curlCode: 43  msg: A libcurl function was given a bad argument
[ERR] curl.cpp:MultipartUploadRequest(3932): failed uploading part(-5)
[ERR] fdcache.cpp:RowFlush(1564): failed to multipart post(start=0, size=8519680) for file(8).

Details about issue

Large file uploads always fail. Multipart upload request are not working. It looks like curl_easy_setopt() is never invoked before curl_easy_perform() from S3fsCurl::UploadMultipartPostRequest()

I copied this code into S3fsCurl::UploadMultipartPostRequest(), and now it works:

  if(!fpLazySetup || !fpLazySetup(this)){
    S3FS_PRN_ERR("Failed to lazy setup in multipart upload post request.");
    return -1;
  }

Originally created by @larsch on GitHub (Jul 14, 2019). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1093 ### Additional Information #### Version of s3fs being used (s3fs --version) ``` Amazon Simple Storage Service File System V1.85(commit:unknown) with OpenSSL ``` #### Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse) ``` % pacman -Qi fuse Name : fuse2 Version : 2.9.9-1 ``` #### s3fs command line used, if applicable ``` s3f -o ensure_diskfree=1500 -o multipart_size=8 -o dbglevel=debug -o curldbg -f bucket /path/to/bucket -o passwd_file=...,url=... ``` #### s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs) _if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages_ ``` [INF] fdcache.cpp:RowFlush(1462): [tpath=][path=/bugfile][fd=8] [INF] curl.cpp:MultipartUploadRequest(3909): [upload_id=ODZk...jY2Mx][tpath=/bugfile][fd=8][offset=0][size=8519680] [INF] curl.cpp:UploadMultipartPostRequest(3667): [tpath=/bugfile][start=0][size=8519680][part=4] [INF] curl.cpp:UploadMultipartPostSetup(3607): [tpath=/bugfile][start=0][size=8519680][part=4] [INF] curl.cpp:prepare_url(4504): URL is https://anon/bugfile?partNumber=4&uploadId=ODZ...2Mx [INF] curl.cpp:prepare_url(4536): URL changed is https://anon/bugfile?partNumber=4&uploadId=OD...2Mx [INF] curl.cpp:insertV4Headers(2587): computing signature [PUT] [/bugfile] [partNumber=4&uploadId=ODZk...TRjY2Mx] [6532...999] [INF] curl.cpp:url_to_host(102): url is https://anon [DBG] curl.cpp:RequestPerform(2212): connecting to URL [ERR] curl.cpp:RequestPerform(2376): ###curlCode: 43 msg: A libcurl function was given a bad argument [ERR] curl.cpp:MultipartUploadRequest(3932): failed uploading part(-5) [ERR] fdcache.cpp:RowFlush(1564): failed to multipart post(start=0, size=8519680) for file(8). ``` ### Details about issue Large file uploads always fail. Multipart upload request are not working. It looks like `curl_easy_setopt()` is never invoked before `curl_easy_perform()` from `S3fsCurl::UploadMultipartPostRequest()` I copied this code into `S3fsCurl::UploadMultipartPostRequest()`, and now it works: ``` if(!fpLazySetup || !fpLazySetup(this)){ S3FS_PRN_ERR("Failed to lazy setup in multipart upload post request."); return -1; } ```

kerem closed this issue

2026-03-04 01:47:01 +03:00

kerem commented

2026-03-04 01:47:02 +03:00

Author

Owner

@ggtakec commented on GitHub (Jul 14, 2019):

@larsch Thanks, and your PR is merged.
Please let us know if you notice any problems in the future.

@ggtakec commented on GitHub (Jul 14, 2019): @larsch Thanks, and your PR is merged. Please let us know if you notice any problems in the future.

kerem referenced this issue

2026-03-04 02:01:04 +03:00

[PR #590] [MERGED] Updated man page for default_acl option - #567 #1562