starred/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2026-04-25 21:35:58 +03:00
Code Issues 305 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #1591] Multipart upload with first part size < 5MB #834

New issue
Closed
opened 2026-03-04 01:49:11 +03:00 by kerem · 6 comments
kerem commented 2026-03-04 01:49:11 +03:00
Owner
Copy link

Originally created by @CarstenGrohmann on GitHub (Mar 3, 2021).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1591

In February I run a test with commit 4c6690f and uploaded more than 20000 files / 2 TB to a local S3 appliance.

Some uploads started with a segment size smaller that 5MB. My appliance accepted This violates the AWS S3 standards.

Example: First segment is smaller than 5MB, the next are 10MB and the final 1.6 MB

$ fgrep "upload_id=d1XLhUcHsM8BZDa8hCXzqim6ad4nLrQxpokWsTTFWGDPGKkWeZKcHuN" s3fs_20210208.log
2021-02-09T03:15:45.997Z [17328] [INF]       curl.cpp:MultipartUploadRequest(4015): [upload_id=d1XLhUcHsM8BZDa8hCXzqim6ad4nLrQxpokWsTTFWGDPGKkWeZKcHuN-Dg][tpath=/dir1/dir2/.file1.7D9K1j][fd=18][offset=0][size=4980736]
2021-02-09T03:15:46.116Z [17328] [INF]       curl.cpp:MultipartUploadRequest(4015): [upload_id=d1XLhUcHsM8BZDa8hCXzqim6ad4nLrQxpokWsTTFWGDPGKkWeZKcHuN-Dg][tpath=/dir1/dir2/.file1.7D9K1j][fd=18][offset=4980736][size=10485760]
021-02-09T03:15:46.310Z [17321] [INF]       curl.cpp:MultipartUploadRequest(4015): [upload_id=d1XLhUcHsM8BZDa8hCXzqim6ad4nLrQxpokWsTTFWGDPGKkWeZKcHuN-Dg][tpath=/dir1/dir2/.file1.7D9K1j][fd=18][offset=15466496][size=10485760]
2021-02-09T03:15:47.450Z [17328] [INF]       curl.cpp:MultipartUploadRequest(4015): [upload_id=d1XLhUcHsM8BZDa8hCXzqim6ad4nLrQxpokWsTTFWGDPGKkWeZKcHuN-Dg][tpath=/dir1/dir2/.file1.7D9K1j][fd=18][offset=25952256][size=10485760]
2021-02-09T03:15:48.053Z [17317] [INF]       curl.cpp:MultipartUploadRequest(4015): [upload_id=d1XLhUcHsM8BZDa8hCXzqim6ad4nLrQxpokWsTTFWGDPGKkWeZKcHuN-Dg][tpath=/dir1/dir2/.file1.7D9K1j][fd=18][offset=36438016][size=1695744]

Source file:

$ ll /src/dir1/dir2/file1
-rw-r--r-- 1 user user 38133760 Jun  6  2019 /src/dir1/dir2/file1

Extract from the debug log:

2021-02-09T03:15:45.980Z [17328] [DBG] s3fs.cpp:s3fs_write(2272): [path=/dir1/dir2/.file1.7D9K1j][size=131072][offset=4980736][fd=18]
2021-02-09T03:15:45.980Z [17328] [DBG] fdcache.cpp:ExistOpen(526): [path=/dir1/dir2/.file1.7D9K1j][fd=18][ignore_existfd=false]
2021-02-09T03:15:45.980Z [17328] [DBG] fdcache.cpp:Open(446): [path=/dir1/dir2/.file1.7D9K1j][size=-1][time=-1]
2021-02-09T03:15:45.980Z [17328] [DBG] fdcache_entity.cpp:Dup(246): [path=/dir1/dir2/.file1.7D9K1j][fd=18][refcnt=2]
2021-02-09T03:15:45.980Z [17328] [DBG] fdcache_entity.cpp:Open(315): [path=/dir1/dir2/.file1.7D9K1j][fd=18][size=-1][time=-1]
2021-02-09T03:15:45.980Z [17328] [DBG] fdcache_entity.cpp:Dup(246): [path=/dir1/dir2/.file1.7D9K1j][fd=18][refcnt=3]
2021-02-09T03:15:45.980Z [17328] [DBG] fdcache_entity.cpp:Close(200): [path=/dir1/dir2/.file1.7D9K1j][fd=18][refcnt=2]
2021-02-09T03:15:45.980Z [17328] [DBG] fdcache_entity.cpp:Write(1439): [path=/dir1/dir2/.file1.7D9K1j][fd=18][offset=4980736][size=131072]
2021-02-09T03:15:45.980Z [17328] [INF]       curl.cpp:PreMultipartPostRequest(3433): [tpath=/dir1/dir2/.file1.7D9K1j]
2021-02-09T03:15:45.980Z [17328] [DBG] curl_handlerpool.cpp:GetHandler(81): Get handler from pool: rest = 28
2021-02-09T03:15:45.980Z [17328] [INF]       curl_util.cpp:prepare_url(250): URL is http://mys3service/mybucket/dir1/dir2/.file1.7D9K1j?uploads
2021-02-09T03:15:45.980Z [17328] [INF]       curl_util.cpp:prepare_url(283): URL changed is http://mys3service/mybucket/dir1/dir2/.file1.7D9K1j?uploads
2021-02-09T03:15:45.980Z [17328] [DBG] curl.cpp:RequestPerform(2246): connecting to URL http://mys3service/mybucket/dir1/dir2/.file1.7D9K1j?uploads
2021-02-09T03:15:45.980Z [17328] [INF]       curl.cpp:insertV4Headers(2627): computing signature [POST] [/dir1/dir2/.file1.7D9K1j] [uploads] []
2021-02-09T03:15:45.980Z [17328] [INF]       curl_util.cpp:url_to_host(327): url is http://mys3service
2021-02-09T03:15:45.981Z [17328] [CURL DBG] * Found bundle for host mys3service: 0x7f10142a4440
2021-02-09T03:15:45.981Z [17328] [CURL DBG] * Re-using existing connection! (#25454) with host mys3service
2021-02-09T03:15:45.981Z [17328] [CURL DBG] * Connected to mys3service (192.168.1.1) port 80 (#25454)
2021-02-09T03:15:45.981Z [17328] [CURL DBG] > POST /mybucket/dir1/dir2/.file1.7D9K1j?uploads HTTP/1.1
2021-02-09T03:15:45.981Z [17328] [CURL DBG] > User-Agent: s3fs/1.88 (commit hash 4c6690f; OpenSSL)
2021-02-09T03:15:45.981Z [17328] [CURL DBG] > Authorization: AWS4-HMAC-SHA256 Credential=NHTL282ERGST9B40156H/20210209/us-east-1/s3/aws4_request, SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-atime;x-amz-m
eta-ctime;x-amz-meta-gid;x-amz-meta-mode;x-amz-meta-mtime;x-amz-meta-uid;x-amz-metadata-directive, Signature=44c13e3833dcdc549b0c8d2f9a2a58ee047b251bd017f8bb86b83019217b0b2a
2021-02-09T03:15:45.981Z [17328] [CURL DBG] > Content-Type: application/x-tar
2021-02-09T03:15:45.981Z [17328] [CURL DBG] > host: mys3service
2021-02-09T03:15:45.981Z [17328] [CURL DBG] > x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2021-02-09T03:15:45.981Z [17328] [CURL DBG] > x-amz-date: 20210209T031545Z
2021-02-09T03:15:45.981Z [17328] [CURL DBG] > x-amz-meta-atime: 1612840545
2021-02-09T03:15:45.981Z [17328] [CURL DBG] > x-amz-meta-ctime: 1612840545
2021-02-09T03:15:45.981Z [17328] [CURL DBG] > x-amz-meta-gid: 0
2021-02-09T03:15:45.981Z [17328] [CURL DBG] > x-amz-meta-mode: 33152
2021-02-09T03:15:45.981Z [17328] [CURL DBG] > x-amz-meta-mtime: 1612840545
2021-02-09T03:15:45.981Z [17328] [CURL DBG] > x-amz-meta-uid: 0
2021-02-09T03:15:45.981Z [17328] [CURL DBG] > x-amz-metadata-directive: REPLACE
2021-02-09T03:15:45.981Z [17328] [CURL DBG] >
2021-02-09T03:15:45.997Z [17328] [CURL DBG] < HTTP/1.1 200 OK
2021-02-09T03:15:45.997Z [17328] [CURL DBG] < Date: Tue, 09 Feb 2021 03:15:45 GMT
2021-02-09T03:15:45.997Z [17328] [CURL DBG] < Connection: KEEP-ALIVE
2021-02-09T03:15:45.997Z [17328] [CURL DBG] < Server: StorageGRID/11.4.0.2
2021-02-09T03:15:45.997Z [17328] [CURL DBG] < x-amz-request-id: 1612840531254473
2021-02-09T03:15:45.997Z [17328] [CURL DBG] < x-amz-id-2: 12700546
2021-02-09T03:15:45.997Z [17328] [CURL DBG] < Content-Length: 386
2021-02-09T03:15:45.997Z [17328] [CURL DBG] < Content-Type: application/xml
2021-02-09T03:15:45.997Z [17328] [CURL DBG] <
2021-02-09T03:15:45.997Z [17328] [CURL DBG] * Connection #25454 to host mys3service left intact
2021-02-09T03:15:45.997Z [17328] [INF]       curl.cpp:RequestPerform(2279): HTTP response code 200
2021-02-09T03:15:45.997Z [17328] [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool
2021-02-09T03:15:45.997Z [17328] [INF]       fdcache_entity.cpp:NoCacheLoadAndPost(962): [path=/dir1/dir2/.file1.7D9K1j][fd=18][offset=0][size=4980736]
2021-02-09T03:15:45.997Z [17328] [INF]       curl.cpp:MultipartUploadRequest(4015): [upload_id=d1XLhUcHsM8BZDa8hCXzqim6ad4nLrQxpokWsTTFWGDPGKkWeZKcHuN-Dg][tpath=/dir1/dir2/.file1.7D9K1j][fd=18][offset=0][size=4980736]
2021-02-09T03:15:45.997Z [17328] [INF]       curl.cpp:UploadMultipartPostRequest(3751): [tpath=/dir1/dir2/.file1.7D9K1j][start=0][size=4980736][part=1]
2021-02-09T03:15:45.997Z [17328] [INF]       curl.cpp:UploadMultipartPostSetup(3692): [tpath=/dir1/dir2/.file1.7D9K1j][start=0][size=4980736][part=1]
2021-02-09T03:15:45.997Z [17328] [INF]       curl_util.cpp:prepare_url(250): URL is http://mys3service/mybucket/dir1/dir2/.file1.7D9K1j?partNumber=1&uploadId=d1XLhUcHsM8BZDa8hCXzqim6ad4nLrQxpokWsTTFWGDPGKkWeZKcHuN-Dg
2021-02-09T03:15:45.997Z [17328] [INF]       curl_util.cpp:prepare_url(283): URL changed is http://mys3service/mybucket/dir1/dir2/.file1.7D9K1j?partNumber=1&uploadId=d1XLhUcHsM8BZDa8hCXzqim6ad4nLrQxpokWsTTFWGDPGKkWeZKcHuN-Dg
2021-02-09T03:15:45.997Z [17328] [DBG] curl_handlerpool.cpp:GetHandler(81): Get handler from pool: rest = 28
2021-02-09T03:15:45.997Z [17328] [DBG] curl.cpp:RequestPerform(2246): connecting to URL http://mys3service/mybucket/dir1/dir2/.file1.7D9K1j?partNumber=1&uploadId=d1XLhUcHsM8BZDa8hCXzqim6ad4nLrQxpokWsTTFWGDPGKkWeZKcHuN-Dg
2021-02-09T03:15:46.021Z [17328] [INF]       curl.cpp:insertV4Headers(2627): computing signature [PUT] [/dir1/dir2/.file1.7D9K1j] [partNumber=1&uploadId=d1XLhUcHsM8BZDa8hCXzqim6ad4nLrQxpokWsTTFWGDPGKkWeZKcHuN-Dg] [53f769421db0267371a5c8be8c71ed3a47b16197e35ceb0807ffdc24de04d1d6]
2021-02-09T03:15:46.021Z [17328] [CURL DBG] * Found bundle for host mys3service: 0x7f10142a4440
2021-02-09T03:15:46.021Z [17328] [CURL DBG] * Re-using existing connection! (#25454) with host mys3service
2021-02-09T03:15:46.021Z [17328] [CURL DBG] * Connected to mys3service (192.168.1.1) port 80 (#25454)
2021-02-09T03:15:46.021Z [17328] [CURL DBG] > PUT /mybucket/dir1/dir2/.file1.7D9K1j?partNumber=1&uploadId=d1XLhUcHsM8BZDa8hCXzqim6ad4nLrQxpokWsTTFWGDPGKkWeZKcHuN-Dg HTTP/1.1
2021-02-09T03:15:46.021Z [17328] [CURL DBG] > User-Agent: s3fs/1.88 (commit hash 4c6690f; OpenSSL)
2021-02-09T03:15:46.021Z [17328] [CURL DBG] > Authorization: AWS4-HMAC-SHA256 Credential=NHTL282ERGST9B40156H/20210209/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=9df0eee599a7958f52f3276ec34952a789f102355803b7515a7cef0c236bac7d
2021-02-09T03:15:46.021Z [17328] [CURL DBG] > host: mys3service
2021-02-09T03:15:46.021Z [17328] [CURL DBG] > x-amz-content-sha256: 53f769421db0267371a5c8be8c71ed3a47b16197e35ceb0807ffdc24de04d1d6
2021-02-09T03:15:46.021Z [17328] [CURL DBG] > x-amz-date: 20210209T031546Z
2021-02-09T03:15:46.021Z [17328] [CURL DBG] > Content-Length: 4980736
2021-02-09T03:15:46.021Z [17328] [CURL DBG] > Expect: 100-continue
2021-02-09T03:15:46.021Z [17328] [CURL DBG] >
2021-02-09T03:15:46.022Z [17328] [CURL DBG] < HTTP/1.1 100 Continue
2021-02-09T03:15:46.029Z [17328] [CURL DBG] * We are completely uploaded and fine
2021-02-09T03:15:46.050Z [17328] [CURL DBG] < HTTP/1.1 200 OK
2021-02-09T03:15:46.050Z [17328] [CURL DBG] < Date: Tue, 09 Feb 2021 03:15:46 GMT
2021-02-09T03:15:46.050Z [17328] [CURL DBG] < Connection: KEEP-ALIVE
2021-02-09T03:15:46.050Z [17328] [CURL DBG] < Server: StorageGRID/11.4.0.2
2021-02-09T03:15:46.050Z [17328] [CURL DBG] < x-amz-request-id: 1612840531254473
2021-02-09T03:15:46.050Z [17328] [CURL DBG] < x-amz-id-2: 12700546
2021-02-09T03:15:46.050Z [17328] [CURL DBG] < Content-Length: 0
2021-02-09T03:15:46.050Z [17328] [CURL DBG] < ETag: "2b37ec4645d5fbb261bed462f3eeec29"
2021-02-09T03:15:46.050Z [17328] [CURL DBG] <
2021-02-09T03:15:46.050Z [17328] [CURL DBG] * Connection #25454 to host mys3service left intact
2021-02-09T03:15:46.050Z [17328] [INF]       curl.cpp:RequestPerform(2279): HTTP response code 200

The debug output is reworked to hide internal names.

Version of s3fs being used (s3fs --version)

Commit: 4c6690f

s3fs command line

$ ./s3fs mybucket /mybucket -o url=http://mys3service,use_path_request_style,nomultipart,curldbg,dbglevel=debug -d -d -f
2021-02-08T09:50:57.430Z [4260] [CRT] s3fs_logger.cpp:LowSetLogLevel(202): change debug level from [CRT] to [DBG]
2021-02-08T09:50:57.430Z [4260] [INF]     s3fs.cpp:set_mountpoint_attribute(3990): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755)
2021-02-08T09:50:57.430Z [4260] [DBG] curl.cpp:InitMimeType(410): Try to load mime types from /etc/mime.types file.
2021-02-08T09:50:57.431Z [4260] [DBG] curl.cpp:InitMimeType(415): The old mime types are cleared to load new mime types.
2021-02-08T09:50:57.432Z [4260] [INF] curl.cpp:InitMimeType(438): Loaded mime information from /etc/mime.types
2021-02-08T09:50:57.432Z [4260] [INF] fdcache_stat.cpp:CheckCacheFileStatTopDir(79): The path to cache top dir is empty, thus not need to check permission.
FUSE library version: 2.9.2
nullpath_ok: 0
nopath: 0
utime_omit_ok: 0
unique: 1, opcode: INIT (26), nodeid: 0, insize: 56, pid: 0
INIT: 7.22
flags=0x0000f7fb
max_readahead=0x00020000
2021-02-08T09:50:57.433Z [4261] [INF] s3fs.cpp:s3fs_init(3301): init v1.88(commit:4c6690f) with OpenSSL
Originally created by @CarstenGrohmann on GitHub (Mar 3, 2021). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1591 In February I run a test with commit 4c6690f and uploaded more than 20000 files / 2 TB to a local S3 appliance. Some uploads started with a segment size smaller that 5MB. My appliance accepted This violates the AWS S3 standards. **Example:** First segment is smaller than 5MB, the next are 10MB and the final 1.6 MB ``` $ fgrep "upload_id=d1XLhUcHsM8BZDa8hCXzqim6ad4nLrQxpokWsTTFWGDPGKkWeZKcHuN" s3fs_20210208.log 2021-02-09T03:15:45.997Z [17328] [INF] curl.cpp:MultipartUploadRequest(4015): [upload_id=d1XLhUcHsM8BZDa8hCXzqim6ad4nLrQxpokWsTTFWGDPGKkWeZKcHuN-Dg][tpath=/dir1/dir2/.file1.7D9K1j][fd=18][offset=0][size=4980736] 2021-02-09T03:15:46.116Z [17328] [INF] curl.cpp:MultipartUploadRequest(4015): [upload_id=d1XLhUcHsM8BZDa8hCXzqim6ad4nLrQxpokWsTTFWGDPGKkWeZKcHuN-Dg][tpath=/dir1/dir2/.file1.7D9K1j][fd=18][offset=4980736][size=10485760] 021-02-09T03:15:46.310Z [17321] [INF] curl.cpp:MultipartUploadRequest(4015): [upload_id=d1XLhUcHsM8BZDa8hCXzqim6ad4nLrQxpokWsTTFWGDPGKkWeZKcHuN-Dg][tpath=/dir1/dir2/.file1.7D9K1j][fd=18][offset=15466496][size=10485760] 2021-02-09T03:15:47.450Z [17328] [INF] curl.cpp:MultipartUploadRequest(4015): [upload_id=d1XLhUcHsM8BZDa8hCXzqim6ad4nLrQxpokWsTTFWGDPGKkWeZKcHuN-Dg][tpath=/dir1/dir2/.file1.7D9K1j][fd=18][offset=25952256][size=10485760] 2021-02-09T03:15:48.053Z [17317] [INF] curl.cpp:MultipartUploadRequest(4015): [upload_id=d1XLhUcHsM8BZDa8hCXzqim6ad4nLrQxpokWsTTFWGDPGKkWeZKcHuN-Dg][tpath=/dir1/dir2/.file1.7D9K1j][fd=18][offset=36438016][size=1695744] ``` **Source file:** ``` $ ll /src/dir1/dir2/file1 -rw-r--r-- 1 user user 38133760 Jun 6 2019 /src/dir1/dir2/file1 ``` **Extract from the debug log:** ``` 2021-02-09T03:15:45.980Z [17328] [DBG] s3fs.cpp:s3fs_write(2272): [path=/dir1/dir2/.file1.7D9K1j][size=131072][offset=4980736][fd=18] 2021-02-09T03:15:45.980Z [17328] [DBG] fdcache.cpp:ExistOpen(526): [path=/dir1/dir2/.file1.7D9K1j][fd=18][ignore_existfd=false] 2021-02-09T03:15:45.980Z [17328] [DBG] fdcache.cpp:Open(446): [path=/dir1/dir2/.file1.7D9K1j][size=-1][time=-1] 2021-02-09T03:15:45.980Z [17328] [DBG] fdcache_entity.cpp:Dup(246): [path=/dir1/dir2/.file1.7D9K1j][fd=18][refcnt=2] 2021-02-09T03:15:45.980Z [17328] [DBG] fdcache_entity.cpp:Open(315): [path=/dir1/dir2/.file1.7D9K1j][fd=18][size=-1][time=-1] 2021-02-09T03:15:45.980Z [17328] [DBG] fdcache_entity.cpp:Dup(246): [path=/dir1/dir2/.file1.7D9K1j][fd=18][refcnt=3] 2021-02-09T03:15:45.980Z [17328] [DBG] fdcache_entity.cpp:Close(200): [path=/dir1/dir2/.file1.7D9K1j][fd=18][refcnt=2] 2021-02-09T03:15:45.980Z [17328] [DBG] fdcache_entity.cpp:Write(1439): [path=/dir1/dir2/.file1.7D9K1j][fd=18][offset=4980736][size=131072] 2021-02-09T03:15:45.980Z [17328] [INF] curl.cpp:PreMultipartPostRequest(3433): [tpath=/dir1/dir2/.file1.7D9K1j] 2021-02-09T03:15:45.980Z [17328] [DBG] curl_handlerpool.cpp:GetHandler(81): Get handler from pool: rest = 28 2021-02-09T03:15:45.980Z [17328] [INF] curl_util.cpp:prepare_url(250): URL is http://mys3service/mybucket/dir1/dir2/.file1.7D9K1j?uploads 2021-02-09T03:15:45.980Z [17328] [INF] curl_util.cpp:prepare_url(283): URL changed is http://mys3service/mybucket/dir1/dir2/.file1.7D9K1j?uploads 2021-02-09T03:15:45.980Z [17328] [DBG] curl.cpp:RequestPerform(2246): connecting to URL http://mys3service/mybucket/dir1/dir2/.file1.7D9K1j?uploads 2021-02-09T03:15:45.980Z [17328] [INF] curl.cpp:insertV4Headers(2627): computing signature [POST] [/dir1/dir2/.file1.7D9K1j] [uploads] [] 2021-02-09T03:15:45.980Z [17328] [INF] curl_util.cpp:url_to_host(327): url is http://mys3service 2021-02-09T03:15:45.981Z [17328] [CURL DBG] * Found bundle for host mys3service: 0x7f10142a4440 2021-02-09T03:15:45.981Z [17328] [CURL DBG] * Re-using existing connection! (#25454) with host mys3service 2021-02-09T03:15:45.981Z [17328] [CURL DBG] * Connected to mys3service (192.168.1.1) port 80 (#25454) 2021-02-09T03:15:45.981Z [17328] [CURL DBG] > POST /mybucket/dir1/dir2/.file1.7D9K1j?uploads HTTP/1.1 2021-02-09T03:15:45.981Z [17328] [CURL DBG] > User-Agent: s3fs/1.88 (commit hash 4c6690f; OpenSSL) 2021-02-09T03:15:45.981Z [17328] [CURL DBG] > Authorization: AWS4-HMAC-SHA256 Credential=NHTL282ERGST9B40156H/20210209/us-east-1/s3/aws4_request, SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-atime;x-amz-m eta-ctime;x-amz-meta-gid;x-amz-meta-mode;x-amz-meta-mtime;x-amz-meta-uid;x-amz-metadata-directive, Signature=44c13e3833dcdc549b0c8d2f9a2a58ee047b251bd017f8bb86b83019217b0b2a 2021-02-09T03:15:45.981Z [17328] [CURL DBG] > Content-Type: application/x-tar 2021-02-09T03:15:45.981Z [17328] [CURL DBG] > host: mys3service 2021-02-09T03:15:45.981Z [17328] [CURL DBG] > x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 2021-02-09T03:15:45.981Z [17328] [CURL DBG] > x-amz-date: 20210209T031545Z 2021-02-09T03:15:45.981Z [17328] [CURL DBG] > x-amz-meta-atime: 1612840545 2021-02-09T03:15:45.981Z [17328] [CURL DBG] > x-amz-meta-ctime: 1612840545 2021-02-09T03:15:45.981Z [17328] [CURL DBG] > x-amz-meta-gid: 0 2021-02-09T03:15:45.981Z [17328] [CURL DBG] > x-amz-meta-mode: 33152 2021-02-09T03:15:45.981Z [17328] [CURL DBG] > x-amz-meta-mtime: 1612840545 2021-02-09T03:15:45.981Z [17328] [CURL DBG] > x-amz-meta-uid: 0 2021-02-09T03:15:45.981Z [17328] [CURL DBG] > x-amz-metadata-directive: REPLACE 2021-02-09T03:15:45.981Z [17328] [CURL DBG] > 2021-02-09T03:15:45.997Z [17328] [CURL DBG] < HTTP/1.1 200 OK 2021-02-09T03:15:45.997Z [17328] [CURL DBG] < Date: Tue, 09 Feb 2021 03:15:45 GMT 2021-02-09T03:15:45.997Z [17328] [CURL DBG] < Connection: KEEP-ALIVE 2021-02-09T03:15:45.997Z [17328] [CURL DBG] < Server: StorageGRID/11.4.0.2 2021-02-09T03:15:45.997Z [17328] [CURL DBG] < x-amz-request-id: 1612840531254473 2021-02-09T03:15:45.997Z [17328] [CURL DBG] < x-amz-id-2: 12700546 2021-02-09T03:15:45.997Z [17328] [CURL DBG] < Content-Length: 386 2021-02-09T03:15:45.997Z [17328] [CURL DBG] < Content-Type: application/xml 2021-02-09T03:15:45.997Z [17328] [CURL DBG] < 2021-02-09T03:15:45.997Z [17328] [CURL DBG] * Connection #25454 to host mys3service left intact 2021-02-09T03:15:45.997Z [17328] [INF] curl.cpp:RequestPerform(2279): HTTP response code 200 2021-02-09T03:15:45.997Z [17328] [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool 2021-02-09T03:15:45.997Z [17328] [INF] fdcache_entity.cpp:NoCacheLoadAndPost(962): [path=/dir1/dir2/.file1.7D9K1j][fd=18][offset=0][size=4980736] 2021-02-09T03:15:45.997Z [17328] [INF] curl.cpp:MultipartUploadRequest(4015): [upload_id=d1XLhUcHsM8BZDa8hCXzqim6ad4nLrQxpokWsTTFWGDPGKkWeZKcHuN-Dg][tpath=/dir1/dir2/.file1.7D9K1j][fd=18][offset=0][size=4980736] 2021-02-09T03:15:45.997Z [17328] [INF] curl.cpp:UploadMultipartPostRequest(3751): [tpath=/dir1/dir2/.file1.7D9K1j][start=0][size=4980736][part=1] 2021-02-09T03:15:45.997Z [17328] [INF] curl.cpp:UploadMultipartPostSetup(3692): [tpath=/dir1/dir2/.file1.7D9K1j][start=0][size=4980736][part=1] 2021-02-09T03:15:45.997Z [17328] [INF] curl_util.cpp:prepare_url(250): URL is http://mys3service/mybucket/dir1/dir2/.file1.7D9K1j?partNumber=1&uploadId=d1XLhUcHsM8BZDa8hCXzqim6ad4nLrQxpokWsTTFWGDPGKkWeZKcHuN-Dg 2021-02-09T03:15:45.997Z [17328] [INF] curl_util.cpp:prepare_url(283): URL changed is http://mys3service/mybucket/dir1/dir2/.file1.7D9K1j?partNumber=1&uploadId=d1XLhUcHsM8BZDa8hCXzqim6ad4nLrQxpokWsTTFWGDPGKkWeZKcHuN-Dg 2021-02-09T03:15:45.997Z [17328] [DBG] curl_handlerpool.cpp:GetHandler(81): Get handler from pool: rest = 28 2021-02-09T03:15:45.997Z [17328] [DBG] curl.cpp:RequestPerform(2246): connecting to URL http://mys3service/mybucket/dir1/dir2/.file1.7D9K1j?partNumber=1&uploadId=d1XLhUcHsM8BZDa8hCXzqim6ad4nLrQxpokWsTTFWGDPGKkWeZKcHuN-Dg 2021-02-09T03:15:46.021Z [17328] [INF] curl.cpp:insertV4Headers(2627): computing signature [PUT] [/dir1/dir2/.file1.7D9K1j] [partNumber=1&uploadId=d1XLhUcHsM8BZDa8hCXzqim6ad4nLrQxpokWsTTFWGDPGKkWeZKcHuN-Dg] [53f769421db0267371a5c8be8c71ed3a47b16197e35ceb0807ffdc24de04d1d6] 2021-02-09T03:15:46.021Z [17328] [CURL DBG] * Found bundle for host mys3service: 0x7f10142a4440 2021-02-09T03:15:46.021Z [17328] [CURL DBG] * Re-using existing connection! (#25454) with host mys3service 2021-02-09T03:15:46.021Z [17328] [CURL DBG] * Connected to mys3service (192.168.1.1) port 80 (#25454) 2021-02-09T03:15:46.021Z [17328] [CURL DBG] > PUT /mybucket/dir1/dir2/.file1.7D9K1j?partNumber=1&uploadId=d1XLhUcHsM8BZDa8hCXzqim6ad4nLrQxpokWsTTFWGDPGKkWeZKcHuN-Dg HTTP/1.1 2021-02-09T03:15:46.021Z [17328] [CURL DBG] > User-Agent: s3fs/1.88 (commit hash 4c6690f; OpenSSL) 2021-02-09T03:15:46.021Z [17328] [CURL DBG] > Authorization: AWS4-HMAC-SHA256 Credential=NHTL282ERGST9B40156H/20210209/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=9df0eee599a7958f52f3276ec34952a789f102355803b7515a7cef0c236bac7d 2021-02-09T03:15:46.021Z [17328] [CURL DBG] > host: mys3service 2021-02-09T03:15:46.021Z [17328] [CURL DBG] > x-amz-content-sha256: 53f769421db0267371a5c8be8c71ed3a47b16197e35ceb0807ffdc24de04d1d6 2021-02-09T03:15:46.021Z [17328] [CURL DBG] > x-amz-date: 20210209T031546Z 2021-02-09T03:15:46.021Z [17328] [CURL DBG] > Content-Length: 4980736 2021-02-09T03:15:46.021Z [17328] [CURL DBG] > Expect: 100-continue 2021-02-09T03:15:46.021Z [17328] [CURL DBG] > 2021-02-09T03:15:46.022Z [17328] [CURL DBG] < HTTP/1.1 100 Continue 2021-02-09T03:15:46.029Z [17328] [CURL DBG] * We are completely uploaded and fine 2021-02-09T03:15:46.050Z [17328] [CURL DBG] < HTTP/1.1 200 OK 2021-02-09T03:15:46.050Z [17328] [CURL DBG] < Date: Tue, 09 Feb 2021 03:15:46 GMT 2021-02-09T03:15:46.050Z [17328] [CURL DBG] < Connection: KEEP-ALIVE 2021-02-09T03:15:46.050Z [17328] [CURL DBG] < Server: StorageGRID/11.4.0.2 2021-02-09T03:15:46.050Z [17328] [CURL DBG] < x-amz-request-id: 1612840531254473 2021-02-09T03:15:46.050Z [17328] [CURL DBG] < x-amz-id-2: 12700546 2021-02-09T03:15:46.050Z [17328] [CURL DBG] < Content-Length: 0 2021-02-09T03:15:46.050Z [17328] [CURL DBG] < ETag: "2b37ec4645d5fbb261bed462f3eeec29" 2021-02-09T03:15:46.050Z [17328] [CURL DBG] < 2021-02-09T03:15:46.050Z [17328] [CURL DBG] * Connection #25454 to host mys3service left intact 2021-02-09T03:15:46.050Z [17328] [INF] curl.cpp:RequestPerform(2279): HTTP response code 200 ``` The debug output is reworked to hide internal names. ## Version of s3fs being used (s3fs --version) Commit: 4c6690f ## s3fs command line ``` $ ./s3fs mybucket /mybucket -o url=http://mys3service,use_path_request_style,nomultipart,curldbg,dbglevel=debug -d -d -f 2021-02-08T09:50:57.430Z [4260] [CRT] s3fs_logger.cpp:LowSetLogLevel(202): change debug level from [CRT] to [DBG] 2021-02-08T09:50:57.430Z [4260] [INF] s3fs.cpp:set_mountpoint_attribute(3990): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755) 2021-02-08T09:50:57.430Z [4260] [DBG] curl.cpp:InitMimeType(410): Try to load mime types from /etc/mime.types file. 2021-02-08T09:50:57.431Z [4260] [DBG] curl.cpp:InitMimeType(415): The old mime types are cleared to load new mime types. 2021-02-08T09:50:57.432Z [4260] [INF] curl.cpp:InitMimeType(438): Loaded mime information from /etc/mime.types 2021-02-08T09:50:57.432Z [4260] [INF] fdcache_stat.cpp:CheckCacheFileStatTopDir(79): The path to cache top dir is empty, thus not need to check permission. FUSE library version: 2.9.2 nullpath_ok: 0 nopath: 0 utime_omit_ok: 0 unique: 1, opcode: INIT (26), nodeid: 0, insize: 56, pid: 0 INIT: 7.22 flags=0x0000f7fb max_readahead=0x00020000 2021-02-08T09:50:57.433Z [4261] [INF] s3fs.cpp:s3fs_init(3301): init v1.88(commit:4c6690f) with OpenSSL ```
kerem 2026-03-04 01:49:11 +03:00
  • closed this issue
  • added the
    need info
         label
kerem commented 2026-03-04 01:49:12 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Mar 3, 2021):

I am confused about the flags you use since nomultipart is set in your example. Can you provide the exact steps you took that reproduce these symptoms?

<!-- gh-comment-id:789669194 --> @gaul commented on GitHub (Mar 3, 2021): I am confused about the flags you use since `nomultipart` is set in your example. Can you provide the exact steps you took that reproduce these symptoms?
kerem commented 2026-03-04 01:49:12 +03:00
Author
Owner
Copy link

@CarstenGrohmann commented on GitHub (Mar 4, 2021):

Currently s3fs used multipart uploads even if nomultipart is set. I've filed a separate issue #1595 to address this question, because it looks like a bug in s3fs.

From my current perspective the nomultipart flag can be ignored in this case.

I see this issue only if I use multiple parallel rsync instances to copy files to my bucket. I couldn't reproduce it with a single cp command. I'll try simplifying the test case and share the results.

<!-- gh-comment-id:790457724 --> @CarstenGrohmann commented on GitHub (Mar 4, 2021): Currently s3fs used multipart uploads even if `nomultipart` is set. I've filed a separate issue #1595 to address this question, because it looks like a bug in s3fs. From my current perspective the `nomultipart` flag can be ignored in this case. I see this issue only if I use multiple parallel `rsync `instances to copy files to my bucket. I couldn't reproduce it with a single `cp ` command. I'll try simplifying the test case and share the results.
kerem commented 2026-03-04 01:49:12 +03:00
Author
Owner
Copy link

@CarstenGrohmann commented on GitHub (Mar 5, 2021):

That's probably the same /tmp issue as described in #1595. I will try to reproduce the problem with the new knowledge and attach a short test case.

<!-- gh-comment-id:791538787 --> @CarstenGrohmann commented on GitHub (Mar 5, 2021): That's probably the same `/tmp` issue as described in #1595. I will try to reproduce the problem with the new knowledge and attach a short test case.
kerem commented 2026-03-04 01:49:12 +03:00
Author
Owner
Copy link

@CarstenGrohmann commented on GitHub (Mar 8, 2021):

Steps to reproduce:

  1. Fill /tmp except 1..2 MB

    $ df -k /tmp/
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/sda3        3997376 3769464      1816 100% /tmp

  2. Start s3fs

    $ ./s3fs mybucket /grid/cfm2 -o url=http://mys3service:8080 -o use_path_request_style,nomultipart,notsup_compat_dir,enable_noobj_cache,multipart_size=100,curldbg,dbglevel=debug -d -d -f
2021-03-07T16:55:40.563Z [CRT] s3fs_logger.cpp:LowSetLogLevel(219): change debug level from [CRT] to [DBG]
2021-03-07T16:55:40.563Z [INF]     s3fs.cpp:set_mountpoint_attribute(4020): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755)
2021-03-07T16:55:40.563Z [DBG] curl.cpp:InitMimeType(408): Try to load mime types from /etc/mime.types file.
2021-03-07T16:55:40.563Z [DBG] curl.cpp:InitMimeType(413): The old mime types are cleared to load new mime types.
2021-03-07T16:55:40.564Z [INF] curl.cpp:InitMimeType(436): Loaded mime information from /etc/mime.types
2021-03-07T16:55:40.565Z [INF] fdcache_stat.cpp:CheckCacheFileStatTopDir(79): The path to cache top dir is empty, thus not need to check permission.
FUSE library version: 2.9.2
nullpath_ok: 0
nopath: 0
utime_omit_ok: 0
unique: 1, opcode: INIT (26), nodeid: 0, insize: 56, pid: 0
INIT: 7.22
flags=0x0000f7fb
max_readahead=0x00020000
2021-03-07T16:55:40.566Z [INF] s3fs.cpp:s3fs_init(3331): init v1.89(commit:8c58ba8) with OpenSSL

  3. Fill copy a file to S3

    $ cp -p 3000M /grid/cfm2/

  4. Check output:

    1. First uploaded part is just 1835008 bytes

      2021-03-07T17:02:16.764Z [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool
2021-03-07T17:02:16.764Z [INF]       fdcache_entity.cpp:NoCacheLoadAndPost(962): [path=/3000M][fd=5][offset=0][size=1835008]
2021-03-07T17:02:16.764Z [INF]       curl.cpp:MultipartUploadRequest(4034): [upload_id=uc101yNzruXeD6pgLfcZWykX6O3iJtxIDl2gT67LbugJy8R3Uj-Efx-cLw][tpath=/3000M][fd=5][offset=0][size=1835008]
2021-03-07T17:02:16.764Z [INF]       curl.cpp:UploadMultipartPostRequest(3770): [tpath=/3000M][start=0][size=1835008][part=1]
2021-03-07T17:02:16.764Z [INF]       curl.cpp:UploadMultipartPostSetup(3711): [tpath=/3000M][start=0][size=1835008][part=1]
[...]
2021-03-07T17:02:16.772Z [CURL DBG] > PUT /mybucket/3000M?partNumber=1&uploadId=uc101yNzruXeD6pgLfcZWykX6O3iJtxIDl2gT67LbugJy8R3Uj-Efx-cLw HTTP/1.1
2021-03-07T17:02:16.772Z [CURL DBG] > User-Agent: s3fs/1.89 (commit hash 8c58ba8; OpenSSL)
2021-03-07T17:02:16.772Z [CURL DBG] > Authorization: AWS4-HMAC-SHA256 Credential=NHTL282FGST9B40156H/20210305/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=8d4bc64b177542ecf790f1e189c92a8c80c6d57067126cde3581c9741e5248fc
2021-03-07T17:02:16.772Z [CURL DBG] > host: mys3service:8080
2021-03-07T17:02:16.773Z [CURL DBG] > x-amz-content-sha256: 2317cfc4f337b22d9b4be6058dbf96ddef8a1a1c8a9567bb6439f57d581d2b5b
2021-03-07T17:02:16.773Z [CURL DBG] > x-amz-date: 20210305T170216Z
2021-03-07T17:02:16.773Z [CURL DBG] > Content-Length: 1835008

    2. Second uploaded part has the right size 100MB as specified with -o multipart_size=100

      2021-03-07T17:02:17.221Z [DBG] fdcache_entity.cpp:Write(1439): [path=/3000M][fd=5][offset=106627072][size=65536]
2021-03-07T17:02:17.221Z [INF]       curl.cpp:MultipartUploadRequest(4034): [upload_id=uc101yNzruXeD6pgLfcZWykX6O3iJtxIDl2gT67LbugJy8R3Uj-Efx-cLw][tpath=/3000M][fd=5][offset=1835008][size=104857600]
2021-03-07T17:02:17.221Z [INF]       curl.cpp:UploadMultipartPostRequest(3770): [tpath=/3000M][start=1835008][size=104857600][part=2]
2021-03-07T17:02:17.221Z [INF]       curl.cpp:UploadMultipartPostSetup(3711): [tpath=/3000M][start=1835008][size=104857600][part=2]
[...]
2021-03-07T17:02:17.669Z [CURL DBG] > PUT /mybucket/3000M?partNumber=2&uploadId=uc101yNzruXeD6pgLfcZWykX6O3iJtxIDl2gT67LbugJy8R3Uj-Efx-cLw HTTP/1.1
2021-03-07T17:02:17.669Z [CURL DBG] > User-Agent: s3fs/1.89 (commit hash 8c58ba8; OpenSSL)
2021-03-07T17:02:17.669Z [CURL DBG] > Authorization: AWS4-HMAC-SHA256 Credential=NHTL282ERGST9B40156H/20210305/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=fb9710536ab62c85e37b890bb2119b025bdbd308127534fe22136819794cc857
2021-03-07T17:02:17.669Z [CURL DBG] > host: mys3service:8080
2021-03-07T17:02:17.669Z [CURL DBG] > x-amz-content-sha256: f10dca51409a8aef099df385de370ce4ac23704ea8b0df7af83543d369bbdb18
2021-03-07T17:02:17.669Z [CURL DBG] > x-amz-date: 20210305T170217Z
2021-03-07T17:02:17.669Z [CURL DBG] > Content-Length: 104857600
<!-- gh-comment-id:792543256 --> @CarstenGrohmann commented on GitHub (Mar 8, 2021): #### Steps to reproduce: 1. Fill `/tmp` except 1..2 MB ``` $ df -k /tmp/ Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda3 3997376 3769464 1816 100% /tmp ``` 2. Start s3fs ``` $ ./s3fs mybucket /grid/cfm2 -o url=http://mys3service:8080 -o use_path_request_style,nomultipart,notsup_compat_dir,enable_noobj_cache,multipart_size=100,curldbg,dbglevel=debug -d -d -f 2021-03-07T16:55:40.563Z [CRT] s3fs_logger.cpp:LowSetLogLevel(219): change debug level from [CRT] to [DBG] 2021-03-07T16:55:40.563Z [INF] s3fs.cpp:set_mountpoint_attribute(4020): PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755) 2021-03-07T16:55:40.563Z [DBG] curl.cpp:InitMimeType(408): Try to load mime types from /etc/mime.types file. 2021-03-07T16:55:40.563Z [DBG] curl.cpp:InitMimeType(413): The old mime types are cleared to load new mime types. 2021-03-07T16:55:40.564Z [INF] curl.cpp:InitMimeType(436): Loaded mime information from /etc/mime.types 2021-03-07T16:55:40.565Z [INF] fdcache_stat.cpp:CheckCacheFileStatTopDir(79): The path to cache top dir is empty, thus not need to check permission. FUSE library version: 2.9.2 nullpath_ok: 0 nopath: 0 utime_omit_ok: 0 unique: 1, opcode: INIT (26), nodeid: 0, insize: 56, pid: 0 INIT: 7.22 flags=0x0000f7fb max_readahead=0x00020000 2021-03-07T16:55:40.566Z [INF] s3fs.cpp:s3fs_init(3331): init v1.89(commit:8c58ba8) with OpenSSL ``` 3. Fill copy a file to S3 ``` $ cp -p 3000M /grid/cfm2/ ``` 4. Check output: 1. First uploaded part is just 1835008 bytes ``` 2021-03-07T17:02:16.764Z [DBG] curl_handlerpool.cpp:ReturnHandler(103): Return handler to pool 2021-03-07T17:02:16.764Z [INF] fdcache_entity.cpp:NoCacheLoadAndPost(962): [path=/3000M][fd=5][offset=0][size=1835008] 2021-03-07T17:02:16.764Z [INF] curl.cpp:MultipartUploadRequest(4034): [upload_id=uc101yNzruXeD6pgLfcZWykX6O3iJtxIDl2gT67LbugJy8R3Uj-Efx-cLw][tpath=/3000M][fd=5][offset=0][size=1835008] 2021-03-07T17:02:16.764Z [INF] curl.cpp:UploadMultipartPostRequest(3770): [tpath=/3000M][start=0][size=1835008][part=1] 2021-03-07T17:02:16.764Z [INF] curl.cpp:UploadMultipartPostSetup(3711): [tpath=/3000M][start=0][size=1835008][part=1] [...] 2021-03-07T17:02:16.772Z [CURL DBG] > PUT /mybucket/3000M?partNumber=1&uploadId=uc101yNzruXeD6pgLfcZWykX6O3iJtxIDl2gT67LbugJy8R3Uj-Efx-cLw HTTP/1.1 2021-03-07T17:02:16.772Z [CURL DBG] > User-Agent: s3fs/1.89 (commit hash 8c58ba8; OpenSSL) 2021-03-07T17:02:16.772Z [CURL DBG] > Authorization: AWS4-HMAC-SHA256 Credential=NHTL282FGST9B40156H/20210305/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=8d4bc64b177542ecf790f1e189c92a8c80c6d57067126cde3581c9741e5248fc 2021-03-07T17:02:16.772Z [CURL DBG] > host: mys3service:8080 2021-03-07T17:02:16.773Z [CURL DBG] > x-amz-content-sha256: 2317cfc4f337b22d9b4be6058dbf96ddef8a1a1c8a9567bb6439f57d581d2b5b 2021-03-07T17:02:16.773Z [CURL DBG] > x-amz-date: 20210305T170216Z 2021-03-07T17:02:16.773Z [CURL DBG] > Content-Length: 1835008 ``` 1. Second uploaded part has the right size 100MB as specified with `-o multipart_size=100` ``` 2021-03-07T17:02:17.221Z [DBG] fdcache_entity.cpp:Write(1439): [path=/3000M][fd=5][offset=106627072][size=65536] 2021-03-07T17:02:17.221Z [INF] curl.cpp:MultipartUploadRequest(4034): [upload_id=uc101yNzruXeD6pgLfcZWykX6O3iJtxIDl2gT67LbugJy8R3Uj-Efx-cLw][tpath=/3000M][fd=5][offset=1835008][size=104857600] 2021-03-07T17:02:17.221Z [INF] curl.cpp:UploadMultipartPostRequest(3770): [tpath=/3000M][start=1835008][size=104857600][part=2] 2021-03-07T17:02:17.221Z [INF] curl.cpp:UploadMultipartPostSetup(3711): [tpath=/3000M][start=1835008][size=104857600][part=2] [...] 2021-03-07T17:02:17.669Z [CURL DBG] > PUT /mybucket/3000M?partNumber=2&uploadId=uc101yNzruXeD6pgLfcZWykX6O3iJtxIDl2gT67LbugJy8R3Uj-Efx-cLw HTTP/1.1 2021-03-07T17:02:17.669Z [CURL DBG] > User-Agent: s3fs/1.89 (commit hash 8c58ba8; OpenSSL) 2021-03-07T17:02:17.669Z [CURL DBG] > Authorization: AWS4-HMAC-SHA256 Credential=NHTL282ERGST9B40156H/20210305/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=fb9710536ab62c85e37b890bb2119b025bdbd308127534fe22136819794cc857 2021-03-07T17:02:17.669Z [CURL DBG] > host: mys3service:8080 2021-03-07T17:02:17.669Z [CURL DBG] > x-amz-content-sha256: f10dca51409a8aef099df385de370ce4ac23704ea8b0df7af83543d369bbdb18 2021-03-07T17:02:17.669Z [CURL DBG] > x-amz-date: 20210305T170217Z 2021-03-07T17:02:17.669Z [CURL DBG] > Content-Length: 104857600 ```
kerem commented 2026-03-04 01:49:12 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Apr 23, 2021):

@CarstenGrohmann Is this issue still valid?

<!-- gh-comment-id:825674901 --> @gaul commented on GitHub (Apr 23, 2021): @CarstenGrohmann Is this issue still valid?
kerem commented 2026-03-04 01:49:12 +03:00
Author
Owner
Copy link

@CarstenGrohmann commented on GitHub (Apr 24, 2021):

I would contribute three changes to address this issue:

  1. a fix to address this issue similar 739f499 (not started yet)
  2. a update to the manpage to describe the local storage consumption (started)
  3. a change to the location for temporary files, currently is /tmp hardcoded (not started yet)
<!-- gh-comment-id:826060458 --> @CarstenGrohmann commented on GitHub (Apr 24, 2021): I would contribute three changes to address this issue: 1. a fix to address this issue similar 739f499 (not started yet) 2. a update to the manpage to describe the local storage consumption (started) 3. a change to the location for temporary files, currently is /tmp hardcoded (not started yet)
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
v1.97
v1.96
v1.95
v1.94
v1.93
v1.92
v1.91
v1.90
v1.89
v1.88
v1.87
v1.86
v1.85
v1.84
v1.83
v1.82
v1.81
v1.80
v1.79
v1.78
Pre-v1.78
v1.77
v1.76
v1.75
v1.74
Labels
Clear labels   
bug

   
bug

   
dataloss

   
duplicate

   
enhancement

   
feature request

   
help wanted

   
invalid

   
need info

   
performance

   
pull-request

Mirrored from GitHub Pull Request

  
question

   
question

   
testing
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#834
No description provided.