[PR #1538] [MERGED] Add warning for failing metadata updates of large files #2034

New issue

Closed

opened 2026-03-04 02:03:21 +03:00 by kerem · 0 comments

kerem commented

2026-03-04 02:03:21 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/s3fs-fuse/s3fs-fuse/pull/1538
Author: @CarstenGrohmann
Created: 1/27/2021
Status: ✅ Merged
Merged: 1/28/2021
Merged by: @gaul

Base: master ← Head: multipart_warning

📝 Commits (1)

ae66b63 Add warning for failing metadata updates of large files

📊 Changes

1 file changed (+1 additions, -0 deletions)

View changed files

📝 src/s3fs.cpp (+1 -0)

📄 Description

Metadata updates fail for large files if "-o nocopyapi" or "-o nomultipart" is set.

This change is related to #1528

Example

# ll bigfile
-rw-r--r-- 1 tcpdump tcpdump 7486832640 Jan 27 11:20 bigfile

# cp -p bigfile /grid/
cp: failed to preserve ownership for "/grid/bigfile": Input/output error

Old debug output w/o warning

unique: 114254, opcode: SETATTR (4), nodeid: 2, insize: 128, pid: 26341
chown /bigfile 72 72
[INF] s3fs.cpp:s3fs_chown(1741): [path=/bigfile][uid=72][gid=72]
[...]
[DBG] s3fs.cpp:get_object_attribute(362): [path=/bigfile]
[DBG] cache.cpp:GetStat(293): stat cache hit [path=/bigfile][time=1128915.515422614][hit count=3]
[DBG] fdcache.cpp:Close(592): [ent->file=/bigfile][ent->fd=7]
[DBG] fdcache_entity.cpp:Close(200): [path=/bigfile][fd=7][refcnt=1]
   unique: 114254, error: -5 (Input/output error), outsize: 16

New debug output w/ warning

unique: 114249, opcode: SETATTR (4), nodeid: 2, insize: 128, pid: 28319
chown /bigfile 72 72
[INF] s3fs.cpp:s3fs_chown(1742): [path=/bigfile][uid=72][gid=72]
[...]
[DBG] s3fs.cpp:get_object_attribute(362): [path=/bigfile]
[DBG] cache.cpp:GetStat(293): stat cache hit [path=/bigfile][time=1130681.479403969][hit count=3]
[WAN] s3fs.cpp:put_headers(729): Metadata update failed because the file is larger than 5GB and the options nocopyapi or nomultipart are set: [path=/bigfile]
[DBG] fdcache.cpp:Close(592): [ent->file=/bigfile][ent->fd=7]
[DBG] fdcache_entity.cpp:Close(200): [path=/bigfile][fd=7][refcnt=1]
   unique: 114249, error: -5 (Input/output error), outsize: 16
unique: 114250, opcode: FLUSH (25), nodeid: 2, insize: 64, pid: 28319

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/s3fs-fuse/s3fs-fuse/pull/1538 **Author:** [@CarstenGrohmann](https://github.com/CarstenGrohmann) **Created:** 1/27/2021 **Status:** ✅ Merged **Merged:** 1/28/2021 **Merged by:** [@gaul](https://github.com/gaul) **Base:** `master` ← **Head:** `multipart_warning` --- ### 📝 Commits (1) - [`ae66b63`](https://github.com/s3fs-fuse/s3fs-fuse/commit/ae66b63e3b430061af1155bcbffe0152d7050780) Add warning for failing metadata updates of large files ### 📊 Changes **1 file changed** (+1 additions, -0 deletions) <details> <summary>View changed files</summary> 📝 `src/s3fs.cpp` (+1 -0) </details> ### 📄 Description Metadata updates fail for large files if "-o nocopyapi" or "-o nomultipart" is set. This change is related to #1528 ### Example ``` # ll bigfile -rw-r--r-- 1 tcpdump tcpdump 7486832640 Jan 27 11:20 bigfile # cp -p bigfile /grid/ cp: failed to preserve ownership for "/grid/bigfile": Input/output error ``` ### Old debug output w/o warning ``` unique: 114254, opcode: SETATTR (4), nodeid: 2, insize: 128, pid: 26341 chown /bigfile 72 72 [INF] s3fs.cpp:s3fs_chown(1741): [path=/bigfile][uid=72][gid=72] [...] [DBG] s3fs.cpp:get_object_attribute(362): [path=/bigfile] [DBG] cache.cpp:GetStat(293): stat cache hit [path=/bigfile][time=1128915.515422614][hit count=3] [DBG] fdcache.cpp:Close(592): [ent->file=/bigfile][ent->fd=7] [DBG] fdcache_entity.cpp:Close(200): [path=/bigfile][fd=7][refcnt=1] unique: 114254, error: -5 (Input/output error), outsize: 16 ``` ### New debug output w/ warning ``` unique: 114249, opcode: SETATTR (4), nodeid: 2, insize: 128, pid: 28319 chown /bigfile 72 72 [INF] s3fs.cpp:s3fs_chown(1742): [path=/bigfile][uid=72][gid=72] [...] [DBG] s3fs.cpp:get_object_attribute(362): [path=/bigfile] [DBG] cache.cpp:GetStat(293): stat cache hit [path=/bigfile][time=1130681.479403969][hit count=3] [WAN] s3fs.cpp:put_headers(729): Metadata update failed because the file is larger than 5GB and the options nocopyapi or nomultipart are set: [path=/bigfile] [DBG] fdcache.cpp:Close(592): [ent->file=/bigfile][ent->fd=7] [DBG] fdcache_entity.cpp:Close(200): [path=/bigfile][fd=7][refcnt=1] unique: 114249, error: -5 (Input/output error), outsize: 16 unique: 114250, opcode: FLUSH (25), nodeid: 2, insize: 64, pid: 28319 ``` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>