starred/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2026-04-25 13:26:00 +03:00
Code Issues 305 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #1086] Unable to update file owner, mode and access-modification time #591

New issue
Closed
opened 2026-03-04 01:47:01 +03:00 by kerem · 2 comments
kerem commented 2026-03-04 01:47:01 +03:00
Owner
Copy link

Originally created by @maheshw-vtas on GitHub (Jul 11, 2019).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1086

Additional Information

Version of s3fs being used (s3fs --version)

1.85

Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse)

2.9.3

Kernel information (uname -r)

4.4.73-5-default

GNU/Linux Distribution, if applicable (cat /etc/os-release)

NAME="SLES"
VERSION="12-SP3"
VERSION_ID="12.3"
PRETTY_NAME="SUSE Linux Enterprise Server 12 SP3"
ID="sles"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sles:12:sp3"

s3fs command line used, if applicable

/usr/bin/s3fs dnd-maheshw5 /mnt -o passwd_file=/etc/passwd-s3fs -o dbglevel="info" -o curldbg

/etc/fstab entry, if applicable

NA

s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)

if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages
s3fs_put_failure.txt

Details about issue

When I try to write large number of small files (size ranging from 50KB to 2MB) on s3fs mount point, for some of the files (1 to 10 files out of approx. 30000 files) I see failures when fchown() or/and fchmod() or/and utimes() functionas are called. I create the file, write data into it and close it.
After that when file owner, file mode, access and modification time are changed using the functions mentioned above, those functions fail with 'Input/output error'. Corresponding PUT failures (HTTP/1.1 404 Not Found) can also be seen in s3fs logs.

If I selectively write the single file (i.e. not as part large number of files) in s3fs mounted bucket and change it's owner, mode, access/modification time then I don't see this problem. Therefore this doesn't seem an issue with a particular file/s. Also when the test was repeated, the files for which PUT failed were different.

Is this expected behavior with large number of small files? Is there any workaround?

Note: For security reasons, I replaced some part in Credential string with XXXXXXXXXXXXXXXXXXXX

s3fs log snippet:

2019-07-05T09:30:18.223378-05:00 ellacl02vm39 s3fs[26254]: [path=/restore_full2/0/000/001/337.txt]
2019-07-05T09:30:18.230232-05:00 ellacl02vm39 s3fs[26254]: [path=/restore_full2/0/000/001/337.txt]
2019-07-05T09:30:18.230411-05:00 ellacl02vm39 s3fs[26254]: [path=/restore_full2/0/000/001/337.txt][uid=0][gid=0]
2019-07-05T09:30:18.230643-05:00 ellacl02vm39 s3fs[26254]:     [path=/restore_full2/0/000/001/337.txt]
2019-07-05T09:30:18.230859-05:00 ellacl02vm39 s3fs[26254]:       [tpath=/restore_full2/0/000/001/337.txt]
2019-07-05T09:30:18.231189-05:00 ellacl02vm39 s3fs[26254]:       URL is https://s3.amazonaws.com/dnd-maheshw5/restore_full2/0/000/001/337.txt
2019-07-05T09:30:18.231410-05:00 ellacl02vm39 s3fs[26254]:       URL changed is https://dnd-maheshw5.s3.amazonaws.com/restore_full2/0/000/001/337.txt
2019-07-05T09:30:18.231621-05:00 ellacl02vm39 s3fs[26254]:       computing signature [PUT] [/restore_full2/0/000/001/337.txt] [] []
2019-07-05T09:30:18.231873-05:00 ellacl02vm39 s3fs[26254]:       url is https://s3.amazonaws.com
2019-07-05T09:30:18.232112-05:00 ellacl02vm39 s3fs[26254]:       copying... [path=/restore_full2/0/000/001/337.txt]
2019-07-05T09:30:18.232404-05:00 ellacl02vm39 s3fs[26254]: * Found bundle for host dnd-maheshw5.s3.amazonaws.com: 0x7f5198000e30
2019-07-05T09:30:18.232728-05:00 ellacl02vm39 s3fs[26254]: * Re-using existing connection! (#282) with host dnd-maheshw5.s3.amazonaws.com
2019-07-05T09:30:18.232991-05:00 ellacl02vm39 s3fs[26254]: * Connected to dnd-maheshw5.s3.amazonaws.com (52.216.145.203) port 443 (#282)
2019-07-05T09:30:18.233268-05:00 ellacl02vm39 s3fs[26254]: * SSLv2, Unknown (23):
2019-07-05T09:30:18.233613-05:00 ellacl02vm39 s3fs[26254]: > PUT /restore_full2/0/000/001/337.txt HTTP/1.1
2019-07-05T09:30:18.233873-05:00 ellacl02vm39 s3fs[26254]: > User-Agent: s3fs/1.85 (commit hash unknown; OpenSSL)
2019-07-05T09:30:18.234134-05:00 ellacl02vm39 s3fs[26254]: > Accept: */*
2019-07-05T09:30:18.234403-05:00 ellacl02vm39 s3fs[26254]: > Authorization: AWS4-HMAC-SHA256 Credential=XXXXXXXXXXXXXXXXXXXX/20190705/us-east-1/s3/aws4_request, SignedHeaders=content-type;host;x-amz-acl;x-amz-content-sha256;x-amz-copy-source;x-amz-date;x-amz-meta-ctime;x-amz-meta-gid;x-amz-meta-mode;x-amz-meta-mtime;x-amz-meta-uid;x-amz-metadata-directive, Signature=5b8b94090b46f84b7eca78221334050470760ecff0bc1cd38d24f090d344f175
2019-07-05T09:30:18.234805-05:00 ellacl02vm39 s3fs[26254]: > Content-Type: text/plain
2019-07-05T09:30:18.235061-05:00 ellacl02vm39 s3fs[26254]: > host: dnd-maheshw5.s3.amazonaws.com
2019-07-05T09:30:18.235334-05:00 ellacl02vm39 s3fs[26254]: > x-amz-acl: private
2019-07-05T09:30:18.235726-05:00 ellacl02vm39 s3fs[26254]: > x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2019-07-05T09:30:18.236050-05:00 ellacl02vm39 s3fs[26254]: > x-amz-copy-source: /dnd-maheshw5/restore_full2/0/000/001/337.txt
2019-07-05T09:30:18.236370-05:00 ellacl02vm39 s3fs[26254]: > x-amz-date: 20190705T143018Z
2019-07-05T09:30:18.236669-05:00 ellacl02vm39 s3fs[26254]: > x-amz-meta-ctime: 1562337018
2019-07-05T09:30:18.237005-05:00 ellacl02vm39 s3fs[26254]: > x-amz-meta-gid: 0
2019-07-05T09:30:18.237258-05:00 ellacl02vm39 s3fs[26254]: > x-amz-meta-mode: 33188
2019-07-05T09:30:18.237512-05:00 ellacl02vm39 s3fs[26254]: > x-amz-meta-mtime: 1562337017
2019-07-05T09:30:18.237779-05:00 ellacl02vm39 s3fs[26254]: > x-amz-meta-uid: 0
2019-07-05T09:30:18.238124-05:00 ellacl02vm39 s3fs[26254]: > x-amz-metadata-directive: REPLACE
2019-07-05T09:30:18.238376-05:00 ellacl02vm39 s3fs[26254]: > Content-Length: 0
2019-07-05T09:30:18.238630-05:00 ellacl02vm39 s3fs[26254]: >
2019-07-05T09:30:18.303515-05:00 ellacl02vm39 s3fs[26254]: * SSLv2, Unknown (23):
2019-07-05T09:30:18.304001-05:00 ellacl02vm39 s3fs[26254]: < HTTP/1.1 404 Not Found
2019-07-05T09:30:18.304274-05:00 ellacl02vm39 s3fs[26254]: < x-amz-request-id: E990D2A5B0EF9EBB
2019-07-05T09:30:18.304725-05:00 ellacl02vm39 s3fs[26254]: < x-amz-id-2: +HjHgebqSRgBlbTe6g+9r62/UWaoZi5Qazq4ThyJw411j49YV3MtTCU0CFkNxU/kXOrABrZhT9Y=
2019-07-05T09:30:18.305108-05:00 ellacl02vm39 s3fs[26254]: < Content-Type: application/xml
2019-07-05T09:30:18.305397-05:00 ellacl02vm39 s3fs[26254]: < Transfer-Encoding: chunked
2019-07-05T09:30:18.305657-05:00 ellacl02vm39 s3fs[26254]: < Date: Fri, 05 Jul 2019 14:30:17 GMT
2019-07-05T09:30:18.305913-05:00 ellacl02vm39 s3fs[26254]: * Server AmazonS3 is not blacklisted
2019-07-05T09:30:18.306307-05:00 ellacl02vm39 s3fs[26254]: < Server: AmazonS3
2019-07-05T09:30:18.306566-05:00 ellacl02vm39 s3fs[26254]: * HTTP error before end of send, stop sending
2019-07-05T09:30:18.306831-05:00 ellacl02vm39 s3fs[26254]: <
Originally created by @maheshw-vtas on GitHub (Jul 11, 2019). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1086 ### Additional Information #### Version of s3fs being used (s3fs --version) 1.85 #### Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse) 2.9.3 #### Kernel information (uname -r) 4.4.73-5-default #### GNU/Linux Distribution, if applicable (cat /etc/os-release) NAME="SLES" VERSION="12-SP3" VERSION_ID="12.3" PRETTY_NAME="SUSE Linux Enterprise Server 12 SP3" ID="sles" ANSI_COLOR="0;32" CPE_NAME="cpe:/o:suse:sles:12:sp3" #### s3fs command line used, if applicable /usr/bin/s3fs dnd-maheshw5 /mnt -o passwd_file=/etc/passwd-s3fs -o dbglevel="info" -o curldbg #### /etc/fstab entry, if applicable NA #### s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs) _if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages_ [s3fs_put_failure.txt](https://github.com/s3fs-fuse/s3fs-fuse/files/3381541/s3fs_put_failure.txt) ### Details about issue When I try to write large number of small files (size ranging from 50KB to 2MB) on s3fs mount point, for some of the files (1 to 10 files out of approx. 30000 files) I see failures when fchown() or/and fchmod() or/and utimes() functionas are called. I create the file, write data into it and close it. After that when file owner, file mode, access and modification time are changed using the functions mentioned above, those functions fail with 'Input/output error'. Corresponding PUT failures (HTTP/1.1 404 Not Found) can also be seen in s3fs logs. If I selectively write the single file (i.e. not as part large number of files) in s3fs mounted bucket and change it's owner, mode, access/modification time then I don't see this problem. Therefore this doesn't seem an issue with a particular file/s. Also when the test was repeated, the files for which PUT failed were different. Is this expected behavior with large number of small files? Is there any workaround? Note: For security reasons, I replaced some part in Credential string with XXXXXXXXXXXXXXXXXXXX s3fs log snippet: ``` 2019-07-05T09:30:18.223378-05:00 ellacl02vm39 s3fs[26254]: [path=/restore_full2/0/000/001/337.txt] 2019-07-05T09:30:18.230232-05:00 ellacl02vm39 s3fs[26254]: [path=/restore_full2/0/000/001/337.txt] 2019-07-05T09:30:18.230411-05:00 ellacl02vm39 s3fs[26254]: [path=/restore_full2/0/000/001/337.txt][uid=0][gid=0] 2019-07-05T09:30:18.230643-05:00 ellacl02vm39 s3fs[26254]: [path=/restore_full2/0/000/001/337.txt] 2019-07-05T09:30:18.230859-05:00 ellacl02vm39 s3fs[26254]: [tpath=/restore_full2/0/000/001/337.txt] 2019-07-05T09:30:18.231189-05:00 ellacl02vm39 s3fs[26254]: URL is https://s3.amazonaws.com/dnd-maheshw5/restore_full2/0/000/001/337.txt 2019-07-05T09:30:18.231410-05:00 ellacl02vm39 s3fs[26254]: URL changed is https://dnd-maheshw5.s3.amazonaws.com/restore_full2/0/000/001/337.txt 2019-07-05T09:30:18.231621-05:00 ellacl02vm39 s3fs[26254]: computing signature [PUT] [/restore_full2/0/000/001/337.txt] [] [] 2019-07-05T09:30:18.231873-05:00 ellacl02vm39 s3fs[26254]: url is https://s3.amazonaws.com 2019-07-05T09:30:18.232112-05:00 ellacl02vm39 s3fs[26254]: copying... [path=/restore_full2/0/000/001/337.txt] 2019-07-05T09:30:18.232404-05:00 ellacl02vm39 s3fs[26254]: * Found bundle for host dnd-maheshw5.s3.amazonaws.com: 0x7f5198000e30 2019-07-05T09:30:18.232728-05:00 ellacl02vm39 s3fs[26254]: * Re-using existing connection! (#282) with host dnd-maheshw5.s3.amazonaws.com 2019-07-05T09:30:18.232991-05:00 ellacl02vm39 s3fs[26254]: * Connected to dnd-maheshw5.s3.amazonaws.com (52.216.145.203) port 443 (#282) 2019-07-05T09:30:18.233268-05:00 ellacl02vm39 s3fs[26254]: * SSLv2, Unknown (23): 2019-07-05T09:30:18.233613-05:00 ellacl02vm39 s3fs[26254]: > PUT /restore_full2/0/000/001/337.txt HTTP/1.1 2019-07-05T09:30:18.233873-05:00 ellacl02vm39 s3fs[26254]: > User-Agent: s3fs/1.85 (commit hash unknown; OpenSSL) 2019-07-05T09:30:18.234134-05:00 ellacl02vm39 s3fs[26254]: > Accept: */* 2019-07-05T09:30:18.234403-05:00 ellacl02vm39 s3fs[26254]: > Authorization: AWS4-HMAC-SHA256 Credential=XXXXXXXXXXXXXXXXXXXX/20190705/us-east-1/s3/aws4_request, SignedHeaders=content-type;host;x-amz-acl;x-amz-content-sha256;x-amz-copy-source;x-amz-date;x-amz-meta-ctime;x-amz-meta-gid;x-amz-meta-mode;x-amz-meta-mtime;x-amz-meta-uid;x-amz-metadata-directive, Signature=5b8b94090b46f84b7eca78221334050470760ecff0bc1cd38d24f090d344f175 2019-07-05T09:30:18.234805-05:00 ellacl02vm39 s3fs[26254]: > Content-Type: text/plain 2019-07-05T09:30:18.235061-05:00 ellacl02vm39 s3fs[26254]: > host: dnd-maheshw5.s3.amazonaws.com 2019-07-05T09:30:18.235334-05:00 ellacl02vm39 s3fs[26254]: > x-amz-acl: private 2019-07-05T09:30:18.235726-05:00 ellacl02vm39 s3fs[26254]: > x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 2019-07-05T09:30:18.236050-05:00 ellacl02vm39 s3fs[26254]: > x-amz-copy-source: /dnd-maheshw5/restore_full2/0/000/001/337.txt 2019-07-05T09:30:18.236370-05:00 ellacl02vm39 s3fs[26254]: > x-amz-date: 20190705T143018Z 2019-07-05T09:30:18.236669-05:00 ellacl02vm39 s3fs[26254]: > x-amz-meta-ctime: 1562337018 2019-07-05T09:30:18.237005-05:00 ellacl02vm39 s3fs[26254]: > x-amz-meta-gid: 0 2019-07-05T09:30:18.237258-05:00 ellacl02vm39 s3fs[26254]: > x-amz-meta-mode: 33188 2019-07-05T09:30:18.237512-05:00 ellacl02vm39 s3fs[26254]: > x-amz-meta-mtime: 1562337017 2019-07-05T09:30:18.237779-05:00 ellacl02vm39 s3fs[26254]: > x-amz-meta-uid: 0 2019-07-05T09:30:18.238124-05:00 ellacl02vm39 s3fs[26254]: > x-amz-metadata-directive: REPLACE 2019-07-05T09:30:18.238376-05:00 ellacl02vm39 s3fs[26254]: > Content-Length: 0 2019-07-05T09:30:18.238630-05:00 ellacl02vm39 s3fs[26254]: > 2019-07-05T09:30:18.303515-05:00 ellacl02vm39 s3fs[26254]: * SSLv2, Unknown (23): 2019-07-05T09:30:18.304001-05:00 ellacl02vm39 s3fs[26254]: < HTTP/1.1 404 Not Found 2019-07-05T09:30:18.304274-05:00 ellacl02vm39 s3fs[26254]: < x-amz-request-id: E990D2A5B0EF9EBB 2019-07-05T09:30:18.304725-05:00 ellacl02vm39 s3fs[26254]: < x-amz-id-2: +HjHgebqSRgBlbTe6g+9r62/UWaoZi5Qazq4ThyJw411j49YV3MtTCU0CFkNxU/kXOrABrZhT9Y= 2019-07-05T09:30:18.305108-05:00 ellacl02vm39 s3fs[26254]: < Content-Type: application/xml 2019-07-05T09:30:18.305397-05:00 ellacl02vm39 s3fs[26254]: < Transfer-Encoding: chunked 2019-07-05T09:30:18.305657-05:00 ellacl02vm39 s3fs[26254]: < Date: Fri, 05 Jul 2019 14:30:17 GMT 2019-07-05T09:30:18.305913-05:00 ellacl02vm39 s3fs[26254]: * Server AmazonS3 is not blacklisted 2019-07-05T09:30:18.306307-05:00 ellacl02vm39 s3fs[26254]: < Server: AmazonS3 2019-07-05T09:30:18.306566-05:00 ellacl02vm39 s3fs[26254]: * HTTP error before end of send, stop sending 2019-07-05T09:30:18.306831-05:00 ellacl02vm39 s3fs[26254]: < ```
kerem closed this issue 2026-03-04 01:47:01 +03:00
kerem commented 2026-03-04 01:47:02 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Jul 19, 2019):

It is possible you have encountered eventual consistency, see these limitations:

https://github.com/s3fs-fuse/s3fs-fuse#limitations

That said, we have tests for the behavior you describe and do not see these errors. Do you have a reproduceable test case that you can share?

<!-- gh-comment-id:513046976 --> @gaul commented on GitHub (Jul 19, 2019): It is possible you have encountered eventual consistency, see these limitations: https://github.com/s3fs-fuse/s3fs-fuse#limitations That said, we have tests for the behavior you describe and do not see these errors. Do you have a reproduceable test case that you can share?
kerem commented 2026-03-04 01:47:02 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Feb 3, 2020):

Closing due to inactivity. Please reopen if symptoms persist.

<!-- gh-comment-id:581288879 --> @gaul commented on GitHub (Feb 3, 2020): Closing due to inactivity. Please reopen if symptoms persist.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
v1.97
v1.96
v1.95
v1.94
v1.93
v1.92
v1.91
v1.90
v1.89
v1.88
v1.87
v1.86
v1.85
v1.84
v1.83
v1.82
v1.81
v1.80
v1.79
v1.78
Pre-v1.78
v1.77
v1.76
v1.75
v1.74
Labels
Clear labels   
bug

   
bug

   
dataloss

   
duplicate

   
enhancement

   
feature request

   
help wanted

   
invalid

   
need info

   
performance

   
pull-request

Mirrored from GitHub Pull Request

  
question

   
question

   
testing
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#591
No description provided.