starred/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2026-04-25 05:16:00 +03:00
Code Issues 305 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #1022] File been incorrect truncated when writing via nfs which mounted on s3fs-fuse #561

New issue
Closed
opened 2026-03-04 01:46:45 +03:00 by kerem · 3 comments
kerem commented 2026-03-04 01:46:45 +03:00
Owner
Copy link

Originally created by @dulingling200 on GitHub (Apr 29, 2019).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1022

Additional Information

I use s3fs-fuse to mount object-storage bucket，and share the mounted directory for others by NFS, for example nfs-ganesha or nfs-utils, when I write a big file (for exmple 100M) to NFS, then the file will be uplouded to S3 ,but the file will be incorrect truncated by s3fs-fuse ,so the datas of file which uplouded to s3 will be wrong.

Version of s3fs being used (s3fs --version)

_example: 1.85

Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse)

example: 2.9.4

Kernel information (uname -r)

_command result: 3.10.0-957.5.1.el7.x86_64

GNU/Linux Distribution, if applicable (cat /etc/os-release)

_command result: cat /etc/os-release

cat /etc/os-release

NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

s3fs command line used, if applicable

/etc/fstab entry, if applicable

s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)

if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages

Details about issue

I use s3fs-fuse to mount object-storage bucket，and share the mounted directory for others by NFS, for example nfs-ganesha or nfs-utils, when I write a big file (for exmple 100M) to NFS, then the file will be uplouded to S3 ,but the file will be incorrect truncated by s3fs-fuse ,so the datas of file which uplouded to s3 will be wrong.

Some log that uploud a file. The size of file is 179550635，but md5sum on the file, the value is wrong.

Apr 28 00:12:10 test-gw-server s3fs[5750]: path:/Testfile. size:0. pagelist.Size():179550635. size_orgmeta:179550635
Apr 28 00:12:10 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0
Apr 28 00:12:10 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0
Apr 28 00:12:10 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:0. pagelist.Size():0. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:0. pagelist.Size():0. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():131072. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():262144. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():393216. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():524288. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():655360. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():786432. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():917504. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():1048576. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():2228224. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():2359296. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():2490368. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():2621440. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():2752512. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():2883584. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3014656. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3145728. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3276800. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3407872. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3538944. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3670016. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:0. pagelist.Size():3801088. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3932160. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():4063232. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():4194304. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():4194304. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():4194304. size_orgmeta:4194304
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():84934656. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():85065728. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():85196800. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():85327872. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():85458944. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():85590016. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():85721088. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():85852160. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():85983232. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():86114304. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():86245376. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():86376448. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():86507520. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():86638592. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():86769664. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():86900736. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():87031808. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():87162880. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:84934656. pagelist.Size():87293952. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():84934656. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():87425024. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():87556096. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():87687168. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():87818240. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():87949312. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():88080384. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():88211456. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():88342528. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():88473600. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():88604672. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():88735744. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():88866816. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():88997888. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():89128960. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():89260032. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():89391104. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():89522176. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():89653248. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():89784320. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():89915392. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():90046464. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():90177536. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():90308608. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():90439680. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():90570752. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():90701824. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():90832896. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():90963968. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():91095040. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():91226112. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():91357184. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():91488256. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():91619328. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():91750400. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():91881472. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():92012544. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():92143616. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():92274688. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():93454336. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():93585408. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():93716480. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():93847552. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():93978624. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94109696. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94240768. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94371840. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94371840. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94371840. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94371840. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94371840. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94371840. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94371840. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94371840. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94371840. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94502912. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94633984. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94765056. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94896128. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():95027200. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():95158272. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():95289344. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():95420416. size_orgmeta:84934656
Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():95420416. size_orgmeta:84934656
Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():95420416. size_orgmeta:95420416
Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():96600064. size_orgmeta:95420416
Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():96731136. size_orgmeta:95420416
Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():96862208. size_orgmeta:95420416
Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():96993280. size_orgmeta:95420416
Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97124352. size_orgmeta:95420416
Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97255424. size_orgmeta:95420416
Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97386496. size_orgmeta:95420416
Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97517568. size_orgmeta:95420416
Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97517568. size_orgmeta:95420416
Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97517568. size_orgmeta:95420416
Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97517568. size_orgmeta:95420416
Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97517568. size_orgmeta:95420416
Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97517568. size_orgmeta:95420416
Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97517568. size_orgmeta:95420416
Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97517568. size_orgmeta:95420416
Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97517568. size_orgmeta:95420416
Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97648640. size_orgmeta:95420416
Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97779712. size_orgmeta:95420416
Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97910784. size_orgmeta:95420416
Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():98041856. size_orgmeta:95420416
Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():98172928. size_orgmeta:95420416
Apr 28 00:13:18 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():179550635. size_orgmeta:179550635
Apr 28 00:13:18 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():179550635. size_orgmeta:179550635

Originally created by @dulingling200 on GitHub (Apr 29, 2019). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1022 ### Additional Information I use s3fs-fuse to mount object-storage bucket，and share the mounted directory for others by NFS, for example nfs-ganesha or nfs-utils, when I write a big file (for exmple 100M) to NFS, then the file will be uplouded to S3 ,but the file will be incorrect truncated by s3fs-fuse ,so the datas of file which uplouded to s3 will be wrong. #### Version of s3fs being used (s3fs --version) _example: 1.85 #### Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse) _example: 2.9.4_ #### Kernel information (uname -r) _command result: 3.10.0-957.5.1.el7.x86_64 #### GNU/Linux Distribution, if applicable (cat /etc/os-release) _command result: cat /etc/os-release # cat /etc/os-release NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/" CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7" #### s3fs command line used, if applicable ``` ``` #### /etc/fstab entry, if applicable ``` ``` #### s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs) _if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages_ ``` ``` ### Details about issue I use s3fs-fuse to mount object-storage bucket，and share the mounted directory for others by NFS, for example nfs-ganesha or nfs-utils, when I write a big file (for exmple 100M) to NFS, then the file will be uplouded to S3 ,but the file will be incorrect truncated by s3fs-fuse ,so the datas of file which uplouded to s3 will be wrong. Some log that uploud a file. The size of file is 179550635，but md5sum on the file, the value is wrong. Apr 28 00:12:10 test-gw-server s3fs[5750]: path:/Testfile. size:0. pagelist.Size():179550635. size_orgmeta:179550635 Apr 28 00:12:10 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0 Apr 28 00:12:10 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0 Apr 28 00:12:10 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:0. pagelist.Size():0. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:0. pagelist.Size():0. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():131072. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():262144. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():393216. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():524288. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():655360. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():786432. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():917504. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():1048576. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():2228224. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():2359296. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():2490368. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():2621440. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():2752512. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():2883584. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3014656. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3145728. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3276800. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3407872. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3538944. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3670016. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:0. pagelist.Size():3801088. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3932160. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():4063232. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():4194304. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():4194304. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():4194304. size_orgmeta:4194304 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():84934656. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():85065728. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():85196800. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():85327872. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():85458944. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():85590016. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():85721088. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():85852160. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():85983232. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():86114304. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():86245376. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():86376448. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():86507520. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():86638592. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():86769664. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():86900736. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():87031808. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():87162880. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:84934656. pagelist.Size():87293952. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():84934656. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():87425024. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():87556096. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():87687168. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():87818240. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():87949312. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():88080384. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():88211456. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():88342528. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():88473600. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():88604672. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():88735744. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():88866816. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():88997888. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():89128960. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():89260032. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():89391104. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():89522176. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():89653248. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():89784320. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():89915392. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():90046464. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():90177536. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():90308608. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():90439680. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():90570752. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():90701824. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():90832896. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():90963968. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():91095040. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():91226112. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():91357184. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():91488256. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():91619328. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():91750400. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():91881472. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():92012544. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():92143616. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():92274688. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():93454336. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():93585408. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():93716480. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():93847552. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():93978624. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94109696. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94240768. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94371840. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94371840. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94371840. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94371840. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94371840. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94371840. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94371840. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94371840. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94371840. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94502912. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94633984. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94765056. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():94896128. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():95027200. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():95158272. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():95289344. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():95420416. size_orgmeta:84934656 Apr 28 00:12:39 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():95420416. size_orgmeta:84934656 Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():95420416. size_orgmeta:95420416 Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():96600064. size_orgmeta:95420416 Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():96731136. size_orgmeta:95420416 Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():96862208. size_orgmeta:95420416 Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():96993280. size_orgmeta:95420416 Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97124352. size_orgmeta:95420416 Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97255424. size_orgmeta:95420416 Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97386496. size_orgmeta:95420416 Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97517568. size_orgmeta:95420416 Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97517568. size_orgmeta:95420416 Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97517568. size_orgmeta:95420416 Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97517568. size_orgmeta:95420416 Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97517568. size_orgmeta:95420416 Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97517568. size_orgmeta:95420416 Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97517568. size_orgmeta:95420416 Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97517568. size_orgmeta:95420416 Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97517568. size_orgmeta:95420416 Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97648640. size_orgmeta:95420416 Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97779712. size_orgmeta:95420416 Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():97910784. size_orgmeta:95420416 Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():98041856. size_orgmeta:95420416 Apr 28 00:12:45 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():98172928. size_orgmeta:95420416 Apr 28 00:13:18 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():179550635. size_orgmeta:179550635 Apr 28 00:13:18 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():179550635. size_orgmeta:179550635
kerem closed this issue 2026-03-04 01:46:45 +03:00
kerem commented 2026-03-04 01:46:46 +03:00
Author
Owner
Copy link

@dulingling200 commented on GitHub (May 5, 2019):

@gaul @ggtakec
this issue occurs when writing file to s3fs-fuse directory via nfs.

This is my operation steps to reproduce the issue:

  1. share the s3fs-fuse directory (e.g. /mnt/gw) via nfs on my server
  2. mount the shared nfs directory on another directory (e.g. /status)
  3. write a file(>10M) to the s3fs-fuse directory (e.g. /mnt/gw) directly, everything is ok.
  4. write a file(>10M) to the shared nfs directory (e.g. /status) , the file content is wrong!

I have spent a few days tracing this issue and resolved it last weeks .
this is my trace steps:

  1. add a line (S3FS_PRN_INFO3) in function FdEntity::Open to log the file size.
int FdEntity::Open(headers_t* pmeta, ssize_t size, time_t time, bool no_fd_lock_wait)
{
 ........ //omit irrelevant code

  if(-1 != fd){
    // already opened, needs to increment refcnt.
    Dup();

    // check only file size(do not need to save cfs and time.
    S3FS_PRN_INFO3("path:%s. size:%ld. pagelist.Size():%ld. size_orgmeta:%ld", path.c_str(), size, pagelist.Size(), size_orgmeta);
    if(0 <= size && pagelist.Size() != static_cast<size_t>(size)){
      // truncate temporary file size
      ......
  1. write a file and analyze the log
Apr 28 00:12:10 test-gw-server s3fs[5750]: path:/Testfile. size:0. pagelist.Size():179550635. size_orgmeta:179550635
Apr 28 00:12:10 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0
Apr 28 00:12:10 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0
Apr 28 00:12:10 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:0. pagelist.Size():0. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:0. pagelist.Size():0. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():131072. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():262144. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():393216. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():524288. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():655360. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():786432. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():917504. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():1048576. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():2228224. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():2359296. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():2490368. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():2621440. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():2752512. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():2883584. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3014656. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3145728. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3276800. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3407872. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3538944. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3670016. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:0. pagelist.Size():3801088. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3932160. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():4063232. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():4194304. size_orgmeta:0
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():4194304. size_orgmeta:0
......

!!! Notice the line :
Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:0. pagelist.Size():3801088. size_orgmeta:0
unexcepted size 0!
the file will be truncated to 0 at this time, while some data have been write to file already !

So I fixed this problem by modify the condition of truncating file operation in function FdEntity::Open

the code modified could be found in pull request #1024

<!-- gh-comment-id:489434186 --> @dulingling200 commented on GitHub (May 5, 2019): @gaul @ggtakec this issue occurs when writing file to s3fs-fuse directory via nfs. **This is my operation steps to reproduce the issue:** 1. share the s3fs-fuse directory (e.g. /mnt/gw) via nfs on my server 2. mount the shared nfs directory on another directory (e.g. /status) 3. write a file(>10M) to the s3fs-fuse directory (e.g. /mnt/gw) directly, everything is ok. 4. write a file(>10M) to the shared nfs directory (e.g. /status) , the file content is wrong! I have spent a few days tracing this issue and resolved it last weeks . **this is my trace steps:** 1. add a line (S3FS_PRN_INFO3) in function FdEntity::Open to log the file size. ``` int FdEntity::Open(headers_t* pmeta, ssize_t size, time_t time, bool no_fd_lock_wait) { ........ //omit irrelevant code if(-1 != fd){ // already opened, needs to increment refcnt. Dup(); // check only file size(do not need to save cfs and time. S3FS_PRN_INFO3("path:%s. size:%ld. pagelist.Size():%ld. size_orgmeta:%ld", path.c_str(), size, pagelist.Size(), size_orgmeta); if(0 <= size && pagelist.Size() != static_cast<size_t>(size)){ // truncate temporary file size ...... ``` 2. write a file and analyze the log ``` Apr 28 00:12:10 test-gw-server s3fs[5750]: path:/Testfile. size:0. pagelist.Size():179550635. size_orgmeta:179550635 Apr 28 00:12:10 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0 Apr 28 00:12:10 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0 Apr 28 00:12:10 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:0. pagelist.Size():0. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:0. pagelist.Size():0. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():131072. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():262144. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():393216. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():524288. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():655360. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():786432. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():917504. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():1048576. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():2228224. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():2359296. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():2490368. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():2621440. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():2752512. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():2883584. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3014656. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3145728. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3276800. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3407872. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3538944. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3670016. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:0. pagelist.Size():3801088. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():0. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():3932160. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():4063232. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():4194304. size_orgmeta:0 Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:-1. pagelist.Size():4194304. size_orgmeta:0 ...... ``` **!!! Notice the line : Apr 28 00:12:11 test-gw-server s3fs[5750]: path:/Testfile. size:0. pagelist.Size():3801088. size_orgmeta:0** unexcepted size 0! **the file will be truncated to 0 at this time, while some data have been write to file already !** So I fixed this problem by modify the condition of truncating file operation in function FdEntity::Open the code modified could be found in pull request #1024
kerem commented 2026-03-04 01:46:46 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Jul 11, 2019):

@dulingling200 you might want to try #1063 which addresses another truncation symptom.

<!-- gh-comment-id:510290207 --> @gaul commented on GitHub (Jul 11, 2019): @dulingling200 you might want to try #1063 which addresses another truncation symptom.
kerem commented 2026-03-04 01:46:46 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Feb 2, 2020):

Closing due to lack of activity.

<!-- gh-comment-id:581139372 --> @gaul commented on GitHub (Feb 2, 2020): Closing due to lack of activity.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
v1.97
v1.96
v1.95
v1.94
v1.93
v1.92
v1.91
v1.90
v1.89
v1.88
v1.87
v1.86
v1.85
v1.84
v1.83
v1.82
v1.81
v1.80
v1.79
v1.78
Pre-v1.78
v1.77
v1.76
v1.75
v1.74
Labels
Clear labels   
bug

   
bug

   
dataloss

   
duplicate

   
enhancement

   
feature request

   
help wanted

   
invalid

   
need info

   
performance

   
pull-request

Mirrored from GitHub Pull Request

  
question

   
question

   
testing
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#561
No description provided.