[GH-ISSUE #2149] [NFS support], can not upload file content. #1097

New issue

Open

opened 2026-03-04 01:51:21 +03:00 by kerem · 1 comment

kerem commented

2026-03-04 01:51:21 +03:00

Owner

Originally created by @VVoidV on GitHub (Apr 13, 2023).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2149

When I export s3fs dir by NFS, file content (size < 5 GB) is not uploaded.

reproduce：

mount s3fs
s3fs -f -o debug,update_parent_dir_stat,noforget,parallel_count=16,dbglevel=debug,use_path_request_style,allow_other,del_cache,connect_timeout=300,readwrite_timeout=120,url=http://100.127.40.130 nfs-0410 /exports/origin
export /exports/origin by NFS
modify /etc/exports, add this line below, then run exportfs -av

/exports/origin -async,no_root_squash,fsid=63e5587656082103d81ce51509e95d4d *(rw)

mount on localhost with mount -o vers=3 localhost:/exports/origin /mnt
echo 123 > /mnt/testfile
cat /mnt/testfile

the content of testfile is emtpy.

cause

It seems NFS does not call flush before release, so the data in cache is not uploaded.

I just add s3fs_flush in s3fs_release, the content of file will be uploaded successfully.

static int s3fs_release(const char* _path, struct fuse_file_info* fi)
{
    WTF8_ENCODE(path)
    S3FS_PRN_INFO("[path=%s][pseudo_fd=%llu]", path, (unsigned long long)(fi->fh));
    s3fs_flush(_path, fi);

    // [NOTE]
    // All opened file's stats is cached with no truncate flag.
    // Thus we unset it here.
    StatCache::getStatCacheData()->ChangeNoTruncateFlag(std::string(path), false);
.......

Additional Information

Version of s3fs being used (`s3fs --version`)

Amazon Simple Storage Service File System V1.91 (commit:9c74014) with OpenSSL

Version of fuse being used (`pkg-config --modversion fuse`, `rpm -qi fuse` or `dpkg -s fuse`)

fuse-libs-2.9.2-11.el7.x86_64

Kernel information (`uname -r`)

7.10.0-957.el7.x86_64

GNU/Linux Distribution, if applicable (`cat /etc/os-release`)

NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

How to run s3fs, if applicable

[x] command line
[] /etc/fstab

s3fs -f -o debug,update_parent_dir_stat,noforget,parallel_count=16,dbglevel=debug,use_path_request_style,allow_other,del_cache,connect_timeout=300,readwrite_timeout=120,url=http://100.127.40.130 nfs-0410 /exports/origin

s3fs syslog messages (`grep s3fs /var/log/syslog`, `journalctl | grep s3fs`, or `s3fs outputs`)

Details about issue

fuse ops:

unique: 50, opcode: LOOKUP (1), nodeid: 1, insize: 49, pid: 17588
LOOKUP /testfile
getattr /testfile
   unique: 50, error: -2 (No such file or directory), outsize: 16
unique: 51, opcode: MKNOD (8), nodeid: 1, insize: 65, pid: 17588
create flags: 0xc1 /testfile 0100644 umask=0000
   create[2] flags: 0xc1 /testfile
getattr /testfile
   NODEID: 18
release[2] flags: 0xc1
   unique: 51, success, outsize: 144
unique: 52, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 17588
getattr /
   unique: 52, success, outsize: 120
unique: 53, opcode: OPEN (14), nodeid: 18, insize: 48, pid: 17588
open flags: 0x8001 /testfile
   open[2] flags: 0x8001 /testfile
   unique: 53, success, outsize: 32
unique: 54, opcode: WRITE (16), nodeid: 18, insize: 84, pid: 17588
write[2] 4 bytes to 0 flags: 0x8001
   write[2] 4 bytes to 0
   unique: 54, success, outsize: 24
unique: 55, opcode: GETATTR (3), nodeid: 18, insize: 56, pid: 17588
getattr /testfile
   unique: 55, success, outsize: 120
unique: 56, opcode: RELEASE (18), nodeid: 18, insize: 64, pid: 0
release[2] flags: 0x8001
   unique: 56, success, outsize: 16

Originally created by @VVoidV on GitHub (Apr 13, 2023). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2149  When I export s3fs dir by NFS, file content (size < 5 GB) is not uploaded. ## reproduce： 1. mount s3fs ```s3fs -f -o debug,update_parent_dir_stat,noforget,parallel_count=16,dbglevel=debug,use_path_request_style,allow_other,del_cache,connect_timeout=300,readwrite_timeout=120,url=http://100.127.40.130 nfs-0410 /exports/origin``` 2. export /exports/origin by NFS modify /etc/exports, add this line below, then run `exportfs -av` ``` /exports/origin -async,no_root_squash,fsid=63e5587656082103d81ce51509e95d4d *(rw) ``` 3. mount on localhost with `mount -o vers=3 localhost:/exports/origin /mnt` 4. `echo 123 > /mnt/testfile` 5. `cat /mnt/testfile` the content of testfile is emtpy. ## cause It seems NFS does not call **flush** before release, so the data in cache is not uploaded. I just add s3fs_flush in s3fs_release, the content of file will be uploaded successfully. ```C static int s3fs_release(const char* _path, struct fuse_file_info* fi) { WTF8_ENCODE(path) S3FS_PRN_INFO("[path=%s][pseudo_fd=%llu]", path, (unsigned long long)(fi->fh)); s3fs_flush(_path, fi); // [NOTE] // All opened file's stats is cached with no truncate flag. // Thus we unset it here. StatCache::getStatCacheData()->ChangeNoTruncateFlag(std::string(path), false); ....... ``` ## Additional Information #### Version of s3fs being used (`s3fs --version`)  Amazon Simple Storage Service File System V1.91 (commit:9c74014) with OpenSSL #### Version of fuse being used (`pkg-config --modversion fuse`, `rpm -qi fuse` or `dpkg -s fuse`)  fuse-libs-2.9.2-11.el7.x86_64 #### Kernel information (`uname -r`)  7.10.0-957.el7.x86_64 #### GNU/Linux Distribution, if applicable (`cat /etc/os-release`)  NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/" CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7" #### How to run s3fs, if applicable  [x] command line [] /etc/fstab  ``` s3fs -f -o debug,update_parent_dir_stat,noforget,parallel_count=16,dbglevel=debug,use_path_request_style,allow_other,del_cache,connect_timeout=300,readwrite_timeout=120,url=http://100.127.40.130 nfs-0410 /exports/origin ``` #### s3fs syslog messages (`grep s3fs /var/log/syslog`, `journalctl | grep s3fs`, or `s3fs outputs`)  ``` ``` ### Details about issue  fuse ops: ``` unique: 50, opcode: LOOKUP (1), nodeid: 1, insize: 49, pid: 17588 LOOKUP /testfile getattr /testfile unique: 50, error: -2 (No such file or directory), outsize: 16 unique: 51, opcode: MKNOD (8), nodeid: 1, insize: 65, pid: 17588 create flags: 0xc1 /testfile 0100644 umask=0000 create[2] flags: 0xc1 /testfile getattr /testfile NODEID: 18 release[2] flags: 0xc1 unique: 51, success, outsize: 144 unique: 52, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 17588 getattr / unique: 52, success, outsize: 120 unique: 53, opcode: OPEN (14), nodeid: 18, insize: 48, pid: 17588 open flags: 0x8001 /testfile open[2] flags: 0x8001 /testfile unique: 53, success, outsize: 32 unique: 54, opcode: WRITE (16), nodeid: 18, insize: 84, pid: 17588 write[2] 4 bytes to 0 flags: 0x8001 write[2] 4 bytes to 0 unique: 54, success, outsize: 24 unique: 55, opcode: GETATTR (3), nodeid: 18, insize: 56, pid: 17588 getattr /testfile unique: 55, success, outsize: 120 unique: 56, opcode: RELEASE (18), nodeid: 18, insize: 64, pid: 0 release[2] flags: 0x8001 unique: 56, success, outsize: 16 ```

kerem commented

2026-03-04 01:51:22 +03:00

Author

Owner

@ggtakec commented on GitHub (Apr 16, 2023):

@VVoidV Thank you for your report.
s3fs expects s3fs_flush to be called before s3fs_relase is called, so it is correct that the file was not uploaded according to this report.

I created PR #2150 to fix this issue.
If you can build the #2150 code, please test it.
Thanks in advance for your assistance.

@ggtakec commented on GitHub (Apr 16, 2023): @VVoidV Thank you for your report. s3fs expects `s3fs_flush` to be called before `s3fs_relase` is called, so it is correct that the file was not uploaded according to this report. I created PR #2150 to fix this issue. If you can build the #2150 code, please test it. Thanks in advance for your assistance.