starred/s3fs-fuse

Fork 0

mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2026-04-25 05:16:00 +03:00

[GH-ISSUE #2183] "cannot fstat / No such file or directory" when writing a file with ecryptfs (creates 0 byte file, then works if retried!). #1110

New issue

Open

opened 2026-03-04 01:51:27 +03:00 by kerem · 4 comments

kerem commented

2026-03-04 01:51:27 +03:00

Owner

Originally created by @netlore on GitHub (Jun 7, 2023).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2183

Additional Information

Version of s3fs being used (`s3fs --version`)

# s3fs --version
Amazon Simple Storage Service File System V1.90 (commit:unknown) with GnuTLS(gcrypt)

Version of fuse being used (`pkg-config --modversion fuse`, `rpm -qi fuse` or `dpkg -s fuse`)

Tried with both default "fuse3", and "fuse" packages, eg:-

# dpkg -s fuse3
Package: fuse3
Status: install ok installed
Priority: optional
Section: utils
Installed-Size: 90
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Architecture: amd64
Version: 3.14.0-4
Replaces: fuse
Provides: fuse (= 3.14.0-4)
Depends: libc6 (>= 2.34), libfuse3-3 (= 3.14.0-4), adduser, mount (>= 2.19.1), sed (>= 4)
Breaks: fuse
Conffiles:
 /etc/fuse.conf ba9c9667f7df01ae7081d2c05d1d24e6

# dpkg -s fuse
Package: fuse
Status: install ok installed
Priority: optional
Section: utils
Installed-Size: 103
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Architecture: amd64
Version: 2.9.9-6
Depends: libc6 (>= 2.34), libfuse2 (= 2.9.9-6), adduser, mount (>= 2.19.1), sed (>= 4)
Conffiles:
 /etc/fuse.conf 298587592c8444196833f317def414f2

Kernel information (`uname -r`)

# uname -r
6.2.0-21-generic

GNU/Linux Distribution, if applicable (`cat /etc/os-release`)

# cat /etc/os-release
PRETTY_NAME="Ubuntu Mantic Minotaur (development branch)"
NAME="Ubuntu"
VERSION_ID="23.10"
VERSION="23.10 (Mantic Minotaur)"
VERSION_CODENAME=mantic

How to run s3fs, if applicable

I've tried running with gradually fewer options, even with single thread mode, with/without xattr & cache etc... but issue always seen.

I tried each of the following options in addition to single thread with "-s":-

-ouse_cache=/srv/.s3fs_cache/
-oenable_noobj_cache
-omultireq_max=10
-ouse_xattr

Also, tried the "ecryptfs_xattr" option on the module load (kernel command line for ubuntu, as ecryptfs is compiled in).

# s3fs archive /srv/.s3fs

s3fs syslog messages (`grep s3fs /var/log/syslog`, `journalctl | grep s3fs`, or `s3fs outputs`)

2023-06-07T02:55:09.623423+01:00 trinity s3fs[18429]: s3fs version 1.90(unknown) : s3fs archive /srv/.s3fs
2023-06-07T02:55:09.624378+01:00 trinity s3fs[18429]: Loaded mime information from /etc/mime.types
2023-06-07T02:55:09.625614+01:00 trinity s3fs[18430]: init v1.90(commit:unknown) with GnuTLS(gcrypt)
2023-06-07T02:55:09.834883+01:00 trinity s3fs[18430]: s3fs.cpp:s3fs_check_service(3550): Failed to connect region 'us-east-1'(default), so retry to connect region 'eu-west-1'.

Details about issue

When using eCryptFS on top of s3fs, I encounter a situation where when writing a file I get the following error (example using cp to write the file, but the same issue is seen with vi, etc. "touch" does not exhibit the issue, but then it doesn't write data into the file):-

cp: cannot fstat '/srv/.s3-archive/test.file': No such file or directory

The files are created, but have 0 size... however if I attempt the copy again, the file is (over)written successfully... or if I "touch" the file first..

In this case, the ecryptfs mount is as follows:-

# mount -t ecryptfs /srv/.s3fs/.ecryptfs /srv/.s3-archive/
Attempting to mount with the following options:
  ecryptfs_unlink_sigs
  ecryptfs_fnek_sig=**************
  ecryptfs_key_bytes=16
  ecryptfs_cipher=aes
  ecryptfs_sig=****************
Mounted eCryptfs

I wonder if the issue is related to https://github.com/s3fs-fuse/s3fs-fuse/issues/1084, the symptoms described don't appear to be exactly the same, but the description is quite vague.

Is there anything else I can test, get debug output for, I feel like this should be quite easy to replicate.

Originally created by @netlore on GitHub (Jun 7, 2023). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2183 ### Additional Information #### Version of s3fs being used (`s3fs --version`) ``` # s3fs --version Amazon Simple Storage Service File System V1.90 (commit:unknown) with GnuTLS(gcrypt) ``` #### Version of fuse being used (`pkg-config --modversion fuse`, `rpm -qi fuse` or `dpkg -s fuse`) Tried with both default "fuse3", and "fuse" packages, eg:- ``` # dpkg -s fuse3 Package: fuse3 Status: install ok installed Priority: optional Section: utils Installed-Size: 90 Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com> Architecture: amd64 Version: 3.14.0-4 Replaces: fuse Provides: fuse (= 3.14.0-4) Depends: libc6 (>= 2.34), libfuse3-3 (= 3.14.0-4), adduser, mount (>= 2.19.1), sed (>= 4) Breaks: fuse Conffiles: /etc/fuse.conf ba9c9667f7df01ae7081d2c05d1d24e6 ``` OR ``` # dpkg -s fuse Package: fuse Status: install ok installed Priority: optional Section: utils Installed-Size: 103 Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com> Architecture: amd64 Version: 2.9.9-6 Depends: libc6 (>= 2.34), libfuse2 (= 2.9.9-6), adduser, mount (>= 2.19.1), sed (>= 4) Conffiles: /etc/fuse.conf 298587592c8444196833f317def414f2 ``` #### Kernel information (`uname -r`) ``` # uname -r 6.2.0-21-generic ``` #### GNU/Linux Distribution, if applicable (`cat /etc/os-release`) ``` # cat /etc/os-release PRETTY_NAME="Ubuntu Mantic Minotaur (development branch)" NAME="Ubuntu" VERSION_ID="23.10" VERSION="23.10 (Mantic Minotaur)" VERSION_CODENAME=mantic ``` #### How to run s3fs, if applicable  I've tried running with gradually fewer options, even with single thread mode, with/without xattr & cache etc... but issue always seen. I tried each of the following options in addition to single thread with "-s":- ``` -ouse_cache=/srv/.s3fs_cache/ -oenable_noobj_cache -omultireq_max=10 -ouse_xattr ``` Also, tried the "ecryptfs_xattr" option on the module load (kernel command line for ubuntu, as ecryptfs is compiled in).  ``` # s3fs archive /srv/.s3fs ``` #### s3fs syslog messages (`grep s3fs /var/log/syslog`, `journalctl | grep s3fs`, or `s3fs outputs`)  ``` 2023-06-07T02:55:09.623423+01:00 trinity s3fs[18429]: s3fs version 1.90(unknown) : s3fs archive /srv/.s3fs 2023-06-07T02:55:09.624378+01:00 trinity s3fs[18429]: Loaded mime information from /etc/mime.types 2023-06-07T02:55:09.625614+01:00 trinity s3fs[18430]: init v1.90(commit:unknown) with GnuTLS(gcrypt) 2023-06-07T02:55:09.834883+01:00 trinity s3fs[18430]: s3fs.cpp:s3fs_check_service(3550): Failed to connect region 'us-east-1'(default), so retry to connect region 'eu-west-1'. ``` ### Details about issue  When using eCryptFS on top of s3fs, I encounter a situation where when writing a file I get the following error (example using cp to write the file, but the same issue is seen with vi, etc. "touch" does not exhibit the issue, but then it doesn't write data into the file):- ``` cp: cannot fstat '/srv/.s3-archive/test.file': No such file or directory ``` The files are created, but have 0 size... however if I attempt the copy again, the file is (over)written successfully... or if I "touch" the file first.. In this case, the ecryptfs mount is as follows:- ``` # mount -t ecryptfs /srv/.s3fs/.ecryptfs /srv/.s3-archive/ Attempting to mount with the following options: ecryptfs_unlink_sigs ecryptfs_fnek_sig=************** ecryptfs_key_bytes=16 ecryptfs_cipher=aes ecryptfs_sig=**************** Mounted eCryptfs I wonder if the issue is related to https://github.com/s3fs-fuse/s3fs-fuse/issues/1084, the symptoms described don't appear to be exactly the same, but the description is quite vague. Is there anything else I can test, get debug output for, I feel like this should be quite easy to replicate.

kerem commented

2026-03-04 01:51:28 +03:00

Author

Owner

@ggtakec commented on GitHub (Jun 10, 2023):

@netlore
It seems that the combination of ecryptfs and s3fs was not finally resolved in the old issue.
I'd like to check, does this error occur not only for large files(using multipart uploads), but also for small files?

We have fixed xattr in v1.92, so if possible, could you try it in v1.92?
And if you can please try the nomultipart option. (However, this is just my concern and may be a waste of work.)
Thanks in advance for your assistance.

@ggtakec commented on GitHub (Jun 10, 2023): @netlore It seems that the combination of ecryptfs and s3fs was not finally resolved in the old issue. I'd like to check, does this error occur not only for large files(using multipart uploads), but also for small files? We have fixed `xattr` in v1.92, so if possible, could you try it in v1.92? And if you can please try the `nomultipart` option. (However, this is just my concern and may be a waste of work.) Thanks in advance for your assistance.

kerem commented

2026-03-04 01:51:28 +03:00

Author

Owner

@netlore commented on GitHub (Jun 12, 2023):

@netlore It seems that the combination of ecryptfs and s3fs was not finally resolved in the old issue. I'd like to check, does this error occur not only for large files(using multipart uploads), but also for small files?

It definitely occurs for small files, I was initially testing with "touch" so I didn't notice the issue until I tried to write data into the files, but when trying to save a 5 byte test file with an editor, save fails once, but creates a 0 byte file, then if I try again, the data gets written... I wonder if it's some kind of race condition where the file isn't created in time for data to start being written.

We have fixed xattr in v1.92, so if possible, could you try it in v1.92? And if you can please try the nomultipart option. (However, this is just my concern and may be a waste of work.) Thanks in advance for your assistance.

I'm currently using the package supplied by Canonical, so I'll have to get hold of the new verison, and build it from source, but I'll let you know with 1.92 once I have had time to do that.

@netlore commented on GitHub (Jun 12, 2023): > @netlore It seems that the combination of ecryptfs and s3fs was not finally resolved in the old issue. I'd like to check, does this error occur not only for large files(using multipart uploads), but also for small files? It definitely occurs for small files, I was initially testing with "touch" so I didn't notice the issue until I tried to write data into the files, but when trying to save a 5 byte test file with an editor, save fails once, but creates a 0 byte file, then if I try again, the data gets written... I wonder if it's some kind of race condition where the file isn't created in time for data to start being written. > > We have fixed `xattr` in v1.92, so if possible, could you try it in v1.92? And if you can please try the `nomultipart` option. (However, this is just my concern and may be a waste of work.) Thanks in advance for your assistance. I'm currently using the package supplied by Canonical, so I'll have to get hold of the new verison, and build it from source, but I'll let you know with 1.92 once I have had time to do that.

kerem commented

2026-03-04 01:51:28 +03:00

Author

Owner

@ggtakec commented on GitHub (Jun 12, 2023):

@netlore Thank you for your cooperation.
If possible during testing, having logs with dbglevel=info (and curldbg) options may help identify the cause.
(However, this option will output a lot of logs, so be careful.)

@ggtakec commented on GitHub (Jun 12, 2023): @netlore Thank you for your cooperation. If possible during testing, having logs with `dbglevel=info` (and `curldbg`) options may help identify the cause. (However, this option will output a lot of logs, so be careful.)

kerem commented

2026-03-04 01:51:28 +03:00

Author

Owner

@netlore commented on GitHub (Jun 26, 2023):

@ggtakec Sorry it has taken so long to get to this... I can confirm that I can't replicate the issue the latest version from GIT, so I don't know how valuable it may be to continue to investigate, but I should mention that it appeared to possibly be some kind of race condition, since I was not able to replicate with the original version when connected to a really fast connection, only when I had a relatively slower connection.

@netlore commented on GitHub (Jun 26, 2023): @ggtakec Sorry it has taken so long to get to this... I can confirm that I can't replicate the issue the latest version from GIT, so I don't know how valuable it may be to continue to investigate, but I should mention that it appeared to possibly be some kind of race condition, since I was not able to replicate with the original version when connected to a really fast connection, only when I had a relatively slower connection.