[GH-ISSUE #1818] Empty directories in S3 bucket lead to corrupted files #932

New issue

Open

opened 2026-03-04 01:50:02 +03:00 by kerem · 0 comments

kerem commented

2026-03-04 01:50:02 +03:00

Owner

Originally created by @tbobm on GitHub (Dec 13, 2021).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1818

It seems like having // in S3 keys lead to weird behaviour when mounting the S3 buckets.

Additional Information

The following information is very important in order to help us to help you. Omission of the following details may delay your support request or receive no attention at all.
Keep in mind that the commands we provide to retrieve information are oriented to GNU/Linux Distributions, so you could need to use others if you use s3fs on macOS or BSD

Version of s3fs being used (s3fs --version)

$ s3fs --version
Amazon Simple Storage Service File System V1.89 (commit:unknown) with GnuTLS(gcrypt)

Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse)

2.9.9-4ubuntu2

Kernel information (uname -r)

5.11.10-051110-generic

GNU/Linux Distribution, if applicable (cat /etc/os-release)

command result: cat /etc/os-release

NAME="Ubuntu"
VERSION="21.04 (Hirsute Hippo)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 21.04"
VERSION_ID="21.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=hirsute
UBUNTU_CODENAME=hirsute

s3fs command line used, if applicable

s3fs tbobm-sample-test-s3fs-bug ./mountpoint/ -o passwd_file=~/.passwd-s3fs

s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)

if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages

Details about issue

Creating a file withing en empty directory in S3 leads to a corrupted file definition in the local filesystem.

How to reprocude

Create an empty bucket, add a file in using the format s3://my-bucket//file, and list the content of the mountpoint.

$ aws s3api create-bucket --bucket tbobm-sample-test-s3fs-bug --region eu-west-1 --create-bucket-configuration LocationConstraint=eu-west-1
{
    "Location": "http://tbobm-sample-test-s3fs-bug.s3.amazonaws.com/"
}
$ touch dummy-file
$ aws s3 cp dummy-file s3://tbobm-sample-test-s3fs-bug/directory//dummy-file
upload: ./dummy-file to s3://tbobm-sample-test-s3fs-bug/directory//dummy-file
$ mkdir mountpoint
$ s3fs tbobm-sample-test-s3fs-bug ./mountpoint/ -o passwd_file=~/.passwd-s3fs
$ ls mountpoint/
directory
$ ls mountpoint/directory/
ls: cannot access 'mountpoint/directory/directory': No such file or directory
directory
$ ls -lhrt mountpoint/directory/
ls: cannot access 'mountpoint/directory/directory': No such file or directory
total 0
?????????? ? ? ? ?              ? directory

Originally created by @tbobm on GitHub (Dec 13, 2021). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1818 It seems like having `//` in S3 keys lead to weird behaviour when mounting the S3 buckets. ### Additional Information _The following information is very important in order to help us to help you. Omission of the following details may delay your support request or receive no attention at all._ _Keep in mind that the commands we provide to retrieve information are oriented to GNU/Linux Distributions, so you could need to use others if you use s3fs on macOS or BSD_ #### Version of s3fs being used (s3fs --version) ```console $ s3fs --version Amazon Simple Storage Service File System V1.89 (commit:unknown) with GnuTLS(gcrypt) ```` #### Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse) **2.9.9-4ubuntu2** #### Kernel information (uname -r) **5.11.10-051110-generic** #### GNU/Linux Distribution, if applicable (cat /etc/os-release) _command result: cat /etc/os-release_ ``` NAME="Ubuntu" VERSION="21.04 (Hirsute Hippo)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 21.04" VERSION_ID="21.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=hirsute UBUNTU_CODENAME=hirsute ``` #### s3fs command line used, if applicable ``` s3fs tbobm-sample-test-s3fs-bug ./mountpoint/ -o passwd_file=~/.passwd-s3fs ``` #### s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs) _if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages_ ``` ``` ### Details about issue Creating a file withing en empty directory in S3 leads to a corrupted file definition in the local filesystem. **How to reprocude** Create an empty bucket, add a file in using the format `s3://my-bucket//file`, and list the content of the mountpoint. ```console $ aws s3api create-bucket --bucket tbobm-sample-test-s3fs-bug --region eu-west-1 --create-bucket-configuration LocationConstraint=eu-west-1 { "Location": "http://tbobm-sample-test-s3fs-bug.s3.amazonaws.com/" } $ touch dummy-file $ aws s3 cp dummy-file s3://tbobm-sample-test-s3fs-bug/directory//dummy-file upload: ./dummy-file to s3://tbobm-sample-test-s3fs-bug/directory//dummy-file $ mkdir mountpoint $ s3fs tbobm-sample-test-s3fs-bug ./mountpoint/ -o passwd_file=~/.passwd-s3fs $ ls mountpoint/ directory $ ls mountpoint/directory/ ls: cannot access 'mountpoint/directory/directory': No such file or directory directory $ ls -lhrt mountpoint/directory/ ls: cannot access 'mountpoint/directory/directory': No such file or directory total 0 ?????????? ? ? ? ? ? directory ```