starred/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2026-04-25 21:35:58 +03:00
Code Issues 305 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #2022] A subdirectory is not accessible just after mounting S3 bucket #1015

New issue
Closed
opened 2026-03-04 01:50:40 +03:00 by kerem · 5 comments
kerem commented 2026-03-04 01:50:40 +03:00
Owner
Copy link

Originally created by @Mage152 on GitHub (Aug 12, 2022).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2022

Additional Information

The following information is very important in order to help us to help you. Omission of the following details may delay your support request or receive no attention at all.
Keep in mind that the commands we provide to retrieve information are oriented to GNU/Linux Distributions, so you could need to use others if you use s3fs on macOS or BSD

Version of s3fs being used (s3fs --version)

# s3fs --version | head -1
Amazon Simple Storage Service File System V1.91 (commit:unknown) with OpenSSL

Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse)

# rpm -q fuse
fuse-2.9.2-11.el7.x86_64

Kernel information (uname -r)

# uname -r
3.10.0-957.27.2.el7.x86_64

GNU/Linux Distribution, if applicable (cat /etc/os-release)

# cat /etc/redhat-release 
CentOS Linux release 7.6.1810 (Core)

s3fs command line used, if applicable

/etc/fstab entry, if applicable

# tail -1 /etc/fstab
bucket_name     /s3/my_directory   fuse.s3fs del_cache,allow_other,uid=1021,gid=1020,umask=000,_netdev,logfile=/var/log/s3fs.log,dbglevel=information,use_cache=/tmp/ 0 0

s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)

I specified into /etc/fstab the options

  • logfile=/var/log/s3fs.log
  • dbglevel=information

Details about issue

Just after mount /s3/my_directory directory, the following command does not work

$ ll /s3/my_directory/DATA/SUBDIR/
ls: cannot access /s3/my_directory/DATA/SUBDIR/: Not a directory

It works, if I do this:

myuser-myserver$ ll /s3/my_directory/DATA/SUBDIR/
ls: cannot access /s3/my_directory/DATA/SUBDIR/: Not a directory
myuser-myserver$ ll /s3/my_directory/DATA/
total 4
drwxrwxrwx 1 
drwxrwxrwx 1 
-rwxrwxrwx 1 

myuser-myserver$ ll /s3/my_directory/DATA/SUBDIR/
total 432274
drwxrwxrwx 1 
drwxrwxrwx 1 
-rwxrwxrwx 1

When it works, I get this into the /var/log/s3fs.log file

2022-08-12T09:27:49.871Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/DATA]
2022-08-12T09:27:49.871Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/DATA/SUBDIR]
2022-08-12T09:27:49.872Z [INF] s3fs.cpp:s3fs_opendir(2514): [path=/DATA/SUBDIR][flags=0x18800]
2022-08-12T09:27:49.872Z [INF] s3fs.cpp:s3fs_readdir(2663): [path=/DATA/SUBDIR]
2022-08-12T09:27:49.872Z [INF]   s3fs.cpp:list_bucket(2707): [path=/DATA/SUBDIR]
2022-08-12T09:27:49.872Z [INF]       curl.cpp:ListBucketRequest(3522): [tpath=/DATA/SUBDIR]
2022-08-12T09:27:49.872Z [INF]       curl_util.cpp:prepare_url(255): URL is https://s3.eu-west-2.amazonaws.com/bucket_name?delimiter=/&max-keys=1000&prefix=DATA/SUBDIR/
2022-08-12T09:27:49.872Z [INF]       curl_util.cpp:prepare_url(288): URL changed is https://bucket_name.s3.eu-west-2.amazonaws.com?delimiter=/&max-keys=1000&prefix=DATA/SUBDIR/
2022-08-12T09:27:49.872Z [INF]       curl.cpp:insertV4Headers(2696): computing signature [GET] [/] [delimiter=/&max-keys=1000&prefix=DATA/SUBDIR/] []
2022-08-12T09:27:49.872Z [INF]       curl_util.cpp:url_to_host(332): url is https://s3.eu-west-2.amazonaws.com
2022-08-12T09:27:49.931Z [INF]       curl.cpp:RequestPerform(2324): HTTP response code 200
2022-08-12T09:27:49.932Z [INF]   s3fs.cpp:readdir_multi_head(2579): [path=/DATA/SUBDIR/][list=0]
2022-08-12T09:27:49.932Z [INF]       curl_multi.cpp:Request(297): [count=0]
2022-08-12T09:27:49.932Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/DATA/SUBDIR/ZONE1]
2022-08-12T09:27:49.932Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/DATA/SUBDIR/INPUT_DATA]
2022-08-12T09:27:49.932Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/DATA/SUBDIR/SERT]
2022-08-12T09:27:49.932Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/DATA/SUBDIR/ZONE2]
2022-08-12T09:27:49.932Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/DATA/SUBDIR/ZONE3]
2022-08-12T09:27:49.932Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/DATA/SUBDIR/ZONE4]
2022-08-12T09:27:49.933Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/DATA/SUBDIR/file.tar]

When it does not work, I get this into the /var/log/s3fs.log file

2022-08-12T09:25:30.956Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/DATA]
2022-08-12T09:25:30.956Z [INF]       curl.cpp:HeadRequest(3100): [tpath=/DATA]
2022-08-12T09:25:30.956Z [INF]       curl.cpp:PreHeadRequest(3060): [tpath=/DATA][bpath=][save=][sseckeypos=18446744073709551615]
2022-08-12T09:25:30.956Z [INF]       curl_util.cpp:prepare_url(255): URL is https://s3.eu-west-2.amazonaws.com/bucket_name/DATA
2022-08-12T09:25:30.956Z [INF]       curl_util.cpp:prepare_url(288): URL changed is https://bucket_name.s3.eu-west-2.amazonaws.com/DATA
2022-08-12T09:25:30.956Z [INF]       curl.cpp:insertV4Headers(2696): computing signature [HEAD] [/DATA] [] []
2022-08-12T09:25:30.956Z [INF]       curl_util.cpp:url_to_host(332): url is https://s3.eu-west-2.amazonaws.com
2022-08-12T09:25:31.013Z [INF]       curl.cpp:RequestPerform(2376): HTTP response code 404 was returned, returning ENOENT
2022-08-12T09:25:31.013Z [INF]       curl.cpp:HeadRequest(3100): [tpath=/DATA/]
2022-08-12T09:25:31.013Z [INF]       curl.cpp:PreHeadRequest(3060): [tpath=/DATA/][bpath=][save=][sseckeypos=18446744073709551615]
2022-08-12T09:25:31.013Z [INF]       curl_util.cpp:prepare_url(255): URL is https://s3.eu-west-2.amazonaws.com/bucket_name/DATA/
2022-08-12T09:25:31.013Z [INF]       curl_util.cpp:prepare_url(288): URL changed is https://bucket_name.s3.eu-west-2.amazonaws.com/DATA/
2022-08-12T09:25:31.013Z [INF]       curl.cpp:insertV4Headers(2696): computing signature [HEAD] [/DATA/] [] []
2022-08-12T09:25:31.013Z [INF]       curl_util.cpp:url_to_host(332): url is https://s3.eu-west-2.amazonaws.com
2022-08-12T09:25:31.027Z [INF]       curl.cpp:RequestPerform(2324): HTTP response code 200
2022-08-12T09:25:31.027Z [INF]       cache.cpp:AddStat(343): add stat cache entry[path=/DATA/]
2022-08-12T09:25:31.028Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/DATA/SUBDIR]
2022-08-12T09:25:31.028Z [INF]       curl.cpp:HeadRequest(3100): [tpath=/DATA/SUBDIR]

Is somebody can tell me why I cannot access to the /s3/my_directory/DATA/SUBDIR/ directory just after mount /s3/my_directory directory?
Thanks for your help!

Originally created by @Mage152 on GitHub (Aug 12, 2022). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2022 ### Additional Information _The following information is very important in order to help us to help you. Omission of the following details may delay your support request or receive no attention at all._ _Keep in mind that the commands we provide to retrieve information are oriented to GNU/Linux Distributions, so you could need to use others if you use s3fs on macOS or BSD_ #### Version of s3fs being used (s3fs --version) ``` # s3fs --version | head -1 Amazon Simple Storage Service File System V1.91 (commit:unknown) with OpenSSL ``` #### Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse) ``` # rpm -q fuse fuse-2.9.2-11.el7.x86_64 ``` #### Kernel information (uname -r) ``` # uname -r 3.10.0-957.27.2.el7.x86_64 ``` #### GNU/Linux Distribution, if applicable (cat /etc/os-release) ``` # cat /etc/redhat-release CentOS Linux release 7.6.1810 (Core) ``` #### s3fs command line used, if applicable ``` ``` #### /etc/fstab entry, if applicable ``` # tail -1 /etc/fstab bucket_name /s3/my_directory fuse.s3fs del_cache,allow_other,uid=1021,gid=1020,umask=000,_netdev,logfile=/var/log/s3fs.log,dbglevel=information,use_cache=/tmp/ 0 0 ``` #### s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs) I specified into /etc/fstab the options - logfile=/var/log/s3fs.log - dbglevel=information ### Details about issue Just after mount /s3/my_directory directory, the following command does not work ``` $ ll /s3/my_directory/DATA/SUBDIR/ ls: cannot access /s3/my_directory/DATA/SUBDIR/: Not a directory ``` It works, if I do this: ``` myuser-myserver$ ll /s3/my_directory/DATA/SUBDIR/ ls: cannot access /s3/my_directory/DATA/SUBDIR/: Not a directory myuser-myserver$ ll /s3/my_directory/DATA/ total 4 drwxrwxrwx 1 drwxrwxrwx 1 -rwxrwxrwx 1 myuser-myserver$ ll /s3/my_directory/DATA/SUBDIR/ total 432274 drwxrwxrwx 1 drwxrwxrwx 1 -rwxrwxrwx 1 ``` **When it works, I get this into the /var/log/s3fs.log file** ``` 2022-08-12T09:27:49.871Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/DATA] 2022-08-12T09:27:49.871Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/DATA/SUBDIR] 2022-08-12T09:27:49.872Z [INF] s3fs.cpp:s3fs_opendir(2514): [path=/DATA/SUBDIR][flags=0x18800] 2022-08-12T09:27:49.872Z [INF] s3fs.cpp:s3fs_readdir(2663): [path=/DATA/SUBDIR] 2022-08-12T09:27:49.872Z [INF] s3fs.cpp:list_bucket(2707): [path=/DATA/SUBDIR] 2022-08-12T09:27:49.872Z [INF] curl.cpp:ListBucketRequest(3522): [tpath=/DATA/SUBDIR] 2022-08-12T09:27:49.872Z [INF] curl_util.cpp:prepare_url(255): URL is https://s3.eu-west-2.amazonaws.com/bucket_name?delimiter=/&max-keys=1000&prefix=DATA/SUBDIR/ 2022-08-12T09:27:49.872Z [INF] curl_util.cpp:prepare_url(288): URL changed is https://bucket_name.s3.eu-west-2.amazonaws.com?delimiter=/&max-keys=1000&prefix=DATA/SUBDIR/ 2022-08-12T09:27:49.872Z [INF] curl.cpp:insertV4Headers(2696): computing signature [GET] [/] [delimiter=/&max-keys=1000&prefix=DATA/SUBDIR/] [] 2022-08-12T09:27:49.872Z [INF] curl_util.cpp:url_to_host(332): url is https://s3.eu-west-2.amazonaws.com 2022-08-12T09:27:49.931Z [INF] curl.cpp:RequestPerform(2324): HTTP response code 200 2022-08-12T09:27:49.932Z [INF] s3fs.cpp:readdir_multi_head(2579): [path=/DATA/SUBDIR/][list=0] 2022-08-12T09:27:49.932Z [INF] curl_multi.cpp:Request(297): [count=0] 2022-08-12T09:27:49.932Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/DATA/SUBDIR/ZONE1] 2022-08-12T09:27:49.932Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/DATA/SUBDIR/INPUT_DATA] 2022-08-12T09:27:49.932Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/DATA/SUBDIR/SERT] 2022-08-12T09:27:49.932Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/DATA/SUBDIR/ZONE2] 2022-08-12T09:27:49.932Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/DATA/SUBDIR/ZONE3] 2022-08-12T09:27:49.932Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/DATA/SUBDIR/ZONE4] 2022-08-12T09:27:49.933Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/DATA/SUBDIR/file.tar] ``` **When it does not work, I get this into the /var/log/s3fs.log file** ``` 2022-08-12T09:25:30.956Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/DATA] 2022-08-12T09:25:30.956Z [INF] curl.cpp:HeadRequest(3100): [tpath=/DATA] 2022-08-12T09:25:30.956Z [INF] curl.cpp:PreHeadRequest(3060): [tpath=/DATA][bpath=][save=][sseckeypos=18446744073709551615] 2022-08-12T09:25:30.956Z [INF] curl_util.cpp:prepare_url(255): URL is https://s3.eu-west-2.amazonaws.com/bucket_name/DATA 2022-08-12T09:25:30.956Z [INF] curl_util.cpp:prepare_url(288): URL changed is https://bucket_name.s3.eu-west-2.amazonaws.com/DATA 2022-08-12T09:25:30.956Z [INF] curl.cpp:insertV4Headers(2696): computing signature [HEAD] [/DATA] [] [] 2022-08-12T09:25:30.956Z [INF] curl_util.cpp:url_to_host(332): url is https://s3.eu-west-2.amazonaws.com 2022-08-12T09:25:31.013Z [INF] curl.cpp:RequestPerform(2376): HTTP response code 404 was returned, returning ENOENT 2022-08-12T09:25:31.013Z [INF] curl.cpp:HeadRequest(3100): [tpath=/DATA/] 2022-08-12T09:25:31.013Z [INF] curl.cpp:PreHeadRequest(3060): [tpath=/DATA/][bpath=][save=][sseckeypos=18446744073709551615] 2022-08-12T09:25:31.013Z [INF] curl_util.cpp:prepare_url(255): URL is https://s3.eu-west-2.amazonaws.com/bucket_name/DATA/ 2022-08-12T09:25:31.013Z [INF] curl_util.cpp:prepare_url(288): URL changed is https://bucket_name.s3.eu-west-2.amazonaws.com/DATA/ 2022-08-12T09:25:31.013Z [INF] curl.cpp:insertV4Headers(2696): computing signature [HEAD] [/DATA/] [] [] 2022-08-12T09:25:31.013Z [INF] curl_util.cpp:url_to_host(332): url is https://s3.eu-west-2.amazonaws.com 2022-08-12T09:25:31.027Z [INF] curl.cpp:RequestPerform(2324): HTTP response code 200 2022-08-12T09:25:31.027Z [INF] cache.cpp:AddStat(343): add stat cache entry[path=/DATA/] 2022-08-12T09:25:31.028Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/DATA/SUBDIR] 2022-08-12T09:25:31.028Z [INF] curl.cpp:HeadRequest(3100): [tpath=/DATA/SUBDIR] ``` **Is somebody can tell me why I cannot access to the /s3/my_directory/DATA/SUBDIR/ directory just after mount /s3/my_directory directory? Thanks for your help!**
kerem closed this issue 2026-03-04 01:50:40 +03:00
kerem commented 2026-03-04 01:50:41 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Aug 14, 2022):

Are the DATA/VISION1/*** files uploaded from other than s3fs?

If a file is uploaded under a directory path that does not exist using the aws command, etc., the directory may not be visible from s3fs.
For example, if you upload myfile under the directory nodir(<bucket>/nodir/myfile), the files in the directory cannot be listed because there is no stat information for the nodir directory.

A workaround is to access the file directly. (e.g. ls nodir/myfile)
Alternatively, update the stat information of the directory so that the directory has stat information that s3fs can recognize. (e.g. touch nodir)

I'll try to fix this issue so that it can be listed by specifying options.
(There is a compat_dir option in the code of the master branch. I will modify it so that even if the stat of the directory does not exist when this option is specified, it will be listed.)

<!-- gh-comment-id:1214397595 --> @ggtakec commented on GitHub (Aug 14, 2022): Are the `DATA/VISION1/***` files uploaded from other than s3fs? If a file is uploaded under a directory path that does not exist using the aws command, etc., the directory may not be visible from s3fs. For example, if you upload `myfile` under the directory `nodir`(`<bucket>/nodir/myfile`), the files in the directory cannot be listed because there is no stat information for the `nodir` directory. A workaround is to access the file directly. (e.g. `ls nodir/myfile`) Alternatively, update the stat information of the directory so that the directory has stat information that s3fs can recognize. (e.g. `touch nodir`) I'll try to fix this issue so that it can be listed by specifying options. (There is a `compat_dir` option in the code of the master branch. I will modify it so that even if the stat of the directory does not exist when this option is specified, it will be listed.)
kerem commented 2026-03-04 01:50:41 +03:00
Author
Owner
Copy link

@Mage152 commented on GitHub (Aug 16, 2022):

Thank you for your reply ggtakec !

Here is the test you ask me to do.

[root@myserver ~]# umount /s3/my_directory
[root@myserver ~]# mount /s3/my_directory

If I try to list file.tar immediately after mount, I cannot access to the file.

myuser-myserver$ ll /s3/my_directory/DATA/SUBDIR/file.tar
ls: cannot access /s3/my_directory/DATA/SUBDIR/file.tar: Not a directory

But, if I list DATA directory, I can access to file.tar once again.

myuser-myserver$ ll /s3/my_directory/DATA/                
total 4
drwxrwxrwx 1 myuser mygroup    0 Jan  1  1970 OTHERDIR
drwxrwxrwx 1 myuser mygroup    0 May 23 14:14 SUBDIR
-rwxrwxrwx 1 myuser mygroup 1552 Jun 16 07:50 csvfile.csv
-rwxrwxrwx 1 myuser mygroup  468 Jun 16 07:49 textfile.txt

myuser-myserver$ ll /s3/my_directory/DATA/SUBDIR/file.tar
-rwxrwxrwx 1 myuser mygroup 442644480 Aug  4 16:37 /s3/my_directory/DATA/SUBDIR/file.tar

I still don't understand what happen.

<!-- gh-comment-id:1216443771 --> @Mage152 commented on GitHub (Aug 16, 2022): Thank you for your reply ggtakec ! Here is the test you ask me to do. ``` [root@myserver ~]# umount /s3/my_directory [root@myserver ~]# mount /s3/my_directory ``` If I try to list file.tar immediately after mount, I cannot access to the file. ``` myuser-myserver$ ll /s3/my_directory/DATA/SUBDIR/file.tar ls: cannot access /s3/my_directory/DATA/SUBDIR/file.tar: Not a directory ``` But, if I list DATA directory, I can access to file.tar once again. ``` myuser-myserver$ ll /s3/my_directory/DATA/ total 4 drwxrwxrwx 1 myuser mygroup 0 Jan 1 1970 OTHERDIR drwxrwxrwx 1 myuser mygroup 0 May 23 14:14 SUBDIR -rwxrwxrwx 1 myuser mygroup 1552 Jun 16 07:50 csvfile.csv -rwxrwxrwx 1 myuser mygroup 468 Jun 16 07:49 textfile.txt myuser-myserver$ ll /s3/my_directory/DATA/SUBDIR/file.tar -rwxrwxrwx 1 myuser mygroup 442644480 Aug 4 16:37 /s3/my_directory/DATA/SUBDIR/file.tar ``` I still don't understand what happen.
kerem commented 2026-03-04 01:50:41 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Aug 16, 2022):

Thanks for the detailed information.
Maybe your condition is the same as the cause I mentioned.

As a directory, s3fs needs an object (<dirname>/) to get its attributes.
However, if you directly upload files under a directory using the aws command, the object in that directory does not exist.
In your case there is a DATA/SUBDIR/file.tar object, but no DATA/ or DATA/SUBDIR/ objects.

I think this can be worked around by specifying the compat_dir option once the code currently in PR is merged.
(still subject to change if this PR gets merged or not)

<!-- gh-comment-id:1216743660 --> @ggtakec commented on GitHub (Aug 16, 2022): Thanks for the detailed information. Maybe your condition is the same as the cause I mentioned. As a directory, s3fs needs an object (`<dirname>/`) to get its attributes. However, if you directly upload files under a directory using the aws command, the object in that directory does not exist. In your case there is a `DATA/SUBDIR/file.tar` object, but no `DATA/` or `DATA/SUBDIR/` objects. I think this can be worked around by specifying the compat_dir option once the code currently in PR is merged. (still subject to change if this PR gets merged or not)
kerem commented 2026-03-04 01:50:41 +03:00
Author
Owner
Copy link

@Mage152 commented on GitHub (Aug 19, 2022):

The issue is due to the presence of two S3 objects with the same name:

  • one file /DATA/SUBDIR,
  • one directory /DATA/SUBDIR/

This is possible since those two objects have two different keys, one with a trailing / and the other without.

When s3fuse try to access to a file in the SUBDIR directory, it tests all the hierarchy. During this test, s3fuse see that SUBDIR is also a file and stop.

For SUBDIR, we find 2 objects !

$ aws s3api list-objects --bucket my_bucket | less 
...
            {
                "Key": "DATA/SUBDIR",
                "LastModified": "2022-08-02T09:45:09+00:00",
                "ETag": "\"866eadd83f2d7d5977fff570dabe6488\"",
                "Size": 1370643,
                "StorageClass": "STANDARD",
                "Owner": {
                    "ID": "618325c147f05a699aee84acd228550edfc29ca5eacea59006fa99454b5aa783"
                }
            },
            {
                "Key": "DATA/SUBDIR/",
                "LastModified": "2022-07-31T16:24:17+00:00",
                "ETag": "\"d41d8cd98f00bbb4e9800998ecf8427e\"",
                "Size": 0,
                "StorageClass": "STANDARD",
                "Owner": {
                    "ID": "618325c147f05a699aee84acd228550afdc29ca5eaaaa59006fa99454b5aa783"
                }
            },

The solution is to rename SUBDIR file
$ aws s3 mv s3://my_bucket/DATA/SUBDIR s3://my_bucket/DATA/SUBDIR.file

Thanks a lot @ggtakec for your help !

<!-- gh-comment-id:1220627149 --> @Mage152 commented on GitHub (Aug 19, 2022): The issue is due to the presence of two S3 objects with the same name: - one file /DATA/SUBDIR, - one directory /DATA/SUBDIR/ This is possible since those two objects have two different keys, one with a trailing / and the other without. When s3fuse try to access to a file in the SUBDIR directory, it tests all the hierarchy. During this test, s3fuse see that SUBDIR is also a file and stop. For SUBDIR, we find 2 objects ! ``` $ aws s3api list-objects --bucket my_bucket | less ... { "Key": "DATA/SUBDIR", "LastModified": "2022-08-02T09:45:09+00:00", "ETag": "\"866eadd83f2d7d5977fff570dabe6488\"", "Size": 1370643, "StorageClass": "STANDARD", "Owner": { "ID": "618325c147f05a699aee84acd228550edfc29ca5eacea59006fa99454b5aa783" } }, { "Key": "DATA/SUBDIR/", "LastModified": "2022-07-31T16:24:17+00:00", "ETag": "\"d41d8cd98f00bbb4e9800998ecf8427e\"", "Size": 0, "StorageClass": "STANDARD", "Owner": { "ID": "618325c147f05a699aee84acd228550afdc29ca5eaaaa59006fa99454b5aa783" } }, ``` The solution is to rename SUBDIR file `$ aws s3 mv s3://my_bucket/DATA/SUBDIR s3://my_bucket/DATA/SUBDIR.file` Thanks a lot @ggtakec for your help !
kerem commented 2026-03-04 01:50:41 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Aug 20, 2022):

@Mage152 Thanks for contacting me about the detailed cause.

We can certainly expect problems with s3fs if there is a file with the same name as the directory.
s3fs in its modification history has seen DIR as a directory object in the past, and keep a compatibility code as failsafe now.
Also, in your case you also have a DIR/ object (I think DIR/ takes precedence) and I think you had a behavioral problem.
Thank you again for your kindness.

<!-- gh-comment-id:1221251357 --> @ggtakec commented on GitHub (Aug 20, 2022): @Mage152 Thanks for contacting me about the detailed cause. We can certainly expect problems with s3fs if there is a file with the same name as the directory. s3fs in its modification history has seen `DIR` as a directory object in the past, and keep a compatibility code as failsafe now. Also, in your case you also have a `DIR/` object (I think `DIR/` takes precedence) and I think you had a behavioral problem. Thank you again for your kindness.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
v1.97
v1.96
v1.95
v1.94
v1.93
v1.92
v1.91
v1.90
v1.89
v1.88
v1.87
v1.86
v1.85
v1.84
v1.83
v1.82
v1.81
v1.80
v1.79
v1.78
Pre-v1.78
v1.77
v1.76
v1.75
v1.74
Labels
Clear labels   
bug

   
bug

   
dataloss

   
duplicate

   
enhancement

   
feature request

   
help wanted

   
invalid

   
need info

   
performance

   
pull-request

Mirrored from GitHub Pull Request

  
question

   
question

   
testing
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#1015
No description provided.