mirror of
https://github.com/s3fs-fuse/s3fs-fuse.git
synced 2026-04-24 21:06:02 +03:00
[GH-ISSUE #2748] Redhat 8.8 and 8.10, awscli image upload to powerscale but when looking through s3fs mounted share shows image as folder #1285
Labels
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/s3fs-fuse#1285
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @zahnmax on GitHub (Oct 14, 2025).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2748
So as the title denotes when image is uploaded via awscli on the storage backend the file is still a file. Awscli for writing or pulling is able to work on the file as expected. With s3fs mounted on 8.8 or 8.10 redhat though new image content shows as a folder. However S3 acl is identical between old and new images. Old images show up as expected. Backed storage which is a powerscale the files show up as expected and identical ls -led outputs. I would assume if the permissions were different per file in different folders on the backend that might explain it as a bug potentially but unfortunately not the case.
s3fs --version
Amazon Simple Storage Service File System V1.95 with OpenSSL
pkg-config --modversion fuse
2.9.7
Readable File
-rwxrwxr-x 1 root root 1378439799 Sep 25 14:40 File_Old.svs
Non-Readable File
drwxrwxr-x 1 root root 4096 Oct 9 10:21 File_New.svs
Mounting Logs
Oct 14 10:33:25 LocalServer s3fs[1164138]: s3fs_logger.cpp:LowSetLogLevel(233): change debug level from [CRT] to [DBG]
Oct 14 10:33:25 LocalServer s3fs[1164138]: PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40777)
Oct 14 10:33:25 LocalServer s3fs[1164138]: Loaded mime information from /etc/mime.types
Oct 14 10:33:25 LocalServer s3fs[1164138]: The path to cache top dir is empty, thus not need to check permission.
Oct 14 10:33:25 LocalServer s3fs[1164138]: Free space ratio default to 10 %, ensure the available disk space is greater than 13919.600 MB
Oct 14 10:33:25 LocalServer s3fs[1164139]: init v1.95 with OpenSSL, credential-library(built-in)
Oct 14 10:33:25 LocalServer s3fs[1164139]: check services.
Oct 14 10:33:25 LocalServer s3fs[1164139]: check a bucket path(/).
Oct 14 10:33:25 LocalServer s3fs[1164139]: URL is https://S3_Storage:port/bucket/
Oct 14 10:33:25 LocalServer s3fs[1164139]: URL changed is https://S3_Storage:port/bucket/
Oct 14 10:33:25 LocalServer s3fs[1164139]: computing signature [GET] [/] [] []
Oct 14 10:33:25 LocalServer s3fs[1164139]: url is https://S3_Storage:port
Oct 14 10:33:26 LocalServer s3fs[1164139]: HTTP response code 200
Oct 14 10:33:26 LocalServer s3fs[1164139]: Any threads are running now, then nothing to do.
Oct 14 10:33:26 LocalServer s3fs[1164139]: Start worker thread in ThreadPoolMan.
Oct 14 10:33:26 LocalServer s3fs[1164139]: Start worker thread in ThreadPoolMan.
Oct 14 10:33:26 LocalServer s3fs[1164139]: Start worker thread in ThreadPoolMan.
Oct 14 10:33:26 LocalServer s3fs[1164139]: Start worker thread in ThreadPoolMan.
Oct 14 10:33:26 LocalServer s3fs[1164139]: Start worker thread in ThreadPoolMan.
Oct 14 10:33:28 LocalServer s3fs[1164139]: [path=/][pid=1164153,uid=0,gid=0]
Oct 14 10:33:28 LocalServer s3fs[1164139]: [tpath=//]
Oct 14 10:33:28 LocalServer s3fs[1164139]: [tpath=//][bpath=][save=][sseckeypos=#'s]
Oct 14 10:33:28 LocalServer s3fs[1164139]: URL is https://S3_Storage:port/bucket//
Oct 14 10:33:28 LocalServer s3fs[1164139]: URL changed is https://S3_Storage:port/bucket//
Oct 14 10:33:28 LocalServer s3fs[1164139]: computing signature [HEAD] [//] [] []
Oct 14 10:33:28 LocalServer s3fs[1164139]: url is https://S3_Storage:port
Oct 14 10:33:28 LocalServer s3fs[1164139]: HTTP response code 200
Oct 14 10:33:28 LocalServer s3fs[1164139]: add stat cache entry[path=/]
@ggtakec commented on GitHub (Oct 25, 2025):
Please let us know the command line or fstab entry used to start s3fs-fuse.
Also, if possible, try changing the logging level using the
dbglevel=dbgoption and comparing the logs when accessing the target file.For reference, the conditions under which s3fs-fuse determines an S3 object to be a directory are as follows:
/(e.g.,dir/).dirordir/do not exist, butdir/fileexists).Also, try the
compat_diroption.@zahnmax commented on GitHub (Oct 28, 2025):
Mainly mounting via cli. General command syntax minus a few specifics and general log output
COMMAND:
s3fs BUCKET MOUNT_LOCATION -o dbglevel=DEBUG -o passwd_file=PASSWORD_FILE -o url=FULL_PATH -o use_path_request_style -o compat_dir
LOGGING WITH ALSO TRYING COMPAT DIR:
Oct 28 09:47:51 LocalServer s3fs[1927243]: s3fs version 1.95 : s3fs -o dbglevel=DEBUG -o passwd_file=PASSWORD_FILE -o url=FULL_PATH -o use_path_request_style -o compat_dir BUCKET MOUNT_LOCATION
Oct 28 09:47:51 LocalServer s3fs[1927243]: s3fs_logger.cpp:LowSetLogLevel(233): change debug level from [CRT] to [DBG]
Oct 28 09:47:51 LocalServer s3fs[1927243]: PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40777)
Oct 28 09:47:51 LocalServer s3fs[1927243]: Loaded mime information from /etc/mime.types
Oct 28 09:47:51 LocalServer s3fs[1927243]: The path to cache top dir is empty, thus not need to check permission.
Oct 28 09:47:51 LocalServer s3fs[1927244]: init v1.95 with OpenSSL, credential-library(built-in)
Oct 28 09:47:51 LocalServer s3fs[1927244]: check services.
Oct 28 09:47:51 LocalServer s3fs[1927244]: check a bucket path(/) containing compatible directory paths.
Oct 28 09:47:51 LocalServer s3fs[1927244]: URL is FULL_PATH/BUCKET/
Oct 28 09:47:51 LocalServer s3fs[1927244]: URL changed is FULL_PATH/BUCKET/
Oct 28 09:47:51 LocalServer s3fs[1927244]: computing signature [GET] [/] [] []
Oct 28 09:47:51 LocalServer s3fs[1927244]: url is FULL_PATH
Oct 28 09:47:51 LocalServer s3fs[1927244]: HTTP response code 200
Oct 28 09:47:51 LocalServer s3fs[1927244]: Any threads are running now, then nothing to do.
Oct 28 09:47:51 LocalServer s3fs[1927244]: Start worker thread in ThreadPoolMan.
Oct 28 09:47:51 LocalServer s3fs[1927244]: Start worker thread in ThreadPoolMan.
Oct 28 09:47:51 LocalServer s3fs[1927244]: Start worker thread in ThreadPoolMan.
Oct 28 09:47:51 LocalServer s3fs[1927244]: Start worker thread in ThreadPoolMan.
Oct 28 09:47:51 LocalServer s3fs[1927244]: Start worker thread in ThreadPoolMan.
New Folder (folder and files same permissions as older folder/files)
Oct 28 11:57:03 cc-cllabred8 s3fs[2008181]: [path=/FOLDER/FILE_FOLDER][pid=2081162,uid=0,gid=0]
Oct 28 11:57:03 cc-cllabred8 s3fs[2008181]: delete stat cache entry[path=/FOLDER/FILE_FOLDER/]
Oct 28 11:57:03 cc-cllabred8 s3fs[2008181]: [tpath=/FOLDER/FILE_FOLDER]
Oct 28 11:57:03 cc-cllabred8 s3fs[2008181]: [tpath=/FOLDER/FILE_FOLDER][bpath=][save=][sseckeypos=#]
Oct 28 11:57:03 cc-cllabred8 s3fs[2008181]: URL is FULL_PATH/BUCKET/FOLDER/FILE_FOLDER
Oct 28 11:57:03 cc-cllabred8 s3fs[2008181]: URL changed is FULL_PATH/BUCKET/FOLDER/FILE_FOLDER
Oct 28 11:57:03 cc-cllabred8 s3fs[2008181]: computing signature [HEAD] [/FOLDER/FILE_FOLDER] [] []
Oct 28 11:57:03 cc-cllabred8 s3fs[2008181]: url is FULL_PATH
Oct 28 11:57:03 cc-cllabred8 s3fs[2008181]: HTTP response code 404 was returned, returning ENOENT
Oct 28 11:57:03 cc-cllabred8 s3fs[2008181]: [tpath=/FOLDER/FILE_FOLDER/]
Oct 28 11:57:03 cc-cllabred8 s3fs[2008181]: [tpath=/FOLDER/FILE_FOLDER/][bpath=][save=][sseckeypos=#]
Oct 28 11:57:03 cc-cllabred8 s3fs[2008181]: URL is FULL_PATH/BUCKET/FOLDER/FILE_FOLDER/
Oct 28 11:57:03 cc-cllabred8 s3fs[2008181]: URL changed is FULL_PATH/BUCKET/FOLDER/FILE_FOLDER/
Oct 28 11:57:03 cc-cllabred8 s3fs[2008181]: computing signature [HEAD] [/FOLDER/FILE_FOLDER/] [] []
Oct 28 11:57:03 cc-cllabred8 s3fs[2008181]: url is FULL_PATH
Oct 28 11:57:03 cc-cllabred8 s3fs[2008181]: HTTP response code 404 was returned, returning ENOENT
Old Folder (folder and files same permissions as newer folder/files)
Oct 28 11:57:12 cc-cllabred8 s3fs[2008181]: [path=/FOLDER][pid=2081162,uid=0,gid=0]
Oct 28 11:57:12 cc-cllabred8 s3fs[2008181]: [path=/FOLDER/FILE_FOLDER][pid=2081162,uid=0,gid=0]
Oct 28 11:57:12 cc-cllabred8 s3fs[2008181]: delete stat cache entry[path=/FOLDER/FILE_FOLDER/]
Oct 28 11:57:12 cc-cllabred8 s3fs[2008181]: [tpath=/FOLDER/FILE_FOLDER]
Oct 28 11:57:12 cc-cllabred8 s3fs[2008181]: [tpath=/FOLDER/FILE_FOLDER][bpath=][save=][sseckeypos=#]
Oct 28 11:57:12 cc-cllabred8 s3fs[2008181]: URL is FULL_PATH/BUCKET/FOLDER/FILE_FOLDER
Oct 28 11:57:12 cc-cllabred8 s3fs[2008181]: URL changed is FULL_PATH/BUCKET/FOLDER/FILE_FOLDER
Oct 28 11:57:12 cc-cllabred8 s3fs[2008181]: computing signature [HEAD] [/FOLDER/FILE_FOLDER] [] []
Oct 28 11:57:12 cc-cllabred8 s3fs[2008181]: url is FULL_PATH
Oct 28 11:57:12 cc-cllabred8 s3fs[2008181]: HTTP response code 200
Oct 28 11:57:12 cc-cllabred8 s3fs[2008181]: add stat cache entry[path=/FOLDER/FILE_FOLDER]
@ggtakec commented on GitHub (Nov 1, 2025):
@zahnmax
In the above log, does
/FOLDER/FILE_FOLDERmeanFile_New.svs?I could not find why the
File_New.svsfile object is being identified as a directory.If you can see the metadata for the
File_New.svsfile object using powerscale(by its console?), please check it.If the object's
Content-Typeisapplication/x-directory, it will be recognized as a directory.Alternatively, check whether
x-amz-meta-modeis set to the value040000(octal), which indicates a directory.If you do not have a way to check the file object's metadata, you can add
-o curldbgwhen starting s3fs. This will output the request response, which you can then check in the log.@zahnmax commented on GitHub (Nov 5, 2025):
@ggtakec
So only under New Folder (folder and files same permissions as older folder/files) then yes.
Powerscale stat
817239842 5044458081 -rwxrwxr-x 1 USER GROUP 18446744073709551615 177552579 "Nov 4 10:56:51 2025" "Oct 21 16:13:28 2025" "Oct 22 09:32:47 2025" "Oct 21 16:13:20 2025" 8192 346880 0x130000e0 NEW_FILE
817239842 5157310236 -rwxrwxr-x 1 USER GROUP 18446744073709551615 812759123 "Sep 25 14:40:55 2025" "Sep 25 14:41:01 2025" "Oct 22 09:32:47 2025" "Sep 25 14:40:55 2025" 8192 1587664 0x130000e0 OLD_FILE
S3FS on Redhat 8
[user@server folder1]# stat NEW_FILE
File: NEW_FILE
Size: 4096 Blocks: 8 IO Block: 4096 directory
Device: 32h/50d Inode: 584 Links: 1
Access: (0775/drwxrwxr-x) Uid: (0/user) Gid: (0/group)
Access: 2025-10-21 14:51:32.000000000 -0400
Modify: 2025-10-21 14:51:32.000000000 -0400
Change: 2025-09-22 16:55:24.823147973 -0400
Birth: -
[user@server folder2]# stat OLD_FILE
File: NEW_FILE
Size: 812759123 Blocks: 1587421 IO Block: 4096 regular file
Device: 32h/50d Inode: 623 Links: 1
Access: (0775/-rwxrwxr-x) Uid: (0/user) Gid: (0/group)
Access: 2025-09-25 14:41:01.000000000 -0400
Modify: 2025-09-25 14:41:01.000000000 -0400
Change: 2025-09-25 14:40:55.908827706 -0400
Birth: -
NFS on Redhat 8
(base) [user@server folder1]# stat NEW_FILE
File: NEW_FILE
Size: 177552579 Blocks: 347360 IO Block: 32768 regular file
Device: 36h/54d Inode: 5178025004 Links: 1
Access: (0775/-rwxrwxr-x) Uid: (1/user) Gid: (1/group)
Access: 2025-10-21 14:51:25.358238000 -0400
Modify: 2025-10-21 14:51:32.395356000 -0400
Change: 2025-10-22 09:32:47.755887000 -0400
Birth: -
(base) [user@server folder2]# stat OLD_FILE
File: NEW_FILE
Size: 812759123 Blocks: 1587664 IO Block: 32768 regular file
Device: 36h/54d Inode: 5157310236 Links: 1
Access: (0775/-rwxrwxr-x) Uid: (1/user) Gid: (1/group)
Access: 2025-11-04 14:11:15.638702000 -0500
Modify: 2025-09-25 14:41:01.613539000 -0400
Change: 2025-10-22 09:32:47.873882000 -0400
Birth: -
@ggtakec commented on GitHub (Nov 9, 2025):
@zahnmax Thanks for your help.
But sorry, I couldn't identify the cause with these results.
If you have access to the AWS CLI, could you please tell me the results of running the following command?
For
<file path>, try specifyingOLD_FILE,OLD_FILE/,NEW_FILE, orNEW_FILE/(the file path or the file path with a/at the end).If you run the above command and receive results, we can check the headers (Content-Types and x-meta-*) of each object.