[GH-ISSUE #730] Existing metadata clobbered on folders when using chown/chmod #415

New issue

Closed

opened 2026-03-04 01:45:23 +03:00 by kerem · 3 comments

kerem commented

2026-03-04 01:45:23 +03:00

Owner

Originally created by @gritnix on GitHub (Mar 9, 2018).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/730

s3fs Version: 1.83 (commit e1dafe7)
fuse version 2.9.2-8
uname -r : 3.10.0-693.17.1.el7.x86_64

Bucket mounted with
s3fs /mountpoint -o iam_role=auto

Preserving existing metadata on files works fine through chown and chmod, but on a directory in an S3 bucket, this is not true. A directory is just an object with a slash at the end of its name and a ContentType of application/x-directory. To duplicate the issue, run the following, create the directory with the aws CLI.

aws s3api put-object --bucket --key myFolder/ --content-type="application/x-directory" --metadata mycustominfo="This is my metadata"

You can see the metadata is in place with
aws s3api get-object --bucket --key myFolder/ /dev/null

If doing an ls in the mounted bucket, the directory object will be owned by root because it has no uid/mode/etc in the metadata. Now as root, do a chmod or a chown of the directory.

chmod 750 /mountpoint/myFolder

The permissions change as directed. Now run the same get-object command again and the custom metadata is gone having been replaced by uid, gid, mtime, and mode.

Doing this on a file preserves the existing and adds what's needed, but on a directory/folder it's overwritten.

Originally created by @gritnix on GitHub (Mar 9, 2018). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/730 s3fs Version: 1.83 (commit e1dafe7) fuse version 2.9.2-8 uname -r : 3.10.0-693.17.1.el7.x86_64 Bucket mounted with s3fs <bucket> /mountpoint -o iam_role=auto Preserving existing metadata on files works fine through chown and chmod, but on a directory in an S3 bucket, this is not true. A directory is just an object with a slash at the end of its name and a ContentType of application/x-directory. To duplicate the issue, run the following, create the directory with the aws CLI. aws s3api put-object --bucket <bucketname> --key myFolder/ --content-type="application/x-directory" --metadata mycustominfo="This is my metadata" You can see the metadata is in place with aws s3api get-object --bucket <bucketname> --key myFolder/ /dev/null If doing an ls in the mounted bucket, the directory object will be owned by root because it has no uid/mode/etc in the metadata. Now as root, do a chmod or a chown of the directory. chmod 750 /mountpoint/myFolder The permissions change as directed. Now run the same get-object command again and the custom metadata is gone having been replaced by uid, gid, mtime, and mode. Doing this on a file preserves the existing and adds what's needed, but on a directory/folder it's overwritten.

kerem closed this issue

2026-03-04 01:45:23 +03:00

kerem commented

2026-03-04 01:45:24 +03:00

Author

Owner

@gaul commented on GitHub (Jan 26, 2019):

Note that s3fs preserves metadata on files, but not for directory files:

$ aws s3api put-object --bucket bucket --key key --metadata mycustominfo="This is my metadata"
{   
    "ETag": "\"d41d8cd98f00b204e9800998ecf8427e\""
}

$ aws s3api get-object --bucket bucket --key key /dev/null
{   
    "AcceptRanges": "bytes",
    "LastModified": "Sat, 26 Jan 2019 00:32:59 GMT",
    "ContentLength": 0,
    "ETag": "\"d41d8cd98f00b204e9800998ecf8427e\"",
    "ContentType": "binary/octet-stream",
    "Metadata": {
        "mycustominfo": "This is my metadata"
    }
}

$ chmod 750 work/s3fs-fuse/mnt/key

$ aws s3api get-object --bucket bucket --key key /dev/null
{
    "AcceptRanges": "bytes",
    "LastModified": "Sat, 26 Jan 2019 00:38:50 GMT",
    "ContentLength": 0,
    "ETag": "\"d41d8cd98f00b204e9800998ecf8427e\"",
    "ContentType": "binary/octet-stream",
    "Metadata": {
        "ctime": "1548463129",
        "mode": "33256",
        "mycustominfo": "This is my metadata"
    }
}

@gaul commented on GitHub (Jan 26, 2019): Note that s3fs preserves metadata on files, but not for directory files: ```ShellSession $ aws s3api put-object --bucket bucket --key key --metadata mycustominfo="This is my metadata" { "ETag": "\"d41d8cd98f00b204e9800998ecf8427e\"" } $ aws s3api get-object --bucket bucket --key key /dev/null { "AcceptRanges": "bytes", "LastModified": "Sat, 26 Jan 2019 00:32:59 GMT", "ContentLength": 0, "ETag": "\"d41d8cd98f00b204e9800998ecf8427e\"", "ContentType": "binary/octet-stream", "Metadata": { "mycustominfo": "This is my metadata" } } $ chmod 750 work/s3fs-fuse/mnt/key $ aws s3api get-object --bucket bucket --key key /dev/null { "AcceptRanges": "bytes", "LastModified": "Sat, 26 Jan 2019 00:38:50 GMT", "ContentLength": 0, "ETag": "\"d41d8cd98f00b204e9800998ecf8427e\"", "ContentType": "binary/octet-stream", "Metadata": { "ctime": "1548463129", "mode": "33256", "mycustominfo": "This is my metadata" } } ```

kerem commented

2026-03-04 01:45:24 +03:00

Author

Owner

@ggtakec commented on GitHub (Mar 29, 2019):

@gritnix I'm sorry for my late reply.
I was able to reproduce this bug.
This only happens with directory objects (dir/).

Although s3fs treats the directory as dir/, s3fs has special processing which is performed for compatibility of the directory object made by other tools.
When updating directory stats information, if it is a directory other than dir/ type, delete the object once and create it again.
There was an error in the judgment of this part, causing a problem.
The defects will be corrected immediately and reflected in the master.

@ggtakec commented on GitHub (Mar 29, 2019): @gritnix I'm sorry for my late reply. I was able to reproduce this bug. This only happens with directory objects (`dir/`). Although s3fs treats the directory as `dir/`, s3fs has special processing which is performed for compatibility of the directory object made by other tools. When updating directory stats information, if it is a directory other than `dir/` type, delete the object once and create it again. There was an error in the judgment of this part, causing a problem. The defects will be corrected immediately and reflected in the master.

kerem commented

2026-03-04 01:45:24 +03:00

Author

Owner

@ggtakec commented on GitHub (Mar 29, 2019):

@gritnix @gaul This bug is fixed by #993
Please try to use master branch code.
I will close this, but if the problem persists, please reopen or post a new issue.

@ggtakec commented on GitHub (Mar 29, 2019): @gritnix @gaul This bug is fixed by #993 Please try to use master branch code. I will close this, but if the problem persists, please reopen or post a new issue.

kerem referenced this issue

2026-03-04 01:54:15 +03:00

[PR #415] [MERGED] Fixed a bug about stat_cache_expire - #382 #1499