[GH-ISSUE #380] Share same bucket s3fs v1.61 with current master #198

New issue

Closed

opened 2026-03-04 01:43:10 +03:00 by kerem · 2 comments

kerem commented

2026-03-04 01:43:10 +03:00

Owner

Originally created by @chrisgo on GitHub (Mar 26, 2016).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/380

We have some old servers using v1.61 of s3fs loaded via /etc/fstab (debian 7)

s3fs#s3fs-test.example.com  /mnt/s3fs-test  fuse    use_cache=/tmp,allow_other  0   0   0

On old server with 1.61 code, doing the following and permissions (non-root user)

cd /mnt
ls -la
drwxr-xr-x  1 root root    0 Jan  1  1970 s3fs-test

cd s3fs-test
mkdir old
ls -la
drwxr-xr-x 1 root root 0 Mar 26 00:24 old

cd old
touch file.txt
vi file.txt
ls -la
-rw-r--r-- 1 root root 12 Mar 26 00:24 file.txt

On the Amazon S3 Console (https://aws.amazon.com), it is all there. On the 1.61 version of the code, there is a "duplicate" file (in this case old) that has a lot of properties of the old folder sitting beside it. The old folder itself only has a Details section

Obviously, this is super old code and the goal is to use s3fs "master" on new servers. However, every time we try to mount a bucket on a server using s3fs with version > 1.61, we could not get the mounts to work together properly.

On the new server (debian 8 with s3fs "master"), we are able to mount it with

s3fs#s3fs-test.example.com  /mnt/s3fs-test  fuse    _netdev,allow_other 0   0   0

Non-root user

cd /mnt
ls -la
drwxrwxrwx  1 root root    0 Jan  1  1970 s3fs-test

cd s3fs-test
ls -la
drwxr-xr-x 1 root    root       0 Mar 26 00:24 old

cd old
ls -la
-rw-r--r-- 1 root root 12 Mar 26 00:24 file.txt
more file.txt (also works)

cd ..
mkdir new
ls -la
drwxr-xr-x 1 vagrant vagrant    0 Mar 26 00:35 new   (note: "vagrant" is the current user)
cd new
touch file.txt
vi file.txt
-rw-r--r-- 1 vagrant vagrant 19 Mar 26 00:36 file.txt

On the AWS S3 Console, I noticed the new "master" code does NOT create a secondary entry in the file system (so there is no metadata). The new folder looks the same.

Things that don't work

On new server, cannot create files or directory on folders created by s3fs v1.61

cd /mnt/s3fs-test/dsr
touch file2.txt
touch: cannot touch ‘file2.txt’: Permission denied

On old server, cannot see folders created by new "master" code

cd /mnt/s3fs-test
ls -la 
drwxr-xr-x 1 root root 0 Mar 26 00:24 old

I am thinking this is because of the missing second file?

Are there any settings we can do on the old (1.61) and new servers (master) so that the directories and files can co-exist with each other?

Some settings in /etc/fstab that involves gid, uid, umask but not really sure how to proceed

Or am I pretty much out of luck? What would be the proper way to migrate from an old s3fs v1.61 mounted/created bucket to a new one while keeping them both operating?

Originally created by @chrisgo on GitHub (Mar 26, 2016). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/380 We have some old servers using v1.61 of s3fs loaded via /etc/fstab (debian 7) ``` s3fs#s3fs-test.example.com /mnt/s3fs-test fuse use_cache=/tmp,allow_other 0 0 0 ``` On old server with 1.61 code, doing the following and permissions (non-root user) ``` cd /mnt ls -la drwxr-xr-x 1 root root 0 Jan 1 1970 s3fs-test cd s3fs-test mkdir old ls -la drwxr-xr-x 1 root root 0 Mar 26 00:24 old cd old touch file.txt vi file.txt ls -la -rw-r--r-- 1 root root 12 Mar 26 00:24 file.txt ``` On the Amazon S3 Console (https://aws.amazon.com), it is all there. On the 1.61 version of the code, there is a "duplicate" file (in this case `old`) that has a lot of properties of the `old` folder sitting beside it. The `old` folder itself only has a `Details` section Obviously, this is super old code and the goal is to use s3fs "master" on new servers. However, every time we try to mount a bucket on a server using s3fs with version > 1.61, we could not get the mounts to work together properly. On the new server (debian 8 with s3fs "master"), we are able to mount it with ``` s3fs#s3fs-test.example.com /mnt/s3fs-test fuse _netdev,allow_other 0 0 0 ``` Non-root user ``` cd /mnt ls -la drwxrwxrwx 1 root root 0 Jan 1 1970 s3fs-test cd s3fs-test ls -la drwxr-xr-x 1 root root 0 Mar 26 00:24 old cd old ls -la -rw-r--r-- 1 root root 12 Mar 26 00:24 file.txt more file.txt (also works) cd .. mkdir new ls -la drwxr-xr-x 1 vagrant vagrant 0 Mar 26 00:35 new (note: "vagrant" is the current user) cd new touch file.txt vi file.txt -rw-r--r-- 1 vagrant vagrant 19 Mar 26 00:36 file.txt ``` On the AWS S3 Console, I noticed the new "master" code does NOT create a secondary entry in the file system (so there is no metadata). The `new` folder looks the same. Things that don't work 1. On new server, cannot create files or directory on folders created by s3fs v1.61 ``` cd /mnt/s3fs-test/dsr touch file2.txt touch: cannot touch ‘file2.txt’: Permission denied ``` 1. On old server, cannot see folders created by new "master" code ``` cd /mnt/s3fs-test ls -la drwxr-xr-x 1 root root 0 Mar 26 00:24 old ``` I am thinking this is because of the missing second file? --- Are there any settings we can do on the old (1.61) and new servers (master) so that the directories and files can co-exist with each other? Some settings in /etc/fstab that involves `gid`, `uid`, `umask` but not really sure how to proceed Or am I pretty much out of luck? What would be the proper way to migrate from an old s3fs v1.61 mounted/created bucket to a new one while keeping them both operating?

kerem closed this issue

2026-03-04 01:43:10 +03:00

kerem commented

2026-03-04 01:43:11 +03:00

Author

Owner

@ggtakec commented on GitHub (Apr 10, 2016):

@chrisgo
s3fs older than 1.62 makes directory name object("dir") for directory.
But s3fs after 1.61 make directory name + "/" object("dir/") instead of it.

Latest s3fs is able to deal with "dir" type object for compatibility, but old s3fs can not handle "dir/".
Maybe, this problem is due to the handling of the directory object, since older s3fs does not have a forward-compatible to the s3fs after 1.61.

How to solve is to rename object name as "dir" to "dir/" by such as s3cmd.
(without changing object attributes)

But I think this method is not recommended.
If you can, we want to upgrade s3fs from 1.61 to latest version.

Thanks in advance for your assistance.

@ggtakec commented on GitHub (Apr 10, 2016): @chrisgo s3fs older than 1.62 makes directory name object("dir") for directory. But s3fs after 1.61 make directory name + "/" object("dir/") instead of it. Latest s3fs is able to deal with "dir" type object for compatibility, but old s3fs can not handle "dir/". Maybe, this problem is due to the handling of the directory object, since older s3fs does not have a forward-compatible to the s3fs after 1.61. How to solve is to rename object name as "dir" to "dir/" by such as s3cmd. (without changing object attributes) But I think this method is not recommended. If you can, we want to upgrade s3fs from 1.61 to latest version. Thanks in advance for your assistance.

kerem commented

2026-03-04 01:43:11 +03:00

Author

Owner

@chrisgo commented on GitHub (May 5, 2016):

@ggtakec
Thanks for your response. The main issue I have is that if I upgrade the s3fs latest, I will not be able to write to any existing folders ... more thinking

@chrisgo commented on GitHub (May 5, 2016): @ggtakec Thanks for your response. The main issue I have is that if I upgrade the s3fs latest, I will not be able to write to any existing folders ... more thinking

kerem referenced this issue

2026-03-04 01:53:43 +03:00

[PR #198] [MERGED] Disasble integration tests for Travis #1392