starred/s3fs-fuse

Fork 0

mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2026-04-25 05:16:00 +03:00

[GH-ISSUE #1566] [DO NOT] mount `non empty` directories (from `host` to `s3`). #822

New issue

Closed

opened 2026-03-04 01:49:05 +03:00 by kerem · 2 comments

kerem commented

2026-03-04 01:49:05 +03:00

Owner

Originally created by @MaxKumundzhiev on GitHub (Feb 12, 2021).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1566

Hello there ! Thank you in advance for the time and consideration !

All the mentioned names of directories and buckets are fictional and was used to explicitly focus on the issue itself.

Goal

The goal was to mount 3 dedicated directories from the host machine (MacOS) into 1 certain s3 bucket within 3 different prefixes, e.g.:

database_A (~ 200 MB) --> bucket/database/volume_A
database_B (~ 5 GB) --> bucket/database/volume_B
database_C (~ 40 GB) --> bucket/database/volume_C

Mandatory Condition : the initial directories (i.e database_A... database_C) are not empty

Installation

I successfully installed s3fs utilizing the steps denoted within s3fs README.md

Addition

Before moving forward with non empty directories I've expiremented with empty one and it was good so far.

$ s3fs sandbox-bucket /Users/.../.../database-test -o passwd_file=${HOME}/.passwd-s3fs

where:

sandbox-bucket - s3 bucket name
/Users/.../.../database-test - path to the empty directory
${HOME}/.passwd-s3fs - path to the file which stored the AWS credentials

Details about issue

I started to mount non empty directories one by one in ascending order.
The first candidate was executed as:

$ s3fs bucket/database/volume_A /Users/.../.../database_A -o passwd_file=${HOME}/.passwd-s3fs -o nonempty

Afterwards the Mac terminal freeze, even bit of data did not appear under s3 within 30 minutes so far, network bandwidth increased a lot, memory usage increased a lot as well (almost all RAM limit).

Eventually I even could not cope with the freezing of the machine and network bandwidth terminating the s3fs process. The certain way was to reboot the machine. I repeated the experiment with the exactly same steps 3 times --> same results --> rebooting machine.

Version of s3fs being used (s3fs --version)

1.87

Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse)

2.9.7

Kernel information (uname -r)

19.6.0

s3fs command line used, if applicable

$ s3fs sandbox-bucket /Users/.../.../database-test -o passwd_file=${HOME}/.passwd-s3fs ✔️ 
$ s3fs bucket/database/volume_A /Users/.../.../database_A -o passwd_file=${HOME}/.passwd-s3fs -o nonempty 🚫

Originally created by @MaxKumundzhiev on GitHub (Feb 12, 2021). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/1566 #### Hello there ! Thank you in advance for the time and consideration ! _`All the mentioned names of directories and buckets are fictional and was used to explicitly focus on the issue itself.`_ ### Goal The goal was to mount 3 dedicated directories from the host machine (MacOS) into 1 certain `s3` bucket within 3 different `prefixes`, e.g.: - `database_A` (~ 200 MB) --> `bucket/database/volume_A` - `database_B` (~ 5 GB) --> `bucket/database/volume_B` - `database_C` (~ 40 GB) --> `bucket/database/volume_C` **Mandatory Condition** : the initial directories (i.e `database_A`... `database_C`) are not empty ### Installation I successfully installed `s3fs` utilizing the steps denoted within s3fs README.md ### Addition Before moving forward with non empty directories I've expiremented with empty one and it was good so far. ```bash $ s3fs sandbox-bucket /Users/.../.../database-test -o passwd_file=${HOME}/.passwd-s3fs ``` where: - sandbox-bucket - s3 bucket name - /Users/.../.../database-test - path to the empty directory - ${HOME}/.passwd-s3fs - path to the file which stored the AWS credentials ### Details about issue I started to mount `non empty` directories one by one in ascending order. The first candidate was executed as: ```bash $ s3fs bucket/database/volume_A /Users/.../.../database_A -o passwd_file=${HOME}/.passwd-s3fs -o nonempty ``` Afterwards the Mac terminal freeze, even bit of data did not appear under s3 within 30 minutes so far, network bandwidth increased a lot, memory usage increased a lot as well (almost all RAM limit). Eventually I even could not cope with the freezing of the machine and network bandwidth terminating the s3fs process. The certain way was to reboot the machine. I repeated the experiment with the exactly same steps 3 times --> same results --> rebooting machine. #### Version of s3fs being used (s3fs --version) 1.87 #### Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse, dpkg -s fuse) 2.9.7 #### Kernel information (uname -r) 19.6.0 #### s3fs command line used, if applicable ``` $ s3fs sandbox-bucket /Users/.../.../database-test -o passwd_file=${HOME}/.passwd-s3fs ✔️ $ s3fs bucket/database/volume_A /Users/.../.../database_A -o passwd_file=${HOME}/.passwd-s3fs -o nonempty 🚫 ```

kerem

2026-03-04 01:49:05 +03:00

closed this issue
added the
need info
label

kerem commented

2026-03-04 01:49:06 +03:00

Author

Owner

@CarstenGrohmann commented on GitHub (Feb 18, 2021):

The s3fs man page mentions a colon between bucket and path:

s3fs bucket[:/path] mountpoint [options]

I miss this in your example. Is it just a typo?

@CarstenGrohmann commented on GitHub (Feb 18, 2021): The s3fs man page mentions a colon between bucket and path: ``` s3fs bucket[:/path] mountpoint [options] ``` I miss this in your example. Is it just a typo?

kerem commented

2026-03-04 01:49:06 +03:00

Author

Owner

@gaul commented on GitHub (Apr 21, 2021):

Please try the suggested fix and reopen if symptoms persist.

@gaul commented on GitHub (Apr 21, 2021): Please try the suggested fix and reopen if symptoms persist.

kerem referenced this issue

2026-03-04 02:01:33 +03:00

[PR #852] [MERGED] Allow credentials from ${HOME}/.aws/credentials #1661