mirror of
https://github.com/s3fs-fuse/s3fs-fuse.git
synced 2026-04-25 13:26:00 +03:00
[GH-ISSUE #623] Can I mount a folder within a bucket instead of the root bucket? #355
Labels
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/s3fs-fuse#355
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @quadgnim on GitHub (Jul 19, 2017).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/623
Hi all, I'm successfully mounting a bucket and through it I have access to all my files and folders. However, in some cases I just want to mount the folder and not provide the client access to everything else in the bucket. Creating separate buckets is less ideal than placing things into folders and mounting a specific folder. However, I can't seem to figure out a way to do this?
Current mount command using fstab =
3fs#quadgnimsbucket /tmp/s3_mount fuse _netdev,iam_role=auto,url=https://s3.amazonaws.com,use_cache=/tmp,allow_other,nonempty,use_sse,endpoint=us-east-1
Within quadgnimsbucket is a folder called mytestfolder
Any tricks on how to mount the specific folder?
Additional Information
The following information is very important in order to help us to help you. Omission of the following details may delay your support request or receive no attention at all.
Version of s3fs being used (s3fs --version)
V1.82
Version of fuse being used (pkg-config --modversion fuse)
2.9.7
System information (uname -a)
2.6.32-696.3.2.el6.x86_64
@gaul commented on GitHub (Jul 22, 2017):
https://github.com/s3fs-fuse/s3fs-fuse/wiki/Fuse-Over-Amazon#prefix-default-coming-soon
@tzahimizrahi commented on GitHub (Sep 24, 2017):
@quadgnim did you manage to to mount s3 folder?
I am also curious about it
@quadgnim commented on GitHub (Sep 24, 2017):
Yes, use the format
S3fs#bucket:/folder if memory serves
Maybe two slashes //
Thanks,
Larry
On Sep 24, 2017 2:22 PM, "tzahimizrahi" notifications@github.com wrote:
@shivkaundal commented on GitHub (Aug 23, 2018):
Hello,
is this issue closed?
@darthvadrismydad commented on GitHub (Oct 8, 2018):
This doesn't work for me. when I mount a sub-folder within a bucket, I get:
[CRT] s3fs.cpp:s3fs_check_service(3784): bucket not found(host=https://s3.amazonaws.com) - result of checking service.When I just mount the bucket, the mounting works but I cannot view anything within the sub-folders. I just get a 'Operation not permitted':
ls: cannot open directory my-bucket/data: Operation not permittedEDIT: System Data
OS: CentoOS (Amazon Linux)
Installation Method: Clone Git & Run Make
SSE/KMS Used: Yes (managed key)
IAM Role: Set to "auto"
@haghabozorgi commented on GitHub (Oct 11, 2018):
@InspiredIdealist it only seems to work with dirs that were created by fuse. so if you mount a bucket, create a 'dir' (e.g myfusedir) inside that bucket then umount bucket and remount that (myfusedir) dir you created it works.
If you create the dir outside of fuse (e.g aws cli) and try to mount it, it does not seem to work. I think it is related to https://github.com/s3fs-fuse/s3fs-fuse/wiki/FAQ#q-could-not-access-filesdirectories-by-permission-denied.
I am trying to confirm and find a solution.
I believe the solution might be to run https://github.com/s3fs-fuse/s3fs-fuse/blob/master/test/mergedir.sh but have not tested yet.@darthvadrismydad commented on GitHub (Oct 14, 2018):
Ah ok, understood. So as a workaround, I just have to make sure we apply those metadata tags first to any given file in the sub-directory of choice within a given bucket.
@haghabozorgi commented on GitHub (Oct 15, 2018):
@InspiredIdealist thats what it seems like but I was not able to get it working. So far if i create a "folder" in s3 bucket using aws management console mounting to folder works as expected, if i use aws cli or terraform for example, to create the "folder", it does not seem to work, even if i then go and add the metadata tags. Let me know if you have any luck.
@darthvadrismydad commented on GitHub (Oct 23, 2018):
Hmm that is strange, I haven't had a chance to try the metadata tags out. I will report back when I have a chance to try it out.
@APAC-DevOps commented on GitHub (Feb 25, 2019):
Thanks for the sharing.
I encountered a similar issue on my end as well.
I have an AWS SFTP, if I created SFTP User with optional home dir, and logged in to my AWS SFTP with that SFTP user, then, sub-folder would be created automatically under the designated S3 bucket.
however, when I tried to mount to that directory from my EC2 instance, it could not get through.
I had to manually create folder for my SFTP user from AWS S3 console, then, do dir mounting from my EC2 instance.
This seems to be a bug in AWS. Did you ever get any update from AWS about this issue?
@ggtakec commented on GitHub (Mar 29, 2019):
We kept this issue open for a long time.
The problem is that the path is not a directory object s3fs can recognize.
You need to specify a path that s3fs can recognize as a directory.
In order for s3fs to recognize an object (path) as a directory, you can create a directory using s3fs or update the information in the directory.
I added about this issue to wiki page:
https://github.com/s3fs-fuse/s3fs-fuse/wiki/FAQ#q-could-not-mount-with-specifying-the-path-under-bucket
I will close this, but if the problem persists, please reopen or post a new issue.
Thanks in advance for your assistance.
@APAC-DevOps commented on GitHub (Mar 29, 2019):
may be you might want to tell users in public community that "folder" object created via AWS web GUI does not have this issue. This seems to be easier than the mounting-and-remounting method mentioned in the link above.
@Wang-Kai commented on GitHub (May 15, 2020):
It is not workable with minio .
@sgandon commented on GitHub (Aug 7, 2020):
@Wang-Kai , It works fine with minio, I am using
mybucket:/subfolderin the s3fs command.You have to make sure the folder is not empty otherwise the mount succedes but no file can be added or read to/from the mount.
@nikiladonya commented on GitHub (Aug 19, 2020):
@sgandon do you use standalone or distributed minio configuration?
@jonassvatos commented on GitHub (Aug 30, 2020):
I can confirm that mounting subdir doesn't work against standalone MINIO.RELEASE.2020-08-27T05-16-20Z
s3fs log:
aug 30 00:00:00 localhost s3fs[8905]: s3fs.cpp:s3fs_check_service(3780): bucket not found(host=http://localhost:9000) - result of checking service.
/etc/fstab entry:
s3fs#bucket:/directory/ /mnt/test-s3fs fuse _netdev,rw,allow_other,noatime,use_path_request_style,uid=1000,gid=1000,umask=0002,passwd_file=/etc/s3cred,multipart_size=5,url=http://localhost:9000/ 0 0
MINIO trace log shows:
00:00:00.000 [404 Not Found] s3.GetObject localhost:9000/bucket/directory/ localhost 1.714ms ↑ 84 B ↓ 592 B
@sgandon commented on GitHub (Aug 31, 2020):
@nikiladonya I was using the standalone version.
@dvaldivia commented on GitHub (Apr 5, 2021):
seems to not be working with MinIO because
s3fstries to perform aGetObjectoperation on the subpath, for example with bucketinfluxdband subpath:/pod-0I see the 404 that @InspiredIdealist reportedfrom
mc admin traceIs there an expected response from the S3 store the
s3fsis looking for specifically? even if the sub-folder exists on my MinIO setup, the getObject will fail every time@dvaldivia commented on GitHub (Apr 5, 2021):
I think probing the prefix via
ListObjectsV2might be a better solution to probe access to the desired "folder"