mirror of
https://github.com/s3fs-fuse/s3fs-fuse.git
synced 2026-04-25 13:26:00 +03:00
[GH-ISSUE #67] S3FS and versioning support? #37
Labels
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/s3fs-fuse#37
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @hetzbh on GitHub (Oct 13, 2014).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/67
Hi,
I'm just wondering - is there (or will there be) any support for Amazon S3 versioning? (http://docs.aws.amazon.com/AmazonS3/latest/dev/Versioning.html) It would be nice not to loose files due to stupid mistakes ;)
Thanks
@ggtakec commented on GitHub (Oct 13, 2014):
Hi,
s3fs does not support versioning(x-amz-versionid header, /?versions request, etc.)
Now s3fs does nothing about versioning, so user only sets versioning attribute to their bucket on AWS console. s3fs reads/writes/does only latest objects.
If s3fs supported versioning, I thought the user interface was very difficult. because same name many object(file)s is in directory(under mount point).
Regards,
@sqlbot commented on GitHub (Apr 15, 2015):
@hetzbh to clarify this point, even though s3fs does not "support" versioning, s3fs still _does work correctly_ on buckets with versioning enabled because Amazon designed versioning to work transparently with clients (like s3fs) that are not aware of versioning, and you get the advantanges of versioning that you mentioned.
That's how I am using it, and for the same reason. You can't see old versions the objects or delete markers in the mounted "filesystem," because s3fs doesn't see them when it interacts with the S3 API using non-version-aware calls -- but the object versions are still accessible via the AWS console and the REST API... so you have the ability to recover from inadvertent overwrites, deletions, etc., that is a significant advantage of bucket versioning. It would probably have been a terrible mistake for s3fs to have attempted to actually "support" versioning in the sense of trying to expose the versioned objects through fuse.
The only caveat is, any change to a stored object (chown, chmod, chgrp, touch, rename, move) creates a whole new version of the object with the new attributes, because it changes the metadata. This happens because that's how versioning works -- the same thing happens when you edit metadata in the AWS console with a versioned bucket, and the "Versions: Hide" button is selected -- it makes a new version of the object, with the new metadata, leaving the previous "version" as it was).
This means eventually you'll want to clean things up, manually or with a script or background job... or you'll want to use bucket lifecycle policies to delete obsolete versions after some period of time. To me, the potential extra storage cost seems justified by the reduced risk. I have not had any issues since I began using it this way.
@ggtakec commented on GitHub (Jan 17, 2016):
I'm sorry for that this issue had been left at a long period of time.
If you have this problem yet, please try to use latest code.
Thanks in advance for your help.
@api-py commented on GitHub (Mar 30, 2023):
Hi @ggtakec ,
Could you confirm, if s3fs supports retrieval (read) of versioned objects? I am struggling with lack of technical documentation.
Thank you
@ggtakec commented on GitHub (Apr 3, 2023):
@plaformsre
You can see following comments in issues(one of them is this issue's comment and it is very clear explanation.)
https://github.com/s3fs-fuse/s3fs-fuse/issues/67#issuecomment-93549104
https://github.com/s3fs-fuse/s3fs-fuse/issues/91#issuecomment-68879200