[GH-ISSUE #743] How to make the performance turning for the big data files? #425

New issue

Closed

opened 2026-03-04 01:45:27 +03:00 by kerem · 3 comments

kerem commented

2026-03-04 01:45:27 +03:00

Owner

Originally created by @wangf8406 on GitHub (Apr 4, 2018).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/743

Version of s3fs being used (s3fs --version)

_example: 1.87

Details about issue

Hi there, we now working on backup the big files (usually 200G to 400G), how to configure the options to make the S3FS have best performance ? currently the connection bandwidth is enough to get the best transport performance? the main scenario is to transfer the big files .

Originally created by @wangf8406 on GitHub (Apr 4, 2018). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/743 #### Version of s3fs being used (s3fs --version) _example: 1.87 ### Details about issue Hi there, we now working on backup the big files (usually 200G to 400G), how to configure the options to make the S3FS have best performance ? currently the connection bandwidth is enough to get the best transport performance? the main scenario is to transfer the big files .

kerem closed this issue

2026-03-04 01:45:27 +03:00

kerem commented

2026-03-04 01:45:28 +03:00

Author

Owner

@gaul commented on GitHub (Jan 26, 2019):

You should try tuning the -o multipart_size flag since its default 10 MB is low for large files. This should influence both read and write speeds. Please share the results of your test which other users will value!

@gaul commented on GitHub (Jan 26, 2019): You should try tuning the `-o multipart_size` flag since its default 10 MB is low for large files. This should influence both read and write speeds. Please share the results of your test which other users will value!

kerem commented

2026-03-04 01:45:28 +03:00

Author

Owner

@jonassvatos commented on GitHub (Feb 18, 2019):

I can confirm a huge increase in read performance after increasing the part size.
In our case, 300% increase on large files (hundreds of GBs) with -o multipart_size=100 using MINIO as a backend.

@jonassvatos commented on GitHub (Feb 18, 2019): I can confirm a huge increase in read performance after increasing the part size. In our case, 300% increase on large files (hundreds of GBs) with `-o multipart_size=100` using MINIO as a backend.

kerem commented

2026-03-04 01:45:28 +03:00

Author

Owner

@gaul commented on GitHub (Jun 25, 2019):

#941 tracks automatically tuning the part size.

@gaul commented on GitHub (Jun 25, 2019): #941 tracks automatically tuning the part size.

kerem referenced this issue

2026-03-04 02:00:46 +03:00

[PR #425] [MERGED] Updated ChangeLog and configure.ac for release 1.80 #1503