starred/s3fs-fuse

Fork 0

mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2026-04-25 21:35:58 +03:00

[GH-ISSUE #813] FUSE request size #470

New issue

Closed

opened 2026-03-04 01:45:53 +03:00 by kerem · 7 comments

kerem commented

2026-03-04 01:45:53 +03:00

Owner

Originally created by @kunallillaney on GitHub (Aug 30, 2018).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/813

Is there a way to change the FUSE request size for S3FS? As of now, it seems that s3fs-fuse only supports 4KB request sizes in FUSE. But, libfuse does allows large I/O sized requests, up to 128KB (Reference).

Is there a plan on the roadmap to expose this as a user-defined parameter?

Originally created by @kunallillaney on GitHub (Aug 30, 2018). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/813 Is there a way to change the FUSE request size for S3FS? As of now, it seems that s3fs-fuse only supports 4KB request sizes in FUSE. But, libfuse does allows large I/O sized requests, up to 128KB ([Reference](http://libfuse.github.io/doxygen/fast17-vangoor.pdf)). Is there a plan on the roadmap to expose this as a user-defined parameter?

kerem closed this issue

2026-03-04 01:45:54 +03:00

kerem commented

2026-03-04 01:45:55 +03:00

Author

Owner

@kahing commented on GitHub (Sep 11, 2018):

Do have you logs that show this? For example, if you do dd if=/dev/zero of=/mnt/file.out bs=1M count=1, and s3fs -o dbglevel=info, do you see that s3fs is getting a smaller write size than expected?

@kahing commented on GitHub (Sep 11, 2018): Do have you logs that show this? For example, if you do `dd if=/dev/zero of=/mnt/file.out bs=1M count=1`, and `s3fs -o dbglevel=info`, do you see that s3fs is getting a smaller write size than expected?

kerem commented

2026-03-04 01:45:55 +03:00

Author

Owner

@kunallillaney commented on GitHub (Sep 11, 2018):

Here is the screenshot. I ran slightly different commands then you mentioned to enable the debug mode in FUSE: s3fs kunalfs /data/testfs -o parallel_count=4 -o multipart_si16 -o use_cache=/data/ramdisk -o debug and dd if=/dev/zero of=file.out bs=1M count=100.

AFAIK, there is no parameter or constant in S3FS which allows us to change this.

@kunallillaney commented on GitHub (Sep 11, 2018): <img width="637" alt="screen shot 2018-09-10 at 11 55 07 pm" src="https://user-images.githubusercontent.com/1380214/45337251-0f662480-b555-11e8-8c06-e75e25f9e5aa.png"> Here is the screenshot. I ran slightly different commands then you mentioned to enable the debug mode in FUSE: ```s3fs kunalfs /data/testfs -o parallel_count=4 -o multipart_si16 -o use_cache=/data/ramdisk -o debug``` and ```dd if=/dev/zero of=file.out bs=1M count=100```. AFAIK, there is no parameter or constant in S3FS which allows us to change this.

kerem commented

2026-03-04 01:45:55 +03:00

Author

Owner

@kunallillaney commented on GitHub (Sep 11, 2018):

@kahing goofys uses 128KB (even though it seems 1MB is specified?). Here are the logs for goofys
screen shot 2018-09-11 at 1 10 33 am

@kunallillaney commented on GitHub (Sep 11, 2018): @kahing goofys uses 128KB (even though it seems 1MB is specified?). Here are the logs for goofys <img width="639" alt="screen shot 2018-09-11 at 1 10 33 am" src="https://user-images.githubusercontent.com/1380214/45339256-a3d58480-b55f-11e8-91ce-3f450770cd77.png">

kerem commented

2026-03-04 01:45:55 +03:00

Author

Owner

@kahing commented on GitHub (Sep 11, 2018):

github.com/jacobsa/fuse@cd3959611b/connection.go (L164) enabled the equivalent big write for goofys

@kahing commented on GitHub (Sep 11, 2018): https://github.com/jacobsa/fuse/blob/cd3959611bcb6bb23d49f1598d98f2bf44476805/connection.go#L164 enabled the equivalent big write for goofys

kerem commented

2026-03-04 01:45:55 +03:00

Author

Owner

@kunallillaney commented on GitHub (Sep 11, 2018):

@kahing My comments did imply that it was enabled for goofys thus the 128KB. FUSE does not allow you any larger I/O sizes (even though I was making 1MB block size calls).

@kunallillaney commented on GitHub (Sep 11, 2018): @kahing My comments did imply that it was enabled for goofys thus the 128KB. FUSE does not allow you any larger I/O sizes (even though I was making 1MB block size calls).

kerem commented

2026-03-04 01:45:55 +03:00

Author

Owner

@kahing commented on GitHub (Sep 11, 2018):

Understood, I am merely pointing out where it's set

@kahing commented on GitHub (Sep 11, 2018): Understood, I am merely pointing out where it's set

kerem commented

2026-03-04 01:45:55 +03:00

Author

Owner

@ggtakec commented on GitHub (Sep 17, 2018):

@kunallillaney @kahing I'm sorry for my late reply.
I merged #820 just now.

@ggtakec commented on GitHub (Sep 17, 2018): @kunallillaney @kahing I'm sorry for my late reply. I merged #820 just now.