starred/s3fs-fuse

Fork 0

mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2026-04-25 13:26:00 +03:00

[GH-ISSUE #791] Uploading large files over samba to s3fs mount. #456

New issue

Closed

opened 2026-03-04 01:45:42 +03:00 by kerem · 4 comments

kerem commented

2026-03-04 01:45:42 +03:00

Owner

Originally created by @matthewmedway on GitHub (Jul 6, 2018).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/791

Additional Information

The following information is very important in order to help us to help you. Omission of the following details may delay your support request or receive no attention at all.

Version of s3fs being used (s3fs --version)

V1.83(commit:3bf05da)

Version of fuse being used (pkg-config --modversion fuse)

2.9.4

System information (uname -r)

4.4.0-1062-aws

Distro (cat /etc/issue)

Ubuntu 16.04.4 LTS \n \l

s3fs command line used (if applicable)

N/A

/etc/fstab entry (if applicable):

bucketnamehere /mnt/sftp fuse.s3fs _netdev,iam_role=roleanme here,allow_other,use_cache=/tmp,gid=sftp,umask=002,nonempty,complement_stat,dbglevel=debug 0 0

s3fs syslog messages (grep s3fs /var/log/syslog, or s3fs outputs)

if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages

N/A

Details about issue

Hello, I'm attempting to export the s3 bucket i have as a samba share to my windows servers but i'm running into issues uploading larger files. When I transfer a ~3gb file to the samba share windows will hang at 0% complete transfer and eventually die off with error : "0x8007003B: an unexpected network error has occurred". After this error occurs if i check the syslog for s3fs logs it shows that the file upload is still in progress to s3. If i wait long enough the upload completes and i am able to refresh the samba share and the file actually is there.

What i believe to be happening is that while the file is being uploaded to the s3fs server the 0 byte files that s3fs creates at the start is causing samba to report the 0% transfer complete to windows. As soon as the entire file is written to the s3fs cache then the actual upload to s3 then takes place and only then does windows see the usual green progress bar. if i set the smb session timeout on windows to 5min (default 60s) mins then error does not occur.

Is there any way i can configure s3fs or maybe even samba to report that the file upload to cache is actually working instead of throwing this false alert?

Originally created by @matthewmedway on GitHub (Jul 6, 2018). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/791 ### Additional Information _The following information is very important in order to help us to help you. Omission of the following details may delay your support request or receive no attention at all._ #### Version of s3fs being used (s3fs --version) V1.83(commit:3bf05da) #### Version of fuse being used (pkg-config --modversion fuse) 2.9.4 #### System information (uname -r) 4.4.0-1062-aws #### Distro (cat /etc/issue) Ubuntu 16.04.4 LTS \n \l #### s3fs command line used (if applicable) ``` N/A ``` #### /etc/fstab entry (if applicable): ``` bucketnamehere /mnt/sftp fuse.s3fs _netdev,iam_role=roleanme here,allow_other,use_cache=/tmp,gid=sftp,umask=002,nonempty,complement_stat,dbglevel=debug 0 0 ``` #### s3fs syslog messages (grep s3fs /var/log/syslog, or s3fs outputs) _if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages_ ``` N/A ``` ### Details about issue Hello, I'm attempting to export the s3 bucket i have as a samba share to my windows servers but i'm running into issues uploading larger files. When I transfer a ~3gb file to the samba share windows will hang at 0% complete transfer and eventually die off with error : "0x8007003B: an unexpected network error has occurred". After this error occurs if i check the syslog for s3fs logs it shows that the file upload is still in progress to s3. If i wait long enough the upload completes and i am able to refresh the samba share and the file actually is there. What i believe to be happening is that while the file is being uploaded to the s3fs server the 0 byte files that s3fs creates at the start is causing samba to report the 0% transfer complete to windows. As soon as the entire file is written to the s3fs cache then the actual upload to s3 then takes place and only then does windows see the usual green progress bar. if i set the smb session timeout on windows to 5min (default 60s) mins then error does not occur. Is there any way i can configure s3fs or maybe even samba to report that the file upload to cache is actually working instead of throwing this false alert?

kerem closed this issue

2026-03-04 01:45:43 +03:00

kerem commented

2026-03-04 01:45:43 +03:00

Author

Owner

@nkadel commented on GitHub (Aug 21, 2018):

Use NFS instead of CIFS ? Seriously, for dealing with bulky files, or many small files, NFS has proven considerably more robust. It does nave different security models, but if you want performance, it can be well worth the extra work. And if you need authenticated access, you can do the extra work to use NFS version 4.

@nkadel commented on GitHub (Aug 21, 2018): Use NFS instead of CIFS ? Seriously, for dealing with bulky files, or many small files, NFS has proven considerably more robust. It does nave different security models, but if you want performance, it can be well worth the extra work. And if you need authenticated access, you can do the extra work to use NFS version 4.

kerem commented

2026-03-04 01:45:43 +03:00

Author

Owner

@matthewmedway commented on GitHub (Aug 21, 2018):

@nkadel Sadly NFS is not supported in my organization :(

@matthewmedway commented on GitHub (Aug 21, 2018): @nkadel Sadly NFS is not supported in my organization :(

kerem commented

2026-03-04 01:45:44 +03:00

Author

Owner

@DHB-liuhong commented on GitHub (Mar 27, 2020):

@matthewmedway i have the same problem，do you slave it? if you slave it ,can tell me how to do this? thanks.

@DHB-liuhong commented on GitHub (Mar 27, 2020): @matthewmedway i have the same problem，do you slave it? if you slave it ,can tell me how to do this? thanks.

kerem commented

2026-03-04 01:45:44 +03:00

Author

Owner

@matthewmedway commented on GitHub (Mar 27, 2020):

@DHB-liuhong Nope. We ended up switching to use AWS Storage Gateway for Files instead of s3fs: https://aws.amazon.com/storagegateway/file/

@matthewmedway commented on GitHub (Mar 27, 2020): @DHB-liuhong Nope. We ended up switching to use AWS Storage Gateway for Files instead of s3fs: https://aws.amazon.com/storagegateway/file/