[GH-ISSUE #634] Large file copies failing on macOS 10.12 #361

New issue

Closed

opened 2026-03-04 01:44:45 +03:00 by kerem · 1 comment

kerem commented

2026-03-04 01:44:45 +03:00

Owner

Originally created by @swt2c on GitHub (Sep 1, 2017).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/634

I'm using s3fs on macOS 10.12.6 and I'm seeing large file copies fail most of the time.

For example:

$ time dd if=file_1GB_new.bin of=~/s3fuse/ScottTest3/1GB_dd bs=1048576
1000+0 records in
1000+0 records out
1048576000 bytes transferred in 189.940385 secs (5520553 bytes/sec)

real	3m13.023s
user	0m0.007s
sys	0m4.264s
$ ls -l ~/s3fuse/ScottTest3/1GB_dd
-rw-r--r--  1 stalbert  staff  0 Sep  1 11:05 /Users/stalbert/s3fuse/ScottTest3/1GB_dd

Some of the time dd or cp will report an I/O error and some of the time it appears successful. However, the result is always a 0-byte file on the S3 side. I am using the latest s3fs from git (fa8c417), compiled with --with-gnutls, and OSXFUSE v3.6.3. Smaller files (e.g., 2MB) are usually copied successfully.

An additional data point, I am using a bucket in US-West, but I'm on the US East coast.

Additional Information

Version of s3fs being used (s3fs --version)
Amazon Simple Storage Service File System V1.82(commit:fa8c417) with GnuTLS(gcrypt)
Version of fuse being used (pkg-config --modversion fuse)
2.9.7
System information (uname -a)
Darwin Scotts-Mac-Pro.local 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64 x86_64
Distro (cat /etc/issue)
macOS 10.12.6
s3fs command line used (if applicable)

s3fs grack12test ~/s3fuse/

/etc/fstab entry (if applicable):

s3fs syslog messages (grep s3fs /var/log/syslog, or s3fs outputs)
if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages

Details about issue

Originally created by @swt2c on GitHub (Sep 1, 2017). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/634 I'm using s3fs on macOS 10.12.6 and I'm seeing large file copies fail most of the time. For example: ``` $ time dd if=file_1GB_new.bin of=~/s3fuse/ScottTest3/1GB_dd bs=1048576 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 189.940385 secs (5520553 bytes/sec) real 3m13.023s user 0m0.007s sys 0m4.264s $ ls -l ~/s3fuse/ScottTest3/1GB_dd -rw-r--r-- 1 stalbert staff 0 Sep 1 11:05 /Users/stalbert/s3fuse/ScottTest3/1GB_dd ``` Some of the time dd or cp will report an I/O error and some of the time it appears successful. However, the result is always a 0-byte file on the S3 side. I am using the latest s3fs from git (fa8c417), compiled with `--with-gnutls`, and OSXFUSE v3.6.3. Smaller files (e.g., 2MB) are usually copied successfully. An additional data point, I am using a bucket in US-West, but I'm on the US East coast. #### Additional Information - Version of s3fs being used (s3fs --version) - Amazon Simple Storage Service File System V1.82(commit:fa8c417) with GnuTLS(gcrypt) - Version of fuse being used (pkg-config --modversion fuse) - 2.9.7 - System information (uname -a) - Darwin Scotts-Mac-Pro.local 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64 x86_64 - Distro (cat /etc/issue) - macOS 10.12.6 - s3fs command line used (if applicable) ``` s3fs grack12test ~/s3fuse/ ``` - /etc/fstab entry (if applicable): ``` ``` - s3fs syslog messages (grep s3fs /var/log/syslog, or s3fs outputs) _if you execute s3fs with dbglevel, curldbg option, you can get detail debug messages_ ``` ``` #### Details about issue

kerem closed this issue

2026-03-04 01:44:45 +03:00

kerem commented

2026-03-04 01:44:46 +03:00

Author

Owner

@ggtakec commented on GitHub (Sep 17, 2017):

@swt2c Thanks for posting a issue and sending codes.
#636 is merged, and I'm closing this issue.
If you find another issue, please post new issue.
Thanks,

@ggtakec commented on GitHub (Sep 17, 2017): @swt2c Thanks for posting a issue and sending codes. #636 is merged, and I'm closing this issue. If you find another issue, please post new issue. Thanks,