[GH-ISSUE #702] Second PUT event missing for files over 25 MB #396

New issue

Closed

opened 2026-03-04 01:45:09 +03:00 by kerem · 2 comments

kerem commented

2026-03-04 01:45:09 +03:00

Owner

Originally created by @stephanechauvin on GitHub (Dec 19, 2017).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/702

My issue

I encounter an odd behavior when I upload "large" files in a s3fs mounted directory (through sftp) :

for files that are bigger than 26MB, I correctly have the first PUT event (with a size of 0 byte), but never the second when the files is completely uploaded.

Although it works correctly for files under 25 MB (first 0 size PUT event, then the "regular" final PUT event).

Nevertheless, all files are correctly copied into the S3 bucket, regardless of their size.
It's just the second PUT event that is missing for files over 26 MB...

The uploaded files are XML files (UTF-8 encoded).

Thanks in advance for any help !

Additional Information

Version of s3fs being used (s3fs --version)

V1.80

Version of fuse being used (pkg-config --modversion fuse)

2.9.4

System information (uname -r)

4.4.0-97-generic

Distro (cat /etc/issue)

Ubuntu 16.04.3 LTS

/etc/fstab entry (if applicable):

s3fs#MY_BUCKET_NAME /mnt/XXXX fuse _netdev,rw,nosuid,nodev,allow_other,dbglevel=info,mp_umask=022 0 0

s3fs syslog messages (grep s3fs /var/log/syslog, or s3fs outputs)

Dec 19 09:18:38 ec2-sftp-production s3fs[9655]: s3fs.cpp:set_s3fs_log_level(254): change debug level from [CRT] to [INF]
Dec 19 09:18:38 ec2-sftp-production s3fs[9655]:     PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755)
Dec 19 09:18:38 ec2-sftp-production s3fs[9655]: s3fs: MOUNTPOINT directory /mnt/XXXX is not empty. if you are sure this is safe, can use the 'nonempty' mount option.

Originally created by @stephanechauvin on GitHub (Dec 19, 2017). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/702 ### My issue I encounter an odd behavior when I upload "large" files in a s3fs mounted directory (through sftp) : for files that are bigger than 26MB, I correctly have the first PUT event (with a size of 0 byte), but never the second when the files is completely uploaded. Although it works correctly for files under 25 MB (first 0 size PUT event, then the "regular" final PUT event). Nevertheless, all files are correctly copied into the S3 bucket, regardless of their size. It's just the second PUT event that is missing for files over 26 MB... The uploaded files are XML files (UTF-8 encoded). Thanks in advance for any help ! ### Additional Information #### Version of s3fs being used (s3fs --version) V1.80 #### Version of fuse being used (pkg-config --modversion fuse) 2.9.4 #### System information (uname -r) 4.4.0-97-generic #### Distro (cat /etc/issue) Ubuntu 16.04.3 LTS #### /etc/fstab entry (if applicable): ``` s3fs#MY_BUCKET_NAME /mnt/XXXX fuse _netdev,rw,nosuid,nodev,allow_other,dbglevel=info,mp_umask=022 0 0 ``` #### s3fs syslog messages (grep s3fs /var/log/syslog, or s3fs outputs) ``` Dec 19 09:18:38 ec2-sftp-production s3fs[9655]: s3fs.cpp:set_s3fs_log_level(254): change debug level from [CRT] to [INF] Dec 19 09:18:38 ec2-sftp-production s3fs[9655]: PROC(uid=0, gid=0) - MountPoint(uid=0, gid=0, mode=40755) Dec 19 09:18:38 ec2-sftp-production s3fs[9655]: s3fs: MOUNTPOINT directory /mnt/XXXX is not empty. if you are sure this is safe, can use the 'nonempty' mount option. ```

kerem closed this issue

2026-03-04 01:45:09 +03:00

kerem commented

2026-03-04 01:45:10 +03:00

Author

Owner

@stephanechauvin commented on GitHub (Jan 2, 2018):

My problem is solved by using the nomultipart option

@stephanechauvin commented on GitHub (Jan 2, 2018): My problem is solved by using the `nomultipart` option

kerem commented

2026-03-04 01:45:10 +03:00

Author

Owner

@sqlbot commented on GitHub (Jan 2, 2018):

@stephanechauvin alternately, don't use ObjectCreated:Put for your S3 event trigger. Using ObjectCreated:*, instead, will catch any object created in the bucket, whether by put, post, copy, or completion of a multipart upload (which is not equivalent to a "put").

@sqlbot commented on GitHub (Jan 2, 2018): @stephanechauvin alternately, don't use `ObjectCreated:Put` for your S3 event trigger. Using `ObjectCreated:*`, instead, will catch any object created in the bucket, whether by put, post, copy, or completion of a multipart upload (which is not equivalent to a "put").

kerem referenced this issue

2026-03-04 01:54:14 +03:00

[PR #396] [CLOSED] Proper Dockerfile + security profile #1494