[GH-ISSUE #264] Intermittent test failures testing against Amazon S3 servers #137

New issue

Closed

opened 2026-03-04 01:42:31 +03:00 by kerem · 3 comments

kerem commented

2026-03-04 01:42:31 +03:00

Owner

Originally created by @RobbKistler on GitHub (Sep 11, 2015).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/264

I opened PR #263 that includes the ability to optionally run the s3fs-fuse integration tests against an Amazon S3 server instead of s3proxy.

I see intermittent failures for the integration-test-main.sh/test_append_file test. It happens more frequently on a multicore test machine. I assume this is an S3 object coherency issue.

+ FILE_LENGTH=13
+ '[' 13 -ne 15 ']'
+ echo 'error: expected 15 , got 13'
error: expected 15 , got 13

When I modify the test to append a sequence number, the dropped lines can occur anywhere:

HELLO WORLD 1
HELLO WORLD 2
HELLO WORLD 4
HELLO WORLD 6
...

Is this the expected behavior (and a bad test)? Unless s3fs-fuse does something heroic like poll HEAD the object to wait for the Etag to update -- I don't see that happening currently.

Originally created by @RobbKistler on GitHub (Sep 11, 2015). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/264 I opened PR #263 that includes the ability to optionally run the s3fs-fuse integration tests against an Amazon S3 server instead of s3proxy. I see intermittent failures for the integration-test-main.sh/test_append_file test. It happens more frequently on a multicore test machine. I assume this is an S3 object coherency issue. ``` + FILE_LENGTH=13 + '[' 13 -ne 15 ']' + echo 'error: expected 15 , got 13' error: expected 15 , got 13 ``` When I modify the test to append a sequence number, the dropped lines can occur anywhere: ``` HELLO WORLD 1 HELLO WORLD 2 HELLO WORLD 4 HELLO WORLD 6 ... ``` Is this the expected behavior (and a bad test)? Unless s3fs-fuse does something heroic like poll HEAD the object to wait for the Etag to update -- I don't see that happening currently.

kerem closed this issue

2026-03-04 01:42:31 +03:00

kerem commented

2026-03-04 01:42:32 +03:00

Author

Owner

@gaul commented on GitHub (Sep 11, 2015):

Integration tests against AWS fail due to eventual consistency of read-after-overwrite. If you use a strongly consistent S3 provider like Ceph/DreamObjects or the default S3Proxy harness you will not experience this. Some background:

https://github.com/andrewgaul/are-we-consistent-yet

We should modify the test to avoid overwrites.

@gaul commented on GitHub (Sep 11, 2015): Integration tests against AWS fail due to eventual consistency of read-after-overwrite. If you use a strongly consistent S3 provider like Ceph/DreamObjects or the default S3Proxy harness you will not experience this. Some background: https://github.com/andrewgaul/are-we-consistent-yet We should modify the test to avoid overwrites.

kerem commented

2026-03-04 01:42:32 +03:00

Author

Owner

@RobbKistler commented on GitHub (Sep 11, 2015):

Great writeup, thanks!

Will try to address with PR #264

@RobbKistler commented on GitHub (Sep 11, 2015): Great writeup, thanks! Will try to address with PR #264

kerem commented

2026-03-04 01:42:32 +03:00

Author

Owner