starred/s3fs-fuse
1
0
Fork 0
mirror of https://github.com/s3fs-fuse/s3fs-fuse.git synced 2026-04-25 13:26:00 +03:00
Code Issues 305 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #2740] CI(GHA) and local test fail on Ubuntu 25.10 #1283

New issue
Closed
opened 2026-03-04 01:52:47 +03:00 by kerem · 8 comments
kerem commented 2026-03-04 01:52:47 +03:00
Owner
Copy link

Originally created by @ggtakec on GitHub (Oct 12, 2025).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2740

Related Issues/PRs

#2738 #2737

Details

CI was passing at the time of PR, but it's now failing frequently on Ubuntu 25.10.

The errors observed are in the following tests:

  • test_update_parent_directory_time
  • test_update_time_mv
  • test_update_time_chmod
  • test_pjdfstest_mkdir
  • test_pjdfstest_mkfifo
  • test_pjdfstest_rename
  • test_pjdfstest_rmdir
  • test_pjdfstest_symlink
Originally created by @ggtakec on GitHub (Oct 12, 2025). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2740 ### Related Issues/PRs #2738 #2737 ### Details CI was passing at the time of PR, but it's now failing frequently on Ubuntu 25.10. The errors observed are in the following tests: - test_update_parent_directory_time - test_update_time_mv - test_update_time_chmod - test_pjdfstest_mkdir - test_pjdfstest_mkfifo - test_pjdfstest_rename - test_pjdfstest_rmdir - test_pjdfstest_symlink
kerem closed this issue 2026-03-04 01:52:47 +03:00
kerem commented 2026-03-04 01:52:48 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Oct 12, 2025):

The five pjdfstest errors appear to be related to the s3proxy settings.

The following errors were found in the test log:

[s3proxy] W 10-12 07:26:13.421 S3Proxy-Jetty-33 o.g.s.o.e.j.http.HttpParser:1149 |::] Header is too large 8193>8192
...
...
:1: parser error : Extra content at the end of the document
<h1>Bad Message 431</h1><pre>reason: Request Header Fields Too Large</pre>
^

In other words, the header size requested by s3fs is larger than 8192 bytes, and it seems to be rejected by s3proxy(jetty).

@gaul
This seems like it could be avoided by configuring jetty's requestHeaderSize setting, but is there a way to load that setting into s3proxy?

I will continue to investigate other errors.

<!-- gh-comment-id:3394688682 --> @ggtakec commented on GitHub (Oct 12, 2025): The five pjdfstest errors appear to be related to the s3proxy settings. The following errors were found in the test log: ``` [s3proxy] W 10-12 07:26:13.421 S3Proxy-Jetty-33 o.g.s.o.e.j.http.HttpParser:1149 |::] Header is too large 8193>8192 ... ... :1: parser error : Extra content at the end of the document <h1>Bad Message 431</h1><pre>reason: Request Header Fields Too Large</pre> ^ ``` In other words, the header size requested by s3fs is larger than 8192 bytes, and it seems to be rejected by s3proxy(jetty). @gaul This seems like it could be avoided by configuring jetty's requestHeaderSize setting, but is there a way to load that setting into s3proxy? _I will continue to investigate other errors._
kerem commented 2026-03-04 01:52:48 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Oct 12, 2025):

There is a tension here between what S3 supports, the S3Proxy behavior, and what the s3fs tests expect. S3 only supports a maximum key length of 1024 bytes. It seems that S3Proxy allows a larger limit, 8192, the default Jetty size. The test appears to create files with PATH_MAX, or 4096 characters, although this increases with headers like x-amz-copy-source. From my perspective the easiest thing to do is increase the header limit as you suggest. Let me see if there is some way that users can pass through arbitrary properties but otherwise I can run another S3Proxy release to fix this.

<!-- gh-comment-id:3394942223 --> @gaul commented on GitHub (Oct 12, 2025): There is a tension here between what S3 supports, the S3Proxy behavior, and what the s3fs tests expect. [S3 only supports a maximum key length of 1024 bytes](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-keys.html). It seems that S3Proxy allows a larger limit, 8192, the default Jetty size. The test appears to create files with `PATH_MAX`, or 4096 characters, although this increases with headers like `x-amz-copy-source`. From my perspective the easiest thing to do is increase the header limit as you suggest. Let me see if there is some way that users can pass through arbitrary properties but otherwise I can run another S3Proxy release to fix this.
kerem commented 2026-03-04 01:52:48 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Oct 13, 2025):

The header size changes depending on the PATH length (or the values ​​of other headers, etc.), so the upper limit was exceeded, but so far this has only occurred on Ubuntu 25.10.

Personally, I think that since this is an issue that occurs in part of s3fs-fuse's pjdfstest, it would be good to avoid it if possible using a configuration file or something similar.
(A fix that does not require integration into s3proxy)

<!-- gh-comment-id:3395629150 --> @ggtakec commented on GitHub (Oct 13, 2025): The header size changes depending on the PATH length (or the values ​​of other headers, etc.), so the upper limit was exceeded, but so far this has only occurred on Ubuntu 25.10. Personally, I think that since this is an issue that occurs in part of s3fs-fuse's pjdfstest, it would be good to avoid it if possible using a configuration file or something similar. (A fix that does not require integration into s3proxy)
kerem commented 2026-03-04 01:52:48 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Oct 13, 2025):

Related to this issue, I submitted PR #2741 to handle errors in xmlReadMemory calls in s3fs.

<!-- gh-comment-id:3395773795 --> @ggtakec commented on GitHub (Oct 13, 2025): Related to this issue, I submitted PR #2741 to handle errors in xmlReadMemory calls in s3fs.
kerem commented 2026-03-04 01:52:48 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Oct 13, 2025):

I found out why the test_update_parent_directory_time(which is only executed when the update_parent_dir_stat option is used) is failing.
There seems to be a problem with the behavior of the stat command used in Ubuntu 25.10.

First, below is a comparison of the stat command with Ubuntu 24.04.

[ubuntu24.04] $  stat --version
stat (GNU coreutils) 9.4
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by Michael Meskes.

[ubuntu25.10] $ stat --version
stat (uutils coreutils) 0.2.2

The results of obtaining each atime/ctime/mtime using the stat command are as follows:

[ubuntu24.04] $ stat --format "%.9X" file
1713791067.000000000
[ubuntu24.04] $ stat --format "%.9Y"  file
1713791067.000000000
[ubuntu24.04] $ stat --format "%.9Z" file
1754100046.563516141

[ubuntu25.10] $ stat --format "%.9X" file
1760345404
[ubuntu25.10] $ stat --format "%.9Y" file
1760345382.491099600
[ubuntu25.10] $ stat --format "%.9Z" file
1760345382

The atime/ctime decimal fraction ​​are being truncated.

This is causing the test_update_parent_directory_time test to fail. (The test fails because the decimal point difference is not reflected.)

I'm still investigating whether this is an issue(bug) with Ubuntu 25.10, but I'm sharing this information for now.

<!-- gh-comment-id:3396609591 --> @ggtakec commented on GitHub (Oct 13, 2025): I found out why the `test_update_parent_directory_time`(which is only executed when the `update_parent_dir_stat option` is used) is failing. There seems to be a problem with the behavior of the `stat` command used in `Ubuntu 25.10`. First, below is a comparison of the stat command with `Ubuntu 24.04`. ``` [ubuntu24.04] $ stat --version stat (GNU coreutils) 9.4 Copyright (C) 2023 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Written by Michael Meskes. [ubuntu25.10] $ stat --version stat (uutils coreutils) 0.2.2 ``` The results of obtaining each atime/ctime/mtime using the stat command are as follows: ``` [ubuntu24.04] $ stat --format "%.9X" file 1713791067.000000000 [ubuntu24.04] $ stat --format "%.9Y" file 1713791067.000000000 [ubuntu24.04] $ stat --format "%.9Z" file 1754100046.563516141 [ubuntu25.10] $ stat --format "%.9X" file 1760345404 [ubuntu25.10] $ stat --format "%.9Y" file 1760345382.491099600 [ubuntu25.10] $ stat --format "%.9Z" file 1760345382 ``` The atime/ctime decimal fraction ​​are being truncated. This is causing the `test_update_parent_directory_time` test to fail. (The test fails because the decimal point difference is not reflected.) I'm still investigating whether this is an issue(bug) with `Ubuntu 25.10`, but I'm sharing this information for now.
kerem commented 2026-03-04 01:52:48 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Oct 13, 2025):

Additional information

Starting with Ubuntu 25.10, stat has been replaced by uutils coreutils (Rust), and it appears that the GNU version cannot be installed as a package.
Looking at the source code, I noticed that the format for the X, Y, and Z options is different in the following section, which intentionally truncates decimal points.
https://github.com/uutils/coreutils/blob/main/src/uu/stat/src/stat.rs#L1118-L1130

If we run the stat command without any options, we can get the atime/ctime values ​​with decimal points, which can be handled specially in Ubuntu 25.10(such as get_ctime).

<!-- gh-comment-id:3396733975 --> @ggtakec commented on GitHub (Oct 13, 2025): #### Additional information Starting with Ubuntu 25.10, stat has been replaced by uutils coreutils (Rust), and it appears that the GNU version cannot be installed as a package. Looking at the source code, I noticed that the format for the `X`, `Y`, and `Z` options is different in the following section, which intentionally truncates decimal points. https://github.com/uutils/coreutils/blob/main/src/uu/stat/src/stat.rs#L1118-L1130 If we run the stat command without any options, we can get the atime/ctime values ​​with decimal points, which can be handled specially in Ubuntu 25.10(such as `get_ctime`).
kerem commented 2026-03-04 01:52:48 +03:00
Author
Owner
Copy link

@gaul commented on GitHub (Oct 13, 2025):

I sent an upstream PR but Ubuntu doesn't update packages within a given release often. So it might be better for us to add a local workaround.

<!-- gh-comment-id:3398402135 --> @gaul commented on GitHub (Oct 13, 2025): I sent an upstream PR but Ubuntu doesn't update packages within a given release often. So it might be better for us to add a local workaround.
kerem commented 2026-03-04 01:52:48 +03:00
Author
Owner
Copy link

@ggtakec commented on GitHub (Oct 14, 2025):

@gaul
I've confirmed that your PR to uutils/coreutils has been merged.
That's good, but as you said, a temporary solution will be necessary until the fix is ​​reflected in the package.
I'll prepare the code (tests) for the temporary solution

<!-- gh-comment-id:3401541945 --> @ggtakec commented on GitHub (Oct 14, 2025): @gaul I've confirmed that your PR to uutils/coreutils has been merged. That's good, but as you said, a temporary solution will be necessary until the fix is ​​reflected in the package. I'll prepare the code (tests) for the temporary solution
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
v1.97
v1.96
v1.95
v1.94
v1.93
v1.92
v1.91
v1.90
v1.89
v1.88
v1.87
v1.86
v1.85
v1.84
v1.83
v1.82
v1.81
v1.80
v1.79
v1.78
Pre-v1.78
v1.77
v1.76
v1.75
v1.74
Labels
Clear labels   
bug

   
bug

   
dataloss

   
duplicate

   
enhancement

   
feature request

   
help wanted

   
invalid

   
need info

   
performance

   
pull-request

Mirrored from GitHub Pull Request

  
question

   
question

   
testing
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#1283
No description provided.