mirror of
https://github.com/s3fs-fuse/s3fs-fuse.git
synced 2026-04-25 21:35:58 +03:00
[GH-ISSUE #2384] Cannot get AWS IMDSv2 token due to curl Expect header #1169
Labels
No labels
bug
bug
dataloss
duplicate
enhancement
feature request
help wanted
invalid
need info
performance
pull-request
question
question
testing
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/s3fs-fuse#1169
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @kuza55 on GitHub (Nov 27, 2023).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2384
Additional Information
Version of s3fs being used (
s3fs --version)Amazon Simple Storage Service File System V1.90 (commit:unknown) with GnuTLS(gcrypt)
Version of fuse being used (
pkg-config --modversion fuse,rpm -qi fuseordpkg -s fuse)Not a fuse problem
Kernel information (
uname -r)6.2.0-1012-aws
GNU/Linux Distribution, if applicable (
cat /etc/os-release)Ubuntu 22.04.3 LTS
How to run s3fs, if applicable
[x] command line
s3fs syslog messages (
grep s3fs /var/log/syslog,journalctl | grep s3fs, ors3fs outputs)Command-line output
Details about issue
s3fs cannot seem to get some sort of IAM token because curl adds the Expect: 100-continue header.
This looks like it should be handled in the code, but as the debug output shows, it is still there:
github.com/s3fs-fuse/s3fs-fuse@b4edad86d6I have verified that when I manually execute the request without the header, I get the token:
I think this is the root cause for some other issues like https://github.com/s3fs-fuse/s3fs-fuse/issues/1743
@kuza55 commented on GitHub (Nov 28, 2023):
It looks like this is fixed in 1.93, but Ubuntu 22.04.3 LTS doesn't have that version.
Have you guys thought of having some other way to get recent versions, rather than relying on package maintainers?
@gaul commented on GitHub (Nov 28, 2023):
You can already compile from source if your distribution is slow. Which other way do you envision of getting more recent versions?
Perhaps you should complain to the Ubuntu maintainers to upgrade packages faster? I notice that only one person indicated that the latest upstream bug affected them (which was me).
Or you could upgrade to a distro like Fedora that upgrades as soon as we release new versions.
@kuza55 commented on GitHub (Nov 29, 2023):
A lot of other open source software provides ways to get it separate from the official repositories.
Having prebuilt binaries that people can download from github or a ppa would be a great alternative to having to build from source.
@gaul commented on GitHub (Nov 29, 2023):
It is not feasible to provide binary Linux packages given the matrix of architectures (ARM32, ARM64, x86-64) and SSL libraries (OpenSSL, GNUtls, maybe NSS). Statically linking these or even just libcurl presents its own security issues.
If you want to set up your own PPA we could link to a PPA in the README although users generally prefer distribution packaging for convenience and policy issues. We can also improve the compilation instructions you want to submit a pull request.
The real solution is to communicate your dissatisfaction to your distribution. You can see more precise suggestions at https://github.com/s3fs-fuse/s3fs-fuse/issues/2360#issuecomment-1793974323.
@kuza55 commented on GitHub (Nov 29, 2023):
I don't see why these options are infeasible. CI infra should make either producing this matrix or producing regular releases with statically linked libraries relatively straight forward.
I am personally not hopeful about distributions doing a good job here, and frankly do am not keen about relying on the distribution packages if possible, though it's good to hear a concrete recommendation for Fedora.
@gaul commented on GitHub (Nov 30, 2023):
Infeasible does not mean impossible. What you propose is an large effort for a tiny benefit and simply working around avoidable brokenness. We will not be packaging binaries or creating a PPA. Please follow up with your distribution or switch to a better one like Fedora.
@juliogonzalez commented on GitHub (Nov 30, 2023):
Sorry for posting on a closed issue :-)
I 100% agree, as I know by experience how hard is to maintain such matrix for building and testing.
Building such stuff is work for the distribution maintainers.
For bugs that are fixed on this official repository, but not on distribution, bug reports must be created on the distribution. Then the maintainers can see if they can prepare backports or not, and of course they will be welcoming any help that can be provided.
To be fair, I'd not put the blame on the Ubuntu maintainers or would say Ubuntu is a bad distribution. It's just that the focus is different. In particular, Ubuntu 22.04 is a LTS release.
As the Fedora/EPEL maintainer, I have to admit I am bending a little bit the rules but pushing the latest version always.
Strictly speaking someone could ask me to stick to the first version that was shipped to a given Fedora version and start backporting patches (as it would be the case for Red Hat and clones such as Alma Linux or Rocky Linux). Unfortunately at that point I'd need to step down as maintainer, as I am afraid I am not a C developer and would likely have problems as soon as two patches from different s3fs-fuse versions start causing conflicts.
Also strictly speaking, for the latest and greatest I'd always recommend a rolling release distribution, or at the very least something with a more frequent release cadence, not LTS distributions.
In any case, for those that want to experiment, I just prepared builds for Debian 12, and Ubuntu >= 22.04 at https://build.opensuse.org/project/show/home:juliogonzalez:s3fs-fuse, using the Debian Unstable package.
Feel free to use the Ubuntu 22.04 repository, the instructions should be the same I'm pasting below, but replacing
Debian_12withUbuntu_22.04.Needless to say, there's no warranty or anything like that.
But if @gaul later agrees, we can see how to add this to the doc, as a 100% unofficial and unsupported way of installing the last s3fs-fuse on old Debian/Ubuntu/Raspbian. But even before that... I'd like more feedback fist, so not sure if we want to close https://github.com/s3fs-fuse/s3fs-fuse/issues/2360 and create a new issue asking people to try the packages for the different OS I am building for with the Open Build Service.
Sorry for not preparing a PPA, but it was quicker with this solution, and that way I can generate packages for Debian/Raspbian as well.
Instructions for Debian12
Tested in my personal desktop computer (they can be copied and pasted)
And with this:
NOTE to myself: I should probably adjust the services the the maintainer at the
.dscfile at OBS is adjusted, just to avoid having people sending reports to Mattia Rizzolo from Debian.@gaul commented on GitHub (Dec 1, 2023):
Backporting newer s3fs versions to older distributions is the packagers choice and I don't take issue with this. I do take issue that Ubuntu doesn't upgrade to the latest version in their current releases and I have to manually remind them to do so which takes an unpredictable amount of time. For example Ubuntu 23.10 still has s3fs 1.90: https://packages.ubuntu.com/search?keywords=s3fs. I am happy that all the distributions are using vanilla packages without patches since Frankenstein binaries would be difficult for us to support.
Ubuntu has a backporting process that I recommended in https://github.com/s3fs-fuse/s3fs-fuse/issues/2360#issuecomment-1793974323. The correct solution is to work with the distribution and not ask the upstream project to duplicate parallel infrastructure. Packaging is a hard task without all of the infrastructure that distributions have.