[PR #1569] [MERGED] Improved processing when HEAD response is 400 #2051

New issue

Closed

opened 2026-03-04 02:03:27 +03:00 by kerem · 0 comments

kerem commented

2026-03-04 02:03:27 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/s3fs-fuse/s3fs-fuse/pull/1569
Author: @ggtakec
Created: 2/13/2021
Status: ✅ Merged
Merged: 2/13/2021
Merged by: @gaul

Base: master ← Head: head_400

📝 Commits (1)

55c7f79 Improved processing when HEAD response is 400

📊 Changes

2 files changed (+32 additions, -4 deletions)

View changed files

📝 src/curl.cpp (+7 -2)
📝 src/s3fs.cpp (+25 -2)

📄 Description

Relevant Issue (if applicable)

#1569 #1567
(But this PR merge does not have to wait for #1567 merge.)

Details

Current

The HEAD request returns a 400 error if access to an object such as SSE is denied.(See also #1567)
If s3fs received this error, it was not registered in the Stats cache.
Therefore, when everytime listing the directories containing the SSE objects that generate the error, a HEAD request was generated for each object that causes the error.
Also, the object that resulted in EPREM with a 403 error also degraded performance as same as it.

Fixes

For HEAD requests, It was changed the 400 error to EPERM instead of EIO.
And when s3fs receives the result of EPERM, s3fs registers the file in the Stats cache.
This will list the files in the list of objects in the directory without any error message.
However, if s3fs leaves the meta header empty and register it in the Stats cache, it will be displayed with the permissions 0644.
In this state, when user access the file, nothing is displayed without any error.
So, when registering in the Stats cache, s3fs sets 0000 permission as the dummy to x-amz-meta-mode and register it.
This allows that user gets a Permission denied error when user access the file.

Details

Before the fix, objects such as SSE(with error) were displayed as follows.

$ ls -la
ls: cannot access sse_file: No such file or directory
total 855349
drwxrwxrwx 1 ggtakec users         0 Jan  1  1970 .
drwxr-xr-x 4 root    root         40 Feb  7 14:30 ..
-rw-r--r-- 1 ggtakec users         0 Sep 21 09:34 normal_file
?????????? ? ?       ?             ?            ? sse_file

As mentioned above, permissions, owner, group, date and time were all ?, And a No such file or directory error was output.
And the HEAD request sent every time when user reads this directory.
First HEAD request receives a 400 error, so after all, the 4 type request(the normal path(/sse_file), the path as a directory(/sse_file/), the special path(/sse_file_$folder$) and ListBucket requests as /sse_file) were sent.

After modification, only HEAD request(normal path(/sse_file)) will be sent.
Also, once registered in the Stats cache, no HEAD requests will be sent while the cache is valid.

And the result of listing the directories is as follows.

$ ll
total 855351
drwxrwxrwx 1 ggtakec users         0 Jan  1  1970 .
drwxr-xr-x 4 root    root         40 Feb  7 14:30 ..
-rw-r--r-- 1 ggtakec users         0 Sep 21 09:34 normal_file
---------- 1 ggtakec users         0 Jan  1  1970 sse_file

As mentioned above, the file permissions will be 0000, and the owner, group, date and time(unixtime=0) will also be displayed.
Also, the No such file or directory error disappears.
When user accesses this file, user gets a Permission denied error as shown below.

$ cat sse_file
cat: sse_file: Permission denied

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/s3fs-fuse/s3fs-fuse/pull/1569 **Author:** [@ggtakec](https://github.com/ggtakec) **Created:** 2/13/2021 **Status:** ✅ Merged **Merged:** 2/13/2021 **Merged by:** [@gaul](https://github.com/gaul) **Base:** `master` ← **Head:** `head_400` --- ### 📝 Commits (1) - [`55c7f79`](https://github.com/s3fs-fuse/s3fs-fuse/commit/55c7f79353247cc23adbdc75a6264724a6ff49ac) Improved processing when HEAD response is 400 ### 📊 Changes **2 files changed** (+32 additions, -4 deletions) <details> <summary>View changed files</summary> 📝 `src/curl.cpp` (+7 -2) 📝 `src/s3fs.cpp` (+25 -2) </details> ### 📄 Description ## Relevant Issue (if applicable) #1569 #1567 (But this PR merge does not have to wait for #1567 merge.) ## Details ### Current The HEAD request returns a 400 error if access to an object such as SSE is denied.(See also #1567) If s3fs received this error, it was not registered in the Stats cache. Therefore, when everytime listing the directories containing the SSE objects that generate the error, a HEAD request was generated for each object that causes the error. Also, the object that resulted in EPREM with a 403 error also degraded performance as same as it. ### Fixes For HEAD requests, It was changed the 400 error to EPERM instead of EIO. And when s3fs receives the result of EPERM, s3fs registers the file in the Stats cache. This will list the files in the list of objects in the directory without any error message. However, if s3fs leaves the meta header empty and register it in the Stats cache, it will be displayed with the permissions `0644`. In this state, when user access the file, nothing is displayed without any error. So, when registering in the Stats cache, s3fs sets `0000` permission as the dummy to` x-amz-meta-mode` and register it. This allows that user gets a `Permission denied` error when user access the file. ### Details Before the fix, objects such as SSE(with error) were displayed as follows. ``` $ ls -la ls: cannot access sse_file: No such file or directory total 855349 drwxrwxrwx 1 ggtakec users 0 Jan 1 1970 . drwxr-xr-x 4 root root 40 Feb 7 14:30 .. -rw-r--r-- 1 ggtakec users 0 Sep 21 09:34 normal_file ?????????? ? ? ? ? ? sse_file ``` As mentioned above, permissions, owner, group, date and time were all `?`, And a `No such file or directory` error was output. And the HEAD request sent every time when user reads this directory. First HEAD request receives a 400 error, so after all, the 4 type request(the normal path(`/sse_file`), the path as a directory(`/sse_file/`), the special path(`/sse_file_$folder$`) and ListBucket requests as `/sse_file`) were sent. After modification, only HEAD request(normal path(`/sse_file`)) will be sent. Also, once registered in the Stats cache, no HEAD requests will be sent while the cache is valid. And the result of listing the directories is as follows. ``` $ ll total 855351 drwxrwxrwx 1 ggtakec users 0 Jan 1 1970 . drwxr-xr-x 4 root root 40 Feb 7 14:30 .. -rw-r--r-- 1 ggtakec users 0 Sep 21 09:34 normal_file ---------- 1 ggtakec users 0 Jan 1 1970 sse_file ``` As mentioned above, the file permissions will be `0000`, and the owner, group, date and time(unixtime=0) will also be displayed. Also, the `No such file or directory` error disappears. When user accesses this file, user gets a `Permission denied` error as shown below. ``` $ cat sse_file cat: sse_file: Permission denied ``` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>