[PR #1334] [MERGED] Added SIGUSR1 option for cache file integrity test #1925

New issue

Closed

opened 2026-03-04 02:02:49 +03:00 by kerem · 0 comments

kerem commented

2026-03-04 02:02:49 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/s3fs-fuse/s3fs-fuse/pull/1334
Author: @ggtakec
Created: 7/12/2020
Status: ✅ Merged
Merged: 7/26/2020
Merged by: @gaul

Base: master ← Head: add_sigusr1

📝 Commits (1)

0fa4046 Added SIGUSR1 option for cache file integrity test

📊 Changes

9 files changed (+864 additions, -7 deletions)

View changed files

📝 doc/man/s3fs.1 (+6 -0)
📝 src/Makefile.am (+2 -1)
📝 src/common.h (+11 -0)
📝 src/fdcache.cpp (+496 -5)
📝 src/fdcache.h (+15 -1)
📝 src/s3fs.cpp (+28 -0)
📝 src/s3fs_util.cpp (+9 -0)
➕ src/sighandlers.cpp (+230 -0)
➕ src/sighandlers.h (+67 -0)

📄 Description

Relevant Issue (if applicable)

#1291

Details

Added the ability to check the cache file and its cache status file for inconsistencies.

Background

The cache file partially holds data by downloading or uploading(editing) S3 object(file).
(The cache file stats file shows which areas of the file s3fs holds as cache.)
If this cache fill and its stats file are inconsistent, the data will be incorrect during upload and download.

Therefore, it was necessary to check the status of these files from the outside at any time.

Functions provided

Prepared an option(set_check_cache_sigusr1) that can check the cache file and this stats file while s3fs is running.
If the option(set_check_cache_sigusr1) is specified, s3fs can handle SIGUSR1 and check the integrity if SIGUSR1 is triggered.
The check result is output to the standard output or the file path passed to the option(set_check_cache_sigusr1).

The following is an output example of the check result:

------------------------------------------------------------
Check cache file and its stats file consistency
------------------------------------------------------------
File:      /file_u25mb    -> [OK] no problem
File:      /small_kms2.txt    -> [OK] no problem
File:      /file_u25mb-mix    -> [OK] no problem
------------------------------------------------------------
Summary - Total files:                3
          Detected error files:       0
          Detected error directories: 0
------------------------------------------------------------

Critical

If a fatal problem is detected in the check, critical([C] keyword prefix) is output.

Error

When an inconsistency is detected in the cache file, an error([E] keyword prefix) is output.

Warning

The cache file of s3fs is created as a sparse file.
Invalid areas that are not cached have holes in the file as HOLE.
Basically, the cached valid data range is in the DATA area of the sparse file and the uncached range is in the HOLE area.
However, the HOLE/DATA area of the sparse file is in OS dependent(or device dependent) block size(ex, 4096 byte).
There are cases where the boundaries of the area cached by s3fs do not match this block size.
If the boundaries do not match, the area will be DATA in the cache file even if s3fs does not have a cache.
The data in this DATA area is assumed to be filled with 0x00.
This checker outputs a warning ([W] keyword prefix) if it is not filled with 0x00.
Even if it is not filled with 0x00, there is no problem in the operation of s3fs.(so it's a warning)

Also, if the cache file is opened(the file is opened), the cache file and the cache stats file will change dynamically, thus giving a warning like not accurate result.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/s3fs-fuse/s3fs-fuse/pull/1334 **Author:** [@ggtakec](https://github.com/ggtakec) **Created:** 7/12/2020 **Status:** ✅ Merged **Merged:** 7/26/2020 **Merged by:** [@gaul](https://github.com/gaul) **Base:** `master` ← **Head:** `add_sigusr1` --- ### 📝 Commits (1) - [`0fa4046`](https://github.com/s3fs-fuse/s3fs-fuse/commit/0fa40460d1340d1d8befbbda588b47994f7b1311) Added SIGUSR1 option for cache file integrity test ### 📊 Changes **9 files changed** (+864 additions, -7 deletions) <details> <summary>View changed files</summary> 📝 `doc/man/s3fs.1` (+6 -0) 📝 `src/Makefile.am` (+2 -1) 📝 `src/common.h` (+11 -0) 📝 `src/fdcache.cpp` (+496 -5) 📝 `src/fdcache.h` (+15 -1) 📝 `src/s3fs.cpp` (+28 -0) 📝 `src/s3fs_util.cpp` (+9 -0) ➕ `src/sighandlers.cpp` (+230 -0) ➕ `src/sighandlers.h` (+67 -0) </details> ### 📄 Description ### Relevant Issue (if applicable) #1291 ### Details Added the ability to check the cache file and its cache status file for inconsistencies. #### Background The cache file partially holds data by downloading or uploading(editing) S3 object(file). (The cache file stats file shows which areas of the file s3fs holds as cache.) If this cache fill and its stats file are inconsistent, the data will be incorrect during upload and download. Therefore, it was necessary to check the status of these files from the outside at any time. #### Functions provided Prepared an option(`set_check_cache_sigusr1`) that can check the cache file and this stats file while s3fs is running. If the option(`set_check_cache_sigusr1`) is specified, s3fs can handle `SIGUSR1` and check the integrity if `SIGUSR1` is triggered. The check result is output to the standard output or the file path passed to the option(`set_check_cache_sigusr1`). The following is an output example of the check result: ``` ------------------------------------------------------------ Check cache file and its stats file consistency ------------------------------------------------------------ File: /file_u25mb -> [OK] no problem File: /small_kms2.txt -> [OK] no problem File: /file_u25mb-mix -> [OK] no problem ------------------------------------------------------------ Summary - Total files: 3 Detected error files: 0 Detected error directories: 0 ------------------------------------------------------------ ``` ##### Critical If a fatal problem is detected in the check, critical(`[C]` keyword prefix) is output. ##### Error When an inconsistency is detected in the cache file, an error(`[E]` keyword prefix) is output. ##### Warning The cache file of s3fs is created as a sparse file. Invalid areas that are not cached have holes in the file as `HOLE`. Basically, the cached valid data range is in the `DATA` area of the sparse file and the uncached range is in the `HOLE` area. However, the `HOLE`/`DATA` area of the sparse file is in OS dependent(or device dependent) block size(ex, 4096 byte). There are cases where the boundaries of the area cached by s3fs do not match this block size. If the boundaries do not match, the area will be DATA in the cache file even if s3fs does not have a cache. The data in this DATA area is assumed to be filled with 0x00. This checker outputs a warning (`[W]` keyword prefix) if it is not filled with 0x00. Even if it is not filled with 0x00, there is no problem in the operation of s3fs.(so it's a warning) Also, if the cache file is opened(the file is opened), the cache file and the cache stats file will change dynamically, thus giving a warning like not accurate result. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>