[GH-ISSUE #2328] Rethink CI targets #1150

Open
opened 2026-03-04 01:51:45 +03:00 by kerem · 7 comments
Owner

Originally created by @gaul on GitHub (Sep 23, 2023).
Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2328

Our CI has a large combination of distros:

  • ubuntu:23.04
  • ubuntu:22.04 (remove?)
  • ubuntu:20.04 (remove?)
  • debian:bookworm
  • debian:bullseye (remove?)
  • debian:buster (remove?)
  • rockylinux:9
  • rockylinux:8
  • centos:centos7
  • fedora:38
  • fedora:37 (remove?)
  • opensuse/leap:15
  • alpine:3.17 (remove?)
  • macOS (temporarily disabled)

I don't think that s3fs is well-served by this coverage since most of them are the same. We should instead only test the latest version or versions that packagers actively maintain. For example, RHEL 9, 8, and 7 all have a good value to ensure that packagers can use older compilers. But Ubuntu 22.04 and 20.04 are not since they don't repackage newer s3fs. Same with Debian so we should only support bookworm. Further alpine has some strange issue that doesn't seem worth investigating.

Further some of the older distros have old versions of cppcheck and shellcheck that we should avoid. Instead I think we should have distinct CI tasks for cppcheck and shellcheck and stop running them as part of each CI task. I guess we can be more aggressive about version checks which is almost the same thing.

Originally created by @gaul on GitHub (Sep 23, 2023). Original GitHub issue: https://github.com/s3fs-fuse/s3fs-fuse/issues/2328 Our CI has a large combination of distros: * ubuntu:23.04 * ubuntu:22.04 (remove?) * ubuntu:20.04 (remove?) * debian:bookworm * debian:bullseye (remove?) * debian:buster (remove?) * rockylinux:9 * rockylinux:8 * centos:centos7 * fedora:38 * fedora:37 (remove?) * opensuse/leap:15 * alpine:3.17 (remove?) * macOS (temporarily disabled) I don't think that s3fs is well-served by this coverage since most of them are the same. We should instead only test the latest version or versions that packagers actively maintain. For example, RHEL 9, 8, and 7 all have a good value to ensure that packagers can use older compilers. But Ubuntu 22.04 and 20.04 are not since they don't repackage newer s3fs. Same with Debian so we should only support bookworm. Further alpine has some strange issue that doesn't seem worth investigating. Further some of the older distros have old versions of cppcheck and shellcheck that we should avoid. Instead I think we should have distinct CI tasks for cppcheck and shellcheck and stop running them as part of each CI task. I guess we can be more aggressive about version checks which is almost the same thing.
Author
Owner

@ggtakec commented on GitHub (Sep 24, 2023):

In my opinion, I would like to preserve the OS provided as LTS.
I think it is okay to exclude OS versions that are outside of LTS.
I would like to support the Current OS version (before it becomes Stable or LTS). (For example, Ubuntu23.04)

Regarding Alpine, I have the opposite opinion and should continue to support this OS.
Considering their maintenance story, I think we can always support only latest released version. (We should also change to 3.18 as soon as possible)
This is because the number of users using Alpine is increasing in consideration of Container, and we expect this to continue to increase in the future.
Regarding Alpine, there are some problems passing the test(now), but the related packages required by s3fs are also well supported, so we don't expect any major problems to occur.
(For example, Alpine already supports OpenSSL3.0, which is attractive.)

Below is the state of the OS in my opinion.

  • ubuntu:23.04
    Either one is still fine. It is possible to wait until LTS
  • ubuntu:22.04
    Main support
  • ubuntu:20.04
    I think it should be supported (optional if unnecessary)
  • debian:bookworm
    Main support
  • debian:bullseye
    I think it should be supported (optional if unnecessary)
  • debian:buster
    I think it should be removed.
  • rockylinux:9
    Main support
  • rockylinux:8
    Main support
  • centos:centos7
    I think this can no longer be supported.
  • fedora:39
    I think it's better to prepare support.(2023, Oct)
  • fedora:38
    Main support
  • fedora:37
    It is clearly stated as EOL on 2023-11-14, so it may be time to exclude it.
  • opensuse/leap:15
    I think it should be continued.
  • alpine:3.18
    It should be switched as Main Support.
  • alpine:3.17
    I think this can no longer be supported.(switch to 3.18)
  • macOS 12.x
    Although there are uncertainties such as osxfuse/fuse-t, we believe that support with fuse-t should be provided. (Currently preparing to support)

As a result, it's not much different from the current situation.
If you want to narrow it down further, you may be able to divide it into Ubunutu or Debian / Rocky or Fedora.
(But even considering package managers, distributions, etc., I feel that the above OS is the minimum)

<!-- gh-comment-id:1732603659 --> @ggtakec commented on GitHub (Sep 24, 2023): In my opinion, I would like to preserve the OS provided as LTS. _I think it is okay to exclude OS versions that are outside of LTS._ I would like to support the Current OS version (before it becomes Stable or LTS). (For example, Ubuntu23.04) Regarding Alpine, I have the opposite opinion and should continue to support this OS. Considering their maintenance story, I think we can always support only latest released version. (We should also change to 3.18 as soon as possible) This is because the number of users using Alpine is increasing in consideration of Container, and we expect this to continue to increase in the future. Regarding Alpine, there are some problems passing the test(now), but the related packages required by s3fs are also well supported, so we don't expect any major problems to occur. (For example, Alpine already supports OpenSSL3.0, which is attractive.) Below is the state of the OS in my opinion. - ubuntu:23.04 Either one is still fine. It is possible to wait until LTS - ubuntu:22.04 Main support - ubuntu:20.04 I think it should be supported (optional if unnecessary) - debian:bookworm Main support - debian:bullseye I think it should be supported (optional if unnecessary) - debian:buster I think it should be removed. - rockylinux:9 Main support - rockylinux:8 Main support - centos:centos7 I think this can no longer be supported. - fedora:39 I think it's better to prepare support.(2023, Oct) - fedora:38 Main support - fedora:37 It is clearly stated as EOL on 2023-11-14, so it may be time to exclude it. - opensuse/leap:15 I think it should be continued. - alpine:3.18 It should be switched as Main Support. - alpine:3.17 I think this can no longer be supported.(switch to 3.18) - macOS 12.x Although there are uncertainties such as osxfuse/fuse-t, we believe that support with fuse-t should be provided. (Currently preparing to support) As a result, it's not much different from the current situation. If you want to narrow it down further, you may be able to divide it into Ubunutu or Debian / Rocky or Fedora. (But even considering package managers, distributions, etc., I feel that the above OS is the minimum)
Author
Owner

@ggtakec commented on GitHub (Nov 20, 2023):

I would like to reconsider the need for CentOS7.
With the transition to C++11, the use of devtoolset is required to support CentOS7.
Actually, I think it can be installed and it can work.
However, I feel like it would be a good idea to end support for CentOS7 with s3fs CI (Github Actions).

what's your opinion.

<!-- gh-comment-id:1819292239 --> @ggtakec commented on GitHub (Nov 20, 2023): I would like to reconsider the need for CentOS7. With the transition to C++11, the use of devtoolset is required to support CentOS7. Actually, I think it can be installed and it can work. However, I feel like it would be a good idea to end support for CentOS7 with s3fs CI (Github Actions). what's your opinion.
Author
Owner

@gaul commented on GitHub (Nov 20, 2023):

RHEL 7 end of life is 30 June 2024 so we will end up removing support for it soon anyway. This is the oldest environment we support and blocks things like FUSE 3 and C++17. I'm neutral on dropping support and we could just revert the std::get_time changes for now.

<!-- gh-comment-id:1819910725 --> @gaul commented on GitHub (Nov 20, 2023): RHEL 7 end of life is 30 June 2024 so we will end up removing support for it soon anyway. This is the oldest environment we support and blocks things like FUSE 3 and C++17. I'm neutral on dropping support and we could just revert the `std::get_time` changes for now.
Author
Owner

@ggtakec commented on GitHub (Nov 20, 2023):

I'm keeping it because CentOS 7(and RHEL 7) are so popular(I'm not sure if I can call it that), but I think it's about time for user to switch to RockyLinux.
Personally, I feel that the need for us to test CentOS 7 on Github Actions is much lower.
Additionally, support can be continued by using the devtoolset, but it requires modifications to the build environment and its scripts.
Just to be sure, I will try it on the weekend and we can judge based on the corresponding modified code(script).

<!-- gh-comment-id:1819992506 --> @ggtakec commented on GitHub (Nov 20, 2023): I'm keeping it because CentOS 7(and RHEL 7) are so popular(I'm not sure if I can call it that), but I think it's about time for user to switch to RockyLinux. Personally, I feel that the need for us to test CentOS 7 on Github Actions is much lower. Additionally, support can be continued by using the devtoolset, but it requires modifications to the build environment and its scripts. Just to be sure, I will try it on the weekend and we can judge based on the corresponding modified code(script).
Author
Owner

@gaul commented on GitHub (Nov 21, 2023):

One argument against removing CentOS 7 support is that we have an active packager so users on legacy CentOS still get good s3fs support. This isn't true for Ubuntu LTS releases, for example, which are stuck on older buggy versions. I think it might be possible to just install the newer GCC compiler for CI but this also pushes the same burden onto the packager.

I guess the addition to libstdc++ landed too late for CentOS 7: https://github.com/gcc-mirror/gcc/blob/master/libstdc%2B%2B-v3/ChangeLog-2014#L91

I'm fine reverting the change and revisiting it in June. But I don't care too much either way.

<!-- gh-comment-id:1820076080 --> @gaul commented on GitHub (Nov 21, 2023): One argument against removing CentOS 7 support is that we have an active packager so users on legacy CentOS still get good s3fs support. This isn't true for Ubuntu LTS releases, for example, which are stuck on older buggy versions. I think it might be possible to just install the newer GCC compiler for CI but this also pushes the same burden onto the packager. I guess the addition to libstdc++ landed too late for CentOS 7: https://github.com/gcc-mirror/gcc/blob/master/libstdc%2B%2B-v3/ChangeLog-2014#L91 I'm fine reverting the change and revisiting it in June. But I don't care too much either way.
Author
Owner

@ggtakec commented on GitHub (Jan 28, 2024):

Additional Information About actions/checkout@v4 and CentOS 7

Going forward, we will need to migrate to actions/checkout@v4, which we are using with Gitub Actions.
But actions/checkout@v4 doesn't work on CnetOS 7.
reference
This incompatibility seems to stem from the fact that NodeJS version 20 supports glibc >=2.28 (as seen in the link below), but CentOS 7 only has glibc==2.17.

We won't be migrating to @v4 right away, but we will do so as soon as we no longer support CentOS 7.

<!-- gh-comment-id:1913419752 --> @ggtakec commented on GitHub (Jan 28, 2024): ### Additional Information About actions/checkout@v4 and CentOS 7 Going forward, we will need to migrate to actions/checkout@v4, which we are using with Gitub Actions. But actions/checkout@v4 doesn't work on CnetOS 7. reference This incompatibility seems to stem from the fact that NodeJS version 20 supports glibc >=2.28 (as seen in the link below), but CentOS 7 only has glibc==2.17. We won't be migrating to @v4 right away, but we will do so as soon as we no longer support CentOS 7.
Author
Owner

@gaul commented on GitHub (Dec 26, 2024):

Revisiting this in 2024, s3fs continues to have a matrix of distributions:

  • ubuntu:24.10
  • ubuntu:24.04
  • ubuntu:22.04
  • ubuntu:20.04
  • debian:bookworm
  • debian:bullseye
  • rockylinux:9
  • rockylinux:8
  • fedora:41
  • fedora:40
  • opensuse/leap:15
  • alpine:3.21
  • macos-13

But also now has functional differences:

  • glibc_debug
  • sanitize_address
  • sanitize_others
  • sanitize_thread
  • thread_safety
  • valgrind
  • clang-tidy

Importantly we have moved clang-tidy to its own task which addressed problems with older versions and buggy behavior. I think CI could also move cppcheck and shellcheck to their own tasks or as part of the clang-tidy task, perhaps renamed to "static checks" or similar. This would have the advantage of reducing run-time for each of the distributions. Note that we should continue to run these as part of macOS which has its own headers and environment. We might also want to add some of the configurations from test/compile_all_targets.sh.

<!-- gh-comment-id:2562877761 --> @gaul commented on GitHub (Dec 26, 2024): Revisiting this in 2024, s3fs continues to have a matrix of distributions: * ubuntu:24.10 * ubuntu:24.04 * ubuntu:22.04 * ubuntu:20.04 * debian:bookworm * debian:bullseye * rockylinux:9 * rockylinux:8 * fedora:41 * fedora:40 * opensuse/leap:15 * alpine:3.21 * macos-13 But also now has functional differences: * glibc_debug * sanitize_address * sanitize_others * sanitize_thread * thread_safety * valgrind * clang-tidy Importantly we have moved clang-tidy to its own task which addressed problems with older versions and buggy behavior. I think CI could also move cppcheck and shellcheck to their own tasks or as part of the clang-tidy task, perhaps renamed to "static checks" or similar. This would have the advantage of reducing run-time for each of the distributions. Note that we should continue to run these as part of macOS which has its own headers and environment. We might also want to add some of the configurations from `test/compile_all_targets.sh`.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/s3fs-fuse#1150
No description provided.