[PR #3434] [MERGED] More ergonomic compatibility tests #3841

New issue

Closed

opened 2026-03-16 12:05:40 +03:00 by kerem · 0 comments

kerem commented

2026-03-16 12:05:40 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/hickory-dns/hickory-dns/pull/3434
Author: @cpu
Created: 1/8/2026
Status: ✅ Merged
Merged: 1/14/2026
Merged by: @cpu

Base: main ← Head: cpu-tsig-test-update_dev

📝 Commits (9)

f557d3a dns-test: change BIND startup check substring
bfd5e10 dns-test: model zone transfer/update configuration
1d05a35 compatibility-tests: top-down ordering for tsig_tests
c2f134e move compatibility tests to conformance workspace
f397642 compatibility-tests: rewrite tsig_tests w/ dns-test
bfa92ef compatibility-tests: rewrite zone_transfer.rs w/ dns-test
99959a2 compatibility-tests: rewrite zone_transfer assert_serial
ebcf2e4 compatibility-tests: remove SIG0 tests, legacy bits
b65fea8 justfile: simplify compatibility-tests, rm init-bind

📊 Changes

29 files changed (+833 additions, -2737 deletions)

View changed files

📝 Cargo.lock (+0 -15)
📝 Cargo.toml (+0 -1)
📝 conformance/Cargo.lock (+358 -4)
📝 conformance/Cargo.toml (+1 -1)
➕ conformance/compatibility-tests/Cargo.toml (+21 -0)
📝 conformance/compatibility-tests/src/lib.rs (+0 -1)
➕ conformance/compatibility-tests/src/tsig_tests.rs (+210 -0)
📝 conformance/compatibility-tests/src/zone_transfer.rs (+67 -19)
📝 conformance/dns-test/src/container.rs (+1 -1)
📝 conformance/dns-test/src/implementation.rs (+11 -2)
📝 conformance/dns-test/src/lib.rs (+1 -0)
📝 conformance/dns-test/src/name_server.rs (+84 -5)
📝 conformance/dns-test/src/templates/named.name-server.conf.jinja (+16 -3)
➕ conformance/dns-test/src/tsig.rs (+49 -0)
📝 justfile (+14 -61)
➖ tests/compatibility-tests/Cargo.toml (+0 -40)
➖ tests/compatibility-tests/src/lib.rs (+0 -173)
➖ tests/compatibility-tests/tests/conf/Kupdate.example.com.+008+56935.key (+0 -1)
➖ tests/compatibility-tests/tests/conf/Kupdate.example.com.+008+56935.pem (+0 -27)
➖ tests/compatibility-tests/tests/conf/Kupdate.example.com.+008+56935.pk8 (+0 -0)

...and 9 more files

📄 Description

An initial step towards addressing https://github.com/hickory-dns/hickory-dns/issues/1981 & helping direct https://github.com/hickory-dns/hickory-dns/pull/3427.

There's some pre-existing TSIG authenticated update and zone transfer tests in the tests/compatibility-tests crate, but there are some downsides:

The tests only support BIND as the remote name server implementation.
The tests require you to build/run named on the host.
The named configuration had any in the allow-update and allow-transfer config for the zone under test, meaning authentication wasn't really being tested.

The dns-test framework from the conformance workspace addresses the first two problems by offering the ability to run different auth. name server implementations (e.g. hickory-dns, named, pdns, and nsd) and does so in isolated Docker containers that are easy to reproduce and configure from tests.

This branch moves the compatibility-tests into a new crate in the conformance workspace and rewrites the tests to rely on dns-test instead of a local named. The SIG(0) authenticated update tests are removed, as BIND has removed support upstream.

Next steps will involve extending the TSIG tests to cover more scenarios (notably, using incorrect auth), and testing against more implementations (pdns, etc).

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/hickory-dns/hickory-dns/pull/3434 **Author:** [@cpu](https://github.com/cpu) **Created:** 1/8/2026 **Status:** ✅ Merged **Merged:** 1/14/2026 **Merged by:** [@cpu](https://github.com/cpu) **Base:** `main` ← **Head:** `cpu-tsig-test-update_dev` --- ### 📝 Commits (9) - [`f557d3a`](https://github.com/hickory-dns/hickory-dns/commit/f557d3a63b782704b1a73ef36e3d91fcc358a573) dns-test: change BIND startup check substring - [`bfd5e10`](https://github.com/hickory-dns/hickory-dns/commit/bfd5e10773eb7922975072ab48d2cf3f17c0e427) dns-test: model zone transfer/update configuration - [`1d05a35`](https://github.com/hickory-dns/hickory-dns/commit/1d05a35b0a1e099faa8f5a7eb3f5f2b13bf8e9fa) compatibility-tests: top-down ordering for tsig_tests - [`c2f134e`](https://github.com/hickory-dns/hickory-dns/commit/c2f134ef1899983e646e0ccb873b81472bb000f3) move compatibility tests to conformance workspace - [`f397642`](https://github.com/hickory-dns/hickory-dns/commit/f3976423be5a944d3576b00c5d490a674c3a590a) compatibility-tests: rewrite tsig_tests w/ dns-test - [`bfa92ef`](https://github.com/hickory-dns/hickory-dns/commit/bfa92efae3d9e1877cb9a6f6cf265be080106918) compatibility-tests: rewrite zone_transfer.rs w/ dns-test - [`99959a2`](https://github.com/hickory-dns/hickory-dns/commit/99959a2cea3260df768b5ad9780e6fe8900fc914) compatibility-tests: rewrite zone_transfer assert_serial - [`ebcf2e4`](https://github.com/hickory-dns/hickory-dns/commit/ebcf2e4f3cae6739aacc9abc7789eae23bcc5193) compatibility-tests: remove SIG0 tests, legacy bits - [`b65fea8`](https://github.com/hickory-dns/hickory-dns/commit/b65fea80262ddc66d09be59852c015b3fa7fef16) justfile: simplify compatibility-tests, rm init-bind ### 📊 Changes **29 files changed** (+833 additions, -2737 deletions) <details> <summary>View changed files</summary> 📝 `Cargo.lock` (+0 -15) 📝 `Cargo.toml` (+0 -1) 📝 `conformance/Cargo.lock` (+358 -4) 📝 `conformance/Cargo.toml` (+1 -1) ➕ `conformance/compatibility-tests/Cargo.toml` (+21 -0) 📝 `conformance/compatibility-tests/src/lib.rs` (+0 -1) ➕ `conformance/compatibility-tests/src/tsig_tests.rs` (+210 -0) 📝 `conformance/compatibility-tests/src/zone_transfer.rs` (+67 -19) 📝 `conformance/dns-test/src/container.rs` (+1 -1) 📝 `conformance/dns-test/src/implementation.rs` (+11 -2) 📝 `conformance/dns-test/src/lib.rs` (+1 -0) 📝 `conformance/dns-test/src/name_server.rs` (+84 -5) 📝 `conformance/dns-test/src/templates/named.name-server.conf.jinja` (+16 -3) ➕ `conformance/dns-test/src/tsig.rs` (+49 -0) 📝 `justfile` (+14 -61) ➖ `tests/compatibility-tests/Cargo.toml` (+0 -40) ➖ `tests/compatibility-tests/src/lib.rs` (+0 -173) ➖ `tests/compatibility-tests/tests/conf/Kupdate.example.com.+008+56935.key` (+0 -1) ➖ `tests/compatibility-tests/tests/conf/Kupdate.example.com.+008+56935.pem` (+0 -27) ➖ `tests/compatibility-tests/tests/conf/Kupdate.example.com.+008+56935.pk8` (+0 -0) _...and 9 more files_ </details> ### 📄 Description An initial step towards addressing https://github.com/hickory-dns/hickory-dns/issues/1981 & helping direct https://github.com/hickory-dns/hickory-dns/pull/3427. There's some pre-existing TSIG authenticated update and zone transfer tests in the `tests/compatibility-tests` crate, but there are some downsides: 1. The tests only support BIND as the remote name server implementation. 2. The tests require you to build/run `named` on the host. 3. The `named` configuration had `any` in the allow-update and allow-transfer config for the zone under test, meaning authentication wasn't really being tested. The `dns-test` framework from the `conformance` workspace addresses the first two problems by offering the ability to run different auth. name server implementations (e.g. `hickory-dns`, `named`, `pdns`, and `nsd`) and does so in isolated Docker containers that are easy to reproduce and configure from tests. This branch moves the `compatibility-tests` into a new crate in the `conformance` workspace and rewrites the tests to rely on `dns-test` instead of a local `named`. The SIG(0) authenticated update tests are removed, as BIND has [removed support](https://kb.isc.org/docs/cve-2024-1975) upstream. Next steps will involve extending the TSIG tests to cover more scenarios (notably, using incorrect auth), and testing against more implementations (pdns, etc). --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>