starred/hickory-dns

Fork 0

mirror of https://github.com/hickory-dns/hickory-dns.git synced 2026-04-25 11:15:54 +03:00

[GH-ISSUE #1803] Storage backend: Key-Value landscape #775

New issue

Open

opened 2026-03-16 00:12:06 +03:00 by kerem · 3 comments

kerem commented

2026-03-16 00:12:06 +03:00

Owner

Originally created by @darnuria on GitHub (Oct 16, 2022).
Original GitHub issue: https://github.com/hickory-dns/hickory-dns/issues/1803

(to maintainers feel free to edit! it's a draft for now; Will try to edit with pro/cons later)

Right now Trust-Dns purpose only the following backends:

Sqlite
InMemory
Forwarding
Filesystem

It might be pertinent to have a Key-Value based store, or even multiple.

In this ticket I share some of the research I already did personally.

From my experience, DNS involves a majority of read, and need to support quite nicely writes without locking all the reads, edge case like big zone edition are already showcased by CloudFlare but it might be covered into an other issue. Aka transactional read/write, thread aware is also a need.

It's important to note that Auth have to handle fastly and with a cache request because some domains has.... hundreds of RR.

A struct like Btree is handy for DNS search based on Name (sorted and iterable)

https://github.com/bluejekyll/trust-dns/issues/553

Pertinent reads:

https://mozilla.github.io/firefox-browser-architecture/text/0017-lmdb-vs-leveldb.html

Sled

https://github.com/spacejam/sled

RKV

https://github.com/mozilla/rkv

Has a LMDB backend might need some commit in this backend to be fully prod ready, the Safe is prod ready.

LMDB

LevelDB

crate.io search: https://crates.io/search?q=leveldb

RockDB

-https://github.com/rust-rocksdb/rust-rocksdb

Berkeley database 5.3

No really good binding, it's old, easy to shot your feet but do the job.

Agathe DB

https://github.com/tikv/agatedb

Still early.

SkyDB (AGPLV3) (not embeded)

https://github.com/skytable/skytable

Berkeley database from 6.x (AGPLV3)

No Rust binding, also worth noting that's AGPLv3 software from 6.x version.

Originally created by @darnuria on GitHub (Oct 16, 2022). Original GitHub issue: https://github.com/hickory-dns/hickory-dns/issues/1803 (to maintainers feel free to edit! it's a draft for now; Will try to edit with pro/cons later) Right now Trust-Dns purpose only the following backends: - Sqlite - InMemory - Forwarding - Filesystem It might be pertinent to have a Key-Value based store, or even multiple. In this ticket I share some of the research I already did personally. From my experience, DNS involves a majority of read, and need to support quite nicely writes without locking all the reads, edge case like big zone edition are already showcased by CloudFlare but it might be covered into an other issue. Aka transactional read/write, thread aware is also a need. It's important to note that Auth have to handle fastly and with a cache request because some domains has.... hundreds of RR. A struct like Btree is handy for DNS search based on Name (sorted and iterable) ## Related issues - https://github.com/bluejekyll/trust-dns/issues/553 ## Pertinent reads: https://mozilla.github.io/firefox-browser-architecture/text/0017-lmdb-vs-leveldb.html ## Sled https://github.com/spacejam/sled ## RKV https://github.com/mozilla/rkv Has a LMDB backend might need some commit in this backend to be fully prod ready, the Safe is prod ready. ## LMDB See also RKV - https://github.com/meilisearch/heed - https://github.com/danburkert/lmdb-rs - https://github.com/AltSysrq/lmdb-zero `crate.io` search: https://crates.io/keywords/lmdb Already used by powerdns: https://github.com/PowerDNS/pdns/tree/master/modules/lmdbbackend ## LevelDB - https://github.com/skade/leveldb - https://github.com/dermesser/leveldb-rs `crate.io` search: https://crates.io/search?q=leveldb ## RockDB -https://github.com/rust-rocksdb/rust-rocksdb ## Berkeley database 5.3 No really good binding, it's old, easy to shot your feet but do the job. - https://github.com/jesterpm/libdb-rs - https://github.com/fishi0x01/libdb-rs ## Agathe DB https://github.com/tikv/agatedb Still early. ## SkyDB (AGPLV3) (not embeded) https://github.com/skytable/skytable ## Berkeley database from 6.x (AGPLV3) No Rust binding, also worth noting that's AGPLv3 software from 6.x version.

kerem commented

2026-03-16 00:12:12 +03:00

Author

Owner

@djc commented on GitHub (Oct 17, 2022):

I basically think we shouldn't take on more implementations like this in the server crate. As I see it, the server crate is already pretty undermaintained, and in my experience there are common use cases where the Authority trait isn't even that helpful. It's also pretty easy to write your own backend, in my experience (I maintain a RequestHandler that's backed by Redis).

@djc commented on GitHub (Oct 17, 2022): I basically think we shouldn't take on more implementations like this in the server crate. As I see it, the server crate is already pretty undermaintained, and in my experience there are common use cases where the `Authority` trait isn't even that helpful. It's also pretty easy to write your own backend, in my experience (I maintain a `RequestHandler` that's backed by `Redis`).

kerem commented

2026-03-16 00:12:17 +03:00

Author

Owner

@bluejekyll commented on GitHub (Nov 7, 2022):

Yeah, maybe we should look into supporting some dylib option for implementing custom backends and loading at runtime?

@bluejekyll commented on GitHub (Nov 7, 2022): Yeah, maybe we should look into supporting some dylib option for implementing custom backends and loading at runtime?

kerem commented

2026-03-16 00:12:22 +03:00

Author

Owner

@darnuria commented on GitHub (Nov 17, 2022):

It's a dual edged sword. Have some experience with this part with PowerDns wouldn't say it's perfect it need a really good management of crash and signals to avoid leaving databases in bad state if they are in process space.

If a C-ABI is exposed it an other kind of beast and error (it need an interface for logging, managing failure and cleaning etc.)

Don't want to say it's "impossible" but would prefers having a solid mature Backend API before exposing a public external dynlib one.

It's only my humble opinion.

A "Thin" protocol remote API for external process backend is indeed interesting, would suggest avoiding JSON, XML or anything needing big parsing it kills a lot performance (already hit by process context change).

@darnuria commented on GitHub (Nov 17, 2022): It's a dual edged sword. Have some experience with this part with PowerDns wouldn't say it's perfect it need a really good management of crash and signals to avoid leaving databases in bad state if they are in process space. If a C-ABI is exposed it an other kind of beast and error (it need an interface for logging, managing failure and cleaning etc.) Don't want to say it's "impossible" but would prefers having a solid mature Backend API before exposing a public external dynlib one. It's only my humble opinion. A "Thin" protocol remote API for external process backend is indeed interesting, would suggest avoiding JSON, XML or anything needing big parsing it kills a lot performance (already hit by process context change).