mirror of
https://github.com/hickory-dns/hickory-dns.git
synced 2026-04-25 11:15:54 +03:00
[GH-ISSUE #1259] Reverselookup Slooooow #634
Labels
No labels
blocked
breaking-change
bug
bug:critical
bug:tests
cleanup
compliance
compliance
compliance
crate:all
crate:client
crate:native-tls
crate:proto
crate:recursor
crate:resolver
crate:resolver
crate:rustls
crate:server
crate:util
dependencies
docs
duplicate
easy
easy
enhance
enhance
enhance
feature:dns-over-https
feature:dns-over-quic
feature:dns-over-tls
feature:dnsssec
feature:global_lb
feature:mdns
feature:tsig
features:edns
has workaround
ops
perf
platform:WASM
platform:android
platform:fuchsia
platform:linux
platform:macos
platform:windows
pull-request
question
test
tools
tools
trust
unclear
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/hickory-dns#634
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @JRAndreassen on GitHub (Oct 20, 2020).
Original GitHub issue: https://github.com/hickory-dns/hickory-dns/issues/1259
Describe the bug
The reverse lookup is extremely slow
As are the SOA queries at times...
It appears it misses the responses some times...
To Reproduce
See attached code...
Adapted from async global example
Expected behavior
Well... faster
System:
stable-x86_64-pc-windows-msvc - Up to date : 1.47.0 (18bf6b4f0 2020-10-07)
stable-x86_64-unknown-linux-gnu - Up to date : 1.47.0 (18bf6b4f0 2020-10-07)
Version:
Crate: trust-dns-resolver
Version: "0.20.0"
cargo.toml
main.rs
@bluejekyll commented on GitHub (Oct 21, 2020):
is it possible you're being rate-limited? I haven't reviewed your code in depth at the moment, maybe there are some areas where this can be sped up. One thing I notice is that your lookups are probably all happening on a single thread. You may want to read up a little bit on spawning in Tokio to achieve a little more parallelism: https://tokio.rs/tokio/tutorial/spawning
@JRAndreassen commented on GitHub (Oct 21, 2020):
Hi...
They are intentionally on a single thread in the test...
The issue I see is that it is missing some of the replies...
Though, seems to only be happening on the the non host(A) queries ...
@JRAndreassen commented on GitHub (Oct 25, 2020):
Hi...
This one works:
@djc commented on GitHub (Oct 26, 2020):
So does that mean your issue has been solved?
@JRAndreassen commented on GitHub (Oct 26, 2020):
@djc ,
I would say so...
It is now only running about 17 threads, besides the 40+ that are left behind for Windows issues...
So, yes...
Thanks you for your assistance...
JR
@bluejekyll commented on GitHub (Oct 26, 2020):
@JRAndreassen be aware, that there is a known mio memory leak (the async-io library used by tokio and most of the Rust ecosystem) on Windows. This is fixed in Tokio 0.3, though it's going to be a little bit before we upgrade to Tokio 0.3. You can follow that progress here: #1262
@JRAndreassen commented on GitHub (Oct 27, 2020):
@bluejekyll
Thanks for the tip..
I've not seen a leak, but I do have a runaway future:poll
@JRAndreassen commented on GitHub (Oct 28, 2020):
@bluejekyll ...
I've not seen a memory leak, but the runaway process I saw showed up on the trust-dns thread...
2020-10-28T02:33:08.142180800+00:00 WARN tokio-runtime-worker trust_dns_proto::xfer: error notifying wait, possible future leak: Err(ProtoError { kind: Message("requestor canceled"), backtrack: None })
And it's deep in the futures::block_on...
What parts of trust-dns is working under 0.3...
I only use resolver (and dependents).
Thanks
JR
@bluejekyll commented on GitHub (Oct 28, 2020):
The resolver is probably the most complex component in the entire library, so it's going to be the last thing to be ready on tokio 0.3.
You can follow the effort here, there's currently a bug around UDP: #1262
@JRAndreassen commented on GitHub (Oct 28, 2020):
@bluejekyll
Ok...
Keep up the good work...
Thanks for letting me know...
JR
@JRAndreassen commented on GitHub (Oct 28, 2020):
@bluejekyll ...
As an aside...
Is there a preferred way to turn off the cache (like setting Cache size to 0) ?
My goal is to time the lookup, not fast cached lookups...
Thanks
JR
@djc commented on GitHub (Oct 28, 2020):
Yes, you can set the cache size in the
ResolverOpts:https://docs.rs/trust-dns-resolver/0.19.5/trust_dns_resolver/config/struct.ResolverOpts.html#structfield.cache_size
@JRAndreassen commented on GitHub (Oct 28, 2020):
Cool,
Thanks
JR