starred/hickory-dns

Fork 0

mirror of https://github.com/hickory-dns/hickory-dns.git synced 2026-04-25 03:05:51 +03:00

[GH-ISSUE #1973] UdpClientStream fails to parse large responses #835

New issue

Closed

opened 2026-03-16 00:29:05 +03:00 by kerem · 5 comments

kerem commented

2026-03-16 00:29:05 +03:00

Owner

Originally created by @nmittler on GitHub (Jun 20, 2023).
Original GitHub issue: https://github.com/hickory-dns/hickory-dns/issues/1973

Describe the bug
I'm trying to build a local test for truncation, and ran into an issue. The test uses UdpClientStream to call a local server (Catalog + InMemoryAuthority) which responds a large record set resulting in a UDP payload of 2080 bytes (verified via WireShark). Running with tracing enabled, I see:

2023-06-20T17:59:48.758321Z  WARN trust_dns_proto::udp::udp_client_stream: dropped malformed message waiting for id: 9851 err: unexpected end of input reached

Digging in a little further, I see that the client is only seeing 2048 of the 2080 bytes. The MTU for lo0 is 16384, so it's not being truncated down the stack (Wireshark confirms).

The problem appears to be thatUdpClientStream explicitly limits the size of the reply buffer to 2048. Since each read assumes a complete UDP packet, parsing fails.

To Reproduce
See description.

Expected behavior
Replies larger than 2048 bytes should be supported by UdpClientStream.

Docs for tokio UdpSocket indicate that reads should generally be done with the max UDP packet size of 65536. It may be a larger buffer than we'd like in general, but it would at least be correct.

System:

OS: mac OSX
Architecture: arm64
Version: 13.4
rustc version: 1.70.0

Version:
Crate: client, server
Version: 0.22.0

Additional context
NA

Originally created by @nmittler on GitHub (Jun 20, 2023). Original GitHub issue: https://github.com/hickory-dns/hickory-dns/issues/1973 **Describe the bug** I'm trying to build a local test for truncation, and ran into an issue. The test uses `UdpClientStream` to call a local server (`Catalog` + `InMemoryAuthority`) which responds a large record set resulting in a UDP payload of 2080 bytes (verified via WireShark). Running with tracing enabled, I see: ``` 2023-06-20T17:59:48.758321Z WARN trust_dns_proto::udp::udp_client_stream: dropped malformed message waiting for id: 9851 err: unexpected end of input reached ``` Digging in a little further, I see that the client is only seeing `2048` of the `2080` bytes. The MTU for `lo0` is `16384`, so it's not being truncated down the stack (Wireshark confirms). The problem appears to be that`UdpClientStream` explicitly [limits the size of the reply buffer to 2048](https://github.com/bluejekyll/trust-dns/blob/a614257fb0b57471c0903d0c88e15528a69faac5/crates/proto/src/udp/udp_client_stream.rs#L317). Since each read assumes a complete UDP packet, parsing fails. **To Reproduce** See description. **Expected behavior** Replies larger than 2048 bytes should be supported by `UdpClientStream`. [Docs for tokio UdpSocket](https://docs.rs/tokio/latest/tokio/net/struct.UdpSocket.html#method.peek_from) indicate that reads should generally be done with the max UDP packet size of 65536. It may be a larger buffer than we'd like in general, but it would at least be correct. **System:** - OS: mac OSX - Architecture: arm64 - Version: 13.4 - rustc version: 1.70.0 **Version:** Crate: client, server Version: 0.22.0 **Additional context** NA

kerem closed this issue

2026-03-16 00:29:11 +03:00

kerem commented

2026-03-16 00:29:16 +03:00

Author

Owner

@nmittler commented on GitHub (Jun 20, 2023):

@bluejekyll

Related to this is the fact that DNS truncation (TC=1) doesn't appear to be happening in my test. EDNS has max_payload set to 512 (the default), yet the response still comes through without truncation.

I believe that issue may be related to the fact that BinEncoder uses a MaximalBuf with a maximum of u16::max_value(). IIUC, this could be why there is no truncation for a payload of 2080.

I'm happy to raise this as a separate bug if you think it's a real issue.

@nmittler commented on GitHub (Jun 20, 2023): @bluejekyll Related to this is the fact that DNS truncation (TC=1) doesn't appear to be happening in my test. EDNS has max_payload set to 512 (the default), yet the response still comes through without truncation. I believe that issue may be related to the fact that [`BinEncoder` uses a `MaximalBuf` with a maximum of `u16::max_value()`](https://github.com/bluejekyll/trust-dns/blob/a614257fb0b57471c0903d0c88e15528a69faac5/crates/proto/src/serialize/binary/encoder.rs#L135). IIUC, this could be why there is no truncation for a payload of `2080`. I'm happy to raise this as a separate bug if you think it's a real issue.

kerem commented

2026-03-16 00:29:21 +03:00

Author

Owner

@djc commented on GitHub (Jun 20, 2023):

Maybe run a little git blame on the UdpClientStream code to figure out if there were changes to the size of the buffer? It's tricky because I guess we only have one shot to receive, but we might not want to keep a 64k buffer allocated throughout the lifetime of the stream when it's unlikely we'll use the entire size of the buffer.

@djc commented on GitHub (Jun 20, 2023): Maybe run a little `git blame` on the `UdpClientStream` code to figure out if there were changes to the size of the buffer? It's tricky because I guess we only have one shot to receive, but we might not want to keep a 64k buffer allocated throughout the lifetime of the stream when it's unlikely we'll use the entire size of the buffer.

kerem commented

2026-03-16 00:29:26 +03:00

Author

Owner

@nmittler commented on GitHub (Jun 20, 2023):

@djc Yeah that was my thought as well. Perhaps we could use the EDNS max_payload value from the request if available, to trim this down a bit? And/or we could use the MTU of the network interface ... this would likely only be large-ish for a loopback interface.

FYI, the 2048 value goes back all the way to the initial implementation: https://github.com/bluejekyll/trust-dns/pull/46.

@nmittler commented on GitHub (Jun 20, 2023): @djc Yeah that was my thought as well. Perhaps we could use the EDNS max_payload value from the request if available, to trim this down a bit? And/or we could use the MTU of the network interface ... this would likely only be large-ish for a loopback interface. FYI, the 2048 value goes back all the way to the initial implementation: https://github.com/bluejekyll/trust-dns/pull/46.

kerem commented

2026-03-16 00:29:31 +03:00

Author

Owner

@nmittler commented on GitHub (Jun 21, 2023):

@bluejekyll @djc I threw together a WIP PR: #1975. Right now it just uses MTU, but we can expand it. Should probably also have a proper e2e test specifically for this problem.

@nmittler commented on GitHub (Jun 21, 2023): @bluejekyll @djc I threw together a WIP PR: #1975. Right now it just uses MTU, but we can expand it. Should probably also have a proper e2e test specifically for this problem.

kerem commented

2026-03-16 00:29:37 +03:00

Author

Owner

@djc commented on GitHub (Jun 21, 2023):

I guess the MTU is accurate for loopback interfaces at least, but I suppose actual network path MTUs would be constrained by any number of other interfaces on the path to the remote, so it's maybe not very meaningful? Additionally at least the crates I've looked at so far for retrieving the MTU are pretty heavyweight dependencies which IMO isn't very attractive.

@djc commented on GitHub (Jun 21, 2023): I guess the MTU is accurate for loopback interfaces at least, but I suppose actual network path MTUs would be constrained by any number of other interfaces on the path to the remote, so it's maybe not very meaningful? Additionally at least the crates I've looked at so far for retrieving the MTU are pretty heavyweight dependencies which IMO isn't very attractive.

kerem referenced this issue

2026-03-16 02:21:56 +03:00

[PR #835] [MERGED] Bump log from 0.4.7 to 0.4.8 #1722