[GH-ISSUE #1687] DNS-over-QUIC (DoQ) does not work with dns.adguard.com #736

Closed
opened 2026-03-16 00:03:27 +03:00 by kerem · 9 comments
Owner

Originally created by @bluejekyll on GitHub (Apr 8, 2022).
Original GitHub issue: https://github.com/hickory-dns/hickory-dns/issues/1687

Describe the bug
When using DoQ for queries, the QUIC protocol errors with:

; using quic:94.140.14.14:853 dns_name:dns.adguard.com
; sending query: www.example.com. IN SOA
[2022-04-08T19:06:28Z DEBUG trust_dns_proto::xfer::dns_handle] querying: www.example.com. SOA
[2022-04-08T19:06:28Z DEBUG trust_dns_proto::xfer] enqueueing message:QUERY:[Query { name: Name("www.example.com."), query_type: SOA, query_class: IN, mdns_unicast_response: false }]
[2022-04-08T19:06:28Z DEBUG trust_dns_proto::quic::quic_stream] received packet len: 44 bytes: b"\0\0\x01\0\0\x01\0\0\0\0\0\x01\x03www\x07example\x03com\0\0\x06\0\x01\0\0)\x04\xd0\0\0\0\0\0\0"
Error: Error { kind: Proto(ProtoError { kind: QuinnReadError(FinishedEarly), backtrack: None }), backtrack: None }

To Reproduce
The current easiest way to reproduce this is to use the new dns utility from the project, see trust-dns-client-cli:

cargo run --all-features --bin dns -- --debug -p quic -t dns.adguard.com -n 94.140.14.14:853 query www.example.com. SOA

edit: it appears that adguard may require a custom ALPN, doq-i02, as opposed to the RFC's ALPN of doq, the proper command for that is, but has the same result:

cargo run --all-features --bin dns -- --debug -p quic -t dns.adguard.com -n 94.140.14.14:853 -a doq-i02 query www.example.com. SOA

Expected behavior
Correct behavior with DoH can be seen with the same server:

$> cargo run --all-features --bin dns -- --debug -p https -t dns.adguard.com -n 94.140.14.14:443 query www.example.com. SOA
    Finished dev [unoptimized + debuginfo] target(s) in 0.13s
     Running `target/debug/dns --debug -p https -t dns.adguard.com -n '94.140.14.14:443' query www.example.com. SOA`
; using https:94.140.14.14:443 dns_name:dns.adguard.com
[2022-04-08T20:16:35Z DEBUG trust_dns_proto::https::https_client_stream] tcp connecting to: 94.140.14.14:443
[2022-04-08T20:16:35Z DEBUG trust_dns_proto::https::https_client_stream] tcp connection established to: 94.140.14.14:443
[2022-04-08T20:16:35Z DEBUG trust_dns_proto::https::https_client_stream] tls connection established to: 94.140.14.14:443
[2022-04-08T20:16:35Z DEBUG trust_dns_proto::https::https_client_stream] h2 connection established to: 94.140.14.14:443
; sending query: www.example.com. IN SOA
[2022-04-08T20:16:35Z DEBUG trust_dns_proto::xfer::dns_handle] querying: www.example.com. SOA
[2022-04-08T20:16:35Z DEBUG trust_dns_proto::xfer] enqueueing message:QUERY:[Query { name: Name("www.example.com."), query_type: SOA, query_class: IN, mdns_unicast_response: false }]
[2022-04-08T20:16:35Z DEBUG trust_dns_proto::https::https_client_stream] request: Request {
        method: POST,
        uri: https://dns.adguard.com/dns-query,
        version: HTTP/2.0,
        headers: {
            "content-type": "application/dns-message",
            "accept": "application/dns-message",
            "content-length": "44",
        },
        body: (),
    }
[2022-04-08T20:16:35Z DEBUG trust_dns_proto::https::https_client_stream] got response: Response {
        status: 200,
        version: HTTP/2.0,
        headers: {
            "cache-control": "max-age=1887.000000",
            "content-type": "application/dns-message",
            "content-length": "120",
            "date": "Fri, 08 Apr 2022 20:16:35 GMT",
        },
        body: RecvStream {
            inner: FlowControl {
                inner: OpaqueStreamRef {
                    stream_id: StreamId(
                        1,
                    ),
                    ref_count: 2,
                },
            },
        },
    }
[2022-04-08T20:16:35Z DEBUG trust_dns_proto::https::https_client_stream] got bytes: 120
; received response
; header 0:RD,RA:NoError:QUERY:0/1/1
; edns version: 0 dnssec_ok: false max_payload: 1232 opts: 0
; query
;; name: www.example.com. type: SOA class: IN mdns_unicast_response: false
; answers 0
; nameservers 1
example.com. 1887 IN SOA ns.icann.org. noc.dns.icann.org. 2022040401 7200 3600 1209600 3600
; additionals 1

Version:
Trust-DNS on up-to-date branch trust-dns-client-cli
Quinn version 0.8.2

Additional context

I've also be testing against a different DoQ service setup, which has fewer details, though I have more logs. It appears that the remote is seeing the stream as "finished" before the client has sent all logs. From that server, there are these logs:

86860b5992ac0deb: T= 0.158073, cwin: 15360,flight: 1591,nb_ret: 0,rtt_min: 156641,rtt: 156689,rtt_var: 44098,max_ack_delay: 0,state: 14
86860b5992ac0deb: Receiving 80 bytes from 192.184.128.105:51361 at T=0.158096 (5dc273b6a918a)
86860b5992ac0deb: Receiving packet type: 6 (1rtt protected), S1, Q1,
86860b5992ac0deb:     <fc83ee005bf9a5a7>, Seq: 2 (2), Phi: 0,
86860b5992ac0deb:     Decrypted 54 bytes
86860b5992ac0deb:     ACK (nb=0), 0
86860b5992ac0deb:     Stream 0, offset 0, length 46, fin = 1: 2c00000001000001...

86860b5992ac0deb: Quicdoq: Stream FIN before query was received fully on stream  #0.

86860b5992ac0deb: Data callback (1, l=46) on stream 0 returns error 0x1
86860b5992ac0deb: Protocol error 0x1
86860b5992ac0deb: T= 0.158096, cwin: 15360,flight: 1591,nb_ret: 0,rtt_min: 156641,rtt: 156689,rtt_var: 44098,max_ack_delay: 0,state: 15

Notice the "Stream FIN before query was received.

In the code we have this structure for sending quic packets:

github.com/bluejekyll/trust-dns@e6ab219f81/crates/proto/src/quic/quic_client_stream.rs (L58-L74)

Which appears to follow the RFC:

https://www.ietf.org/archive/id/draft-ietf-dprive-dnsoquic-11.html#name-stream-mapping-and-usage

@Ralith or @djc not sure if this looks familiar to you?

I'm not sure if these issues would have anything to do with the compatibility tests here: https://interop.seemann.io/
https://github.com/private-octopus/quicdoq

Originally created by @bluejekyll on GitHub (Apr 8, 2022). Original GitHub issue: https://github.com/hickory-dns/hickory-dns/issues/1687 **Describe the bug** When using DoQ for queries, the QUIC protocol errors with: ```shell ; using quic:94.140.14.14:853 dns_name:dns.adguard.com ; sending query: www.example.com. IN SOA [2022-04-08T19:06:28Z DEBUG trust_dns_proto::xfer::dns_handle] querying: www.example.com. SOA [2022-04-08T19:06:28Z DEBUG trust_dns_proto::xfer] enqueueing message:QUERY:[Query { name: Name("www.example.com."), query_type: SOA, query_class: IN, mdns_unicast_response: false }] [2022-04-08T19:06:28Z DEBUG trust_dns_proto::quic::quic_stream] received packet len: 44 bytes: b"\0\0\x01\0\0\x01\0\0\0\0\0\x01\x03www\x07example\x03com\0\0\x06\0\x01\0\0)\x04\xd0\0\0\0\0\0\0" Error: Error { kind: Proto(ProtoError { kind: QuinnReadError(FinishedEarly), backtrack: None }), backtrack: None } ``` **To Reproduce** The current easiest way to reproduce this is to use the new `dns` utility from the project, see [trust-dns-client-cli](https://github.com/bluejekyll/trust-dns/tree/trust-dns-client-cli): `cargo run --all-features --bin dns -- --debug -p quic -t dns.adguard.com -n 94.140.14.14:853 query www.example.com. SOA` edit: it appears that adguard may require a custom ALPN, `doq-i02`, as opposed to the RFC's ALPN of `doq`, the proper command for that is, but has the same result: `cargo run --all-features --bin dns -- --debug -p quic -t dns.adguard.com -n 94.140.14.14:853 -a doq-i02 query www.example.com. SOA` **Expected behavior** Correct behavior with DoH can be seen with the same server: ```shell $> cargo run --all-features --bin dns -- --debug -p https -t dns.adguard.com -n 94.140.14.14:443 query www.example.com. SOA Finished dev [unoptimized + debuginfo] target(s) in 0.13s Running `target/debug/dns --debug -p https -t dns.adguard.com -n '94.140.14.14:443' query www.example.com. SOA` ; using https:94.140.14.14:443 dns_name:dns.adguard.com [2022-04-08T20:16:35Z DEBUG trust_dns_proto::https::https_client_stream] tcp connecting to: 94.140.14.14:443 [2022-04-08T20:16:35Z DEBUG trust_dns_proto::https::https_client_stream] tcp connection established to: 94.140.14.14:443 [2022-04-08T20:16:35Z DEBUG trust_dns_proto::https::https_client_stream] tls connection established to: 94.140.14.14:443 [2022-04-08T20:16:35Z DEBUG trust_dns_proto::https::https_client_stream] h2 connection established to: 94.140.14.14:443 ; sending query: www.example.com. IN SOA [2022-04-08T20:16:35Z DEBUG trust_dns_proto::xfer::dns_handle] querying: www.example.com. SOA [2022-04-08T20:16:35Z DEBUG trust_dns_proto::xfer] enqueueing message:QUERY:[Query { name: Name("www.example.com."), query_type: SOA, query_class: IN, mdns_unicast_response: false }] [2022-04-08T20:16:35Z DEBUG trust_dns_proto::https::https_client_stream] request: Request { method: POST, uri: https://dns.adguard.com/dns-query, version: HTTP/2.0, headers: { "content-type": "application/dns-message", "accept": "application/dns-message", "content-length": "44", }, body: (), } [2022-04-08T20:16:35Z DEBUG trust_dns_proto::https::https_client_stream] got response: Response { status: 200, version: HTTP/2.0, headers: { "cache-control": "max-age=1887.000000", "content-type": "application/dns-message", "content-length": "120", "date": "Fri, 08 Apr 2022 20:16:35 GMT", }, body: RecvStream { inner: FlowControl { inner: OpaqueStreamRef { stream_id: StreamId( 1, ), ref_count: 2, }, }, }, } [2022-04-08T20:16:35Z DEBUG trust_dns_proto::https::https_client_stream] got bytes: 120 ; received response ; header 0:RD,RA:NoError:QUERY:0/1/1 ; edns version: 0 dnssec_ok: false max_payload: 1232 opts: 0 ; query ;; name: www.example.com. type: SOA class: IN mdns_unicast_response: false ; answers 0 ; nameservers 1 example.com. 1887 IN SOA ns.icann.org. noc.dns.icann.org. 2022040401 7200 3600 1209600 3600 ; additionals 1 ``` **Version:** Trust-DNS on up-to-date branch [trust-dns-client-cli](https://github.com/bluejekyll/trust-dns/tree/trust-dns-client-cli) Quinn version `0.8.2` **Additional context** I've also be testing against a different DoQ service setup, which has fewer details, though I have more logs. It appears that the remote is seeing the stream as "finished" before the client has sent all logs. From that server, there are these logs: ```text 86860b5992ac0deb: T= 0.158073, cwin: 15360,flight: 1591,nb_ret: 0,rtt_min: 156641,rtt: 156689,rtt_var: 44098,max_ack_delay: 0,state: 14 86860b5992ac0deb: Receiving 80 bytes from 192.184.128.105:51361 at T=0.158096 (5dc273b6a918a) 86860b5992ac0deb: Receiving packet type: 6 (1rtt protected), S1, Q1, 86860b5992ac0deb: <fc83ee005bf9a5a7>, Seq: 2 (2), Phi: 0, 86860b5992ac0deb: Decrypted 54 bytes 86860b5992ac0deb: ACK (nb=0), 0 86860b5992ac0deb: Stream 0, offset 0, length 46, fin = 1: 2c00000001000001... 86860b5992ac0deb: Quicdoq: Stream FIN before query was received fully on stream #0. 86860b5992ac0deb: Data callback (1, l=46) on stream 0 returns error 0x1 86860b5992ac0deb: Protocol error 0x1 86860b5992ac0deb: T= 0.158096, cwin: 15360,flight: 1591,nb_ret: 0,rtt_min: 156641,rtt: 156689,rtt_var: 44098,max_ack_delay: 0,state: 15 ``` Notice the "Stream FIN before query was received. In the code we have this structure for sending quic packets: https://github.com/bluejekyll/trust-dns/blob/e6ab219f81443f69765862d6a69f9f4f37319280/crates/proto/src/quic/quic_client_stream.rs#L58-L74 Which appears to follow the RFC: https://www.ietf.org/archive/id/draft-ietf-dprive-dnsoquic-11.html#name-stream-mapping-and-usage @Ralith or @djc not sure if this looks familiar to you? I'm not sure if these issues would have anything to do with the compatibility tests here: https://interop.seemann.io/ https://github.com/private-octopus/quicdoq
Author
Owner

@Ralith commented on GitHub (Apr 8, 2022):

Assuming nothing interesting is happening in the QuicStream wrapper, your QUIC use looks sensible. I'd verify that message.into_parts().0 is correct, then try to determine where exactly the peer sees the stream getting cut off, and verify that send isn't failing with some other error halfway through queuing data. Wireshark might be useful to provide insight into what data was actually sent.

<!-- gh-comment-id:1093389160 --> @Ralith commented on GitHub (Apr 8, 2022): Assuming nothing interesting is happening in the `QuicStream` wrapper, your QUIC use looks sensible. I'd verify that `message.into_parts().0` is correct, then try to determine where exactly the peer sees the stream getting cut off, and verify that `send` isn't failing with some other error halfway through queuing data. Wireshark might be useful to provide insight into what data was actually sent.
Author
Owner

@bluejekyll commented on GitHub (Apr 8, 2022):

End-to-end tests work locally so there isn't an issue with the DNS packet as far as I can tell. I'll see what we can see with wireshark.

<!-- gh-comment-id:1093451208 --> @bluejekyll commented on GitHub (Apr 8, 2022): End-to-end tests work locally so there isn't an issue with the DNS packet as far as I can tell. I'll see what we can see with wireshark.
Author
Owner

@bluejekyll commented on GitHub (Apr 8, 2022):

And thanks for taking the time.

<!-- gh-comment-id:1093451407 --> @bluejekyll commented on GitHub (Apr 8, 2022): And thanks for taking the time.
Author
Owner

@IvanNardi commented on GitHub (Apr 9, 2022):

edit: it appears that adguard may require a custom ALPN, doq-i02, as opposed to the RFC's ALPN of doq, the proper command for that is, but has the same result:

Not an expert, but is it possible that Adguard is still using DoQ draft-02, instead of the latest versions?
Draft-02 is incompatible with latest versions because it doesn't have the "2-octet length field" of the beginning of the message.
My 2 cents

<!-- gh-comment-id:1093839217 --> @IvanNardi commented on GitHub (Apr 9, 2022): > edit: it appears that adguard may require a custom ALPN, `doq-i02`, as opposed to the RFC's ALPN of `doq`, the proper command for that is, but has the same result: Not an expert, but is it possible that Adguard is still using DoQ draft-02, instead of the latest versions? Draft-02 is incompatible with latest versions because it doesn't have the "2-octet length field" of the beginning of the message. My 2 cents
Author
Owner

@djc commented on GitHub (Apr 9, 2022):

That sounds very plausible.

<!-- gh-comment-id:1093902796 --> @djc commented on GitHub (Apr 9, 2022): That sounds very plausible.
Author
Owner

@bluejekyll commented on GitHub (Apr 9, 2022):

Draft-02 is incompatible with latest versions because it doesn't have the "2-octet length field" of the beginning of the message.

Oh... that's significant, and I didn't notice that. I'll focus on the other service. There's a setup guide for testing with it that I need to go through. I'll see when I have time to do that. I don't think it's necessarily worth providing an option for that, but I suppose I could hack it temporarily to see if I can get it to work.

This is the current doc I'm working off of that links to other implementations: https://docs.google.com/document/d/1aMVwZf979-xa6wbQUx6pm56Kk0AYI0FoY41tBqmniKk/edit#

<!-- gh-comment-id:1094096830 --> @bluejekyll commented on GitHub (Apr 9, 2022): > Draft-02 is incompatible with latest versions because it doesn't have the "2-octet length field" of the beginning of the message. Oh... that's significant, and I didn't notice that. I'll focus on the other service. There's a setup guide for testing with it that I need to go through. I'll see when I have time to do that. I don't think it's necessarily worth providing an option for that, but I suppose I could hack it temporarily to see if I can get it to work. This is the current doc I'm working off of that links to other implementations: https://docs.google.com/document/d/1aMVwZf979-xa6wbQUx6pm56Kk0AYI0FoY41tBqmniKk/edit#
Author
Owner

@bluejekyll commented on GitHub (Apr 9, 2022):

@Ralith thanks for noticing that.

I just hacked the current implementation to remove the 2 byte length from the stream, and it indeed works as expected with dns.adguard.coms quic implementation on doq-i02.

So that's a nice validation.

<!-- gh-comment-id:1094132867 --> @bluejekyll commented on GitHub (Apr 9, 2022): @Ralith thanks for noticing that. I just hacked the current implementation to remove the 2 byte length from the stream, and it indeed works as expected with `dns.adguard.com`s quic implementation on `doq-i02`. So that's a nice validation.
Author
Owner

@bluejekyll commented on GitHub (Apr 9, 2022):

I'm going to open a separate issue in regards to the doq-i03 issues.

<!-- gh-comment-id:1094133002 --> @bluejekyll commented on GitHub (Apr 9, 2022): I'm going to open a separate issue in regards to the `doq-i03` issues.
Author
Owner

@bluejekyll commented on GitHub (Apr 9, 2022):

We won't fix this issue, as the dns.adguard.com name server isn't on the current RFC, I don't think it makes sense for us to try and support different versions of the RFC. We'll only focus on the current (as of this writing, version 11): https://www.ietf.org/archive/id/draft-ietf-dprive-dnsoquic-11.html#name-reservation-of-dedicated-po

<!-- gh-comment-id:1094133339 --> @bluejekyll commented on GitHub (Apr 9, 2022): We won't fix this issue, as the `dns.adguard.com` name server isn't on the current RFC, I don't think it makes sense for us to try and support different versions of the RFC. We'll only focus on the current (as of this writing, version 11): https://www.ietf.org/archive/id/draft-ietf-dprive-dnsoquic-11.html#name-reservation-of-dedicated-po
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/hickory-dns#736
No description provided.