[GH-ISSUE #339] Panic when serializing some messages #153

Closed
opened 2026-03-07 22:29:01 +03:00 by kerem · 5 comments
Owner

Originally created by @SAPikachu on GitHub (Feb 9, 2018).
Original GitHub issue: https://github.com/hickory-dns/hickory-dns/issues/339

Test case:

#[test]
fn test_serialization_panic() {
    use std::str::FromStr;
    use ::client::{Client, SyncClient};
    use ::udp::UdpClientConnection;
    use ::op::Message;
    use ::rr::{DNSClass, Name, RecordType};
    use trust_dns_proto::serialize::binary::BinEncodable;

    let address = "8.8.8.8:53".parse().unwrap();
    let conn = UdpClientConnection::new(address).unwrap();
    let client = SyncClient::new(conn);

    let name = Name::from_str("bluedot.is.autonavi.com.gds.alibabadns.com.").unwrap();

    let response: Message = client.query(&name, DNSClass::IN, RecordType::AAAA).unwrap();
    response.to_bytes().unwrap();
}

Backtrace:

        thread 'test_serialization_panic' panicked at 'assertion failed: end <= self.buffer.len()', /home/sapikachu/Dropbox/devel/trust-dns/proto/src/serialize/binary/encoder.rs:135:8
stack backtrace:
   0:     0x562e533e4583 - std::sys::imp::backtrace::tracing::imp::unwind_backtrace::hd278582b7bf125b4
                               at /checkout/src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1:     0x562e533e0a1c - std::sys_common::backtrace::_print::h585aeaf99da81bd9
                               at /checkout/src/libstd/sys_common/backtrace.rs:68
   2:     0x562e533e6ca1 - std::panicking::default_hook::{{closure}}::h2cf5ae2c765258ee
                               at /checkout/src/libstd/sys_common/backtrace.rs:57
                               at /checkout/src/libstd/panicking.rs:381
   3:     0x562e533e69ae - std::panicking::default_hook::h4df9c6962fc8f370
                               at /checkout/src/libstd/panicking.rs:391
   4:     0x562e533e71aa - std::panicking::rust_panic_with_hook::hec57cb9076b0053e
                               at /checkout/src/libstd/panicking.rs:577
   5:     0x562e53234627 - std::panicking::begin_panic::hb4ad32baf2f8e19c
                               at /checkout/src/libstd/panicking.rs:538
   6:     0x562e53223ed0 - trust_dns_proto::serialize::binary::encoder::BinEncoder::slice_of::hd6bc6edb528a46da
                               at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/serialize/binary/encoder.rs:135
   7:     0x562e53224137 - trust_dns_proto::serialize::binary::encoder::BinEncoder::get_label_pointer::h0051f57ec2c92965
                               at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/serialize/binary/encoder.rs:157
   8:     0x562e53200962 - trust_dns_proto::rr::domain::name::Name::emit_as_canonical::h3c6e550a75edd073
                               at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/rr/domain/name.rs:573
   9:     0x562e53200e80 - trust_dns_proto::rr::domain::name::Name::emit_with_lowercase::hce53fa0679e6e297
                               at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/rr/domain/name.rs:623
  10:     0x562e53256cea - trust_dns_proto::rr::rdata::soa::emit::h64f06f9690b13a9a
                               at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/rr/rdata/soa.rs:247
  11:     0x562e53259da7 - trust_dns_proto::rr::record_data::RData::emit::h1f75731488845f42
                               at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/rr/record_data.rs:533
  12:     0x562e5321922e - <trust_dns_proto::rr::resource::Record as trust_dns_proto::serialize::binary::BinEncodable>::emit::h3e81a2c0206db448
                               at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/rr/resource.rs:240
  13:     0x562e53224d8c - trust_dns_proto::serialize::binary::encoder::BinEncoder::emit_all::h67e7fbab89ff90e6
                               at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/serialize/binary/encoder.rs:252
  14:     0x562e5323375c - <trust_dns_proto::op::message::Message as trust_dns_proto::op::message::EncodableMessage>::emit_name_servers::h671f98ee3d0ee2b4
                               at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/op/message.rs:151
  15:     0x562e53192537 - trust_dns_proto::op::message::<impl trust_dns_proto::serialize::binary::BinEncodable for M>::emit::ha0f97a0f572ec3ef
                               at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/op/message.rs:761
  16:     0x562e53195560 - trust_dns_proto::serialize::binary::BinEncodable::to_bytes::h5f56ce727adfd1ee
                               at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/serialize/binary/mod.rs:41
  17:     0x562e5318c7a3 - trust_dns::test_serialization_panic::h8a270555a23fc466
                               at src/lib.rs:274
  18:     0x562e531caa11 - <F as test::FnBox<T>>::call_box::h0d75d89fa03308a9
                               at /checkout/src/libtest/lib.rs:1510
                               at /checkout/src/libcore/ops/function.rs:223
                               at /checkout/src/libtest/lib.rs:142
  19:     0x562e533efc4e - __rust_maybe_catch_panic
                               at /checkout/src/libpanic_unwind/lib.rs:101
  20:     0x562e531bbfb0 - std::sys_common::backtrace::__rust_begin_short_backtrace::he7d719eb470b8fd9
                               at /checkout/src/libstd/panicking.rs:459
                               at /checkout/src/libstd/panic.rs:365
                               at /checkout/src/libtest/lib.rs:1447
                               at /checkout/src/libstd/sys_common/backtrace.rs:133
  21:     0x562e531bcc82 - std::panicking::try::do_call::h58eedba497ec1dce
                               at /checkout/src/libstd/thread/mod.rs:406
                               at /checkout/src/libstd/panic.rs:300
                               at /checkout/src/libstd/panicking.rs:480
  22:     0x562e533efc4e - __rust_maybe_catch_panic
                               at /checkout/src/libpanic_unwind/lib.rs:101
  23:     0x562e531c4502 - <F as alloc::boxed::FnBox<A>>::call_box::h73a3b3041228d75a
                               at /checkout/src/libstd/panicking.rs:459
                               at /checkout/src/libstd/panic.rs:365
                               at /checkout/src/libstd/thread/mod.rs:405
                               at /checkout/src/liballoc/boxed.rs:762
  24:     0x562e533e607b - std::sys::imp::thread::Thread::new::thread_start::h379d0fc65e150416
                               at /checkout/src/liballoc/boxed.rs:772
                               at /checkout/src/libstd/sys_common/thread.rs:24
                               at /checkout/src/libstd/sys/unix/thread.rs:90
  25:     0x7f5ca0baa6b9 - start_thread
  26:     0x7f5ca06ca41c - clone
  27:                0x0 - <unknown>

I am using latest master code. Any hint for debugging this? :)

Originally created by @SAPikachu on GitHub (Feb 9, 2018). Original GitHub issue: https://github.com/hickory-dns/hickory-dns/issues/339 Test case: ```rust #[test] fn test_serialization_panic() { use std::str::FromStr; use ::client::{Client, SyncClient}; use ::udp::UdpClientConnection; use ::op::Message; use ::rr::{DNSClass, Name, RecordType}; use trust_dns_proto::serialize::binary::BinEncodable; let address = "8.8.8.8:53".parse().unwrap(); let conn = UdpClientConnection::new(address).unwrap(); let client = SyncClient::new(conn); let name = Name::from_str("bluedot.is.autonavi.com.gds.alibabadns.com.").unwrap(); let response: Message = client.query(&name, DNSClass::IN, RecordType::AAAA).unwrap(); response.to_bytes().unwrap(); } ``` Backtrace: ``` thread 'test_serialization_panic' panicked at 'assertion failed: end <= self.buffer.len()', /home/sapikachu/Dropbox/devel/trust-dns/proto/src/serialize/binary/encoder.rs:135:8 stack backtrace: 0: 0x562e533e4583 - std::sys::imp::backtrace::tracing::imp::unwind_backtrace::hd278582b7bf125b4 at /checkout/src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:49 1: 0x562e533e0a1c - std::sys_common::backtrace::_print::h585aeaf99da81bd9 at /checkout/src/libstd/sys_common/backtrace.rs:68 2: 0x562e533e6ca1 - std::panicking::default_hook::{{closure}}::h2cf5ae2c765258ee at /checkout/src/libstd/sys_common/backtrace.rs:57 at /checkout/src/libstd/panicking.rs:381 3: 0x562e533e69ae - std::panicking::default_hook::h4df9c6962fc8f370 at /checkout/src/libstd/panicking.rs:391 4: 0x562e533e71aa - std::panicking::rust_panic_with_hook::hec57cb9076b0053e at /checkout/src/libstd/panicking.rs:577 5: 0x562e53234627 - std::panicking::begin_panic::hb4ad32baf2f8e19c at /checkout/src/libstd/panicking.rs:538 6: 0x562e53223ed0 - trust_dns_proto::serialize::binary::encoder::BinEncoder::slice_of::hd6bc6edb528a46da at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/serialize/binary/encoder.rs:135 7: 0x562e53224137 - trust_dns_proto::serialize::binary::encoder::BinEncoder::get_label_pointer::h0051f57ec2c92965 at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/serialize/binary/encoder.rs:157 8: 0x562e53200962 - trust_dns_proto::rr::domain::name::Name::emit_as_canonical::h3c6e550a75edd073 at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/rr/domain/name.rs:573 9: 0x562e53200e80 - trust_dns_proto::rr::domain::name::Name::emit_with_lowercase::hce53fa0679e6e297 at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/rr/domain/name.rs:623 10: 0x562e53256cea - trust_dns_proto::rr::rdata::soa::emit::h64f06f9690b13a9a at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/rr/rdata/soa.rs:247 11: 0x562e53259da7 - trust_dns_proto::rr::record_data::RData::emit::h1f75731488845f42 at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/rr/record_data.rs:533 12: 0x562e5321922e - <trust_dns_proto::rr::resource::Record as trust_dns_proto::serialize::binary::BinEncodable>::emit::h3e81a2c0206db448 at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/rr/resource.rs:240 13: 0x562e53224d8c - trust_dns_proto::serialize::binary::encoder::BinEncoder::emit_all::h67e7fbab89ff90e6 at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/serialize/binary/encoder.rs:252 14: 0x562e5323375c - <trust_dns_proto::op::message::Message as trust_dns_proto::op::message::EncodableMessage>::emit_name_servers::h671f98ee3d0ee2b4 at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/op/message.rs:151 15: 0x562e53192537 - trust_dns_proto::op::message::<impl trust_dns_proto::serialize::binary::BinEncodable for M>::emit::ha0f97a0f572ec3ef at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/op/message.rs:761 16: 0x562e53195560 - trust_dns_proto::serialize::binary::BinEncodable::to_bytes::h5f56ce727adfd1ee at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/serialize/binary/mod.rs:41 17: 0x562e5318c7a3 - trust_dns::test_serialization_panic::h8a270555a23fc466 at src/lib.rs:274 18: 0x562e531caa11 - <F as test::FnBox<T>>::call_box::h0d75d89fa03308a9 at /checkout/src/libtest/lib.rs:1510 at /checkout/src/libcore/ops/function.rs:223 at /checkout/src/libtest/lib.rs:142 19: 0x562e533efc4e - __rust_maybe_catch_panic at /checkout/src/libpanic_unwind/lib.rs:101 20: 0x562e531bbfb0 - std::sys_common::backtrace::__rust_begin_short_backtrace::he7d719eb470b8fd9 at /checkout/src/libstd/panicking.rs:459 at /checkout/src/libstd/panic.rs:365 at /checkout/src/libtest/lib.rs:1447 at /checkout/src/libstd/sys_common/backtrace.rs:133 21: 0x562e531bcc82 - std::panicking::try::do_call::h58eedba497ec1dce at /checkout/src/libstd/thread/mod.rs:406 at /checkout/src/libstd/panic.rs:300 at /checkout/src/libstd/panicking.rs:480 22: 0x562e533efc4e - __rust_maybe_catch_panic at /checkout/src/libpanic_unwind/lib.rs:101 23: 0x562e531c4502 - <F as alloc::boxed::FnBox<A>>::call_box::h73a3b3041228d75a at /checkout/src/libstd/panicking.rs:459 at /checkout/src/libstd/panic.rs:365 at /checkout/src/libstd/thread/mod.rs:405 at /checkout/src/liballoc/boxed.rs:762 24: 0x562e533e607b - std::sys::imp::thread::Thread::new::thread_start::h379d0fc65e150416 at /checkout/src/liballoc/boxed.rs:772 at /checkout/src/libstd/sys_common/thread.rs:24 at /checkout/src/libstd/sys/unix/thread.rs:90 25: 0x7f5ca0baa6b9 - start_thread 26: 0x7f5ca06ca41c - clone 27: 0x0 - <unknown> ``` I am using latest master code. Any hint for debugging this? :)
kerem 2026-03-07 22:29:01 +03:00
Author
Owner

@bluejekyll commented on GitHub (Feb 9, 2018):

These two lines from the trace look to be the most interesting:

6: 0x562e53223ed0 - trust_dns_proto::serialize::binary::encoder::BinEncoder::slice_of::hd6bc6edb528a46da
at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/serialize/binary/encoder.rs:135
7: 0x562e53224137 - trust_dns_proto::serialize::binary::encoder::BinEncoder::get_label_pointer::h0051f57ec2c92965
at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/serialize/binary/encoder.rs:15

It looks like the label offset pointers are having a problem for some reason. My guess is that the logic around that in the serialization of the Name as it relates to the label compression is having an issue, not sure why (I did rewrite that section of code in 0.8):

https://github.com/bluejekyll/trust-dns/blob/master/proto/src/rr/domain/name.rs#L549-L592

But that looks like where it's probably coming from?

<!-- gh-comment-id:364600514 --> @bluejekyll commented on GitHub (Feb 9, 2018): These two lines from the trace look to be the most interesting: > 6: 0x562e53223ed0 - trust_dns_proto::serialize::binary::encoder::BinEncoder::slice_of::hd6bc6edb528a46da at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/serialize/binary/encoder.rs:135 > 7: 0x562e53224137 - trust_dns_proto::serialize::binary::encoder::BinEncoder::get_label_pointer::h0051f57ec2c92965 at /home/sapikachu/Dropbox/devel/trust-dns/proto/src/serialize/binary/encoder.rs:15 It looks like the label offset pointers are having a problem for some reason. My guess is that the logic around that in the serialization of the Name as it relates to the label compression is having an issue, not sure why (I did rewrite that section of code in 0.8): https://github.com/bluejekyll/trust-dns/blob/master/proto/src/rr/domain/name.rs#L549-L592 But that looks like where it's probably coming from?
Author
Owner

@bluejekyll commented on GitHub (Feb 9, 2018):

If you use VSCode, I like to throw this in the lldb plugin and step through the code to see what's going on, personally.

<!-- gh-comment-id:364600604 --> @bluejekyll commented on GitHub (Feb 9, 2018): If you use VSCode, I like to throw this in the lldb plugin and step through the code to see what's going on, personally.
Author
Owner

@SAPikachu commented on GitHub (Feb 10, 2018):

Thanks for the hint! It seems Name::emit_as_canonical stores label pointers for all labels, then rewrites one of the label in middle of the name and invalidates saved pointers beyond that label, and caused the panic. If I change trim to:

    pub fn trim(&mut self) {
        let offset = self.offset;
        self.buffer.truncate(offset as usize);
        self.name_pointers.retain(|&(start, end)| start < offset && end <= offset);
    }

Then the test would pass. Do you see any problem with this change?

<!-- gh-comment-id:364625778 --> @SAPikachu commented on GitHub (Feb 10, 2018): Thanks for the hint! It seems `Name::emit_as_canonical` stores label pointers for all labels, then rewrites one of the label in middle of the name and invalidates saved pointers beyond that label, and caused the panic. If I change `trim` to: ``` pub fn trim(&mut self) { let offset = self.offset; self.buffer.truncate(offset as usize); self.name_pointers.retain(|&(start, end)| start < offset && end <= offset); } ``` Then the test would pass. Do you see any problem with this change?
Author
Owner

@bluejekyll commented on GitHub (Feb 10, 2018):

Ah, that was an edge condition I hadn’t considered. I think your change is correct, Nice work!

Want to submit a PR and maybe a test case (just with serialization) so that we don’t run into this against?

Thanks!

<!-- gh-comment-id:364628114 --> @bluejekyll commented on GitHub (Feb 10, 2018): Ah, that was an edge condition I hadn’t considered. I think your change is correct, Nice work! Want to submit a PR and maybe a test case (just with serialization) so that we don’t run into this against? Thanks!
Author
Owner

@SAPikachu commented on GitHub (Feb 10, 2018):

Oh great, will submit PR a bit later. :)

<!-- gh-comment-id:364628854 --> @SAPikachu commented on GitHub (Feb 10, 2018): Oh great, will submit PR a bit later. :)
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/hickory-dns#153
No description provided.