[GH-ISSUE #310] Return traffic from local DNS container not traversing VPN tunnel #286

Closed
opened 2026-03-02 08:01:05 +03:00 by kerem · 5 comments
Owner

Originally created by @diamondsw on GitHub (Sep 20, 2022).
Original GitHub issue: https://github.com/hwdsl2/docker-ipsec-vpn-server/issues/310

Summary: VPN and DNS containers on the same host; both work as expected independently, but DNS return traffic does not reach VPN clients. Seems like a docker networking issue, but I'm at my wits' end.

This is going to be a bit of a weird one; thank you for looking! Despite the amount of DNS detail, I swear it's not a DNS issue. The goal here is to provide an internal DNS server for private hostnames on the remote network. While this is my first time using this container, I've configured (painfully) several IPSEC/L2TP VPNs before, with mobileconfig, site-to-site tunnels, and private DNS. First time I've ever seen something quite like this, and I'm completely stumped.

Checklist

I don't know on that last one! If anything, it seems like a very weird Docker network thing.

Describe the issue
After establishing a VPN connection to the ipsec-vpn-server container, return traffic from a dnsmasq container on the same host does not arrive.

More detail below, but in short:

  • The VPN connects without issue (by the way, BRAVO at making IPSEC so easy!)
  • Ping, SSH, and other bidirectional traffic to the host server or internet traverse the VPN without issue.
  • If no DNS server is specified for the VPN (i.e. using built-in 8.8.8.8), then DNS resolution works
  • If a public DNS server is specified (e.g. 1.1.1.1), then DNS resolution works
  • If a local dnsmasq container is specified using the host private IP address, then dnsmasq sees the requests and responds - but the client never sees these responses
  • Meanwhile, accessing the same DNS container over the internet works, and using dig locally on the server works

That's the weirdest thing - the DNS container works, and even sees the requests over the VPN and responds - but the client never gets the traffic back. Other protocols are working fine. The only difference I can see is this is traffic between containers, whereas ping, ssh, etc are all either to the host or headed outside the network.

Network details:

  • Server is on Oracle Cloud and has both a public IP (141.147.x.y - redacted) and private IP (10.2.1.8), and the security list allows USP 500/4500 from all IPs. For the purposes of troubleshooting, it also allows port 53 from my home IP.
  • The public IP is reachable at 'relay.redacted.com', and the private IP at 'relay.oracle.redacted.com'. Apologies if this is confusing!

To Reproduce
Steps to reproduce the behavior:

  1. Setup the VPN server with the following docker-compose.yml:
version: "3"
services:
  ipsec-vpn:
    image: hwdsl2/ipsec-vpn-server:latest
    container_name: ipsec-vpn
    restart: unless-stopped
    environment:
      VPN_IKEV2_ONLY: 'yes'
      VPN_DNS_NAME: 'relay.redacted.com'
      VPN_DNS_SRV1: '10.2.1.8'
    ports:
      - '500:500/udp'
      - '4500:4500/udp'
    volumes:
      - /lib/modules:/lib/modules:ro
      - ./data:/etc/ipsec.d
    cap_add:
      - NET_ADMIN
    devices:
      - "/dev/ppp:/dev/ppp"
    sysctls:
      - net.ipv4.ip_forward=1
      - net.ipv4.conf.all.accept_redirects=0
      - net.ipv4.conf.all.send_redirects=0
      - net.ipv4.conf.all.rp_filter=0
      - net.ipv4.conf.default.accept_redirects=0
      - net.ipv4.conf.default.send_redirects=0
      - net.ipv4.conf.default.rp_filter=0
      - net.ipv4.conf.eth0.send_redirects=0
      - net.ipv4.conf.eth0.rp_filter=0
  1. Setup the DNS server with the following docker-compose.yml and config files:

Dockerfile

FROM alpine:edge
RUN apk --no-cache add dnsmasq
EXPOSE 53 53/udp
ENTRYPOINT ["dnsmasq", "-k", "--log-facility=-"]

docker-compose.yml

version: "3"
services:
  dnsmasq:
    build: .
    image: diamondsw/dnsmasq
    container_name: dnsmasq
    restart: unless-stopped
    environment:
      TZ: 'America/Chicago'
    ports:
      - "53:53/udp"
    volumes:
      - ./dnsmasq.conf:/etc/dnsmasq.d/dnsmasq.conf
      - ./hosts:/etc/dnsmasq.d/hosts
    cap_add:
      - NET_ADMIN

dnsmasq.conf

# Listen on the Docker eth0
interface=eth0

# Standard options
no-resolv
cache-size=10000
local-ttl=2

# Internal DNS
domain-needed
bogus-priv
domain=oracle.redacted.com
expand-hosts
addn-hosts=/etc/dnsmasq.d/hosts

# Forward all public queries to the internet
server=1.1.1.1
server=8.8.8.8

# Debugging
log-queries

Note here that I have specified an interface to listen to, which turns off the dnsmasq '--local-service' option.

hosts

10.2.1.8        relay.oracle.redacted.com

(Note, I have changed the domain names to remove private information.)

Expected behavior
DNS responses should arrive back to the requestor (i.e. the VPN client machine).

Logs
Check logs and VPN status, and add error logs to help explain the problem, if applicable.

DNSmasq log when VPN connected (with a LOT of Apple services connection attempts cut out):

me@kelvin ~ % ssh 10.2.1.8
me@relay ~ $ docker logs --follow dnsmasq
dnsmasq[1]: started, version 2.86 cachesize 10000
dnsmasq[1]: compile time options: IPv6 GNU-getopt no-DBus no-UBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth no-cryptohash no-DNSSEC loop-detect inotify dumpfile
dnsmasq[1]: using nameserver 1.1.1.1#53
dnsmasq[1]: using nameserver 8.8.8.8#53
dnsmasq[1]: read /etc/hosts - 7 addresses
dnsmasq[1]: read /etc/dnsmasq.d/hosts - 1 addresses
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: query[A] relay2.oracle.redacted.com from 172.22.0.1
dnsmasq[1]: forwarded relay2.oracle.redacted.com to 8.8.8.8
dnsmasq[1]: reply relay2.oracle.redacted.com is NODATA-IPv4
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: query[A] relay.redacted.com from 172.22.0.1
dnsmasq[1]: forwarded relay.redacted.com to 8.8.8.8
dnsmasq[1]: reply relay.redacted.com is 141.147.x.y
dnsmasq[1]: query[A] relay.redacted.com from 172.22.0.1
dnsmasq[1]: cached relay.redacted.com is 141.147.x.y
dnsmasq[1]: query[A] relay.redacted.com from 172.22.0.1
dnsmasq[1]: cached relay.redacted.com is 141.147.x.y
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: query[A] relay.redacted.com from 172.22.0.1
dnsmasq[1]: cached relay.redacted.com is 141.147.x.y
dnsmasq[1]: query[A] www.google.com from 172.22.0.1
dnsmasq[1]: forwarded www.google.com to 8.8.8.8
dnsmasq[1]: reply www.google.com is 142.250.206.228
dnsmasq[1]: query[A] www.google.com from 172.22.0.1
dnsmasq[1]: cached www.google.com is 142.250.206.228
dnsmasq[1]: query[A] www.google.com from 172.22.0.1
dnsmasq[1]: cached www.google.com is 142.250.206.228
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1
dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1
dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8
dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1
dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8
dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1
dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8
dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1
dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8
dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1
dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8
dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1
dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1
dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8
^C
me@relay ~ $ 

My dig attempts show up in the dnsmasq log as these lines:

dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1
dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8

Nothing much to show on the VPN side - it's happy:

me@relay vpn $ docker logs ipsec-vpn

VPN credentials not set by user. Generating random PSK and password...

Setting DNS server to 10.2.1.8...

Note: Running in IKEv2-only mode via env file option.
      IPsec/L2TP and IPsec/XAuth ("Cisco IPsec") modes are disabled.

Starting IPsec service...

Setting up IKEv2. This may take a few moments...

================================================

IKEv2 setup successful. Details for IKEv2 mode:

VPN server address: relay.redacted.com
VPN client name: vpnclient

Client configuration is available inside the
Docker container at:
/etc/ipsec.d/vpnclient.p12 (for Windows & Linux)
/etc/ipsec.d/vpnclient.sswan (for Android)
/etc/ipsec.d/vpnclient.mobileconfig (for iOS & macOS)

Next steps: Configure IKEv2 clients. See:
https://vpnsetup.net/clients2

================================================

xl2tpd[1]: Not looking for kernel SAref support.
xl2tpd[1]: Using l2tp kernel support.
xl2tpd[1]: xl2tpd version xl2tpd-1.3.17 started on 51a8334895ce PID:1
xl2tpd[1]: Written by Mark Spencer, Copyright (C) 1998, Adtran, Inc.
xl2tpd[1]: Forked by Scott Balmos and David Stipp, © 2001
xl2tpd[1]: Inherited by Jeff McAdams, © 2002
xl2tpd[1]: Forked again by Xelerance (www.xelerance.com) (C) 2006-2016
xl2tpd[1]: Listening on IP address 0.0.0.0, port 1701

Server (please complete the following information)

  • OS: Debian 11.5, running the cloudinit qcow2 image, but with the standard linux-image-arm64 kernel for ppp support
  • Hosting provider (if applicable): Oracle Cloud, free tier

Client (please complete the following information)

  • Device: MacBook Air M1
  • OS: macOS 12.6 "Monterey"
  • VPN mode: IKEv2

Client (please complete the following information)

  • Device: iPhone X
  • OS: iOS 16.0
  • VPN mode: IKEv2

Additional context

Connecting to the DNS over the internet (restricted to allow my IP only; no open resolvers):

me@kelvin ~ % dig @relay.redacted.com relay.oracle.redacted.com

; <<>> DiG 9.10.6 <<>> @relay.redacted.com relay.oracle.redacted.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 31390
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;relay.oracle.redacted.com.	IN	A

;; ANSWER SECTION:
relay.oracle.redacted.com. 2	IN	A	10.2.1.8

;; Query time: 154 msec
;; SERVER: 141.147.x.y#53(141.147.x.y)
;; WHEN: Mon Sep 19 08:29:49 CDT 2022
;; MSG SIZE  rcvd: 72

Connecting to the DNS from the server itself (i.e. connecting to localhost):

me@kelvin ~ % ssh relay.redacted.com
me@relay ~ $ dig @127.0.0.1 relay.oracle.redacted.com

; <<>> DiG 9.16.27-Debian <<>> @127.0.0.1 relay.oracle.redacted.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49168
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;relay.oracle.redacted.com.	IN	A

;; ANSWER SECTION:
relay.oracle.redacted.com. 2	IN	A	10.2.1.8

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sun Sep 18 14:53:19 CDT 2022
;; MSG SIZE  rcvd: 72

Connecting to the DNS once connected to the VPN (both macOS DNS resolution and dig):
connect vpn

me@kelvin ~ % dig @10.2.1.8 relay.oracle.redacted.com 

; <<>> DiG 9.10.6 <<>> @10.2.1.8 relay.oracle.redacted.com
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached
me@kelvin ~ % ping relay.oracle.redacted.com
ping: cannot resolve relay.oracle.redacted.com: Unknown host
me@kelvin ~ % 

disconnect vpn

Nothing. Weird!

Originally created by @diamondsw on GitHub (Sep 20, 2022). Original GitHub issue: https://github.com/hwdsl2/docker-ipsec-vpn-server/issues/310 Summary: VPN and DNS containers on the same host; both work as expected independently, but DNS return traffic does not reach VPN clients. Seems like a docker networking issue, but I'm at my wits' end. This is going to be a bit of a weird one; thank you for looking! Despite the amount of DNS detail, I swear it's not a DNS issue. The goal here is to provide an internal DNS server for private hostnames on the remote network. While this is my first time using this container, I've configured (painfully) several IPSEC/L2TP VPNs before, with mobileconfig, site-to-site tunnels, and private DNS. First time I've ever seen something quite like this, and I'm completely stumped. **Checklist** - [x] I read the [README](https://github.com/hwdsl2/setup-ipsec-vpn/blob/master/README.md) - [x] I read the [Important notes](https://github.com/hwdsl2/setup-ipsec-vpn/blob/master/README.md#important-notes) - [x] I followed instructions to [configure VPN clients](https://github.com/hwdsl2/setup-ipsec-vpn/blob/master/README.md#next-steps) - [x] I checked [Troubleshooting](https://github.com/hwdsl2/setup-ipsec-vpn/blob/master/docs/clients.md#troubleshooting), [IKEv2 troubleshooting](https://github.com/hwdsl2/setup-ipsec-vpn/blob/master/docs/ikev2-howto.md#troubleshooting) and [VPN status](https://github.com/hwdsl2/setup-ipsec-vpn/blob/master/docs/clients.md#check-logs-and-vpn-status) - [x] I searched existing [Issues](https://github.com/hwdsl2/setup-ipsec-vpn/issues?q=is%3Aissue) - [ ] This bug is about the VPN setup scripts, and not IPsec VPN itself I don't know on that last one! If anything, it seems like a very weird Docker network thing. **Describe the issue** After establishing a VPN connection to the ipsec-vpn-server container, return traffic from a dnsmasq container on the same host does not arrive. More detail below, but in short: - The VPN connects without issue (by the way, BRAVO at making IPSEC so easy!) - Ping, SSH, and other bidirectional traffic to the host server or internet traverse the VPN without issue. - If no DNS server is specified for the VPN (i.e. using built-in 8.8.8.8), then DNS resolution works - If a public DNS server is specified (e.g. 1.1.1.1), then DNS resolution works - If a local dnsmasq container is specified using the host private IP address, then dnsmasq sees the requests and responds - but the client never sees these responses - Meanwhile, accessing the same DNS container over the internet works, and using dig locally on the server works That's the weirdest thing - the DNS container works, and even sees the requests over the VPN and responds - but the client never gets the traffic back. Other protocols are working fine. The only difference I can see is this is **traffic between containers**, whereas ping, ssh, etc are all either to the host or headed outside the network. Network details: - Server is on Oracle Cloud and has both a public IP (141.147.x.y - redacted) and private IP (10.2.1.8), and the security list allows USP 500/4500 from all IPs. For the purposes of troubleshooting, it also allows port 53 from my home IP. - The public IP is reachable at 'relay.redacted.com', and the private IP at 'relay.**oracle**.redacted.com'. Apologies if this is confusing! **To Reproduce** Steps to reproduce the behavior: 1. Setup the VPN server with the following docker-compose.yml: ``` version: "3" services: ipsec-vpn: image: hwdsl2/ipsec-vpn-server:latest container_name: ipsec-vpn restart: unless-stopped environment: VPN_IKEV2_ONLY: 'yes' VPN_DNS_NAME: 'relay.redacted.com' VPN_DNS_SRV1: '10.2.1.8' ports: - '500:500/udp' - '4500:4500/udp' volumes: - /lib/modules:/lib/modules:ro - ./data:/etc/ipsec.d cap_add: - NET_ADMIN devices: - "/dev/ppp:/dev/ppp" sysctls: - net.ipv4.ip_forward=1 - net.ipv4.conf.all.accept_redirects=0 - net.ipv4.conf.all.send_redirects=0 - net.ipv4.conf.all.rp_filter=0 - net.ipv4.conf.default.accept_redirects=0 - net.ipv4.conf.default.send_redirects=0 - net.ipv4.conf.default.rp_filter=0 - net.ipv4.conf.eth0.send_redirects=0 - net.ipv4.conf.eth0.rp_filter=0 ``` 3. Setup the DNS server with the following docker-compose.yml and config files: Dockerfile ``` FROM alpine:edge RUN apk --no-cache add dnsmasq EXPOSE 53 53/udp ENTRYPOINT ["dnsmasq", "-k", "--log-facility=-"] ``` docker-compose.yml ``` version: "3" services: dnsmasq: build: . image: diamondsw/dnsmasq container_name: dnsmasq restart: unless-stopped environment: TZ: 'America/Chicago' ports: - "53:53/udp" volumes: - ./dnsmasq.conf:/etc/dnsmasq.d/dnsmasq.conf - ./hosts:/etc/dnsmasq.d/hosts cap_add: - NET_ADMIN ``` dnsmasq.conf ``` # Listen on the Docker eth0 interface=eth0 # Standard options no-resolv cache-size=10000 local-ttl=2 # Internal DNS domain-needed bogus-priv domain=oracle.redacted.com expand-hosts addn-hosts=/etc/dnsmasq.d/hosts # Forward all public queries to the internet server=1.1.1.1 server=8.8.8.8 # Debugging log-queries ``` Note here that I have specified an interface to listen to, which turns off the dnsmasq '--local-service' option. hosts ``` 10.2.1.8 relay.oracle.redacted.com ``` (Note, I have changed the domain names to remove private information.) **Expected behavior** DNS responses should arrive back to the requestor (i.e. the VPN client machine). **Logs** [Check logs and VPN status](https://github.com/hwdsl2/setup-ipsec-vpn/blob/master/docs/clients.md#check-logs-and-vpn-status), and add error logs to help explain the problem, if applicable. DNSmasq log when VPN connected (with a LOT of Apple services connection attempts cut out): ``` me@kelvin ~ % ssh 10.2.1.8 me@relay ~ $ docker logs --follow dnsmasq dnsmasq[1]: started, version 2.86 cachesize 10000 dnsmasq[1]: compile time options: IPv6 GNU-getopt no-DBus no-UBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth no-cryptohash no-DNSSEC loop-detect inotify dumpfile dnsmasq[1]: using nameserver 1.1.1.1#53 dnsmasq[1]: using nameserver 8.8.8.8#53 dnsmasq[1]: read /etc/hosts - 7 addresses dnsmasq[1]: read /etc/dnsmasq.d/hosts - 1 addresses dnsmasq[1]: config 192.168.43.10 is NXDOMAIN dnsmasq[1]: config 192.168.43.10 is NXDOMAIN dnsmasq[1]: config 192.168.43.10 is NXDOMAIN dnsmasq[1]: query[A] relay2.oracle.redacted.com from 172.22.0.1 dnsmasq[1]: forwarded relay2.oracle.redacted.com to 8.8.8.8 dnsmasq[1]: reply relay2.oracle.redacted.com is NODATA-IPv4 dnsmasq[1]: config 192.168.43.10 is NXDOMAIN dnsmasq[1]: query[A] relay.redacted.com from 172.22.0.1 dnsmasq[1]: forwarded relay.redacted.com to 8.8.8.8 dnsmasq[1]: reply relay.redacted.com is 141.147.x.y dnsmasq[1]: query[A] relay.redacted.com from 172.22.0.1 dnsmasq[1]: cached relay.redacted.com is 141.147.x.y dnsmasq[1]: query[A] relay.redacted.com from 172.22.0.1 dnsmasq[1]: cached relay.redacted.com is 141.147.x.y dnsmasq[1]: config 192.168.43.10 is NXDOMAIN dnsmasq[1]: query[A] relay.redacted.com from 172.22.0.1 dnsmasq[1]: cached relay.redacted.com is 141.147.x.y dnsmasq[1]: query[A] www.google.com from 172.22.0.1 dnsmasq[1]: forwarded www.google.com to 8.8.8.8 dnsmasq[1]: reply www.google.com is 142.250.206.228 dnsmasq[1]: query[A] www.google.com from 172.22.0.1 dnsmasq[1]: cached www.google.com is 142.250.206.228 dnsmasq[1]: query[A] www.google.com from 172.22.0.1 dnsmasq[1]: cached www.google.com is 142.250.206.228 dnsmasq[1]: config 192.168.43.10 is NXDOMAIN dnsmasq[1]: config 192.168.43.10 is NXDOMAIN dnsmasq[1]: config 192.168.43.10 is NXDOMAIN dnsmasq[1]: config 192.168.43.10 is NXDOMAIN dnsmasq[1]: config 192.168.43.10 is NXDOMAIN dnsmasq[1]: config 192.168.43.10 is NXDOMAIN dnsmasq[1]: config 192.168.43.10 is NXDOMAIN dnsmasq[1]: config 192.168.43.10 is NXDOMAIN dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1 dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8 dnsmasq[1]: config 192.168.43.10 is NXDOMAIN dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1 dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8 dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1 dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8 dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1 dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8 dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1 dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8 dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1 dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8 dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1 dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8 dnsmasq[1]: config 192.168.43.10 is NXDOMAIN dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1 dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8 ^C me@relay ~ $ ``` My dig attempts show up in the dnsmasq log as these lines: ``` dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1 dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8 ``` Nothing much to show on the VPN side - it's happy: ``` me@relay vpn $ docker logs ipsec-vpn VPN credentials not set by user. Generating random PSK and password... Setting DNS server to 10.2.1.8... Note: Running in IKEv2-only mode via env file option. IPsec/L2TP and IPsec/XAuth ("Cisco IPsec") modes are disabled. Starting IPsec service... Setting up IKEv2. This may take a few moments... ================================================ IKEv2 setup successful. Details for IKEv2 mode: VPN server address: relay.redacted.com VPN client name: vpnclient Client configuration is available inside the Docker container at: /etc/ipsec.d/vpnclient.p12 (for Windows & Linux) /etc/ipsec.d/vpnclient.sswan (for Android) /etc/ipsec.d/vpnclient.mobileconfig (for iOS & macOS) Next steps: Configure IKEv2 clients. See: https://vpnsetup.net/clients2 ================================================ xl2tpd[1]: Not looking for kernel SAref support. xl2tpd[1]: Using l2tp kernel support. xl2tpd[1]: xl2tpd version xl2tpd-1.3.17 started on 51a8334895ce PID:1 xl2tpd[1]: Written by Mark Spencer, Copyright (C) 1998, Adtran, Inc. xl2tpd[1]: Forked by Scott Balmos and David Stipp, © 2001 xl2tpd[1]: Inherited by Jeff McAdams, © 2002 xl2tpd[1]: Forked again by Xelerance (www.xelerance.com) (C) 2006-2016 xl2tpd[1]: Listening on IP address 0.0.0.0, port 1701 ``` **Server (please complete the following information)** - OS: Debian 11.5, running the cloudinit qcow2 image, but with the standard linux-image-arm64 kernel for ppp support - Hosting provider (if applicable): Oracle Cloud, free tier **Client (please complete the following information)** - Device: MacBook Air M1 - OS: macOS 12.6 "Monterey" - VPN mode: IKEv2 **Client (please complete the following information)** - Device: iPhone X - OS: iOS 16.0 - VPN mode: IKEv2 **Additional context** Connecting to the DNS over the internet (restricted to allow my IP only; no open resolvers): ``` me@kelvin ~ % dig @relay.redacted.com relay.oracle.redacted.com ; <<>> DiG 9.10.6 <<>> @relay.redacted.com relay.oracle.redacted.com ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 31390 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ;; QUESTION SECTION: ;relay.oracle.redacted.com. IN A ;; ANSWER SECTION: relay.oracle.redacted.com. 2 IN A 10.2.1.8 ;; Query time: 154 msec ;; SERVER: 141.147.x.y#53(141.147.x.y) ;; WHEN: Mon Sep 19 08:29:49 CDT 2022 ;; MSG SIZE rcvd: 72 ``` Connecting to the DNS from the server itself (i.e. connecting to localhost): ``` me@kelvin ~ % ssh relay.redacted.com me@relay ~ $ dig @127.0.0.1 relay.oracle.redacted.com ; <<>> DiG 9.16.27-Debian <<>> @127.0.0.1 relay.oracle.redacted.com ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49168 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ;; QUESTION SECTION: ;relay.oracle.redacted.com. IN A ;; ANSWER SECTION: relay.oracle.redacted.com. 2 IN A 10.2.1.8 ;; Query time: 0 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Sun Sep 18 14:53:19 CDT 2022 ;; MSG SIZE rcvd: 72 ``` Connecting to the DNS once connected to the VPN (both macOS DNS resolution and dig): _connect vpn_ ``` me@kelvin ~ % dig @10.2.1.8 relay.oracle.redacted.com ; <<>> DiG 9.10.6 <<>> @10.2.1.8 relay.oracle.redacted.com ; (1 server found) ;; global options: +cmd ;; connection timed out; no servers could be reached me@kelvin ~ % ping relay.oracle.redacted.com ping: cannot resolve relay.oracle.redacted.com: Unknown host me@kelvin ~ % ``` _disconnect vpn_ Nothing. Weird!
kerem closed this issue 2026-03-02 08:01:06 +03:00
Author
Owner

@letoams commented on GitHub (Sep 21, 2022):

Two common issues could be affecting you

One is that the ip range you assign to the vpn clients isn’t fully routed to it from the dns server and the reply packet goes out the wrong way via a default gateway bypassing the ipsec server.

Two is that somewhere this reply packet is getting NATed and then no longer matches the ipsec policies and is getting dropped.

Sent using a virtual keyboard on a phone

On Sep 20, 2022, at 19:10, diamondsw @.***> wrote:


This is going to be a bit of a weird one; thank you for looking! Despite the amount of DNS detail, I swear it's not a DNS issue. The goal here is to provide an internal DNS server for private hostnames on the remote network.

While this is my first time using this container, I've configured (painfully) several IPSEC/L2TP VPNs before, with mobileconfig, site-to-site tunnels, and private DNS. First time I've ever seen something quite like this, and I'm completely stumped.

Checklist

I read the README
I read the Important notes
I followed instructions to configure VPN clients
I checked Troubleshooting, IKEv2 troubleshooting and VPN status
I searched existing Issues
This bug is about the VPN setup scripts, and not IPsec VPN itself
I don't know on that last one! If anything, it seems like a very weird Docker network thing.

Describe the issue
After establishing a VPN connection to the ipsec-vpn-server container, return traffic from a dnsmasq container on the same host does not arrive.

More detail below, but in short:

The VPN connects without issue (by the way, BRAVO at making IPSEC so easy!)
Ping, SSH, and other bidirectional traffic to the host server or internet traverse the VPN without issue.
If no DNS server is specified for the VPN (i.e. using built-in 8.8.8.8), then DNS resolution works
If a public DNS server is specified (e.g. 1.1.1.1), then DNS resolution works
If a local dnsmasq container is specified using the host private IP address, then dnsmasq sees the requests and responds - but the client never sees these responses
Meanwhile, accessing the same DNS container over the internet works, and using dig locally on the server works
That's the weirdest thing - the DNS container works, and even sees the requests over the VPN and responds - but the client never gets the traffic back. Other protocols are working fine. The only difference I can see is this is traffic between containers, whereas ping, ssh, etc are all either to the host or headed outside the network.

Network details:

Server is on Oracle Cloud and has both a public IP (141.147.x.y - redacted) and private IP (10.2.1.8), and the security list allows USP 500/4500 from all IPs. For the purposes of troubleshooting, it also allows port 53 from my home IP.
The public IP is reachable at 'relay.redacted.com', and the private IP at 'relay.oracle.redacted.com'. Apologies if this is confusing!
To Reproduce
Steps to reproduce the behavior:

Setup the VPN server with the following docker-compose.yml:
version: "3"
services:
ipsec-vpn:
image: hwdsl2/ipsec-vpn-server:latest
container_name: ipsec-vpn
restart: unless-stopped
environment:
VPN_IKEV2_ONLY: 'yes'
VPN_DNS_NAME: 'relay.redacted.com'
VPN_DNS_SRV1: '10.2.1.8'
ports:
- '500:500/udp'
- '4500:4500/udp'
volumes:
- /lib/modules:/lib/modules:ro
- ./data:/etc/ipsec.d
cap_add:
- NET_ADMIN
devices:
- "/dev/ppp:/dev/ppp"
sysctls:
- net.ipv4.ip_forward=1
- net.ipv4.conf.all.accept_redirects=0
- net.ipv4.conf.all.send_redirects=0
- net.ipv4.conf.all.rp_filter=0
- net.ipv4.conf.default.accept_redirects=0
- net.ipv4.conf.default.send_redirects=0
- net.ipv4.conf.default.rp_filter=0
- net.ipv4.conf.eth0.send_redirects=0
- net.ipv4.conf.eth0.rp_filter=0
Setup the DNS server with the following docker-compose.yml and config files:
Dockerfile

FROM alpine:edge
RUN apk --no-cache add dnsmasq
EXPOSE 53 53/udp
ENTRYPOINT ["dnsmasq", "-k", "--log-facility=-"]
docker-compose.yml

version: "3"
services:
dnsmasq:
build: .
image: diamondsw/dnsmasq
container_name: dnsmasq
restart: unless-stopped
environment:
TZ: 'America/Chicago'
ports:
- "53:53/udp"
volumes:
- ./dnsmasq.conf:/etc/dnsmasq.d/dnsmasq.conf
- ./hosts:/etc/dnsmasq.d/hosts
cap_add:
- NET_ADMIN
dnsmasq.conf

Listen on the Docker eth0

interface=eth0

Standard options

no-resolv
cache-size=10000
local-ttl=2

Internal DNS

domain-needed
bogus-priv
domain=oracle.redacted.com
expand-hosts
addn-hosts=/etc/dnsmasq.d/hosts

Forward all public queries to the internet

server=1.1.1.1
server=8.8.8.8

Debugging

log-queries
Note here that I have specified an interface to listen to, which turns off the dnsmasq '--local-service' option.

hosts

10.2.1.8 relay.oracle.redacted.com
(Note, I have changed the domain names to remove private information.)

Expected behavior
DNS responses should arrive back to the requestor (i.e. the VPN client machine).

Logs
Check logs and VPN status, and add error logs to help explain the problem, if applicable.

DNSmasq log when VPN connected (with a LOT of Apple services connection attempts cut out):

@.*** ~ % ssh 10.2.1.8
@.*** ~ $ docker logs --follow dnsmasq
dnsmasq[1]: started, version 2.86 cachesize 10000
dnsmasq[1]: compile time options: IPv6 GNU-getopt no-DBus no-UBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth no-cryptohash no-DNSSEC loop-detect inotify dumpfile
dnsmasq[1]: using nameserver 1.1.1.1#53
dnsmasq[1]: using nameserver 8.8.8.8#53
dnsmasq[1]: read /etc/hosts - 7 addresses
dnsmasq[1]: read /etc/dnsmasq.d/hosts - 1 addresses
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: query[A] relay2.oracle.redacted.com from 172.22.0.1
dnsmasq[1]: forwarded relay2.oracle.redacted.com to 8.8.8.8
dnsmasq[1]: reply relay2.oracle.redacted.com is NODATA-IPv4
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: query[A] relay.redacted.com from 172.22.0.1
dnsmasq[1]: forwarded relay.redacted.com to 8.8.8.8
dnsmasq[1]: reply relay.redacted.com is 141.147.x.y
dnsmasq[1]: query[A] relay.redacted.com from 172.22.0.1
dnsmasq[1]: cached relay.redacted.com is 141.147.x.y
dnsmasq[1]: query[A] relay.redacted.com from 172.22.0.1
dnsmasq[1]: cached relay.redacted.com is 141.147.x.y
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: query[A] relay.redacted.com from 172.22.0.1
dnsmasq[1]: cached relay.redacted.com is 141.147.x.y
dnsmasq[1]: query[A] www.google.com from 172.22.0.1
dnsmasq[1]: forwarded www.google.com to 8.8.8.8
dnsmasq[1]: reply www.google.com is 142.250.206.228
dnsmasq[1]: query[A] www.google.com from 172.22.0.1
dnsmasq[1]: cached www.google.com is 142.250.206.228
dnsmasq[1]: query[A] www.google.com from 172.22.0.1
dnsmasq[1]: cached www.google.com is 142.250.206.228
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1
dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1
dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8
dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1
dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8
dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1
dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8
dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1
dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8
dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1
dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8
dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1
dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8
dnsmasq[1]: config 192.168.43.10 is NXDOMAIN
dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1
dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8
^C
@.*** ~ $
My dig attempts show up in the dnsmasq log as these lines:

dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1
dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8
Nothing much to show on the VPN side - it's happy:

@.*** vpn $ docker logs ipsec-vpn

VPN credentials not set by user. Generating random PSK and password...

Setting DNS server to 10.2.1.8...

Note: Running in IKEv2-only mode via env file option.
IPsec/L2TP and IPsec/XAuth ("Cisco IPsec") modes are disabled.

Starting IPsec service...

Setting up IKEv2. This may take a few moments...

================================================

IKEv2 setup successful. Details for IKEv2 mode:

VPN server address: relay.redacted.com
VPN client name: vpnclient

Client configuration is available inside the
Docker container at:
/etc/ipsec.d/vpnclient.p12 (for Windows & Linux)
/etc/ipsec.d/vpnclient.sswan (for Android)
/etc/ipsec.d/vpnclient.mobileconfig (for iOS & macOS)

Next steps: Configure IKEv2 clients. See:
https://vpnsetup.net/clients2

================================================

xl2tpd[1]: Not looking for kernel SAref support.
xl2tpd[1]: Using l2tp kernel support.
xl2tpd[1]: xl2tpd version xl2tpd-1.3.17 started on 51a8334895ce PID:1
xl2tpd[1]: Written by Mark Spencer, Copyright (C) 1998, Adtran, Inc.
xl2tpd[1]: Forked by Scott Balmos and David Stipp, © 2001
xl2tpd[1]: Inherited by Jeff McAdams, © 2002
xl2tpd[1]: Forked again by Xelerance (www.xelerance.com) (C) 2006-2016
xl2tpd[1]: Listening on IP address 0.0.0.0, port 1701
Server (please complete the following information)

OS: Debian 11.5, running the cloudinit qcow2 image, but with the standard linux-image-arm64 kernel for ppp support
Hosting provider (if applicable): Oracle Cloud, free tier
Client (please complete the following information)

Device: MacBook Air M1
OS: macOS 12.6 "Monterey"
VPN mode: IKEv2
Client (please complete the following information)

Device: iPhone X
OS: iOS 16.0
VPN mode: IKEv2
Additional context

Connecting to the DNS over the internet (restricted to allow my IP only; no open resolvers):

@.*** ~ % dig @relay.redacted.com relay.oracle.redacted.com

; <<>> DiG 9.10.6 <<>> @relay.redacted.com relay.oracle.redacted.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 31390
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;relay.oracle.redacted.com. IN A

;; ANSWER SECTION:
relay.oracle.redacted.com. 2 IN A 10.2.1.8

;; Query time: 154 msec
;; SERVER: 141.147.x.y#53(141.147.x.y)
;; WHEN: Mon Sep 19 08:29:49 CDT 2022
;; MSG SIZE rcvd: 72

Connecting to the DNS from the server itself (i.e. connecting to localhost):

@.*** ~ % ssh relay.redacted.com
@.*** ~ $ dig @127.0.0.1 relay.oracle.redacted.com

; <<>> DiG 9.16.27-Debian <<>> @127.0.0.1 relay.oracle.redacted.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49168
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;relay.oracle.redacted.com. IN A

;; ANSWER SECTION:
relay.oracle.redacted.com. 2 IN A 10.2.1.8

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sun Sep 18 14:53:19 CDT 2022
;; MSG SIZE rcvd: 72

Connecting to the DNS once connected to the VPN (both macOS DNS resolution and dig):
connect vpn

@.*** ~ % dig @10.2.1.8 relay.oracle.redacted.com

; <<>> DiG 9.10.6 <<>> @10.2.1.8 relay.oracle.redacted.com
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached
@.*** ~ % ping relay.oracle.redacted.com
ping: cannot resolve relay.oracle.redacted.com: Unknown host
@.*** ~ %

disconnect vpn

Nothing. Weird!


Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.

<!-- gh-comment-id:1254153130 --> @letoams commented on GitHub (Sep 21, 2022): Two common issues could be affecting you One is that the ip range you assign to the vpn clients isn’t fully routed to it from the dns server and the reply packet goes out the wrong way via a default gateway bypassing the ipsec server. Two is that somewhere this reply packet is getting NATed and then no longer matches the ipsec policies and is getting dropped. Sent using a virtual keyboard on a phone > On Sep 20, 2022, at 19:10, diamondsw ***@***.***> wrote: > >  > This is going to be a bit of a weird one; thank you for looking! Despite the amount of DNS detail, I swear it's not a DNS issue. The goal here is to provide an internal DNS server for private hostnames on the remote network. > > While this is my first time using this container, I've configured (painfully) several IPSEC/L2TP VPNs before, with mobileconfig, site-to-site tunnels, and private DNS. First time I've ever seen something quite like this, and I'm completely stumped. > > Checklist > > I read the README > I read the Important notes > I followed instructions to configure VPN clients > I checked Troubleshooting, IKEv2 troubleshooting and VPN status > I searched existing Issues > This bug is about the VPN setup scripts, and not IPsec VPN itself > I don't know on that last one! If anything, it seems like a very weird Docker network thing. > > Describe the issue > After establishing a VPN connection to the ipsec-vpn-server container, return traffic from a dnsmasq container on the same host does not arrive. > > More detail below, but in short: > > The VPN connects without issue (by the way, BRAVO at making IPSEC so easy!) > Ping, SSH, and other bidirectional traffic to the host server or internet traverse the VPN without issue. > If no DNS server is specified for the VPN (i.e. using built-in 8.8.8.8), then DNS resolution works > If a public DNS server is specified (e.g. 1.1.1.1), then DNS resolution works > If a local dnsmasq container is specified using the host private IP address, then dnsmasq sees the requests and responds - but the client never sees these responses > Meanwhile, accessing the same DNS container over the internet works, and using dig locally on the server works > That's the weirdest thing - the DNS container works, and even sees the requests over the VPN and responds - but the client never gets the traffic back. Other protocols are working fine. The only difference I can see is this is traffic between containers, whereas ping, ssh, etc are all either to the host or headed outside the network. > > Network details: > > Server is on Oracle Cloud and has both a public IP (141.147.x.y - redacted) and private IP (10.2.1.8), and the security list allows USP 500/4500 from all IPs. For the purposes of troubleshooting, it also allows port 53 from my home IP. > The public IP is reachable at 'relay.redacted.com', and the private IP at 'relay.oracle.redacted.com'. Apologies if this is confusing! > To Reproduce > Steps to reproduce the behavior: > > Setup the VPN server with the following docker-compose.yml: > version: "3" > services: > ipsec-vpn: > image: hwdsl2/ipsec-vpn-server:latest > container_name: ipsec-vpn > restart: unless-stopped > environment: > VPN_IKEV2_ONLY: 'yes' > VPN_DNS_NAME: 'relay.redacted.com' > VPN_DNS_SRV1: '10.2.1.8' > ports: > - '500:500/udp' > - '4500:4500/udp' > volumes: > - /lib/modules:/lib/modules:ro > - ./data:/etc/ipsec.d > cap_add: > - NET_ADMIN > devices: > - "/dev/ppp:/dev/ppp" > sysctls: > - net.ipv4.ip_forward=1 > - net.ipv4.conf.all.accept_redirects=0 > - net.ipv4.conf.all.send_redirects=0 > - net.ipv4.conf.all.rp_filter=0 > - net.ipv4.conf.default.accept_redirects=0 > - net.ipv4.conf.default.send_redirects=0 > - net.ipv4.conf.default.rp_filter=0 > - net.ipv4.conf.eth0.send_redirects=0 > - net.ipv4.conf.eth0.rp_filter=0 > Setup the DNS server with the following docker-compose.yml and config files: > Dockerfile > > FROM alpine:edge > RUN apk --no-cache add dnsmasq > EXPOSE 53 53/udp > ENTRYPOINT ["dnsmasq", "-k", "--log-facility=-"] > docker-compose.yml > > version: "3" > services: > dnsmasq: > build: . > image: diamondsw/dnsmasq > container_name: dnsmasq > restart: unless-stopped > environment: > TZ: 'America/Chicago' > ports: > - "53:53/udp" > volumes: > - ./dnsmasq.conf:/etc/dnsmasq.d/dnsmasq.conf > - ./hosts:/etc/dnsmasq.d/hosts > cap_add: > - NET_ADMIN > dnsmasq.conf > > # Listen on the Docker eth0 > interface=eth0 > > # Standard options > no-resolv > cache-size=10000 > local-ttl=2 > > # Internal DNS > domain-needed > bogus-priv > domain=oracle.redacted.com > expand-hosts > addn-hosts=/etc/dnsmasq.d/hosts > > # Forward all public queries to the internet > server=1.1.1.1 > server=8.8.8.8 > > # Debugging > log-queries > Note here that I have specified an interface to listen to, which turns off the dnsmasq '--local-service' option. > > hosts > > 10.2.1.8 relay.oracle.redacted.com > (Note, I have changed the domain names to remove private information.) > > Expected behavior > DNS responses should arrive back to the requestor (i.e. the VPN client machine). > > Logs > Check logs and VPN status, and add error logs to help explain the problem, if applicable. > > DNSmasq log when VPN connected (with a LOT of Apple services connection attempts cut out): > > ***@***.*** ~ % ssh 10.2.1.8 > ***@***.*** ~ $ docker logs --follow dnsmasq > dnsmasq[1]: started, version 2.86 cachesize 10000 > dnsmasq[1]: compile time options: IPv6 GNU-getopt no-DBus no-UBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth no-cryptohash no-DNSSEC loop-detect inotify dumpfile > dnsmasq[1]: using nameserver 1.1.1.1#53 > dnsmasq[1]: using nameserver 8.8.8.8#53 > dnsmasq[1]: read /etc/hosts - 7 addresses > dnsmasq[1]: read /etc/dnsmasq.d/hosts - 1 addresses > dnsmasq[1]: config 192.168.43.10 is NXDOMAIN > dnsmasq[1]: config 192.168.43.10 is NXDOMAIN > dnsmasq[1]: config 192.168.43.10 is NXDOMAIN > dnsmasq[1]: query[A] relay2.oracle.redacted.com from 172.22.0.1 > dnsmasq[1]: forwarded relay2.oracle.redacted.com to 8.8.8.8 > dnsmasq[1]: reply relay2.oracle.redacted.com is NODATA-IPv4 > dnsmasq[1]: config 192.168.43.10 is NXDOMAIN > dnsmasq[1]: query[A] relay.redacted.com from 172.22.0.1 > dnsmasq[1]: forwarded relay.redacted.com to 8.8.8.8 > dnsmasq[1]: reply relay.redacted.com is 141.147.x.y > dnsmasq[1]: query[A] relay.redacted.com from 172.22.0.1 > dnsmasq[1]: cached relay.redacted.com is 141.147.x.y > dnsmasq[1]: query[A] relay.redacted.com from 172.22.0.1 > dnsmasq[1]: cached relay.redacted.com is 141.147.x.y > dnsmasq[1]: config 192.168.43.10 is NXDOMAIN > dnsmasq[1]: query[A] relay.redacted.com from 172.22.0.1 > dnsmasq[1]: cached relay.redacted.com is 141.147.x.y > dnsmasq[1]: query[A] www.google.com from 172.22.0.1 > dnsmasq[1]: forwarded www.google.com to 8.8.8.8 > dnsmasq[1]: reply www.google.com is 142.250.206.228 > dnsmasq[1]: query[A] www.google.com from 172.22.0.1 > dnsmasq[1]: cached www.google.com is 142.250.206.228 > dnsmasq[1]: query[A] www.google.com from 172.22.0.1 > dnsmasq[1]: cached www.google.com is 142.250.206.228 > dnsmasq[1]: config 192.168.43.10 is NXDOMAIN > dnsmasq[1]: config 192.168.43.10 is NXDOMAIN > dnsmasq[1]: config 192.168.43.10 is NXDOMAIN > dnsmasq[1]: config 192.168.43.10 is NXDOMAIN > dnsmasq[1]: config 192.168.43.10 is NXDOMAIN > dnsmasq[1]: config 192.168.43.10 is NXDOMAIN > dnsmasq[1]: config 192.168.43.10 is NXDOMAIN > dnsmasq[1]: config 192.168.43.10 is NXDOMAIN > dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1 > dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8 > dnsmasq[1]: config 192.168.43.10 is NXDOMAIN > dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1 > dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8 > dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1 > dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8 > dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1 > dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8 > dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1 > dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8 > dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1 > dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8 > dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1 > dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8 > dnsmasq[1]: config 192.168.43.10 is NXDOMAIN > dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1 > dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8 > ^C > ***@***.*** ~ $ > My dig attempts show up in the dnsmasq log as these lines: > > dnsmasq[1]: query[A] relay.oracle.redacted.com from 172.22.0.1 > dnsmasq[1]: /etc/dnsmasq.d/hosts relay.oracle.redacted.com is 10.2.1.8 > Nothing much to show on the VPN side - it's happy: > > ***@***.*** vpn $ docker logs ipsec-vpn > > VPN credentials not set by user. Generating random PSK and password... > > Setting DNS server to 10.2.1.8... > > Note: Running in IKEv2-only mode via env file option. > IPsec/L2TP and IPsec/XAuth ("Cisco IPsec") modes are disabled. > > Starting IPsec service... > > Setting up IKEv2. This may take a few moments... > > ================================================ > > IKEv2 setup successful. Details for IKEv2 mode: > > VPN server address: relay.redacted.com > VPN client name: vpnclient > > Client configuration is available inside the > Docker container at: > /etc/ipsec.d/vpnclient.p12 (for Windows & Linux) > /etc/ipsec.d/vpnclient.sswan (for Android) > /etc/ipsec.d/vpnclient.mobileconfig (for iOS & macOS) > > Next steps: Configure IKEv2 clients. See: > https://vpnsetup.net/clients2 > > ================================================ > > xl2tpd[1]: Not looking for kernel SAref support. > xl2tpd[1]: Using l2tp kernel support. > xl2tpd[1]: xl2tpd version xl2tpd-1.3.17 started on 51a8334895ce PID:1 > xl2tpd[1]: Written by Mark Spencer, Copyright (C) 1998, Adtran, Inc. > xl2tpd[1]: Forked by Scott Balmos and David Stipp, © 2001 > xl2tpd[1]: Inherited by Jeff McAdams, © 2002 > xl2tpd[1]: Forked again by Xelerance (www.xelerance.com) (C) 2006-2016 > xl2tpd[1]: Listening on IP address 0.0.0.0, port 1701 > Server (please complete the following information) > > OS: Debian 11.5, running the cloudinit qcow2 image, but with the standard linux-image-arm64 kernel for ppp support > Hosting provider (if applicable): Oracle Cloud, free tier > Client (please complete the following information) > > Device: MacBook Air M1 > OS: macOS 12.6 "Monterey" > VPN mode: IKEv2 > Client (please complete the following information) > > Device: iPhone X > OS: iOS 16.0 > VPN mode: IKEv2 > Additional context > > Connecting to the DNS over the internet (restricted to allow my IP only; no open resolvers): > > ***@***.*** ~ % dig @relay.redacted.com relay.oracle.redacted.com > > ; <<>> DiG 9.10.6 <<>> @relay.redacted.com relay.oracle.redacted.com > ; (1 server found) > ;; global options: +cmd > ;; Got answer: > ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 31390 > ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 > > ;; OPT PSEUDOSECTION: > ; EDNS: version: 0, flags:; udp: 4096 > ;; QUESTION SECTION: > ;relay.oracle.redacted.com. IN A > > ;; ANSWER SECTION: > relay.oracle.redacted.com. 2 IN A 10.2.1.8 > > ;; Query time: 154 msec > ;; SERVER: 141.147.x.y#53(141.147.x.y) > ;; WHEN: Mon Sep 19 08:29:49 CDT 2022 > ;; MSG SIZE rcvd: 72 > > Connecting to the DNS from the server itself (i.e. connecting to localhost): > > ***@***.*** ~ % ssh relay.redacted.com > ***@***.*** ~ $ dig @127.0.0.1 relay.oracle.redacted.com > > ; <<>> DiG 9.16.27-Debian <<>> @127.0.0.1 relay.oracle.redacted.com > ; (1 server found) > ;; global options: +cmd > ;; Got answer: > ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49168 > ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 > > ;; OPT PSEUDOSECTION: > ; EDNS: version: 0, flags:; udp: 4096 > ;; QUESTION SECTION: > ;relay.oracle.redacted.com. IN A > > ;; ANSWER SECTION: > relay.oracle.redacted.com. 2 IN A 10.2.1.8 > > ;; Query time: 0 msec > ;; SERVER: 127.0.0.1#53(127.0.0.1) > ;; WHEN: Sun Sep 18 14:53:19 CDT 2022 > ;; MSG SIZE rcvd: 72 > > Connecting to the DNS once connected to the VPN (both macOS DNS resolution and dig): > connect vpn > > ***@***.*** ~ % dig @10.2.1.8 relay.oracle.redacted.com > > ; <<>> DiG 9.10.6 <<>> @10.2.1.8 relay.oracle.redacted.com > ; (1 server found) > ;; global options: +cmd > ;; connection timed out; no servers could be reached > ***@***.*** ~ % ping relay.oracle.redacted.com > ping: cannot resolve relay.oracle.redacted.com: Unknown host > ***@***.*** ~ % > > disconnect vpn > > Nothing. Weird! > > — > Reply to this email directly, view it on GitHub, or unsubscribe. > You are receiving this because you are subscribed to this thread.
Author
Owner

@diamondsw commented on GitHub (Sep 21, 2022):

It looks like the reply is reaching the IPSEC container, but being dropped since it's appearing from a docker bridge subnet rather than the host IP as expected. The odd thing is 172.24.0.1 is the gateway IP of the IPSEC container, not the DNS container.

Although I just realized this may not be a valid test since the source IP of the DNS request is the VPN server, not a VPN client. Man, too many routing layers are involved here.

(Oh, and my apologies for not seeing the docker-specific repo!)

me@relay ~ $ docker exec -it ipsec-vpn /bin/sh
/opt/src # dig @10.2.1.8 relay.oracle.redacted.com
;; reply from unexpected source: 172.24.0.1#53, expected 10.2.1.8#53

;; reply from unexpected source: 172.24.0.1#53, expected 10.2.1.8#53
^C
me@relay ~ $ docker network inspect vpn_default
[
    {
        "Name": "vpn_default",
        "Id": "ad227735e402c98f5b5296f0ee3ad85915fc927944fc0307ab3d49f86d294ad2",
        "Created": "2022-09-19T10:16:33.91033209-05:00",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "172.24.0.0/16",
                    "Gateway": "172.24.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": true,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "77ed36d3db0cb1ffe45749e8142a83547978862c897c8dc8732da0c1c8e2b720": {
                "Name": "ipsec-vpn",
                "EndpointID": "89bbf345d979f75cb44e704be63e4777bc44426128bcbd85641c8b2f6327c773",
                "MacAddress": "02:42:ac:18:00:02",
                "IPv4Address": "172.24.0.2/16",
                "IPv6Address": ""
            }
        },
        "Options": {},
        "Labels": {
            "com.docker.compose.network": "default",
            "com.docker.compose.project": "vpn",
            "com.docker.compose.version": "1.25.0"
        }
    }
]

<!-- gh-comment-id:1254164329 --> @diamondsw commented on GitHub (Sep 21, 2022): It looks like the reply is reaching the IPSEC container, but being dropped since it's appearing from a docker bridge subnet rather than the host IP as expected. The odd thing is 172.24.0.1 is the gateway IP of the IPSEC container, not the DNS container. Although I just realized this may not be a valid test since the source IP of the DNS request is the VPN server, not a VPN client. Man, too many routing layers are involved here. (Oh, and my apologies for not seeing the docker-specific repo!) ``` me@relay ~ $ docker exec -it ipsec-vpn /bin/sh /opt/src # dig @10.2.1.8 relay.oracle.redacted.com ;; reply from unexpected source: 172.24.0.1#53, expected 10.2.1.8#53 ;; reply from unexpected source: 172.24.0.1#53, expected 10.2.1.8#53 ^C me@relay ~ $ docker network inspect vpn_default [ { "Name": "vpn_default", "Id": "ad227735e402c98f5b5296f0ee3ad85915fc927944fc0307ab3d49f86d294ad2", "Created": "2022-09-19T10:16:33.91033209-05:00", "Scope": "local", "Driver": "bridge", "EnableIPv6": false, "IPAM": { "Driver": "default", "Options": null, "Config": [ { "Subnet": "172.24.0.0/16", "Gateway": "172.24.0.1" } ] }, "Internal": false, "Attachable": true, "Ingress": false, "ConfigFrom": { "Network": "" }, "ConfigOnly": false, "Containers": { "77ed36d3db0cb1ffe45749e8142a83547978862c897c8dc8732da0c1c8e2b720": { "Name": "ipsec-vpn", "EndpointID": "89bbf345d979f75cb44e704be63e4777bc44426128bcbd85641c8b2f6327c773", "MacAddress": "02:42:ac:18:00:02", "IPv4Address": "172.24.0.2/16", "IPv6Address": "" } }, "Options": {}, "Labels": { "com.docker.compose.network": "default", "com.docker.compose.project": "vpn", "com.docker.compose.version": "1.25.0" } } ] ```
Author
Owner

@diamondsw commented on GitHub (Sep 22, 2022):

Still honestly not sure if this is a bug, or just life with Docker container/host routing - I don't know if there's anything you can do to change how this is being networked between containers. So I'll close the issue, but here's my solution in case anyone else stumbles across this.

Given that this appears to be a complex routing issue between Docker containers, host networking, etc, I took the path of least resistance and removed the host from the equation. I created an external network:

subspace\docker-compose.yml

version: "3"
services:
  dax:
    image: alpine:latest
    container_name: subspace-relay
    networks:
      relay:

networks:
  relay:
    ipam:
      config:
        - subnet: 172.20.0.0/16

And then placed the DNS container on it:

    networks:
      subspace_relay:
        ipv4_address: 172.20.0.2

networks:
  subspace_relay:
    external: true

And the VPN container as well:

    networks:
      subspace_relay:
        ipv4_address: 172.20.0.3

networks:
  subspace_relay:
    external: true

And finally, set the VPN configuration to use the new IP address:

    environment:
      VPN_IKEV2_ONLY: 'yes'
      VPN_DNS_NAME: 'relay.redacted.com'
      VPN_DNS_SRV1: '172.20.0.2'

Regenerated the VPN container, reloaded the VPN profiles and finally, all is working.

<!-- gh-comment-id:1254390483 --> @diamondsw commented on GitHub (Sep 22, 2022): Still honestly not sure if this is a bug, or just life with Docker container/host routing - I don't know if there's anything you can do to change how this is being networked between containers. So I'll close the issue, but here's my solution in case anyone else stumbles across this. Given that this appears to be a complex routing issue between Docker containers, host networking, etc, I took the path of least resistance and removed the host from the equation. I created an external network: subspace\docker-compose.yml ``` version: "3" services: dax: image: alpine:latest container_name: subspace-relay networks: relay: networks: relay: ipam: config: - subnet: 172.20.0.0/16 ``` And then placed the DNS container on it: ``` networks: subspace_relay: ipv4_address: 172.20.0.2 networks: subspace_relay: external: true ``` And the VPN container as well: ``` networks: subspace_relay: ipv4_address: 172.20.0.3 networks: subspace_relay: external: true ``` And finally, set the VPN configuration to use the new IP address: ``` environment: VPN_IKEV2_ONLY: 'yes' VPN_DNS_NAME: 'relay.redacted.com' VPN_DNS_SRV1: '172.20.0.2' ``` Regenerated the VPN container, reloaded the VPN profiles and _finally_, all is working.
Author
Owner

@hwdsl2 commented on GitHub (Sep 22, 2022):

@diamondsw Thank you for sharing your solution with us.

<!-- gh-comment-id:1254519230 --> @hwdsl2 commented on GitHub (Sep 22, 2022): @diamondsw Thank you for sharing your solution with us.
Author
Owner

@atdimitrov commented on GitHub (Feb 11, 2023):

@diamondsw Thank you for sharing your research and your solution on this.

I had the exact same setup as you did - DNS and VPN servers running on the same host machine in the default bridge network and of course I had the exact same problem. However, I don't think the problem is with the routing - the response seems to get back to the machine that initiates the dig command because of this log: reply from unexpected source: 172.24.0.1#53, expected 10.2.1.8#53. It looks like it is a security feature of the DNS resolvers and the NAT is tripping it.

So the main problem seems to be having the DNS server in a bridge network. If other containers in the same bridge network try to use the DNS with the host IP they would fail the same way. The way I solved it is to put the DNS server on the host network avoiding going through NAT. Another solution that should work would be to use the bridge network IP instead of the host IP. Essentially, the same solution you found without the additional network.

I'm by no means a Docker networking expert but I don't think there's anything that has to be done in the VPN image, nor it has to do anything with it.

<!-- gh-comment-id:1426705590 --> @atdimitrov commented on GitHub (Feb 11, 2023): @diamondsw Thank you for sharing your research and your solution on this. I had the exact same setup as you did - DNS and VPN servers running on the same host machine in the default bridge network and of course I had the exact same problem. However, I don't think the problem is with the routing - the response seems to get back to the machine that initiates the `dig` command because of this log: `reply from unexpected source: 172.24.0.1#53, expected 10.2.1.8#53`. It looks like it is a security feature of the DNS resolvers and the NAT is tripping it. So the main problem seems to be having the DNS server in a bridge network. If other containers in the same bridge network try to use the DNS with the host IP they would fail the same way. The way I solved it is to put the DNS server on the host network avoiding going through NAT. Another solution that should work would be to use the bridge network IP instead of the host IP. Essentially, the same solution you found without the additional network. I'm by no means a Docker networking expert but I don't think there's anything that has to be done in the VPN image, nor it has to do anything with it.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/docker-ipsec-vpn-server#286
No description provided.