starred/dns-proxy-server-mageddo
1
0
Fork 0
mirror of https://github.com/mageddo/dns-proxy-server.git synced 2026-04-25 09:25:56 +03:00
Code Issues 10 Projects Releases 10 Packages Wiki Activity

[GH-ISSUE #408] DPS Network autoconnect stops working #142

New issue
Closed
opened 2026-02-26 04:34:10 +03:00 by kerem · 8 comments
kerem commented 2026-02-26 04:34:10 +03:00
Owner
Copy link

Originally created by @Sajito on GitHub (Apr 4, 2023).
Original GitHub issue: https://github.com/mageddo/dns-proxy-server/issues/408

Originally assigned to: @mageddo on GitHub.

What is Happening / What is expected

I'm using dps in a docker container with the dps network and auto connect features enabled. After the startup everything works fine, new containers are detected and added to the network. Also the logs are written, that containers were added to the network.
But after some time, new containers are not added to the network anymore. Nothing appears in the logs. There are no errors or messages regarding docker events.
The dns itself is working fine and the logging itself seems to work also. It's only that new containers are not registered in any way. Maybe the event listener closes silently in the background or something?
I don't know how to provide information on how to reproduce this, as it seems to happen randomly. Sometimes it happens after 10 minutes, sometimes it works fine for an hour.

Specs

  • OS: EndeavourOS
  • Docker Version: 
    Client:
 Version:           23.0.2
 API version:       1.42
 Go version:        go1.20.2
 Git commit:        569dd73db1
 Built:             Sun Apr  2 11:30:09 2023
 OS/Arch:           linux/amd64
 Context:           default

Server:
 Engine:
  Version:          23.0.2
  API version:      1.42 (minimum version 1.12)
  Go version:       go1.20.2
  Git commit:       219f21bf07
  Built:            Sun Apr  2 11:30:09 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.7.0
  GitCommit:        1fbd70374134b891f97ce19c70b6e50c7b9f4e0d.m
 runc:
  Version:          1.1.5
  GitCommit:        
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
  • DPS Version: 3.15.9-snapshot
Originally created by @Sajito on GitHub (Apr 4, 2023). Original GitHub issue: https://github.com/mageddo/dns-proxy-server/issues/408 Originally assigned to: @mageddo on GitHub. ### What is Happening / What is expected I'm using dps in a docker container with the dps network and auto connect features enabled. After the startup everything works fine, new containers are detected and added to the network. Also the logs are written, that containers were added to the network. But after some time, new containers are not added to the network anymore. Nothing appears in the logs. There are no errors or messages regarding docker events. The dns itself is working fine and the logging itself seems to work also. It's only that new containers are not registered in any way. Maybe the event listener closes silently in the background or something? I don't know how to provide information on how to reproduce this, as it seems to happen randomly. Sometimes it happens after 10 minutes, sometimes it works fine for an hour. ### Specs * OS: EndeavourOS * Docker Version: ``` Client: Version: 23.0.2 API version: 1.42 Go version: go1.20.2 Git commit: 569dd73db1 Built: Sun Apr 2 11:30:09 2023 OS/Arch: linux/amd64 Context: default Server: Engine: Version: 23.0.2 API version: 1.42 (minimum version 1.12) Go version: go1.20.2 Git commit: 219f21bf07 Built: Sun Apr 2 11:30:09 2023 OS/Arch: linux/amd64 Experimental: false containerd: Version: v1.7.0 GitCommit: 1fbd70374134b891f97ce19c70b6e50c7b9f4e0d.m runc: Version: 1.1.5 GitCommit: docker-init: Version: 0.19.0 GitCommit: de40ad0 ``` * DPS Version: `3.15.9-snapshot`
kerem 2026-02-26 04:34:10 +03:00
  • closed this issue
  • added the
    bug
         label
kerem commented 2026-02-26 04:34:11 +03:00
Author
Owner
Copy link

@mageddo commented on GitHub (Apr 5, 2023):

Please provide the command you are using to run DPS.

Also enable the DEBUG log and be aware of the log "status=event, id={}, action={}, type={}, status={}, event={}", it's supposed to log every time a container is created or destroyed, status=connected, networkNameOrId is also supposed to be logged in sequence or some leak could be happening between the event listening and the action taken.

If you can't see that log after the "new containers are not added to the network anymore." then lack of event trigger can be the root cause.

I'll make some tests at the event listener

<!-- gh-comment-id:1496910817 --> @mageddo commented on GitHub (Apr 5, 2023): Please provide the command you are using to run DPS. Also enable the DEBUG log and be aware of the log `"status=event, id={}, action={}, type={}, status={}, event={}"`, it's supposed to log every time a container is created or destroyed, `status=connected, networkNameOrId` is also supposed to be logged in sequence or some leak could be happening between the event listening and the action taken. If you can't see that log after the "new containers are not added to the network anymore." then lack of event trigger can be the root cause. I'll make some tests at the event listener
kerem commented 2026-02-26 04:34:11 +03:00
Author
Owner
Copy link

@Sajito commented on GitHub (Apr 5, 2023):

I'm using this docker-compose snippet, to run DPS.

version: '3.6'

services:
  dns:
    image: defreitas/dns-proxy-server:3.15.9-snapshot
    restart: unless-stopped
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      MG_DPS_NETWORK: true
      MG_DPS_NETWORK_AUTO_CONNECT: true

This is the config I'm using currently.

{
  "version": 2,
  "activeEnv": "",
  "webServerPort": null,
  "dnsServerPort": null,
  "defaultDns": false,
  "logLevel": null,
  "logFile": null,
  "registerContainerNames": null,
  "hostMachineHostname": null,
  "domain": null,
  "dpsNetwork": null,
  "dpsNetworkAutoConnect": null,
  "remoteDnsServers": [],
  "envs": [
    {
      "name": "",
      "hostnames": []
    }
  ]
}

I've enabled the DEBUG log and initially the logs are printed as described, but when the containers are not added anymore, the logs don't contain the "status=event" log anymore. Since the full log exceeded 10MB, I have cleared the log file and reproduced the issue. Here is the log of that period, I can't find anything related to docker events in that snippet.
log.log

<!-- gh-comment-id:1497216766 --> @Sajito commented on GitHub (Apr 5, 2023): I'm using this docker-compose snippet, to run DPS. ``` version: '3.6' services: dns: image: defreitas/dns-proxy-server:3.15.9-snapshot restart: unless-stopped volumes: - /var/run/docker.sock:/var/run/docker.sock environment: MG_DPS_NETWORK: true MG_DPS_NETWORK_AUTO_CONNECT: true ``` This is the config I'm using currently. ``` { "version": 2, "activeEnv": "", "webServerPort": null, "dnsServerPort": null, "defaultDns": false, "logLevel": null, "logFile": null, "registerContainerNames": null, "hostMachineHostname": null, "domain": null, "dpsNetwork": null, "dpsNetworkAutoConnect": null, "remoteDnsServers": [], "envs": [ { "name": "", "hostnames": [] } ] } ``` I've enabled the DEBUG log and initially the logs are printed as described, but when the containers are not added anymore, the logs don't contain the "status=event" log anymore. Since the full log exceeded 10MB, I have cleared the log file and reproduced the issue. Here is the log of that period, I can't find anything related to docker events in that snippet. [log.log](https://github.com/mageddo/dns-proxy-server/files/11157455/log.log)
kerem commented 2026-02-26 04:34:11 +03:00
Author
Owner
Copy link

@mageddo commented on GitHub (Apr 7, 2023):

I'm trying to reproduce the issue with the docker-compose.yml provided and the following docker stack.

This command will start a container, keep it alive for 3 seconds then it will die and start another one, it would be great if you check if it will also cause the issue for you.

$ while [ true ] ; do docker run --rm alpine sleep 3 ; done

My docker version is different from yours, if I couldn't get the issue then it can be the cause.

Client:
 Version:           20.10.14
 API version:       1.41
 Go version:        go1.16.15
 Git commit:        a224086
 Built:             Thu Mar 24 01:45:09 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.14
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.15
  Git commit:       87a90dc
  Built:            Thu Mar 24 01:49:54 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.5.11
  GitCommit:        3df54a852345ae127d1fa3092b95168e4a88e2f8
 runc:
  Version:          1.0.3
  GitCommit:        v1.0.3-0-gf46b6ba2
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

For 5 minutes, no issue:

408-dns-1  | 16:50:14.978 [stream-69888102] DEB com.mageddo.dnsproxyserver.docker.EventListener   l=60   m=onNext                          status=event, id=d0294f04634718bdf2718d26c39c4b0e2663f35038f99c87655581bb52967905, action=start, type=CONTAINER, status=start, event=Event(status=start, id=d0294f04634718bdf2718d26c39c4b0e2663f35038f99c87655581bb52967905, from=alpine, node=null, type=CONTAINER, action=start, actor=EventActor(id=d0294f04634718bdf2718d26c39c4b0e2663f35038f99c87655581bb52967905, attributes={image=alpine, name=busy_mirzakhani}), time=1680886214, timeNano=1680886214978251780)
408-dns-1  | 16:50:15.270 [stream-69888102] INF c.m.dnsproxyserver.docker.DockerNetworkDAODefault l=92   m=connect                         status=connected, networkNameOrId=dps, containerId=d0294f04634718bdf2718d26c39c4b0e2663f35038f99c87655581bb52967905
<!-- gh-comment-id:1500456762 --> @mageddo commented on GitHub (Apr 7, 2023): I'm trying to reproduce the issue with the docker-compose.yml provided and the following docker stack. This command will start a container, keep it alive for 3 seconds then it will die and start another one, it would be great if you check if it will also cause the issue for you. ```bash $ while [ true ] ; do docker run --rm alpine sleep 3 ; done ``` My docker version is different from yours, if I couldn't get the issue then it can be the cause. ``` Client: Version: 20.10.14 API version: 1.41 Go version: go1.16.15 Git commit: a224086 Built: Thu Mar 24 01:45:09 2022 OS/Arch: linux/amd64 Context: default Experimental: true Server: Docker Engine - Community Engine: Version: 20.10.14 API version: 1.41 (minimum version 1.12) Go version: go1.16.15 Git commit: 87a90dc Built: Thu Mar 24 01:49:54 2022 OS/Arch: linux/amd64 Experimental: false containerd: Version: v1.5.11 GitCommit: 3df54a852345ae127d1fa3092b95168e4a88e2f8 runc: Version: 1.0.3 GitCommit: v1.0.3-0-gf46b6ba2 docker-init: Version: 0.19.0 GitCommit: de40ad0 ``` For 5 minutes, no issue: ``` 408-dns-1 | 16:50:14.978 [stream-69888102] DEB com.mageddo.dnsproxyserver.docker.EventListener l=60 m=onNext status=event, id=d0294f04634718bdf2718d26c39c4b0e2663f35038f99c87655581bb52967905, action=start, type=CONTAINER, status=start, event=Event(status=start, id=d0294f04634718bdf2718d26c39c4b0e2663f35038f99c87655581bb52967905, from=alpine, node=null, type=CONTAINER, action=start, actor=EventActor(id=d0294f04634718bdf2718d26c39c4b0e2663f35038f99c87655581bb52967905, attributes={image=alpine, name=busy_mirzakhani}), time=1680886214, timeNano=1680886214978251780) 408-dns-1 | 16:50:15.270 [stream-69888102] INF c.m.dnsproxyserver.docker.DockerNetworkDAODefault l=92 m=connect status=connected, networkNameOrId=dps, containerId=d0294f04634718bdf2718d26c39c4b0e2663f35038f99c87655581bb52967905 ```
kerem commented 2026-02-26 04:34:11 +03:00
Author
Owner
Copy link

@mageddo commented on GitHub (Apr 7, 2023):

Didn't get the issue 2 hours later.

408-dns-1  | 18:20:38.662 [stream-69888102] DEB com.mageddo.dnsproxyserver.docker.EventListener   l=60   m=onNext                          status=event, id=500a201ef3a1843d5b2366366d63bceef33b490cabb8fbe9ee3b0ff282bd6984, action=start, type=CONTAINER, status=start, event=Event(status=start, id=500a201ef3a1843d5b2366366d63bceef33b490cabb8fbe9ee3b0ff282bd6984, from=alpine, node=null, type=CONTAINER, action=start, actor=EventActor(id=500a201ef3a1843d5b2366366d63bceef33b490cabb8fbe9ee3b0ff282bd6984, attributes={image=alpine, name=exciting_morse}), time=1680891638, timeNano=1680891638662064363)
408-dns-1  | 18:20:38.935 [stream-69888102] INF c.m.dnsproxyserver.docker.DockerNetworkDAODefault l=92   m=connect                         status=connected, networkNameOrId=dps, containerId=500a201ef3a1843d5b2366366d63bceef33b490cabb8fbe9ee3b0ff282bd6984
<!-- gh-comment-id:1500526126 --> @mageddo commented on GitHub (Apr 7, 2023): Didn't get the issue 2 hours later. ``` 408-dns-1 | 18:20:38.662 [stream-69888102] DEB com.mageddo.dnsproxyserver.docker.EventListener l=60 m=onNext status=event, id=500a201ef3a1843d5b2366366d63bceef33b490cabb8fbe9ee3b0ff282bd6984, action=start, type=CONTAINER, status=start, event=Event(status=start, id=500a201ef3a1843d5b2366366d63bceef33b490cabb8fbe9ee3b0ff282bd6984, from=alpine, node=null, type=CONTAINER, action=start, actor=EventActor(id=500a201ef3a1843d5b2366366d63bceef33b490cabb8fbe9ee3b0ff282bd6984, attributes={image=alpine, name=exciting_morse}), time=1680891638, timeNano=1680891638662064363) 408-dns-1 | 18:20:38.935 [stream-69888102] INF c.m.dnsproxyserver.docker.DockerNetworkDAODefault l=92 m=connect status=connected, networkNameOrId=dps, containerId=500a201ef3a1843d5b2366366d63bceef33b490cabb8fbe9ee3b0ff282bd6984 ```
kerem commented 2026-02-26 04:34:11 +03:00
Author
Owner
Copy link

@mageddo commented on GitHub (Apr 7, 2023):

Got the issue, If a container already connected to DPS network stops and starts again, the event listener will hang and no new event will be able to be processed.

# Okay
$ docker run --name cobaia alpine sleep 10 
$ docker start cobaia
# further events won't be processed
$ docker run alpine sleep 10
<!-- gh-comment-id:1500533201 --> @mageddo commented on GitHub (Apr 7, 2023): Got the issue, If a container already connected to DPS network stops and starts again, the event listener will hang and no new event will be able to be processed. ```bash # Okay $ docker run --name cobaia alpine sleep 10 $ docker start cobaia # further events won't be processed $ docker run alpine sleep 10 ```
kerem commented 2026-02-26 04:34:11 +03:00
Author
Owner
Copy link

@mageddo commented on GitHub (Apr 7, 2023):

I'm releasing 3.15.12-snapshot with the fix right now, will be available in 10 minutes.

<!-- gh-comment-id:1500566997 --> @mageddo commented on GitHub (Apr 7, 2023): I'm releasing `3.15.12-snapshot` with the fix right now, will be available in 10 minutes.
kerem commented 2026-02-26 04:34:11 +03:00
Author
Owner
Copy link

@mageddo commented on GitHub (Apr 12, 2023):

I'll consider this issue as fixed, please reopen it if the issue persists.

<!-- gh-comment-id:1506083038 --> @mageddo commented on GitHub (Apr 12, 2023): I'll consider this issue as fixed, please reopen it if the issue persists.
kerem commented 2026-02-26 04:34:11 +03:00
Author
Owner
Copy link

@Sajito commented on GitHub (Apr 13, 2023):

Sorry, I forgot to reply here. Having no issues with this anymore. Thanks!

<!-- gh-comment-id:1506388524 --> @Sajito commented on GitHub (Apr 13, 2023): Sorry, I forgot to reply here. Having no issues with this anymore. Thanks!
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
gh-pages
develop
feat/138_2
feat/629
feat/594
5.9.0
5.9.0-snapshot
5.8.4
5.8.4-snapshot
5.8.3-snapshot
5.8.2-snapshot
5.8.1-snapshot
5.8.0-snapshot
5.7.0-snapshot
5.6.2-snapshot
5.6.1-snapshot
5.6.0-snapshot
5.5.1-snapshot
5.5.0-snapshot
5.4.0-snapshot
5.3.0-snapshot
5.2.0-snapshot
5.1.3-snapshot
5.1.2-snapshot
5.1.0-snapshot
5.0.5-snapshot
5.0.3-snapshot
5.0.2-snapshot
5.0.1-snapshot
5.0.0-snapshot
4.3.0-snapshot
4.2.0-snapshot
4.0.0
4.0.0-snapshot
3.32.7
3.32.7-snapshot
3.32.6
3.32.6-snapshot
3.32.5-snapshot
3.32.4
3.32.4-snapshot
3.32.3
3.32.3-snapshot
3.32.2-snapshot
3.32.1-snapshot
3.32.0-snapshot
3.31.1-snapshot
3.31.0-snapshot
3.30.5-snapshot
3.30.4
3.30.4-snapshot
3.30.3-snapshot
3.30.2-snapshot
3.30.1
3.30.1-snapshot
3.30.0-snapshot
3.29.0-snapshot
3.27.0
3.27.0-snapshot
3.26.0-snapshot
3.25.14
3.25.14-snapshot
3.25.13-snapshot
3.25.12-snapshot
3.25.11
3.25.11-snapshot
3.25.10
3.25.10-snapshot
3.25.9-snapshot
3.25.8-snapshot
3.25.7-snapshot
3.25.6-snapshot
3.25.4-snapshot
3.25.3-snapshot
3.25.2-snapshot
3.25.1-snapshot
3.25.0-snapshot
3.24.2
3.24.2-snapshot
3.24.1
3.24.1-snapshot
3.24.0-snapshot
3.23.0-snapshot
3.22.2-snapshot
3.22.3-snapshot
3.22.1-snapshot
3.21.3-snapshot
3.22.0-snapshot
3.21.2-snapshot
3.21.1-snapshot
3.20.0-snapshot
3.21.0-snapshot
3.19.7-snapshot
3.19.6-snapshot
3.19.5-snapshot
3.19.4-snapshot
3.19.3-snapshot
3.19.2-snapshot
3.19.1
3.19.1-snapshot
3.19.0
3.18.5-snapshot
3.18.4-snapshot
3.18.3-snapshot
3.18.2
3.18.2-snapshot
3.18.1-snapshot
3.18.0-snapshot
3.17.3-snapshot
3.17.2-snapshot
3.17.1-snapshot
3.17.0-snapshot
3.16.3-snapshot
3.16.2-snapshot
3.16.1-snapshot
3.16.0-snapshot
3.15.13-snapshot
3.15.12-snapshot
3.15.11-snapshot
3.15.10-snapshot
3.15.9-snapshot
3.15.8-snapshot
3.15.7-snapshot
3.15.6-snapshot
3.15.5-snapshot
3.15.4-snapshot
3.15.3-snapshot
3.15.2-snapshot
3.15.1-snapshot
3.15.0-snapshot
3.14.5
3.14.4
3.14.3
3.14.3-snapshot
3.14.2-snapshot
3.14.1-snapshot
3.14.0-snapshot
3.13.1
3.13.1-snapshot
3.13.0-snapshot
3.12.1
3.12.0
3.11.0
3.10.2-snapshot
3.10.3-snapshot
3.10.4-snapshot
3.10.5-snapshot
3.10.1-snapshot
3.9.2
3.9.1
3.9.0
3.8.1-last-version-with-quarkus
3.8.1
3.8.0
3.7.0
3.6.0
3.5.3
3.5.2
3.5.0
3.4.0-beta
3.3.0-beta
3.2.5-beta
3.2.4-beta
3.2.3-beta
3.2.2-beta
3.2.1-beta
3.2.0-beta
3.1.7-beta
3.1.6-beta
3.1.5-beta
3.1.4-beta
3.1.3-beta
3.1.2-beta
b-jre-8
3.1.1-beta
3.1.0-beta
3.0.4-beta
3.0.3-beta
3.0.2-beta
3.0.1-beta
v2-golang
2.19.5
2.19.4
2.19.3
2.19.2
2.19.1
2.19.0
2.18.7
2.18.6
2.18.5
2.18.4
2.18.3
2.18.2
2.18.1
2.18.0
2.17.4
2.17.3
2.17.2
2.17.1
2.17.0
2.16.0
2.15.0
2.14.6
2.14.5
2.14.4
2.14.2
2.14.1
2.14.0
2.13.2
2.13.1
2.13.0
2.12.0
2.11.0
2.10.4
2.10.3
2.10.2
2.10.1
2.10.0
2.9.1
2.9.0
2.8.0
2.7.0
2.6.1
2.6.0
2.5.4
2.5.3
2.5.2
2.5.1
2.5.0
2.4.1
2.4.0
2.3.3
2.2.3
2.2.2
2.2.1
2.2.0
2.1.8
2.1.7
2.1.6
2.1.5
2.1.2
2.1.1
2.1.0
2.0.21
2.0.20
2.0.19
2.0.18
2.0.17
2.0.16
2.0.9
2.0.5
2.0.4
v1-nodejs
1.8.1
1.8.0
1.7.7
1.7.6
1.7.5
1.7.4
1.7.3
1.7.2
1.7.1
1.7.0
1.6.9
1.6.8
1.6.7
1.6.6
1.6.5
1.6.4
1.6.3
1.6.2
1.6.1
1.6.0
1.5.0
1.4.0
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
peteris-version
1.1.4-peteris-version
1.2.0
1.1.3
1.1.2
1.1.1
1.1.0
1.0.1
1.0.0
Labels
Clear labels   
bug

   
confirmed

   
discussion

   
duplicate

   
enhancement

   
feature

   
feature-request

   
not-planned

   
pull-request

Mirrored from GitHub Pull Request

  
secondary-feature

   
stale

   
triage

   
waiting-feedback
No labels
bug
confirmed
discussion
duplicate
enhancement
feature
feature-request
not-planned
pull-request
secondary-feature
stale
triage
waiting-feedback
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/dns-proxy-server-mageddo#142
No description provided.