[GH-ISSUE #114] netbootxyx not running on Docker Swarm #150

Open
opened 2026-03-01 18:33:32 +03:00 by kerem · 4 comments
Owner

Originally created by @masterlog80 on GitHub (Dec 4, 2025).
Original GitHub issue: https://github.com/netbootxyz/docker-netbootxyz/issues/114

Hello,
I have upgraded my environment from Docker Compose to Docker Swarm.
As result, altrough most of the application I am using keep working "netbootxyx" doesn't.
Specifically the PC booting in PXE get the IP from the DHCP Server, but once connecting to TFTP faces a timeout or similar Errors (depend by the BIOS I think).
Taking a look at the container, here is the output:

[init] Setting up user nbxyz with PUID=1000 and PGID=1000
            _   _                 _                      
 _ __   ___| |_| |__   ___   ___ | |_  __  ___   _ ____  
| '_ \ / _ \ __| '_ \ / _ \ / _ \| __| \ \/ / | | |_  /  
| | | |  __/ |_| |_) | (_) | (_) | |_ _ >  <| |_| |/ /   
|_| |_|\___|\__|_.__/ \___/ \___/ \__(_)_/\_\__,  /___| 
                                             |___/       
If you enjoy netboot.xyz projects, please support us at:

https://opencollective.com/netbootxyz
https://github.com/sponsors/netbootxyz

[start] Starting supervisord (programs will run as nbxyz)
/usr/lib/python3.12/site-packages/supervisor/options.py:13: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
2025-12-04 05:10:47,323 INFO Set uid to user 0 succeeded
2025-12-04 05:10:47,337 INFO supervisord started with pid 1
2025-12-04 05:10:48,358 INFO spawned: 'nginx' with pid 23
2025-12-04 05:10:48,380 INFO spawned: 'dnsmasq' with pid 24
2025-12-04 05:10:48,397 INFO spawned: 'webapp' with pid 25
[dnsmasq] Starting TFTP server on port 69
[dnsmasq] TFTP root: /config/menus
[dnsmasq] TFTP security: enabled
[dnsmasq] Logging: enabled (dhcp and queries)
dnsmasq[24]: started, version 2.91 DNS disabled
dnsmasq[24]: compile time options: IPv6 GNU-getopt no-DBus no-UBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset no-nftset auth no-DNSSEC loop-detect inotify dumpfile
dnsmasq-tftp[24]: TFTP root is /config/menus secure mode 
2025-12-04 05:10:49,486 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2025-12-04 05:10:49,487 INFO success: dnsmasq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2025-12-04 05:10:49,488 INFO success: webapp entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
dnsmasq[24]: failed to send packet: Operation not permitted
dnsmasq[24]: failed to send packet: Operation not permitted
dnsmasq[24]: failed to send packet: Operation not permitted
dnsmasq[24]: failed to send packet: Operation not permitted
dnsmasq-tftp[24]: sent /config/menus/netboot.xyz.kpxe to 10.0.0.2
dnsmasq[24]: failed to send packet: Operation not permitted
dnsmasq-tftp[24]: sent /config/menus/netboot.xyz.kpxe to 10.0.0.2
....

That IP 10.0.0.2 is part of the Ingress Network which is where the Container is reached from outside, so it doesn't have any relation with the PC booting on PXE.

I am assuming this behaviour being somehow related to the Network settings on Container:

9b4b56c9f2ff:/# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
22982: eth0@if22983: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue state UP 
    link/ether 02:42:0a:00:00:e6 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.230/24 brd 10.0.0.255 scope global eth0
       valid_lft forever preferred_lft forever
22984: eth1@if22985: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP 
    link/ether 86:a5:53:cf:42:da brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.11/16 brd 172.18.255.255 scope global eth1
       valid_lft forever preferred_lft forever
22986: eth2@if22987: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue state UP 
    link/ether 02:42:0a:00:01:d5 brd ff:ff:ff:ff:ff:ff
    inet 10.0.1.213/24 brd 10.0.1.255 scope global eth2
       valid_lft forever preferred_lft forever
9b4b56c9f2ff:/# 

Those settings are completely different from the one on a Docker (not Swarm) environment:

80cfdd96e8dc:/# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN qlen 1000
    link/ipip 0.0.0.0 brd 0.0.0.0
3: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN qlen 1000
    link/gre 0.0.0.0 brd 0.0.0.0
4: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1476 qdisc noop state DOWN qlen 1000
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
5: erspan0@NONE: <BROADCAST,MULTICAST> mtu 1464 qdisc noop state DOWN qlen 1000
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
6: ip_vti0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN qlen 1000
    link/ipip 0.0.0.0 brd 0.0.0.0
7: ip6_vti0@NONE: <NOARP> mtu 1364 qdisc noop state DOWN qlen 1000
    link/tunnel6 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00 brd 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
8: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN qlen 1000
    link/sit 0.0.0.0 brd 0.0.0.0
9: ip6tnl0@NONE: <NOARP> mtu 1452 qdisc noop state DOWN qlen 1000
    link/tunnel6 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00 brd 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
10: ip6gre0@NONE: <NOARP> mtu 1448 qdisc noop state DOWN qlen 1000
    link/[823] 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00 brd 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
11: eth0@if6264: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP 
    link/ether 46:74:29:71:e8:3c brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.6/16 brd 172.18.255.255 scope global eth0
       valid_lft forever preferred_lft forever
80cfdd96e8dc:/# 

As I would like keep using Docker Swarm to guarantee HA, is there a workaround for this?

Regards,

Originally created by @masterlog80 on GitHub (Dec 4, 2025). Original GitHub issue: https://github.com/netbootxyz/docker-netbootxyz/issues/114 Hello, I have upgraded my environment from Docker Compose to Docker Swarm. As result, altrough most of the application I am using keep working "netbootxyx" doesn't. Specifically the PC booting in PXE get the IP from the DHCP Server, but once connecting to TFTP faces a [timeout](https://github.com/user-attachments/assets/c22a3a03-bc58-4b11-8d77-39041831f067) or similar Errors (depend by the BIOS I think). Taking a look at the container, here is the output: ``` [init] Setting up user nbxyz with PUID=1000 and PGID=1000 _ _ _ _ __ ___| |_| |__ ___ ___ | |_ __ ___ _ ____ | '_ \ / _ \ __| '_ \ / _ \ / _ \| __| \ \/ / | | |_ / | | | | __/ |_| |_) | (_) | (_) | |_ _ > <| |_| |/ / |_| |_|\___|\__|_.__/ \___/ \___/ \__(_)_/\_\__, /___| |___/ If you enjoy netboot.xyz projects, please support us at: https://opencollective.com/netbootxyz https://github.com/sponsors/netbootxyz [start] Starting supervisord (programs will run as nbxyz) /usr/lib/python3.12/site-packages/supervisor/options.py:13: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. import pkg_resources 2025-12-04 05:10:47,323 INFO Set uid to user 0 succeeded 2025-12-04 05:10:47,337 INFO supervisord started with pid 1 2025-12-04 05:10:48,358 INFO spawned: 'nginx' with pid 23 2025-12-04 05:10:48,380 INFO spawned: 'dnsmasq' with pid 24 2025-12-04 05:10:48,397 INFO spawned: 'webapp' with pid 25 [dnsmasq] Starting TFTP server on port 69 [dnsmasq] TFTP root: /config/menus [dnsmasq] TFTP security: enabled [dnsmasq] Logging: enabled (dhcp and queries) dnsmasq[24]: started, version 2.91 DNS disabled dnsmasq[24]: compile time options: IPv6 GNU-getopt no-DBus no-UBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset no-nftset auth no-DNSSEC loop-detect inotify dumpfile dnsmasq-tftp[24]: TFTP root is /config/menus secure mode 2025-12-04 05:10:49,486 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2025-12-04 05:10:49,487 INFO success: dnsmasq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2025-12-04 05:10:49,488 INFO success: webapp entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) dnsmasq[24]: failed to send packet: Operation not permitted dnsmasq[24]: failed to send packet: Operation not permitted dnsmasq[24]: failed to send packet: Operation not permitted dnsmasq[24]: failed to send packet: Operation not permitted dnsmasq-tftp[24]: sent /config/menus/netboot.xyz.kpxe to 10.0.0.2 dnsmasq[24]: failed to send packet: Operation not permitted dnsmasq-tftp[24]: sent /config/menus/netboot.xyz.kpxe to 10.0.0.2 .... ``` That IP 10.0.0.2 is part of the Ingress Network which is where the Container is reached from outside, so it doesn't have any relation with the PC booting on PXE. I am assuming this behaviour being somehow related to the Network settings on Container: ``` 9b4b56c9f2ff:/# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 22982: eth0@if22983: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue state UP link/ether 02:42:0a:00:00:e6 brd ff:ff:ff:ff:ff:ff inet 10.0.0.230/24 brd 10.0.0.255 scope global eth0 valid_lft forever preferred_lft forever 22984: eth1@if22985: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP link/ether 86:a5:53:cf:42:da brd ff:ff:ff:ff:ff:ff inet 172.18.0.11/16 brd 172.18.255.255 scope global eth1 valid_lft forever preferred_lft forever 22986: eth2@if22987: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue state UP link/ether 02:42:0a:00:01:d5 brd ff:ff:ff:ff:ff:ff inet 10.0.1.213/24 brd 10.0.1.255 scope global eth2 valid_lft forever preferred_lft forever 9b4b56c9f2ff:/# ``` Those settings are completely different from the one on a Docker (not Swarm) environment: ``` 80cfdd96e8dc:/# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN qlen 1000 link/ipip 0.0.0.0 brd 0.0.0.0 3: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN qlen 1000 link/gre 0.0.0.0 brd 0.0.0.0 4: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1476 qdisc noop state DOWN qlen 1000 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff 5: erspan0@NONE: <BROADCAST,MULTICAST> mtu 1464 qdisc noop state DOWN qlen 1000 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff 6: ip_vti0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN qlen 1000 link/ipip 0.0.0.0 brd 0.0.0.0 7: ip6_vti0@NONE: <NOARP> mtu 1364 qdisc noop state DOWN qlen 1000 link/tunnel6 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00 brd 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00 8: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN qlen 1000 link/sit 0.0.0.0 brd 0.0.0.0 9: ip6tnl0@NONE: <NOARP> mtu 1452 qdisc noop state DOWN qlen 1000 link/tunnel6 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00 brd 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00 10: ip6gre0@NONE: <NOARP> mtu 1448 qdisc noop state DOWN qlen 1000 link/[823] 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00 brd 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00 11: eth0@if6264: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP link/ether 46:74:29:71:e8:3c brd ff:ff:ff:ff:ff:ff inet 172.18.0.6/16 brd 172.18.255.255 scope global eth0 valid_lft forever preferred_lft forever 80cfdd96e8dc:/# ``` As I would like keep using Docker Swarm to guarantee HA, is there a workaround for this? Regards,
Author
Owner

@kaysond commented on GitHub (Dec 19, 2025):

Can you post your docker-compose? Are you using TFTPD_OPTS: '--tftp-single-port'? You probably need to otherwise the ingress network won't forward the data.

<!-- gh-comment-id:3672870881 --> @kaysond commented on GitHub (Dec 19, 2025): Can you post your docker-compose? Are you using `TFTPD_OPTS: '--tftp-single-port'`? You probably need to otherwise the ingress network won't forward the data.
Author
Owner

@masterlog80 commented on GitHub (Dec 21, 2025):

Hello @kaysond,

Can you post your docker-compose?

Sure. Here it is:

services:
  netbootxyz:
    container_name: netbootxyz
    image: ghcr.io/netbootxyz/netbootxyz:latest
    hostname: netbootxyz
    volumes:
      - /mnt/CEPHFS/volumes/netbootxyz/config:/config
      - NAS_NETBOOT_CIFS:/assets
    networks:
      - proxy
    environment:
      - TZ=Asia/Tokyo
      - PUID=1000 #current user
      - PGID=1000 #current group
      - PORT_RANGE=30000:30010 #optional
      - SUBFOLDER=/ #optional
      - WEB_APP_PORT=3000 #optional
      - MENU_VERSION=latest
    ports:
      - 69:69/udp
      - 3000:3000
    deploy:
      mode: replicated
      replicas: 1
      resources:
        reservations:
          cpus: '1.0'
          memory: 256M
        limits:
          cpus: '2.0'
          memory: 512M
    restart: unless-stopped

###########################

volumes:
  NAS_NETBOOT_CIFS:
    driver_opts:
      type: cifs
      o: vers=2.0,username=${NAS_CIFS_USER},password=${NAS_CIFS_PASSWORD},iocharset=utf8,file_mode=0777,dir_mode=0777,uid=1000,gid=1000
      device: //192.168.***.***/NETBOOT/assets/

###########################

networks:
  proxy:
    external: true

Are you using TFTPD_OPTS: '--tftp-single-port'? You probably need to otherwise the ingress network won't forward the data.

Nope. Let me try with that, as soon as possible.

Regards,

<!-- gh-comment-id:3678552639 --> @masterlog80 commented on GitHub (Dec 21, 2025): Hello @kaysond, > Can you post your docker-compose? Sure. Here it is: ``` services: netbootxyz: container_name: netbootxyz image: ghcr.io/netbootxyz/netbootxyz:latest hostname: netbootxyz volumes: - /mnt/CEPHFS/volumes/netbootxyz/config:/config - NAS_NETBOOT_CIFS:/assets networks: - proxy environment: - TZ=Asia/Tokyo - PUID=1000 #current user - PGID=1000 #current group - PORT_RANGE=30000:30010 #optional - SUBFOLDER=/ #optional - WEB_APP_PORT=3000 #optional - MENU_VERSION=latest ports: - 69:69/udp - 3000:3000 deploy: mode: replicated replicas: 1 resources: reservations: cpus: '1.0' memory: 256M limits: cpus: '2.0' memory: 512M restart: unless-stopped ########################### volumes: NAS_NETBOOT_CIFS: driver_opts: type: cifs o: vers=2.0,username=${NAS_CIFS_USER},password=${NAS_CIFS_PASSWORD},iocharset=utf8,file_mode=0777,dir_mode=0777,uid=1000,gid=1000 device: //192.168.***.***/NETBOOT/assets/ ########################### networks: proxy: external: true ``` > Are you using TFTPD_OPTS: '--tftp-single-port'? You probably need to otherwise the ingress network won't forward the data. Nope. Let me try with that, as soon as possible. Regards,
Author
Owner

@kaysond commented on GitHub (Dec 21, 2025):

Hello @kaysond,

Can you post your docker-compose?

Sure. Here it is:

    environment:
      - PORT_RANGE=30000:30010 #optional

Are you using TFTPD_OPTS: '--tftp-single-port'? You probably need to otherwise the ingress network won't forward the data.

Nope. Let me try with that, as soon as possible.

Regards,

I'm guessing you also moved over from the LSIO container? PORT_RANGE isn't supported by this image; it's not strictly a drop-in replacement despite what LSIO said. There should be a way to get the tftp server in this image to use a specific port range for data, but it was easier just to share the same port 69.

You were probably getting timeouts because the data ports were random and being blocked by a firewall.

<!-- gh-comment-id:3679488884 --> @kaysond commented on GitHub (Dec 21, 2025): > Hello [@kaysond](https://github.com/kaysond), > > > Can you post your docker-compose? > > Sure. Here it is: > > ``` > environment: > - PORT_RANGE=30000:30010 #optional > ``` > > > Are you using TFTPD_OPTS: '--tftp-single-port'? You probably need to otherwise the ingress network won't forward the data. > > Nope. Let me try with that, as soon as possible. > > Regards, I'm guessing you also moved over from the LSIO container? `PORT_RANGE` isn't supported by this image; it's not strictly a drop-in replacement despite what LSIO said. There should be a way to get the tftp server in this image to use a specific port range for data, but it was easier just to share the same port 69. You were probably getting timeouts because the data ports were random and being blocked by a firewall.
Author
Owner

@masterlog80 commented on GitHub (Jan 5, 2026):

Hello @kaysond ,
Sorry for late reply.

I'm guessing you also moved over from the LSIO container?

Nope, this is a new deployment on Docker Swarm (previously running on Docker standalone).
I have just put PORT_RANGE because I have seen it in the example YAML file.

There should be a way to get the tftp server in this image to use a specific port range for data, but it was easier just to share the same port 69.

The default port is indeed fine for me, and it worked with Docker standalone (non Swarm).

You were probably getting timeouts because the data ports were random and being blocked by a firewall.

I don't have any FW active on the device.

Do you know if netbootxyz was tested successfully on Docker Swarm?

Regards,

<!-- gh-comment-id:3709169145 --> @masterlog80 commented on GitHub (Jan 5, 2026): Hello @kaysond , Sorry for late reply. > I'm guessing you also moved over from the LSIO container? Nope, this is a new deployment on Docker Swarm (previously running on Docker standalone). I have just put `PORT_RANGE` because I have seen it in the example YAML file. > There should be a way to get the tftp server in this image to use a specific port range for data, but it was easier just to share the same port 69. The default port is indeed fine for me, and it worked with Docker standalone (non Swarm). > You were probably getting timeouts because the data ports were random and being blocked by a firewall. I don't have any FW active on the device. Do you know if netbootxyz was tested successfully on Docker Swarm? Regards,
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/docker-netbootxyz#150
No description provided.