[GH-ISSUE #276] Sendkey API call randomly ending in EOF failing packer build #60

Open
opened 2026-02-28 00:40:24 +03:00 by kerem · 1 comment
Owner

Originally created by @newedgex on GitHub (Oct 14, 2023).
Original GitHub issue: https://github.com/Telmate/proxmox-api-go/issues/276

https://github.com/Telmate/proxmox-api-go/blob/master/proxmox/client.go#L411

About the environment :

  • Server side : This has been verified on Proxmox v6, v7 and the latest 8.0.4 , and also with different versions of packer. I always connect with https through a valid letsencrypt certificate, the connection is also encapsulated through wireguard VPN.
  • Client side : This happened with different versions of packer and also packer-plugin-proxmox , and it also happens with different templates wherever I use the "Typing the boot command" which relies on a PUT request to the Sendkey API endpoint.
    What happens :
    short log :
2023/10/12 10:12:35 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:35 [INFO] Waiting 2s
2023/10/12 10:12:37 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:37 [INFO] Waiting 2s
2023/10/12 10:12:44 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:44 [INFO] Waiting 5s
==> proxmox-iso.ubuntu-server: Error running boot command: Put "https://proxmox-02-01.upstream.domain.com:8006/api2/json/nodes/proxmox-02-01/qemu/102/sendkey": EOF
==> proxmox-iso.ubuntu-server: Stopping VM
==> proxmox-iso.ubuntu-server: Deleting VM

long log :


2023/10/12 10:12:06 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:06 Plugin address: unix /tmp/packer-plugin4053739862
2023/10/12 10:12:06 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:06 Waiting for connection...
2023/10/12 10:12:06 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:06 Serving a plugin connection...
2023/10/12 10:12:06 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:06 [TRACE] starting builder iso
2023/10/12 10:12:06 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:06 Number of sockets 0 is too small, using default: 1
2023/10/12 10:12:06 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:06 OS not set, using default 'other'
2023/10/12 10:12:06 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:06 Disk 0 cache mode not set, using default 'none'
2023/10/12 10:12:07 Build debug mode: false
2023/10/12 10:12:07 Force build: false
2023/10/12 10:12:07 On error: 
2023/10/12 10:12:07 Waiting on builds to complete...
2023/10/12 10:12:07 Starting build run: proxmox-iso.ubuntu-server
proxmox-iso.ubuntu-server: output will be in this color.
2023/10/12 10:12:07 Running builder: 
2023/10/12 10:12:07 [INFO] (telemetry) Starting builder proxmox-iso.ubuntu-server

2023/10/12 10:12:07 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:07 using token auth
2023/10/12 10:12:07 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:07 No URLs were provided to Step Download. Continuing...
==> proxmox-iso.ubuntu-server: Creating VM
==> proxmox-iso.ubuntu-server: No VM ID given, getting next free from Proxmox
==> proxmox-iso.ubuntu-server: Starting VM
==> proxmox-iso.ubuntu-server: Waiting 20s for boot
==> proxmox-iso.ubuntu-server: Typing the boot command
2023/10/12 10:12:35 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:35 [INFO] Waiting 2s
2023/10/12 10:12:37 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:37 [INFO] Waiting 2s
2023/10/12 10:12:44 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:44 [INFO] Waiting 5s
==> proxmox-iso.ubuntu-server: Error running boot command: Put "https://proxmox-02-01.upstream.domain.com:8006/api2/json/nodes/proxmox-02-01/qemu/102/sendkey": EOF
==> proxmox-iso.ubuntu-server: Stopping VM
==> proxmox-iso.ubuntu-server: Deleting VM
2023/10/12 10:12:55 [INFO] (telemetry) ending proxmox-iso.ubuntu-server
==> Wait completed after 48 seconds 224 milliseconds
2023/10/12 10:12:55 machine readable: error-count []string{"1"}
==> Some builds didn't complete successfully and had errors:
2023/10/12 10:12:55 machine readable: proxmox-iso.ubuntu-server,error []string{"Error running boot command: Put \"https://proxmox-02-01.upstream.domain.com:8006/api2/json/nodes/proxmox-02-01/qemu/102/sendkey\": EOF"}
==> Builds finished but no artifacts were created.
2023/10/12 10:12:55 [INFO] (telemetry) Finalizing.
Build 'proxmox-iso.ubuntu-server' errored after 48 seconds 224 milliseconds: Error running boot command: Put "https://proxmox-02-01.upstream.domain.com:8006/api2/json/nodes/proxmox-02-01/qemu/102/sendkey": EOF

==> Wait completed after 48 seconds 224 milliseconds

==> Some builds didn't complete successfully and had errors:
--> proxmox-iso.ubuntu-server: Error running boot command: Put "https://proxmox-02-01.upstream.domain.com:8006/api2/json/nodes/proxmox-02-01/qemu/102/sendkey": EOF

==> Builds finished but no artifacts were created.
2023/10/12 10:12:56 waiting for all plugin processes to complete...
2023/10/12 10:12:56 /home/user/.config/packer/plugins/github.com/hashicorp/proxmox/packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64: plugin process exited

Since EOF usually happens due to connectivity issues, I'm wondering what is actually happening since both my and proxmox's connections are fine, and I'm the only one using it so there's no network saturation.

I have reported the issue also on :

Originally created by @newedgex on GitHub (Oct 14, 2023). Original GitHub issue: https://github.com/Telmate/proxmox-api-go/issues/276 https://github.com/Telmate/proxmox-api-go/blob/master/proxmox/client.go#L411 About the environment : - **Server side** : This has been verified on Proxmox v6, v7 and the latest 8.0.4 , and also with different versions of packer. I always connect with https through a valid letsencrypt certificate, the connection is also encapsulated through wireguard VPN. - **Client side** : This happened with different versions of packer and also packer-plugin-proxmox , and it also happens with different templates wherever I use the "Typing the boot command" which relies on a PUT request to the Sendkey API endpoint. What happens : short log : ``` 2023/10/12 10:12:35 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:35 [INFO] Waiting 2s 2023/10/12 10:12:37 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:37 [INFO] Waiting 2s 2023/10/12 10:12:44 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:44 [INFO] Waiting 5s ==> proxmox-iso.ubuntu-server: Error running boot command: Put "https://proxmox-02-01.upstream.domain.com:8006/api2/json/nodes/proxmox-02-01/qemu/102/sendkey": EOF ==> proxmox-iso.ubuntu-server: Stopping VM ==> proxmox-iso.ubuntu-server: Deleting VM ``` long log : ```bash 2023/10/12 10:12:06 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:06 Plugin address: unix /tmp/packer-plugin4053739862 2023/10/12 10:12:06 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:06 Waiting for connection... 2023/10/12 10:12:06 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:06 Serving a plugin connection... 2023/10/12 10:12:06 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:06 [TRACE] starting builder iso 2023/10/12 10:12:06 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:06 Number of sockets 0 is too small, using default: 1 2023/10/12 10:12:06 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:06 OS not set, using default 'other' 2023/10/12 10:12:06 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:06 Disk 0 cache mode not set, using default 'none' 2023/10/12 10:12:07 Build debug mode: false 2023/10/12 10:12:07 Force build: false 2023/10/12 10:12:07 On error: 2023/10/12 10:12:07 Waiting on builds to complete... 2023/10/12 10:12:07 Starting build run: proxmox-iso.ubuntu-server proxmox-iso.ubuntu-server: output will be in this color. 2023/10/12 10:12:07 Running builder: 2023/10/12 10:12:07 [INFO] (telemetry) Starting builder proxmox-iso.ubuntu-server 2023/10/12 10:12:07 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:07 using token auth 2023/10/12 10:12:07 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:07 No URLs were provided to Step Download. Continuing... ==> proxmox-iso.ubuntu-server: Creating VM ==> proxmox-iso.ubuntu-server: No VM ID given, getting next free from Proxmox ==> proxmox-iso.ubuntu-server: Starting VM ==> proxmox-iso.ubuntu-server: Waiting 20s for boot ==> proxmox-iso.ubuntu-server: Typing the boot command 2023/10/12 10:12:35 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:35 [INFO] Waiting 2s 2023/10/12 10:12:37 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:37 [INFO] Waiting 2s 2023/10/12 10:12:44 packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64 plugin: 2023/10/12 10:12:44 [INFO] Waiting 5s ==> proxmox-iso.ubuntu-server: Error running boot command: Put "https://proxmox-02-01.upstream.domain.com:8006/api2/json/nodes/proxmox-02-01/qemu/102/sendkey": EOF ==> proxmox-iso.ubuntu-server: Stopping VM ==> proxmox-iso.ubuntu-server: Deleting VM 2023/10/12 10:12:55 [INFO] (telemetry) ending proxmox-iso.ubuntu-server ==> Wait completed after 48 seconds 224 milliseconds 2023/10/12 10:12:55 machine readable: error-count []string{"1"} ==> Some builds didn't complete successfully and had errors: 2023/10/12 10:12:55 machine readable: proxmox-iso.ubuntu-server,error []string{"Error running boot command: Put \"https://proxmox-02-01.upstream.domain.com:8006/api2/json/nodes/proxmox-02-01/qemu/102/sendkey\": EOF"} ==> Builds finished but no artifacts were created. 2023/10/12 10:12:55 [INFO] (telemetry) Finalizing. Build 'proxmox-iso.ubuntu-server' errored after 48 seconds 224 milliseconds: Error running boot command: Put "https://proxmox-02-01.upstream.domain.com:8006/api2/json/nodes/proxmox-02-01/qemu/102/sendkey": EOF ==> Wait completed after 48 seconds 224 milliseconds ==> Some builds didn't complete successfully and had errors: --> proxmox-iso.ubuntu-server: Error running boot command: Put "https://proxmox-02-01.upstream.domain.com:8006/api2/json/nodes/proxmox-02-01/qemu/102/sendkey": EOF ==> Builds finished but no artifacts were created. 2023/10/12 10:12:56 waiting for all plugin processes to complete... 2023/10/12 10:12:56 /home/user/.config/packer/plugins/github.com/hashicorp/proxmox/packer-plugin-proxmox_v1.1.5_x5.0_linux_amd64: plugin process exited ``` Since EOF usually happens due to connectivity issues, I'm wondering what is actually happening since both my and proxmox's connections are fine, and I'm the only one using it so there's no network saturation. I have reported the issue also on : - https://github.com/hashicorp/packer-plugin-proxmox/issues/237 - https://forum.proxmox.com/threads/api-error-running-boot-command-sendkey-eof.134796/
Author
Owner

@stmcginnis commented on GitHub (Feb 8, 2024):

Same error here. Using packer 1.9.5, running commands from the latest CAPI image builder.

PACKER_FLAGS="--var 'kubernetes_rpm_version=1.28.3' --var 'kubernetes_semver=v1.28.3' --var 'kubernetes_series=v1.28' --var 'kubernetes_deb_version=1.28.3-1.1'" make build-proxmox-ubuntu-2204
# pveversion
pve-manager/8.1.4/ec5affc9e41f1d79 (running kernel: 6.5.11-7-pve)
<!-- gh-comment-id:1934446473 --> @stmcginnis commented on GitHub (Feb 8, 2024): Same error here. Using packer 1.9.5, running commands from the latest CAPI [image builder](https://github.com/kubernetes-sigs/image-builder/tree/main/images/capi/packer/proxmox). ```sh PACKER_FLAGS="--var 'kubernetes_rpm_version=1.28.3' --var 'kubernetes_semver=v1.28.3' --var 'kubernetes_series=v1.28' --var 'kubernetes_deb_version=1.28.3-1.1'" make build-proxmox-ubuntu-2204 ``` ```sh # pveversion pve-manager/8.1.4/ec5affc9e41f1d79 (running kernel: 6.5.11-7-pve) ```
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/proxmox-api-go#60
No description provided.