[GH-ISSUE #12079] RustDesk Server: SSO/OAuth login breaks connectivity (WebSocket/secure tcp timeout) after successful install #2534

Open
opened 2026-02-26 12:52:45 +03:00 by kerem · 1 comment
Owner

Originally created by @HoBeom on GitHub (Feb 19, 2026).
Original GitHub issue: https://github.com/community-scripts/ProxmoxVE/issues/12079

Have you read and understood the above guidelines?

yes

🔎 Did you run the script with verbose mode enabled?

No (this issue will likely be closed automatically)

📜 What is the name of the script you are using?

RustDesk Server

📂 What was the exact command used to execute the script?

bash -c "$(curl -fsSL https://raw.githubusercontent.com/community-scripts/ProxmoxVE/main/ct/rustdeskserver.sh)"

⚙️ What settings are you using?

  • Default Settings
  • Advanced Settings

🖥️ Which Linux distribution are you using?

Debian 12

📈 Which Proxmox version are you on?

8.4.16

📝 Provide a clear and concise description of the issue.

The rustdeskserver LXC script installs successfully (hbbs/hbbr + rustdesk-api), but after enabling SSO/OAuth login via rustdesk-api and logging in from RustDesk Desktop, remote connectivity breaks.

  • Logged out: remote connections work normally.
  • After SSO/OAuth login succeeds: clients frequently become offline / rendezvous fails, with errors like:
    "Failed to secure tcp: deadline has elapsed"
    "Rendezvous server connection failed, try later"

This makes the login feature unusable even though installation completes without errors.

I suspect a WebSocket / reverse-proxy related issue triggered after login (client starts using ws endpoints), and/or a compatibility mismatch between rustdesk-api (lejianwen) and hbbs/hbbr packages installed by the script.

🔄 Steps to reproduce the issue.

Step 1: Install the container using the community-scripts rustdeskserver LXC script.
Step 2: Configure RustDesk Desktop clients to use the self-hosted ID/Relay server (works normally while logged out).
Step 3: Install/enable rustdesk-api (installed by the script) and open the admin UI.
Step 4: Configure OAuth/SSO (tested with Google and GitHub) and login successfully from the RustDesk Desktop client.
Step 5: After login succeeds, try remote control: connections fail and/or clients become offline.

Paste the full error output (if available).

RustDesk Desktop client errors after successful login:

  • "Failed to secure tcp: deadline has elapsed"
  • "Rendezvous server connection failed, try later"

WebSocket endpoints behind Nginx Proxy Manager (OpenResty) behave inconsistently:

  • HTTP/2 requests return 502
  • Forcing HTTP/1.1 shows WebSocket upgrade (101 Switching Protocols)

🖼️ Additional context (optional).

Related upstream discussion that seems relevant:
https://github.com/lejianwen/rustdesk-api/discussions/396

  • Please confirm whether this is a known issue / incompatibility when combining rustdesk-api (lejianwen) with hbbs/hbbr packages installed by the script.
  • Alternatively, consider installing a compatible hbbs/hbbr build set when rustdesk-api is installed.
Originally created by @HoBeom on GitHub (Feb 19, 2026). Original GitHub issue: https://github.com/community-scripts/ProxmoxVE/issues/12079 ### ✅ Have you read and understood the above guidelines? yes ### 🔎 Did you run the script with verbose mode enabled? No (this issue will likely be closed automatically) ### 📜 What is the name of the script you are using? RustDesk Server ### 📂 What was the exact command used to execute the script? bash -c "$(curl -fsSL https://raw.githubusercontent.com/community-scripts/ProxmoxVE/main/ct/rustdeskserver.sh)" ### ⚙️ What settings are you using? - [x] Default Settings - [ ] Advanced Settings ### 🖥️ Which Linux distribution are you using? Debian 12 ### 📈 Which Proxmox version are you on? 8.4.16 ### 📝 Provide a clear and concise description of the issue. The rustdeskserver LXC script installs successfully (hbbs/hbbr + rustdesk-api), but after enabling SSO/OAuth login via rustdesk-api and logging in from RustDesk Desktop, remote connectivity breaks. - Logged out: remote connections work normally. - After SSO/OAuth login succeeds: clients frequently become offline / rendezvous fails, with errors like: "Failed to secure tcp: deadline has elapsed" "Rendezvous server connection failed, try later" This makes the login feature unusable even though installation completes without errors. I suspect a WebSocket / reverse-proxy related issue triggered after login (client starts using ws endpoints), and/or a compatibility mismatch between rustdesk-api (lejianwen) and hbbs/hbbr packages installed by the script. ### 🔄 Steps to reproduce the issue. Step 1: Install the container using the community-scripts rustdeskserver LXC script. Step 2: Configure RustDesk Desktop clients to use the self-hosted ID/Relay server (works normally while logged out). Step 3: Install/enable rustdesk-api (installed by the script) and open the admin UI. Step 4: Configure OAuth/SSO (tested with Google and GitHub) and login successfully from the RustDesk Desktop client. Step 5: After login succeeds, try remote control: connections fail and/or clients become offline. ### ❌ Paste the full error output (if available). RustDesk Desktop client errors after successful login: - "Failed to secure tcp: deadline has elapsed" - "Rendezvous server connection failed, try later" WebSocket endpoints behind Nginx Proxy Manager (OpenResty) behave inconsistently: - HTTP/2 requests return 502 - Forcing HTTP/1.1 shows WebSocket upgrade (101 Switching Protocols) ### 🖼️ Additional context (optional). Related upstream discussion that seems relevant: https://github.com/lejianwen/rustdesk-api/discussions/396 - Please confirm whether this is a known issue / incompatibility when combining rustdesk-api (lejianwen) with hbbs/hbbr packages installed by the script. - Alternatively, consider installing a compatible hbbs/hbbr build set when rustdesk-api is installed.
Author
Owner

@HoBeom commented on GitHub (Feb 19, 2026):

Update / Workaround (no script changes required)

I was able to get login + post-login connectivity working by updating hbbs/hbbr inside the LXC to the lejianwen/rustdesk-server deb builds (hbbs/hbbr/utils), without modifying this ProxmoxVE script.

After swapping hbbs/hbbr binaries with lejianwen/rustdesk-server deb packages (hbbs/hbbr/utils) and restarting services, the issue disappears.

Commands (run inside the RustDesk LXC):

set -euo pipefail
apt-get update
apt-get install -y curl ca-certificates jq

WORK="/root/rustdesk_lejianwen_upgrade"
mkdir -p "$WORK"
cd "$WORK"

REL_JSON="$(curl -fsSL https://api.github.com/repos/lejianwen/rustdesk-server/releases/latest)"

HBBS_URL="$(echo "$REL_JSON" | jq -r '.assets[] | select(.name|test("rustdesk-server-hbbs.*amd64\\.deb$")) | .browser_download_url' | head -n1)"
HBBR_URL="$(echo "$REL_JSON" | jq -r '.assets[] | select(.name|test("rustdesk-server-hbbr.*amd64\\.deb$")) | .browser_download_url' | head -n1)"
UTIL_URL="$(echo "$REL_JSON" | jq -r '.assets[] | select(.name|test("rustdesk-server-utils.*amd64\\.deb$")) | .browser_download_url' | head -n1)"

curl -fL "$UTIL_URL" -o utils_amd64.deb
curl -fL "$HBBR_URL" -o hbbr_amd64.deb
curl -fL "$HBBS_URL" -o hbbs_amd64.deb

dpkg -i ./utils_amd64.deb ./hbbr_amd64.deb ./hbbs_amd64.deb || apt-get -f install -y

systemctl daemon-reload
systemctl restart rustdesk-hbbs rustdesk-hbbr rustdesk-api

/usr/bin/hbbs -V || true
/usr/bin/hbbr -V || true
ss -lntup | egrep ':(21114|21115|21116|21117|21118|21119)\b' || true

This looks like a hbbs/hbbr build compatibility issue with rustdesk-api login flows.

<!-- gh-comment-id:3925672299 --> @HoBeom commented on GitHub (Feb 19, 2026): Update / Workaround (no script changes required) I was able to get login + post-login connectivity working by updating hbbs/hbbr inside the LXC to the lejianwen/rustdesk-server deb builds (hbbs/hbbr/utils), without modifying this ProxmoxVE script. After swapping hbbs/hbbr binaries with lejianwen/rustdesk-server deb packages (hbbs/hbbr/utils) and restarting services, the issue disappears. Commands (run inside the RustDesk LXC): ``` set -euo pipefail apt-get update apt-get install -y curl ca-certificates jq WORK="/root/rustdesk_lejianwen_upgrade" mkdir -p "$WORK" cd "$WORK" REL_JSON="$(curl -fsSL https://api.github.com/repos/lejianwen/rustdesk-server/releases/latest)" HBBS_URL="$(echo "$REL_JSON" | jq -r '.assets[] | select(.name|test("rustdesk-server-hbbs.*amd64\\.deb$")) | .browser_download_url' | head -n1)" HBBR_URL="$(echo "$REL_JSON" | jq -r '.assets[] | select(.name|test("rustdesk-server-hbbr.*amd64\\.deb$")) | .browser_download_url' | head -n1)" UTIL_URL="$(echo "$REL_JSON" | jq -r '.assets[] | select(.name|test("rustdesk-server-utils.*amd64\\.deb$")) | .browser_download_url' | head -n1)" curl -fL "$UTIL_URL" -o utils_amd64.deb curl -fL "$HBBR_URL" -o hbbr_amd64.deb curl -fL "$HBBS_URL" -o hbbs_amd64.deb dpkg -i ./utils_amd64.deb ./hbbr_amd64.deb ./hbbs_amd64.deb || apt-get -f install -y systemctl daemon-reload systemctl restart rustdesk-hbbs rustdesk-hbbr rustdesk-api /usr/bin/hbbs -V || true /usr/bin/hbbr -V || true ss -lntup | egrep ':(21114|21115|21116|21117|21118|21119)\b' || true ``` This looks like a hbbs/hbbr build compatibility issue with rustdesk-api login flows.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/ProxmoxVE#2534
No description provided.