[GH-ISSUE #465] The L2TP Connection attempt failed because the security layer encountered a processing error during initial negotiations with the remote computer. #437

Closed
opened 2026-03-02 08:18:56 +03:00 by kerem · 4 comments
Owner

Originally created by @raychan907 on GitHub (Dec 23, 2024).
Original GitHub issue: https://github.com/hwdsl2/docker-ipsec-vpn-server/issues/465

Server (please complete the following information)

  • OS:
    Operating System: Ubuntu 24.04.1 LTS
    Kernel: Linux 6.8.0-1017-raspi
    Architecture: arm64

  • Hosting provider (if applicable): [e.g. GCP, AWS]
    Raspberry Pi 5

Client (please complete the following information)

  • Device: PC
  • OS: Windows 11 Pro
  • VPN mode: [IPsec/L2TP and IKEv2]

Additional context
VPN client name 是預設
這是env file 我設置過的.
VPN_IPSEC_PSK=
VPN_DNS_NAME=
VPN_USER=
VPN_PASSWORD=
VPN_ADDL_USERS=
VPN_ADDL_PASSWORDS=

VPN log.txt
docker compose.txt

Screenshot 2024-12-23 090354

  1. Windows regedit 已經設定好
    REG ADD HKLM\SYSTEM\CurrentControlSet\Services\PolicyAgent /v AssumeUDPEncapsulationContextOnSendRule /t REG_DWORD /d 0x2 /f
    REG ADD HKLM\SYSTEM\CurrentControlSet\Services\RasMan\Parameters /v ProhibitIpSec /t REG_DWORD /d 0x0 /f
  2. 用過iKEv2 和L2TP 一樣是有問題 連不上.
  3. 已將port 500 1701 4500 UDP forward到Pi5上. TNC hostname -P 是連得上. port 是沒問題的
  4. 除了L2TP 安全問題之外, 我也遇過 這個問題.
    Screenshot 2024-12-18 085522
  5. VPN 設定上 我試過把它改成允許PAP. 一樣是有相同問題
    Screenshot 2024-12-23 090503
  6. 暫時試過兩部電腦 雖然都是Windows 11, 但是一樣是連不上, 之前是試過用router 設定IPSec VPN. 但只成功1天, 之後還是連不上, 跟第4是一樣問題.
  7. Linux 上 ufw 已經允許500 1701 4500 udp, windows 上也做了.
    請問我是做少了哪一個步驟?
Originally created by @raychan907 on GitHub (Dec 23, 2024). Original GitHub issue: https://github.com/hwdsl2/docker-ipsec-vpn-server/issues/465 **Server (please complete the following information)** - OS: Operating System: Ubuntu 24.04.1 LTS Kernel: Linux 6.8.0-1017-raspi Architecture: arm64 - Hosting provider (if applicable): [e.g. GCP, AWS] Raspberry Pi 5 **Client (please complete the following information)** - Device: PC - OS: Windows 11 Pro - VPN mode: [IPsec/L2TP and IKEv2] **Additional context** VPN client name 是預設 這是env file 我設置過的. VPN_IPSEC_PSK= VPN_DNS_NAME= VPN_USER= VPN_PASSWORD= VPN_ADDL_USERS= VPN_ADDL_PASSWORDS= [VPN log.txt](https://github.com/user-attachments/files/18230114/VPN.log.txt) [docker compose.txt](https://github.com/user-attachments/files/18230262/docker.compose.txt) ![Screenshot 2024-12-23 090354](https://github.com/user-attachments/assets/258e081f-32e4-4855-9857-03f5e9a9e2b9) 1. Windows regedit 已經設定好 REG ADD HKLM\SYSTEM\CurrentControlSet\Services\PolicyAgent /v AssumeUDPEncapsulationContextOnSendRule /t REG_DWORD /d 0x2 /f REG ADD HKLM\SYSTEM\CurrentControlSet\Services\RasMan\Parameters /v ProhibitIpSec /t REG_DWORD /d 0x0 /f 2. 用過iKEv2 和L2TP 一樣是有問題 連不上. 3. 已將port 500 1701 4500 UDP forward到Pi5上. TNC hostname -P 是連得上. port 是沒問題的 4. 除了L2TP 安全問題之外, 我也遇過 這個問題. ![Screenshot 2024-12-18 085522](https://github.com/user-attachments/assets/940cd0d6-286a-4225-988f-c6a3cfe14547) 5. VPN 設定上 我試過把它改成允許PAP. 一樣是有相同問題 ![Screenshot 2024-12-23 090503](https://github.com/user-attachments/assets/3ae0dcd8-4873-4766-bfb6-197eb96ddc03) 6. 暫時試過兩部電腦 雖然都是Windows 11, 但是一樣是連不上, 之前是試過用router 設定IPSec VPN. 但只成功1天, 之後還是連不上, 跟第4是一樣問題. 7. Linux 上 ufw 已經允許500 1701 4500 udp, windows 上也做了. 請問我是做少了哪一個步驟?
kerem closed this issue 2026-03-02 08:18:56 +03:00
Author
Owner

@hwdsl2 commented on GitHub (Dec 23, 2024):

@raychan907 你好!你的 VPN 服务器配置和日志以及客户端注册表改动看起来基本正常。Docker compose 中 /lib/modules 目录应该映射到主机上的/lib/modules目录,而不是/opt/ipsec。另外,UDP 1701端口不需要打开或转发。

你可以启用 Docker 容器内的 Libreswan 日志,然后重新尝试连接,最后检查日志是否有错误。请参见:
https://github.com/hwdsl2/docker-ipsec-vpn-server/blob/master/docs/advanced-usage-zh.md#%E5%90%AF%E7%94%A8-libreswan-%E6%97%A5%E5%BF%97

如果尝试连接后没有新的日志,说明连接请求没有到达服务器,你可以自己继续进行故障排除。另外,如果日志中有 retransmission 相关错误,则说明可能是 VPN 客户端和服务器之间的网络问题。

另外,你也可以尝试不使用 Docker 直接搭建 VPN 服务器,或者试一下 IPsec VPN 之外的其他解决方案。

<!-- gh-comment-id:2560364995 --> @hwdsl2 commented on GitHub (Dec 23, 2024): @raychan907 你好!你的 VPN 服务器配置和日志以及客户端注册表改动看起来基本正常。Docker compose 中 /lib/modules 目录应该映射到主机上的/lib/modules目录,而不是/opt/ipsec。另外,UDP 1701端口不需要打开或转发。 你可以启用 Docker 容器内的 Libreswan 日志,然后重新尝试连接,最后检查日志是否有错误。请参见: https://github.com/hwdsl2/docker-ipsec-vpn-server/blob/master/docs/advanced-usage-zh.md#%E5%90%AF%E7%94%A8-libreswan-%E6%97%A5%E5%BF%97 如果尝试连接后没有新的日志,说明连接请求没有到达服务器,你可以自己继续进行故障排除。另外,如果日志中有 retransmission 相关错误,则说明可能是 VPN 客户端和服务器之间的网络问题。 另外,你也可以尝试不使用 Docker 直接搭建 VPN 服务器,或者试一下 IPsec VPN 之外的其他解决方案。
Author
Owner

@raychan907 commented on GitHub (Dec 24, 2024):

Docker compose 中 /lib/modules 目录应该映射到主机上的/lib/modules目录

這個不能更改本地位置嗎? 因為我想把資料都放在/opt裡

另外我本身有另外設定了wireguard 但我同時亦需要IPsec.

以下是我運行後的結果

root@rasp-pi5:~# docker exec -it ipsecvpn env TERM=xterm bash -l
ipsecvpn:/opt/src# # For Alpine-based image
apk add --no-cache rsyslog
rsyslogd
rc-service ipsec stop; rc-service -D ipsec start >/dev/null 2>&1
sed -i '\|pluto\.pid|a rm -f /var/run/rsyslogd.pid; rsyslogd' /opt/src/run.sh
exit
# For Debian-based image
apt-get update && apt-get -y install rsyslog
rsyslogd
service ipsec restart
sed -i '\|pluto\.pid|a rm -f /var/run/rsyslogd.pid; rsyslogd' /opt/src/run.sh
exit
fetch https://dl-cdn.alpinelinux.org/alpine/v3.20/main/aarch64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.20/community/aarch64/APKINDEX.tar.gz
(1/4) Installing libestr (0.1.11-r4)
(2/4) Installing libfastjson (1.2304.0-r4)
(3/4) Installing rsyslog (8.2404.0-r0)
(4/4) Installing rsyslog-openrc (8.2404.0-r0)
Executing busybox-1.36.1-r29.trigger
OK: 62 MiB in 88 packages
 * Caching service dependencies ...
Service `hwdrivers' needs non existent service `dev'
Service `machine-id' needs non existent service `dev'                                                                                                                                  [ ok ]
 * Stopping ipsec ...
ipsec addconn: /etc/ipsec.conf:19: warning: obsolete keyword ignored: dpdaction=clear
ipsec addconn: /etc/ipsec.d/ikev2.conf:16: warning: obsolete keyword ignored: dpdaction=clear                                                                                          [ ok ]
logout
<!-- gh-comment-id:2560476496 --> @raychan907 commented on GitHub (Dec 24, 2024): Docker compose 中 /lib/modules 目录应该映射到主机上的/lib/modules目录 這個不能更改本地位置嗎? 因為我想把資料都放在/opt裡 另外我本身有另外設定了wireguard 但我同時亦需要IPsec. 以下是我運行後的結果 ``` root@rasp-pi5:~# docker exec -it ipsecvpn env TERM=xterm bash -l ipsecvpn:/opt/src# # For Alpine-based image apk add --no-cache rsyslog rsyslogd rc-service ipsec stop; rc-service -D ipsec start >/dev/null 2>&1 sed -i '\|pluto\.pid|a rm -f /var/run/rsyslogd.pid; rsyslogd' /opt/src/run.sh exit # For Debian-based image apt-get update && apt-get -y install rsyslog rsyslogd service ipsec restart sed -i '\|pluto\.pid|a rm -f /var/run/rsyslogd.pid; rsyslogd' /opt/src/run.sh exit fetch https://dl-cdn.alpinelinux.org/alpine/v3.20/main/aarch64/APKINDEX.tar.gz fetch https://dl-cdn.alpinelinux.org/alpine/v3.20/community/aarch64/APKINDEX.tar.gz (1/4) Installing libestr (0.1.11-r4) (2/4) Installing libfastjson (1.2304.0-r4) (3/4) Installing rsyslog (8.2404.0-r0) (4/4) Installing rsyslog-openrc (8.2404.0-r0) Executing busybox-1.36.1-r29.trigger OK: 62 MiB in 88 packages * Caching service dependencies ... Service `hwdrivers' needs non existent service `dev' Service `machine-id' needs non existent service `dev' [ ok ] * Stopping ipsec ... ipsec addconn: /etc/ipsec.conf:19: warning: obsolete keyword ignored: dpdaction=clear ipsec addconn: /etc/ipsec.d/ikev2.conf:16: warning: obsolete keyword ignored: dpdaction=clear [ ok ] logout ```
Author
Owner

@hwdsl2 commented on GitHub (Dec 24, 2024):

@raychan907 是的,这样应该就启用 Libreswan 日志成功了。如果你把 /lib/modules 内容复制到别的目录的话,映射这个别的目录可能也可以。

<!-- gh-comment-id:2560517786 --> @hwdsl2 commented on GitHub (Dec 24, 2024): @raychan907 是的,这样应该就启用 Libreswan 日志成功了。如果你把 /lib/modules 内容复制到别的目录的话,映射这个别的目录可能也可以。
Author
Owner

@raychan907 commented on GitHub (Dec 24, 2024):

謝謝, 問題已解決.

<!-- gh-comment-id:2561146616 --> @raychan907 commented on GitHub (Dec 24, 2024): 謝謝, 問題已解決.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/docker-ipsec-vpn-server#437
No description provided.