[PR #2672] FIX: Ngnix fails to start if custom location upstream is unavailable/unreachable #3607

Open
opened 2026-02-26 08:31:16 +03:00 by kerem · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/NginxProxyManager/nginx-proxy-manager/pull/2672
Author: @kabadisha
Created: 3/12/2023
Status: 🔄 Open

Base: developHead: fix-startup-fail-when-upstream-is-down


📝 Commits (6)

  • 88e0295 Add variables for scheme, host & port
  • a0c945a Add trailing whitespace
  • 58c42a6 Add UI hints
  • e9a6b7b Merge branch 'NginxProxyManager:develop' into fix-startup-fail-when-upstream-is-down
  • 214efca Fix proper handling of paths and parameters
  • cf3a3bc Improve comments for clarity

📊 Changes

2 files changed (+36 additions, -2 deletions)

View changed files

📝 backend/templates/_location.conf (+30 -2)
📝 frontend/js/app/nginx/proxy/location-item.ejs (+6 -0)

📄 Description

⚠️ Potentially breaking change:
Upstream services with special configurations to compensate for not being hosted at root of domain are affected (e.g. Sonarr & Radarr). See section A potential problem below for an explanation.

Issue:

Currently, if a Proxy Host has a custom location and the host being forwarded to is down, Nginx fails to start.
See here, here and here for examples of others who have encountered this issue.

Impact:

In this scenario:

  • All other hosts managed by NPM are now also taken out.
  • NPM cannot be started in order to disable or edit the host configuration in order to rectify the issue.

Solution:

Declare a variable for forward_host instead of simply injecting it directly into the proxy_pass directive.

Although not strictly necessary to fix the issue, this PR also adds similar variables for forward_scheme and forward_port as this keeps things consistent and these variables could be useful in any custom elements an end-user might want to add.

Dialog

Handling paths and GET params properly:

With further testing and reading I discovered that the Nginx proxy_pass directive does not gracefully handle paths when using variables (source):

Changing

proxy_pass https://localhost:5000/api/;

to

set $upstream https://localhost:5000;
proxy_pass $upstream/api/;

you might think should result in exactly the same, you might be surprised. The former will hit your upstream server with /api/foo?bar=baz with our example request to /webapp/foo?bar=baz. The latter, however, will hit your upstream server with /api/. No foo. No bar. And no baz. :-(

I also found that @chaptergy had attempted something similar in the past, but reverted it because of this issue.
@chaptergy I believe the approach I have found works well. It certainly works in my testing. I'd appreciate your second opinion & testing though.

Testing:

It can be a bit tricky testing this because it isn't always obvious exactly what request is being passed to the upstream.
For my testing, I used http-https-echo as my upstream as it simply shows the details of the request being passed through directly in the browser. Super handy.

A potential problem:

For one of my proxy hosts, I had a custom location: /sonarr with the forward hostname set to sonarr.
It turns out that prior to this change, the proxy_pass directive was automatically including the custom location path /sonarr in the proxied request to the upstream Sonarr.
This meant that in Sonarr's configuration, I had configured the property URL Base to /sonarr to compensate.

With this PR, the proxy_pass directive now actually does what the NPM UI describes and so uses the path specified by the user in the Forward Hostname / IP field instead. If no path is provided there, then no path is used in the upstream request.
This means that with this change in place, I either have to set the URL Base property in Sonarr back to the default / or set the Forward Hostname / IP field in NPM to sonarr/sonarr.

This also affects Radarr in the same way.

Conclusion:

Personally, I think this PR actually makes NPM behave in a way that is more flexible, explicit and in keeping with what the UI actually presents. However, I'm pretty sure that this makes it a potentially breaking change for many users who have 'worked around' this issue by configuring their upstream systems to recognise that they are not hosted at the root of the domain. It also needs some decent testing by the community.

On the flip side, this PR would:

  • Make it less likely that less experienced users would get tripped up by NPM silently adding a path to upstream requests and make compensating configuration at the upstream service unnecessary.
  • Enable compatibility with upstream services that are not able to be configured to operate at non-root paths.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/NginxProxyManager/nginx-proxy-manager/pull/2672 **Author:** [@kabadisha](https://github.com/kabadisha) **Created:** 3/12/2023 **Status:** 🔄 Open **Base:** `develop` ← **Head:** `fix-startup-fail-when-upstream-is-down` --- ### 📝 Commits (6) - [`88e0295`](https://github.com/NginxProxyManager/nginx-proxy-manager/commit/88e0295a06361082e589209e3d4bd5df5c2d1ac4) Add variables for scheme, host & port - [`a0c945a`](https://github.com/NginxProxyManager/nginx-proxy-manager/commit/a0c945a6148b7686fe9d332b233c55b879a5e167) Add trailing whitespace - [`58c42a6`](https://github.com/NginxProxyManager/nginx-proxy-manager/commit/58c42a6ce8bdb58f6917918c5ee50a51ca3a2244) Add UI hints - [`e9a6b7b`](https://github.com/NginxProxyManager/nginx-proxy-manager/commit/e9a6b7b2c65a3142576af00b2fda682e1ef670cd) Merge branch 'NginxProxyManager:develop' into fix-startup-fail-when-upstream-is-down - [`214efca`](https://github.com/NginxProxyManager/nginx-proxy-manager/commit/214efca62c236be01a866602561e96e3a78be677) Fix proper handling of paths and parameters - [`cf3a3bc`](https://github.com/NginxProxyManager/nginx-proxy-manager/commit/cf3a3bc4b9ae75949d46aa6d3a8b67de2a96f610) Improve comments for clarity ### 📊 Changes **2 files changed** (+36 additions, -2 deletions) <details> <summary>View changed files</summary> 📝 `backend/templates/_location.conf` (+30 -2) 📝 `frontend/js/app/nginx/proxy/location-item.ejs` (+6 -0) </details> ### 📄 Description :warning: **Potentially breaking change**: Upstream services with special configurations to compensate for not being hosted at root of domain are affected (e.g. Sonarr & Radarr). See section **A potential problem** below for an explanation. ### Issue: Currently, if a Proxy Host has a custom location and the host being forwarded to is down, Nginx fails to start. See [here](https://github.com/NginxProxyManager/nginx-proxy-manager/discussions/2228), [here](https://github.com/NginxProxyManager/nginx-proxy-manager/issues/633) and [here](https://github.com/NginxProxyManager/nginx-proxy-manager/issues/669) for examples of others who have encountered this issue. ### Impact: In this scenario: - All other hosts managed by NPM are now also taken out. - NPM cannot be started in order to disable or edit the host configuration in order to rectify the issue. ### Solution: Declare a variable for `forward_host` instead of simply injecting it directly into the `proxy_pass` directive. Although not strictly necessary to fix the issue, this PR also adds similar variables for `forward_scheme` and `forward_port` as this keeps things consistent and these variables could be useful in any custom elements an end-user might want to add. <img width="300" alt="Dialog" src="https://user-images.githubusercontent.com/593306/224549012-4bfcfcbc-0f8e-46c7-abd1-d611d4c30935.png"> ### Handling paths and GET params properly: With further testing and reading I discovered that the Nginx `proxy_pass` directive does not gracefully handle paths when using variables ([source](https://dev.to/danielkun/nginx-everything-about-proxypass-2ona#let-nginx-start-even-when-not-all-upstream-hosts-are-available)): > Changing > ``` > proxy_pass https://localhost:5000/api/; > ``` > to > ``` >set $upstream https://localhost:5000; > proxy_pass $upstream/api/; > ``` > you might think should result in exactly the same, you might be surprised. The former will hit your upstream server with /api/foo?bar=baz with our example request to /webapp/foo?bar=baz. The latter, however, will hit your upstream server with /api/. No foo. No bar. And no baz. :-( I also found that @chaptergy had [attempted something similar in the past, but reverted it](https://github.com/NginxProxyManager/nginx-proxy-manager/commit/1f879f67a9bed8f5eb444d00ad47c353b8a75db8?diff=unified) because of this issue. @chaptergy I believe the approach I have found works well. It certainly works in my testing. I'd appreciate your second opinion & testing though. ### Testing: It can be a bit tricky testing this because it isn't always obvious exactly what request is being passed to the upstream. For my testing, I used [http-https-echo](https://code.mendhak.com/docker-http-https-echo/) as my upstream as it simply shows the details of the request being passed through directly in the browser. Super handy. ### A potential problem: For one of my proxy hosts, I had a custom location: `/sonarr` with the forward hostname set to `sonarr`. It turns out that prior to this change, the `proxy_pass` directive was automatically including the custom location path `/sonarr` in the proxied request to the upstream Sonarr. This meant that in Sonarr's configuration, I had configured the property `URL Base` to `/sonarr` to compensate. With this PR, the `proxy_pass` directive now actually does what the NPM UI describes and so uses the path specified by the user in the `Forward Hostname / IP` field instead. If no path is provided there, then no path is used in the upstream request. This means that with this change in place, I either have to set the `URL Base` property in Sonarr back to the default `/` or set the `Forward Hostname / IP` field in NPM to `sonarr/sonarr`. This also affects Radarr in the same way. ### Conclusion: Personally, I think this PR actually makes NPM behave in a way that is more flexible, explicit and in keeping with what the UI actually presents. However, I'm pretty sure that this makes it a potentially **breaking change** for many users who have 'worked around' this issue by configuring their upstream systems to recognise that they are not hosted at the root of the domain. It also needs some decent testing by the community. **On the flip side, this PR would:** - Make it less likely that less experienced users would get tripped up by NPM silently adding a path to upstream requests and make compensating configuration at the upstream service unnecessary. - Enable compatibility with upstream services that are not able to be configured to operate at non-root paths. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/nginx-proxy-manager-NginxProxyManager#3607
No description provided.