[GH-ISSUE #795] Unhealthy container #552

Closed
opened 2026-03-03 01:30:28 +03:00 by kerem · 12 comments
Owner

Originally created by @STaRDoGG on GitHub (Jan 2, 2020).
Original GitHub issue: https://github.com/dani-garcia/vaultwarden/issues/795

#713 # Subject of the issue
My container is showing as "Unhealthy"; restarting the container seems to have no effect, as it will always still show as unhealthy.

image

Your environment

  • Bitwarden_rs version:

image

  • Install method: Docker image using compose.
version: '2'

# https://hub.docker.com/r/bitwardenrs/server

services:

  bitwarden:
    container_name: bitwarden
    image: 'bitwardenrs/server:latest'
    environment:
      - TZ='America/Chicago'
      - PUID=197609
      - PGID=197121
    restart: unless-stopped
    volumes:
      - 'scotts_bitwarden_data:/data'
    ports:
        - '9069:80'
    networks:
      - om

networks:
  om:
    external:
      name: oly-misc
  • Reverse proxy and version: Nginx
  • Version of mysql/postgresql: N/A

Steps to reproduce

Just with the above compose code, and once created, just the standard start/stop commands.

Expected behaviour

A healthy container. ;P

Actual behaviour

An unhealthy container ;P

Relevant logs

Looking at the main log it shows nothing of interest related to errors, etc. If there's another log to view that may provide better info, lemme know and I'll post it.

Originally created by @STaRDoGG on GitHub (Jan 2, 2020). Original GitHub issue: https://github.com/dani-garcia/vaultwarden/issues/795 #713 # Subject of the issue My container is showing as "Unhealthy"; restarting the container seems to have no effect, as it will always still show as unhealthy. ![image](https://user-images.githubusercontent.com/1524526/71656406-aa61af80-2d00-11ea-9fd8-52b2b241bfbb.png) ### Your environment <!-- The version number, obtained from the logs or the admin page --> * Bitwarden_rs version: ![image](https://user-images.githubusercontent.com/1524526/71656441-d2511300-2d00-11ea-9a79-bec76453c88d.png) <!-- How the server was installed: Docker image / package / built from source --> * Install method: Docker image using compose. ``` version: '2' # https://hub.docker.com/r/bitwardenrs/server services: bitwarden: container_name: bitwarden image: 'bitwardenrs/server:latest' environment: - TZ='America/Chicago' - PUID=197609 - PGID=197121 restart: unless-stopped volumes: - 'scotts_bitwarden_data:/data' ports: - '9069:80' networks: - om networks: om: external: name: oly-misc ``` * Reverse proxy and version: Nginx * Version of mysql/postgresql: N/A ### Steps to reproduce <!-- Tell us how to reproduce this issue. What parameters did you set (differently from the defaults) and how did you start bitwarden_rs? --> Just with the above compose code, and once created, just the standard start/stop commands. ### Expected behaviour A **healthy** container. ;P ### Actual behaviour An **unhealthy** container ;P ### Relevant logs <!-- Share some logfiles, screenshots or output of relevant programs with us. --> Looking at the main log it shows nothing of interest related to errors, etc. If there's another log to view that may provide better info, lemme know and I'll post it.
kerem closed this issue 2026-03-03 01:30:28 +03:00
Author
Owner

@mprasil commented on GitHub (Jan 2, 2020):

That sounds like health check failing for whatever reason or maybe container being reported as unhealthy before first health check returning success.

Can you docker exec health check manually and see if that works?

<!-- gh-comment-id:570278441 --> @mprasil commented on GitHub (Jan 2, 2020): That sounds like health check failing for whatever reason or maybe container being reported as unhealthy before first health check returning success. Can you docker exec health check manually and see if that works?
Author
Owner

@STaRDoGG commented on GitHub (Jan 3, 2020):

Can you docker exec health check manually and see if that works?

Sorry, I'm still learning my way around Docker and not familiar with manually running a health check; can you save me a bit of time digging up the command to do it?

<!-- gh-comment-id:570661901 --> @STaRDoGG commented on GitHub (Jan 3, 2020): > Can you docker exec health check manually and see if that works? Sorry, I'm still learning my way around Docker and not familiar with manually running a health check; can you save me a bit of time digging up the command to do it?
Author
Owner

@mprasil commented on GitHub (Jan 3, 2020):

Ah apologies. So you can docker exec <container name> <command> commands inside container:

docker exec -ti bitwarden healthcheck.sh

You can also try running curl directly for some extra verbosity:

docker exec -ti bitwarden curl -v http://localhost:80/alive
<!-- gh-comment-id:570672145 --> @mprasil commented on GitHub (Jan 3, 2020): Ah apologies. So you can `docker exec <container name> <command>` commands inside container: ```bash docker exec -ti bitwarden healthcheck.sh ``` You can also try running `curl` directly for some extra verbosity: ```bash docker exec -ti bitwarden curl -v http://localhost:80/alive ```
Author
Owner

@STaRDoGG commented on GitHub (Jan 4, 2020):

Thanks @mprasil, I appreciate it.

Trying to run:

docker exec -ti bitwarden healthcheck.sh

Just gives me:

OCI runtime exec failed: exec failed: container_linux.go:346: starting container process caused "exec: "healthcheck.sh": executable file not found in $PATH": unknown

and:

 docker exec -ti bitwarden curl -v http://localhost:80/alive

curl: error while loading shared libraries: /usr/lib/x86_64-linux-gnu/libidn2.so.0: file too short

I should mention that I'm running Docker Desktop on Win 10, in case it helps.

<!-- gh-comment-id:570752803 --> @STaRDoGG commented on GitHub (Jan 4, 2020): Thanks @mprasil, I appreciate it. Trying to run: ```shell docker exec -ti bitwarden healthcheck.sh ``` Just gives me: > OCI runtime exec failed: exec failed: container_linux.go:346: starting container process caused "exec: \"healthcheck.sh\": executable file not found in $PATH": unknown and: ```shell docker exec -ti bitwarden curl -v http://localhost:80/alive ``` > curl: error while loading shared libraries: /usr/lib/x86_64-linux-gnu/libidn2.so.0: file too short I should mention that I'm running Docker Desktop on Win 10, in case it helps.
Author
Owner

@mprasil commented on GitHub (Jan 14, 2020):

Hmm, looks like for some reason the curl fails to run in windows docker. I currently have no way of testing how that behaves, but as a quick workaround, you can disable health check.

For your docker compose:

version: '2'

# https://hub.docker.com/r/bitwardenrs/server

services:

  bitwarden:
    container_name: bitwarden
    image: 'bitwardenrs/server:latest'
    environment:
      - TZ='America/Chicago'
      - PUID=197609
      - PGID=197121
    restart: unless-stopped
    volumes:
      - 'scotts_bitwarden_data:/data'
    ports:
        - '9069:80'
    networks:
      - om
    healthcheck:
      disable: true

networks:
  om:
    external:
      name: oly-misc
<!-- gh-comment-id:574275672 --> @mprasil commented on GitHub (Jan 14, 2020): Hmm, looks like for some reason the `curl` fails to run in windows docker. I currently have no way of testing how that behaves, but as a quick workaround, you can disable health check. For your docker compose: ```yaml version: '2' # https://hub.docker.com/r/bitwardenrs/server services: bitwarden: container_name: bitwarden image: 'bitwardenrs/server:latest' environment: - TZ='America/Chicago' - PUID=197609 - PGID=197121 restart: unless-stopped volumes: - 'scotts_bitwarden_data:/data' ports: - '9069:80' networks: - om healthcheck: disable: true networks: om: external: name: oly-misc ```
Author
Owner

@STaRDoGG commented on GitHub (Jan 14, 2020):

@mprasil Thanks, I'm curious as to actually figuring out the issue rather than hiding it. =)

I managed to mess around a bit and get curl to work from another machine on the LAN, calling BW on the port I assigned to the container, and here's the results if it gives any insight?

curl -v http://192.168.0.194:9028/alive
*   Trying 192.168.0.194...
* TCP_NODELAY set
* Connected to 192.168.0.194 (192.168.0.194) port 9028 (#0)
> GET /alive HTTP/1.1
> Host: 192.168.0.194:9028
> User-Agent: curl/7.55.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: application/json
< Server: Rocket
< Feature-Policy: accelerometer 'none'; ambient-light-sensor 'none'; autoplay 'none'; camera 'none'; encrypted-media 'none'; fullscreen 'none'; geolocation 'none'; gyroscope 'none'; magnetometer 'none'; microphone 'none'; midi 'none'; payment 'none'; picture-in-picture 'none'; sync-xhr 'self' https://haveibeenpwned.com https://twofactorauth.org; usb 'none'; vr 'none'
< Referrer-Policy: same-origin
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< Content-Security-Policy: frame-ancestors 'self' chrome-extension://nngceckbapebfimnlniiiahkandclblb moz-extension://*;
< Cache-Control: no-cache, no-store, max-age=0
< Access-Control-Allow-Origin:
< Content-Length: 29
< Date: Tue, 14 Jan 2020 21:01:53 GMT
<
"2020-01-14T21:01:53.613227Z"* Connection #0 to host 192.168.0.194 left intact

If I try to run healthcheck.sh from within the container, I just get (not sure I'm doing that right?):

root@4c41b4599966:/# healthcheck.sh
bash: healthcheck.sh: command not found

If I go directly to http://192.168.0.194:9028/alive I get:

"2020-01-14T21:09:25.547858Z"

Another strange thing; when it's not permanently shown as "unhealthy", it also often shows itself locked as "starting".

Any ideas?

<!-- gh-comment-id:574375797 --> @STaRDoGG commented on GitHub (Jan 14, 2020): @mprasil Thanks, I'm curious as to actually figuring out the issue rather than hiding it. =) I managed to mess around a bit and get curl to work from another machine on the LAN, calling BW on the port I assigned to the container, and here's the results if it gives any insight? ``` curl -v http://192.168.0.194:9028/alive * Trying 192.168.0.194... * TCP_NODELAY set * Connected to 192.168.0.194 (192.168.0.194) port 9028 (#0) > GET /alive HTTP/1.1 > Host: 192.168.0.194:9028 > User-Agent: curl/7.55.1 > Accept: */* > < HTTP/1.1 200 OK < Content-Type: application/json < Server: Rocket < Feature-Policy: accelerometer 'none'; ambient-light-sensor 'none'; autoplay 'none'; camera 'none'; encrypted-media 'none'; fullscreen 'none'; geolocation 'none'; gyroscope 'none'; magnetometer 'none'; microphone 'none'; midi 'none'; payment 'none'; picture-in-picture 'none'; sync-xhr 'self' https://haveibeenpwned.com https://twofactorauth.org; usb 'none'; vr 'none' < Referrer-Policy: same-origin < X-Frame-Options: SAMEORIGIN < X-Content-Type-Options: nosniff < X-XSS-Protection: 1; mode=block < Content-Security-Policy: frame-ancestors 'self' chrome-extension://nngceckbapebfimnlniiiahkandclblb moz-extension://*; < Cache-Control: no-cache, no-store, max-age=0 < Access-Control-Allow-Origin: < Content-Length: 29 < Date: Tue, 14 Jan 2020 21:01:53 GMT < "2020-01-14T21:01:53.613227Z"* Connection #0 to host 192.168.0.194 left intact ``` If I try to run `healthcheck.sh` from within the container, I just get (not sure I'm doing that right?): > root@4c41b4599966:/# healthcheck.sh > bash: healthcheck.sh: command not found If I go directly to `http://192.168.0.194:9028/alive` I get: > "2020-01-14T21:09:25.547858Z" Another strange thing; when it's not permanently shown as "unhealthy", it also often shows itself locked as "starting". Any ideas?
Author
Owner

@mprasil commented on GitHub (Jan 16, 2020):

I think the healthcheck needs to be executed with the full path:

docker exec -ti bitwarden /healthcheck.sh

If you want to test with curl you can just exec inside the container:

docker exec -ti bitwarden /bin/bash

And then you can try out curl:

curl -v http://localhost:80/alive

You can play around inside and exit with exit.

<!-- gh-comment-id:575197930 --> @mprasil commented on GitHub (Jan 16, 2020): I think the healthcheck needs to be executed with the full path: ```bash docker exec -ti bitwarden /healthcheck.sh ``` If you want to test with `curl` you can just exec inside the container: ```bash docker exec -ti bitwarden /bin/bash ``` And then you can try out curl: ```bash curl -v http://localhost:80/alive ``` You can play around inside and exit with `exit`.
Author
Owner

@mprasil commented on GitHub (Jan 16, 2020):

Another strange thing; when it's not permanently shown as "unhealthy", it also often shows itself locked as "starting".

It could be restarted due to failing healthcheck? There's a delay before the check is executed and it could be showing as starting? Just guessing.

<!-- gh-comment-id:575198463 --> @mprasil commented on GitHub (Jan 16, 2020): > Another strange thing; when it's not permanently shown as "unhealthy", it also often shows itself locked as "starting". It could be restarted due to failing healthcheck? There's a delay before the check is executed and it could be showing as starting? Just guessing.
Author
Owner

@dani-garcia commented on GitHub (May 13, 2020):

Closed due to inactivity.

<!-- gh-comment-id:628280666 --> @dani-garcia commented on GitHub (May 13, 2020): Closed due to inactivity.
Author
Owner

@Sparkenstein commented on GitHub (May 15, 2020):

I am having the same issue, docker ps returns

CONTAINER ID        IMAGE                          COMMAND             CREATED             STATUS                   PORTS                            NAMES

aa4c9a762ee1        bitwardenrs/server:latest      "/bitwarden_rs"     12 days ago         Up 12 days (unhealthy)   3012/tcp, 0.0.0.0:8080->80/tcp   bitwarden

running curl -v http://localhost:80/alive inside container gives

* Expire in 0 ms for 6 (transfer 0x55bb32ccef50)
* Expire in 1 ms for 1 (transfer 0x55bb32ccef50)
* Expire in 0 ms for 1 (transfer 0x55bb32ccef50)
* Expire in 2 ms for 1 (transfer 0x55bb32ccef50)
* Expire in 0 ms for 1 (transfer 0x55bb32ccef50)
* Expire in 0 ms for 1 (transfer 0x55bb32ccef50)
* Expire in 2 ms for 1 (transfer 0x55bb32ccef50)
* Expire in 1 ms for 1 (transfer 0x55bb32ccef50)
* Expire in 1 ms for 1 (transfer 0x55bb32ccef50)
* Expire in 1 ms for 1 (transfer 0x55bb32ccef50)
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Expire in 149997 ms for 3 (transfer 0x55bb32ccef50)
* Expire in 200 ms for 4 (transfer 0x55bb32ccef50)
*   Trying ::1...
* TCP_NODELAY set
* Expire in 149997 ms for 3 (transfer 0x55bb32ccef50)
* Immediate connect fail for ::1: Cannot assign requested address
*   Trying ::1...
* TCP_NODELAY set
....

running ./healthcheck.sh inside container just stuck, not printing anything for a long time

so I ran sh -x healthcheck.sh and it's stopping at

+ : data
+ : 80
+ CONFIG_FILE=data/config.json
+ [ -r data/config.json ]
+ get_base_path 
+ sed -e s|.*://|| -e s|[^/]\+|| -e s|/*$||
+ echo 
+ base_path=
+ [ -n  ]
+ curl --insecure --fail --silent --show-error http://localhost:80/alive

Is restarting the container a solution? my site has stopped responding at all let me know what should be done here I am not much experienced with docker

<!-- gh-comment-id:629121077 --> @Sparkenstein commented on GitHub (May 15, 2020): I am having the same issue, `docker ps` returns ```sh CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES aa4c9a762ee1 bitwardenrs/server:latest "/bitwarden_rs" 12 days ago Up 12 days (unhealthy) 3012/tcp, 0.0.0.0:8080->80/tcp bitwarden ``` running `curl -v http://localhost:80/alive` inside container gives ```sh * Expire in 0 ms for 6 (transfer 0x55bb32ccef50) * Expire in 1 ms for 1 (transfer 0x55bb32ccef50) * Expire in 0 ms for 1 (transfer 0x55bb32ccef50) * Expire in 2 ms for 1 (transfer 0x55bb32ccef50) * Expire in 0 ms for 1 (transfer 0x55bb32ccef50) * Expire in 0 ms for 1 (transfer 0x55bb32ccef50) * Expire in 2 ms for 1 (transfer 0x55bb32ccef50) * Expire in 1 ms for 1 (transfer 0x55bb32ccef50) * Expire in 1 ms for 1 (transfer 0x55bb32ccef50) * Expire in 1 ms for 1 (transfer 0x55bb32ccef50) * Trying 127.0.0.1... * TCP_NODELAY set * Expire in 149997 ms for 3 (transfer 0x55bb32ccef50) * Expire in 200 ms for 4 (transfer 0x55bb32ccef50) * Trying ::1... * TCP_NODELAY set * Expire in 149997 ms for 3 (transfer 0x55bb32ccef50) * Immediate connect fail for ::1: Cannot assign requested address * Trying ::1... * TCP_NODELAY set .... ``` running `./healthcheck.sh` inside container just stuck, not printing anything for a long time so I ran `sh -x healthcheck.sh ` and it's stopping at ```sh + : data + : 80 + CONFIG_FILE=data/config.json + [ -r data/config.json ] + get_base_path + sed -e s|.*://|| -e s|[^/]\+|| -e s|/*$|| + echo + base_path= + [ -n ] + curl --insecure --fail --silent --show-error http://localhost:80/alive ``` Is restarting the container a solution? my site has stopped responding at all let me know what should be done here I am not much experienced with docker
Author
Owner

@Sparkenstein commented on GitHub (May 15, 2020):

I believe the site is up as of now, I ran docker restart bitwarden and it looks healthy. Let me know if you need any inputs from me if or when it goes down next time. I will make sure to capture it

<!-- gh-comment-id:629126101 --> @Sparkenstein commented on GitHub (May 15, 2020): I believe the site is up as of now, I ran `docker restart bitwarden` and it looks healthy. Let me know if you need any inputs from me if or when it goes down next time. I will make sure to capture it
Author
Owner

@jjlin commented on GitHub (May 15, 2020):

@Sparkenstein See https://github.com/dani-garcia/bitwarden_rs/issues/950#issuecomment-612346331

<!-- gh-comment-id:629488866 --> @jjlin commented on GitHub (May 15, 2020): @Sparkenstein See https://github.com/dani-garcia/bitwarden_rs/issues/950#issuecomment-612346331
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/vaultwarden#552
No description provided.