[GH-ISSUE #1108] 3.8.2 stays unhealthy - same config working with 3.8.1 #770

New issue

Closed

opened 2026-02-25 23:43:31 +03:00 by kerem · 4 comments

kerem commented

2026-02-25 23:43:31 +03:00

Owner

Originally created by @BeyondVertical on GitHub (Dec 20, 2024).
Original GitHub issue: https://github.com/healthchecks/healthchecks/issues/1108

My Healthchecks-container was unhealthy today and I dug a little deeper:
Watchtower was updating it overnight from 3.8.1 to 3.8.2 and it could not get to run (status: unhealthy).
Here is my config from the docker compose, maybe you can see what the problem is. In the meantime I went back to 3.8.1.

    services:
      healthchecks-app:
        image: healthchecks/healthchecks
        container_name: healthchecks-app
        restart: always
        security_opt:
          - no-new-privileges:true
        command: bash -c 'while !</dev/tcp/healthchecks-db/5432; do sleep 1; done; uwsgi /opt/healthchecks/docker/uwsgi.ini'
        environment:
          - ADMINS=admin@domain.com
          - ALLOWED_HOSTS=domain.com
          - DB=postgres
          - DB_HOST=healthchecks-db
          - DB_PORT=5432
          - DB_NAME=healthchecks
          - DB_USER=postgres
          - DB_PASSWORD=my_db_password
          - DEBUG=False
          - DEFAULT_FROM_EMAIL=my_default_mail_user
          - EMAIL_HOST=smtp.maildomain.com
          - EMAIL_HOST_USER=my_mail_user
          - EMAIL_HOST_PASSWORD=my_mail_password
          - EMAIL_PORT=587
          - EMAIL_USE_TLS=True
          - EMAIL_USE_VERIFICATION=True
          - MATTERMOST_ENABLED=False
          - MSTEAMS_ENABLED=False
          - OPSGENIE_ENABLED=False
          - PAGERTREE_ENABLED=False
          - PD_ENABLED=False
          - PING_BODY_LIMIT=None
          - PROMETHEUS_ENABLED=False
          - REGISTRATION_OPEN=False
          - ROCKETCHAT_ENABLED=False
          - SECRET_KEY=my_super_secret_key
          - SITE_NAME=Healthchecks
          - SITE_ROOT=https://domain.com
          - SITE_LOGO_URL=https://otherdomain.com/img/logo.svg
          - SLACK_ENABLED=False
          - SPIKE_ENABLED=False
          - USE_PAYMENTS=False
          - VICTOROPS_ENABLED=False
          - WEBHOOKS_ENABLED=False
          - ZULIP_ENABLED=False
          - TZ=Europe/Berlin
        volumes:
          - /etc/localtime:/etc/localtime:ro
        networks:
          - proxy
          - default
        depends_on:
          - healthchecks-db

Originally created by @BeyondVertical on GitHub (Dec 20, 2024). Original GitHub issue: https://github.com/healthchecks/healthchecks/issues/1108 My Healthchecks-container was unhealthy today and I dug a little deeper: Watchtower was updating it overnight from 3.8.1 to 3.8.2 and it could not get to run (status: unhealthy). Here is my config from the docker compose, maybe you can see what the problem is. In the meantime I went back to 3.8.1. ``` services: healthchecks-app: image: healthchecks/healthchecks container_name: healthchecks-app restart: always security_opt: - no-new-privileges:true command: bash -c 'while !</dev/tcp/healthchecks-db/5432; do sleep 1; done; uwsgi /opt/healthchecks/docker/uwsgi.ini' environment: - ADMINS=admin@domain.com - ALLOWED_HOSTS=domain.com - DB=postgres - DB_HOST=healthchecks-db - DB_PORT=5432 - DB_NAME=healthchecks - DB_USER=postgres - DB_PASSWORD=my_db_password - DEBUG=False - DEFAULT_FROM_EMAIL=my_default_mail_user - EMAIL_HOST=smtp.maildomain.com - EMAIL_HOST_USER=my_mail_user - EMAIL_HOST_PASSWORD=my_mail_password - EMAIL_PORT=587 - EMAIL_USE_TLS=True - EMAIL_USE_VERIFICATION=True - MATTERMOST_ENABLED=False - MSTEAMS_ENABLED=False - OPSGENIE_ENABLED=False - PAGERTREE_ENABLED=False - PD_ENABLED=False - PING_BODY_LIMIT=None - PROMETHEUS_ENABLED=False - REGISTRATION_OPEN=False - ROCKETCHAT_ENABLED=False - SECRET_KEY=my_super_secret_key - SITE_NAME=Healthchecks - SITE_ROOT=https://domain.com - SITE_LOGO_URL=https://otherdomain.com/img/logo.svg - SLACK_ENABLED=False - SPIKE_ENABLED=False - USE_PAYMENTS=False - VICTOROPS_ENABLED=False - WEBHOOKS_ENABLED=False - ZULIP_ENABLED=False - TZ=Europe/Berlin volumes: - /etc/localtime:/etc/localtime:ro networks: - proxy - default depends_on: - healthchecks-db ```

kerem closed this issue

2026-02-25 23:43:31 +03:00

kerem commented

2026-02-25 23:43:32 +03:00

Author

Owner

@andreasfaerber commented on GitHub (Dec 20, 2024):

Same issue here. From health status (docker inspect --format "{{json .State.Health }}" container | jq):

{
"Status": "unhealthy",
"FailingStreak": 1,
"Log": [
{
"Start": "2024-12-20T09:56:30.309793605+01:00",
"End": "2024-12-20T09:56:30.582685056+01:00",
"ExitCode": 1,
"Output": "Traceback (most recent call last):\n File "/opt/healthchecks/./fetchstatus.py", line 34, in \n with urlopen(f"{SITE_ROOT}/api/v3/status/") as response:\n ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.13/urllib/request.py", line 189, in urlopen\n return opener.open(url, data, timeout)\n ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.13/urllib/request.py", line 495, in open\n response = meth(req, response)\n File "/usr/local/lib/python3.13/urllib/request.py", line 604, in http_response\n response = self.parent.error(\n 'http', request, response, code, msg, hdrs)\n File "/usr/local/lib/python3.13/urllib/request.py", line 533, in error\n return self._call_chain(*args)\n ~~~~~~~~~~~~~~~~^^^^^^^\n File "/usr/local/lib/python3.13/urllib/request.py", line 466, in _call_chain\n result = func(*args)\n File "/usr/local/lib/python3.13/urllib/request.py", line 613, in http_error_default\n raise HTTPError(req.full_url, code, msg, hdrs, fp)\nurllib.error.HTTPError: HTTP Error 404: Not Found\n"
}
]
}

I am behind traefik. Maybe trying to request something from itself via traefik while traefik did not pick up the container and will not as long as it's unhealthy?

@andreasfaerber commented on GitHub (Dec 20, 2024): Same issue here. From health status (docker inspect --format "{{json .State.Health }}" container | jq): { "Status": "unhealthy", "FailingStreak": 1, "Log": [ { "Start": "2024-12-20T09:56:30.309793605+01:00", "End": "2024-12-20T09:56:30.582685056+01:00", "ExitCode": 1, "Output": "Traceback (most recent call last):\n File \"/opt/healthchecks/./fetchstatus.py\", line 34, in <module>\n with urlopen(f\"{SITE_ROOT}/api/v3/status/\") as response:\n ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/usr/local/lib/python3.13/urllib/request.py\", line 189, in urlopen\n return opener.open(url, data, timeout)\n ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^\n File \"/usr/local/lib/python3.13/urllib/request.py\", line 495, in open\n response = meth(req, response)\n File \"/usr/local/lib/python3.13/urllib/request.py\", line 604, in http_response\n response = self.parent.error(\n 'http', request, response, code, msg, hdrs)\n File \"/usr/local/lib/python3.13/urllib/request.py\", line 533, in error\n return self._call_chain(*args)\n ~~~~~~~~~~~~~~~~^^^^^^^\n File \"/usr/local/lib/python3.13/urllib/request.py\", line 466, in _call_chain\n result = func(*args)\n File \"/usr/local/lib/python3.13/urllib/request.py\", line 613, in http_error_default\n raise HTTPError(req.full_url, code, msg, hdrs, fp)\nurllib.error.HTTPError: HTTP Error 404: Not Found\n" } ] } I am behind traefik. Maybe trying to request something from itself via traefik while traefik did not pick up the container and will not as long as it's unhealthy?

kerem commented

2026-02-25 23:43:32 +03:00

Author

Owner

@cuu508 commented on GitHub (Dec 20, 2024):

Maybe trying to request something from itself via traefik while traefik did not pick up the container and will not as long as it's unhealthy?

Yes, that looks like it. It's a bug by me, sorry. I'll have another go at fetchstatus.py.

@cuu508 commented on GitHub (Dec 20, 2024): > Maybe trying to request something from itself via traefik while traefik did not pick up the container and will not as long as it's unhealthy? Yes, that looks like it. It's a bug by me, sorry. I'll have another go at fetchstatus.py.

kerem commented

2026-02-25 23:43:32 +03:00

Author

Owner

@cuu508 commented on GitHub (Dec 20, 2024):

I've released v3.9 which should fix this.

@cuu508 commented on GitHub (Dec 20, 2024): I've released v3.9 which should fix this.

kerem commented

2026-02-25 23:43:32 +03:00

Author

Owner

@BeyondVertical commented on GitHub (Dec 22, 2024):

Yeah, working again. Thanks. The healthy-status is taking a little longer than expected, but it works.

@BeyondVertical commented on GitHub (Dec 22, 2024): Yeah, working again. Thanks. The healthy-status is taking a little longer than expected, but it works.

kerem referenced this issue

2026-02-26 00:30:39 +03:00

[PR #770] [CLOSED] Idempotence key #1021