[GH-ISSUE #866] E-mail sending does not work if server (postfix) uses ECDSA certificate #604

Closed
opened 2026-02-25 23:42:59 +03:00 by kerem · 7 comments
Owner

Originally created by @eddyJK on GitHub (Jul 26, 2023).
Original GitHub issue: https://github.com/healthchecks/healthchecks/issues/866

Dear Healthchecks Developer Team and GitHub Community,

to reproduce the issue you need a mailserver using only ECDSA based certificates for encryption and Healthchecks docker.
I tested it with the Healtchchecks versions 2.4 and 2.10.
On the other hand the configuration works for the same mailserver unsing only RSA based certificates for encryption.

The following output appears:

web_1  | [pid: 10|app: 0|req: 465/1857] 192.168.2.2 () {54 vars in 1061 bytes} [Wed Jul 26 07:10:29 2023] GET / => generated 0 bytes in 2 msecs (HTTP/1.1 302) 8 headers i
n 250 bytes (1 switches on core 0)
web_1  | [pid: 12|app: 0|req: 465/1858] 192.168.2.2 () {54 vars in 1091 bytes} [Wed Jul 26 07:10:29 2023] GET /accounts/login/ => generated 2056 bytes in 10 msecs (HTTP/1
.1 200) 9 headers in 399 bytes (1 switches on core 0)
web_1  | [pid: 11|app: 0|req: 465/1859] 192.168.2.2 () {64 vars in 1319 bytes} [Wed Jul 26 07:10:35 2023] POST /accounts/login/ => generated 0 bytes in 643 msecs (HTTP/1.
1 302) 9 headers in 371 bytes (1 switches on core 0)
web_1  | [pid: 13|app: 0|req: 465/1860] 192.168.2.2 () {58 vars in 1231 bytes} [Wed Jul 26 07:10:36 2023] GET /accounts/login_link_sent/ => generated 1190 bytes in 10 mse
cs (HTTP/1.1 200) 8 headers in 263 bytes (1 switches on core 0)
web_1  | Exception in thread Thread-2:
web_1  | Traceback (most recent call last):
web_1  |   File "/usr/local/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
web_1  |     self.run()
web_1  |   File "/opt/healthchecks/hc/lib/emails.py", line 25, in run
web_1  |     self.message.send()
web_1  |   File "/usr/local/lib/python3.11/site-packages/django/core/mail/message.py", line 298, in send
web_1  |     return self.get_connection(fail_silently).send_messages([self])
web_1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
web_1  |   File "/usr/local/lib/python3.11/site-packages/django/core/mail/backends/smtp.py", line 127, in send_messages
web_1  |     new_conn_created = self.open()
web_1  |                        ^^^^^^^^^^^
web_1  |   File "/usr/local/lib/python3.11/site-packages/django/core/mail/backends/smtp.py", line 92, in open
web_1  |     self.connection.starttls(context=self.ssl_context)
web_1  |   File "/usr/local/lib/python3.11/smtplib.py", line 790, in starttls
web_1  |     self.sock = context.wrap_socket(self.sock,
web_1  |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
web_1  |   File "/usr/local/lib/python3.11/ssl.py", line 517, in wrap_socket
web_1  |     return self.sslsocket_class._create(
web_1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
web_1  |   File "/usr/local/lib/python3.11/ssl.py", line 1075, in _create
web_1  |     self.do_handshake()
web_1  |   File "/usr/local/lib/python3.11/ssl.py", line 1346, in do_handshake
web_1  |     self._sslobj.do_handshake()
web_1  | ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002)
web_1  | [pid: 10|app: 0|req: 466/1861] 192.168.2.2 () {34 vars in 566 bytes} [Wed Jul 26 07:10:37 2023] GET / => generated 0 bytes in 2 msecs (HTTP/1.1 302) 8 headers in
 250 bytes (1 switches on core 0)
web_1  | [pid: 12|app: 0|req: 466/1862] 192.168.2.2 () {34 vars in 596 bytes} [Wed Jul 26 07:10:37 2023] GET /accounts/login/ => generated 8087 bytes in 8 msecs (HTTP/1.1
 200) 8 headers in 375 bytes (1 switches on core 0)

I am looking forward to discuss possible fixes and / or workarounds.

Originally created by @eddyJK on GitHub (Jul 26, 2023). Original GitHub issue: https://github.com/healthchecks/healthchecks/issues/866 Dear Healthchecks Developer Team and GitHub Community, to reproduce the issue you need a mailserver using only ECDSA based certificates for encryption and Healthchecks docker. I tested it with the Healtchchecks versions 2.4 and 2.10. On the other hand the configuration works for the same mailserver unsing only RSA based certificates for encryption. The following output appears: ``` web_1 | [pid: 10|app: 0|req: 465/1857] 192.168.2.2 () {54 vars in 1061 bytes} [Wed Jul 26 07:10:29 2023] GET / => generated 0 bytes in 2 msecs (HTTP/1.1 302) 8 headers i n 250 bytes (1 switches on core 0) web_1 | [pid: 12|app: 0|req: 465/1858] 192.168.2.2 () {54 vars in 1091 bytes} [Wed Jul 26 07:10:29 2023] GET /accounts/login/ => generated 2056 bytes in 10 msecs (HTTP/1 .1 200) 9 headers in 399 bytes (1 switches on core 0) web_1 | [pid: 11|app: 0|req: 465/1859] 192.168.2.2 () {64 vars in 1319 bytes} [Wed Jul 26 07:10:35 2023] POST /accounts/login/ => generated 0 bytes in 643 msecs (HTTP/1. 1 302) 9 headers in 371 bytes (1 switches on core 0) web_1 | [pid: 13|app: 0|req: 465/1860] 192.168.2.2 () {58 vars in 1231 bytes} [Wed Jul 26 07:10:36 2023] GET /accounts/login_link_sent/ => generated 1190 bytes in 10 mse cs (HTTP/1.1 200) 8 headers in 263 bytes (1 switches on core 0) web_1 | Exception in thread Thread-2: web_1 | Traceback (most recent call last): web_1 | File "/usr/local/lib/python3.11/threading.py", line 1038, in _bootstrap_inner web_1 | self.run() web_1 | File "/opt/healthchecks/hc/lib/emails.py", line 25, in run web_1 | self.message.send() web_1 | File "/usr/local/lib/python3.11/site-packages/django/core/mail/message.py", line 298, in send web_1 | return self.get_connection(fail_silently).send_messages([self]) web_1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ web_1 | File "/usr/local/lib/python3.11/site-packages/django/core/mail/backends/smtp.py", line 127, in send_messages web_1 | new_conn_created = self.open() web_1 | ^^^^^^^^^^^ web_1 | File "/usr/local/lib/python3.11/site-packages/django/core/mail/backends/smtp.py", line 92, in open web_1 | self.connection.starttls(context=self.ssl_context) web_1 | File "/usr/local/lib/python3.11/smtplib.py", line 790, in starttls web_1 | self.sock = context.wrap_socket(self.sock, web_1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ web_1 | File "/usr/local/lib/python3.11/ssl.py", line 517, in wrap_socket web_1 | return self.sslsocket_class._create( web_1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ web_1 | File "/usr/local/lib/python3.11/ssl.py", line 1075, in _create web_1 | self.do_handshake() web_1 | File "/usr/local/lib/python3.11/ssl.py", line 1346, in do_handshake web_1 | self._sslobj.do_handshake() web_1 | ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002) web_1 | [pid: 10|app: 0|req: 466/1861] 192.168.2.2 () {34 vars in 566 bytes} [Wed Jul 26 07:10:37 2023] GET / => generated 0 bytes in 2 msecs (HTTP/1.1 302) 8 headers in 250 bytes (1 switches on core 0) web_1 | [pid: 12|app: 0|req: 466/1862] 192.168.2.2 () {34 vars in 596 bytes} [Wed Jul 26 07:10:37 2023] GET /accounts/login/ => generated 8087 bytes in 8 msecs (HTTP/1.1 200) 8 headers in 375 bytes (1 switches on core 0) ``` I am looking forward to discuss possible fixes and / or workarounds.
kerem closed this issue 2026-02-25 23:42:59 +03:00
Author
Owner

@cuu508 commented on GitHub (Jul 28, 2023):

To help me reproduce the issue – are you using LetsEncrypt-issued certificates?

<!-- gh-comment-id:1655188181 --> @cuu508 commented on GitHub (Jul 28, 2023): To help me reproduce the issue – are you using LetsEncrypt-issued certificates?
Author
Owner

@eddyJK commented on GitHub (Jul 28, 2023):

To help me reproduce the issue – are you using LetsEncrypt-issued certificates?

Yes. With the following config:

text = True
non-interactive = True
webroot-path = /data/letsencrypt-acme-challenge
key-type = ecdsa
elliptic-curve = secp384r1
preferred-chain = ISRG Root X1

<!-- gh-comment-id:1655410400 --> @eddyJK commented on GitHub (Jul 28, 2023): > To help me reproduce the issue – are you using LetsEncrypt-issued certificates? Yes. With the following config: text = True non-interactive = True webroot-path = /data/letsencrypt-acme-challenge key-type = ecdsa elliptic-curve = secp384r1 preferred-chain = ISRG Root X1
Author
Owner

@cuu508 commented on GitHub (Jul 31, 2023):

I haven't had luck reproducing this yet.

I found a random mail server on shodan that listens on port 587 (STARTTLS) and seems to be using ECC certificate.

I started a throwaway Healthchecks instance like so:

docker run --rm \
  --name=healthchecks \
  -p 8000:8000 \
  -e ALLOWED_HOSTS=localhost \
  -e DB=sqlite \
  -e DB_NAME=/data/hc.sqlite \
  -e DEBUG=False \
  -e DEFAULT_FROM_EMAIL=fixme-email-address-here \
  -e EMAIL_HOST=random-mailservers-hostname-here.com \
  -e EMAIL_HOST_PASSWORD=foo \
  -e EMAIL_HOST_USER=bar \
  -e EMAIL_PORT=587 \
  -e EMAIL_USE_TLS=True \
  -e SECRET_KEY=--- \
  -e SITE_ROOT=http://localhost:8000 \
  -v healthchecks-data:/data \
healthchecks/healthchecks:v2.10

In the web UI, I submitted the "Create Account" form which should trigger an outgoing email. The error I got was:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
    self.run()
  File "/opt/healthchecks/hc/lib/emails.py", line 25, in run
    self.message.send()
  File "/usr/local/lib/python3.11/site-packages/django/core/mail/message.py", line 298, in send
    return self.get_connection(fail_silently).send_messages([self])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/core/mail/backends/smtp.py", line 127, in send_messages
    new_conn_created = self.open()
                       ^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/core/mail/backends/smtp.py", line 94, in open
    self.connection.login(self.username, self.password)
  File "/usr/local/lib/python3.11/smtplib.py", line 750, in login
    raise last_exception
  File "/usr/local/lib/python3.11/smtplib.py", line 739, in login
    (code, resp) = self.auth(
                   ^^^^^^^^^^
  File "/usr/local/lib/python3.11/smtplib.py", line 662, in auth
    raise SMTPAuthenticationError(code, resp)
smtplib.SMTPAuthenticationError: (535, b'Incorrect authentication data')

From the error message it looks like it got past the TLS handshake, but the SMTP credentials were wrong – which makes sense.

Can you point me to a publicly available mail server that I can test with (don't need username/password, just the hostname), or provide instructions to reproduce the issue in some other form?

<!-- gh-comment-id:1657901428 --> @cuu508 commented on GitHub (Jul 31, 2023): I haven't had luck reproducing this yet. I found a random mail server on shodan that listens on port 587 (STARTTLS) and seems to be using ECC certificate. I started a throwaway Healthchecks instance like so: ``` docker run --rm \ --name=healthchecks \ -p 8000:8000 \ -e ALLOWED_HOSTS=localhost \ -e DB=sqlite \ -e DB_NAME=/data/hc.sqlite \ -e DEBUG=False \ -e DEFAULT_FROM_EMAIL=fixme-email-address-here \ -e EMAIL_HOST=random-mailservers-hostname-here.com \ -e EMAIL_HOST_PASSWORD=foo \ -e EMAIL_HOST_USER=bar \ -e EMAIL_PORT=587 \ -e EMAIL_USE_TLS=True \ -e SECRET_KEY=--- \ -e SITE_ROOT=http://localhost:8000 \ -v healthchecks-data:/data \ healthchecks/healthchecks:v2.10 ``` In the web UI, I submitted the "Create Account" form which should trigger an outgoing email. The error I got was: ``` Exception in thread Thread-1: Traceback (most recent call last): File "/usr/local/lib/python3.11/threading.py", line 1038, in _bootstrap_inner self.run() File "/opt/healthchecks/hc/lib/emails.py", line 25, in run self.message.send() File "/usr/local/lib/python3.11/site-packages/django/core/mail/message.py", line 298, in send return self.get_connection(fail_silently).send_messages([self]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/core/mail/backends/smtp.py", line 127, in send_messages new_conn_created = self.open() ^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/core/mail/backends/smtp.py", line 94, in open self.connection.login(self.username, self.password) File "/usr/local/lib/python3.11/smtplib.py", line 750, in login raise last_exception File "/usr/local/lib/python3.11/smtplib.py", line 739, in login (code, resp) = self.auth( ^^^^^^^^^^ File "/usr/local/lib/python3.11/smtplib.py", line 662, in auth raise SMTPAuthenticationError(code, resp) smtplib.SMTPAuthenticationError: (535, b'Incorrect authentication data') ``` From the error message it looks like it got past the TLS handshake, but the SMTP credentials were wrong – which makes sense. Can you point me to a publicly available mail server that I can test with (don't need username/password, just the hostname), or provide instructions to reproduce the issue in some other form?
Author
Owner

@eddyJK commented on GitHub (Jul 31, 2023):

You can use the following:

EMAIL_HOST=mail.tal-deloitte.de
EMAIL_HOST_PASSWORD=xxx
EMAIL_HOST_USER=test@tal-deloitte.de
EMAIL_PORT=587
EMAIL_USE_SSL=False
EMAIL_USE_TLS=True
EMAIL_USE_VERIFICATION=True
<!-- gh-comment-id:1658048794 --> @eddyJK commented on GitHub (Jul 31, 2023): You can use the following: ``` EMAIL_HOST=mail.tal-deloitte.de EMAIL_HOST_PASSWORD=xxx EMAIL_HOST_USER=test@tal-deloitte.de EMAIL_PORT=587 EMAIL_USE_SSL=False EMAIL_USE_TLS=True EMAIL_USE_VERIFICATION=True ```
Author
Owner

@eddyJK commented on GitHub (Jul 31, 2023):

Please apologize. You are right. The correct certificate was not presented.

How to use your own certificate
Make sure you disable mailcows internal LE client (see above).
To use your own certificates, just save the combined certificate (containing the certificate and intermediate CA/CA if any) to data/assets/ssl/cert.pem and the corresponding key to data/assets/ssl/key.pem.

I did not follow the whole instruction of the mail server.

A last question: Setting EMAIL_USE_VERIFICATION to False did not work as well. Should this option not disable the certificate check?

<!-- gh-comment-id:1658180775 --> @eddyJK commented on GitHub (Jul 31, 2023): Please apologize. You are right. The correct certificate was not presented. > How to use your own certificate Make sure you disable mailcows internal LE client (see above). To use your own certificates, just save the combined certificate (**containing the certificate and intermediate CA/CA if any**) to data/assets/ssl/cert.pem and the corresponding key to data/assets/ssl/key.pem. I did not follow the whole instruction of the mail server. A last question: Setting EMAIL_USE_VERIFICATION to _False_ did not work as well. Should this option not disable the certificate check?
Author
Owner

@cuu508 commented on GitHub (Jul 31, 2023):

Awesome, mystery solved :-)

EMAIL_USE_VERIFICATION controls whether Healthchecks sends an email with a confirmation link when adding an email integration. (See https://healthchecks.io/docs/self_hosted_configuration/#EMAIL_USE_VERIFICATION)

<!-- gh-comment-id:1658185991 --> @cuu508 commented on GitHub (Jul 31, 2023): Awesome, mystery solved :-) `EMAIL_USE_VERIFICATION` controls whether Healthchecks sends an email with a confirmation link when adding an email integration. (See https://healthchecks.io/docs/self_hosted_configuration/#EMAIL_USE_VERIFICATION)
Author
Owner

@eddyJK commented on GitHub (Aug 1, 2023):

Thank you again and I want to apologize for stealing your time.
Of course the deployment of the correct full chain certificate is already fixed.
At the moment I am investigating, why other web services did not alarm in this configuration.

<!-- gh-comment-id:1659660738 --> @eddyJK commented on GitHub (Aug 1, 2023): Thank you again and I want to apologize for stealing your time. Of course the deployment of the correct full chain certificate is already fixed. At the moment I am investigating, why other web services did not alarm in this configuration.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/healthchecks#604
No description provided.