[GH-ISSUE #923] Fallback notifications #650

Open
opened 2026-02-25 23:43:09 +03:00 by kerem · 8 comments
Owner

Originally created by @rwjack on GitHub (Dec 6, 2023).
Original GitHub issue: https://github.com/healthchecks/healthchecks/issues/923

I use Matrix and Email notifications for a few checks, although I realized I don't really need both at the same time. I only enabled Email since it is "more mature" than Matrix, in case something's off with my Matrix server.

The ideal solution would be to have a separate option for fallback notifications, which would be triggered/sent only if the primary notification method fails.

Originally created by @rwjack on GitHub (Dec 6, 2023). Original GitHub issue: https://github.com/healthchecks/healthchecks/issues/923 I use Matrix and Email notifications for a few checks, although I realized I don't really need both at the same time. I only enabled Email since it is "more mature" than Matrix, in case something's off with my Matrix server. The ideal solution would be to have a separate option for fallback notifications, which would be triggered/sent only if the primary notification method fails.
Author
Owner

@cuu508 commented on GitHub (Dec 7, 2023):

I'd like to keep things as simple as possible and avoid a configuration UI where the user can say "deliver to channel A. If delivery fails, deliver to B."

But perhaps we could have a fixed, predetermined fallback: when we detect a delivery failure, inform the account owner via email about it.

We already do this for Signal notifications when Signal rejects our message due to rate-limiting on their end. We send a message to the account owner saying:

We could not deliver a Signal notification to <phone-number> due to
hitting a rate-limit on the Signal network, so we are sending it via email
instead:

***
<original notification's text here>
***

We could perhaps do something similar for other types of Signal failures, and for other channel types too.

<!-- gh-comment-id:1845432534 --> @cuu508 commented on GitHub (Dec 7, 2023): I'd like to keep things as simple as possible and avoid a configuration UI where the user can say "deliver to channel A. If delivery fails, deliver to B." But perhaps we could have a fixed, predetermined fallback: when we detect a delivery failure, inform the account owner via email about it. We already do this for Signal notifications when Signal rejects our message due to rate-limiting on their end. We send a message to the account owner saying: ``` We could not deliver a Signal notification to <phone-number> due to hitting a rate-limit on the Signal network, so we are sending it via email instead: *** <original notification's text here> *** ``` We could perhaps do something similar for other types of Signal failures, and for other channel types too.
Author
Owner

@rwjack commented on GitHub (Dec 7, 2023):

I'd like to keep things as simple as possible and avoid a configuration UI where the user can say "deliver to channel A. If delivery fails, deliver to B."

That makes sense.

But perhaps we could have a fixed, predetermined fallback: when we detect a delivery failure, inform the account owner via email about it.

For my use case, this would work, though having the ability to chose any notification method as the predetermined fallback seems like pure bonus cutomizability.

<!-- gh-comment-id:1845543232 --> @rwjack commented on GitHub (Dec 7, 2023): > I'd like to keep things as simple as possible and avoid a configuration UI where the user can say "deliver to channel A. If delivery fails, deliver to B." That makes sense. > But perhaps we could have a fixed, predetermined fallback: when we detect a delivery failure, inform the account owner via email about it. For my use case, this would work, though having the ability to chose any notification method as the predetermined fallback seems like pure bonus cutomizability.
Author
Owner

@rebmcr commented on GitHub (Feb 16, 2024):

We could not deliver a Signal notification to due to
hitting a rate-limit on the Signal network, so we are sending it via email
instead:

@cuu508 These Signal rate-limits appear to be permanently imposed on new integrations (unchanged since at least Tuesday this week), is there any further insight into a solution beyond the blog post (which would seem to be outdated at this point)?

<!-- gh-comment-id:1947747241 --> @rebmcr commented on GitHub (Feb 16, 2024): > We could not deliver a Signal notification to <phone-number> due to > hitting a rate-limit on the Signal network, so we are sending it via email > instead: @cuu508 These Signal rate-limits appear to be permanently imposed on new integrations (unchanged since at least Tuesday this week), is there any further insight into a solution beyond the [blog post](https://blog.healthchecks.io/2023/01/how-healthchecks-sends-signal-notifications/) (which would seem to be outdated at this point)?
Author
Owner

@cuu508 commented on GitHub (Feb 16, 2024):

@rebmcr since the blog post, I have not figured anything new on how to work around the Signal service's rate limits :-/

Can you please try the following:

  • Click the "Test!" button next to the Signal integration in your Healthchecks account. Make sure you get a delivery failure, and a fallback email message (i.e., you can reproduce the problem).
  • Then, start a chat with the Healthchecks account (+1 202-793-8823), and send a dummy "Hello!" message
  • Try clicking "Test!" again

And another thing to test:

  • Add the Healthchecks phone number to your phone contacts
  • Try clicking "Test!" again and see if it works this time

When I view the Healthchecks account in the Signal app, I see it is marked as "Signal Connection". My guess is the being or not being a Signal Connection might influence the rate limiting logic.

image

<!-- gh-comment-id:1947967693 --> @cuu508 commented on GitHub (Feb 16, 2024): @rebmcr since the blog post, I have not figured anything new on how to work around the Signal service's rate limits :-/ Can you please try the following: * Click the "Test!" button next to the Signal integration in your Healthchecks account. Make sure you get a delivery failure, and a fallback email message (i.e., you can reproduce the problem). * Then, start a chat with the Healthchecks account (**+1 202-793-8823**), and send a dummy "Hello!" message * Try clicking "Test!" again And another thing to test: * Add the Healthchecks phone number to your phone contacts * Try clicking "Test!" again and see if it works this time When I view the Healthchecks account in the Signal app, I see it is marked as "Signal Connection". My guess is the being or not being a Signal Connection might influence the rate limiting logic. ![image](https://github.com/healthchecks/healthchecks/assets/661859/580a8f45-9f90-40a0-aeed-e6405892262f)
Author
Owner

@rebmcr commented on GitHub (Feb 17, 2024):

Click the "Test!" button next to the Signal integration in your Healthchecks account. Make sure you get a delivery failure, and a fallback email message (i.e., you can reproduce the problem).

I can't do this, because the safety feature means I cannot create the integration at all. (But the test function seems to be as described, screenshot attached)
healthchecks-signal

Then, start a chat with the Healthchecks account (+1 202-793-8823), and send a dummy "Hello!" message
Try clicking "Test!" again

I have done both of these (days ago) but to no success.

I get the exact same screenshots as you have posted, when I check the account screen on Android.

<!-- gh-comment-id:1949663814 --> @rebmcr commented on GitHub (Feb 17, 2024): > Click the "Test!" button next to the Signal integration in your Healthchecks account. Make sure you get a delivery failure, and a fallback email message (i.e., you can reproduce the problem). I can't do this, because the safety feature means I cannot create the integration at all. (But the test function seems to be as described, screenshot attached) ![healthchecks-signal](https://github.com/healthchecks/healthchecks/assets/50411833/e2fc197c-5707-4b63-91d6-c13087a5ffb3) > Then, start a chat with the Healthchecks account (+1 202-793-8823), and send a dummy "Hello!" message > Try clicking "Test!" again I have done both of these (days ago) but to no success. I get the **_exact_** same screenshots as you have posted, when I check the account screen on Android.
Author
Owner

@cuu508 commented on GitHub (Feb 17, 2024):

@rebmcr can you please share the phone number privately with me (send it to contact@healthchecks.io)? I'll poke around with sending Signal notifications to it (and you may get a test notification if it works out).

<!-- gh-comment-id:1949895443 --> @cuu508 commented on GitHub (Feb 17, 2024): @rebmcr can you please share the phone number privately with me (send it to contact@healthchecks.io)? I'll poke around with sending Signal notifications to it (and you may get a test notification if it works out).
Author
Owner

@rebmcr commented on GitHub (Feb 17, 2024):

@rebmcr can you please share the phone number privately with me (send it to contact@healthchecks.io)? I'll poke around with sending Signal notifications to it (and you may get a test notification if it works out).

Done! Thanks.

<!-- gh-comment-id:1949896620 --> @rebmcr commented on GitHub (Feb 17, 2024): > @rebmcr can you please share the phone number privately with me (send it to [contact@healthchecks.io](mailto:contact@healthchecks.io))? I'll poke around with sending Signal notifications to it (and you may get a test notification if it works out). Done! Thanks.
Author
Owner

@fightingdreamer commented on GitHub (Jul 30, 2024):

I also have problem with passing the rate-limit test in Signal integration:

  • contact is marked as Signal Connector
  • I already sent few messages to Signal account number
<!-- gh-comment-id:2259319494 --> @fightingdreamer commented on GitHub (Jul 30, 2024): I also have problem with passing the rate-limit test in Signal integration: - contact is marked as Signal Connector - I already sent few messages to Signal account number
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/healthchecks#650
No description provided.