[GH-ISSUE #1069] State reason for failure (late or explicit failure) in notifications #742

Closed
opened 2026-02-25 23:43:26 +03:00 by kerem · 5 comments
Owner

Originally created by @rwjack on GitHub (Oct 10, 2024).
Original GitHub issue: https://github.com/healthchecks/healthchecks/issues/1069

Related: https://github.com/healthchecks/healthchecks/discussions/1067

Originally created by @rwjack on GitHub (Oct 10, 2024). Original GitHub issue: https://github.com/healthchecks/healthchecks/issues/1069 Related: https://github.com/healthchecks/healthchecks/discussions/1067
kerem 2026-02-25 23:43:26 +03:00
  • closed this issue
  • added the
    feature
    label
Author
Owner

@cuu508 commented on GitHub (Nov 8, 2024):

I updated the Flip object to track failure reason, and updated the email template. When the grace time runs out:

image

When the check receives a failure signal:

image

@rwjack, @davidkcarey thoughts on this, does this help?

Changing the contents of the subject line is risky, as people may already have email filters set up that look for the "DOWN" keyword.

I thought about adding a more verbose paragraph explaining why a check went down. Something like:

Downtime Reason
The period of this check is 10 minutes and the grace time is 5 minutes.
The last success signal was at Nov 8, 06:50.
The next success signal was expected at Nov 8, 07:00.
There was no success signal by Nov 8, 07:05, so this check has changed status to DOWN.

And even fancier way would be to show a visual timeline of (non-)events:

image

Problems with this:

  • Need to track more information (as minimum, one additional date)
  • The email notification gets longer. It gets harder to find information that the user is looking for.
  • The visual timeline would be a PITA to implement given HTML email limitations
<!-- gh-comment-id:2464264956 --> @cuu508 commented on GitHub (Nov 8, 2024): I updated the Flip object to track failure reason, and updated the email template. When the grace time runs out: ![image](https://github.com/user-attachments/assets/aa3b1527-509e-4225-8156-94512c339ee6) When the check receives a failure signal: ![image](https://github.com/user-attachments/assets/306fa36d-093a-45b6-b695-70e1a10baab9) @rwjack, @davidkcarey thoughts on this, does this help? Changing the contents of the subject line is risky, as people may already have email filters set up that look for the "DOWN" keyword. I thought about adding a more verbose paragraph explaining why a check went down. Something like: > **Downtime Reason** > The period of this check is **10 minutes** and the grace time is **5 minutes**. > The last success signal was at **Nov 8, 06:50**. > The next success signal was expected at **Nov 8, 07:00**. > There was no success signal by Nov 8, 07:05, so this check has changed status to **DOWN**. And even fancier way would be to show a visual timeline of (non-)events: ![image](https://github.com/user-attachments/assets/06b5ffbe-a33e-48cb-93cd-c410330aa000) Problems with this: * Need to track more information (as minimum, one additional date) * The email notification gets longer. It gets harder to find information that the user is looking for. * The visual timeline would be a PITA to implement given HTML email limitations
Author
Owner

@rwjack commented on GitHub (Dec 1, 2024):

Excuse the late reply, been busy lately.

Yeah, I think your implementation would solve the original issue, in the email notification description at least. That's where verbose paragraph would come into play, but for existing simple notifications (I use matrix): [STATE] - check name, something like [STATE] [MISSED GRACE] - check name would make more sense.

Although it might be a better idea to just update the matrix template to include data from your 2nd screenshot. Then the verbose paragraph solution would solve it everywhere.

Also, seeing your 3rd image of the timeline, upped the initial idea, and it really would look great. Not necessarily in notifications, but on the webapp itself.

<!-- gh-comment-id:2509789025 --> @rwjack commented on GitHub (Dec 1, 2024): Excuse the late reply, been busy lately. Yeah, I think your implementation would solve the original issue, in the email notification description at least. That's where verbose paragraph would come into play, but for existing simple notifications (I use matrix): `[STATE] - check name`, something like `[STATE] [MISSED GRACE] - check name` would make more sense. Although it might be a better idea to just update the matrix template to include data from your 2nd screenshot. Then the verbose paragraph solution would solve it everywhere. Also, seeing your 3rd image of the timeline, upped the initial idea, and it really would look great. Not necessarily in notifications, but on the webapp itself.
Author
Owner

@cuu508 commented on GitHub (Dec 13, 2024):

@rwjack, by the verbose paragraph do you mean these?

"{check_name}" is DOWN (success signal did not arrive on time, grace time passed).

and

"{check_name}" is DOWN (received a failure signal).

I started with the email integration as it is the most used one, but once we settle on a pattern I'm going to gradually update all other integrations to use the same or a similar pattern.

The issue I see with [STATE] [MISSED GRACE] - check name is for some users it would be unclear what "MISSED GRACE" means. You know it, and from the context in this issue I know it, but for a new user it could be confusing.

<!-- gh-comment-id:2541002348 --> @cuu508 commented on GitHub (Dec 13, 2024): @rwjack, by the verbose paragraph do you mean these? > "{check_name}" is DOWN (success signal did not arrive on time, grace time passed). and > "{check_name}" is DOWN (received a failure signal). I started with the email integration as it is the most used one, but once we settle on a pattern I'm going to gradually update all other integrations to use the same or a similar pattern. The issue I see with `[STATE] [MISSED GRACE] - check name` is for some users it would be unclear what "MISSED GRACE" means. You know it, and from the context in this issue I know it, but for a new user it could be confusing.
Author
Owner

@cuu508 commented on GitHub (Dec 13, 2024):

Matrix messages support fair few HTML tags and attributes. I updated the message template to be similar to Signal, Telegram templates, and to include tags, schedule, last ping body, a list of other checks that are down.

Here's how it looks now:

image

I couldn't get two-column layout as in Slack and email messages. There are table tags, but Element renders the tables with borders and alternating row background colors, so not useful for layout:

image

<!-- gh-comment-id:2541257227 --> @cuu508 commented on GitHub (Dec 13, 2024): Matrix messages support fair few HTML tags and attributes. I updated the message template to be similar to Signal, Telegram templates, and to include tags, schedule, last ping body, a list of other checks that are down. Here's how it looks now: ![image](https://github.com/user-attachments/assets/04134119-cbbb-4413-90bd-9653c67a77ea) I couldn't get two-column layout as in Slack and email messages. There are table tags, but Element renders the tables with borders and alternating row background colors, so not useful for layout: ![image](https://github.com/user-attachments/assets/b36891fb-f249-4c05-8d29-86a30817a07f)
Author
Owner

@rwjack commented on GitHub (Dec 15, 2024):

The matrix template is perfect now, that should be all!

The issue I see with [STATE] [MISSED GRACE] - check name is for some users it would be unclear what "MISSED GRACE" means. You know it, and from the context in this issue I know it, but for a new user it could be confusing.

I get what you're saying.

By verbose paragraph, I mean the whole new updated email template:

image

<!-- gh-comment-id:2543929460 --> @rwjack commented on GitHub (Dec 15, 2024): The matrix template is perfect now, that should be all! > The issue I see with [STATE] [MISSED GRACE] - check name is for some users it would be unclear what "MISSED GRACE" means. You know it, and from the context in this issue I know it, but for a new user it could be confusing. I get what you're saying. By verbose paragraph, I mean the whole new updated email template: ![image](https://github.com/user-attachments/assets/c86ced09-4294-4ff8-9eee-c955865e7a66)
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/healthchecks#742
No description provided.