[GH-ISSUE #166] Make this easier to run automate integration tests

kerem commented

2026-03-15 12:36:24 +03:00

Owner

Originally created by @kamontat on GitHub (Sep 19, 2023).
Original GitHub issue: https://github.com/axllent/mailpit/issues/166

Current UI is a bit hard to run automate integration test (e.g. cypress or playwright).

My suggestion:

Add data-testid

Originally created by @kamontat on GitHub (Sep 19, 2023). Original GitHub issue: https://github.com/axllent/mailpit/issues/166 Current UI is a bit hard to run automate integration test (e.g. cypress or playwright). My suggestion: 1. Add data-testid

kerem

2026-03-15 12:36:24 +03:00

closed this issue
added the
enhancement
label

kerem commented

2026-03-15 12:36:40 +03:00

Author

Owner

@axllent commented on GitHub (Sep 19, 2023):

I find this very interesting as I have never considered the web UI to be a method of integration testing. The UI is just an interface/ to view the data. You appear to be building a tool to test another tool to test the data...

First off, I would strongly suggest you use the API directly for integration testing, not the web UI. It is stable, structured and documented. The web UI is prone to change at any point, in fact I am completely rebuilding it at the moment, and while it may appear very similar to the current layout, it will work quite differently in many areas.

What kind of integration testing are you doing exactly, and is there any specific reason you require the web UI to do this? It just doesn't seem the right approach to me.

@axllent commented on GitHub (Sep 19, 2023): I find this very interesting as I have never considered the web UI to be a method of integration testing. The UI is just an interface/ to view the data. You appear to be building a tool to test another tool to test the data... First off, I would strongly suggest you use the API directly for integration testing, not the web UI. It is stable, structured and documented. The web UI is prone to change at any point, in fact I am completely rebuilding it at the moment, and while it may appear very similar to the current layout, it will work quite differently in many areas. What kind of integration testing are you doing exactly, and is there any specific reason you require the web UI to do this? It just doesn't seem the right approach to me.

kerem commented

2026-03-15 12:36:45 +03:00

Author

Owner

@crdoconnor commented on GitHub (Sep 24, 2023):

I've been using the web UI for integration testing also. It works pretty well. I wouldn't be bothered if mixed the UI up a bit, but it would be really nice if I could navigate to a URL that:

A) shows only the email and not the surrounding UI.
B) Let me navigate directly to the latest email with /latest or something rather than having to do an API to get the ID and plugging that into a URL.

I do this so that I can:

Click on links in the email using selectors in the email.
Do the same to grab codes - using HTML selectors.
Use playwright to take screenshots and do snapshot testing the same way I do with the rest of the web app.

The alternative would be to use the API to pull the HTML and then stick that in a folder and then set up a separate web server to display that but I'd rather have this.

Thank you for writing such an amazing app by the way. I really like the link check and HTML check apps they are super useful. This definitely the best app in its class.

@crdoconnor commented on GitHub (Sep 24, 2023): I've been using the web UI for integration testing also. It works pretty well. I wouldn't be bothered if mixed the UI up a bit, but it would be really nice if I could navigate to a URL that: A) shows only the email and not the surrounding UI. B) Let me navigate directly to the latest email with /latest or something rather than having to do an API to get the ID and plugging that into a URL. I do this so that I can: * Click on links in the email using selectors in the email. * Do the same to grab codes - using HTML selectors. * Use playwright to take screenshots and do snapshot testing the same way I do with the rest of the web app. The alternative would be to use the API to pull the HTML and then stick that in a folder and then set up a separate web server to display that but I'd rather have this. Thank you for writing such an amazing app by the way. I *really* like the link check and HTML check apps they are super useful. This definitely the best app in its class.

kerem commented

2026-03-15 12:36:50 +03:00

Author

Owner

@axllent commented on GitHub (Sep 24, 2023):

@crdoconnor Thanks for your input and kind words. As I mentioned in my previous comment, I am not keen on customising Mailpit's web UI to try suit frontend testing, as this then becomes a commitment to maintaining a consistent UI in the same manner as an API, which then prevents me from developing the UI and adding new features without the continuos risk of breaking someone's integration. Just to state it clearly: I cannot stop developers from using the web UI for "automated frontend testing", and if things break in automated testing (using the web UI) because I added a new tag/button/design element, then that's really not my problem.

But, in saying that, if there are things I can do to make it easier for developers such as yourself to build things on top of Mailpit, then I am eager to discuss ideas. This could include things like data-testid attributes to main HTML elements. So let's go through your points so I can better understand your requirements.

A) shows only the email and not the surrounding UI.

What do you mean exactly with "the email", and what would be the reasoning for this exactly? Surely using HTML selectors you would be able to target the message section in the page without requiring a specific dedicated frontend component & route to just show "the email"?

B) Let me navigate directly to the latest email with /latest or something rather than having to do an API to get the ID and plugging that into a URL.

I understand your reasoning here. What if /latest was a 302 redirect to the latest message, so for example /latest would be redirected to /view/123-asd-465-fdfdg?

The alternative would be to use the API to pull the HTML and then stick that in a folder and then set up a separate web server to display that but I'd rather have this.

Well, it's possibly not that simple, at least not in a reliable way. The HTML returned from the email (assuming it has HTML) is not necessarily the same as what is displayed in the web UI. Email HTML can contain all sorts of things, including references to inline (attached) images (eg: <img src="cid:image001.gif@01D9EA83.5AABBC80">), removal of HTML <base> headers, etc. Mailpit's JavaScript frontend manipulates this to work in the frontend, for example "translating" (rewriting) inline image src paths. So if you simply extracted the original HTML (returned by the API) and saved that as a HTML file, it may not work as you expect - depending on what HTML you have. If you're referring specifically to the HTML part of an email then you may have more luck extracting the iframe's src-data after it has rendered on the frontend.

@axllent commented on GitHub (Sep 24, 2023): @crdoconnor Thanks for your input and kind words. As I mentioned in my previous comment, I am not keen on customising Mailpit's web UI to try suit frontend testing, as this then becomes a commitment to maintaining a consistent UI in the same manner as an API, which then prevents me from developing the UI and adding new features without the continuos risk of breaking someone's integration. Just to state it clearly: **I cannot stop developers from using the web UI for "automated frontend testing", and if things break in automated testing (using the web UI) because I added a new tag/button/design element, then that's really not my problem**. But, in saying that, if there are things I can do to make it _easier_ for developers such as yourself to build things on top of Mailpit, then I am eager to discuss ideas. This could include things like `data-testid` attributes to main HTML elements. So let's go through your points so I can better understand your requirements. > A) shows only the email and not the surrounding UI. What do you mean exactly with "the email", and what would be the reasoning for this exactly? Surely using HTML selectors you would be able to target the message section in the page without requiring a specific dedicated frontend component & route to just show "the email"? > B) Let me navigate directly to the latest email with /latest or something rather than having to do an API to get the ID and plugging that into a URL. I understand your reasoning here. What if `/latest` was a 302 redirect to the latest message, so for example `/latest` would be redirected to `/view/123-asd-465-fdfdg`? > The alternative would be to use the API to pull the HTML and then stick that in a folder and then set up a separate web server to display that but I'd rather have this. Well, it's possibly not that simple, at least not in a reliable way. The HTML returned from the email (assuming it has HTML) is not necessarily the same as what is displayed in the web UI. Email HTML can contain all sorts of things, including references to inline (attached) images (eg: `<img src="cid:image001.gif@01D9EA83.5AABBC80">`), removal of HTML `<base>` headers, etc. Mailpit's JavaScript frontend manipulates this to work in the frontend, for example "translating" (rewriting) inline image `src` paths. So if you simply extracted the original HTML (returned by the API) and saved that as a HTML file, it _may_ not work as you expect - depending on what HTML you have. If you're referring specifically to the HTML part of an email then you may have more luck extracting the iframe's `src-data` after it has rendered on the frontend.

kerem commented

2026-03-15 12:36:55 +03:00

Author

Owner

@kamontat commented on GitHub (Sep 25, 2023):

@axllent Despite the absence of explicit gratitude in my initial description, I intend to express my genuine admiration for your efforts, and anticipate contributing to the same measure.

Not being a native English speaker, there's a chance my words may inadvertently project hostility or offense. This is something I frequently encounter and for which I would extend my sincerest apologies if it resulted in any form of affront.

Regarding to the issue

My current integration test, I have 2 use cases.

The first one is function tests, which I did migrated to use APIs instead which works nicely with regex to select actionable link or text.

However the second one is UI tests, which I still rely on UI because it render HTML for me to comparing (ensuring) UI are consistently through all changes.

@kamontat commented on GitHub (Sep 25, 2023): @axllent Despite the absence of explicit gratitude in my initial description, I intend to express my genuine admiration for your efforts, and anticipate contributing to the same measure. Not being a native English speaker, there's a chance my words may inadvertently project hostility or offense. This is something I frequently encounter and for which I would extend my sincerest apologies if it resulted in any form of affront. ## Regarding to the issue My current integration test, I have 2 use cases. The first one is _function tests_, which I did migrated to use APIs instead which works nicely with regex to select actionable link or text. However the second one is _UI tests_, which I still rely on UI because it render HTML for me to comparing (ensuring) UI are consistently through all changes.

kerem commented

2026-03-15 12:37:00 +03:00

Author

Owner

@axllent commented on GitHub (Sep 25, 2023):

@kamontat No apology needed, and thank you for the more detailed explanation. Let's see what @crdoconnor first has to say regarding my questions so I can then hopefully come up with a solution/changes that suits you both 👍

@axllent commented on GitHub (Sep 25, 2023): @kamontat No apology needed, and thank you for the more detailed explanation. Let's see what @crdoconnor first has to say regarding my questions so I can then hopefully come up with a solution/changes that suits you both 👍

kerem commented

2026-03-15 12:37:05 +03:00

Author

Owner

@crdoconnor commented on GitHub (Sep 25, 2023):

As I mentioned in my previous comment, I am not keen on customising Mailpit's web UI to try suit frontend testing, as this then becomes a commitment to maintaining a consistent UI in the same manner as an API

I don't think that's a good idea either. The UI as it is is perfect for debugging or email inspection for users but it is not made for integration testing. This is why I suggested a new URL that just displays the HTML version of the email.

What do you mean exactly with "the email", and what would be the reasoning for this exactly? Surely using HTML selectors you would be able to target the message section in the page without requiring a specific dedicated frontend component & route to just show "the email"?

I mean a new URL with only the contents of what is currently shown in the HTML tab displayed on the page.

My playwright tests currently look a bit like this:

Click on registration button. Email sends and gets received by mailpit.
I load the UI and click on the latest email email in the UI and click on the link to confirm the email address.
Back to the website.

These tests also take screenshots and video while they do this, which I use to create documentation. It's really nice to be able to actually see the email being used and interacted with with these screenshots and the video, especially if it's rendered as it would be in a mail client.

Currently the integration testing screenshots and the video show the mailpit UI, which I would prefer to avoid and of course, I have to click on the latest email and grab the iframe in order to grab the "confirm email address" link inside of it.

For step 2 it would be nicer if I could just load a special "integration testing URL" which only displays the contents of what is currently displayed in the HTML tab for the latest email.

I would still want to use the existing UI as it is for debugging, of course.

I understand your reasoning here. What if /latest was a 302 redirect to the latest message, so for example /latest would be redirected to /view/123-asd-465-fdfdg?

Yep of course.

Well, it's possibly not that simple, at least not in a reliable way. The HTML returned from the email (assuming it has HTML) is not necessarily the same as what is displayed in the web UI. Email HTML can contain all sorts of things, including references to inline (attached) images (eg: ), removal of HTML headers, etc. Mailpit's JavaScript frontend manipulates this to work in the frontend, for example "translating" (rewriting) inline image src paths. So if you simply extracted the original HTML (returned by the API) and saved that as a HTML file, it may not work as you expect - depending on what HTML you have. If you're referring specifically to the HTML part of an email then you may have more luck extracting the iframe's src-data after it has rendered on the frontend.

Exactly. I could probably do some magic that extracts the iframe's src-data, paste it to a file and then load a separate webserver to display it in the integration test but it'd be 100x nicer to just load a URL that displays it :)

@crdoconnor commented on GitHub (Sep 25, 2023): >As I mentioned in my previous comment, I am not keen on customising Mailpit's web UI to try suit frontend testing, as this then becomes a commitment to maintaining a consistent UI in the same manner as an API I don't think that's a good idea either. The UI as it is is perfect for debugging or email inspection for users but it is not made for integration testing. This is why I suggested a new URL that just displays the HTML version of the email. >What do you mean exactly with "the email", and what would be the reasoning for this exactly? Surely using HTML selectors you would be able to target the message section in the page without requiring a specific dedicated frontend component & route to just show "the email"? I mean a new URL with *only* the contents of what is currently shown in the HTML tab displayed on the page. My playwright tests currently look a bit like this: 1. Click on registration button. Email sends and gets received by mailpit. 2. I load the UI and click on the latest email email in the UI and click on the link to confirm the email address. 3. Back to the website. These tests also take screenshots and video while they do this, which I use to create documentation. It's really nice to be able to actually see the email being used and interacted with with these screenshots and the video, especially if it's rendered as it would be in a mail client. Currently the integration testing screenshots and the video show the mailpit UI, which I would prefer to avoid and of course, I have to click on the latest email and grab the iframe in order to grab the "confirm email address" link inside of it. For step 2 it would be nicer if I could just load a special "integration testing URL" which only displays the contents of what is currently displayed in the HTML tab for the latest email. I would still want to use the existing UI as it is for debugging, of course. >I understand your reasoning here. What if /latest was a 302 redirect to the latest message, so for example /latest would be redirected to /view/123-asd-465-fdfdg? Yep of course. >Well, it's possibly not that simple, at least not in a reliable way. The HTML returned from the email (assuming it has HTML) is not necessarily the same as what is displayed in the web UI. Email HTML can contain all sorts of things, including references to inline (attached) images (eg: <img src="cid:image001.gif@01D9EA83.5AABBC80">), removal of HTML <base> headers, etc. Mailpit's JavaScript frontend manipulates this to work in the frontend, for example "translating" (rewriting) inline image src paths. So if you simply extracted the original HTML (returned by the API) and saved that as a HTML file, it may not work as you expect - depending on what HTML you have. If you're referring specifically to the HTML part of an email then you may have more luck extracting the iframe's src-data after it has rendered on the frontend. Exactly. I could probably do some magic that extracts the iframe's src-data, paste it to a file and then load a separate webserver to display it in the integration test but it'd be 100x nicer to just load a URL that displays it :)

kerem commented

2026-03-15 12:37:10 +03:00

Author

Owner

@axllent commented on GitHub (Sep 25, 2023):

This is really helpful as it helps me to understand. This question is for both of you @crdoconnor and @kamontat: I still will most likely still have to render the HTML email in an iframe like I do now, but if I could provide a url to display only the iframe and nothing else, would that work for you both and make your testing easier?

I haven't thought through the solution completely yet, but at this point I'm trying to explore ideas and just need to know if this will work for you both.

@axllent commented on GitHub (Sep 25, 2023): This is really helpful as it helps me to understand. This question is for both of you @crdoconnor and @kamontat: I still will most likely still have to render the HTML email in an iframe like I do now, but if I could provide a url to display only the iframe and nothing else, would that work for you both and make your testing easier? I haven't thought through the solution completely yet, but at this point I'm trying to explore ideas and just need to know if this will work for you both.

kerem commented

2026-03-15 12:37:15 +03:00

Author

Owner

@crdoconnor commented on GitHub (Sep 25, 2023):

Yes, that would work great.

@crdoconnor commented on GitHub (Sep 25, 2023): Yes, that would work great.

kerem commented

2026-03-15 12:37:20 +03:00

Author

Owner

@axllent commented on GitHub (Sep 26, 2023):

So I've been playing with this locally and I can simply return the html as plain html without an iframe fairly easily (because it has no UI, styles, JS etc to contend with). In other words, all that is returned is the HTML content from the email. It won't contain any message info (from, to, subject, attachments etc), just the message mime counterpart. HTML versions will however have CID embedded images relinked to the API like the web UI does so online attached images display correctly.

The hardest question I have is what to actually call the url part/section for this kind of testing. It will be standalone to the rest of the app, and the best I came up with is /testing/<id>.html (there will also be a .txt counterpart for the text version) but I'm sure there's something better than "testing". Suggestions?

@axllent commented on GitHub (Sep 26, 2023): So I've been playing with this locally and I can simply return the html as plain html without an iframe fairly easily (because it has no UI, styles, JS etc to contend with). In other words, all that is returned is the HTML content from the email. It won't contain any message info (from, to, subject, attachments etc), just the message mime counterpart. HTML versions will however have CID embedded images relinked to the API like the web UI does so online attached images display correctly. The hardest question I have is what to actually call the url part/section for this kind of testing. It will be standalone to the rest of the app, and the best I came up with is `/testing/<id>.html` (there will also be a .txt counterpart for the text version) but I'm sure there's something better than "testing". Suggestions?

kerem commented

2026-03-15 12:37:25 +03:00

Author

Owner

@crdoconnor commented on GitHub (Sep 26, 2023):

/view/email/html/<id>.html ?

I think it's fine to avoid putting in message info - an integration test can always just grab those from the API if they want to validate them.

@crdoconnor commented on GitHub (Sep 26, 2023): ```/view/email/html/<id>.html``` ? I think it's fine to avoid putting in message info - an integration test can always just grab those from the API if they want to validate them.

kerem commented

2026-03-15 12:37:31 +03:00

Author

Owner

@axllent commented on GitHub (Sep 27, 2023):

@kamontat & @crdoconnor - I have just released v1.9.3 which includes two new "endpoints" to render either the HTML or the TXT message parts: /view/<id>.html and /view/<id>.txt. This is not in an iframe, but rendered directly in the page with no application UI, CSS or JavaScript. I haven't document anything yet as I wanted to get your feedback from you both first.

The ID is obviously the message ID, however you can also use latest - as in /view/latest.html to return the latest message. If a message does not have an HTML part though, it will return a 404 message.

Please give this a test and let me know if it works well for you both? Thanks.

@axllent commented on GitHub (Sep 27, 2023): @kamontat & @crdoconnor - I have just released v1.9.3 which includes two new "endpoints" to render either the HTML or the TXT message parts: `/view/<id>.html` and `/view/<id>.txt`. This is not in an iframe, but rendered directly in the page with no application UI, CSS or JavaScript. I haven't document anything yet as I wanted to get your feedback from you both first. The ID is obviously the message ID, however you can also use `latest` - as in `/view/latest.html` to return the latest message. If a message does not have an HTML part though, it will return a 404 message. Please give this a test and let me know if it works well for you both? Thanks.

kerem commented

2026-03-15 12:37:36 +03:00

Author

Owner

@crdoconnor commented on GitHub (Sep 27, 2023):

Thats amazing, thank you!

I'll give it a whirl today.

On Wed, 27 Sep 2023, 05:39 Ralph Slooten, @.***> wrote:

@kamontat https://github.com/kamontat & @crdoconnor
https://github.com/crdoconnor - I have just released v1.9.3 which
includes two new "endpoints" to render either the HTML or the TXT message
parts: /view/.html and /view/.txt. This is not in an iframe, but
rendered directly in the page with no application UI, CSS or JavaScript. I
haven't document anything yet as I wanted to get your feedback from you
both first.

The ID is obviously the message ID, however you can also use latest - as
in /view/latest.html to return the latest message. If a message does not
have an HTML part though, it will return a 404 message.

Please give this a test and let me know if it works well for you both?
Thanks.

—
Reply to this email directly, view it on GitHub
https://github.com/axllent/mailpit/issues/166#issuecomment-1736662170,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABOJKNKI2CEU6RMFD3DOZ2LX4OUXXANCNFSM6AAAAAA455AMNA
.
You are receiving this because you were mentioned.Message ID:
@.***>

@crdoconnor commented on GitHub (Sep 27, 2023): Thats amazing, thank you! I'll give it a whirl today. On Wed, 27 Sep 2023, 05:39 Ralph Slooten, ***@***.***> wrote: > @kamontat <https://github.com/kamontat> & @crdoconnor > <https://github.com/crdoconnor> - I have just released v1.9.3 which > includes two new "endpoints" to render either the HTML or the TXT message > parts: /view/<id>.html and /view/<id>.txt. This is not in an iframe, but > rendered directly in the page with no application UI, CSS or JavaScript. I > haven't document anything yet as I wanted to get your feedback from you > both first. > > The ID is obviously the message ID, however you can also use latest - as > in /view/latest.html to return the latest message. If a message does not > have an HTML part though, it will return a 404 message. > > Please give this a test and let me know if it works well for you both? > Thanks. > > — > Reply to this email directly, view it on GitHub > <https://github.com/axllent/mailpit/issues/166#issuecomment-1736662170>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ABOJKNKI2CEU6RMFD3DOZ2LX4OUXXANCNFSM6AAAAAA455AMNA> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >

kerem commented

2026-03-15 12:37:41 +03:00

Author

Owner

@axllent commented on GitHub (Sep 29, 2023):

@crdoconnor Not sure if you got around to testing the new functionality, and whether it is working as you had hoped?

@axllent commented on GitHub (Sep 29, 2023): @crdoconnor Not sure if you got around to testing the new functionality, and whether it is working as you had hoped?

kerem commented

2026-03-15 12:37:46 +03:00

Author

Owner

@crdoconnor commented on GitHub (Sep 29, 2023):

I'm afraid I had an emergency yesterday, but I'll try to take a look today.

@crdoconnor commented on GitHub (Sep 29, 2023): I'm afraid I had an emergency yesterday, but I'll try to take a look today.

kerem commented

2026-03-15 12:37:51 +03:00

Author

Owner

@crdoconnor commented on GitHub (Sep 30, 2023):

Works perfectly, thanks!

@crdoconnor commented on GitHub (Sep 30, 2023): Works perfectly, thanks!

kerem commented

2026-03-15 12:37:56 +03:00

Author

Owner

@axllent commented on GitHub (Oct 1, 2023):

Awesome, thanks for the feedback @crdoconnor, glad to hear it! I have added some documentation in the Wiki relating to this feature.

@axllent commented on GitHub (Oct 1, 2023): Awesome, thanks for the feedback @crdoconnor, glad to hear it! I have added some documentation in [the Wiki](https://github.com/axllent/mailpit/wiki/Integration-testing) relating to this feature.

kerem commented

2026-03-15 12:38:01 +03:00

Author

Owner

@kamontat commented on GitHub (Oct 5, 2023):

I didn't try yet, but I appreciated.

@kamontat commented on GitHub (Oct 5, 2023): I didn't try yet, but I appreciated.

kerem commented

2026-03-15 12:38:06 +03:00

Author

Owner

@wadabum commented on GitHub (Nov 1, 2023):

I understand your reasoning here. What if /latest was a 302 redirect to the latest message, so for example /latest would be redirected to /view/123-asd-465-fdfdg?

@axllent could I ask for a reopen, or do you want a fresh issue to get this Route (/view/latest) implemented?

Ive seen the latest.html and latest.txt links here that are already great for automation,
but i´d like to have just a regular bookmark =)

@wadabum commented on GitHub (Nov 1, 2023): > I understand your reasoning here. What if `/latest` was a 302 redirect to the latest message, so for example `/latest` would be redirected to `/view/123-asd-465-fdfdg`? @axllent could I ask for a reopen, or do you want a fresh issue to get this Route (`/view/latest`) implemented? - Ive seen the latest.html and latest.txt links here that are already great for automation, **but i´d like to have just a regular bookmark** =)

kerem commented

2026-03-15 12:38:11 +03:00

Author

Owner

@axllent commented on GitHub (Nov 2, 2023):

@wadabum I have just added this feature (not yet released) - /view/latest will redirect to the latest message in the web UI. If there are no messages then it will redirect to the inbox (which tells you there are no messages). I don't know exactly when this will be released, but it will be in the next release, and most likely within the next week (I am trying to reduce the number of releases I do as some package maintainers weren't too happy about 2 or 3 releases per week;)).

@axllent commented on GitHub (Nov 2, 2023): @wadabum I have just added this feature (not yet released) - `/view/latest` will redirect to the latest message in the web UI. If there are no messages then it will redirect to the inbox (which tells you there are no messages). I don't know exactly when this will be released, but it will be in the next release, and most likely within the next week (I am trying to reduce the number of releases I do as some package maintainers weren't too happy about 2 or 3 releases per week;)).

Rows
Columns

[GH-ISSUE #166] Make this easier to run automate integration tests #111

Regarding to the issue