[GH-ISSUE #443] Add API call to dump all messages in text format .. WAS Bring back MailDir storage. #288

Closed
opened 2026-03-15 13:41:56 +03:00 by kerem · 5 comments
Owner

Originally created by @skyscooby on GitHub (Feb 14, 2025).
Original GitHub issue: https://github.com/axllent/mailpit/issues/443

Can you re-enable the MailDir repository feature so that messages get dumped on the filesystem like was possible with MailHog? It wouldn't have to replace the DB setup... the source of truth could continue being the DB, just automatically write out the messages to a filesystem optionally.. tthe Maildir format is a standard that goes way back.. it's something easy to tar up and upload as an artifact in CI or pass around. Having the raw messages in text format where sed, awk and grep can be used against them was really convenient in MailHog for basic test cases.. it seems like this would be pretty easy and is really the only thing that makes mailpit is worse than Mailhog was.

Originally created by @skyscooby on GitHub (Feb 14, 2025). Original GitHub issue: https://github.com/axllent/mailpit/issues/443 Can you re-enable the MailDir repository feature so that messages get dumped on the filesystem like was possible with MailHog? It wouldn't have to replace the DB setup... the source of truth could continue being the DB, just automatically write out the messages to a filesystem optionally.. tthe Maildir format is a standard that goes way back.. it's something easy to tar up and upload as an artifact in CI or pass around. Having the raw messages in text format where `sed`, `awk` and `grep` can be used against them was really convenient in MailHog for basic test cases.. it seems like this would be pretty easy and is really the only thing that makes mailpit is worse than Mailhog was.
kerem 2026-03-15 13:41:56 +03:00
Author
Owner

@axllent commented on GitHub (Feb 14, 2025):

Let me start off by pointing out that MailHog's "maildir" storage is not maildir at all, it just writes the emails directly to a specified folder similar to maildir, except the maildir format includes a structured format which MailHog does not.

Also, Mailpit is not a fork of MailHog, it's a completely separate application that shares similar functionality, so there is nothing to "bring back" as it never had maildir storage ;-)

Finally, I will also state that Mailpit will never use maildir to store & read messages, vs: the database. I know you already said this, I am just stating it for anyone else who may read this. MailHog's "maildir" storage is ridiculously inefficient and CPU intensive. It's not all related to maildir storage of course, much of it is due to the inefficiencies of how MailHog processes messages, but it's a good indicator of the general performance differences between the two.
Mailpit's SQLite database also allows it to have a fast, flexible and structured mess listing & search which maildir format does not provide. To provide a quick benchmark on my laptop:

  • To store 816 messages it takes MailHog 67s, in Mailpit 9s (7+ times faster)
  • To load an overview of 816 messages in MailHog it takes 47s, in Mailpit 0.15s (300+ times faster)
  • To search 816 messages for a single word "test" in MailHog it takes 19.7s, in Mailpit 0.1s (197 times faster)

So to get back to your request... I am trying to understand the real use-case here. The intended approach to integration testing is to use the API, which could be as simple as curl -o message.txt htp://<mailpit-host>/api/v1/message/latest/raw which would download the latest message (raw source) which could then be processed with sed, awk or grep etc, and isn't limited to the same host as Mailpit either. I don't understand how having multiple messages saved to a directory would be helpful for testing when you don't know which one is which, as they would get saved with random IDs.

Exporting all messages from the database I can understand the use-case for though, so there is currently no need to elaborate on this. I think a manual way of exporting all messages in plain text format (similar to "maildir") is a good idea.

So, for now, could you please explain any downside to using the API to retrieve the latest message in your basic testing?

<!-- gh-comment-id:2660319996 --> @axllent commented on GitHub (Feb 14, 2025): Let me start off by pointing out that MailHog's "maildir" storage is **not** maildir at all, it just writes the emails directly to a specified folder similar to maildir, except the maildir format includes a [structured format](https://en.wikipedia.org/wiki/Maildir) which MailHog does not. Also, Mailpit is not a fork of MailHog, it's a completely separate application that shares similar functionality, so there is nothing to "bring back" as it never had maildir storage ;-) Finally, I will also state that Mailpit will never use maildir to store & read messages, vs: the database. I know you already said this, I am just stating it for anyone else who may read this. MailHog's "maildir" storage is ridiculously inefficient and CPU intensive. It's not all related to maildir storage of course, much of it is due to the inefficiencies of how MailHog processes messages, but it's a good indicator of the general performance differences between the two. Mailpit's SQLite database also allows it to have a fast, flexible and structured mess listing & search which maildir format does not provide. To provide a quick benchmark on my laptop: - To store 816 messages it takes MailHog 67s, in Mailpit 9s (7+ times faster) - To load an overview of 816 messages in MailHog it takes 47s, in Mailpit 0.15s (300+ times faster) - To search 816 messages for a single word "test" in MailHog it takes 19.7s, in Mailpit 0.1s (197 times faster) So to get back to your request... I am trying to understand the real use-case here. The intended approach to integration testing is to use the API, which could be as simple as `curl -o message.txt htp://<mailpit-host>/api/v1/message/latest/raw` which would download the **latest** message (raw source) which could then be processed with `sed`, `awk` or `grep` etc, and isn't limited to the same host as Mailpit either. I don't understand how having multiple messages saved to a directory would be helpful for testing when you don't know which one is which, as they would get saved with random IDs. Exporting all messages from the database I can understand the use-case for though, so there is currently no need to elaborate on this. I think a manual way of exporting all messages in plain text format (similar to "maildir") is a good idea. So, for now, could you please explain any downside to using the API to retrieve the latest message in your basic testing?
Author
Owner

@skyscooby commented on GitHub (Feb 14, 2025):

It really is awesome! That's a heck of a performance boost, congrats.. Sorry I thought since you have that silly Chaos Jim monkey thing in this service too that it was a continuation... vs a scratch thing.. thanks for the correction.

If there was an API call to dump all the messages to text that would completely address my need as an easy way to preserve them all after a CI run. (even it seems like a tool to do this through the API might be possible but I assume slower than a single API call to do it) .. and yes we will use the API for any new test cases... Thx u.

<!-- gh-comment-id:2660379388 --> @skyscooby commented on GitHub (Feb 14, 2025): It really is awesome! That's a heck of a performance boost, congrats.. Sorry I thought since you have that silly Chaos Jim monkey thing in this service too that it was a continuation... vs a scratch thing.. thanks for the correction. If there was an API call to dump all the messages to text that would completely address my need as an easy way to preserve them all after a CI run. (even it seems like a tool to do this through the API might be possible but I assume slower than a single API call to do it) .. and yes we will use the API for any new test cases... Thx u.
Author
Owner

@axllent commented on GitHub (Feb 14, 2025):

Many features from MailHog are "replicated" due to user demand as they are required for various levels of integration testing, however Mailpit is ultimately a "from scratch" thing. Mailpit does however bring a range of additional features, completely different UI, and of course major performance improvements (which was originally the whole reason I started it).

The API doesn't currently have a "dump all" feature, however I will look into a method of utilising either the API or the database file directly (as a choice) to dump all messages to a directory. I see this as a useful general addition for archiving/export purposes.

Leave this with me 👍

<!-- gh-comment-id:2660452451 --> @axllent commented on GitHub (Feb 14, 2025): Many features from MailHog are "replicated" due to user demand as they are required for various levels of integration testing, however Mailpit is ultimately a "from scratch" thing. Mailpit does however bring a range of additional features, completely different UI, and of course major performance improvements (which was originally the whole reason I started it). The API doesn't currently have a "dump all" feature, however I will look into a method of utilising either the API or the database file directly (as a choice) to dump all messages to a directory. I see this as a useful general addition for archiving/export purposes. Leave this with me 👍
Author
Owner

@axllent commented on GitHub (Feb 15, 2025):

I've added this new feature to v1.22.3 and added documentation to the website. This should allow you to easily export all messages from the DB to a folder.

Please confirm this works as expected? Thanks.

<!-- gh-comment-id:2661099826 --> @axllent commented on GitHub (Feb 15, 2025): I've added this new feature to [v1.22.3](https://github.com/axllent/mailpit/releases/tag/v1.22.3) and [added documentation](https://mailpit.axllent.org/docs/usage/import-export/) to the website. This should allow you to easily export all messages from the DB to a folder. Please confirm this works as expected? Thanks.
Author
Owner

@skyscooby commented on GitHub (Feb 15, 2025):

Fantastic!!! Thank you.. this is perfect.

<!-- gh-comment-id:2661134149 --> @skyscooby commented on GitHub (Feb 15, 2025): Fantastic!!! Thank you.. this is perfect.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/mailpit#288
No description provided.