mirror of
https://github.com/axllent/mailpit.git
synced 2026-04-26 08:45:54 +03:00
[GH-ISSUE #8] Harddrive storage size #6
Labels
No labels
awaiting feedback
bug
docker
documentation
enhancement
github_actions
invalid
pull-request
question
stale
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/mailpit#6
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @Florent4782 on GitHub (Aug 17, 2022).
Original GitHub issue: https://github.com/axllent/mailpit/issues/8
Hi,
first of all, a big thank you for your amazing work !
It is not necessarily a big issue, but I find a bit weird that I got a 2GB+ file in the storage directory while there is only 4 emails... with small attachments.
@Florent4782 commented on GitHub (Aug 17, 2022):
Ok, I saw in the issue #3 that it is a temporary file from CloverDB. I feel like it's huge. Is there any way we can decrease this size ? or the file to "autogrow" ?
@axllent commented on GitHub (Aug 17, 2022):
@Florent4782 Thanks for the appreciation and feedback. This confused me at first too, but it is an internal BadgerDB (the underlying database that CloverDB uses) memory-mapping file. On Linux (I'm not 100% sure about Windows), this file looks like a physical file but it doesn't actually take up any physical space (
du -hs <folder>returns the total size minus the 2GB).@axllent commented on GitHub (Aug 17, 2022):
To follow on from this, CloverDB/BadgerDB does use more memory (even when saving to disk) than I would like. Ideally this would be far less, however I've put that into the "too hard basket" as it would mean a large change to Mailpit, as I would probably have to choose another database altogether. CloverDB (functionality-wise) does exactly what I need it to, and I wasn't able to find another similar solution (that didn't involve a hosted database server which I do not want).
@Florent4782 commented on GitHub (Aug 17, 2022):
OK, I understand.
Let me just ask a question though with my little development knowledge: what about SQLite ?
Will let you know about what real size it takes on Windows.
@axllent commented on GitHub (Aug 17, 2022):
SQLite has two main issues - normally it requires CGO (not native to Go) which is horrible for cross-platform binaries. There is now a "native" Go version for SQLite, however I don't think it's yet production-ready. Secondly, SQLite is not suitable re: speed/performance (imagine storing 10k full emails - or 5GB of data in a single SQLite file). I did look at this initially when choosing a database and decided against it :)
@axllent commented on GitHub (Aug 28, 2022):
@Florent4782 Just to keep you in the loop - after finding a native Go SQLite library, I have been trialling SQLite here with very promising results ~ even with a database containing 300k emails / 7.7GB (which is far more than anyone would every have but still...). This addresses several issues too, including storage and memory usage. I still have more extensive testing to do first, but so far so good!
@axllent commented on GitHub (Aug 30, 2022):
@Florent4782 I have just released a beta version containing a major change (the switch to SQLite). This should address the storage issue, and severely reduce memory (RAM) use. I documented the full reasons for the switch to SQLite in #10 - and thank you for the suggestion (which made me revisit SQLite as a viable option for Mailpit). Please let me know if this works for you as expected?
@Florent4782 commented on GitHub (Aug 30, 2022):
@axllent, from a user's perspective this appears to be a good move ! Mailpit starts much faster (I guess because it doesn't have to write the 2GB file), and indeed memory usage is much small too !
Just as confirmation, the 2GB file seemed real on Windows:
I did some tests with attachments and this seems great.
It may be a good idea to "VACCUM" the db file on closing Mailpit. I noticed it doesn't seem to be the case now.
@axllent commented on GitHub (Aug 30, 2022):
@Florent4782 Thanks for the feedback, I'll consider this issue resolved then 👍
Regarding running a
VACUUMwhen the application exists - VACUUM takes time and is processor-intensive (the entire database is compacted and rewritten). This would mean that existing the application could take 30 - 60 seconds which isn't good, especially if running Mailpit within docker (which will by default kill running processes after 30 seconds or so). VACUUM is also only useful when there are deleted messages, and currently runs automatically when:Delete allhas happened, or