starred/papermerge

Fork 0

mirror of https://github.com/ciur/papermerge.git synced 2026-04-25 12:05:58 +03:00

[GH-ISSUE #159] [Feature] Email inbox enhancement #124

New issue

Closed

opened 2026-02-25 21:31:15 +03:00 by kerem · 6 comments

kerem commented

2026-02-25 21:31:15 +03:00

Owner

Originally created by @francescocarzaniga on GitHub (Oct 11, 2020).
Original GitHub issue: https://github.com/ciur/papermerge/issues/159

Originally assigned to: @ciur on GitHub.

Email importing is a really cool feature, but at the moment the implementation seems pretty barebones. The flag IMPORT_MAIL_SECRET is not used anywhere in the code as far as I can tell. The DocumentImporter/IMAPClient code is however extremely flexible, so a variety of enhancements are possible with few lines of code.

I would like to propose two alternatives:

Sort incoming emails by sender:
Email address is mandatory for each user, so once a mail is received the sender will be matched to a user and the document will be put into said user's inbox. If there is no match, the server can proceed as it does now.
Sort incoming emails by secret:
Each user could specify a secret (or it could be provided by the server as a passphrase to avoid collisions). Then incoming emails would be sorted by secret into the user's inbox.
Both could be implemented as well, with either a server-wide flag (IMPORT_MAIL_BY_SECRET/IMPORT_MAIL_BY_USER) or a per-user setting.

Moreover, I couldn't check whether the mail importer runs regularly or must be run manually each time. A further improvement would be to either run it every 5(or 10?) minutes or to use IDLE on mailservers that support it.

Once decided which approach is best I could start working on it.

Originally created by @francescocarzaniga on GitHub (Oct 11, 2020). Original GitHub issue: https://github.com/ciur/papermerge/issues/159 Originally assigned to: @ciur on GitHub. Email importing is a really cool feature, but at the moment the implementation seems pretty barebones. The flag IMPORT_MAIL_SECRET is not used anywhere in the code as far as I can tell. The DocumentImporter/IMAPClient code is however extremely flexible, so a variety of enhancements are possible with few lines of code. I would like to propose two alternatives: 1. **Sort incoming emails by sender**: Email address is mandatory for each user, so once a mail is received the sender will be matched to a user and the document will be put into said user's inbox. If there is no match, the server can proceed as it does now. 2. **Sort incoming emails by secret**: Each user could specify a secret (or it could be provided by the server as a passphrase to avoid collisions). Then incoming emails would be sorted by secret into the user's inbox. Both could be implemented as well, with either a server-wide flag (IMPORT_MAIL_BY_SECRET/IMPORT_MAIL_BY_USER) or a per-user setting. Moreover, I couldn't check whether the mail importer runs regularly or must be run manually each time. A further improvement would be to either run it every 5(or 10?) minutes or to use IDLE on mailservers that support it. Once decided which approach is best I could start working on it.

kerem

2026-02-25 21:31:15 +03:00

closed this issue
added the
enhancement

feature request
labels

kerem commented

2026-02-25 21:31:16 +03:00

Author

Owner

@ciur commented on GitHub (Oct 11, 2020):

@francescocarzaniga, don't rush with this one, as it is not so easy as it looks like.

What you omitted in above feature description is following: how is IMAPClient supposed to store (imap account) passwords for each individual user?
Configuring password for each user is not possible as you don't know in advance users in the system.
Another way would be is to store IMAP passwords (for each user) in DB... but that means you need to store passwords cleartext in DB ! This is a no go because of huge security hole it would add.
Yet another possibility is to ask (each individual) user for password and fetch attachment at that point - but I don't like this solution.

Because of above explanation, Papermerge is now limited to handle only one IMAP account (IMPORT_EMAIL_PASS, ...) which is associated with first administrative user/superuser.

Cleartext password is required here.
Do you have any suggestion where/how to store securely IMAP password for each individual IMAP user?

I don't :)
If you don't have any secure solution, I would just close the ticket. As I mentioned at the beginning, it is more complicated than one might initially think.

@ciur commented on GitHub (Oct 11, 2020): @francescocarzaniga, don't rush with this one, as it is not so easy as it looks like. What you omitted in above feature description is following: **how is IMAPClient supposed to store (imap account) passwords for each individual user?** Configuring password for each user is not possible as you don't know in advance users in the system. Another way would be is to store IMAP passwords (for each user) in DB... but that means you need to store passwords cleartext in DB ! This is a no go because of huge security hole it would add. Yet another possibility is to ask (each individual) user for password and fetch attachment at that point - but I don't like this solution. Because of above explanation, Papermerge is now limited to handle only one IMAP account ([IMPORT_EMAIL_PASS](https://papermerge.readthedocs.io/en/latest/settings.html#import-mail-pass), ...) which is associated with first administrative user/superuser. [Cleartext password is required here.](https://github.com/ciur/papermerge/blob/master/papermerge/core/importers/imap.py#L105) Do you have any suggestion where/how to **store securely IMAP password** for each individual IMAP user? I don't :) If you don't have any *secure* solution, I would just close the ticket. As I mentioned at the beginning, it is more complicated than one might initially think.

kerem commented

2026-02-25 21:31:16 +03:00

Author

Owner

@francescocarzaniga commented on GitHub (Oct 11, 2020):

@ciur I completely understand your worries but this is not what I meant. Let me try to make this a bit clearer.

Papermerge would interface to one IMAP account, let's say papermerge@domain.com. There are two users: user1 with email user1@domain1.com, and user2 with email user2@domain2.com. user1 sends an email to papermerge@domain.com with a document to be filed. Currently Papermerge puts this document into the superuser's inbox, but I would propose that it would perform a lookup for user1@domain1.com and file it into user1's inbox. Same with user2. Let's call this IMPORT_MAIL_BY_USER.

Alternatively Papermerge could generate a passphrase (e.g. 3 random words separated by a hyphen in the preferred language of the user) and attach it to each user. Then when user1 and user2 send an email to papermerge@domain.com, the server would look for this string and put the document in the corresponding user's inbox. Let's call this IMPORT_MAIL_BY_SECRET.

Ideally there would be a setting in the user's page which allows them to toggle both options on and off.

@francescocarzaniga commented on GitHub (Oct 11, 2020): @ciur I completely understand your worries but this is not what I meant. Let me try to make this a bit clearer. Papermerge would interface to one IMAP account, let's say papermerge@domain.com. There are two users: user1 with email user1@domain1.com, and user2 with email user2@domain2.com. user1 sends an email to papermerge@domain.com with a document to be filed. Currently Papermerge puts this document into the superuser's inbox, but I would propose that it would perform a lookup for user1@domain1.com and file it into user1's inbox. Same with user2. Let's call this IMPORT_MAIL_BY_USER. Alternatively Papermerge could generate a passphrase (e.g. 3 random words separated by a hyphen in the preferred language of the user) and attach it to each user. Then when user1 and user2 send an email to papermerge@domain.com, the server would look for this string and put the document in the corresponding user's inbox. Let's call this IMPORT_MAIL_BY_SECRET. Ideally there would be a setting in the user's page which allows them to toggle both options on and off.

kerem commented

2026-02-25 21:31:16 +03:00

Author

Owner

@ciur commented on GitHub (Oct 11, 2020):

@francescocarzaniga, now I understand!
Really great idea! It never crossed my mind to match by sender... really great!

Sure, this PR I would love to include. I will include it in 1.6 though. Too many changes for 1.5 so far.

@ciur commented on GitHub (Oct 11, 2020): @francescocarzaniga, now I understand! Really great idea! It never crossed my mind to match by sender... really great! Sure, this PR I would love to include. I will include it in 1.6 though. Too many changes for 1.5 so far.

kerem commented

2026-02-25 21:31:16 +03:00

Author

Owner

@francescocarzaniga commented on GitHub (Oct 11, 2020):

Before starting to write code I need to know a few things:. How is the code called? Is it called every few minutes automatically or does the admin need to run it manually? Ideally the code would need to be run every 5 minutes (or 25 with IDLE), and then killed and reopened at the end of the timeout.
Moreover I am terrible at frontend, so somebody else will probably need to write that part.

@francescocarzaniga commented on GitHub (Oct 11, 2020): Before starting to write code I need to know a few things:. How is the code called? Is it called every few minutes automatically or does the admin need to run it manually? Ideally the code would need to be run every 5 minutes (or 25 with IDLE), and then killed and reopened at the end of the timeout. Moreover I am terrible at frontend, so somebody else will probably need to write that part.

kerem commented

2026-02-25 21:31:16 +03:00

Author

Owner

@ciur commented on GitHub (Oct 11, 2020):

The email importing part is handled by worker. Which is started with:

./manage.py worker

This is where import_from_email is called

import_from_email periodic task (called in python/celery jargon: celery beat) is called every 30 seconds. 30 seconds period is hardcoded and I don't see any problem with that.

About frontend don't worry, I will handle that part (again, it will be part of 1.6).

Usually you start application in two terminals one for webserver (./manage.py runserver) and another one is for OCR/Email import/Watch folder (./manage.py worker). Worker takes care of periodic tasks like email import and asynchronous OCR.

@ciur commented on GitHub (Oct 11, 2020): The email importing part is handled by [worker](https://papermerge.readthedocs.io/en/latest/utilities.html#worker). Which is started with: ``` ./manage.py worker ``` [This is where import_from_email is called](https://github.com/ciur/papermerge/blob/master/papermerge/core/management/commands/worker.py#L92) import_from_email periodic task (called in python/celery jargon: celery beat) is called every 30 seconds. 30 seconds period is hardcoded and I don't see any problem with that. About frontend don't worry, I will handle that part (again, it will be part of 1.6). Usually you start application in two terminals one for webserver (``./manage.py runserver``) and another one is for OCR/Email import/Watch folder (``./manage.py worker``). Worker takes care of periodic tasks like email import and asynchronous OCR.

kerem commented

2026-02-25 21:31:16 +03:00

Author

Owner

@dani commented on GitHub (Oct 18, 2020):

Sorting by dest would also be very useful. My use case : Configure an IMAP account for papermerge, and share it to several users. When user1 receive an email with a PDF attached (with their own email as dest), they can just drag&drop it into the papermerge IMAP folder, papermerge could match the dest being user1@example.org and assign it to the correct user.

@dani commented on GitHub (Oct 18, 2020): Sorting by dest would also be very useful. My use case : Configure an IMAP account for papermerge, and share it to several users. When user1 receive an email with a PDF attached (with their own email as dest), they can just drag&drop it into the papermerge IMAP folder, papermerge could match the dest being user1@example.org and assign it to the correct user.