[PR #232] [CLOSED] WIP: pipeline overhaul #573

New issue

Closed

opened 2026-02-25 21:32:17 +03:00 by kerem · 0 comments

kerem commented

2026-02-25 21:32:17 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/ciur/papermerge/pull/232
Author: @francescocarzaniga
Created: 11/28/2020
Status: ❌ Closed

Base: master ← Head: pipeline_doc

📝 Commits (10+)

b6874d8 Refactor pipeline and importers
ae53015 Add pipeline tests
e56042f Better tests for simple pipeline
18d82e3 Added OCR tests
98193c7 Add local importer tests
1c5ee3f Added IMAP tests
54239b4 Added import_pipeline to API
adbb8e0 Removed document_importer and sped up tests
e2635e7 Separated OCR from pipeline proper
be2e07f Fixed conflict

📊 Changes

11 files changed (+803 additions, -498 deletions)

View changed files

➖ papermerge/core/document_importer.py (+0 -164)
📝 papermerge/core/import_pipeline.py (+154 -87)
📝 papermerge/core/importers/imap.py (+71 -87)
📝 papermerge/core/importers/local.py (+25 -55)
📝 papermerge/core/views/api.py (+13 -11)
📝 papermerge/core/views/documents.py (+4 -38)
📝 papermerge/test/test_document.py (+16 -35)
➕ papermerge/test/test_imap_import.py (+195 -0)
📝 papermerge/test/test_import_pipelines.py (+140 -21)
➕ papermerge/test/test_local_import.py (+84 -0)
➕ papermerge/test/test_ocr_pipeline.py (+101 -0)

📄 Description

Description

Refactored DefaultPipeline to be more streamlined. Added class and methods documentation.
Added tests for Pipelines, OCR, directory importer and IMAP importer.

Fixes #229 #233

Needs modified version of mglib to work (see my PR).

Type of change

Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Added pipeline tests.

Test Configuration:

Python version: 3.8
OS version: Ubuntu 20.04

Checklist:

I have read the Contributing file available here
I have formatted this PR according to PEP8 rules
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/ciur/papermerge/pull/232 **Author:** [@francescocarzaniga](https://github.com/francescocarzaniga) **Created:** 11/28/2020 **Status:** ❌ Closed **Base:** `master` ← **Head:** `pipeline_doc` --- ### 📝 Commits (10+) - [`b6874d8`](https://github.com/ciur/papermerge/commit/b6874d819883427e951d4762bb9e1d6dc78b9d91) Refactor pipeline and importers - [`ae53015`](https://github.com/ciur/papermerge/commit/ae53015ef5dcfaed4977523963c85893145a22b8) Add pipeline tests - [`e56042f`](https://github.com/ciur/papermerge/commit/e56042f0b35b3ae1e0f1ea147134331bdfa7b2b9) Better tests for simple pipeline - [`18d82e3`](https://github.com/ciur/papermerge/commit/18d82e3e714a278c98bf4b9748753380bfb9f642) Added OCR tests - [`98193c7`](https://github.com/ciur/papermerge/commit/98193c7079dcc47e7f7455c419cb71965504b1fd) Add local importer tests - [`1c5ee3f`](https://github.com/ciur/papermerge/commit/1c5ee3f5377a4150ab5d1c6546dce4dd9acfe77b) Added IMAP tests - [`54239b4`](https://github.com/ciur/papermerge/commit/54239b4331794b4da69967f13d125852a4fae40a) Added import_pipeline to API - [`adbb8e0`](https://github.com/ciur/papermerge/commit/adbb8e0f237be5d6c9cdb5bf81e36a27a6c9f47e) Removed document_importer and sped up tests - [`e2635e7`](https://github.com/ciur/papermerge/commit/e2635e7df09ccb354091186f7610bcd875a7046d) Separated OCR from pipeline proper - [`be2e07f`](https://github.com/ciur/papermerge/commit/be2e07f098c1c31db61d95c201be08b6adbe805e) Fixed conflict ### 📊 Changes **11 files changed** (+803 additions, -498 deletions) <details> <summary>View changed files</summary> ➖ `papermerge/core/document_importer.py` (+0 -164) 📝 `papermerge/core/import_pipeline.py` (+154 -87) 📝 `papermerge/core/importers/imap.py` (+71 -87) 📝 `papermerge/core/importers/local.py` (+25 -55) 📝 `papermerge/core/views/api.py` (+13 -11) 📝 `papermerge/core/views/documents.py` (+4 -38) 📝 `papermerge/test/test_document.py` (+16 -35) ➕ `papermerge/test/test_imap_import.py` (+195 -0) 📝 `papermerge/test/test_import_pipelines.py` (+140 -21) ➕ `papermerge/test/test_local_import.py` (+84 -0) ➕ `papermerge/test/test_ocr_pipeline.py` (+101 -0) </details> ### 📄 Description # Description Refactored DefaultPipeline to be more streamlined. Added class and methods documentation. Added tests for Pipelines, OCR, directory importer and IMAP importer. Fixes #229 #233 **Needs modified version of mglib to work (see my PR).** ## Type of change Please delete options that are not relevant. - [x] Bug fix (non-breaking change which fixes an issue) # How Has This Been Tested? Added pipeline tests. **Test Configuration**: * Python version: 3.8 * OS version: Ubuntu 20.04 # Checklist: - [x] I have read the [Contributing file available here](https://github.com/ciur/papermerge/blob/master/CONTRIBUTING.md) - [x] I have formatted this PR according to [PEP8 rules](https://www.python.org/dev/peps/pep-0008/) - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>