[PR #942] feat: allow direct pdf archives #883

New issue

Open

opened 2026-02-25 23:35:47 +03:00 by kerem · 0 comments

kerem commented

2026-02-25 23:35:47 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/go-shiori/shiori/pull/942
Author: @fmartingr
Created: 6/25/2024
Status: 🔄 Open

Base: master ← Head: feat/pdf-archiver

📝 Commits (10+)

0896957 feat: new pdf archiver
1fe353e archiver, archiver type
8f7837d mysql pg migrations
3f27d7c iterate archivers if error
da41211 fixed getbookmark
9d34bad basic archive page
c8dd55d archive page
978be82 bookmark logic
8455991 swagger
e08334b ebook logic

📊 Changes

27 files changed (+602 additions, -155 deletions)

View changed files

📝 docs/swagger/docs.go (+6 -0)
📝 docs/swagger/swagger.json (+6 -0)
📝 docs/swagger/swagger.yaml (+4 -0)
➕ internal/archiver/pdf.go (+57 -0)
➕ internal/archiver/warc.go (+78 -0)
📝 internal/cmd/add.go (+11 -27)
📝 internal/core/ebook.go (+8 -3)
📝 internal/core/ebook_test.go (+9 -45)
📝 internal/core/processing.go (+5 -4)
📝 internal/database/database_test.go (+7 -2)
📝 internal/database/migrations.go (+4 -2)
➕ internal/database/migrations/0001_migrate_archiver.go (+141 -0)
➕ internal/database/migrations/mysql/0011_bookmark_archiver.up.sql (+3 -0)
➕ internal/database/migrations/postgres/0003_bookmark_archiver.up.sql (+3 -0)
➕ internal/database/migrations/sqlite/0005_bookmark_archiver.up.sql (+2 -0)
📝 internal/database/mysql.go (+19 -6)
📝 internal/database/pg.go (+25 -9)
📝 internal/database/sqlite.go (+14 -6)
📝 internal/domains/archiver.go (+49 -8)
📝 internal/domains/bookmarks_test.go (+1 -1)

...and 7 more files

📄 Description

This pull request aims to provide PDF archives for bookmarks by directly downloading the file.

Notable changes

Logic for archivers now uses the ArchiverDomain
- ArchiverDomain initializes a list of archivers that try to match the content type, if it does, that archiver is used. If an error is raised, the next archiver is used and so on.
Added PDFArchiver: Just downloads the file.
Added WARCArchiver as a "catch all" (current default)
Added database columns bookmark.archiver (warc/pdf) and archive_path.
- Bookmark has an archive if archive_path != ""

Pending

Migrate the archive_path for existing bookmarks: iterate over the bookmarks, check if the archive exists and set the path and the archiver to warc.
Refactor EPub logic separate from the archiver
Allow different archivers in archive page
Tests
e2e tests?

Closes #929

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/go-shiori/shiori/pull/942 **Author:** [@fmartingr](https://github.com/fmartingr) **Created:** 6/25/2024 **Status:** 🔄 Open **Base:** `master` ← **Head:** `feat/pdf-archiver` --- ### 📝 Commits (10+) - [`0896957`](https://github.com/go-shiori/shiori/commit/089695764dfe4bedeeb642e484515457a251d6ce) feat: new pdf archiver - [`1fe353e`](https://github.com/go-shiori/shiori/commit/1fe353e16289beb1cfe6be67cf9193149bb30495) archiver, archiver type - [`8f7837d`](https://github.com/go-shiori/shiori/commit/8f7837d57f7c7a2c19c05e26ef562ded1867d0d9) mysql pg migrations - [`3f27d7c`](https://github.com/go-shiori/shiori/commit/3f27d7c4cb072bea3ef96991cba19b7b88f5dad2) iterate archivers if error - [`da41211`](https://github.com/go-shiori/shiori/commit/da412113d9e7e19a1661a512da29962db55f33bb) fixed getbookmark - [`9d34bad`](https://github.com/go-shiori/shiori/commit/9d34badeb2db5350905093a48631fae603cc5592) basic archive page - [`c8dd55d`](https://github.com/go-shiori/shiori/commit/c8dd55dac604580640b9b44d922703de74e2748c) archive page - [`978be82`](https://github.com/go-shiori/shiori/commit/978be82cf7e03244ae850d88361c38712ea378bf) bookmark logic - [`8455991`](https://github.com/go-shiori/shiori/commit/84559914d6f45eea6e8f52218435b1290bdc6f83) swagger - [`e08334b`](https://github.com/go-shiori/shiori/commit/e08334b27a3f185c5a240677699ddb68c1372619) ebook logic ### 📊 Changes **27 files changed** (+602 additions, -155 deletions) <details> <summary>View changed files</summary> 📝 `docs/swagger/docs.go` (+6 -0) 📝 `docs/swagger/swagger.json` (+6 -0) 📝 `docs/swagger/swagger.yaml` (+4 -0) ➕ `internal/archiver/pdf.go` (+57 -0) ➕ `internal/archiver/warc.go` (+78 -0) 📝 `internal/cmd/add.go` (+11 -27) 📝 `internal/core/ebook.go` (+8 -3) 📝 `internal/core/ebook_test.go` (+9 -45) 📝 `internal/core/processing.go` (+5 -4) 📝 `internal/database/database_test.go` (+7 -2) 📝 `internal/database/migrations.go` (+4 -2) ➕ `internal/database/migrations/0001_migrate_archiver.go` (+141 -0) ➕ `internal/database/migrations/mysql/0011_bookmark_archiver.up.sql` (+3 -0) ➕ `internal/database/migrations/postgres/0003_bookmark_archiver.up.sql` (+3 -0) ➕ `internal/database/migrations/sqlite/0005_bookmark_archiver.up.sql` (+2 -0) 📝 `internal/database/mysql.go` (+19 -6) 📝 `internal/database/pg.go` (+25 -9) 📝 `internal/database/sqlite.go` (+14 -6) 📝 `internal/domains/archiver.go` (+49 -8) 📝 `internal/domains/bookmarks_test.go` (+1 -1) _...and 7 more files_ </details> ### 📄 Description This pull request aims to provide PDF archives for bookmarks by directly downloading the file. #### Notable changes - Logic for archivers now uses the `ArchiverDomain` - `ArchiverDomain` initializes a list of archivers that try to match the content type, if it does, that archiver is used. If an error is raised, the next archiver is used and so on. - Added `PDFArchiver`: Just downloads the file. - Added `WARCArchiver` as a "catch all" (current default) - Added database columns `bookmark.archiver` (`warc`/`pdf`) and `archive_path`. - Bookmark has an archive if `archive_path != ""` ### Pending - [x] Migrate the `archive_path` for existing bookmarks: iterate over the bookmarks, check if the archive exists and set the path and the `archiver` to `warc`. - [ ] Refactor EPub logic separate from the archiver - [x] Allow different archivers in archive page - [ ] Tests - [ ] e2e tests? Closes #929 --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>