[PR #481] [CLOSED] Draft: Replace WARC with Obelisk #608

New issue

Closed

opened 2026-02-25 23:34:53 +03:00 by kerem · 0 comments

kerem commented

2026-02-25 23:34:53 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/go-shiori/shiori/pull/481
Author: @fmartingr
Created: 9/30/2022
Status: ❌ Closed

Base: master ← Head: feat/obelisk

📝 Commits (5)

02cbce7 modules: add obelisk
8f460f2 typo: fix user agent for downloads
95bec6e feat: archive using obelisk
404764f feat: serve archive correctly from obelisk
6ed3b14 avoid using archive when serving content

📊 Changes

5 files changed (+122 additions, -93 deletions)

View changed files

📝 go.mod (+6 -0)
📝 go.sum (+12 -1)
📝 internal/core/core.go (+1 -1)
📝 internal/core/processing.go (+26 -9)
📝 internal/webserver/handler-ui.go (+77 -82)

📄 Description

This is a draft to replace WARC with Obelisk in order to know what would be the impact of such a change.

Notes:

Serving content (not archives) uses the archive in some way. Don't know why, need to investigate
Old archives would require a migration from the old system (WARC) to the new one (Obelisk) potentially saving up space and being much more confortable to manage since it will create single files.
Right now storage is file-based, I would move things to database with a cache layer so its easier for us to manage and for users to backup, manage and migrate.

Closes #353

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/go-shiori/shiori/pull/481 **Author:** [@fmartingr](https://github.com/fmartingr) **Created:** 9/30/2022 **Status:** ❌ Closed **Base:** `master` ← **Head:** `feat/obelisk` --- ### 📝 Commits (5) - [`02cbce7`](https://github.com/go-shiori/shiori/commit/02cbce7dc6ba1f361b5de4c61bb476b51cfec5c8) modules: add obelisk - [`8f460f2`](https://github.com/go-shiori/shiori/commit/8f460f2b81122b2c92c7211497e94d916e5bb404) typo: fix user agent for downloads - [`95bec6e`](https://github.com/go-shiori/shiori/commit/95bec6ef5fde6ef67c09e653a0cf5ea36ad31252) feat: archive using obelisk - [`404764f`](https://github.com/go-shiori/shiori/commit/404764f1a90ffc69a4b07cdb4e3c40b307156b30) feat: serve archive correctly from obelisk - [`6ed3b14`](https://github.com/go-shiori/shiori/commit/6ed3b1449a5540c6f12a2438e66ed2b6eef0269f) avoid using archive when serving content ### 📊 Changes **5 files changed** (+122 additions, -93 deletions) <details> <summary>View changed files</summary> 📝 `go.mod` (+6 -0) 📝 `go.sum` (+12 -1) 📝 `internal/core/core.go` (+1 -1) 📝 `internal/core/processing.go` (+26 -9) 📝 `internal/webserver/handler-ui.go` (+77 -82) </details> ### 📄 Description This is a draft to replace WARC with Obelisk in order to know what would be the impact of such a change. Notes: - Serving content (not archives) uses the archive in some way. Don't know why, need to investigate - Old archives would require a migration from the old system (WARC) to the new one (Obelisk) potentially saving up space and being much more confortable to manage since it will create single files. - Right now storage is file-based, I would move things to database with a cache layer so its easier for us to manage and for users to backup, manage and migrate. Closes #353 --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>