[PR #707] [CLOSED] #578: Add ability to schedule and manage recurring imports via the admin UI #1264

New issue

Closed

opened 2026-03-01 14:49:05 +03:00 by kerem · 0 comments

kerem commented

2026-03-01 14:49:05 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/ArchiveBox/ArchiveBox/pull/707
Author: @pirate
Created: 4/16/2021
Status: ❌ Closed

Base: dev ← Head: scheduler-ui

📝 Commits (2)

c2f2f4f make add command accept Path as import and return all_links, new_links
940b9fe add beginnings of new scheduler model for recurring imports

📊 Changes

2 files changed (+106 additions, -7 deletions)

View changed files

📝 archivebox/main.py (+7 -7)
➕ archivebox/scheduler/models.py (+99 -0)

📄 Description

Fixes: #578

Remaining TODOs:

figure out which python scheduler to use
- huey (my current favorite)
- celery (ugh...)
- APScheduler (will require lots of manual models and concurrency control code)
- yacron (not sure if it can be configured dynamically)
- dramatiq (doesn't support sqlite)
decide whether to continue supporting system crontab at all, or tear it out (imo we should just tear it out and move to using an internal scheduler)
fork the scheduled task worker off the server process automatically on startup, so no need to run separate archivebox schedule --foreground process manually
figure out how to enforce "at least once" or "at most once" concurrency model for scheduled tasks

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/ArchiveBox/ArchiveBox/pull/707 **Author:** [@pirate](https://github.com/pirate) **Created:** 4/16/2021 **Status:** ❌ Closed **Base:** `dev` ← **Head:** `scheduler-ui` --- ### 📝 Commits (2) - [`c2f2f4f`](https://github.com/ArchiveBox/ArchiveBox/commit/c2f2f4ff57b88c55cc557a73ce28ed06e04d3c28) make add command accept Path as import and return all_links, new_links - [`940b9fe`](https://github.com/ArchiveBox/ArchiveBox/commit/940b9fe181b465c9c1483bea124c2d819c9ca8a4) add beginnings of new scheduler model for recurring imports ### 📊 Changes **2 files changed** (+106 additions, -7 deletions) <details> <summary>View changed files</summary> 📝 `archivebox/main.py` (+7 -7) ➕ `archivebox/scheduler/models.py` (+99 -0) </details> ### 📄 Description Fixes: #578 **Remaining TODOs:** - [ ] figure out which python scheduler to use - huey (my current favorite) - celery (ugh...) - APScheduler (will require lots of manual models and concurrency control code) - yacron (not sure if it can be configured dynamically) - dramatiq (doesn't support sqlite) - [ ] decide whether to continue supporting system crontab at all, or tear it out (imo we should just tear it out and move to using an internal scheduler) - [ ] fork the scheduled task worker off the server process automatically on startup, so no need to run separate `archivebox schedule --foreground` process manually - [ ] figure out how to enforce "at least once" or "at most once" concurrency model for scheduled tasks --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>