[PR #707] #578: Add ability to schedule and manage recurring imports via the admin UI #2776

New issue

Closed

opened 2026-03-01 18:00:43 +03:00 by kerem · 0 comments

kerem commented

2026-03-01 18:00:43 +03:00

Owner

Original Pull Request: https://github.com/ArchiveBox/ArchiveBox/pull/707

State: closed
Merged: No

Fixes: #578

Remaining TODOs:

figure out which python scheduler to use
- huey (my current favorite)
- celery (ugh...)
- APScheduler (will require lots of manual models and concurrency control code)
- yacron (not sure if it can be configured dynamically)
- dramatiq (doesn't support sqlite)
decide whether to continue supporting system crontab at all, or tear it out (imo we should just tear it out and move to using an internal scheduler)
fork the scheduled task worker off the server process automatically on startup, so no need to run separate archivebox schedule --foreground process manually
figure out how to enforce "at least once" or "at most once" concurrency model for scheduled tasks

**Original Pull Request:** https://github.com/ArchiveBox/ArchiveBox/pull/707 **State:** closed **Merged:** No --- Fixes: #578 **Remaining TODOs:** - [ ] figure out which python scheduler to use - huey (my current favorite) - celery (ugh...) - APScheduler (will require lots of manual models and concurrency control code) - yacron (not sure if it can be configured dynamically) - dramatiq (doesn't support sqlite) - [ ] decide whether to continue supporting system crontab at all, or tear it out (imo we should just tear it out and move to using an internal scheduler) - [ ] fork the scheduled task worker off the server process automatically on startup, so no need to run separate `archivebox schedule --foreground` process manually - [ ] figure out how to enforce "at least once" or "at most once" concurrency model for scheduled tasks

kerem

2026-03-01 18:00:43 +03:00

closed this issue
added the
pull-request
label

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/ArchiveBox#2776

No description provided.

Rows
Columns