[GH-ISSUE #1289] Rename `archivebox oneshot` -> `archivebox snapshot` & `archivebox extract`, expose atomic subtasks for future job queue system #3814

New issue

Open

opened 2026-03-15 00:33:02 +03:00 by kerem · 0 comments

kerem commented

2026-03-15 00:33:02 +03:00

Owner

Originally created by @pirate on GitHub (Dec 15, 2023).
Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/1289

I want each thing that ArchiveBox does internally to be a subcommand exposed directly to users or callable by the Huey job queue system.

It will also have the side effect of making the process tree nicely represent what ArchiveBox is doing internally, and allow users to kill stuck subtasks independently without stopping the entire "add" import.

The old oneshot will be renamed and joined by a new command to run a single extractor method:

`archivebox snapshot`

Can be run to snapshot an individual URL into the current directory (runs all extractors by default).

archivebox snapshot --methods=all 'https://example.com/somepage.html'
# creates a subfolder for each extractor method, and an index.html and index.json file in $PWD

This works the same way as oneshot does now, and I'll alias oneshot to the new command so we don't break backwards compatibility.

`archivebox extract`

This runs an individual extractor method and outputs into the current directory.

archivebox extract --method=PDF --method-args-here 'https://example.com/somepage.html'
# writes output.pdf (and an index.json containing cmd+output for each run) into $PWD using the headless browser

After the refactor, archivebox add will work by internally enqueuing a job that runs archivebox snapshot ... for each imported URL.
The snapshot job then in turn enqueues a job for each extractor needed on that URL.
Each extractor job then runs archivebox extract --method=... internally to write the output into the final archive directory.

Originally created by @pirate on GitHub (Dec 15, 2023). Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/1289 I want each thing that ArchiveBox does internally to be a subcommand exposed directly to users or callable by the Huey job queue system. It will also have the side effect of making the process tree nicely represent what ArchiveBox is doing internally, and allow users to kill stuck subtasks independently without stopping the entire "add" import. --- The old `oneshot` will be renamed and joined by a new command to run a single extractor method: #### `archivebox snapshot` Can be run to snapshot an individual URL into the current directory (runs all extractors by default). ```bash archivebox snapshot --methods=all 'https://example.com/somepage.html' # creates a subfolder for each extractor method, and an index.html and index.json file in $PWD ``` This works the same way as `oneshot` does now, and I'll alias `oneshot` to the new command so we don't break backwards compatibility. #### `archivebox extract` This runs an individual extractor method and outputs into the current directory. ```bash archivebox extract --method=PDF --method-args-here 'https://example.com/somepage.html' # writes output.pdf (and an index.json containing cmd+output for each run) into $PWD using the headless browser ``` --- After the refactor, `archivebox add` will work by internally enqueuing a job that runs `archivebox snapshot ...` for each imported URL. The snapshot job then in turn enqueues a job for each extractor needed on that URL. Each extractor job then runs `archivebox extract --method=...` internally to write the output into the final archive directory.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/ArchiveBox#3814

No description provided.

Rows
Columns

[GH-ISSUE #1289] Rename archivebox oneshot -> archivebox snapshot & archivebox extract, expose atomic subtasks for future job queue system #3814

archivebox snapshot

archivebox extract

[GH-ISSUE #1289] Rename `archivebox oneshot` -> `archivebox snapshot` & `archivebox extract`, expose atomic subtasks for future job queue system #3814

`archivebox snapshot`

`archivebox extract`