[GH-ISSUE #767] Feature Request: Use of tools like gallery-dl to archive specific websites. #1996

New issue

Closed

opened 2026-03-01 17:55:41 +03:00 by kerem · 1 comment

kerem commented

2026-03-01 17:55:41 +03:00

Owner

Originally created by @TheAnachronism on GitHub (Jun 10, 2021).
Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/767

Type

General question or discussion
Propose a brand new feature
Request modification of existing behavior or design

What is the problem that your feature request solves

Archiving stuff from sites like DeviantArt is very difficult if not impossible because of how those sites kind of prevent such crawling or downloading the content by a simple wget.

Describe the ideal specific solution you'd want, and whether it fits into any broader scope of changes

Tools like gallery-dl solve this issue because they're made to download and archive content from those sites.

What hacks or alternative solutions have you tried to solve the problem?

The only thing that would come to mind is to download the site manually and then somehow upload it to ArchiveBox. But because WARC upload isn't supported yet this gets difficult

How badly do you want this new feature?

It's an urgent deal-breaker, I can't live without it
It's important to add it in the near-mid term future
It would be nice to have eventually

I'm willing to contribute dev time / money to fix this issue
I like ArchiveBox so far / would recommend it to a friend
I've had a lot of difficulty getting ArchiveBox set up

Originally created by @TheAnachronism on GitHub (Jun 10, 2021). Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/767 ## Type - [ ] General question or discussion - [x] Propose a brand new feature - [ ] Request modification of existing behavior or design ## What is the problem that your feature request solves Archiving stuff from sites like DeviantArt is very difficult if not impossible because of how those sites kind of prevent such crawling or downloading the content by a simple wget. ## Describe the ideal specific solution you'd want, and whether it fits into any broader scope of changes Tools like gallery-dl solve this issue because they're made to download and archive content from those sites. ## What hacks or alternative solutions have you tried to solve the problem? The only thing that would come to mind is to download the site manually and then somehow upload it to ArchiveBox. But because WARC upload isn't supported yet this gets difficult ## How badly do you want this new feature? - [ ] It's an urgent deal-breaker, I can't live without it - [x] It's important to add it in the near-mid term future - [x] It would be nice to have eventually --- - [x] I'm willing to contribute [dev time](https://github.com/ArchiveBox/ArchiveBox#archivebox-development) / [money](https://github.com/sponsors/pirate) to fix this issue - [x] I like ArchiveBox so far / would recommend it to a friend - [ ] I've had a lot of difficulty getting ArchiveBox set up

kerem closed this issue

2026-03-01 17:55:41 +03:00

kerem commented

2026-03-01 17:55:42 +03:00

Author

Owner

@pirate commented on GitHub (Jun 11, 2021):

Duplicate of: https://github.com/ArchiveBox/ArchiveBox/issues/564

@pirate commented on GitHub (Jun 11, 2021): Duplicate of: https://github.com/ArchiveBox/ArchiveBox/issues/564

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/ArchiveBox#1996

No description provided.

Rows
Columns