[GH-ISSUE #164] Does it follow links to archive? #113

New issue

Closed

opened 2026-03-01 14:40:43 +03:00 by kerem · 2 comments

kerem commented

2026-03-01 14:40:43 +03:00

Owner

Originally created by @rajaravivarma-r on GitHub (Mar 8, 2019).
Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/164

Type:

General Question or Disussion
Propose a brand new feature
Request modification of existing behavior or design

What is the problem that your feature request solves
Not really sure if there is a configuration which allows this behavior but I can't find one by checking wikis or issues. I want to archive all the pages of a website. I don't want to archive the links which points to other websites, but if I archive something in github.com I want all the links in that to be available offline.

Describe the ideal specific solution you'd want, and whether it fits into any broader scope of changes
A configuration to enable following links to archive.

What hacks or alternative solutions have you tried to solve the problem?
None

How badly do you want this new feature?

It's an urgent deal-breaker, I cant live without it
It's important to add it in the near-mid term future
It would be nice to have eventualy
I'm willing to contribute to development

Originally created by @rajaravivarma-r on GitHub (Mar 8, 2019). Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/164 Type: - [x] General Question or Disussion - [ ] Propose a brand new feature - [ ] Request modification of existing behavior or design **What is the problem that your feature request solves** Not really sure if there is a configuration which allows this behavior but I can't find one by checking wikis or issues. I want to archive all the pages of a website. I don't want to archive the links which points to other websites, but if I archive something in `github.com` I want all the links in that to be available offline. **Describe the ideal specific solution you'd want, and whether it fits into any broader scope of changes** A configuration to enable following links to archive. **What hacks or alternative solutions have you tried to solve the problem?** None **How badly do you want this new feature?** - [ ] It's an urgent deal-breaker, I cant live without it - [x] It's important to add it in the near-mid term future - [ ] It would be nice to have eventualy - [x] I'm willing to contribute to development

kerem closed this issue

2026-03-01 14:40:43 +03:00

kerem commented

2026-03-01 14:40:44 +03:00

Author

Owner

@pirate commented on GitHub (Mar 8, 2019):

Thanks for the suggestion!

This is not currently supported by ArchiveBox, but it's definitely something we've already planned on adding in the future. You can see it demonstrated on the archivebox add --mirror example line in the Roadmap: https://github.com/pirate/ArchiveBox/issues/120

If you need this funcitonality right now, you can already do it manually with the wget --recursive option: https://www.gnu.org/software/wget/manual/wget.html#Recursive-Retrieval-Options or an app like SiteSucker.

@pirate commented on GitHub (Mar 8, 2019): Thanks for the suggestion! This is not currently supported by ArchiveBox, but it's definitely something we've already planned on adding in the future. You can see it demonstrated on the `archivebox add --mirror` example line in the Roadmap: https://github.com/pirate/ArchiveBox/issues/120 If you need this funcitonality right now, you can already do it manually with the `wget --recursive` option: https://www.gnu.org/software/wget/manual/wget.html#Recursive-Retrieval-Options or an app like [SiteSucker](https://itunes.apple.com/us/app/sitesucker/id442168834?mt=12).

kerem commented

2026-03-01 14:40:44 +03:00

Author

Owner

@rajaravivarma-r commented on GitHub (Mar 9, 2019):

@pirate Thanks.

@rajaravivarma-r commented on GitHub (Mar 9, 2019): @pirate Thanks.

kerem referenced this issue

2026-03-01 17:51:22 +03:00

[GH-ISSUE #6] Archive Method: Add WARC file output #1516

kerem referenced this issue

2026-03-01 17:51:59 +03:00