[GH-ISSUE #318] Feature Request: Add SingleFile CLI option to fire scroll event and load deferred images to better support WeChat archiving

kerem commented

2026-03-14 21:46:22 +03:00

Owner

Originally created by @KagurazakaShirosatosu on GitHub (Feb 2, 2020).
Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/318

Type

General question or discussion
Propose a brand new feature
Request modification of existing behavior or design

What is the problem that your feature request solves

I archived a page form Wechat Open Platform (such as https://mp.weixin.qq.com/s/ri4nDgPQo4OVWaIWG9EQZA)
but I found that all images on the page can't be archived.
(https://archive.sager.wang/archive/1580637904/mp.weixin.qq.com/s/ri4nDgPQo4OVWaIWG9EQZA.html)

and the title also show "Unable to detect page title" in index of the archive box.

WeChat is the biggest IM in China and it has the strictest censor there.
So I am hopping archive box can archive the page with images.

I am sorry for my bad English :-)

Describe the ideal specific solution you'd want, and whether it fits into any broader scope of changes

I hope archive box can archive the page (with images) from Wechat Open Platform.

What hacks or alternative solutions have you tried to solve the problem?

archive.is can archive the page from wechat open platform with images.

How badly do you want this new feature?

It's an urgent deal-breaker, I can't live without it
It's important to add it in the near-mid term future
It would be nice to have eventually

I'm willing to contribute dev time / money to fix this issue
I like ArchiveBox so far / would recommend it to a friend
I've had a lot of difficulty getting ArchiveBox set up

Originally created by @KagurazakaShirosatosu on GitHub (Feb 2, 2020). Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/318  ## Type - [ ] General question or discussion - [ ] Propose a brand new feature - [x] Request modification of existing behavior or design ## What is the problem that your feature request solves I archived a page form Wechat Open Platform (such as https://mp.weixin.qq.com/s/ri4nDgPQo4OVWaIWG9EQZA) but I found that all images on the page can't be archived. (https://archive.sager.wang/archive/1580637904/mp.weixin.qq.com/s/ri4nDgPQo4OVWaIWG9EQZA.html) and the title also show "Unable to detect page title" in index of the archive box. WeChat is the biggest IM in China and it has the strictest censor there. So I am hopping archive box can archive the page with images. I am sorry for my bad English :-) ## Describe the ideal specific solution you'd want, and whether it fits into any broader scope of changes I hope archive box can archive the page (with images) from Wechat Open Platform. ## What hacks or alternative solutions have you tried to solve the problem? archive.is can archive the page from wechat open platform with images. ## How badly do you want this new feature? - [ ] It's an urgent deal-breaker, I can't live without it - [x] It's important to add it in the near-mid term future - [ ] It would be nice to have eventually --- - [ ] I'm willing to contribute dev time / money to fix this issue - [x] I like ArchiveBox so far / would recommend it to a friend - [ ] I've had a lot of difficulty getting ArchiveBox set up

kerem added the

touches: dependencies/packaging

labels

2026-03-14 21:46:22 +03:00

kerem commented

2026-03-14 21:46:33 +03:00

Author

Owner

@wych42 commented on GitHub (Mar 22, 2020):

Unable to detect page title

It's caused by wechat media platform(article) page has an empty

@wych42 commented on GitHub (Mar 22, 2020): > Unable to detect page title It's caused by wechat media platform(article) page has an empty <title> field. Maybe archivebox could try to parse og:title twitter:title fields to get title. title screenshot https://www.dropbox.com/s/k90irdvcdicl986/Screenshot%202020-03-22%2023.04.16.png?dl=0

kerem commented

2026-03-14 21:46:38 +03:00

Author

Owner

@cdvv7788 commented on GitHub (Jul 17, 2020):

@pirate the title is not present, is it ok to look in the og:title tag as a fallback? (the retrieval is stopping because of this)
About the images, they are being lazy-loaded. I am not sure if wget can handle that, but that is something we can check after the title issue is fixed.

@cdvv7788 commented on GitHub (Jul 17, 2020): @pirate the title is not present, is it ok to look in the `og:title` tag as a fallback? (the retrieval is stopping because of this) About the images, they are being lazy-loaded. I am not sure if wget can handle that, but that is something we can check after the title issue is fixed.

kerem commented

2026-03-14 21:46:44 +03:00

Author

Owner

@pirate commented on GitHub (Jul 17, 2020):

Yeah you can look in og:title, but lets not handle the image lazy loading right now, that's a very complex problem.

@pirate commented on GitHub (Jul 17, 2020): Yeah you can look in og:title, but lets not handle the image lazy loading right now, that's a very complex problem.

kerem commented

2026-03-14 21:46:49 +03:00

Author

Owner

@hope1 commented on GitHub (Jan 11, 2022):

Greetings. I wonder if it is possible at this point to revisit this problem?

Given the prominence of Wechat (and the most comprehensive censoring, as OP also mentioned), I'd wager that articles on mp.weixin.qq.com are probably the most common target for archiving for users in China. It certainly is the case for me, as most of the articles I have felt a need to archive are on there. It would be wonderful to have ArchiveBox available for this usage, especially as the archive.* sites now block all web host proxy users.

Apologies for digging up an old thread and thank you all for your hard work.

@hope1 commented on GitHub (Jan 11, 2022): Greetings. I wonder if it is possible at this point to revisit this problem? Given the prominence of Wechat (and the most comprehensive censoring, as OP also mentioned), I'd wager that articles on mp.weixin.qq.com are probably the most common target for archiving for users in China. It certainly is the case for me, as most of the articles I have felt a need to archive are on there. It would be wonderful to have ArchiveBox available for this usage, especially as the archive.* sites now block all web host proxy users. Apologies for digging up an old thread and thank you all for your hard work.

kerem commented

2026-03-14 21:46:54 +03:00

Author

Owner

@pirate commented on GitHub (Jan 11, 2022):

I recommend seeing if they can be archived with SingleFile, and if not, raising the issue on that repo. ArchiveBox does not itself do any archiving, it's just a collection of other utilities that do the actual archiving. If there are issues with archive fidelity in general the issue is to raise those issues with the sub-utilities or add a new extractor.

@pirate commented on GitHub (Jan 11, 2022): I recommend seeing if they can be archived with SingleFile, and if not, raising the issue on that repo. ArchiveBox does not itself do any archiving, it's just a collection of other utilities that do the actual archiving. If there are issues with archive fidelity in general the issue is to raise those issues with the sub-utilities or add a new extractor.

kerem commented

2026-03-14 21:46:59 +03:00

Author

Owner

@KagurazakaShirosatosu commented on GitHub (Jan 19, 2022):

Hi,
I found that mp.weixin.qq.com can be captured by SingleFile including pictures if I scroll down the entire webpage manually and wait all pictures finish loading. I think ArchiveBox can scroll to the bottom of the page, then wait for networkidle0 and then call SignleFile to capture it.

@KagurazakaShirosatosu commented on GitHub (Jan 19, 2022): Hi, I found that mp.weixin.qq.com can be captured by SingleFile including pictures if I scroll down the entire webpage manually and wait all pictures finish loading. I think ArchiveBox can scroll to the bottom of the page, then wait for networkidle0 and then call SignleFile to capture it.

kerem commented

2026-03-14 21:47:04 +03:00

Author

Owner

@pirate commented on GitHub (Jan 22, 2022):

ArchiveBox does not have granular control over how the singlefile capture is done, we only call the SingleFile CLI. Unless they provide a CLI option to scroll before capturing, we cannot do that.

@pirate commented on GitHub (Jan 22, 2022): ArchiveBox does not have granular control over how the singlefile capture is done, we only call the SingleFile CLI. Unless they provide a CLI option to scroll before capturing, we cannot do that.

kerem commented

2026-03-14 21:47:09 +03:00

Author

Owner

@canoziia commented on GitHub (Dec 1, 2022):

Hello, I found that SingleFile CLI has an option --load-deferred-images-dispatch-scroll-event. When it is enable, lazy-loaded images can be saved perfectly (at least on WeChat's Page).
https://github.com/gildas-lormeau/single-file-cli/blob/master/args.js#L206

@canoziia commented on GitHub (Dec 1, 2022): Hello, I found that SingleFile CLI has an option `--load-deferred-images-dispatch-scroll-event`. When it is enable, lazy-loaded images can be saved perfectly (at least on WeChat's Page). https://github.com/gildas-lormeau/single-file-cli/blob/master/args.js#L206

kerem commented

2026-03-14 21:47:14 +03:00

Author

Owner

@pirate commented on GitHub (Dec 3, 2022):

Thats a great find! 🥳 Thanks. Lets add that option to archivebox by default then.

@pirate commented on GitHub (Dec 3, 2022): Thats a great find! 🥳 Thanks. Lets add that option to archivebox by default then.

kerem commented

2026-03-14 21:47:19 +03:00

Author

Owner

@melyux commented on GitHub (Jul 16, 2023):

On a lot of my pages, turning --load-deferred-images-dispatch-scroll-event to true causes some "subscribe to my newsletter" popup to come up while having it off prevents this and still loads deferred images. So it probably shouldn't be a default.

This is all moot though because the bundled singlefile is outdated and doesn't support deferred images at all right now

@melyux commented on GitHub (Jul 16, 2023): On a lot of my pages, turning `--load-deferred-images-dispatch-scroll-event` to true causes some "subscribe to my newsletter" popup to come up while having it off prevents this and still loads deferred images. So it probably shouldn't be a default. This is all moot though because the bundled singlefile is outdated and doesn't support deferred images at all right now

Rows
Columns

[GH-ISSUE #318] Feature Request: Add SingleFile CLI option to fire scroll event and load deferred images to better support WeChat archiving #3251

Type

What is the problem that your feature request solves

Describe the ideal specific solution you'd want, and whether it fits into any broader scope of changes

What hacks or alternative solutions have you tried to solve the problem?

How badly do you want this new feature?