[GH-ISSUE #690] Bug: Installing missing 3 recommended dependencies never completes #1941

New issue

Closed

opened 2026-03-01 17:55:12 +03:00 by kerem · 8 comments

kerem commented 2026-03-01 17:55:12 +03:00

Owner

Copy link

Originally created by @rxw1 on GitHub (Apr 3, 2021).
Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/690

$ npm install --prefix . "git+https://github.com/ArchiveBox/ArchiveBox.git"
npm WARN deprecated request-promise-native@1.0.9: request-promise-native has been deprecated because it extends the now deprecated request package, see https://github.com/request/request/issues/3142
npm WARN deprecated har-validator@5.1.5: this library is no longer supported
npm WARN deprecated request-promise@4.2.6: request-promise has been deprecated because it extends the now deprecated request package, see https://github.com/request/request/issues/3142
npm WARN deprecated left-pad@1.3.0: use String.prototype.padStart()
npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated core-js@2.6.12: core-js@<3 is no longer maintained and not recommended for usage due to the number of issues. Please, upgrade your dependencies to the actual version of core-js@3.
[                  ] / reify:single-file: timing reifyNode:node_modules/core-js Completed in 2177ms

npm install --prefix . "git+https://github.com/ArchiveBox/ArchiveBox.git" does never complete and hangs at the last line above.

1 archivebox bla
2 cd bla
3 npm install --prefix . "git+https://github.com/ArchiveBox/ArchiveBox.git"

ArchiveBox version

ArchiveBox v0.5.4
Cpython Linux Linux-5.11.10-gentoo-x86_64-x86_64-Intel-R-_Core-TM-_i7-4790K_CPU_@_4.00GHz-with-glibc2.2.5 x86_64 (not in Docker)

[i] Dependency versions:
 √  ARCHIVEBOX_BINARY     v0.5.4          valid     /home/asdf/.python/bin/archivebox
 √  PYTHON_BINARY         v3.8.8          valid     /usr/bin/python3.8
 √  DJANGO_BINARY         v3.1.3          valid     /home/asdf/.python/lib/python3.8/site-packages/django/bin/django-admin.py
 √  CURL_BINARY           v7.74.0         valid     /usr/bin/curl
 √  WGET_BINARY           v1.20.3         valid     /usr/bin/wget
 √  NODE_BINARY           v15.11.0        valid     /usr/bin/node
 X  SINGLEFILE_BINARY     ?               invalid   single-file
 X  READABILITY_BINARY    ?               invalid   readability-extractor
 X  MERCURY_BINARY        ?               invalid   mercury-parser
 √  GIT_BINARY            v2.26.3         valid     /usr/bin/git
 √  YOUTUBEDL_BINARY      v2021.04.01     valid     /home/asdf/.python/bin/youtube-dl
 √  CHROME_BINARY         v90.0.4430.51   valid     /usr/bin/google-chrome-beta
 √  RIPGREP_BINARY        v0.10.0         valid     /usr/local/bin/rg

[i] Source-code locations:
 √  PACKAGE_DIR           23 files        valid     /home/asdf/.python/lib/python3.8/site-packages/archivebox
 √  TEMPLATES_DIR         3 files         valid     /home/asdf/.python/lib/python3.8/site-packages/archivebox/templates

[i] Secrets locations:
 -  CHROME_USER_DATA_DIR  -               disabled
 -  COOKIES_FILE          -               disabled

[i] Data locations:
 √  OUTPUT_DIR            9 files         valid     /mnt/unsort/archive/www
 √  SOURCES_DIR           108 files       valid     ./sources
 √  LOGS_DIR              0 files         valid     ./logs
 √  ARCHIVE_DIR           87 files        valid     ./archive
 √  CONFIG_FILE           81.0 Bytes      valid     ./ArchiveBox.conf
 √  SQL_INDEX             892.0 KB        valid     ./index.sqlite3

[!] Warning: Missing 3 recommended dependencies
    ! SINGLEFILE_BINARY: single-file (unable to detect version)
      Hint: npm install --prefix . "git+https://github.com/ArchiveBox/ArchiveBox.git"
            or archivebox config --set SAVE_SINGLEFILE=False to silence this warning

    ! READABILITY_BINARY: readability-extractor (unable to detect version)
      Hint: npm install --prefix . "git+https://github.com/ArchiveBox/ArchiveBox.git"
            or archivebox config --set SAVE_READABILITY=False to silence this warning

    ! MERCURY_BINARY: mercury-parser (unable to detect version)
      Hint: npm install --prefix . "git+https://github.com/ArchiveBox/ArchiveBox.git"
            or archivebox config --set SAVE_MERCURY=False to silence this warning

thx

Originally created by @rxw1 on GitHub (Apr 3, 2021). Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/690 ``` $ npm install --prefix . "git+https://github.com/ArchiveBox/ArchiveBox.git" npm WARN deprecated request-promise-native@1.0.9: request-promise-native has been deprecated because it extends the now deprecated request package, see https://github.com/request/request/issues/3142 npm WARN deprecated har-validator@5.1.5: this library is no longer supported npm WARN deprecated request-promise@4.2.6: request-promise has been deprecated because it extends the now deprecated request package, see https://github.com/request/request/issues/3142 npm WARN deprecated left-pad@1.3.0: use String.prototype.padStart() npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142 npm WARN deprecated core-js@2.6.12: core-js@<3 is no longer maintained and not recommended for usage due to the number of issues. Please, upgrade your dependencies to the actual version of core-js@3. [ ] / reify:single-file: timing reifyNode:node_modules/core-js Completed in 2177ms ``` `npm install --prefix . "git+https://github.com/ArchiveBox/ArchiveBox.git"` does never complete and hangs at the last line above. 1 `archivebox bla` 2 `cd bla` 3 `npm install --prefix . "git+https://github.com/ArchiveBox/ArchiveBox.git"`  #### ArchiveBox version ``` ArchiveBox v0.5.4 Cpython Linux Linux-5.11.10-gentoo-x86_64-x86_64-Intel-R-_Core-TM-_i7-4790K_CPU_@_4.00GHz-with-glibc2.2.5 x86_64 (not in Docker) [i] Dependency versions: √ ARCHIVEBOX_BINARY v0.5.4 valid /home/asdf/.python/bin/archivebox √ PYTHON_BINARY v3.8.8 valid /usr/bin/python3.8 √ DJANGO_BINARY v3.1.3 valid /home/asdf/.python/lib/python3.8/site-packages/django/bin/django-admin.py √ CURL_BINARY v7.74.0 valid /usr/bin/curl √ WGET_BINARY v1.20.3 valid /usr/bin/wget √ NODE_BINARY v15.11.0 valid /usr/bin/node X SINGLEFILE_BINARY ? invalid single-file X READABILITY_BINARY ? invalid readability-extractor X MERCURY_BINARY ? invalid mercury-parser √ GIT_BINARY v2.26.3 valid /usr/bin/git √ YOUTUBEDL_BINARY v2021.04.01 valid /home/asdf/.python/bin/youtube-dl √ CHROME_BINARY v90.0.4430.51 valid /usr/bin/google-chrome-beta √ RIPGREP_BINARY v0.10.0 valid /usr/local/bin/rg [i] Source-code locations: √ PACKAGE_DIR 23 files valid /home/asdf/.python/lib/python3.8/site-packages/archivebox √ TEMPLATES_DIR 3 files valid /home/asdf/.python/lib/python3.8/site-packages/archivebox/templates [i] Secrets locations: - CHROME_USER_DATA_DIR - disabled - COOKIES_FILE - disabled [i] Data locations: √ OUTPUT_DIR 9 files valid /mnt/unsort/archive/www √ SOURCES_DIR 108 files valid ./sources √ LOGS_DIR 0 files valid ./logs √ ARCHIVE_DIR 87 files valid ./archive √ CONFIG_FILE 81.0 Bytes valid ./ArchiveBox.conf √ SQL_INDEX 892.0 KB valid ./index.sqlite3 [!] Warning: Missing 3 recommended dependencies ! SINGLEFILE_BINARY: single-file (unable to detect version) Hint: npm install --prefix . "git+https://github.com/ArchiveBox/ArchiveBox.git" or archivebox config --set SAVE_SINGLEFILE=False to silence this warning ! READABILITY_BINARY: readability-extractor (unable to detect version) Hint: npm install --prefix . "git+https://github.com/ArchiveBox/ArchiveBox.git" or archivebox config --set SAVE_READABILITY=False to silence this warning ! MERCURY_BINARY: mercury-parser (unable to detect version) Hint: npm install --prefix . "git+https://github.com/ArchiveBox/ArchiveBox.git" or archivebox config --set SAVE_MERCURY=False to silence this warning ``` thx

kerem closed this issue 2026-03-01 17:55:12 +03:00

kerem commented 2026-03-01 17:55:13 +03:00

Author

Owner

Copy link

@pirate commented on GitHub (Apr 3, 2021):

Strange, possibly a network issue, upstream dependency issue in one of the packages, or something hanging the npm audit process? OR maybe it's the new postinstall step I added recently that tries to run python3 -m pip install archivebox after the npm install completes if it doesn't detect the archivebox python package already installed.

npm install --prefix . "git+https://github.com/ArchiveBox/ArchiveBox.git" is a handy shortcut for installing the 3 actual JS dependencies, can you try installing them individually instead to see which one is causing the hang?

npm install --no-save --no-audit --no-fund --loglevel=error git+https://github.com/gildas-lormeau/SingleFile.git
npm install --no-save --no-audit --no-fund --loglevel=error git+https://github.com/ArchiveBox/readability-extractor.git
npm install --no-save --no-audit --no-fund --loglevel=error @postlight/mercury-parser

<!-- gh-comment-id:812913404 --> @pirate commented on GitHub (Apr 3, 2021): Strange, possibly a network issue, upstream dependency issue in one of the packages, or something hanging the npm audit process? OR maybe it's the new `postinstall` step I added recently that tries to run `python3 -m pip install archivebox` after the npm install completes if it doesn't detect the archivebox python package already installed. `npm install --prefix . "git+https://github.com/ArchiveBox/ArchiveBox.git"` is a handy shortcut for installing the 3 actual JS dependencies, can you try installing them individually instead to see which one is causing the hang? ```bash npm install --no-save --no-audit --no-fund --loglevel=error git+https://github.com/gildas-lormeau/SingleFile.git npm install --no-save --no-audit --no-fund --loglevel=error git+https://github.com/ArchiveBox/readability-extractor.git npm install --no-save --no-audit --no-fund --loglevel=error @postlight/mercury-parser ```

kerem commented 2026-03-01 17:55:13 +03:00

Author

Owner

Copy link

@dunnno commented on GitHub (Apr 5, 2021):

That's the problem I face as well : https://github.com/ArchiveBox/ArchiveBox/discussions/687#discussioncomment-553225

I ran all the 3 commands but only the last one remain valid. ==> I ran the SingleFile install command, it gets installed, I ran the readability one, it gets installed, but now SingleFile becomes invalid.

<!-- gh-comment-id:813481172 --> @dunnno commented on GitHub (Apr 5, 2021): That's the problem I face as well : https://github.com/ArchiveBox/ArchiveBox/discussions/687#discussioncomment-553225 I ran all the 3 commands but only the last one remain valid. ==> I ran the SingleFile install command, it gets installed, I ran the readability one, it gets installed, but now SingleFile becomes invalid.

kerem commented 2026-03-01 17:55:13 +03:00

Author

Owner

Copy link

@pirate commented on GitHub (Apr 5, 2021):

Delete any package.json or package-lock.json present in your data folder and re-run those commands. It's still trying to install the broken archivebox npm package after those commands because it saved it into a lockfile earlier.

<!-- gh-comment-id:813505651 --> @pirate commented on GitHub (Apr 5, 2021): Delete any package.json or package-lock.json present in your data folder and re-run those commands. It's still trying to install the broken archivebox npm package after those commands because it saved it into a lockfile earlier.

kerem commented 2026-03-01 17:55:13 +03:00

Author

Owner

Copy link

@dunnno commented on GitHub (Apr 5, 2021):

I ran again the single file line in order to get it installed. Readability and mercury aren't installed.

Unless I look at the wrong place, I don't see any readability or mercury folders in the node_modules folder :/

<!-- gh-comment-id:813582598 --> @dunnno commented on GitHub (Apr 5, 2021): I ran again the single file line in order to get it installed. Readability and mercury aren't installed. Unless I look at the wrong place, I don't see any readability or mercury folders in the node_modules folder :/

kerem commented 2026-03-01 17:55:13 +03:00

Author

Owner

Copy link

@pirate commented on GitHub (Apr 5, 2021):

Just to confirm, you tried running exactly these commands in this order?

rm -Rf package.json package-lock.json node_modules
npm install --no-save --no-audit --no-fund --loglevel=error git+https://github.com/gildas-lormeau/SingleFile.git
npm install --no-save --no-audit --no-fund --loglevel=error git+https://github.com/ArchiveBox/readability-extractor.git
npm install --no-save --no-audit --no-fund --loglevel=error @postlight/mercury-parser

And afterwards node_modules only contains one of them?

<!-- gh-comment-id:813606726 --> @pirate commented on GitHub (Apr 5, 2021): Just to confirm, you tried running exactly these commands in this order? ```bash rm -Rf package.json package-lock.json node_modules npm install --no-save --no-audit --no-fund --loglevel=error git+https://github.com/gildas-lormeau/SingleFile.git npm install --no-save --no-audit --no-fund --loglevel=error git+https://github.com/ArchiveBox/readability-extractor.git npm install --no-save --no-audit --no-fund --loglevel=error @postlight/mercury-parser ``` And afterwards `node_modules` only contains one of them?

kerem commented 2026-03-01 17:55:13 +03:00

Author

Owner

Copy link

@dunnno commented on GitHub (Apr 5, 2021):

Yep, tried it again, singlefile installs fine, readability installs fine, but singlefile got uninstalled, and so on with readability when doing mercury

<!-- gh-comment-id:813639582 --> @dunnno commented on GitHub (Apr 5, 2021): Yep, tried it again, singlefile installs fine, readability installs fine, but singlefile got uninstalled, and so on with readability when doing mercury

kerem commented 2026-03-01 17:55:13 +03:00

Author

Owner

Copy link

@pirate commented on GitHub (Apr 5, 2021):

Arg, it turns out this is a difference in npm behavior between mac and linux. On mac it will install all 3, on linux it will overwrite the previous node_modules each time. 😡

On linux you'll have to use --save instead of --no-save:

rm -Rf package.json package-lock.json node_modules
npm install --save --no-audit --no-fund --loglevel=error git+https://github.com/gildas-lormeau/SingleFile.git
npm install --save --no-audit --no-fund --loglevel=error git+https://github.com/ArchiveBox/readability-extractor.git
npm install --save --no-audit --no-fund --loglevel=error @postlight/mercury-parser

<!-- gh-comment-id:813646826 --> @pirate commented on GitHub (Apr 5, 2021): Arg, it turns out this is a difference in `npm` behavior between mac and linux. On mac it will install all 3, on linux it will overwrite the previous node_modules each time. 😡 <img width="400px" src="https://user-images.githubusercontent.com/511499/113625666-33b41000-962f-11eb-81f2-cf1ad261074a.png"> <img width="400px" src="https://user-images.githubusercontent.com/511499/113625716-47f80d00-962f-11eb-9894-9d97aa8cedfb.png"> On linux you'll have to use `--save` instead of `--no-save`: ```bash rm -Rf package.json package-lock.json node_modules npm install --save --no-audit --no-fund --loglevel=error git+https://github.com/gildas-lormeau/SingleFile.git npm install --save --no-audit --no-fund --loglevel=error git+https://github.com/ArchiveBox/readability-extractor.git npm install --save --no-audit --no-fund --loglevel=error @postlight/mercury-parser ``` <img width="400px" src="https://user-images.githubusercontent.com/511499/113626490-3e22d980-9630-11eb-8670-b40fcdfd81cb.png">

kerem commented 2026-03-01 17:55:13 +03:00

Author

Owner

Copy link

@dunnno commented on GitHub (Apr 5, 2021):

Well, that was it, that did the trick, thanks a bunch :)

<!-- gh-comment-id:813648827 --> @dunnno commented on GitHub (Apr 5, 2021): Well, that was it, that did the trick, thanks a bunch :)

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

dev

main

abx-dl-refactor

claude/issue-1688-20251229-2233

claude/s3-support-migration-JZQcf

claude/review-snapshot-archive-RJOkr

claude/review-code-quality-macdO

claude/reduce-chrome-duplication-JgWVr

fix/chrome-args-comma-parsing

claude/issue-668-20251229-2145

claude/issue-1664-20251229-2236

v086

claude/improve-test-suite-xm6Bh

claude/typescript-archivebox-rewrite-011CUmPyEmECnkXwyHRT9RPF

claude/archivebox-schema-orm-comparison-011CUn26GurEpLiBUq4zjVLJ

stable

logfire

plugins-browsertrix

v072

link-removal2

v0.8.6rc1

v0.7.3

v0.8.6rc0

v0.8.5rc53

v0.8.5rc51

v0.8.5rc50

v0.8.5rc49

v0.8.5rc48

v0.8.5rc47

v0.8.5rc46

v0.8.5rc45

v0.8.5rc44

v0.8.5rc43

v0.8.5rc42

v0.8.5rc41

v0.8.5rc40

v0.8.5rc39

v0.8.5rc38

v0.8.5rc37

v0.8.5rc36

v0.8.5rc35

v0.8.5rc34

v0.8.5rc33

v0.8.5rc32

v0.8.5rc31

v0.8.5rc30

v0.8.5rc29

v0.8.5rc28

v0.8.5rc27

v0.8.5rc26

v0.8.5rc25

v0.8.5rc24

v0.8.5rc23

v0.8.5rc22

v0.8.5rc13

v0.8.5rc12

v0.8.5rc11

v0.8.5rc10

v0.8.5rc9

v0.8.5rc8

v0.8.5rc6

v0.8.5rc5

v0.8.5rc4

v0.8.5rc3

v0.8.5rc2

v0.8.5-rc

v0.8.4-rc

v0.8.3-rc

v0.8.2-rc

v0.8.0-rc

v0.7.2

v0.7.1

v0.7.0

v0.6.2

v0.6.0

v0.5.6

v0.5.4

v0.5.3

v0.4.24

v0.4.21

v0.4.20

v0.4.19

v0.4.18

v0.4.17

v0.4.16

v0.4.15

v0.4.14

v0.4.13

v0.4.12

v0.4.11

v0.4.9

v0.2.4

v0.2.3

v0.2.2

v0.2.1

v0.2.0

v0.1.0

v0.0.3

v0.0.2

v0.0.1

Labels

Clear labels   

expected: maybe someday

  

expected: next release

  

expected: release after next

  

expected: unlikely unless contributed

  

good first ticket

  

help wanted

  

pull-request

Mirrored from GitHub Pull Request

  

scope: all users

  

scope: windows users

  

size: easy

  

size: hard

  

size: medium

  

size: medium

  

status: backlog

  

status: blocked

  

status: done

  

status: idea-phase

  

status: needs followup

  

status: wip

  

status: wontfix

  

touches: API/CLI/Spec

  

touches: configuration

  

touches: data/schema/architecture

  

touches: dependencies/packaging

  

touches: docs

  

touches: js

  

touches: views/replayers/html/css

  

why: correctness

  

why: functionality

  

why: performance

  

why: security

No labels

expected: maybe someday

expected: next release

expected: release after next

expected: unlikely unless contributed

good first ticket

help wanted

pull-request

scope: all users

scope: windows users

size: easy

size: hard

size: medium

size: medium

status: backlog

status: blocked

status: done

status: idea-phase

status: needs followup

status: wip

status: wontfix

touches: API/CLI/Spec

touches: configuration

touches: data/schema/architecture

touches: dependencies/packaging

touches: docs

touches: js

touches: views/replayers/html/css

why: correctness

why: functionality

why: performance

why: security

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/ArchiveBox#1941

No description provided.