starred/ArchiveBox
1
0
Fork 0
mirror of https://github.com/ArchiveBox/ArchiveBox.git synced 2026-04-25 17:16:00 +03:00
Code Issues 624 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #463] Bugfix: UnhandledPromiseRejectionWarning with singlefile attempt #308

New issue
Closed
opened 2026-03-01 14:42:19 +03:00 by kerem · 9 comments
kerem commented 2026-03-01 14:42:19 +03:00
Owner
Copy link

Originally created by @drpfenderson on GitHub (Sep 2, 2020).
Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/463

Describe the bug

Decided to move forward with a clean archive, and keep the old one as a historical snapshot. Added a new list of links to archive, but after 6 or 7, almost all of the tasks start erroring out in cascade fashion, starting with singlefile step. Error is TimeoutExpired Command for chromium-browser, but it gives me a command to run, which also errors out with (node:162485) UnhandledPromiseRejectionWarning: SyntaxError: Unexpected number in JSON at position 2

Steps to reproduce

  1. Installed with pip install archivebox
  2. Changed config to disable MEDIA and timeout set to 180 seconds.
  3. Added list of links with archivebox add ./list.txt.
  4. Starts adding things correctly, but after link 5 or 6 starts to error out.

Screenshots or log output

Original error

[i] [2020-09-02 17:41:18] ArchiveBox v0.4.21: archivebox update
    > /mnt/volume/.archivebox-output/new-archive


[*] [2020-09-02 17:41:21] Writing 33 links to main index...
    √ /mnt/volume/.archivebox-output/new-archive/index.sqlite3
    √ /mnt/volume/.archivebox-output/new-archive/index.json
    √ /mnt/volume/.archivebox-output/new-archive/index.html

[▶] [2020-09-02 17:41:23] Collecting content for 8 Snapshots in archive...

[√] [2020-09-02 17:44:28] "NYPD used facial recognition to track down Black Lives Matter activist - The Verge"
    https://theverge.com/2020/8/18/21373316/nypd-facial-recognition-black-lives-matter-activist-derrick-ingram
    √ ./archive/1598996327.937664
      > singlefile
        Extractor failed:
            TimeoutExpired Command '['/mnt/volume/.archivebox-output/new-archive/node_modules/archivebox/node_modules/single-file/cli/single-file', '--browser-executable-path=chromium-browser', '--browser-args="["--headless", "--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.75 Safari/537.36", "--window-size=1440,2000"]"', 'https://theverge.com/2020/8/18/21373316/nypd-facial-recognition-black-lives-matter-activist-derrick-ingram', '/mnt/volume/.archivebox-output/new-archive/archive/1598996327.937664/singlefile.html']' timed out after 180 seconds
        Run to see full output:
            cd /mnt/volume/.archivebox-output/new-archive/archive/1598996327.937664;
            /mnt/volume/.archivebox-output/new-archive/node_modules/archivebox/node_modules/single-file/cli/single-file --browser-executable-path=chromium-browser "--browser-args="["--headless", "--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.75 Safari/537.36", "--window-size=1440,2000"]"" https://theverge.com/2020/8/18/21373316/nypd-facial-recognition-black-lives-matter-activist-derrick-ingram /mnt/volume/.archivebox-output/new-archive/archive/1598996327.937664/singlefile.html

      > pdf
        Extractor failed:
            Exception Failed to chmod: output.pdf does not exist (did the previous step fail?)
        Run to see full output:
            cd /mnt/volume/.archivebox-output/new-archive/archive/1598996327.937664;
            chromium-browser --headless "--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.75 Safari/537.36" --window-size=1440,2000 --timeout=180000 --print-to-pdf https://theverge.com/2020/8/18/21373316/nypd-facial-recognition-black-lives-matter-activist-derrick-ingram

When I run

cd /mnt/volume/.archivebox-output/new-archive/archive/1598996327.937664;
/mnt/volume/.archivebox-output/new-archive/node_modules/archivebox/node_modules/single-file/cli/single-file --browser-executable-path=chromium-browser "--browser-args="["--headless", "--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.75 Safari/537.36", "--window-size=1440,2000"]"" https://theverge.com/2020/8/18/21373316/nypd-facial-recognition-black-lives-matter-activist-derrick-ingram /mnt/volume/.archivebox-output/new-archive/archive/1598996327.937664/singlefile.html

Output:

(node:162485) UnhandledPromiseRejectionWarning: SyntaxError: Unexpected number in JSON at position 2
    at JSON.parse (<anonymous>)
    at getBrowserOptions (/mnt/volume/.archivebox-output/new-archive/node_modules/archivebox/node_modules/single-file/cli/back-ends/puppeteer.js:63:51)
    at Object.exports.initialize (/mnt/volume/.archivebox-output/new-archive/node_modules/archivebox/node_modules/single-file/cli/back-ends/puppeteer.js:36:35)
    at initialize (/mnt/volume/.archivebox-output/new-archive/node_modules/archivebox/node_modules/single-file/cli/single-file-cli-api.js:46:16)
    at run (/mnt/volume/.archivebox-output/new-archive/node_modules/archivebox/node_modules/single-file/cli/single-file:33:59)
    at Object.<anonymous> (/mnt/volume/.archivebox-output/new-archive/node_modules/archivebox/node_modules/single-file/cli/single-file:30:1)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
(node:162485) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
(node:162485) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Software versions

  • OS: Ubuntu 20.04.1 LTS
  • ArchiveBox version: 0.21
  • Python version: Python 3.7.9
  • Chrome version: Chromium 84.0.4147.105
Originally created by @drpfenderson on GitHub (Sep 2, 2020). Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/463 #### Describe the bug Decided to move forward with a clean archive, and keep the old one as a historical snapshot. Added a new list of links to archive, but after 6 or 7, almost all of the tasks start erroring out in cascade fashion, starting with `singlefile` step. Error is `TimeoutExpired Command` for chromium-browser, but it gives me a command to run, which also errors out with `(node:162485) UnhandledPromiseRejectionWarning: SyntaxError: Unexpected number in JSON at position 2` #### Steps to reproduce 1. Installed with `pip install archivebox` 2. Changed config to disable MEDIA and timeout set to 180 seconds. 3. Added list of links with `archivebox add ./list.txt`. 4. Starts adding things correctly, but after link 5 or 6 starts to error out. #### Screenshots or log output Original error ``` [i] [2020-09-02 17:41:18] ArchiveBox v0.4.21: archivebox update > /mnt/volume/.archivebox-output/new-archive [*] [2020-09-02 17:41:21] Writing 33 links to main index... √ /mnt/volume/.archivebox-output/new-archive/index.sqlite3 √ /mnt/volume/.archivebox-output/new-archive/index.json √ /mnt/volume/.archivebox-output/new-archive/index.html [▶] [2020-09-02 17:41:23] Collecting content for 8 Snapshots in archive... [√] [2020-09-02 17:44:28] "NYPD used facial recognition to track down Black Lives Matter activist - The Verge" https://theverge.com/2020/8/18/21373316/nypd-facial-recognition-black-lives-matter-activist-derrick-ingram √ ./archive/1598996327.937664 > singlefile Extractor failed: TimeoutExpired Command '['/mnt/volume/.archivebox-output/new-archive/node_modules/archivebox/node_modules/single-file/cli/single-file', '--browser-executable-path=chromium-browser', '--browser-args="["--headless", "--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.75 Safari/537.36", "--window-size=1440,2000"]"', 'https://theverge.com/2020/8/18/21373316/nypd-facial-recognition-black-lives-matter-activist-derrick-ingram', '/mnt/volume/.archivebox-output/new-archive/archive/1598996327.937664/singlefile.html']' timed out after 180 seconds Run to see full output: cd /mnt/volume/.archivebox-output/new-archive/archive/1598996327.937664; /mnt/volume/.archivebox-output/new-archive/node_modules/archivebox/node_modules/single-file/cli/single-file --browser-executable-path=chromium-browser "--browser-args="["--headless", "--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.75 Safari/537.36", "--window-size=1440,2000"]"" https://theverge.com/2020/8/18/21373316/nypd-facial-recognition-black-lives-matter-activist-derrick-ingram /mnt/volume/.archivebox-output/new-archive/archive/1598996327.937664/singlefile.html > pdf Extractor failed: Exception Failed to chmod: output.pdf does not exist (did the previous step fail?) Run to see full output: cd /mnt/volume/.archivebox-output/new-archive/archive/1598996327.937664; chromium-browser --headless "--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.75 Safari/537.36" --window-size=1440,2000 --timeout=180000 --print-to-pdf https://theverge.com/2020/8/18/21373316/nypd-facial-recognition-black-lives-matter-activist-derrick-ingram ``` When I run ``` cd /mnt/volume/.archivebox-output/new-archive/archive/1598996327.937664; /mnt/volume/.archivebox-output/new-archive/node_modules/archivebox/node_modules/single-file/cli/single-file --browser-executable-path=chromium-browser "--browser-args="["--headless", "--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.75 Safari/537.36", "--window-size=1440,2000"]"" https://theverge.com/2020/8/18/21373316/nypd-facial-recognition-black-lives-matter-activist-derrick-ingram /mnt/volume/.archivebox-output/new-archive/archive/1598996327.937664/singlefile.html ``` Output: ``` (node:162485) UnhandledPromiseRejectionWarning: SyntaxError: Unexpected number in JSON at position 2 at JSON.parse (<anonymous>) at getBrowserOptions (/mnt/volume/.archivebox-output/new-archive/node_modules/archivebox/node_modules/single-file/cli/back-ends/puppeteer.js:63:51) at Object.exports.initialize (/mnt/volume/.archivebox-output/new-archive/node_modules/archivebox/node_modules/single-file/cli/back-ends/puppeteer.js:36:35) at initialize (/mnt/volume/.archivebox-output/new-archive/node_modules/archivebox/node_modules/single-file/cli/single-file-cli-api.js:46:16) at run (/mnt/volume/.archivebox-output/new-archive/node_modules/archivebox/node_modules/single-file/cli/single-file:33:59) at Object.<anonymous> (/mnt/volume/.archivebox-output/new-archive/node_modules/archivebox/node_modules/single-file/cli/single-file:30:1) at Module._compile (internal/modules/cjs/loader.js:778:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10) at Module.load (internal/modules/cjs/loader.js:653:32) at tryModuleLoad (internal/modules/cjs/loader.js:593:12) (node:162485) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2) (node:162485) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code. ``` #### Software versions - OS: Ubuntu 20.04.1 LTS - ArchiveBox version: 0.21 - Python version: Python 3.7.9 - Chrome version: Chromium 84.0.4147.105
kerem closed this issue 2026-03-01 14:42:19 +03:00
kerem commented 2026-03-01 14:42:20 +03:00
Author
Owner
Copy link

@cdvv7788 commented on GitHub (Sep 2, 2020):

@drpfenderson there is a PR that should fix the issue with the index. You can give it a try (it should be merged this week tho): https://github.com/pirate/ArchiveBox/pull/452

About this issue, I will give it a check.

<!-- gh-comment-id:685924081 --> @cdvv7788 commented on GitHub (Sep 2, 2020): @drpfenderson there is a PR that should fix the issue with the index. You can give it a try (it should be merged this week tho): https://github.com/pirate/ArchiveBox/pull/452 About this issue, I will give it a check.
kerem commented 2026-03-01 14:42:20 +03:00
Author
Owner
Copy link

@cdvv7788 commented on GitHub (Sep 3, 2020):

@drpfenderson It works for me. Can you try running it with docker or installing the npm dependencies? (the quickstart has been updated with the instructions)

<!-- gh-comment-id:686544324 --> @cdvv7788 commented on GitHub (Sep 3, 2020): @drpfenderson It works for me. Can you try running it with docker or installing the npm dependencies? (the quickstart has been updated with the instructions)
kerem commented 2026-03-01 14:42:20 +03:00
Author
Owner
Copy link

@drpfenderson commented on GitHub (Sep 3, 2020):

@cdvv7788 I get the same timeout using docker/docker-compose, though I am really not sure how to run the node_modules commands from inside the docker to see the exact error. If I cd to the directory listed, don't I lose the docker-compose.yml and the path for node_modules? I tried researching the exec/run commands a bit to figure out how to execute the single-file command from within a container, but I can't really grok it.

$ docker-compose run archivebox add https://theverge.com/2020/8/18/21373316/nypd-facial-recognition-black-lives-matter-activist-derrick-ingram

[i] [2020-09-03 15:55:38] ArchiveBox v0.4.21: archivebox add https://theverge.com/2020/8/18/21373316/nypd-facial-recognition-black-lives-matter-activist-derrick-ingram
    > /data

[+] [2020-09-03 15:55:41] Adding 1 links to index (crawl depth=0)...
    > Saved verbatim input to sources/1599148541-import.txt
    > Parsed 1 URLs from input (Plain Text)
    > Found 1 new URLs not already in index

[*] [2020-09-03 15:55:41] Writing 33 links to main index...
    √ /data/index.sqlite3
    √ /data/index.json
    √ /data/index.html

[▶] [2020-09-03 15:55:42] Collecting content for 1 Snapshots in archive...

[+] [2020-09-03 15:55:42] "theverge.com/2020/8/18/21373316/nypd-facial-recognition-black-lives-matter-activist-derrick-ingram"
    https://theverge.com/2020/8/18/21373316/nypd-facial-recognition-black-lives-matter-activist-derrick-ingram
    > ./archive/1599148541.818964
      > title
      > favicon
      > wget
      > singlefile
        Extractor failed:
            TimeoutExpired Command '['/node/node_modules/single-file/cli/single-file', '--browser-executable-path=chromium', '--browser-args="["--headless", "--no-sandbox", "--disable-gpu", "--disable-dev-shm-usage", "--disable-software-rasterizer", "--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.75 Safari/537.36", "--window-size=1440,2000"]"', 'https://theverge.com/2020/8/18/21373316/nypd-facial-recognition-black-lives-matter-activist-derrick-ingram', '/data/archive/1599148541.818964/singlefile.html']' timed out after 180 seconds
        Run to see full output:
            cd /data/archive/1599148541.818964;
            /node/node_modules/single-file/cli/single-file --browser-executable-path=chromium "--browser-args="["--headless", "--no-sandbox", "--disable-gpu", "--disable-dev-shm-usage", "--disable-software-rasterizer", "--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.75 Safari/537.36", "--window-size=1440,2000"]"" https://theverge.com/2020/8/18/21373316/nypd-facial-recognition-black-lives-matter-activist-derrick-ingram /data/archive/1599148541.818964/singlefile.html

EDIT: To be clear, the npm dependencies were for sure installed, as far as I can tell. No errors on install using npm install --prefix . 'git+https://github.com/pirate/ArchiveBox.git'. I actually installed both using the prefix for the master branch, as well as your sql_index branch.

<!-- gh-comment-id:686597524 --> @drpfenderson commented on GitHub (Sep 3, 2020): @cdvv7788 I get the same timeout using docker/docker-compose, though I am really not sure how to run the `node_modules` commands from inside the docker to see the exact error. If I cd to the directory listed, don't I lose the docker-compose.yml and the path for `node_modules`? I tried researching the exec/run commands a bit to figure out how to execute the `single-file` command from within a container, but I can't really grok it. ``` $ docker-compose run archivebox add https://theverge.com/2020/8/18/21373316/nypd-facial-recognition-black-lives-matter-activist-derrick-ingram [i] [2020-09-03 15:55:38] ArchiveBox v0.4.21: archivebox add https://theverge.com/2020/8/18/21373316/nypd-facial-recognition-black-lives-matter-activist-derrick-ingram > /data [+] [2020-09-03 15:55:41] Adding 1 links to index (crawl depth=0)... > Saved verbatim input to sources/1599148541-import.txt > Parsed 1 URLs from input (Plain Text) > Found 1 new URLs not already in index [*] [2020-09-03 15:55:41] Writing 33 links to main index... √ /data/index.sqlite3 √ /data/index.json √ /data/index.html [▶] [2020-09-03 15:55:42] Collecting content for 1 Snapshots in archive... [+] [2020-09-03 15:55:42] "theverge.com/2020/8/18/21373316/nypd-facial-recognition-black-lives-matter-activist-derrick-ingram" https://theverge.com/2020/8/18/21373316/nypd-facial-recognition-black-lives-matter-activist-derrick-ingram > ./archive/1599148541.818964 > title > favicon > wget > singlefile Extractor failed: TimeoutExpired Command '['/node/node_modules/single-file/cli/single-file', '--browser-executable-path=chromium', '--browser-args="["--headless", "--no-sandbox", "--disable-gpu", "--disable-dev-shm-usage", "--disable-software-rasterizer", "--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.75 Safari/537.36", "--window-size=1440,2000"]"', 'https://theverge.com/2020/8/18/21373316/nypd-facial-recognition-black-lives-matter-activist-derrick-ingram', '/data/archive/1599148541.818964/singlefile.html']' timed out after 180 seconds Run to see full output: cd /data/archive/1599148541.818964; /node/node_modules/single-file/cli/single-file --browser-executable-path=chromium "--browser-args="["--headless", "--no-sandbox", "--disable-gpu", "--disable-dev-shm-usage", "--disable-software-rasterizer", "--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.75 Safari/537.36", "--window-size=1440,2000"]"" https://theverge.com/2020/8/18/21373316/nypd-facial-recognition-black-lives-matter-activist-derrick-ingram /data/archive/1599148541.818964/singlefile.html ``` EDIT: To be clear, the npm dependencies were for sure installed, as far as I can tell. No errors on install using `npm install --prefix . 'git+https://github.com/pirate/ArchiveBox.git'`. I actually installed both using the prefix for the master branch, as well as your sql_index branch.
kerem commented 2026-03-01 14:42:20 +03:00
Author
Owner
Copy link

@gildas-lormeau commented on GitHub (Sep 4, 2020):

Author of SingleFile here. This is a bug in SingleFile due to the presence of "," and spaces in the --browser-args switch (e.g. 1440,2000 and Mozilla/5.0 (Windows NT 10.0; Win64; x64)). I'm trying to see how it could be fixed or circumvented. I guess proposing to pass a JSON string was not a good idea.

<!-- gh-comment-id:687032560 --> @gildas-lormeau commented on GitHub (Sep 4, 2020): Author of SingleFile here. This is a bug in SingleFile due to the presence of "," and spaces in the `--browser-args` switch (e.g. `1440,2000` and `Mozilla/5.0 (Windows NT 10.0; Win64; x64)`). I'm trying to see how it could be fixed or circumvented. I guess proposing to pass a JSON string was not a good idea.
kerem commented 2026-03-01 14:42:20 +03:00
Author
Owner
Copy link

@gildas-lormeau commented on GitHub (Sep 4, 2020):

Finally, I was able to fix the issue by formatting the --browser-args switch like this (surrounding quotes included) in your example:

"--browser-args=""[""--headless"", ""--no-sandbox"", ""--disable-gpu"", ""--disable-dev-shm-usage"", ""--disable-software-rasterizer"", ""--window-size=1440,2000"", ""--user-agent"="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.75 Safari/537.36""]"""

instead of:
"--browser-args="["--headless", "--no-sandbox", "--disable-gpu", "--disable-dev-shm-usage", "--disable-software-rasterizer", "--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.75 Safari/537.36", "--window-size=1440,2000"]""

<!-- gh-comment-id:687149737 --> @gildas-lormeau commented on GitHub (Sep 4, 2020): Finally, I was able to fix the issue by formatting the `--browser-args` switch like this (surrounding quotes included) in your example: `"--browser-args=""[""--headless"", ""--no-sandbox"", ""--disable-gpu"", ""--disable-dev-shm-usage"", ""--disable-software-rasterizer"", ""--window-size=1440,2000"", ""--user-agent"="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.75 Safari/537.36""]"""` instead of: `"--browser-args="["--headless", "--no-sandbox", "--disable-gpu", "--disable-dev-shm-usage", "--disable-software-rasterizer", "--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.75 Safari/537.36", "--window-size=1440,2000"]""`
kerem commented 2026-03-01 14:42:20 +03:00
Author
Owner
Copy link

@cdvv7788 commented on GitHub (Sep 4, 2020):

Thanks. We definitely need to improve the command so it can be run directly on case of error. I will add that to the pending tasks.

@drpfenderson What is your TIMEOUT setting? Have you modified any setting or changed any environment variable? I cannot reproduce it, not even with docker, so I am suspecting you have something unusual in your local setup.

<!-- gh-comment-id:687187457 --> @cdvv7788 commented on GitHub (Sep 4, 2020): Thanks. We definitely need to improve the command so it can be run directly on case of error. I will add that to the pending tasks. @drpfenderson What is your `TIMEOUT` setting? Have you modified any setting or changed any environment variable? I cannot reproduce it, not even with docker, so I am suspecting you have something unusual in your local setup.
kerem commented 2026-03-01 14:42:20 +03:00
Author
Owner
Copy link

@drpfenderson commented on GitHub (Sep 4, 2020):

@cdvv7788 My timeout is set to 180, which I set in the docker-compose.yml file. I've tried with much longer times, like 3600 as I've seen mentioned a couple times, but it didn't change anything. The only other change is SHOW_PROGRESS, but I've also tested it with the default docker-compose file.

I'm wondering if there is some config or library file somewhere on my system from the much older versions of archivebox that is floating around and interfering. Before upgrading to v0.21 or docker, I made sure and scrubbed the .conf files and .local stuff related to it, in my user folder as well as the archive folder itself, but there still must be something somewhere that I'm missing since the problem happens with the brand new archive as well. I'll do some deeper searching and let you know.

<!-- gh-comment-id:687217813 --> @drpfenderson commented on GitHub (Sep 4, 2020): @cdvv7788 My timeout is set to 180, which I set in the docker-compose.yml file. I've tried with much longer times, like 3600 as I've seen mentioned a couple times, but it didn't change anything. The only other change is `SHOW_PROGRESS`, but I've also tested it with the default docker-compose file. I'm wondering if there is some config or library file somewhere on my system from the much older versions of archivebox that is floating around and interfering. Before upgrading to v0.21 or docker, I made sure and scrubbed the .conf files and .local stuff related to it, in my user folder as well as the archive folder itself, but there still must be something somewhere that I'm missing since the problem happens with the brand new archive as well. I'll do some deeper searching and let you know.
kerem commented 2026-03-01 14:42:20 +03:00
Author
Owner
Copy link

@cdvv7788 commented on GitHub (Sep 4, 2020):

@drpfenderson I just created a PR that should escape correctly the argument. With that you should be able to, at least, run the command properly, and come back with the error that singlefile is returning.

<!-- gh-comment-id:687220116 --> @cdvv7788 commented on GitHub (Sep 4, 2020): @drpfenderson I just created a PR that should escape correctly the argument. With that you should be able to, at least, run the command properly, and come back with the error that `singlefile` is returning.
kerem commented 2026-03-01 14:42:20 +03:00
Author
Owner
Copy link

@drpfenderson commented on GitHub (Sep 4, 2020):

Well, I essentially did a find/replace/delete for every archivebox/node/python file on my system that could be related. There were a number of weird places with files due to the various installation methods I've used over the years for this program. Realizing the amount of work to disentangle everything, I spun up a new server, attached the archive to it, wiped all the local config/conf files, ran the docker-compose run archivebox add and it all worked!

Thank you all for your infinite patience with me.

<!-- gh-comment-id:687303510 --> @drpfenderson commented on GitHub (Sep 4, 2020): Well, I essentially did a find/replace/delete for every archivebox/node/python file on my system that could be related. There were a number of weird places with files due to the various installation methods I've used over the years for this program. Realizing the amount of work to disentangle everything, I spun up a new server, attached the archive to it, wiped all the local config/conf files, ran the `docker-compose run archivebox add` and it all worked! Thank you all for your infinite patience with me.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
dev
main
abx-dl-refactor
claude/issue-1688-20251229-2233
claude/s3-support-migration-JZQcf
claude/review-snapshot-archive-RJOkr
claude/review-code-quality-macdO
claude/reduce-chrome-duplication-JgWVr
fix/chrome-args-comma-parsing
claude/issue-668-20251229-2145
claude/issue-1664-20251229-2236
v086
claude/improve-test-suite-xm6Bh
claude/typescript-archivebox-rewrite-011CUmPyEmECnkXwyHRT9RPF
claude/archivebox-schema-orm-comparison-011CUn26GurEpLiBUq4zjVLJ
stable
logfire
plugins-browsertrix
v072
link-removal2
v0.8.6rc1
v0.7.3
v0.8.6rc0
v0.8.5rc53
v0.8.5rc51
v0.8.5rc50
v0.8.5rc49
v0.8.5rc48
v0.8.5rc47
v0.8.5rc46
v0.8.5rc45
v0.8.5rc44
v0.8.5rc43
v0.8.5rc42
v0.8.5rc41
v0.8.5rc40
v0.8.5rc39
v0.8.5rc38
v0.8.5rc37
v0.8.5rc36
v0.8.5rc35
v0.8.5rc34
v0.8.5rc33
v0.8.5rc32
v0.8.5rc31
v0.8.5rc30
v0.8.5rc29
v0.8.5rc28
v0.8.5rc27
v0.8.5rc26
v0.8.5rc25
v0.8.5rc24
v0.8.5rc23
v0.8.5rc22
v0.8.5rc13
v0.8.5rc12
v0.8.5rc11
v0.8.5rc10
v0.8.5rc9
v0.8.5rc8
v0.8.5rc6
v0.8.5rc5
v0.8.5rc4
v0.8.5rc3
v0.8.5rc2
v0.8.5-rc
v0.8.4-rc
v0.8.3-rc
v0.8.2-rc
v0.8.0-rc
v0.7.2
v0.7.1
v0.7.0
v0.6.2
v0.6.0
v0.5.6
v0.5.4
v0.5.3
v0.4.24
v0.4.21
v0.4.20
v0.4.19
v0.4.18
v0.4.17
v0.4.16
v0.4.15
v0.4.14
v0.4.13
v0.4.12
v0.4.11
v0.4.9
v0.2.4
v0.2.3
v0.2.2
v0.2.1
v0.2.0
v0.1.0
v0.0.3
v0.0.2
v0.0.1
Labels
Clear labels   
expected: maybe someday

   
expected: next release

   
expected: release after next

   
expected: unlikely unless contributed

   
good first ticket

   
help wanted

   
pull-request

Mirrored from GitHub Pull Request

  
scope: all users

   
scope: windows users

   
size: easy

   
size: hard

   
size: medium

   
size: medium

   
status: backlog

   
status: blocked

   
status: done

   
status: idea-phase

   
status: needs followup

   
status: wip

   
status: wontfix

   
touches: API/CLI/Spec

   
touches: configuration

   
touches: data/schema/architecture

   
touches: dependencies/packaging

   
touches: docs

   
touches: js

   
touches: views/replayers/html/css

   
why: correctness

   
why: functionality

   
why: performance

   
why: security
No labels
expected: maybe someday
expected: next release
expected: release after next
expected: unlikely unless contributed
good first ticket
help wanted
pull-request
scope: all users
scope: windows users
size: easy
size: hard
size: medium
size: medium
status: backlog
status: blocked
status: done
status: idea-phase
status: needs followup
status: wip
status: wontfix
touches: API/CLI/Spec
touches: configuration
touches: data/schema/architecture
touches: dependencies/packaging
touches: docs
touches: js
touches: views/replayers/html/css
why: correctness
why: functionality
why: performance
why: security
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/ArchiveBox#308
No description provided.