starred/ArchiveBox
1
0
Fork 0
mirror of https://github.com/ArchiveBox/ArchiveBox.git synced 2026-04-25 17:16:00 +03:00
Code Issues 624 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #1646] Bug: All timers/progress bars timeout when archiving from REST API #4001

New issue
Open
opened 2026-03-15 01:15:24 +03:00 by kerem · 1 comment
kerem commented 2026-03-15 01:15:24 +03:00
Owner
Copy link

Originally created by @benmuth on GitHub (Feb 1, 2025).
Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/1646

Originally assigned to: @pirate on GitHub.

Provide a screenshot and describe the bug

When using the extension (in the redesign branch, using the REST API) to archive any URL, all timers/progress bars take the maximum amount of time. Parsers take 240 seconds, most extractors take 60 seconds, the media extractor takes an hour (I think, I didn't wait around to find out). This doesn't happen when running archivebox add through the CLI, only the REST API.

Image

I kind of figured out what's going on, but not why. In archivebox/logging_util.py, TimedProgress.end() tries to terminate the progress_bar process. For some reason (busy writing to stdout?), the process ignores the terminate, then the join() call after the terminate just blocks until the progress_bar function finishes execution.

Adding an explicit signal handler to the beginning of the progress_bar function seems to fix the problem:

    def handle_term(signum, frame):
        sys.exit(0)

    signal.signal(signal.SIGTERM, handle_term)

That works, but I'm not sure if there's a better solution. Should I open a PR with this fix against dev?

Steps to reproduce

1. run docker run -it -p 8000:8000 \                                                                                    
                       -v $PWD/data:/data \
                       -v $PWD/archivebox:/app/archivebox \
                       archivebox server 0.0.0.0:8000 --debug
2. load the redesign folder on the redesign branch of the archivebox-browser-extension as an extension.
3. try to archive a webpage by clicking on the extension in the Chrome menubar.

Logs or errors

╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ [2025-02-01 11:35:03] ArchiveBox v0.8.5rc53: archivebox server 0.0.0.0:8000                                                                                 │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
[+] Starting ArchiveBox webserver...
    > Starting ArchiveBox webserver on http://0.0.0.0:8000
    > Log in to ArchiveBox Admin UI on http://0.0.0.0:8000/admin
    > Writing ArchiveBox error log to ./logs/errors.log
Performing system checks...

System check identified no issues (0 silenced).
February 01, 2025 - 11:35:04
Django version 5.1.2, using settings 'core.settings'
Starting ASGI/Daphne version 4.1.2 development server at http://0.0.0.0:8000/
Quit the server with CONTROL-C.
[2025-02-01 11:35:04] INFO     daphne.server HTTP/2 support not enabled (install the http2 and tls Twisted extras)                                server.py:120
                      INFO     daphne.server Configuring endpoint tcp:port=8000:interface=0.0.0.0                                                 server.py:129
                      INFO     daphne.server Listening on TCP address 0.0.0.0:8000                                                                server.py:160
[+] [2025-02-01 11:35:14] Adding 1 links to index (crawl depth=0)...
    > Saved verbatim input to sources/1738409714-import.txt
███████████████████████████████████████████████████                                                                                         5.5% (13/240sec)[2025-02-01 11:35:30] INFO     django.channels.server [mHTTP GET /health/ 200 [0.02, 127.0.0.1:58462][0m                                       runserver.py:168
███████████████████████████████████████████████████████████████████████████████████                                                         15.5% (37/240sec)[2025-02-01 11:36:00] INFO     django.channels.server [mHTTP GET /health/ 200 [0.00, 127.0.0.1:47328][0m                                       runserver.py:168
█████████████████████████████████████████████████████████████████████████████████████████████████                                           25.3% (61/240sec)[2025-02-01 11:36:31] INFO     django.channels.server [mHTTP GET /health/ 200 [0.01, 127.0.0.1:41106][0m                                       runserver.py:168
███████████████████████████████████████████████████████████████████████████████████████████████████████████                                 35.1% (84/240sec)[2025-02-01 11:37:01] INFO     django.channels.server [mHTTP GET /health/ 200 [0.01, 127.0.0.1:55162][0m                                       runserver.py:168
███████████████████████████████████████████████████████████████████████████████████████████████████████████████████                         45.0% (108/240sec)[2025-02-01 11:37:31] INFO     django.channels.server [mHTTP GET /health/ 200 [0.00, 127.0.0.1:58490][0m                                       runserver.py:168
█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████                   54.8% (132/240sec)[2025-02-01 11:38:01] INFO     django.channels.server [mHTTP GET /health/ 200 [0.01, 127.0.0.1:46180][0m                                       runserver.py:168
██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████              64.6% (155/240sec)[2025-02-01 11:38:31] INFO     django.channels.server [mHTTP GET /health/ 200 [0.01, 127.0.0.1:58234][0m                                       runserver.py:168
██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████          74.5% (179/240sec)[2025-02-01 11:39:01] INFO     django.channels.server [mHTTP GET /health/ 200 [0.01, 127.0.0.1:60778][0m                                       runserver.py:168
██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████      84.4% (203/240sec)[2025-02-01 11:39:31] INFO     django.channels.server [mHTTP GET /health/ 200 [0.01, 127.0.0.1:33708][0m                                       runserver.py:168
█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████   94.3% (226/240sec)[2025-02-01 11:40:01] INFO     django.channels.server [mHTTP GET /health/ 200 [0.00, 127.0.0.1:50100][0m                                       runserver.py:168
████████████████████████████████████████████                                                                                                4.2% (10/240sec)[2025-02-01 11:40:31] INFO     django.channels.server [mHTTP GET /health/ 200 [0.01, 127.0.0.1:41456][0m                                       runserver.py:168
████████████████████████████████████████████████████████████████████████████████                                                            14.2% (34/240sec)[2025-02-01 11:41:01] INFO     django.channels.server [mHTTP GET /health/ 200 [0.01, 127.0.0.1:56638][0m                                       runserver.py:168
███████████████████████████████████████████████████████████████████████████████████                                                         15.6% (38/240sec)[2025-02-01 11:41:06] WARNING  daphne.server Application instance <Task pending name='Task-1' coro=<ASGIStaticFilesHandler.__call__() running at  server.py:278
                               /usr/local/lib/python3.11/site-packages/django/contrib/staticfiles/handlers.py:101> wait_for=<Task cancelling
                               name='Task-4' coro=<ASGIHandler.handle.<locals>.process_request() running at
                               /usr/local/lib/python3.11/site-packages/django/core/handlers/asgi.py:185> wait_for=<Future pending
                               cb=[_chain_future.<locals>._call_check_cancel() at /usr/local/lib/python3.11/asyncio/futures.py:387,
                               Task.task_wakeup()]> cb=[Task.task_wakeup()]>> for connection <WebRequest at 0xffff948a6810 method=POST
                               uri=/api/v1/cli/add clientproto=HTTP/1.1> took too long to shut down and was killed.
████████████████████████████████████████████████████████████████████████████████████                                                        16.0% (38/240sec)[2025-02-01 11:41:07] WARNING  daphne.server Application instance <Task cancelling name='Task-1' coro=<ASGIStaticFilesHandler.__call__() running  server.py:278
                               at /usr/local/lib/python3.11/site-packages/django/contrib/staticfiles/handlers.py:101> wait_for=<_GatheringFuture
                               pending cb=[Task.task_wakeup()]>> for connection <WebRequest at 0xffff948a6810 method=POST uri=/api/v1/cli/add
                               clientproto=HTTP/1.1> took too long to shut down and was killed.
    > Parsed 1 URLs from input (URL List)
    > Found 1 new URLs not already in index

[*] [2025-02-01 11:45:23] Writing 1 links to main index...
    √ ./index.sqlite3

[*] [2025-02-01 11:47:54] Archiving 1/26 URLs from added set...

[▶] [2025-02-01 11:47:54] Starting archiving of 1 snapshots in index...

[+] [2025-02-01 11:47:54] "nullprogram.com/blog/2017/09/01"
    https://nullprogram.com/blog/2017/09/01/
    > ./archive/1738409714.199639
      > favicon
      > headers
      > wget
      > title
      > readability
      > htmltotext
      > media
      █████████████████████                                                                                                                 2.1% (76/3600sec)

ArchiveBox Version

0.8.5rc53
ArchiveBox v0.8.5rc53 COMMIT_HASH=3e1cdcf BUILD_TIME=2025-01-31 18:02:40 1738346560
IN_DOCKER=True IN_QEMU=False ARCH=aarch64 OS=Linux PLATFORM=Linux-6.10.14-linuxkit-aarch64-with-glibc2.36 PYTHON=Cpython
EUID=911:0 UID=911:0 PUID=911:0 FS_UID=911:0 FS_PERMS=644 FS_ATOMIC=True FS_REMOTE=True
DEBUG=False IS_TTY=True SUDO=False ID=2cba7b94:9b67ccd9 SEARCH_BACKEND=ripgrep LDAP=False

 Binary Dependencies:
 √  python                3.11.11      sys_pip    /usr/local/bin/python3.11
 √  django                5.1.2        sys_pip    /usr/local/lib/python3.11/site-packages/django/__init__.py
 √  sqlite                2.6.0        sys_pip    /usr/local/lib/python3.11/site-packages/django/db/backends/sqlite3/base.py
 √  pip                   24.2.0       sys_pip    /usr/local/bin/pip
 √  pipx                  1.1.0        sys_pip    /usr/bin/pipx
 √  node                  22.13.1      apt        /usr/bin/node
 √  npm                   11.1.0       apt        /usr/bin/npm
 √  npx                   11.1.0       apt        /usr/bin/npx
 √  playwright            1.49.1       sys_pip    /usr/local/bin/playwright
 √  puppeteer             23.11.1      lib_npm    ~/.npm/bin/puppeteer
 √  ldap                  3.4.4        sys_pip    /usr/local/lib/python3.11/site-packages/ldap/__init__.py
 √  rg                    13.0.0       apt        /usr/bin/rg
 √  sonic                 1.4.9        env        /usr/local/bin/sonic
 √  chrome                131.0.6778   env        /usr/bin/chromium-browser
 √  curl                  8.11.1       apt        /usr/bin/curl
 √  git                   2.39.5       apt        /usr/bin/git
 √  postlight-parser      2.2.3        sys_npm    ~/.npm/bin/postlight-parser
 √  readability-extractor 0.0.11       lib_npm    ~/.npm/bin/readability-extractor
 √  single-file           1.1.54       lib_npm    ~/.npm/bin/single-file
 √  wget                  1.21.3       apt        /usr/bin/wget
 √  yt-dlp                2024.10.7    sys_pip    /usr/local/bin/yt-dlp
 √  ffmpeg                5.1.6        env        /usr/bin/ffmpeg

 Package Managers:
 √  env         /usr/bin/which                                       UID=911  PATH=~/.npm/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin…
 √  apt         /usr/bin/apt-get                                     UID=0    PATH=/usr/bin:/bin
 -  brew        not available                                        UID=911  PATH=
 √  sys_pip     /usr/local/bin/pip                                   UID=911  PATH=~/.local/bin:/usr/local/bin:/usr/bin
 -  venv_pip    not available                                        UID=911  PATH=/tmp/NotInsideAVenv/lib/bin
 -  lib_pip     not available                                        UID=911  PATH=./lib/aarch64-linux-docker/pip/venv/bin
 √  sys_npm     /usr/bin/npm                                         UID=911  PATH=~/.npm/bin
 -  lib_npm     /usr/bin/npm                                         UID=911  PATH=./lib/aarch64-linux-docker/npm/node_modules/.bin:./node_modules/.bin:~/.npm…
 √  playwright  /usr/local/bin/playwright                            UID=0    PATH=./lib/aarch64-linux-docker/bin:~/.npm/bin:/usr/local/bin:/usr/local/sbin:/u…
 √  puppeteer   /usr/bin/npx                                         UID=911  PATH=./lib/aarch64-linux-docker/bin

 Code locations:
 √  PACKAGE_DIR           41 files        valid     /app/archivebox
 √  TEMPLATES_DIR         3 files         valid     /app/archivebox/templates
 -  CUSTOM_TEMPLATES_DIR  missing         unused    ./user_templates
 -  USER_PLUGINS_DIR      missing         unused    ./user_plugins
 √  LIB_DIR               0 files         valid     /usr/share/archivebox/lib

 Data locations:
 √  DATA_DIR              14 files @      valid     /data
 √  CONFIG_FILE           368.0 Bytes     valid     ./ArchiveBox.conf
 √  SQL_INDEX             736.0 KB        valid     ./index.sqlite3
 √  QUEUE_DATABASE        92.0 KB         valid     ./queue.sqlite3
 √  ARCHIVE_DIR           25 files        valid     ./archive
 √  SOURCES_DIR           101 files       valid     ./sources
 √  PERSONAS_DIR          1 files         valid     ./personas
 √  LOGS_DIR              1 files         valid     ./logs
 √  TMP_DIR               0 files         valid     /tmp/archivebox

How did you install the version of ArchiveBox you are using?

Other

What operating system are you running on?

macOS (including Docker on macOS)

What type of drive are you using to store your ArchiveBox data?

  • some of data/ is on a local SSD or NVMe drive
  • some of data/ is on a spinning hard drive or external USB drive
  • some of data/ is on a network mount (e.g. NFS/SMB/Ceph/GlusterFS/etc.)
  • some of data/ is on a FUSE mount (e.g. SSHFS/RClone/S3/B2/Google Drive/Dropbox/etc.)

Docker Compose Configuration

docker run -it -p 8000:8000 \                                                                             
                       -v $PWD/data:/data \
                       -v $PWD/archivebox:/app/archivebox \
                       archivebox server 0.0.0.0:8000 --debug

ArchiveBox Configuration

# Converted from INI to TOML format: https://toml.io/en/

[SERVER_CONFIG]
SECRET_KEY = "n************************************************y"
Originally created by @benmuth on GitHub (Feb 1, 2025). Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/1646 Originally assigned to: @pirate on GitHub. ### Provide a screenshot and describe the bug When using the extension (in the `redesign` branch, using the REST API) to archive any URL, all timers/progress bars take the maximum amount of time. Parsers take 240 seconds, most extractors take 60 seconds, the media extractor takes an hour (I think, I didn't wait around to find out). This doesn't happen when running `archivebox add` through the CLI, only the REST API. <img width="1280" alt="Image" src="https://github.com/user-attachments/assets/0e7d032f-45d2-4aca-be62-45912e9ffc21" /> I kind of figured out what's going on, but not why. In `archivebox/logging_util.py`, `TimedProgress.end()` tries to `terminate` the `progress_bar` process. For some reason (busy writing to stdout?), the process ignores the terminate, then the `join()` call after the terminate just blocks until the `progress_bar` function finishes execution. Adding an explicit signal handler to the beginning of the `progress_bar` function seems to fix the problem: ``` def handle_term(signum, frame): sys.exit(0) signal.signal(signal.SIGTERM, handle_term) ``` That works, but I'm not sure if there's a better solution. Should I open a PR with this fix against dev? ### Steps to reproduce ```markdown 1. run docker run -it -p 8000:8000 \ -v $PWD/data:/data \ -v $PWD/archivebox:/app/archivebox \ archivebox server 0.0.0.0:8000 --debug 2. load the redesign folder on the redesign branch of the archivebox-browser-extension as an extension. 3. try to archive a webpage by clicking on the extension in the Chrome menubar. ``` ### Logs or errors ```shell ╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ [2025-02-01 11:35:03] ArchiveBox v0.8.5rc53: archivebox server 0.0.0.0:8000 │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ [+] Starting ArchiveBox webserver... > Starting ArchiveBox webserver on http://0.0.0.0:8000 > Log in to ArchiveBox Admin UI on http://0.0.0.0:8000/admin > Writing ArchiveBox error log to ./logs/errors.log Performing system checks... System check identified no issues (0 silenced). February 01, 2025 - 11:35:04 Django version 5.1.2, using settings 'core.settings' Starting ASGI/Daphne version 4.1.2 development server at http://0.0.0.0:8000/ Quit the server with CONTROL-C. [2025-02-01 11:35:04] INFO daphne.server HTTP/2 support not enabled (install the http2 and tls Twisted extras) server.py:120 INFO daphne.server Configuring endpoint tcp:port=8000:interface=0.0.0.0 server.py:129 INFO daphne.server Listening on TCP address 0.0.0.0:8000 server.py:160 [+] [2025-02-01 11:35:14] Adding 1 links to index (crawl depth=0)... > Saved verbatim input to sources/1738409714-import.txt ███████████████████████████████████████████████████ 5.5% (13/240sec)[2025-02-01 11:35:30] INFO django.channels.server [mHTTP GET /health/ 200 [0.02, 127.0.0.1:58462][0m runserver.py:168 ███████████████████████████████████████████████████████████████████████████████████ 15.5% (37/240sec)[2025-02-01 11:36:00] INFO django.channels.server [mHTTP GET /health/ 200 [0.00, 127.0.0.1:47328][0m runserver.py:168 █████████████████████████████████████████████████████████████████████████████████████████████████ 25.3% (61/240sec)[2025-02-01 11:36:31] INFO django.channels.server [mHTTP GET /health/ 200 [0.01, 127.0.0.1:41106][0m runserver.py:168 ███████████████████████████████████████████████████████████████████████████████████████████████████████████ 35.1% (84/240sec)[2025-02-01 11:37:01] INFO django.channels.server [mHTTP GET /health/ 200 [0.01, 127.0.0.1:55162][0m runserver.py:168 ███████████████████████████████████████████████████████████████████████████████████████████████████████████████████ 45.0% (108/240sec)[2025-02-01 11:37:31] INFO django.channels.server [mHTTP GET /health/ 200 [0.00, 127.0.0.1:58490][0m runserver.py:168 █████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ 54.8% (132/240sec)[2025-02-01 11:38:01] INFO django.channels.server [mHTTP GET /health/ 200 [0.01, 127.0.0.1:46180][0m runserver.py:168 ██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ 64.6% (155/240sec)[2025-02-01 11:38:31] INFO django.channels.server [mHTTP GET /health/ 200 [0.01, 127.0.0.1:58234][0m runserver.py:168 ██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ 74.5% (179/240sec)[2025-02-01 11:39:01] INFO django.channels.server [mHTTP GET /health/ 200 [0.01, 127.0.0.1:60778][0m runserver.py:168 ██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ 84.4% (203/240sec)[2025-02-01 11:39:31] INFO django.channels.server [mHTTP GET /health/ 200 [0.01, 127.0.0.1:33708][0m runserver.py:168 █████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ 94.3% (226/240sec)[2025-02-01 11:40:01] INFO django.channels.server [mHTTP GET /health/ 200 [0.00, 127.0.0.1:50100][0m runserver.py:168 ████████████████████████████████████████████ 4.2% (10/240sec)[2025-02-01 11:40:31] INFO django.channels.server [mHTTP GET /health/ 200 [0.01, 127.0.0.1:41456][0m runserver.py:168 ████████████████████████████████████████████████████████████████████████████████ 14.2% (34/240sec)[2025-02-01 11:41:01] INFO django.channels.server [mHTTP GET /health/ 200 [0.01, 127.0.0.1:56638][0m runserver.py:168 ███████████████████████████████████████████████████████████████████████████████████ 15.6% (38/240sec)[2025-02-01 11:41:06] WARNING daphne.server Application instance <Task pending name='Task-1' coro=<ASGIStaticFilesHandler.__call__() running at server.py:278 /usr/local/lib/python3.11/site-packages/django/contrib/staticfiles/handlers.py:101> wait_for=<Task cancelling name='Task-4' coro=<ASGIHandler.handle.<locals>.process_request() running at /usr/local/lib/python3.11/site-packages/django/core/handlers/asgi.py:185> wait_for=<Future pending cb=[_chain_future.<locals>._call_check_cancel() at /usr/local/lib/python3.11/asyncio/futures.py:387, Task.task_wakeup()]> cb=[Task.task_wakeup()]>> for connection <WebRequest at 0xffff948a6810 method=POST uri=/api/v1/cli/add clientproto=HTTP/1.1> took too long to shut down and was killed. ████████████████████████████████████████████████████████████████████████████████████ 16.0% (38/240sec)[2025-02-01 11:41:07] WARNING daphne.server Application instance <Task cancelling name='Task-1' coro=<ASGIStaticFilesHandler.__call__() running server.py:278 at /usr/local/lib/python3.11/site-packages/django/contrib/staticfiles/handlers.py:101> wait_for=<_GatheringFuture pending cb=[Task.task_wakeup()]>> for connection <WebRequest at 0xffff948a6810 method=POST uri=/api/v1/cli/add clientproto=HTTP/1.1> took too long to shut down and was killed. > Parsed 1 URLs from input (URL List) > Found 1 new URLs not already in index [*] [2025-02-01 11:45:23] Writing 1 links to main index... √ ./index.sqlite3 [*] [2025-02-01 11:47:54] Archiving 1/26 URLs from added set... [▶] [2025-02-01 11:47:54] Starting archiving of 1 snapshots in index... [+] [2025-02-01 11:47:54] "nullprogram.com/blog/2017/09/01" https://nullprogram.com/blog/2017/09/01/ > ./archive/1738409714.199639 > favicon > headers > wget > title > readability > htmltotext > media █████████████████████ 2.1% (76/3600sec) ``` ### ArchiveBox Version ```shell 0.8.5rc53 ArchiveBox v0.8.5rc53 COMMIT_HASH=3e1cdcf BUILD_TIME=2025-01-31 18:02:40 1738346560 IN_DOCKER=True IN_QEMU=False ARCH=aarch64 OS=Linux PLATFORM=Linux-6.10.14-linuxkit-aarch64-with-glibc2.36 PYTHON=Cpython EUID=911:0 UID=911:0 PUID=911:0 FS_UID=911:0 FS_PERMS=644 FS_ATOMIC=True FS_REMOTE=True DEBUG=False IS_TTY=True SUDO=False ID=2cba7b94:9b67ccd9 SEARCH_BACKEND=ripgrep LDAP=False Binary Dependencies: √ python 3.11.11 sys_pip /usr/local/bin/python3.11 √ django 5.1.2 sys_pip /usr/local/lib/python3.11/site-packages/django/__init__.py √ sqlite 2.6.0 sys_pip /usr/local/lib/python3.11/site-packages/django/db/backends/sqlite3/base.py √ pip 24.2.0 sys_pip /usr/local/bin/pip √ pipx 1.1.0 sys_pip /usr/bin/pipx √ node 22.13.1 apt /usr/bin/node √ npm 11.1.0 apt /usr/bin/npm √ npx 11.1.0 apt /usr/bin/npx √ playwright 1.49.1 sys_pip /usr/local/bin/playwright √ puppeteer 23.11.1 lib_npm ~/.npm/bin/puppeteer √ ldap 3.4.4 sys_pip /usr/local/lib/python3.11/site-packages/ldap/__init__.py √ rg 13.0.0 apt /usr/bin/rg √ sonic 1.4.9 env /usr/local/bin/sonic √ chrome 131.0.6778 env /usr/bin/chromium-browser √ curl 8.11.1 apt /usr/bin/curl √ git 2.39.5 apt /usr/bin/git √ postlight-parser 2.2.3 sys_npm ~/.npm/bin/postlight-parser √ readability-extractor 0.0.11 lib_npm ~/.npm/bin/readability-extractor √ single-file 1.1.54 lib_npm ~/.npm/bin/single-file √ wget 1.21.3 apt /usr/bin/wget √ yt-dlp 2024.10.7 sys_pip /usr/local/bin/yt-dlp √ ffmpeg 5.1.6 env /usr/bin/ffmpeg Package Managers: √ env /usr/bin/which UID=911 PATH=~/.npm/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin… √ apt /usr/bin/apt-get UID=0 PATH=/usr/bin:/bin - brew not available UID=911 PATH= √ sys_pip /usr/local/bin/pip UID=911 PATH=~/.local/bin:/usr/local/bin:/usr/bin - venv_pip not available UID=911 PATH=/tmp/NotInsideAVenv/lib/bin - lib_pip not available UID=911 PATH=./lib/aarch64-linux-docker/pip/venv/bin √ sys_npm /usr/bin/npm UID=911 PATH=~/.npm/bin - lib_npm /usr/bin/npm UID=911 PATH=./lib/aarch64-linux-docker/npm/node_modules/.bin:./node_modules/.bin:~/.npm… √ playwright /usr/local/bin/playwright UID=0 PATH=./lib/aarch64-linux-docker/bin:~/.npm/bin:/usr/local/bin:/usr/local/sbin:/u… √ puppeteer /usr/bin/npx UID=911 PATH=./lib/aarch64-linux-docker/bin Code locations: √ PACKAGE_DIR 41 files valid /app/archivebox √ TEMPLATES_DIR 3 files valid /app/archivebox/templates - CUSTOM_TEMPLATES_DIR missing unused ./user_templates - USER_PLUGINS_DIR missing unused ./user_plugins √ LIB_DIR 0 files valid /usr/share/archivebox/lib Data locations: √ DATA_DIR 14 files @ valid /data √ CONFIG_FILE 368.0 Bytes valid ./ArchiveBox.conf √ SQL_INDEX 736.0 KB valid ./index.sqlite3 √ QUEUE_DATABASE 92.0 KB valid ./queue.sqlite3 √ ARCHIVE_DIR 25 files valid ./archive √ SOURCES_DIR 101 files valid ./sources √ PERSONAS_DIR 1 files valid ./personas √ LOGS_DIR 1 files valid ./logs √ TMP_DIR 0 files valid /tmp/archivebox ``` ### How did you install the version of ArchiveBox you are using? Other ### What operating system are you running on? macOS (including Docker on macOS) ### What type of drive are you using to store your ArchiveBox data? - [x] some of `data/` is on a local SSD or NVMe drive - [ ] some of `data/` is on a spinning hard drive or external USB drive - [ ] some of `data/` is on a network mount (e.g. NFS/SMB/Ceph/GlusterFS/etc.) - [ ] some of `data/` is on a FUSE mount (e.g. SSHFS/RClone/S3/B2/Google Drive/Dropbox/etc.) ### Docker Compose Configuration ```shell docker run -it -p 8000:8000 \ -v $PWD/data:/data \ -v $PWD/archivebox:/app/archivebox \ archivebox server 0.0.0.0:8000 --debug ``` ### ArchiveBox Configuration ```shell # Converted from INI to TOML format: https://toml.io/en/ [SERVER_CONFIG] SECRET_KEY = "n************************************************y" ```
kerem commented 2026-03-15 01:15:30 +03:00
Author
Owner
Copy link

@pirate commented on GitHub (Feb 1, 2025):

Sure I'll take the PR :)

thanks

<!-- gh-comment-id:2629095128 --> @pirate commented on GitHub (Feb 1, 2025): Sure I'll take the PR :) thanks
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
dev
main
abx-dl-refactor
claude/issue-1688-20251229-2233
claude/s3-support-migration-JZQcf
claude/review-snapshot-archive-RJOkr
claude/review-code-quality-macdO
claude/reduce-chrome-duplication-JgWVr
fix/chrome-args-comma-parsing
claude/issue-668-20251229-2145
claude/issue-1664-20251229-2236
v086
claude/improve-test-suite-xm6Bh
claude/typescript-archivebox-rewrite-011CUmPyEmECnkXwyHRT9RPF
claude/archivebox-schema-orm-comparison-011CUn26GurEpLiBUq4zjVLJ
stable
logfire
plugins-browsertrix
v072
link-removal2
v0.8.6rc1
v0.7.3
v0.8.6rc0
v0.8.5rc53
v0.8.5rc51
v0.8.5rc50
v0.8.5rc49
v0.8.5rc48
v0.8.5rc47
v0.8.5rc46
v0.8.5rc45
v0.8.5rc44
v0.8.5rc43
v0.8.5rc42
v0.8.5rc41
v0.8.5rc40
v0.8.5rc39
v0.8.5rc38
v0.8.5rc37
v0.8.5rc36
v0.8.5rc35
v0.8.5rc34
v0.8.5rc33
v0.8.5rc32
v0.8.5rc31
v0.8.5rc30
v0.8.5rc29
v0.8.5rc28
v0.8.5rc27
v0.8.5rc26
v0.8.5rc25
v0.8.5rc24
v0.8.5rc23
v0.8.5rc22
v0.8.5rc13
v0.8.5rc12
v0.8.5rc11
v0.8.5rc10
v0.8.5rc9
v0.8.5rc8
v0.8.5rc6
v0.8.5rc5
v0.8.5rc4
v0.8.5rc3
v0.8.5rc2
v0.8.5-rc
v0.8.4-rc
v0.8.3-rc
v0.8.2-rc
v0.8.0-rc
v0.7.2
v0.7.1
v0.7.0
v0.6.2
v0.6.0
v0.5.6
v0.5.4
v0.5.3
v0.4.24
v0.4.21
v0.4.20
v0.4.19
v0.4.18
v0.4.17
v0.4.16
v0.4.15
v0.4.14
v0.4.13
v0.4.12
v0.4.11
v0.4.9
v0.2.4
v0.2.3
v0.2.2
v0.2.1
v0.2.0
v0.1.0
v0.0.3
v0.0.2
v0.0.1
Labels
Clear labels   
expected: maybe someday

   
expected: next release

   
expected: release after next

   
expected: unlikely unless contributed

   
good first ticket

   
help wanted

   
pull-request

Mirrored from GitHub Pull Request

  
scope: all users

   
scope: windows users

   
size: easy

   
size: hard

   
size: medium

   
size: medium

   
status: backlog

   
status: blocked

   
status: done

   
status: idea-phase

   
status: needs followup

   
status: wip

   
status: wontfix

   
touches: API/CLI/Spec

   
touches: configuration

   
touches: data/schema/architecture

   
touches: dependencies/packaging

   
touches: docs

   
touches: js

   
touches: views/replayers/html/css

   
why: correctness

   
why: functionality

   
why: performance

   
why: security
No labels
expected: maybe someday
expected: next release
expected: release after next
expected: unlikely unless contributed
good first ticket
help wanted
pull-request
scope: all users
scope: windows users
size: easy
size: hard
size: medium
size: medium
status: backlog
status: blocked
status: done
status: idea-phase
status: needs followup
status: wip
status: wontfix
touches: API/CLI/Spec
touches: configuration
touches: data/schema/architecture
touches: dependencies/packaging
touches: docs
touches: js
touches: views/replayers/html/css
why: correctness
why: functionality
why: performance
why: security
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/ArchiveBox#4001
No description provided.