starred/ArchiveBox
1
0
Fork 0
mirror of https://github.com/ArchiveBox/ArchiveBox.git synced 2026-04-25 17:16:00 +03:00
Code Issues 624 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #1544] Bug: failing to start supervisord in v0.8.5rc44 #2425

New issue
Closed
opened 2026-03-01 17:58:56 +03:00 by kerem · 11 comments
kerem commented 2026-03-01 17:58:56 +03:00
Owner
Copy link

Originally created by @tdec on GitHub (Oct 17, 2024).
Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/1544

Describe the bug

supervisord fails to start with v0.8.5rc44 Docker build

Steps to reproduce

  1. Build container with docker build -t archivebox:dev https://github.com/ArchiveBox/ArchiveBox.git#dev
  2. Run archivebox init. docker run --name archivebox -v /data:/data -p 8000:8000 archivebox:dev archivebox init
  3. Run container with docker run --name archivebox -v /data:/data -p 8000:8000 archivebox:dev
  4. ArchiveBox fails to start

Screenshots or log output

 /usr/bin/docker run --name archivebox -v /data:/data -p 8000:8000 archivebox:dev
╭──────────────────────────────────────────────────────────────────────────────╮
│ [2024-10-17 15:15:07] ArchiveBox v0.8.5rc44: archivebox server --quick-init  │
│ 0.0.0.0:8000                                                                 │
╰──────────────────────────────────────────────────────────────────────────────╯

[*] Verifying and updating existing ArchiveBox collection to v0.8.5rc44...
----------------------------------------------------------------------

[*] Verifying archive folder structure...
    + ./archive, ./sources, ./logs...
    + ./ArchiveBox.conf...

[*] Verifying main SQL index and running any migrations needed...
    Operations to perform:
    Apply all migrations: admin, api, auth, contenttypes, core, huey_monitor, machine, sessions, singlefile
    Running migrations:
    No migrations to apply.
    Operations to perform:
    Apply all migrations: huey_monitor
    Running migrations:
    No migrations to apply.

    √ ./index.sqlite3

[*] Checking links from indexes and archive folders (safe to Ctrl+C)...
    √ Loaded 146 links from existing main index.
    > Skipping full snapshot directory check (quick mode)

----------------------------------------------------------------------
[√] Done. Verified and updated the existing ArchiveBox collection.

[+] Starting ArchiveBox webserver...
    > Logging errors to ./logs/errors.log
    > Starting ArchiveBox webserver on http://0.0.0.0:8000

[🦸‍♂️] Supervisord starting...
2024-10-17 15:15:09,878 WARN No file matches via include "/data/tmp/3226a4d7/workers/*.conf"
2024-10-17 15:15:09,878 INFO Set uid to user 911 succeeded
Unlinking stale socket /data/tmp/3226a4d7/supervisord.sock
Unlinking stale socket /data/tmp/3226a4d7/supervisord.sock
Unlinking stale socket /data/tmp/3226a4d7/supervisord.sock
Unlinking stale socket /data/tmp/3226a4d7/supervisord.sock
Unlinking stale socket /data/tmp/3226a4d7/supervisord.sock
Unlinking stale socket /data/tmp/3226a4d7/supervisord.sock
Unlinking stale socket /data/tmp/3226a4d7/supervisord.sock
Unlinking stale socket /data/tmp/3226a4d7/supervisord.sock
Unlinking stale socket /data/tmp/3226a4d7/supervisord.sock
Unlinking stale socket /data/tmp/3226a4d7/supervisord.sock
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /usr/local/bin/archivebox:8 in <module>                                      │
│                                                                              │
│   6 if __name__ == '__main__':                                               │
│   7 │   sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])     │
│ ❱ 8 │   sys.exit(main())                                                     │
│   9                                                                          │
│                                                                              │
│ ╭────────────────────────────── locals ───────────────────────────────╮      │
│ │  re = <module 're' from '/usr/local/lib/python3.11/re/__init__.py'> │      │
│ │ sys = <module 'sys' (built-in)>                                     │      │
│ ╰─────────────────────────────────────────────────────────────────────╯      │
│                                                                              │
│ /app/archivebox/cli/__init__.py:253 in main                                  │
│                                                                              │
│   251 │                                                                      │
│   252 │   try:                                                               │
│ ❱ 253 │   │   run_subcommand(                                                │
│   254 │   │   │   subcommand=command.subcommand,                             │
│   255 │   │   │   subcommand_args=command.subcommand_args,                   │
│                                                                              │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │    args = ['server', '--quick-init', '0.0.0.0:8000']                     │ │
│ │ command = Namespace(help=False, version=False, subcommand='server',      │ │
│ │           subcommand_args=['--quick-init', '0.0.0.0:8000'])              │ │
│ │   group = <argparse._MutuallyExclusiveGroup object at 0x7f437e8dcb10>    │ │
│ │  parser = ArgumentParser(prog='archivebox', usage=None,                  │ │
│ │           description='ArchiveBox: The self-hosted internet archive',    │ │
│ │           formatter_class=<class 'argparse.HelpFormatter'>,              │ │
│ │           conflict_handler='error', add_help=False)                      │ │
│ │     pwd = None                                                           │ │
│ │   stdin = <_io.TextIOWrapper name='<stdin>' mode='r' encoding='utf-8'>   │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│                                                                              │
│ /app/archivebox/cli/__init__.py:179 in run_subcommand                        │
│                                                                              │
│   177 │                                                                      │
│   178 │   module = import_module('.archivebox_{}'.format(subcommand), __pack │
│ ❱ 179 │   module.main(args=subcommand_args, stdin=stdin, pwd=pwd)    # type: │
│   180 │                                                                      │
│   181 │   # wait for webhooks, signals, and other background jobs to finish  │
│                                                                              │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │        check_db = False                                                  │ │
│ │ cmd_requires_db = True                                                   │ │
│ │    init_pending = True                                                   │ │
│ │          module = <module 'archivebox.cli.archivebox_server' from        │ │
│ │                   '/app/archivebox/cli/archivebox_server.py'>            │ │
│ │             pwd = None                                                   │ │
│ │           stdin = <_io.TextIOWrapper name='<stdin>' mode='r'             │ │
│ │                   encoding='utf-8'>                                      │ │
│ │      subcommand = 'server'                                               │ │
│ │ subcommand_args = ['--quick-init', '0.0.0.0:8000']                       │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│                                                                              │
│ /app/archivebox/cli/archivebox_server.py:70 in main                          │
│                                                                              │
│   68 │   reject_stdin(__command__, stdin)                                    │
│   69 │                                                                       │
│ ❱ 70 │   server(                                                             │
│   71 │   │   runserver_args=command.runserver_args + (['--nothreading'] if c │
│   72 │   │   reload=command.reload,                                          │
│                                                                              │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │    args = ['--quick-init', '0.0.0.0:8000']                               │ │
│ │ command = Namespace(runserver_args=['0.0.0.0:8000'], reload=False,       │ │
│ │           debug=False, nothreading=False, init=False, quick_init=True,   │ │
│ │           createsuperuser=False, daemonize=False)                        │ │
│ │  parser = ArgumentParser(prog='archivebox server', usage=None,           │ │
│ │           description='Run the ArchiveBox HTTP server',                  │ │
│ │           formatter_class=<class                                         │ │
│ │           'archivebox.logging_util.SmartFormatter'>,                     │ │
│ │           conflict_handler='error', add_help=True)                       │ │
│ │     pwd = None                                                           │ │
│ │   stdin = <_io.TextIOWrapper name='<stdin>' mode='r' encoding='utf-8'>   │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│                                                                              │
│ /app/archivebox/misc/util.py:163 in typechecked_function                     │
│                                                                              │
│   161 │   │   │   check_argument_type(arg_key, arg_val)                      │
│   162 │   │                                                                  │
│ ❱ 163 │   │   return func(*args, **kwargs)                                   │
│   164 │                                                                      │
│   165 │   return typechecked_function                                        │
│                                                                              │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ arg_key = 'out_dir'                                                      │ │
│ │ arg_val = PosixPath('/data')                                             │ │
│ │    args = ()                                                             │ │
│ │  kwargs = {                                                              │ │
│ │           │   'runserver_args': ['0.0.0.0:8000'],                        │ │
│ │           │   'reload': False,                                           │ │
│ │           │   'debug': False,                                            │ │
│ │           │   'init': False,                                             │ │
│ │           │   'quick_init': True,                                        │ │
│ │           │   'createsuperuser': False,                                  │ │
│ │           │   'daemonize': False,                                        │ │
│ │           │   'out_dir': PosixPath('/data')                              │ │
│ │           }                                                              │ │
│ │     sig = <Signature (runserver_args: Optional[List[str]] = None,        │ │
│ │           reload: bool = False, debug: bool = False, init: bool = False, │ │
│ │           quick_init: bool = False, createsuperuser: bool = False,       │ │
│ │           daemonize: bool = False, out_dir: pathlib.Path =               │ │
│ │           PosixPath('/data')) -> None>                                   │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│                                                                              │
│ /app/archivebox/main.py:1440 in server                                       │
│                                                                              │
│   1438 │   │   print()                                                       │
│   1439 │   │                                                                 │
│ ❱ 1440 │   │   start_server_workers(host=host, port=port, daemonize=False)   │
│   1441 │   │                                                                 │
│   1442 │   │   print("\n[i][green][🟩] ArchiveBox server shut down gracefull │
│                                                                              │
│ ╭─────────────── locals ───────────────╮                                     │
│ │ createsuperuser = False              │                                     │
│ │       daemonize = False              │                                     │
│ │           debug = False              │                                     │
│ │            host = '0.0.0.0'          │                                     │
│ │   host_and_port = '0.0.0.0:8000'     │                                     │
│ │            init = False              │                                     │
│ │         out_dir = PosixPath('/data') │                                     │
│ │            port = '8000'             │                                     │
│ │      quick_init = True               │                                     │
│ │          reload = False              │                                     │
│ │  runserver_args = ['0.0.0.0:8000']   │                                     │
│ ╰──────────────────────────────────────╯                                     │
│                                                                              │
│ /app/archivebox/queues/supervisor_util.py:285 in start_server_workers        │
│                                                                              │
│   283                                                                        │
│   284 def start_server_workers(host='0.0.0.0', port='8000', daemonize=False) │
│ ❱ 285 │   supervisor = get_or_create_supervisord_process(daemonize=daemonize │
│   286 │                                                                      │
│   287 │   bg_workers = [                                                     │
│                                                                              │
│ ╭─────── locals ────────╮                                                    │
│ │ daemonize = False     │                                                    │
│ │      host = '0.0.0.0' │                                                    │
│ │      port = '8000'    │                                                    │
│ ╰───────────────────────╯                                                    │
│                                                                              │
│ /app/archivebox/queues/supervisor_util.py:163 in                             │
│ get_or_create_supervisord_process                                            │
│                                                                              │
│   161 │   │   time.sleep(0.5)                                                │
│   162 │                                                                      │
│ ❱ 163 │   assert supervisor, "Failed to start supervisord or connect to it!" │
│   164 │   supervisor.getPID()  # make sure it doesn't throw an exception     │
│   165                                                                        │
│                                                                              │
│ ╭────── locals ──────╮                                                       │
│ │  daemonize = False │                                                       │
│ │ supervisor = None  │                                                       │
│ ╰────────────────────╯                                                       │
╰──────────────────────────────────────────────────────────────────────────────╯
AssertionError: Failed to start supervisord or connect to it!
Unlinking stale socket /data/tmp/3226a4d7/supervisord.sock

The content of the directory the script is looking for is

ls -lR 3226a4d7/
3226a4d7/:
total 2
-rwxrwxrwx 1 root root 555 Oct 17 15:15 supervisord.conf
srwxrwxrwx 1 root root   1 Oct 17 15:15 supervisord.sock.131

The local data directory is a drive mounted over USB
/dev/sdb2 on /mnt/sdb2 type fuseblk (rw,relatime,user_id=0,group_id=0,allow_other,blksize=4096)

ArchiveBox version

0.8.5rc44
ArchiveBox v0.8.5rc44 COMMIT_HASH=unknown BUILD_TIME=2024-10-17 15:13:37
1729178017
IN_DOCKER=True IN_QEMU=False ARCH=x86_64 OS=Linux
PLATFORM=Linux-5.15.0-122-generic-x86_64-with-glibc2.36 PYTHON=Cpython
EUID=911:0 UID=911:0 PUID=911:0 FS_UID=0:0 FS_PERMS=644 FS_ATOMIC=True
FS_REMOTE=True
DEBUG=False IS_TTY=False SUDO=False ID=3226a4d7:80e862a2 SEARCH_BACKEND=ripgrep
LDAP=False

 Binary Dependencies:
 √  pip                   24.0.0       sys_pip    /usr/local/bin/pip
 √  pipx                  1.1.0        sys_pip    /bin/pipx
 √  python                3.11.10      sys_pip    /usr/local/bin/python3.11
 √  sqlite                2.6.0        sys_pip    /usr/local/lib/python3.11/site-packages/django/db/backends/sqlite3/base.py
 √  django                5.1.2        sys_pip    /usr/local/lib/python3.11/site-packages/django/__init__.py
 √  node                  22.9.0       apt        /usr/bin/node
 √  npm                   10.9.0       apt        /usr/bin/npm
 √  npx                   10.9.0       apt        /usr/bin/npx
 √  playwright            1.47.0       sys_pip    /usr/local/bin/playwright
 √  puppeteer             23.6.0       lib_npm    ~/.npm/bin/puppeteer
 √  rg                    13.0.0       apt        /usr/bin/rg
 √  chrome                129.0.6668   env        /usr/bin/chromium-browser
 √  curl                  8.10.1       apt        /usr/bin/curl
 √  git                   2.39.5       apt        /usr/bin/git
 √  postlight-parser      2.2.3        sys_npm    ~/.npm/bin/postlight-parser
 √  readability-extractor 0.0.11       lib_npm    ~/.npm/bin/readability-extractor
 √  single-file           1.1.54       lib_npm    ~/.npm/bin/single-file
 √  wget                  1.21.3       apt        /usr/bin/wget
 √  yt-dlp                2024.10.7    env        /usr/local/bin/yt-dlp
 √  ffmpeg                5.1.6        env        /usr/bin/ffmpeg

 Package Managers:
 √  sys_pip     /usr/local/bin/pip                                   UID=911  P…
 √  pipx        /usr/bin/pipx                                        UID=911  P…
 -  venv_pip    not available                                        UID=911  P…
 -  lib_pip     not available                                        UID=911  P…
 √  sys_npm     /usr/bin/npm                                         UID=911  P…
 -  lib_npm     /usr/bin/npm                                         UID=911  P…
 √  playwright  /usr/local/bin/playwright                            UID=0    P…
 √  puppeteer   /usr/bin/npx                                         UID=911  P…
 √  env         /usr/bin/which                                       UID=911  P…
 -  brew        not available                                        UID=911  P…
 √  apt         /usr/bin/apt-get                                     UID=0    P…

 Code locations:
 √  PACKAGE_DIR           29 files        valid     /app/archivebox
 √  TEMPLATES_DIR         3 files         valid     /app/archivebox/templates
 -  CUSTOM_TEMPLATES_DIR  missing         unused    ./user_templates
 -  USER_PLUGINS_DIR      missing         unused    ./user_plugins
 √  LIB_DIR               1 files         valid     ./lib/x86_64-linux-docker

 Data locations:
 √  DATA_DIR              14 files @      valid     /data
 √  CONFIG_FILE           106.0 Bytes     valid     ./ArchiveBox.conf
 √  SQL_INDEX             924.0 KB        valid     ./index.sqlite3
 √  QUEUE_DATABASE        92.0 KB         valid     ./queue.sqlite3
 √  ARCHIVE_DIR           20 files        valid     ./archive
 √  SOURCES_DIR           11 files        valid     ./sources
 √  PERSONAS_DIR          1 files         valid     ./personas
 √  LOGS_DIR              2 files         valid     ./logs
 √  TMP_DIR               2 files         valid     ./tmp/3226a4d7
Originally created by @tdec on GitHub (Oct 17, 2024). Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/1544 #### Describe the bug supervisord fails to start with v0.8.5rc44 Docker build #### Steps to reproduce 1. Build container with docker build -t archivebox:dev https://github.com/ArchiveBox/ArchiveBox.git#dev 2. Run archivebox init. docker run --name archivebox -v /data:/data -p 8000:8000 archivebox:dev archivebox init 3. Run container with docker run --name archivebox -v /data:/data -p 8000:8000 archivebox:dev 4. ArchiveBox fails to start #### Screenshots or log output ```logs /usr/bin/docker run --name archivebox -v /data:/data -p 8000:8000 archivebox:dev ╭──────────────────────────────────────────────────────────────────────────────╮ │ [2024-10-17 15:15:07] ArchiveBox v0.8.5rc44: archivebox server --quick-init │ │ 0.0.0.0:8000 │ ╰──────────────────────────────────────────────────────────────────────────────╯ [*] Verifying and updating existing ArchiveBox collection to v0.8.5rc44... ---------------------------------------------------------------------- [*] Verifying archive folder structure... + ./archive, ./sources, ./logs... + ./ArchiveBox.conf... [*] Verifying main SQL index and running any migrations needed... Operations to perform: Apply all migrations: admin, api, auth, contenttypes, core, huey_monitor, machine, sessions, singlefile Running migrations: No migrations to apply. Operations to perform: Apply all migrations: huey_monitor Running migrations: No migrations to apply. √ ./index.sqlite3 [*] Checking links from indexes and archive folders (safe to Ctrl+C)... √ Loaded 146 links from existing main index. > Skipping full snapshot directory check (quick mode) ---------------------------------------------------------------------- [√] Done. Verified and updated the existing ArchiveBox collection. [+] Starting ArchiveBox webserver... > Logging errors to ./logs/errors.log > Starting ArchiveBox webserver on http://0.0.0.0:8000 [🦸‍♂️] Supervisord starting... 2024-10-17 15:15:09,878 WARN No file matches via include "/data/tmp/3226a4d7/workers/*.conf" 2024-10-17 15:15:09,878 INFO Set uid to user 911 succeeded Unlinking stale socket /data/tmp/3226a4d7/supervisord.sock Unlinking stale socket /data/tmp/3226a4d7/supervisord.sock Unlinking stale socket /data/tmp/3226a4d7/supervisord.sock Unlinking stale socket /data/tmp/3226a4d7/supervisord.sock Unlinking stale socket /data/tmp/3226a4d7/supervisord.sock Unlinking stale socket /data/tmp/3226a4d7/supervisord.sock Unlinking stale socket /data/tmp/3226a4d7/supervisord.sock Unlinking stale socket /data/tmp/3226a4d7/supervisord.sock Unlinking stale socket /data/tmp/3226a4d7/supervisord.sock Unlinking stale socket /data/tmp/3226a4d7/supervisord.sock ╭───────────────────── Traceback (most recent call last) ──────────────────────╮ │ /usr/local/bin/archivebox:8 in <module> │ │ │ │ 6 if __name__ == '__main__': │ │ 7 │ sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) │ │ ❱ 8 │ sys.exit(main()) │ │ 9 │ │ │ │ ╭────────────────────────────── locals ───────────────────────────────╮ │ │ │ re = <module 're' from '/usr/local/lib/python3.11/re/__init__.py'> │ │ │ │ sys = <module 'sys' (built-in)> │ │ │ ╰─────────────────────────────────────────────────────────────────────╯ │ │ │ │ /app/archivebox/cli/__init__.py:253 in main │ │ │ │ 251 │ │ │ 252 │ try: │ │ ❱ 253 │ │ run_subcommand( │ │ 254 │ │ │ subcommand=command.subcommand, │ │ 255 │ │ │ subcommand_args=command.subcommand_args, │ │ │ │ ╭───────────────────────────────── locals ─────────────────────────────────╮ │ │ │ args = ['server', '--quick-init', '0.0.0.0:8000'] │ │ │ │ command = Namespace(help=False, version=False, subcommand='server', │ │ │ │ subcommand_args=['--quick-init', '0.0.0.0:8000']) │ │ │ │ group = <argparse._MutuallyExclusiveGroup object at 0x7f437e8dcb10> │ │ │ │ parser = ArgumentParser(prog='archivebox', usage=None, │ │ │ │ description='ArchiveBox: The self-hosted internet archive', │ │ │ │ formatter_class=<class 'argparse.HelpFormatter'>, │ │ │ │ conflict_handler='error', add_help=False) │ │ │ │ pwd = None │ │ │ │ stdin = <_io.TextIOWrapper name='<stdin>' mode='r' encoding='utf-8'> │ │ │ ╰──────────────────────────────────────────────────────────────────────────╯ │ │ │ │ /app/archivebox/cli/__init__.py:179 in run_subcommand │ │ │ │ 177 │ │ │ 178 │ module = import_module('.archivebox_{}'.format(subcommand), __pack │ │ ❱ 179 │ module.main(args=subcommand_args, stdin=stdin, pwd=pwd) # type: │ │ 180 │ │ │ 181 │ # wait for webhooks, signals, and other background jobs to finish │ │ │ │ ╭───────────────────────────────── locals ─────────────────────────────────╮ │ │ │ check_db = False │ │ │ │ cmd_requires_db = True │ │ │ │ init_pending = True │ │ │ │ module = <module 'archivebox.cli.archivebox_server' from │ │ │ │ '/app/archivebox/cli/archivebox_server.py'> │ │ │ │ pwd = None │ │ │ │ stdin = <_io.TextIOWrapper name='<stdin>' mode='r' │ │ │ │ encoding='utf-8'> │ │ │ │ subcommand = 'server' │ │ │ │ subcommand_args = ['--quick-init', '0.0.0.0:8000'] │ │ │ ╰──────────────────────────────────────────────────────────────────────────╯ │ │ │ │ /app/archivebox/cli/archivebox_server.py:70 in main │ │ │ │ 68 │ reject_stdin(__command__, stdin) │ │ 69 │ │ │ ❱ 70 │ server( │ │ 71 │ │ runserver_args=command.runserver_args + (['--nothreading'] if c │ │ 72 │ │ reload=command.reload, │ │ │ │ ╭───────────────────────────────── locals ─────────────────────────────────╮ │ │ │ args = ['--quick-init', '0.0.0.0:8000'] │ │ │ │ command = Namespace(runserver_args=['0.0.0.0:8000'], reload=False, │ │ │ │ debug=False, nothreading=False, init=False, quick_init=True, │ │ │ │ createsuperuser=False, daemonize=False) │ │ │ │ parser = ArgumentParser(prog='archivebox server', usage=None, │ │ │ │ description='Run the ArchiveBox HTTP server', │ │ │ │ formatter_class=<class │ │ │ │ 'archivebox.logging_util.SmartFormatter'>, │ │ │ │ conflict_handler='error', add_help=True) │ │ │ │ pwd = None │ │ │ │ stdin = <_io.TextIOWrapper name='<stdin>' mode='r' encoding='utf-8'> │ │ │ ╰──────────────────────────────────────────────────────────────────────────╯ │ │ │ │ /app/archivebox/misc/util.py:163 in typechecked_function │ │ │ │ 161 │ │ │ check_argument_type(arg_key, arg_val) │ │ 162 │ │ │ │ ❱ 163 │ │ return func(*args, **kwargs) │ │ 164 │ │ │ 165 │ return typechecked_function │ │ │ │ ╭───────────────────────────────── locals ─────────────────────────────────╮ │ │ │ arg_key = 'out_dir' │ │ │ │ arg_val = PosixPath('/data') │ │ │ │ args = () │ │ │ │ kwargs = { │ │ │ │ │ 'runserver_args': ['0.0.0.0:8000'], │ │ │ │ │ 'reload': False, │ │ │ │ │ 'debug': False, │ │ │ │ │ 'init': False, │ │ │ │ │ 'quick_init': True, │ │ │ │ │ 'createsuperuser': False, │ │ │ │ │ 'daemonize': False, │ │ │ │ │ 'out_dir': PosixPath('/data') │ │ │ │ } │ │ │ │ sig = <Signature (runserver_args: Optional[List[str]] = None, │ │ │ │ reload: bool = False, debug: bool = False, init: bool = False, │ │ │ │ quick_init: bool = False, createsuperuser: bool = False, │ │ │ │ daemonize: bool = False, out_dir: pathlib.Path = │ │ │ │ PosixPath('/data')) -> None> │ │ │ ╰──────────────────────────────────────────────────────────────────────────╯ │ │ │ │ /app/archivebox/main.py:1440 in server │ │ │ │ 1438 │ │ print() │ │ 1439 │ │ │ │ ❱ 1440 │ │ start_server_workers(host=host, port=port, daemonize=False) │ │ 1441 │ │ │ │ 1442 │ │ print("\n[i][green][🟩] ArchiveBox server shut down gracefull │ │ │ │ ╭─────────────── locals ───────────────╮ │ │ │ createsuperuser = False │ │ │ │ daemonize = False │ │ │ │ debug = False │ │ │ │ host = '0.0.0.0' │ │ │ │ host_and_port = '0.0.0.0:8000' │ │ │ │ init = False │ │ │ │ out_dir = PosixPath('/data') │ │ │ │ port = '8000' │ │ │ │ quick_init = True │ │ │ │ reload = False │ │ │ │ runserver_args = ['0.0.0.0:8000'] │ │ │ ╰──────────────────────────────────────╯ │ │ │ │ /app/archivebox/queues/supervisor_util.py:285 in start_server_workers │ │ │ │ 283 │ │ 284 def start_server_workers(host='0.0.0.0', port='8000', daemonize=False) │ │ ❱ 285 │ supervisor = get_or_create_supervisord_process(daemonize=daemonize │ │ 286 │ │ │ 287 │ bg_workers = [ │ │ │ │ ╭─────── locals ────────╮ │ │ │ daemonize = False │ │ │ │ host = '0.0.0.0' │ │ │ │ port = '8000' │ │ │ ╰───────────────────────╯ │ │ │ │ /app/archivebox/queues/supervisor_util.py:163 in │ │ get_or_create_supervisord_process │ │ │ │ 161 │ │ time.sleep(0.5) │ │ 162 │ │ │ ❱ 163 │ assert supervisor, "Failed to start supervisord or connect to it!" │ │ 164 │ supervisor.getPID() # make sure it doesn't throw an exception │ │ 165 │ │ │ │ ╭────── locals ──────╮ │ │ │ daemonize = False │ │ │ │ supervisor = None │ │ │ ╰────────────────────╯ │ ╰──────────────────────────────────────────────────────────────────────────────╯ AssertionError: Failed to start supervisord or connect to it! Unlinking stale socket /data/tmp/3226a4d7/supervisord.sock ``` The content of the directory the script is looking for is ``` ls -lR 3226a4d7/ 3226a4d7/: total 2 -rwxrwxrwx 1 root root 555 Oct 17 15:15 supervisord.conf srwxrwxrwx 1 root root 1 Oct 17 15:15 supervisord.sock.131 ``` The local data directory is a drive mounted over USB `/dev/sdb2 on /mnt/sdb2 type fuseblk (rw,relatime,user_id=0,group_id=0,allow_other,blksize=4096)` #### ArchiveBox version <!-- Run the `archivebox version` command locally then copy paste the result here: --> ```logs 0.8.5rc44 ArchiveBox v0.8.5rc44 COMMIT_HASH=unknown BUILD_TIME=2024-10-17 15:13:37 1729178017 IN_DOCKER=True IN_QEMU=False ARCH=x86_64 OS=Linux PLATFORM=Linux-5.15.0-122-generic-x86_64-with-glibc2.36 PYTHON=Cpython EUID=911:0 UID=911:0 PUID=911:0 FS_UID=0:0 FS_PERMS=644 FS_ATOMIC=True FS_REMOTE=True DEBUG=False IS_TTY=False SUDO=False ID=3226a4d7:80e862a2 SEARCH_BACKEND=ripgrep LDAP=False Binary Dependencies: √ pip 24.0.0 sys_pip /usr/local/bin/pip √ pipx 1.1.0 sys_pip /bin/pipx √ python 3.11.10 sys_pip /usr/local/bin/python3.11 √ sqlite 2.6.0 sys_pip /usr/local/lib/python3.11/site-packages/django/db/backends/sqlite3/base.py √ django 5.1.2 sys_pip /usr/local/lib/python3.11/site-packages/django/__init__.py √ node 22.9.0 apt /usr/bin/node √ npm 10.9.0 apt /usr/bin/npm √ npx 10.9.0 apt /usr/bin/npx √ playwright 1.47.0 sys_pip /usr/local/bin/playwright √ puppeteer 23.6.0 lib_npm ~/.npm/bin/puppeteer √ rg 13.0.0 apt /usr/bin/rg √ chrome 129.0.6668 env /usr/bin/chromium-browser √ curl 8.10.1 apt /usr/bin/curl √ git 2.39.5 apt /usr/bin/git √ postlight-parser 2.2.3 sys_npm ~/.npm/bin/postlight-parser √ readability-extractor 0.0.11 lib_npm ~/.npm/bin/readability-extractor √ single-file 1.1.54 lib_npm ~/.npm/bin/single-file √ wget 1.21.3 apt /usr/bin/wget √ yt-dlp 2024.10.7 env /usr/local/bin/yt-dlp √ ffmpeg 5.1.6 env /usr/bin/ffmpeg Package Managers: √ sys_pip /usr/local/bin/pip UID=911 P… √ pipx /usr/bin/pipx UID=911 P… - venv_pip not available UID=911 P… - lib_pip not available UID=911 P… √ sys_npm /usr/bin/npm UID=911 P… - lib_npm /usr/bin/npm UID=911 P… √ playwright /usr/local/bin/playwright UID=0 P… √ puppeteer /usr/bin/npx UID=911 P… √ env /usr/bin/which UID=911 P… - brew not available UID=911 P… √ apt /usr/bin/apt-get UID=0 P… Code locations: √ PACKAGE_DIR 29 files valid /app/archivebox √ TEMPLATES_DIR 3 files valid /app/archivebox/templates - CUSTOM_TEMPLATES_DIR missing unused ./user_templates - USER_PLUGINS_DIR missing unused ./user_plugins √ LIB_DIR 1 files valid ./lib/x86_64-linux-docker Data locations: √ DATA_DIR 14 files @ valid /data √ CONFIG_FILE 106.0 Bytes valid ./ArchiveBox.conf √ SQL_INDEX 924.0 KB valid ./index.sqlite3 √ QUEUE_DATABASE 92.0 KB valid ./queue.sqlite3 √ ARCHIVE_DIR 20 files valid ./archive √ SOURCES_DIR 11 files valid ./sources √ PERSONAS_DIR 1 files valid ./personas √ LOGS_DIR 2 files valid ./logs √ TMP_DIR 2 files valid ./tmp/3226a4d7 ```
kerem commented 2026-03-01 17:58:57 +03:00
Author
Owner
Copy link

@pirate commented on GitHub (Oct 17, 2024):

Please run chown -R 911 data outside of docker. If possible also change your fstab entry to not force the volume ownership to be 0:0, ideally use 911:0. (If not 911 then set the PUID env var to the value you prefer)

Archivebox does not work with volumes that are forcibly set to root ownership as it always tries to drop permissions to a non root user at startup.

There is some info in the troubleshooting docs in the wiki about this.

<!-- gh-comment-id:2419941994 --> @pirate commented on GitHub (Oct 17, 2024): Please run `chown -R 911 data` outside of docker. If possible also change your fstab entry to not force the volume ownership to be 0:0, ideally use 911:0. (If not 911 then set the PUID env var to the value you prefer) Archivebox does not work with volumes that are forcibly set to root ownership as it always tries to drop permissions to a non root user at startup. There is some info in the troubleshooting docs in the wiki about this.
kerem commented 2026-03-01 17:58:57 +03:00
Author
Owner
Copy link

@pirate commented on GitHub (Oct 17, 2024):

Here's the relevant docs: https://github.com/ArchiveBox/ArchiveBox/wiki/Configuration#puid--pgid

The fstab entry you have is basically the same as root_squash on an NFS mount, it's forcing ownership to 0:0, which may prevent ArchiveBox from changing it to 911:0.

<!-- gh-comment-id:2420060678 --> @pirate commented on GitHub (Oct 17, 2024): Here's the relevant docs: https://github.com/ArchiveBox/ArchiveBox/wiki/Configuration#puid--pgid The fstab entry you have is basically the same as root_squash on an NFS mount, it's forcing ownership to 0:0, which may prevent ArchiveBox from changing it to 911:0.
kerem commented 2026-03-01 17:58:57 +03:00
Author
Owner
Copy link

@nightbrd commented on GitHub (Oct 18, 2024):

Hey there, I'm currently also having the same problem, but I'm running it through Docker on Windows, so I'm not sure how to go on about this, as I'm also not as savvy as some people, or it's simply me getting limited by Windows. This problem only started in version 0.8.5rc43.

╭──────────────────────────────────────────────────────────────────────────────╮
│ [2024-10-18 02:34:10] ArchiveBox v0.8.5rc44: archivebox server --quick-init  │
│ 0.0.0.0:8000                                                                 │
╰──────────────────────────────────────────────────────────────────────────────╯

[*] Verifying and updating existing ArchiveBox collection to v0.8.5rc44...
----------------------------------------------------------------------

[*] Verifying archive folder structure...
    + ./archive, ./sources, ./logs...
    + ./ArchiveBox.conf...

[*] Verifying main SQL index and running any migrations needed...
    Operations to perform:
    Apply all migrations: admin, api, auth, contenttypes, core, huey_monitor, machine, sessions, singlefile
    Running migrations:
    No migrations to apply.
    Operations to perform:
    Apply all migrations: huey_monitor
    Running migrations:
    No migrations to apply.

    √ ./index.sqlite3

[*] Checking links from indexes and archive folders (safe to Ctrl+C)...
    √ Loaded 830 links from existing main index.
    > Skipping full snapshot directory check (quick mode)

----------------------------------------------------------------------
[√] Done. Verified and updated the existing ArchiveBox collection.

[+] Starting ArchiveBox webserver...
    > Logging errors to ./logs/errors.log
    > Starting ArchiveBox webserver on http://0.0.0.0:8000

Error connecting to existing supervisord: [Errno 111] Connection refused
[🦸‍♂️] Stopping supervisord process (pid=131)...
[🦸‍♂️] Supervisord starting...
2024-10-18 02:34:12,837 WARN No file matches via include "/data/tmp/c42fa52e/workers/*.conf"
2024-10-18 02:34:12,838 INFO Set uid to user 911 succeeded
2024-10-18 02:34:12,850 INFO RPC interface 'supervisor' initialized
2024-10-18 02:34:12,850 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2024-10-18 02:34:12,853 INFO supervisord started with pid 131
Error connecting to existing supervisord: [Errno 111] Connection refused
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /usr/local/bin/archivebox:8 in <module>                                      │
│                                                                              │
│   6 if __name__ == '__main__':                                               │
│   7 │   sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])     │
│ ❱ 8 │   sys.exit(main())                                                     │
│   9                                                                          │
│                                                                              │
│ ╭────────────────────────────── locals ───────────────────────────────╮      │
│ │  re = <module 're' from '/usr/local/lib/python3.11/re/__init__.py'> │      │
│ │ sys = <module 'sys' (built-in)>                                     │      │
│ ╰─────────────────────────────────────────────────────────────────────╯      │
│                                                                              │
│ /app/archivebox/cli/__init__.py:253 in main                                  │
│                                                                              │
│   251 │                                                                      │
│   252 │   try:                                                               │
│ ❱ 253 │   │   run_subcommand(                                                │
│   254 │   │   │   subcommand=command.subcommand,                             │
│   255 │   │   │   subcommand_args=command.subcommand_args,                   │
│                                                                              │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │    args = ['server', '--quick-init', '0.0.0.0:8000']                     │ │
│ │ command = Namespace(help=False, version=False, subcommand='server',      │ │
│ │           subcommand_args=['--quick-init', '0.0.0.0:8000'])              │ │
│ │   group = <argparse._MutuallyExclusiveGroup object at 0x7f955d33e0d0>    │ │
│ │  parser = ArgumentParser(prog='archivebox', usage=None,                  │ │
│ │           description='ArchiveBox: The self-hosted internet archive',    │ │
│ │           formatter_class=<class 'argparse.HelpFormatter'>,              │ │
│ │           conflict_handler='error', add_help=False)                      │ │
│ │     pwd = None                                                           │ │
│ │   stdin = <_io.TextIOWrapper name='<stdin>' mode='r' encoding='utf-8'>   │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│                                                                              │
│ /app/archivebox/cli/__init__.py:179 in run_subcommand                        │
│                                                                              │
│   177 │                                                                      │
│   178 │   module = import_module('.archivebox_{}'.format(subcommand), __pack │
│ ❱ 179 │   module.main(args=subcommand_args, stdin=stdin, pwd=pwd)    # type: │
│   180 │                                                                      │
│   181 │   # wait for webhooks, signals, and other background jobs to finish  │
│                                                                              │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │        check_db = False                                                  │ │
│ │ cmd_requires_db = True                                                   │ │
│ │    init_pending = True                                                   │ │
│ │          module = <module 'archivebox.cli.archivebox_server' from        │ │
│ │                   '/app/archivebox/cli/archivebox_server.py'>            │ │
│ │             pwd = None                                                   │ │
│ │           stdin = <_io.TextIOWrapper name='<stdin>' mode='r'             │ │
│ │                   encoding='utf-8'>                                      │ │
│ │      subcommand = 'server'                                               │ │
│ │ subcommand_args = ['--quick-init', '0.0.0.0:8000']                       │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│                                                                              │
│ /app/archivebox/cli/archivebox_server.py:70 in main                          │
│                                                                              │
│   68 │   reject_stdin(__command__, stdin)                                    │
│   69 │                                                                       │
│ ❱ 70 │   server(                                                             │
│   71 │   │   runserver_args=command.runserver_args + (['--nothreading'] if c │
│   72 │   │   reload=command.reload,                                          │
│                                                                              │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │    args = ['--quick-init', '0.0.0.0:8000']                               │ │
│ │ command = Namespace(runserver_args=['0.0.0.0:8000'], reload=False,       │ │
│ │           debug=False, nothreading=False, init=False, quick_init=True,   │ │
│ │           createsuperuser=False, daemonize=False)                        │ │
│ │  parser = ArgumentParser(prog='archivebox server', usage=None,           │ │
│ │           description='Run the ArchiveBox HTTP server',                  │ │
│ │           formatter_class=<class                                         │ │
│ │           'archivebox.logging_util.SmartFormatter'>,                     │ │
│ │           conflict_handler='error', add_help=True)                       │ │
│ │     pwd = None                                                           │ │
│ │   stdin = <_io.TextIOWrapper name='<stdin>' mode='r' encoding='utf-8'>   │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│                                                                              │
│ /app/archivebox/misc/util.py:163 in typechecked_function                     │
│                                                                              │
│   161 │   │   │   check_argument_type(arg_key, arg_val)                      │
│   162 │   │                                                                  │
│ ❱ 163 │   │   return func(*args, **kwargs)                                   │
│   164 │                                                                      │
│   165 │   return typechecked_function                                        │
│                                                                              │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ arg_key = 'out_dir'                                                      │ │
│ │ arg_val = PosixPath('/data')                                             │ │
│ │    args = ()                                                             │ │
│ │  kwargs = {                                                              │ │
│ │           │   'runserver_args': ['0.0.0.0:8000'],                        │ │
│ │           │   'reload': False,                                           │ │
│ │           │   'debug': False,                                            │ │
│ │           │   'init': False,                                             │ │
│ │           │   'quick_init': True,                                        │ │
│ │           │   'createsuperuser': False,                                  │ │
│ │           │   'daemonize': False,                                        │ │
│ │           │   'out_dir': PosixPath('/data')                              │ │
│ │           }                                                              │ │
│ │     sig = <Signature (runserver_args: Optional[List[str]] = None,        │ │
│ │           reload: bool = False, debug: bool = False, init: bool = False, │ │
│ │           quick_init: bool = False, createsuperuser: bool = False,       │ │
│ │           daemonize: bool = False, out_dir: pathlib.Path =               │ │
│ │           PosixPath('/data')) -> None>                                   │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│                                                                              │
│ /app/archivebox/main.py:1440 in server                                       │
│                                                                              │
│   1438 │   │   print()                                                       │
│   1439 │   │                                                                 │
│ ❱ 1440 │   │   start_server_workers(host=host, port=port, daemonize=False)   │
│   1441 │   │                                                                 │
│   1442 │   │   print("\n[i][green][🟩] ArchiveBox server shut down gracefull │
│                                                                              │
│ ╭─────────────── locals ───────────────╮                                     │
│ │ createsuperuser = False              │                                     │
│ │       daemonize = False              │                                     │
│ │           debug = False              │                                     │
│ │            host = '0.0.0.0'          │                                     │
│ │   host_and_port = '0.0.0.0:8000'     │                                     │
│ │            init = False              │                                     │
│ │         out_dir = PosixPath('/data') │                                     │
│ │            port = '8000'             │                                     │
│ │      quick_init = True               │                                     │
│ │          reload = False              │                                     │
│ │  runserver_args = ['0.0.0.0:8000']   │                                     │
│ ╰──────────────────────────────────────╯                                     │
│                                                                              │
│ /app/archivebox/queues/supervisor_util.py:285 in start_server_workers        │
│                                                                              │
│   283                                                                        │
│   284 def start_server_workers(host='0.0.0.0', port='8000', daemonize=False) │
│ ❱ 285 │   supervisor = get_or_create_supervisord_process(daemonize=daemonize │
│   286 │                                                                      │
│   287 │   bg_workers = [                                                     │
│                                                                              │
│ ╭─────── locals ────────╮                                                    │
│ │ daemonize = False     │                                                    │
│ │      host = '0.0.0.0' │                                                    │
│ │      port = '8000'    │                                                    │
│ ╰───────────────────────╯                                                    │
│                                                                              │
│ /app/archivebox/queues/supervisor_util.py:163 in                             │
│ get_or_create_supervisord_process                                            │
│                                                                              │
│   161 │   │   time.sleep(0.5)                                                │
│   162 │                                                                      │
│ ❱ 163 │   assert supervisor, "Failed to start supervisord or connect to it!" │
│   164 │   supervisor.getPID()  # make sure it doesn't throw an exception     │
│   165                                                                        │
│                                                                              │
│ ╭────── locals ──────╮                                                       │
│ │  daemonize = False │                                                       │
│ │ supervisor = None  │                                                       │
│ ╰────────────────────╯                                                       │
╰──────────────────────────────────────────────────────────────────────────────╯
AssertionError: Failed to start supervisord or connect to it!


 
0.8.5rc44
ArchiveBox v0.8.5rc44 COMMIT_HASH=d820257 BUILD_TIME=2024-10-11 09:24:55 1728638695
IN_DOCKER=True IN_QEMU=False ARCH=x86_64 OS=Linux 
PLATFORM=Linux-5.15.153.1-microsoft-standard-WSL2-x86_64-with-glibc2.36 PYTHON=Cpython
EUID=911:911 UID=911:911 PUID=911:911 FS_UID=911:911 FS_PERMS=644 FS_ATOMIC=True FS_REMOTE=True
DEBUG=False IS_TTY=True SUDO=False ID=c42fa52e:7ca74139 SEARCH_BACKEND=sonic LDAP=False

 Binary Dependencies:
 √  pip                   24.0.0       sys_pip    /usr/local/bin/pip
 √  pipx                  1.1.0        sys_pip    /usr/bin/pipx
 √  python                3.11.10      sys_pip    /usr/local/bin/python3.11
 √  sqlite                2.6.0        sys_pip    /usr/local/lib/python3.11/site-packages/django/db/backends/sqlite3/base.py
 √  django                5.1.2        sys_pip    /usr/local/lib/python3.11/site-packages/django/__init__.py
 √  node                  22.9.0       apt        /usr/bin/node
 √  npm                   10.9.0       apt        /usr/bin/npm
 √  npx                   10.9.0       apt        /usr/bin/npx
 √  playwright            1.47.0       sys_pip    /usr/local/bin/playwright
 √  puppeteer             23.5.3       lib_npm    ~/.npm/bin/puppeteer
 √  rg                    13.0.0       apt        /usr/bin/rg
 √  sonic                 1.4.9        env        /usr/local/bin/sonic
 √  chrome                129.0.6668   env        /usr/bin/chromium-browser
 √  curl                  8.10.1       apt        /usr/bin/curl
 √  git                   2.39.5       apt        /usr/bin/git
 √  postlight-parser      2.2.3        sys_npm    ~/.npm/bin/postlight-parser
 √  readability-extractor 0.0.11       lib_npm    ~/.npm/bin/readability-extractor
 √  single-file           1.1.54       lib_npm    ~/.npm/bin/single-file
 √  wget                  1.21.3       apt        /usr/bin/wget
 √  yt-dlp                2024.10.7    env        /usr/local/bin/yt-dlp
 √  ffmpeg                5.1.6        env        /usr/bin/ffmpeg

 Package Managers:
 √  sys_pip     /usr/local/bin/pip                                   UID=911  PATH=~/.local/bin:/usr/bin:/usr/local/bin
 √  pipx        /usr/bin/pipx                                        UID=911  PATH=~/.local/bin
 -  venv_pip    not available                                        UID=911  PATH=/tmp/NotInsideAVenv/lib/bin
 -  lib_pip     not available                                        UID=911  PATH=./lib/x86_64-linux-docker/pip/venv/b…
 √  sys_npm     /usr/bin/npm                                         UID=911  PATH=~/.npm/bin
 -  lib_npm     /usr/bin/npm                                         UID=911  PATH=./lib/x86_64-linux-docker/npm/node_m…
 √  playwright  /usr/local/bin/playwright                            UID=0    PATH=./lib/x86_64-linux-docker/bin:~/.npm…
 √  puppeteer   /usr/bin/npx                                         UID=911  PATH=./lib/x86_64-linux-docker/bin
 √  env         /usr/bin/which                                       UID=911  PATH=~/.npm/bin:/usr/local/bin:/usr/local…
 -  brew        not available                                        UID=911  PATH=
 √  apt         /usr/bin/apt-get                                     UID=0    PATH=/usr/bin:/bin

 Code locations:
 √  PACKAGE_DIR           34 files        valid     /app/archivebox                                                     
 √  TEMPLATES_DIR         4 files         valid     /app/archivebox/templates                                           
 -  CUSTOM_TEMPLATES_DIR  missing         unused    ./user_templates                       
 -  USER_PLUGINS_DIR      missing         unused    ./user_plugins                         
 √  LIB_DIR               1 files         valid     ./lib/x86_64-linux-docker              

 Data locations:
 √  DATA_DIR              15 files @      valid     /data                                                               
 √  CONFIG_FILE           83.0 Bytes      valid     ./ArchiveBox.conf                      
 √  SQL_INDEX             9.8 MB          valid     ./index.sqlite3                        
 √  QUEUE_DATABASE        92.0 KB         valid     ./queue.sqlite3                        
 √  ARCHIVE_DIR           0 files         valid     ./archive                              
 √  SOURCES_DIR           0 files         valid     ./sources                              
 √  PERSONAS_DIR          1 files         valid     ./personas                             
 √  LOGS_DIR              5 files         valid     ./logs                                 
 √  TMP_DIR               3 files         valid     ./tmp/c42fa52e
<!-- gh-comment-id:2421160426 --> @nightbrd commented on GitHub (Oct 18, 2024): Hey there, I'm currently also having the same problem, but I'm running it through Docker on Windows, so I'm not sure how to go on about this, as I'm also not as savvy as some people, or it's simply me getting limited by Windows. This problem only started in version 0.8.5rc43. ``` ╭──────────────────────────────────────────────────────────────────────────────╮ │ [2024-10-18 02:34:10] ArchiveBox v0.8.5rc44: archivebox server --quick-init │ │ 0.0.0.0:8000 │ ╰──────────────────────────────────────────────────────────────────────────────╯ [*] Verifying and updating existing ArchiveBox collection to v0.8.5rc44... ---------------------------------------------------------------------- [*] Verifying archive folder structure... + ./archive, ./sources, ./logs... + ./ArchiveBox.conf... [*] Verifying main SQL index and running any migrations needed... Operations to perform: Apply all migrations: admin, api, auth, contenttypes, core, huey_monitor, machine, sessions, singlefile Running migrations: No migrations to apply. Operations to perform: Apply all migrations: huey_monitor Running migrations: No migrations to apply. √ ./index.sqlite3 [*] Checking links from indexes and archive folders (safe to Ctrl+C)... √ Loaded 830 links from existing main index. > Skipping full snapshot directory check (quick mode) ---------------------------------------------------------------------- [√] Done. Verified and updated the existing ArchiveBox collection. [+] Starting ArchiveBox webserver... > Logging errors to ./logs/errors.log > Starting ArchiveBox webserver on http://0.0.0.0:8000 Error connecting to existing supervisord: [Errno 111] Connection refused [🦸‍♂️] Stopping supervisord process (pid=131)... [🦸‍♂️] Supervisord starting... 2024-10-18 02:34:12,837 WARN No file matches via include "/data/tmp/c42fa52e/workers/*.conf" 2024-10-18 02:34:12,838 INFO Set uid to user 911 succeeded 2024-10-18 02:34:12,850 INFO RPC interface 'supervisor' initialized 2024-10-18 02:34:12,850 CRIT Server 'unix_http_server' running without any HTTP authentication checking 2024-10-18 02:34:12,853 INFO supervisord started with pid 131 Error connecting to existing supervisord: [Errno 111] Connection refused ╭───────────────────── Traceback (most recent call last) ──────────────────────╮ │ /usr/local/bin/archivebox:8 in <module> │ │ │ │ 6 if __name__ == '__main__': │ │ 7 │ sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) │ │ ❱ 8 │ sys.exit(main()) │ │ 9 │ │ │ │ ╭────────────────────────────── locals ───────────────────────────────╮ │ │ │ re = <module 're' from '/usr/local/lib/python3.11/re/__init__.py'> │ │ │ │ sys = <module 'sys' (built-in)> │ │ │ ╰─────────────────────────────────────────────────────────────────────╯ │ │ │ │ /app/archivebox/cli/__init__.py:253 in main │ │ │ │ 251 │ │ │ 252 │ try: │ │ ❱ 253 │ │ run_subcommand( │ │ 254 │ │ │ subcommand=command.subcommand, │ │ 255 │ │ │ subcommand_args=command.subcommand_args, │ │ │ │ ╭───────────────────────────────── locals ─────────────────────────────────╮ │ │ │ args = ['server', '--quick-init', '0.0.0.0:8000'] │ │ │ │ command = Namespace(help=False, version=False, subcommand='server', │ │ │ │ subcommand_args=['--quick-init', '0.0.0.0:8000']) │ │ │ │ group = <argparse._MutuallyExclusiveGroup object at 0x7f955d33e0d0> │ │ │ │ parser = ArgumentParser(prog='archivebox', usage=None, │ │ │ │ description='ArchiveBox: The self-hosted internet archive', │ │ │ │ formatter_class=<class 'argparse.HelpFormatter'>, │ │ │ │ conflict_handler='error', add_help=False) │ │ │ │ pwd = None │ │ │ │ stdin = <_io.TextIOWrapper name='<stdin>' mode='r' encoding='utf-8'> │ │ │ ╰──────────────────────────────────────────────────────────────────────────╯ │ │ │ │ /app/archivebox/cli/__init__.py:179 in run_subcommand │ │ │ │ 177 │ │ │ 178 │ module = import_module('.archivebox_{}'.format(subcommand), __pack │ │ ❱ 179 │ module.main(args=subcommand_args, stdin=stdin, pwd=pwd) # type: │ │ 180 │ │ │ 181 │ # wait for webhooks, signals, and other background jobs to finish │ │ │ │ ╭───────────────────────────────── locals ─────────────────────────────────╮ │ │ │ check_db = False │ │ │ │ cmd_requires_db = True │ │ │ │ init_pending = True │ │ │ │ module = <module 'archivebox.cli.archivebox_server' from │ │ │ │ '/app/archivebox/cli/archivebox_server.py'> │ │ │ │ pwd = None │ │ │ │ stdin = <_io.TextIOWrapper name='<stdin>' mode='r' │ │ │ │ encoding='utf-8'> │ │ │ │ subcommand = 'server' │ │ │ │ subcommand_args = ['--quick-init', '0.0.0.0:8000'] │ │ │ ╰──────────────────────────────────────────────────────────────────────────╯ │ │ │ │ /app/archivebox/cli/archivebox_server.py:70 in main │ │ │ │ 68 │ reject_stdin(__command__, stdin) │ │ 69 │ │ │ ❱ 70 │ server( │ │ 71 │ │ runserver_args=command.runserver_args + (['--nothreading'] if c │ │ 72 │ │ reload=command.reload, │ │ │ │ ╭───────────────────────────────── locals ─────────────────────────────────╮ │ │ │ args = ['--quick-init', '0.0.0.0:8000'] │ │ │ │ command = Namespace(runserver_args=['0.0.0.0:8000'], reload=False, │ │ │ │ debug=False, nothreading=False, init=False, quick_init=True, │ │ │ │ createsuperuser=False, daemonize=False) │ │ │ │ parser = ArgumentParser(prog='archivebox server', usage=None, │ │ │ │ description='Run the ArchiveBox HTTP server', │ │ │ │ formatter_class=<class │ │ │ │ 'archivebox.logging_util.SmartFormatter'>, │ │ │ │ conflict_handler='error', add_help=True) │ │ │ │ pwd = None │ │ │ │ stdin = <_io.TextIOWrapper name='<stdin>' mode='r' encoding='utf-8'> │ │ │ ╰──────────────────────────────────────────────────────────────────────────╯ │ │ │ │ /app/archivebox/misc/util.py:163 in typechecked_function │ │ │ │ 161 │ │ │ check_argument_type(arg_key, arg_val) │ │ 162 │ │ │ │ ❱ 163 │ │ return func(*args, **kwargs) │ │ 164 │ │ │ 165 │ return typechecked_function │ │ │ │ ╭───────────────────────────────── locals ─────────────────────────────────╮ │ │ │ arg_key = 'out_dir' │ │ │ │ arg_val = PosixPath('/data') │ │ │ │ args = () │ │ │ │ kwargs = { │ │ │ │ │ 'runserver_args': ['0.0.0.0:8000'], │ │ │ │ │ 'reload': False, │ │ │ │ │ 'debug': False, │ │ │ │ │ 'init': False, │ │ │ │ │ 'quick_init': True, │ │ │ │ │ 'createsuperuser': False, │ │ │ │ │ 'daemonize': False, │ │ │ │ │ 'out_dir': PosixPath('/data') │ │ │ │ } │ │ │ │ sig = <Signature (runserver_args: Optional[List[str]] = None, │ │ │ │ reload: bool = False, debug: bool = False, init: bool = False, │ │ │ │ quick_init: bool = False, createsuperuser: bool = False, │ │ │ │ daemonize: bool = False, out_dir: pathlib.Path = │ │ │ │ PosixPath('/data')) -> None> │ │ │ ╰──────────────────────────────────────────────────────────────────────────╯ │ │ │ │ /app/archivebox/main.py:1440 in server │ │ │ │ 1438 │ │ print() │ │ 1439 │ │ │ │ ❱ 1440 │ │ start_server_workers(host=host, port=port, daemonize=False) │ │ 1441 │ │ │ │ 1442 │ │ print("\n[i][green][🟩] ArchiveBox server shut down gracefull │ │ │ │ ╭─────────────── locals ───────────────╮ │ │ │ createsuperuser = False │ │ │ │ daemonize = False │ │ │ │ debug = False │ │ │ │ host = '0.0.0.0' │ │ │ │ host_and_port = '0.0.0.0:8000' │ │ │ │ init = False │ │ │ │ out_dir = PosixPath('/data') │ │ │ │ port = '8000' │ │ │ │ quick_init = True │ │ │ │ reload = False │ │ │ │ runserver_args = ['0.0.0.0:8000'] │ │ │ ╰──────────────────────────────────────╯ │ │ │ │ /app/archivebox/queues/supervisor_util.py:285 in start_server_workers │ │ │ │ 283 │ │ 284 def start_server_workers(host='0.0.0.0', port='8000', daemonize=False) │ │ ❱ 285 │ supervisor = get_or_create_supervisord_process(daemonize=daemonize │ │ 286 │ │ │ 287 │ bg_workers = [ │ │ │ │ ╭─────── locals ────────╮ │ │ │ daemonize = False │ │ │ │ host = '0.0.0.0' │ │ │ │ port = '8000' │ │ │ ╰───────────────────────╯ │ │ │ │ /app/archivebox/queues/supervisor_util.py:163 in │ │ get_or_create_supervisord_process │ │ │ │ 161 │ │ time.sleep(0.5) │ │ 162 │ │ │ ❱ 163 │ assert supervisor, "Failed to start supervisord or connect to it!" │ │ 164 │ supervisor.getPID() # make sure it doesn't throw an exception │ │ 165 │ │ │ │ ╭────── locals ──────╮ │ │ │ daemonize = False │ │ │ │ supervisor = None │ │ │ ╰────────────────────╯ │ ╰──────────────────────────────────────────────────────────────────────────────╯ AssertionError: Failed to start supervisord or connect to it! ``` ``` 0.8.5rc44 ArchiveBox v0.8.5rc44 COMMIT_HASH=d820257 BUILD_TIME=2024-10-11 09:24:55 1728638695 IN_DOCKER=True IN_QEMU=False ARCH=x86_64 OS=Linux PLATFORM=Linux-5.15.153.1-microsoft-standard-WSL2-x86_64-with-glibc2.36 PYTHON=Cpython EUID=911:911 UID=911:911 PUID=911:911 FS_UID=911:911 FS_PERMS=644 FS_ATOMIC=True FS_REMOTE=True DEBUG=False IS_TTY=True SUDO=False ID=c42fa52e:7ca74139 SEARCH_BACKEND=sonic LDAP=False Binary Dependencies: √ pip 24.0.0 sys_pip /usr/local/bin/pip √ pipx 1.1.0 sys_pip /usr/bin/pipx √ python 3.11.10 sys_pip /usr/local/bin/python3.11 √ sqlite 2.6.0 sys_pip /usr/local/lib/python3.11/site-packages/django/db/backends/sqlite3/base.py √ django 5.1.2 sys_pip /usr/local/lib/python3.11/site-packages/django/__init__.py √ node 22.9.0 apt /usr/bin/node √ npm 10.9.0 apt /usr/bin/npm √ npx 10.9.0 apt /usr/bin/npx √ playwright 1.47.0 sys_pip /usr/local/bin/playwright √ puppeteer 23.5.3 lib_npm ~/.npm/bin/puppeteer √ rg 13.0.0 apt /usr/bin/rg √ sonic 1.4.9 env /usr/local/bin/sonic √ chrome 129.0.6668 env /usr/bin/chromium-browser √ curl 8.10.1 apt /usr/bin/curl √ git 2.39.5 apt /usr/bin/git √ postlight-parser 2.2.3 sys_npm ~/.npm/bin/postlight-parser √ readability-extractor 0.0.11 lib_npm ~/.npm/bin/readability-extractor √ single-file 1.1.54 lib_npm ~/.npm/bin/single-file √ wget 1.21.3 apt /usr/bin/wget √ yt-dlp 2024.10.7 env /usr/local/bin/yt-dlp √ ffmpeg 5.1.6 env /usr/bin/ffmpeg Package Managers: √ sys_pip /usr/local/bin/pip UID=911 PATH=~/.local/bin:/usr/bin:/usr/local/bin √ pipx /usr/bin/pipx UID=911 PATH=~/.local/bin - venv_pip not available UID=911 PATH=/tmp/NotInsideAVenv/lib/bin - lib_pip not available UID=911 PATH=./lib/x86_64-linux-docker/pip/venv/b… √ sys_npm /usr/bin/npm UID=911 PATH=~/.npm/bin - lib_npm /usr/bin/npm UID=911 PATH=./lib/x86_64-linux-docker/npm/node_m… √ playwright /usr/local/bin/playwright UID=0 PATH=./lib/x86_64-linux-docker/bin:~/.npm… √ puppeteer /usr/bin/npx UID=911 PATH=./lib/x86_64-linux-docker/bin √ env /usr/bin/which UID=911 PATH=~/.npm/bin:/usr/local/bin:/usr/local… - brew not available UID=911 PATH= √ apt /usr/bin/apt-get UID=0 PATH=/usr/bin:/bin Code locations: √ PACKAGE_DIR 34 files valid /app/archivebox √ TEMPLATES_DIR 4 files valid /app/archivebox/templates - CUSTOM_TEMPLATES_DIR missing unused ./user_templates - USER_PLUGINS_DIR missing unused ./user_plugins √ LIB_DIR 1 files valid ./lib/x86_64-linux-docker Data locations: √ DATA_DIR 15 files @ valid /data √ CONFIG_FILE 83.0 Bytes valid ./ArchiveBox.conf √ SQL_INDEX 9.8 MB valid ./index.sqlite3 √ QUEUE_DATABASE 92.0 KB valid ./queue.sqlite3 √ ARCHIVE_DIR 0 files valid ./archive √ SOURCES_DIR 0 files valid ./sources √ PERSONAS_DIR 1 files valid ./personas √ LOGS_DIR 5 files valid ./logs √ TMP_DIR 3 files valid ./tmp/c42fa52e ```
kerem commented 2026-03-01 17:58:57 +03:00
Author
Owner
Copy link

@pirate commented on GitHub (Oct 18, 2024):

Maybe windows drives are simply incompatible with Unix socket files?

I can do some research.

Sorry I don't really know anything about windows.

<!-- gh-comment-id:2421170171 --> @pirate commented on GitHub (Oct 18, 2024): Maybe windows drives are simply incompatible with Unix socket files? I can do some research. Sorry I don't really know anything about windows.
kerem commented 2026-03-01 17:58:58 +03:00
Author
Owner
Copy link

@kdkd commented on GitHub (Oct 18, 2024):

If it helps, the same is occurring on unraid's docker, if you tell it to pull the dev branch instead of the last release.

<!-- gh-comment-id:2421233851 --> @kdkd commented on GitHub (Oct 18, 2024): If it helps, the same is occurring on unraid's docker, if you tell it to pull the dev branch instead of the last release.
kerem commented 2026-03-01 17:58:58 +03:00
Author
Owner
Copy link

@kdkd commented on GitHub (Oct 18, 2024):

The issue seems to be that create_worker_config() is being called AFTER supervisord is being started up, so supervisord is erroring out because it can't find the workers directory.

start_server_workers calls get_or_create_supervisord_process which deletes the workers directory if it already exists, then immediately starts up supervisord which isn't happy, because it's trying to include "/data/tmp/c42fa52e/workers/*.conf". The workers directory and the files in it are only being created after supervisord is started. This was possibly working before because the directory and its contents were already there from a previous startup.

Somehow create_worker_config for all the workers needs to be called before get_or_create_supervisord_process in start_server_workers, and the cleanup of the stale workers directory needs to happen before both of these things.

<!-- gh-comment-id:2423276102 --> @kdkd commented on GitHub (Oct 18, 2024): The issue seems to be that create_worker_config() is being called AFTER supervisord is being started up, so supervisord is erroring out because it can't find the workers directory. start_server_workers calls get_or_create_supervisord_process which deletes the workers directory if it already exists, then immediately starts up supervisord which isn't happy, because it's trying to include "/data/tmp/c42fa52e/workers/*.conf". The workers directory and the files in it are only being created after supervisord is started. This was possibly working before because the directory and its contents were already there from a previous startup. Somehow create_worker_config for all the workers needs to be called before get_or_create_supervisord_process in start_server_workers, and the cleanup of the stale workers directory needs to happen before both of these things.
kerem commented 2026-03-01 17:58:58 +03:00
Author
Owner
Copy link

@pirate commented on GitHub (Oct 18, 2024):

Nah that warning is a red herring. The workers dir does not need to exist before it's started.

create_worker_config is supposed to come after supervisors starts. Because we need to confirm it will start it in a stable state without workers before we start adding them.

The worker dir is auto created lazily whenever a worker is created that depends on it.

I will hide those errors at some point, or create a dir to silence them, but for now it's not as important as the permissions error that happens when supervisord tries to create the socket. Can you share the most recent last bit in data/logs/supervisord.log

<!-- gh-comment-id:2423369237 --> @pirate commented on GitHub (Oct 18, 2024): Nah that warning is a red herring. The workers dir does not need to exist before it's started. create_worker_config is supposed to come after supervisors starts. Because we need to confirm it will start it in a stable state without workers before we start adding them. The worker dir is auto created lazily whenever a worker is created that depends on it. I will hide those errors at some point, or create a dir to silence them, but for now it's not as important as the permissions error that happens when supervisord tries to create the socket. Can you share the most recent last bit in data/logs/supervisord.log
kerem commented 2026-03-01 17:58:58 +03:00
Author
Owner
Copy link

@Mika- commented on GitHub (Oct 19, 2024):

I'm also running docker on Unraid and got the same problem.

supervisord.log says it starts:

2024-10-19 15:07:35  344 WARN No file matches via include "/data/tmp/c42fa52e/workers/*.conf"
2024-10-19 15:07:35  344 INFO Set uid to user 911 succeeded
2024-10-19 15:07:35  346 INFO RPC interface 'supervisor' initialized
2024-10-19 15:07:35  346 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2024-10-19 15:07:35  346 INFO supervisord started with pid 130

Docker on Unraid and Windows uses remote filesystem so nfs propably can't handle socket files.

<!-- gh-comment-id:2423991930 --> @Mika- commented on GitHub (Oct 19, 2024): I'm also running docker on Unraid and got the same problem. supervisord.log says it starts: ``` 2024-10-19 15:07:35 344 WARN No file matches via include "/data/tmp/c42fa52e/workers/*.conf" 2024-10-19 15:07:35 344 INFO Set uid to user 911 succeeded 2024-10-19 15:07:35 346 INFO RPC interface 'supervisor' initialized 2024-10-19 15:07:35 346 CRIT Server 'unix_http_server' running without any HTTP authentication checking 2024-10-19 15:07:35 346 INFO supervisord started with pid 130 ``` Docker on Unraid and Windows uses remote filesystem so nfs propably can't handle socket files.
kerem commented 2026-03-01 17:58:58 +03:00
Author
Owner
Copy link

@pirate commented on GitHub (Oct 19, 2024):

I can easily fix it by falling back the socket location to being inside /tmp in the container.

It's a shame though, I was hoping we could keep the socket in the bind mount, but I guess it doesn't work with too many systems / remote filesystems.

<!-- gh-comment-id:2424103024 --> @pirate commented on GitHub (Oct 19, 2024): I can easily fix it by falling back the socket location to being inside /tmp in the container. It's a shame though, I was hoping we could keep the socket in the bind mount, but I guess it doesn't work with too many systems / remote filesystems.
kerem commented 2026-03-01 17:58:58 +03:00
Author
Owner
Copy link

@pirate commented on GitHub (Oct 21, 2024):

Should be fixed in a211461f

Can you give :dev a try?

<!-- gh-comment-id:2426340038 --> @pirate commented on GitHub (Oct 21, 2024): Should be fixed in a211461f Can you give `:dev` a try?
kerem commented 2026-03-01 17:58:58 +03:00
Author
Owner
Copy link

@Mika- commented on GitHub (Oct 21, 2024):

Fix confirmed as working

<!-- gh-comment-id:2426861173 --> @Mika- commented on GitHub (Oct 21, 2024): Fix confirmed as working
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
dev
main
abx-dl-refactor
claude/issue-1688-20251229-2233
claude/s3-support-migration-JZQcf
claude/review-snapshot-archive-RJOkr
claude/review-code-quality-macdO
claude/reduce-chrome-duplication-JgWVr
fix/chrome-args-comma-parsing
claude/issue-668-20251229-2145
claude/issue-1664-20251229-2236
v086
claude/improve-test-suite-xm6Bh
claude/typescript-archivebox-rewrite-011CUmPyEmECnkXwyHRT9RPF
claude/archivebox-schema-orm-comparison-011CUn26GurEpLiBUq4zjVLJ
stable
logfire
plugins-browsertrix
v072
link-removal2
v0.8.6rc1
v0.7.3
v0.8.6rc0
v0.8.5rc53
v0.8.5rc51
v0.8.5rc50
v0.8.5rc49
v0.8.5rc48
v0.8.5rc47
v0.8.5rc46
v0.8.5rc45
v0.8.5rc44
v0.8.5rc43
v0.8.5rc42
v0.8.5rc41
v0.8.5rc40
v0.8.5rc39
v0.8.5rc38
v0.8.5rc37
v0.8.5rc36
v0.8.5rc35
v0.8.5rc34
v0.8.5rc33
v0.8.5rc32
v0.8.5rc31
v0.8.5rc30
v0.8.5rc29
v0.8.5rc28
v0.8.5rc27
v0.8.5rc26
v0.8.5rc25
v0.8.5rc24
v0.8.5rc23
v0.8.5rc22
v0.8.5rc13
v0.8.5rc12
v0.8.5rc11
v0.8.5rc10
v0.8.5rc9
v0.8.5rc8
v0.8.5rc6
v0.8.5rc5
v0.8.5rc4
v0.8.5rc3
v0.8.5rc2
v0.8.5-rc
v0.8.4-rc
v0.8.3-rc
v0.8.2-rc
v0.8.0-rc
v0.7.2
v0.7.1
v0.7.0
v0.6.2
v0.6.0
v0.5.6
v0.5.4
v0.5.3
v0.4.24
v0.4.21
v0.4.20
v0.4.19
v0.4.18
v0.4.17
v0.4.16
v0.4.15
v0.4.14
v0.4.13
v0.4.12
v0.4.11
v0.4.9
v0.2.4
v0.2.3
v0.2.2
v0.2.1
v0.2.0
v0.1.0
v0.0.3
v0.0.2
v0.0.1
Labels
Clear labels   
expected: maybe someday

   
expected: next release

   
expected: release after next

   
expected: unlikely unless contributed

   
good first ticket

   
help wanted

   
pull-request

Mirrored from GitHub Pull Request

  
scope: all users

   
scope: windows users

   
size: easy

   
size: hard

   
size: medium

   
size: medium

   
status: backlog

   
status: blocked

   
status: done

   
status: idea-phase

   
status: needs followup

   
status: wip

   
status: wontfix

   
touches: API/CLI/Spec

   
touches: configuration

   
touches: data/schema/architecture

   
touches: dependencies/packaging

   
touches: docs

   
touches: js

   
touches: views/replayers/html/css

   
why: correctness

   
why: functionality

   
why: performance

   
why: security
No labels
expected: maybe someday
expected: next release
expected: release after next
expected: unlikely unless contributed
good first ticket
help wanted
pull-request
scope: all users
scope: windows users
size: easy
size: hard
size: medium
size: medium
status: backlog
status: blocked
status: done
status: idea-phase
status: needs followup
status: wip
status: wontfix
touches: API/CLI/Spec
touches: configuration
touches: data/schema/architecture
touches: dependencies/packaging
touches: docs
touches: js
touches: views/replayers/html/css
why: correctness
why: functionality
why: performance
why: security
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/ArchiveBox#2425
No description provided.