[GH-ISSUE #1562] Bug: DEBUG mode broken on dev (ModuleNotFoundError: No module named 'django_autotyping') #3953

New issue

Closed

opened 2026-03-15 01:05:48 +03:00 by kerem · 1 comment

kerem commented

2026-03-15 01:05:48 +03:00

Owner

Originally created by @nguyenmp on GitHub (Oct 21, 2024).
Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/1562

Describe the bug

When running archivebox:dev in debug mode, it crashes trying to use django-autotyping and requests-tracker.

I think what's happening is these are defined as "dev-dependencies" in pyproject.toml and aren't packaged in the server on docker builds, so when we run the server via docker, debug mode leads to crashes importing these two.

Workaround: manually disable these two in archivebox/core/settings.py with something like DEBUG_REQUESTS_TRACKER = False.

For context, I wanted debug mode to figure out https://github.com/ArchiveBox/ArchiveBox/issues/1449#issuecomment-2427125470.

Steps to reproduce

docker run -p 8000:8000 archivebox/archivebox:dev --debug

Screenshots or log output

docker run -p 8000:8000 archivebox/archivebox:dev --debug
╭──────────────────────────────────────────────────────────────────────────────╮
│ [2024-10-21 18:28:58] ArchiveBox v0.8.5rc50: archivebox help                 │
╰──────────────────────────────────────────────────────────────────────────────╯

╭───────   [X] Error while trying to load database! ───────╮
│                                                          │
│ ModuleNotFoundError: No module named 'django_autotyping' │
│ Please check your config and DATA_DIR permissions.       │
│                                                          │
╰─────────────── NO WRITES CAN BE PERFORMED ───────────────╯

╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /app/archivebox/config/django.py:96 in setup_django                          │
│                                                                              │
│    93 │   │   │   │   # Otherwise use default sqlite3 file-based database an │
│    94 │   │   │   │   # without running migrations automatically (user runs  │
│    95 │   │   │   │   try:                                                   │
│ ❱  96 │   │   │   │   │   django.setup()                                     │
│    97 │   │   │   │   except Exception as e:                                 │
│    98 │   │   │   │   │   bump_startup_progress_bar(advance=1000)            │
│    99                                                                        │
│                                                                              │
│ /usr/local/lib/python3.11/site-packages/django/__init__.py:19 in setup       │
│                                                                              │
│   16 │   from django.urls import set_script_prefix                           │
│   17 │   from django.utils.log import configure_logging                      │
│   18 │                                                                       │
│ ❱ 19 │   configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)        │
│   20 │   if set_prefix:                                                      │
│   21 │   │   set_script_prefix(                                              │
│   22 │   │   │   "/" if settings.FORCE_SCRIPT_NAME is None else settings.FOR │
│                                                                              │
│ /usr/local/lib/python3.11/site-packages/django/conf/__init__.py:81 in        │
│ __getattr__                                                                  │
│                                                                              │
│    78 │   def __getattr__(self, name):                                       │
│    79 │   │   """Return the value of a setting and cache it in self.__dict__ │
│    80 │   │   if (_wrapped := self._wrapped) is empty:                       │
│ ❱  81 │   │   │   self._setup(name)                                          │
│    82 │   │   │   _wrapped = self._wrapped                                   │
│    83 │   │   val = getattr(_wrapped, name)                                  │
│    84                                                                        │
│                                                                              │
│ /usr/local/lib/python3.11/site-packages/django/conf/__init__.py:68 in _setup │
│                                                                              │
│    65 │   │   │   │   % (desc, ENVIRONMENT_VARIABLE)                         │
│    66 │   │   │   )                                                          │
│    67 │   │                                                                  │
│ ❱  68 │   │   self._wrapped = Settings(settings_module)                      │
│    69 │                                                                      │
│    70 │   def __repr__(self):                                                │
│    71 │   │   # Hardcode the class name as otherwise it yields 'Settings'.   │
│                                                                              │
│ /usr/local/lib/python3.11/site-packages/django/conf/__init__.py:166 in       │
│ __init__                                                                     │
│                                                                              │
│   163 │   │   # store the settings module in case someone later cares        │
│   164 │   │   self.SETTINGS_MODULE = settings_module                         │
│   165 │   │                                                                  │
│ ❱ 166 │   │   mod = importlib.import_module(self.SETTINGS_MODULE)            │
│   167 │   │                                                                  │
│   168 │   │   tuple_settings = (                                             │
│   169 │   │   │   "ALLOWED_HOSTS",                                           │
│                                                                              │
│ /usr/local/lib/python3.11/importlib/__init__.py:126 in import_module         │
│                                                                              │
│   123 │   │   │   if character != '.':                                       │
│   124 │   │   │   │   break                                                  │
│   125 │   │   │   level += 1                                                 │
│ ❱ 126 │   return _bootstrap._gcd_import(name[level:], package, level)        │
│   127                                                                        │
│   128                                                                        │
│   129 _RELOADING = {}                                                        │
│ in _gcd_import:1204                                                          │
│ in _find_and_load:1176                                                       │
│ in _find_and_load_unlocked:1147                                              │
│ in _load_unlocked:690                                                        │
│ in exec_module:940                                                           │
│ in _call_with_frames_removed:241                                             │
│                                                                              │
│ /app/archivebox/core/settings.py:566 in <module>                             │
│                                                                              │
│   563 │   MIDDLEWARE = [*MIDDLEWARE, 'debug_toolbar.middleware.DebugToolbarM │
│   564                                                                        │
│   565 if DEBUG:                                                              │
│ ❱ 566 │   from django_autotyping.typing import AutotypingSettingsDict        │
│   567 │                                                                      │
│   568 │   INSTALLED_APPS += ['django_autotyping']                            │
│   569 │   AUTOTYPING: AutotypingSettingsDict = {                             │
╰──────────────────────────────────────────────────────────────────────────────╯
ModuleNotFoundError: No module named 'django_autotyping'


Docker Usage:
    # using Docker Compose:
    docker compose run archivebox [command] [...args] [--help] [--version]

    # using Docker:
    docker run -v $PWD:/data -p 8000:8000 -it archivebox/archivebox [command] 
[...args] [--help] [--version]

Usage:
    # outside of Docker:
    archivebox [command] [...args] [--help] [--version]

Commands:
    help                 Print the ArchiveBox help message and usage
    version              Print the ArchiveBox version and dependency information

    init                 Initialize a new ArchiveBox collection in the current 
directory
    install              Automatically install all ArchiveBox dependencies and 
extras

    add                  Add a new URL or list of URLs to your archive
    remove               Remove the specified URLs from the archive
    update               Import any new links from subscriptions and retry any 
previously failed/skipped links
    list                 List, filter, and export information about archive 
entries
    status               Print out some info and statistics about the archive 
collection
    schedule             Set ArchiveBox to regularly import URLs at specific 
times using cron
    server               Run the ArchiveBox HTTP server
    shell                Enter an interactive ArchiveBox Django shell
    manage               Run an ArchiveBox Django management command

    config               Get and set your ArchiveBox project configuration 
values

Documentation:
    https://github.com/ArchiveBox/ArchiveBox/wiki
    https://github.com/ArchiveBox/ArchiveBox/wiki/Docker
    https://github.com/ArchiveBox/ArchiveBox/wiki/Usage
    https://github.com/ArchiveBox/ArchiveBox/wiki/Configuration

╭──────────────────── ❌ No collection is currently active ────────────────────╮
│                                                                              │
│ Hint: When using Docker, you need to mount a volume to use as your data dir: │
│     docker run -v /some/path/data:/data archivebox/archivebox ...            │
│                                                                              │
│ To load an existing collection:                                              │
│     1. cd ~/archivebox/data     # go into existing DATA DIR (can be          │
│ anywhere)                                                                    │
│     2. docker ... archivebox init          # migrate to latest version (safe │
│ to run multiple times)                                                       │
│     3. docker ... archivebox install       # auto-update all plugins (wget,  │
│ chrome, singlefile, etc.)                                                    │
│     4. docker ... archivebox help          # ...get help with next steps...  │
│                                                                              │
│ To start a new collection:                                                   │
│     1. mkdir ~/archivebox/data  # create a new, empty DATA DIR (can be       │
│ anywhere)                                                                    │
│     2. cd ~/archivebox/data     # cd into the new directory                  │
│     3. docker ... archivebox init          # initialize ArchiveBox in the    │
│ new data dir                                                                 │
│     4. docker ... archivebox install       # auto-install all plugins (wget, │
│ chrome, singlefile, etc.)                                                    │
│     5. docker ... archivebox help          # ... get help with next steps... │
│                                                                              │
╰── All archivebox commands should be run from inside a collection DATA DIR ───╯

ArchiveBox version

docker run -p 8000:8000 archivebox/archivebox:dev version  

0.8.5rc50
ArchiveBox v0.8.5rc50 COMMIT_HASH=1dff8ba BUILD_TIME=2024-10-21 10:47:52 
1729507672
IN_DOCKER=True IN_QEMU=False ARCH=aarch64 OS=Linux 
PLATFORM=Linux-6.10.4-linuxkit-aarch64-with-glibc2.36 PYTHON=Cpython
EUID=911:911 UID=911:911 PUID=911:911 FS_UID=911:911 FS_PERMS=644 FS_ATOMIC=True
FS_REMOTE=True
DEBUG=False IS_TTY=False SUDO=False ID=195c3bcf:a5bdacad SEARCH_BACKEND=ripgrep 
LDAP=False

╭─────────────── ❗ No collection DATA_DIR is currently active ───────────────╮
│                                                                             │
│ Hint: cd into a collection DATA_DIR and run archivebox version again...     │
│       OR run archivebox init to create a new collection in the current dir. │
│                                                                             │
│       (this is REQUIRED if you are opening a Github Issue to get help)      │
│                                                                             │
╰─── Full version info is only available when inside a collection DATA DIR ───╯

Originally created by @nguyenmp on GitHub (Oct 21, 2024). Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/1562 #### Describe the bug When running archivebox:dev in debug mode, it crashes trying to use `django-autotyping` and `requests-tracker`. I think what's happening is these are defined as "dev-dependencies" in `pyproject.toml` and aren't packaged in the server on docker builds, so when we run the server via docker, debug mode leads to crashes importing these two. Workaround: manually disable these two in `archivebox/core/settings.py` with something like `DEBUG_REQUESTS_TRACKER = False`. For context, I wanted debug mode to figure out https://github.com/ArchiveBox/ArchiveBox/issues/1449#issuecomment-2427125470. #### Steps to reproduce 1. `docker run -p 8000:8000 archivebox/archivebox:dev --debug` #### Screenshots or log output ``` docker run -p 8000:8000 archivebox/archivebox:dev --debug ╭──────────────────────────────────────────────────────────────────────────────╮ │ [2024-10-21 18:28:58] ArchiveBox v0.8.5rc50: archivebox help │ ╰──────────────────────────────────────────────────────────────────────────────╯ ╭─────── [X] Error while trying to load database! ───────╮ │ │ │ ModuleNotFoundError: No module named 'django_autotyping' │ │ Please check your config and DATA_DIR permissions. │ │ │ ╰─────────────── NO WRITES CAN BE PERFORMED ───────────────╯ ╭───────────────────── Traceback (most recent call last) ──────────────────────╮ │ /app/archivebox/config/django.py:96 in setup_django │ │ │ │ 93 │ │ │ │ # Otherwise use default sqlite3 file-based database an │ │ 94 │ │ │ │ # without running migrations automatically (user runs │ │ 95 │ │ │ │ try: │ │ ❱ 96 │ │ │ │ │ django.setup() │ │ 97 │ │ │ │ except Exception as e: │ │ 98 │ │ │ │ │ bump_startup_progress_bar(advance=1000) │ │ 99 │ │ │ │ /usr/local/lib/python3.11/site-packages/django/__init__.py:19 in setup │ │ │ │ 16 │ from django.urls import set_script_prefix │ │ 17 │ from django.utils.log import configure_logging │ │ 18 │ │ │ ❱ 19 │ configure_logging(settings.LOGGING_CONFIG, settings.LOGGING) │ │ 20 │ if set_prefix: │ │ 21 │ │ set_script_prefix( │ │ 22 │ │ │ "/" if settings.FORCE_SCRIPT_NAME is None else settings.FOR │ │ │ │ /usr/local/lib/python3.11/site-packages/django/conf/__init__.py:81 in │ │ __getattr__ │ │ │ │ 78 │ def __getattr__(self, name): │ │ 79 │ │ """Return the value of a setting and cache it in self.__dict__ │ │ 80 │ │ if (_wrapped := self._wrapped) is empty: │ │ ❱ 81 │ │ │ self._setup(name) │ │ 82 │ │ │ _wrapped = self._wrapped │ │ 83 │ │ val = getattr(_wrapped, name) │ │ 84 │ │ │ │ /usr/local/lib/python3.11/site-packages/django/conf/__init__.py:68 in _setup │ │ │ │ 65 │ │ │ │ % (desc, ENVIRONMENT_VARIABLE) │ │ 66 │ │ │ ) │ │ 67 │ │ │ │ ❱ 68 │ │ self._wrapped = Settings(settings_module) │ │ 69 │ │ │ 70 │ def __repr__(self): │ │ 71 │ │ # Hardcode the class name as otherwise it yields 'Settings'. │ │ │ │ /usr/local/lib/python3.11/site-packages/django/conf/__init__.py:166 in │ │ __init__ │ │ │ │ 163 │ │ # store the settings module in case someone later cares │ │ 164 │ │ self.SETTINGS_MODULE = settings_module │ │ 165 │ │ │ │ ❱ 166 │ │ mod = importlib.import_module(self.SETTINGS_MODULE) │ │ 167 │ │ │ │ 168 │ │ tuple_settings = ( │ │ 169 │ │ │ "ALLOWED_HOSTS", │ │ │ │ /usr/local/lib/python3.11/importlib/__init__.py:126 in import_module │ │ │ │ 123 │ │ │ if character != '.': │ │ 124 │ │ │ │ break │ │ 125 │ │ │ level += 1 │ │ ❱ 126 │ return _bootstrap._gcd_import(name[level:], package, level) │ │ 127 │ │ 128 │ │ 129 _RELOADING = {} │ │ in _gcd_import:1204 │ │ in _find_and_load:1176 │ │ in _find_and_load_unlocked:1147 │ │ in _load_unlocked:690 │ │ in exec_module:940 │ │ in _call_with_frames_removed:241 │ │ │ │ /app/archivebox/core/settings.py:566 in <module> │ │ │ │ 563 │ MIDDLEWARE = [*MIDDLEWARE, 'debug_toolbar.middleware.DebugToolbarM │ │ 564 │ │ 565 if DEBUG: │ │ ❱ 566 │ from django_autotyping.typing import AutotypingSettingsDict │ │ 567 │ │ │ 568 │ INSTALLED_APPS += ['django_autotyping'] │ │ 569 │ AUTOTYPING: AutotypingSettingsDict = { │ ╰──────────────────────────────────────────────────────────────────────────────╯ ModuleNotFoundError: No module named 'django_autotyping' Docker Usage: # using Docker Compose: docker compose run archivebox [command] [...args] [--help] [--version] # using Docker: docker run -v $PWD:/data -p 8000:8000 -it archivebox/archivebox [command] [...args] [--help] [--version] Usage: # outside of Docker: archivebox [command] [...args] [--help] [--version] Commands: help Print the ArchiveBox help message and usage version Print the ArchiveBox version and dependency information init Initialize a new ArchiveBox collection in the current directory install Automatically install all ArchiveBox dependencies and extras add Add a new URL or list of URLs to your archive remove Remove the specified URLs from the archive update Import any new links from subscriptions and retry any previously failed/skipped links list List, filter, and export information about archive entries status Print out some info and statistics about the archive collection schedule Set ArchiveBox to regularly import URLs at specific times using cron server Run the ArchiveBox HTTP server shell Enter an interactive ArchiveBox Django shell manage Run an ArchiveBox Django management command config Get and set your ArchiveBox project configuration values Documentation: https://github.com/ArchiveBox/ArchiveBox/wiki https://github.com/ArchiveBox/ArchiveBox/wiki/Docker https://github.com/ArchiveBox/ArchiveBox/wiki/Usage https://github.com/ArchiveBox/ArchiveBox/wiki/Configuration ╭──────────────────── ❌ No collection is currently active ────────────────────╮ │ │ │ Hint: When using Docker, you need to mount a volume to use as your data dir: │ │ docker run -v /some/path/data:/data archivebox/archivebox ... │ │ │ │ To load an existing collection: │ │ 1. cd ~/archivebox/data # go into existing DATA DIR (can be │ │ anywhere) │ │ 2. docker ... archivebox init # migrate to latest version (safe │ │ to run multiple times) │ │ 3. docker ... archivebox install # auto-update all plugins (wget, │ │ chrome, singlefile, etc.) │ │ 4. docker ... archivebox help # ...get help with next steps... │ │ │ │ To start a new collection: │ │ 1. mkdir ~/archivebox/data # create a new, empty DATA DIR (can be │ │ anywhere) │ │ 2. cd ~/archivebox/data # cd into the new directory │ │ 3. docker ... archivebox init # initialize ArchiveBox in the │ │ new data dir │ │ 4. docker ... archivebox install # auto-install all plugins (wget, │ │ chrome, singlefile, etc.) │ │ 5. docker ... archivebox help # ... get help with next steps... │ │ │ ╰── All archivebox commands should be run from inside a collection DATA DIR ───╯ ``` #### ArchiveBox version ``` docker run -p 8000:8000 archivebox/archivebox:dev version 0.8.5rc50 ArchiveBox v0.8.5rc50 COMMIT_HASH=1dff8ba BUILD_TIME=2024-10-21 10:47:52 1729507672 IN_DOCKER=True IN_QEMU=False ARCH=aarch64 OS=Linux PLATFORM=Linux-6.10.4-linuxkit-aarch64-with-glibc2.36 PYTHON=Cpython EUID=911:911 UID=911:911 PUID=911:911 FS_UID=911:911 FS_PERMS=644 FS_ATOMIC=True FS_REMOTE=True DEBUG=False IS_TTY=False SUDO=False ID=195c3bcf:a5bdacad SEARCH_BACKEND=ripgrep LDAP=False ╭─────────────── ❗ No collection DATA_DIR is currently active ───────────────╮ │ │ │ Hint: cd into a collection DATA_DIR and run archivebox version again... │ │ OR run archivebox init to create a new collection in the current dir. │ │ │ │ (this is REQUIRED if you are opening a Github Issue to get help) │ │ │ ╰─── Full version info is only available when inside a collection DATA DIR ───╯ ```

kerem

2026-03-15 01:05:48 +03:00

closed this issue
added the
why: functionality

size: medium

expected: next release

status: backlog

touches: dependencies/packaging
labels

kerem commented

2026-03-15 01:06:09 +03:00

Author

Owner

@pirate commented on GitHub (Oct 21, 2024):

Good catch, I'll add these to the production Dockerfile so --debug is availble.

@pirate commented on GitHub (Oct 21, 2024): Good catch, I'll add these to the production Dockerfile so `--debug` is availble.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/ArchiveBox#3953

No description provided.

Rows
Columns