starred/ArchiveBox

Fork 0

mirror of https://github.com/ArchiveBox/ArchiveBox.git synced 2026-04-26 01:26:00 +03:00

[GH-ISSUE #372] DJango branch: circular import problem: crontab and logging #3278

New issue

Closed

opened 2026-03-14 21:53:38 +03:00 by kerem · 7 comments

kerem commented

2026-03-14 21:53:38 +03:00

Owner

Originally created by @terxw on GitHub (Jul 19, 2020).
Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/372

Describe the bug

After updating to newest django branch I no longer can launch app

Steps to reproduce

1. Run ArchiveBox e.g. ./bin/archivebox add

Screenshots or log output

Traceback (most recent call last):                                                                                                                                                                                                            
  File "/home/kangus/src/ArchiveBox/bin/archivebox", line 7, in <module>                                                                                                                                                                      
    from .cli import main                                                                                                                                                                                                                     
  File "/home/kangus/src/ArchiveBox/archivebox/cli/__init__.py", line 65, in <module>                                                                                                                                                         
    SUBCOMMANDS = list_subcommands()                                                                                                                                                                                                          
  File "/home/kangus/src/ArchiveBox/archivebox/cli/__init__.py", line 41, in list_subcommands                                                                                                                                                 
    module = import_module('.archivebox_{}'.format(subcommand), __package__)                                                                                                                                                                  
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module                                                                                                                                                                 
    return _bootstrap._gcd_import(name[level:], package, level)                                                                                                                                                                               
  File "/home/kangus/src/ArchiveBox/archivebox/cli/archivebox_init.py", line 11, in <module>                                                                                                                                                  
    from ..main import init, docstring                                                                                                                                                                                                        
  File "/home/kangus/src/ArchiveBox/archivebox/main.py", line 8, in <module>                                                                                                                                                                  
    from crontab import CronTab, CronSlices                                                                                                                                                                                                   
  File "/usr/local/lib/python3.8/dist-packages/crontab.py", line 91, in <module>                                                                                                                                                              
    import logging                                                                                                                                                                                                                            
  File "/home/kangus/src/ArchiveBox/archivebox/logging.py", line 16, in <module>                                                                                                                                                              
    from .index.schema import Link, ArchiveResult                                                                                                                                                                                             
  File "/home/kangus/src/ArchiveBox/archivebox/index/__init__.py", line 13, in <module>                                                                                                                                                       
    from ..system import atomic_write                                                                                                                                                                                                         
  File "/home/kangus/src/ArchiveBox/archivebox/system.py", line 12, in <module>                                                                                                                                                               
    from crontab import CronTab                                                                                                                                                                                                               
ImportError: cannot import name 'CronTab' from partially initialized module 'crontab' (most likely due to a circular import) (/usr/local/lib/python3.8/dist-packages/crontab.py)                                                              
Error in sys.excepthook: 
Traceback (most recent call last):                                                                                                                                                                                                    [0/6631]
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 72, in apport_excepthook
    from apport.fileutils import likely_packaged, get_recent_crashes
  File "/usr/lib/python3/dist-packages/apport/__init__.py", line 5, in <module>
    from apport.report import Report
  File "/usr/lib/python3/dist-packages/apport/report.py", line 32, in <module>
    import apport.fileutils
  File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 12, in <module>
    import os, glob, subprocess, os.path, time, pwd, sys, requests_unixsocket
  File "/usr/lib/python3/dist-packages/requests_unixsocket/__init__.py", line 1, in <module>
    import requests
  File "/usr/lib/python3/dist-packages/requests/__init__.py", line 43, in <module>
    import urllib3
  File "/usr/lib/python3/dist-packages/urllib3/__init__.py", line 7, in <module>
    from .connectionpool import HTTPConnectionPool, HTTPSConnectionPool, connection_from_url
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 3, in <module>
    import logging
  File "/home/kangus/src/ArchiveBox/archivebox/logging.py", line 16, in <module>
    from .index.schema import Link, ArchiveResult
  File "/home/kangus/src/ArchiveBox/archivebox/index/__init__.py", line 13, in <module>
    from ..system import atomic_write
  File "/home/kangus/src/ArchiveBox/archivebox/system.py", line 12, in <module>
    from crontab import CronTab
  File "/usr/local/lib/python3.8/dist-packages/crontab.py", line 154, in <module>
    LOG = logging.getLogger('crontab')
AttributeError: partially initialized module 'logging' has no attribute 'getLogger' (most likely due to a circular import)

Original exception was:
Traceback (most recent call last):
  File "/home/kangus/src/ArchiveBox/bin/archivebox", line 7, in <module>
    from .cli import main
  File "/home/kangus/src/ArchiveBox/archivebox/cli/__init__.py", line 65, in <module>
    SUBCOMMANDS = list_subcommands()
  File "/home/kangus/src/ArchiveBox/archivebox/cli/__init__.py", line 41, in list_subcommands
    module = import_module('.archivebox_{}'.format(subcommand), __package__)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/home/kangus/src/ArchiveBox/archivebox/cli/archivebox_init.py", line 11, in <module>
    from ..main import init, docstring
  File "/home/kangus/src/ArchiveBox/archivebox/main.py", line 8, in <module>
    from crontab import CronTab, CronSlices
  File "/usr/local/lib/python3.8/dist-packages/crontab.py", line 91, in <module>
    import logging
  File "/home/kangus/src/ArchiveBox/archivebox/logging.py", line 16, in <module>
    from .index.schema import Link, ArchiveResult
  File "/home/kangus/src/ArchiveBox/archivebox/index/__init__.py", line 13, in <module>
    from ..system import atomic_write
  File "/home/kangus/src/ArchiveBox/archivebox/system.py", line 12, in <module>
    from crontab import CronTab
ImportError: cannot import name 'CronTab' from partially initialized module 'crontab' (most likely due to a circular import) (/usr/local/lib/python3.8/dist-packages/crontab.py)

Software versions

OS: Linux Mint 20/ubuntu 20.04
ArchiveBox version: 848977e
Python version: python 3.8.2

Originally created by @terxw on GitHub (Jul 19, 2020). Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/372 #### Describe the bug ``` After updating to newest django branch I no longer can launch app ``` #### Steps to reproduce ``` 1. Run ArchiveBox e.g. ./bin/archivebox add ``` #### Screenshots or log output ``` Traceback (most recent call last): File "/home/kangus/src/ArchiveBox/bin/archivebox", line 7, in <module> from .cli import main File "/home/kangus/src/ArchiveBox/archivebox/cli/__init__.py", line 65, in <module> SUBCOMMANDS = list_subcommands() File "/home/kangus/src/ArchiveBox/archivebox/cli/__init__.py", line 41, in list_subcommands module = import_module('.archivebox_{}'.format(subcommand), __package__) File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/home/kangus/src/ArchiveBox/archivebox/cli/archivebox_init.py", line 11, in <module> from ..main import init, docstring File "/home/kangus/src/ArchiveBox/archivebox/main.py", line 8, in <module> from crontab import CronTab, CronSlices File "/usr/local/lib/python3.8/dist-packages/crontab.py", line 91, in <module> import logging File "/home/kangus/src/ArchiveBox/archivebox/logging.py", line 16, in <module> from .index.schema import Link, ArchiveResult File "/home/kangus/src/ArchiveBox/archivebox/index/__init__.py", line 13, in <module> from ..system import atomic_write File "/home/kangus/src/ArchiveBox/archivebox/system.py", line 12, in <module> from crontab import CronTab ImportError: cannot import name 'CronTab' from partially initialized module 'crontab' (most likely due to a circular import) (/usr/local/lib/python3.8/dist-packages/crontab.py) Error in sys.excepthook: Traceback (most recent call last): [0/6631] File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 72, in apport_excepthook from apport.fileutils import likely_packaged, get_recent_crashes File "/usr/lib/python3/dist-packages/apport/__init__.py", line 5, in <module> from apport.report import Report File "/usr/lib/python3/dist-packages/apport/report.py", line 32, in <module> import apport.fileutils File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 12, in <module> import os, glob, subprocess, os.path, time, pwd, sys, requests_unixsocket File "/usr/lib/python3/dist-packages/requests_unixsocket/__init__.py", line 1, in <module> import requests File "/usr/lib/python3/dist-packages/requests/__init__.py", line 43, in <module> import urllib3 File "/usr/lib/python3/dist-packages/urllib3/__init__.py", line 7, in <module> from .connectionpool import HTTPConnectionPool, HTTPSConnectionPool, connection_from_url File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 3, in <module> import logging File "/home/kangus/src/ArchiveBox/archivebox/logging.py", line 16, in <module> from .index.schema import Link, ArchiveResult File "/home/kangus/src/ArchiveBox/archivebox/index/__init__.py", line 13, in <module> from ..system import atomic_write File "/home/kangus/src/ArchiveBox/archivebox/system.py", line 12, in <module> from crontab import CronTab File "/usr/local/lib/python3.8/dist-packages/crontab.py", line 154, in <module> LOG = logging.getLogger('crontab') AttributeError: partially initialized module 'logging' has no attribute 'getLogger' (most likely due to a circular import) Original exception was: Traceback (most recent call last): File "/home/kangus/src/ArchiveBox/bin/archivebox", line 7, in <module> from .cli import main File "/home/kangus/src/ArchiveBox/archivebox/cli/__init__.py", line 65, in <module> SUBCOMMANDS = list_subcommands() File "/home/kangus/src/ArchiveBox/archivebox/cli/__init__.py", line 41, in list_subcommands module = import_module('.archivebox_{}'.format(subcommand), __package__) File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/home/kangus/src/ArchiveBox/archivebox/cli/archivebox_init.py", line 11, in <module> from ..main import init, docstring File "/home/kangus/src/ArchiveBox/archivebox/main.py", line 8, in <module> from crontab import CronTab, CronSlices File "/usr/local/lib/python3.8/dist-packages/crontab.py", line 91, in <module> import logging File "/home/kangus/src/ArchiveBox/archivebox/logging.py", line 16, in <module> from .index.schema import Link, ArchiveResult File "/home/kangus/src/ArchiveBox/archivebox/index/__init__.py", line 13, in <module> from ..system import atomic_write File "/home/kangus/src/ArchiveBox/archivebox/system.py", line 12, in <module> from crontab import CronTab ImportError: cannot import name 'CronTab' from partially initialized module 'crontab' (most likely due to a circular import) (/usr/local/lib/python3.8/dist-packages/crontab.py) ``` #### Software versions - OS: Linux Mint 20/ubuntu 20.04 - ArchiveBox version: 848977e - Python version: python 3.8.2

kerem closed this issue

2026-03-14 21:53:44 +03:00

kerem commented

2026-03-14 21:53:49 +03:00

Author

Owner

@cdvv7788 commented on GitHub (Jul 20, 2020):

@terxw what page are you trying to index? I ran archivebox add https://example.com and it worked.
I will review it anyway, but if you can provide a little more information to reproduce the issue, that would be helpful.

@cdvv7788 commented on GitHub (Jul 20, 2020): @terxw what page are you trying to index? I ran `archivebox add https://example.com` and it worked. I will review it anyway, but if you can provide a little more information to reproduce the issue, that would be helpful.

kerem commented

2026-03-14 21:53:55 +03:00

Author

Owner

@cdvv7788 commented on GitHub (Jul 20, 2020):

I was able to reproduce it!

@cdvv7788 commented on GitHub (Jul 20, 2020): I was able to reproduce it!

kerem commented

2026-03-14 21:54:00 +03:00

Author

Owner

@cdvv7788 commented on GitHub (Jul 20, 2020):

@terxw I had the issue with circular imports, but not from crontab. Can you please pull the latest django branch and try again.

@cdvv7788 commented on GitHub (Jul 20, 2020): @terxw I had the issue with circular imports, but not from crontab. Can you please pull the latest `django` branch and try again.

kerem commented

2026-03-14 21:54:05 +03:00

Author

Owner

@terxw commented on GitHub (Jul 22, 2020):

The error appeared at any command e.g. archivebox add, update, and with any url.
I tried updating to the new version 0965031 after fix by cdvv7788 and I got this error:

most recent call last):
  File "./bin/archivebox", line 7, in <module>
    from .cli import main
  File "/home/kangus/src/ArchiveBox/archivebox/cli/__init__.py", line 65, in <module>
    SUBCOMMANDS = list_subcommands()
  File "/home/kangus/src/ArchiveBox/archivebox/cli/__init__.py", line 41, in list_subcommands
    module = import_module('.archivebox_{}'.format(subcommand), __package__)
  File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/home/kangus/src/ArchiveBox/archivebox/cli/archivebox_help.py", line 11, in <module>
    from ..main import help, docstring
  File "/home/kangus/src/ArchiveBox/archivebox/main.py", line 8, in <module>
    from crontab import CronTab, CronSlices
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 618, in _load_backward_compatible
  File "<frozen zipimport>", line 259, in load_module
  File "/home/kangus/.local/share/virtualenvs/archivebox_django/lib/python3.8/site-packages/python_crontab-2.5.1-py3.8.egg/crontab.py", line 91, in <module>
  File "/home/kangus/src/ArchiveBox/archivebox/logging.py", line 19, in <module>
    from .util import enforce_types
  File "/home/kangus/src/ArchiveBox/archivebox/util.py", line 14, in <module>
    import requests
  File "/home/kangus/.local/share/virtualenvs/archivebox_django/lib/python3.8/site-packages/requests/__init__.py", line 43, in <module>
    import urllib3
  File "/home/kangus/.local/share/virtualenvs/archivebox_django/lib/python3.8/site-packages/urllib3/__init__.py", line 7, in <module>
    from .connectionpool import HTTPConnectionPool, HTTPSConnectionPool, connection_from_url
  File "/home/kangus/.local/share/virtualenvs/archivebox_django/lib/python3.8/site-packages/urllib3/connectionpool.py", line 29, in <module>
    from .connection import (
  File "/home/kangus/.local/share/virtualenvs/archivebox_django/lib/python3.8/site-packages/urllib3/connection.py", line 40, in <module>
    from .util.ssl_ import (
  File "/home/kangus/.local/share/virtualenvs/archivebox_django/lib/python3.8/site-packages/urllib3/util/__init__.py", line 20, in <module>
    from .retry import Retry
  File "/home/kangus/.local/share/virtualenvs/archivebox_django/lib/python3.8/site-packages/urllib3/util/retry.py", line 20, in <module>
    log = logging.getLogger(__name__)
AttributeError: partially initialized module 'logging' has no attribute 'getLogger' (most likely due to a circular import)

i used pew for venv:

cd /home/kangus/src/ArchiveBox
pew new -p /usr/local/bin/python3.8 -a $(pwd) archivebox_django
pip install -U setuptools
python3 ./setup.,py install

python -V
Python 3.8.2
python3 -V
Python 3.8.2

It looks to my like urllib3 is trying to run getLogger() from ArchiveBox/archivebox/logging.py instead of python lgger module

@terxw commented on GitHub (Jul 22, 2020): The error appeared at any command e.g. archivebox add, update, and with any url. I tried updating to the new version 0965031 after fix by [cdvv7788 ](https://github.com/cdvv7788) and I got this error: ``` most recent call last): File "./bin/archivebox", line 7, in <module> from .cli import main File "/home/kangus/src/ArchiveBox/archivebox/cli/__init__.py", line 65, in <module> SUBCOMMANDS = list_subcommands() File "/home/kangus/src/ArchiveBox/archivebox/cli/__init__.py", line 41, in list_subcommands module = import_module('.archivebox_{}'.format(subcommand), __package__) File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/home/kangus/src/ArchiveBox/archivebox/cli/archivebox_help.py", line 11, in <module> from ..main import help, docstring File "/home/kangus/src/ArchiveBox/archivebox/main.py", line 8, in <module> from crontab import CronTab, CronSlices File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 655, in _load_unlocked File "<frozen importlib._bootstrap>", line 618, in _load_backward_compatible File "<frozen zipimport>", line 259, in load_module File "/home/kangus/.local/share/virtualenvs/archivebox_django/lib/python3.8/site-packages/python_crontab-2.5.1-py3.8.egg/crontab.py", line 91, in <module> File "/home/kangus/src/ArchiveBox/archivebox/logging.py", line 19, in <module> from .util import enforce_types File "/home/kangus/src/ArchiveBox/archivebox/util.py", line 14, in <module> import requests File "/home/kangus/.local/share/virtualenvs/archivebox_django/lib/python3.8/site-packages/requests/__init__.py", line 43, in <module> import urllib3 File "/home/kangus/.local/share/virtualenvs/archivebox_django/lib/python3.8/site-packages/urllib3/__init__.py", line 7, in <module> from .connectionpool import HTTPConnectionPool, HTTPSConnectionPool, connection_from_url File "/home/kangus/.local/share/virtualenvs/archivebox_django/lib/python3.8/site-packages/urllib3/connectionpool.py", line 29, in <module> from .connection import ( File "/home/kangus/.local/share/virtualenvs/archivebox_django/lib/python3.8/site-packages/urllib3/connection.py", line 40, in <module> from .util.ssl_ import ( File "/home/kangus/.local/share/virtualenvs/archivebox_django/lib/python3.8/site-packages/urllib3/util/__init__.py", line 20, in <module> from .retry import Retry File "/home/kangus/.local/share/virtualenvs/archivebox_django/lib/python3.8/site-packages/urllib3/util/retry.py", line 20, in <module> log = logging.getLogger(__name__) AttributeError: partially initialized module 'logging' has no attribute 'getLogger' (most likely due to a circular import) ``` i used pew for venv: ``` cd /home/kangus/src/ArchiveBox pew new -p /usr/local/bin/python3.8 -a $(pwd) archivebox_django pip install -U setuptools python3 ./setup.,py install ``` ``` python -V Python 3.8.2 python3 -V Python 3.8.2 ``` It looks to my like urllib3 is trying to run getLogger() from ArchiveBox/archivebox/logging.py instead of python lgger module

kerem commented

2026-03-14 21:54:10 +03:00

Author

Owner

@cdvv7788 commented on GitHub (Jul 22, 2020):

@terxw How are you running the commands?
Can you try installing with pip install -e . in the folder, and then running the commands with archivebox add http://example.com?
I am having the same issue you are having when running directly with bin/archivebox (checking that atm).

@cdvv7788 commented on GitHub (Jul 22, 2020): @terxw How are you running the commands? Can you try installing with `pip install -e .` in the folder, and then running the commands with `archivebox add http://example.com`? I am having the same issue you are having when running directly with `bin/archivebox` (checking that atm).

kerem commented

2026-03-14 21:54:15 +03:00

Author

Owner

@cdvv7788 commented on GitHub (Jul 22, 2020):

@terxw We renamed the module to prevent the issue. Once #379 is merged, you should be able to use it as you normally do.

@cdvv7788 commented on GitHub (Jul 22, 2020): @terxw We renamed the module to prevent the issue. Once #379 is merged, you should be able to use it as you normally do.

kerem commented

2026-03-14 21:54:20 +03:00

Author

Owner

@terxw commented on GitHub (Jul 25, 2020):

Tried latest commit, installed in venv with pip3 install -e . and no import problem!

@terxw commented on GitHub (Jul 25, 2020): Tried latest commit, installed in venv with pip3 install -e . and no import problem!

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/ArchiveBox#3278

No description provided.

Rows
Columns