starred/ArchiveBox
1
0
Fork 0
mirror of https://github.com/ArchiveBox/ArchiveBox.git synced 2026-04-25 17:16:00 +03:00
Code Issues 624 Projects Releases 5 Packages Wiki Activity

[GH-ISSUE #409] Bug: 500 Error after finishing archive request. #3291

New issue
Closed
opened 2026-03-14 21:57:58 +03:00 by kerem · 5 comments
kerem commented 2026-03-14 21:57:58 +03:00
Owner
Copy link

Originally created by @PokeyTheCat on GitHub (Jul 31, 2020).
Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/409

Using docker and ArchiveBox v0.4.9 web server interface. I'm getting a Server Error (500) when the post completes after I submit a new archive request in the web.

This is a fresh install and newly initialized instance. Can I increase the verbosity to see what's making it mad?

REPOSITORY                TAG                 IMAGE ID            CREATED             SIZE
nikisweeting/archivebox   latest              3e02bd28e479        5 hours ago         1.07GB

Attaching to archivebox_1
archivebox_1  | [i] [2020-07-30 22:44:34] ArchiveBox v0.4.9: archivebox server 0.0.0.0:8000
archivebox_1  |     > /data
archivebox_1  |
archivebox_1  | [+] Starting ArchiveBox webserver...
archivebox_1  | [i] The admin username is: archivebox
archivebox_1  | Performing system checks...
archivebox_1  |
archivebox_1  | System check identified no issues (0 silenced).
archivebox_1  | July 30, 2020 - 22:44:35
archivebox_1  | Django version 3.0.8, using settings 'core.settings'
archivebox_1  | Starting development server at http://0.0.0.0:8000/
archivebox_1  | Quit the server with CONTROL-C.
archivebox_1  | [30/Jul/2020 22:45:01] "POST /admin/login/?next=/admin/core/snapshot/add/ HTTP/1.1" 302 0
archivebox_1  | [30/Jul/2020 22:45:01] "GET /admin/core/snapshot/add/ HTTP/1.1" 200 8756
archivebox_1  | [30/Jul/2020 22:45:01] "GET /static/admin/css/dashboard.css HTTP/1.1" 200 412
archivebox_1  | [+] Adding URL: https://medium.com/@nielssj/docker-volumes-and-file-system-permissions-772c1aee23ca
archivebox_1  | [30/Jul/2020 22:45:19] "GET /static/admin/fonts/Roboto-Bold-webfont.woff HTTP/1.1" 200 86184
archivebox_1  | [+] [2020-07-30 22:45:19] Adding 1 links to index (crawl depth=0)...
archivebox_1  |     > Saved verbatim input to sources/1596149119-import.txt
archivebox_1  |     > Parsed 1 URLs from input (Plain Text)
archivebox_1  |     > Found 1 new URLs not already in index
archivebox_1  |
archivebox_1  | [*] [2020-07-30 22:45:19] Writing 1 links to main index...
    √ /data/index.sqlite3ta/index.sqlite3
    √ /data/index.json/data/index.json
    √ /data/index.html/data/index.html
archivebox_1  |
archivebox_1  | [▶] [2020-07-30 22:45:19] Collecting content for 1 Snapshots in archive...
archivebox_1  |
archivebox_1  | [+] [2020-07-30 22:45:19] "medium.com/@nielssj/docker-volumes-and-file-system-permissions-772c1aee23ca"
archivebox_1  |     https://medium.com/@nielssj/docker-volumes-and-file-system-permissions-772c1aee23ca
archivebox_1  |     > ./archive/1596149119
archivebox_1  |       > title
archivebox_1  |       > favicon
archivebox_1  |       > wget
archivebox_1  |       > pdf
archivebox_1  |       > screenshot
archivebox_1  |       > dom
archivebox_1  |       > media
archivebox_1  |       > archive_org
archivebox_1  |         Failed:
archivebox_1  |             TimeoutExpired Command '['curl', '--silent', '--location', '--head', '--compressed', '--max-time', '60', '--user-agent', 'ArchiveBox/0.4.9 (+https://github.com/pirate/ArchiveBox/) curl/curl 7.64.0 (x86_64-pc-linux-gnu)', 'https://web.archive.org/save/https://medium.com/@nielssj/docker-volumes-and-file-system-permissions-772c1aee23ca']' timed out after 60 seconds
archivebox_1  |         Run to see full output:
archivebox_1  |             cd /data/archive/1596149119;
archivebox_1  |             curl --silent --location --head --compressed --max-time 60 --user-agent "ArchiveBox/0.4.9 (+https://github.com/pirate/ArchiveBox/) curl/curl 7.64.0 (x86_64-pc-linux-gnu)" https://web.archive.org/save/https://medium.com/@nielssj/docker-volumes-and-file-system-permissions-772c1aee23ca
archivebox_1  |
archivebox_1  |
archivebox_1  | [√] [2020-07-30 22:46:48] Update of 1 pages complete (1.48 min)
archivebox_1  |     - 0 links skipped
archivebox_1  |     - 0 links updated
archivebox_1  |     - 1 links had errors
archivebox_1  |
archivebox_1  |     Hint: To view your archive index, open:
archivebox_1  |         /data/index.html
archivebox_1  |     Or run the built-in webserver:
archivebox_1  |         archivebox server
archivebox_1  |
archivebox_1  | [*] [2020-07-30 22:46:48] Writing 1 links to main index...
    √ /data/index.sqlite3ta/index.sqlite3
    √ /data/index.json/data/index.json
    √ /data/index.html/data/index.html
archivebox_1  |
archivebox_1  | [30/Jul/2020 22:46:48] "POST /admin/core/snapshot/add/? HTTP/1.1" 500 145
Originally created by @PokeyTheCat on GitHub (Jul 31, 2020). Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/409 Using docker and `ArchiveBox v0.4.9` web server interface. I'm getting a `Server Error (500)` when the post completes after I submit a new archive request in the web. This is a fresh install and newly initialized instance. Can I increase the verbosity to see what's making it mad? ``` REPOSITORY TAG IMAGE ID CREATED SIZE nikisweeting/archivebox latest 3e02bd28e479 5 hours ago 1.07GB ``` ``` Attaching to archivebox_1 archivebox_1 | [i] [2020-07-30 22:44:34] ArchiveBox v0.4.9: archivebox server 0.0.0.0:8000 archivebox_1 | > /data archivebox_1 | archivebox_1 | [+] Starting ArchiveBox webserver... archivebox_1 | [i] The admin username is: archivebox archivebox_1 | Performing system checks... archivebox_1 | archivebox_1 | System check identified no issues (0 silenced). archivebox_1 | July 30, 2020 - 22:44:35 archivebox_1 | Django version 3.0.8, using settings 'core.settings' archivebox_1 | Starting development server at http://0.0.0.0:8000/ archivebox_1 | Quit the server with CONTROL-C. archivebox_1 | [30/Jul/2020 22:45:01] "POST /admin/login/?next=/admin/core/snapshot/add/ HTTP/1.1" 302 0 archivebox_1 | [30/Jul/2020 22:45:01] "GET /admin/core/snapshot/add/ HTTP/1.1" 200 8756 archivebox_1 | [30/Jul/2020 22:45:01] "GET /static/admin/css/dashboard.css HTTP/1.1" 200 412 archivebox_1 | [+] Adding URL: https://medium.com/@nielssj/docker-volumes-and-file-system-permissions-772c1aee23ca archivebox_1 | [30/Jul/2020 22:45:19] "GET /static/admin/fonts/Roboto-Bold-webfont.woff HTTP/1.1" 200 86184 archivebox_1 | [+] [2020-07-30 22:45:19] Adding 1 links to index (crawl depth=0)... archivebox_1 | > Saved verbatim input to sources/1596149119-import.txt archivebox_1 | > Parsed 1 URLs from input (Plain Text) archivebox_1 | > Found 1 new URLs not already in index archivebox_1 | archivebox_1 | [*] [2020-07-30 22:45:19] Writing 1 links to main index... √ /data/index.sqlite3ta/index.sqlite3 √ /data/index.json/data/index.json √ /data/index.html/data/index.html archivebox_1 | archivebox_1 | [▶] [2020-07-30 22:45:19] Collecting content for 1 Snapshots in archive... archivebox_1 | archivebox_1 | [+] [2020-07-30 22:45:19] "medium.com/@nielssj/docker-volumes-and-file-system-permissions-772c1aee23ca" archivebox_1 | https://medium.com/@nielssj/docker-volumes-and-file-system-permissions-772c1aee23ca archivebox_1 | > ./archive/1596149119 archivebox_1 | > title archivebox_1 | > favicon archivebox_1 | > wget archivebox_1 | > pdf archivebox_1 | > screenshot archivebox_1 | > dom archivebox_1 | > media archivebox_1 | > archive_org archivebox_1 | Failed: archivebox_1 | TimeoutExpired Command '['curl', '--silent', '--location', '--head', '--compressed', '--max-time', '60', '--user-agent', 'ArchiveBox/0.4.9 (+https://github.com/pirate/ArchiveBox/) curl/curl 7.64.0 (x86_64-pc-linux-gnu)', 'https://web.archive.org/save/https://medium.com/@nielssj/docker-volumes-and-file-system-permissions-772c1aee23ca']' timed out after 60 seconds archivebox_1 | Run to see full output: archivebox_1 | cd /data/archive/1596149119; archivebox_1 | curl --silent --location --head --compressed --max-time 60 --user-agent "ArchiveBox/0.4.9 (+https://github.com/pirate/ArchiveBox/) curl/curl 7.64.0 (x86_64-pc-linux-gnu)" https://web.archive.org/save/https://medium.com/@nielssj/docker-volumes-and-file-system-permissions-772c1aee23ca archivebox_1 | archivebox_1 | archivebox_1 | [√] [2020-07-30 22:46:48] Update of 1 pages complete (1.48 min) archivebox_1 | - 0 links skipped archivebox_1 | - 0 links updated archivebox_1 | - 1 links had errors archivebox_1 | archivebox_1 | Hint: To view your archive index, open: archivebox_1 | /data/index.html archivebox_1 | Or run the built-in webserver: archivebox_1 | archivebox server archivebox_1 | archivebox_1 | [*] [2020-07-30 22:46:48] Writing 1 links to main index... √ /data/index.sqlite3ta/index.sqlite3 √ /data/index.json/data/index.json √ /data/index.html/data/index.html archivebox_1 | archivebox_1 | [30/Jul/2020 22:46:48] "POST /admin/core/snapshot/add/? HTTP/1.1" 500 145 ```
kerem closed this issue 2026-03-14 21:58:03 +03:00
kerem commented 2026-03-14 21:58:09 +03:00
Author
Owner
Copy link

@PokeyTheCat commented on GitHub (Jul 31, 2020):

Ok I found the --debug flag for the server command. This is the cause of my issue.

Environment:

Request Method: POST
Request URL: http://localhost:8000/admin/core/snapshot/add/

Django Version: 3.0.8
Python Version: 3.8.5
Installed Applications:
['django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.admin',
 'core',
 'django_extensions']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware']

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/app/archivebox/core/admin.py", line 188, in add_view
    "stdout": ansi_to_html(add_stdout.getvalue().strip()),
  File "/app/archivebox/util.py", line 220, in ansi_to_html
    from .config import COLOR_DICT

Exception Type: ImportError at /admin/core/snapshot/add/
Exception Value: attempted relative import with no known parent package

Request Method: | POST
-- | --
http://localhost:8000/admin/core/snapshot/add/
3.0.8
ImportError
attempted relative import with no known parent package
/app/archivebox/util.py in ansi_to_html, line 220
/usr/local/bin/python
3.8.5
['/usr/local/bin',  '/usr/local/lib/python38.zip',  '/usr/local/lib/python3.8',  '/usr/local/lib/python3.8/lib-dynload',  '/usr/local/lib/python3.8/site-packages',  '/app',  '/app/archivebox',  '/app/archivebox',  '/app/archivebox',  '/app/archivebox',  '/app/archivebox',  '/app/archivebox',  '/app/archivebox',  '/app/archivebox',  '/app/archivebox',  '/app/archivebox']
Fri, 31 Jul 2020 03:49:54 +0000
<!-- gh-comment-id:666902049 --> @PokeyTheCat commented on GitHub (Jul 31, 2020): Ok I found the `--debug` flag for the server command. This is the cause of my issue. ``` Environment: Request Method: POST Request URL: http://localhost:8000/admin/core/snapshot/add/ Django Version: 3.0.8 Python Version: 3.8.5 Installed Applications: ['django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'django.contrib.admin', 'core', 'django_extensions'] Installed Middleware: ['django.middleware.security.SecurityMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware'] Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner response = get_response(request) File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 115, in _get_response response = self.process_exception_by_middleware(e, request) File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/app/archivebox/core/admin.py", line 188, in add_view "stdout": ansi_to_html(add_stdout.getvalue().strip()), File "/app/archivebox/util.py", line 220, in ansi_to_html from .config import COLOR_DICT Exception Type: ImportError at /admin/core/snapshot/add/ Exception Value: attempted relative import with no known parent package ``` ``` Request Method: | POST -- | -- http://localhost:8000/admin/core/snapshot/add/ 3.0.8 ImportError attempted relative import with no known parent package /app/archivebox/util.py in ansi_to_html, line 220 /usr/local/bin/python 3.8.5 ['/usr/local/bin', '/usr/local/lib/python38.zip', '/usr/local/lib/python3.8', '/usr/local/lib/python3.8/lib-dynload', '/usr/local/lib/python3.8/site-packages', '/app', '/app/archivebox', '/app/archivebox', '/app/archivebox', '/app/archivebox', '/app/archivebox', '/app/archivebox', '/app/archivebox', '/app/archivebox', '/app/archivebox', '/app/archivebox'] Fri, 31 Jul 2020 03:49:54 +0000 ```
kerem commented 2026-03-14 21:58:14 +03:00
Author
Owner
Copy link

@pirate commented on GitHub (Aug 1, 2020):

This should be fixed by #409. If you still encounter any problems with the latest version comment back here and I'll reopen the ticket.

git checkout master
git pull
docker build . -t archivebox
...
docker run -v /archive:/data archivebox server 0.0.0.0:8000 --debug
<!-- gh-comment-id:667552941 --> @pirate commented on GitHub (Aug 1, 2020): This should be fixed by #409. If you still encounter any problems with the latest version comment back here and I'll reopen the ticket. ```bash git checkout master git pull docker build . -t archivebox ... docker run -v /archive:/data archivebox server 0.0.0.0:8000 --debug ```
kerem commented 2026-03-14 21:58:19 +03:00
Author
Owner
Copy link

@mobedoor commented on GitHub (Sep 26, 2020):

@pirate I got the following error when I ran docker build . -t archivebox. Could I get some help.

Step 22/29 : RUN pip install -e .
 ---> Running in da38b7bcd4b0
Obtaining file:///app
    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/app/setup.py'"'"'; __file__='"'"'/app/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-saghmanw
         cwd: /app/
    Complete output (11 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/app/setup.py", line 16, in <module>
        README = (PYTHON_DIR / "README.md").read_text()
      File "/usr/local/lib/python3.8/pathlib.py", line 1232, in read_text
        with self.open(mode='r', encoding=encoding, errors=errors) as f:
      File "/usr/local/lib/python3.8/pathlib.py", line 1218, in open
        return io.open(self, mode, buffering, encoding, errors, newline,
      File "/usr/local/lib/python3.8/pathlib.py", line 1074, in _opener
        return self._accessor.open(self, flags, mode)
    FileNotFoundError: [Errno 2] No such file or directory: '/app/archivebox/README.md'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
<!-- gh-comment-id:699483135 --> @mobedoor commented on GitHub (Sep 26, 2020): @pirate I got the following error when I ran docker build . -t archivebox. Could I get some help. ```logs Step 22/29 : RUN pip install -e . ---> Running in da38b7bcd4b0 Obtaining file:///app ERROR: Command errored out with exit status 1: command: /usr/local/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/app/setup.py'"'"'; __file__='"'"'/app/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-saghmanw cwd: /app/ Complete output (11 lines): Traceback (most recent call last): File "<string>", line 1, in <module> File "/app/setup.py", line 16, in <module> README = (PYTHON_DIR / "README.md").read_text() File "/usr/local/lib/python3.8/pathlib.py", line 1232, in read_text with self.open(mode='r', encoding=encoding, errors=errors) as f: File "/usr/local/lib/python3.8/pathlib.py", line 1218, in open return io.open(self, mode, buffering, encoding, errors, newline, File "/usr/local/lib/python3.8/pathlib.py", line 1074, in _opener return self._accessor.open(self, flags, mode) FileNotFoundError: [Errno 2] No such file or directory: '/app/archivebox/README.md' ---------------------------------------- ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output. ```
kerem commented 2026-03-14 21:58:24 +03:00
Author
Owner
Copy link

@pirate commented on GitHub (Oct 2, 2020):

@mobedoor Possibly something up with your build/git environment, can you try deleting the repo dir and re-cloning it?

Alternatively try pulling the v0.4.21 tag instead of the master branch and building that. https://github.com/pirate/ArchiveBox/releases/tag/v0.4.21

<!-- gh-comment-id:702796211 --> @pirate commented on GitHub (Oct 2, 2020): @mobedoor Possibly something up with your build/git environment, can you try deleting the repo dir and re-cloning it? Alternatively try pulling the v0.4.21 tag instead of the master branch and building that. https://github.com/pirate/ArchiveBox/releases/tag/v0.4.21
kerem commented 2026-03-14 21:58:29 +03:00
Author
Owner
Copy link

@mobedoor commented on GitHub (Oct 3, 2020):

@pirate Thanks. That fixed it.

<!-- gh-comment-id:703038447 --> @mobedoor commented on GitHub (Oct 3, 2020): @pirate Thanks. That fixed it.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
dev
main
abx-dl-refactor
claude/issue-1688-20251229-2233
claude/s3-support-migration-JZQcf
claude/review-snapshot-archive-RJOkr
claude/review-code-quality-macdO
claude/reduce-chrome-duplication-JgWVr
fix/chrome-args-comma-parsing
claude/issue-668-20251229-2145
claude/issue-1664-20251229-2236
v086
claude/improve-test-suite-xm6Bh
claude/typescript-archivebox-rewrite-011CUmPyEmECnkXwyHRT9RPF
claude/archivebox-schema-orm-comparison-011CUn26GurEpLiBUq4zjVLJ
stable
logfire
plugins-browsertrix
v072
link-removal2
v0.8.6rc1
v0.7.3
v0.8.6rc0
v0.8.5rc53
v0.8.5rc51
v0.8.5rc50
v0.8.5rc49
v0.8.5rc48
v0.8.5rc47
v0.8.5rc46
v0.8.5rc45
v0.8.5rc44
v0.8.5rc43
v0.8.5rc42
v0.8.5rc41
v0.8.5rc40
v0.8.5rc39
v0.8.5rc38
v0.8.5rc37
v0.8.5rc36
v0.8.5rc35
v0.8.5rc34
v0.8.5rc33
v0.8.5rc32
v0.8.5rc31
v0.8.5rc30
v0.8.5rc29
v0.8.5rc28
v0.8.5rc27
v0.8.5rc26
v0.8.5rc25
v0.8.5rc24
v0.8.5rc23
v0.8.5rc22
v0.8.5rc13
v0.8.5rc12
v0.8.5rc11
v0.8.5rc10
v0.8.5rc9
v0.8.5rc8
v0.8.5rc6
v0.8.5rc5
v0.8.5rc4
v0.8.5rc3
v0.8.5rc2
v0.8.5-rc
v0.8.4-rc
v0.8.3-rc
v0.8.2-rc
v0.8.0-rc
v0.7.2
v0.7.1
v0.7.0
v0.6.2
v0.6.0
v0.5.6
v0.5.4
v0.5.3
v0.4.24
v0.4.21
v0.4.20
v0.4.19
v0.4.18
v0.4.17
v0.4.16
v0.4.15
v0.4.14
v0.4.13
v0.4.12
v0.4.11
v0.4.9
v0.2.4
v0.2.3
v0.2.2
v0.2.1
v0.2.0
v0.1.0
v0.0.3
v0.0.2
v0.0.1
Labels
Clear labels   
expected: maybe someday

   
expected: next release

   
expected: release after next

   
expected: unlikely unless contributed

   
good first ticket

   
help wanted

   
pull-request

Mirrored from GitHub Pull Request

  
scope: all users

   
scope: windows users

   
size: easy

   
size: hard

   
size: medium

   
size: medium

   
status: backlog

   
status: blocked

   
status: done

   
status: idea-phase

   
status: needs followup

   
status: wip

   
status: wontfix

   
touches: API/CLI/Spec

   
touches: configuration

   
touches: data/schema/architecture

   
touches: dependencies/packaging

   
touches: docs

   
touches: js

   
touches: views/replayers/html/css

   
why: correctness

   
why: functionality

   
why: performance

   
why: security
No labels
expected: maybe someday
expected: next release
expected: release after next
expected: unlikely unless contributed
good first ticket
help wanted
pull-request
scope: all users
scope: windows users
size: easy
size: hard
size: medium
size: medium
status: backlog
status: blocked
status: done
status: idea-phase
status: needs followup
status: wip
status: wontfix
touches: API/CLI/Spec
touches: configuration
touches: data/schema/architecture
touches: dependencies/packaging
touches: docs
touches: js
touches: views/replayers/html/css
why: correctness
why: functionality
why: performance
why: security
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/ArchiveBox#3291
No description provided.