starred/ArchiveBox

Fork 0

mirror of https://github.com/ArchiveBox/ArchiveBox.git synced 2026-04-25 17:16:00 +03:00

[GH-ISSUE #946] Question: Unable to create the django_migrations table (database is locked) - When OUTPUT_DIR to SAMBA share #3608

New issue

Closed

opened 2026-03-14 23:41:49 +03:00 by kerem · 6 comments

kerem commented

2026-03-14 23:41:49 +03:00

Owner

Originally created by @stephenmetzger on GitHub (Mar 16, 2022).
Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/946

Hello,

Running ArchiveBox in docker (docker compose) am trying to output my archives to a samba share mounted on the archivebox host.

I am configuring the directories, and the docker-compose.yml file, then running the docker-compose run archivebox init --setup command. Once finished, I'm updating the ArchiveBox.conf file to contain OUTPUT_DIR = /mnt/archivebox (which is where i have my mounted SAMBA share directory mounted to the container from the host using a bind mount), and running the docker-compose up -d command.

In the logs, there is a long pause at [+] Building main SQL index and running initial migrations..., then a failure to load. Full log:

    > /mnt/archivebox
[+] Initializing a new ArchiveBox v0.6.2 collection...
----------------------------------------------------------------------
[+] Building archive folder structure...
    + ./archive, ./sources, ./logs...
    + ./ArchiveBox.conf...
[+] Building main SQL index and running initial migrations...
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/base/base.py", line 242, in _commit
    return self.connection.commit()
sqlite3.OperationalError: database is locked
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/django/db/migrations/recorder.py", line 68, in ensure_schema
    editor.create_model(self.Migration)
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/sqlite3/schema.py", line 36, in __exit__
    super().__exit__(exc_type, exc_value, traceback)
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/base/schema.py", line 117, in __exit__
    self.atomic.__exit__(exc_type, exc_value, traceback)
  File "/usr/local/lib/python3.9/site-packages/django/db/transaction.py", line 232, in __exit__
    connection.commit()
  File "/usr/local/lib/python3.9/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/base/base.py", line 266, in commit
    self._commit()
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/base/base.py", line 242, in _commit
    return self.connection.commit()
  File "/usr/local/lib/python3.9/site-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/base/base.py", line 242, in _commit
    return self.connection.commit()
django.db.utils.OperationalError: database is locked
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/bin/archivebox", line 33, in <module>
    sys.exit(load_entry_point('archivebox', 'console_scripts', 'archivebox')())
  File "/app/archivebox/cli/__init__.py", line 140, in main
    run_subcommand(
  File "/app/archivebox/cli/__init__.py", line 80, in run_subcommand
    module.main(args=subcommand_args, stdin=stdin, pwd=pwd)    # type: ignore
  File "/app/archivebox/cli/archivebox_server.py", line 64, in main
    server(
  File "/app/archivebox/util.py", line 114, in typechecked_function
    return func(*args, **kwargs)
  File "/app/archivebox/main.py", line 1243, in server
    run_subcommand('init', subcommand_args=['--quick'], stdin=None, pwd=out_dir)
  File "/app/archivebox/cli/__init__.py", line 80, in run_subcommand
    module.main(args=subcommand_args, stdin=stdin, pwd=pwd)    # type: ignore
  File "/app/archivebox/cli/archivebox_init.py", line 43, in main

(
  File "/app/archivebox/util.py", line 114, in typechecked_function
    return func(*args, **kwargs)
  File "/app/archivebox/main.py", line 328, in init
    for migration_line in apply_migrations(out_dir):
  File "/app/archivebox/util.py", line 114, in typechecked_function
    return func(*args, **kwargs)
  File "/app/archivebox/index/sql.py", line 138, in apply_migrations
    call_command("migrate", interactive=False, stdout=out)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 168, in call_command
    return command.execute(*args, **defaults)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 371, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 85, in wrapped
    res = handle_func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/commands/migrate.py", line 243, in handle
    post_migrate_state = executor.migrate(
  File "/usr/local/lib/python3.9/site-packages/django/db/migrations/executor.py", line 91, in migrate
    self.recorder.ensure_schema()
  File "/usr/local/lib/python3.9/site-packages/django/db/migrations/recorder.py", line 70, in ensure_schema
    raise MigrationSchemaMissing("Unable to create the django_migrations table (%s)" % exc)
django.db.migrations.exceptions.MigrationSchemaMissing: Unable to create the django_migrations table (database is locked)

I have exact setup working succesfully working another machine with ArchiveBox running in docker, but cannot seem to get it to work on this machine (Ubuntu Server 21.10).

Originally created by @stephenmetzger on GitHub (Mar 16, 2022). Original GitHub issue: https://github.com/ArchiveBox/ArchiveBox/issues/946 Hello, Running ArchiveBox in docker (docker compose) am trying to output my archives to a samba share mounted on the archivebox host. I am configuring the directories, and the docker-compose.yml file, then running the `docker-compose run archivebox init --setup` command. Once finished, I'm updating the ArchiveBox.conf file to contain `OUTPUT_DIR = /mnt/archivebox` (which is where i have my mounted SAMBA share directory mounted to the container from the host using a bind mount), and running the `docker-compose up -d` command. In the logs, there is a long pause at `[+] Building main SQL index and running initial migrations...`, then a failure to load. Full log: ```[i] [2022-03-16 18:41:37] ArchiveBox v0.6.2: archivebox server --quick-init 0.0.0.0:8000 > /mnt/archivebox [+] Initializing a new ArchiveBox v0.6.2 collection... ---------------------------------------------------------------------- [+] Building archive folder structure... + ./archive, ./sources, ./logs... + ./ArchiveBox.conf... [+] Building main SQL index and running initial migrations... Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/django/db/backends/base/base.py", line 242, in _commit return self.connection.commit() sqlite3.OperationalError: database is locked The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/django/db/migrations/recorder.py", line 68, in ensure_schema editor.create_model(self.Migration) File "/usr/local/lib/python3.9/site-packages/django/db/backends/sqlite3/schema.py", line 36, in __exit__ super().__exit__(exc_type, exc_value, traceback) File "/usr/local/lib/python3.9/site-packages/django/db/backends/base/schema.py", line 117, in __exit__ self.atomic.__exit__(exc_type, exc_value, traceback) File "/usr/local/lib/python3.9/site-packages/django/db/transaction.py", line 232, in __exit__ connection.commit() File "/usr/local/lib/python3.9/site-packages/django/utils/asyncio.py", line 26, in inner return func(*args, **kwargs) File "/usr/local/lib/python3.9/site-packages/django/db/backends/base/base.py", line 266, in commit self._commit() File "/usr/local/lib/python3.9/site-packages/django/db/backends/base/base.py", line 242, in _commit return self.connection.commit() File "/usr/local/lib/python3.9/site-packages/django/db/utils.py", line 90, in __exit__ raise dj_exc_value.with_traceback(traceback) from exc_value File "/usr/local/lib/python3.9/site-packages/django/db/backends/base/base.py", line 242, in _commit return self.connection.commit() django.db.utils.OperationalError: database is locked During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/bin/archivebox", line 33, in <module> sys.exit(load_entry_point('archivebox', 'console_scripts', 'archivebox')()) File "/app/archivebox/cli/__init__.py", line 140, in main run_subcommand( File "/app/archivebox/cli/__init__.py", line 80, in run_subcommand module.main(args=subcommand_args, stdin=stdin, pwd=pwd) # type: ignore File "/app/archivebox/cli/archivebox_server.py", line 64, in main server( File "/app/archivebox/util.py", line 114, in typechecked_function return func(*args, **kwargs) File "/app/archivebox/main.py", line 1243, in server run_subcommand('init', subcommand_args=['--quick'], stdin=None, pwd=out_dir) File "/app/archivebox/cli/__init__.py", line 80, in run_subcommand module.main(args=subcommand_args, stdin=stdin, pwd=pwd) # type: ignore File "/app/archivebox/cli/archivebox_init.py", line 43, in main ( File "/app/archivebox/util.py", line 114, in typechecked_function return func(*args, **kwargs) File "/app/archivebox/main.py", line 328, in init for migration_line in apply_migrations(out_dir): File "/app/archivebox/util.py", line 114, in typechecked_function return func(*args, **kwargs) File "/app/archivebox/index/sql.py", line 138, in apply_migrations call_command("migrate", interactive=False, stdout=out) File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 168, in call_command return command.execute(*args, **defaults) File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 371, in execute output = self.handle(*args, **options) File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 85, in wrapped res = handle_func(*args, **kwargs) File "/usr/local/lib/python3.9/site-packages/django/core/management/commands/migrate.py", line 243, in handle post_migrate_state = executor.migrate( File "/usr/local/lib/python3.9/site-packages/django/db/migrations/executor.py", line 91, in migrate self.recorder.ensure_schema() File "/usr/local/lib/python3.9/site-packages/django/db/migrations/recorder.py", line 70, in ensure_schema raise MigrationSchemaMissing("Unable to create the django_migrations table (%s)" % exc) django.db.migrations.exceptions.MigrationSchemaMissing: Unable to create the django_migrations table (database is locked) ``` I have exact setup working succesfully working another machine with ArchiveBox running in docker, but cannot seem to get it to work on this machine (Ubuntu Server 21.10).

kerem closed this issue

2026-03-14 23:41:54 +03:00

kerem commented

2026-03-14 23:42:00 +03:00

Author

Owner

@stephenmetzger commented on GitHub (Mar 16, 2022):

Not sure why I was getting the issue above, but ended up finding a workaround (probably the 'right' way to do it).

I started from scratch and ran a new --setup and docker-compose up with a typical configuration as outlined in the docs to get the package to get the directories and files set up, then edited my docker-compose.yml file to manually mount everything where I wanted it to live. My volumes section under archivebox now looks like this:

            - /mnt/BAlink/archivebox/archive:/data/archive
            - ./archivebox/data/ArchiveBox.conf:/data/ArchiveBox.conf
            - ./archivebox/data/index.sqlite3:/data/index.sqlite3
            - ./archivebox/data/logs:/data/logs
            - ./archivebox/data/sources:/data/sources

Any issues doing it this way?

@stephenmetzger commented on GitHub (Mar 16, 2022): Not sure why I was getting the issue above, but ended up finding a workaround (probably the 'right' way to do it). I started from scratch and ran a new `--setup` and `docker-compose up` with a typical configuration as outlined in the docs to get the package to get the directories and files set up, then edited my docker-compose.yml file to manually mount everything where I wanted it to live. My volumes section under archivebox now looks like this: ``` volumes: - /mnt/BAlink/archivebox/archive:/data/archive - ./archivebox/data/ArchiveBox.conf:/data/ArchiveBox.conf - ./archivebox/data/index.sqlite3:/data/index.sqlite3 - ./archivebox/data/logs:/data/logs - ./archivebox/data/sources:/data/sources ``` Any issues doing it this way?

kerem commented

2026-03-14 23:42:05 +03:00

Author

Owner

@pirate commented on GitHub (Mar 16, 2022):

Nope, that's a valid way to do it! But I recommend this instead so you don't have to hardcode those file paths:

            - ./archivebox/data:/data
            - /mnt/BAlink/archivebox/archive:/data/archive

@pirate commented on GitHub (Mar 16, 2022): Nope, that's a valid way to do it! But I recommend this instead so you don't have to hardcode those file paths: ```yaml - ./archivebox/data:/data - /mnt/BAlink/archivebox/archive:/data/archive ```

kerem commented

2026-03-14 23:42:10 +03:00

Author

Owner

@stephenmetzger commented on GitHub (Mar 16, 2022):

Fantastic. Thanks. So, for docker generally, you can specify a subdirectory of a directory to be linked to a different path than the rest of the base-directory? Does the order they are listed matter?

@stephenmetzger commented on GitHub (Mar 16, 2022): Fantastic. Thanks. So, for docker generally, you can specify a subdirectory of a directory to be linked to a different path than the rest of the base-directory? Does the order they are listed matter?

kerem commented

2026-03-14 23:42:15 +03:00

Author

Owner

@pirate commented on GitHub (Mar 16, 2022):

Yes, and the order matters. Sub-path overrides for paths within a parent must be listed after the parent.

@pirate commented on GitHub (Mar 16, 2022): Yes, and the order matters. Sub-path overrides for paths within a parent must be listed after the parent.

kerem commented

2026-03-14 23:42:21 +03:00

Author

Owner

@pirate commented on GitHub (Apr 12, 2022):

Note I've added a new DB/filesystem troubleshooting area to the wiki that may help people arriving here from Google: https://github.com/ArchiveBox/ArchiveBox/wiki/Upgrading-or-Merging-Archives#database-troubleshooting

Contributions/suggestions welcome there.

@pirate commented on GitHub (Apr 12, 2022): Note I've added a new DB/filesystem troubleshooting area to the wiki that may help people arriving here from Google: https://github.com/ArchiveBox/ArchiveBox/wiki/Upgrading-or-Merging-Archives#database-troubleshooting Contributions/suggestions welcome there.

kerem commented

2026-03-14 23:42:26 +03:00

Author

Owner

@p0n1 commented on GitHub (Oct 26, 2023):

Nope, that's a valid way to do it! But I recommend this instead so you don't have to hardcode those file paths:
            - ./archivebox/data:/data
            - /mnt/BAlink/archivebox/archive:/data/archive

This is working. When the index.sqlite3 created successfully in local, I just moved all files into /mnt/smb/archivebox and redeployed the container. The ArchiveBox could operate normally. No more database is locked issues. Not sure why.

BTW, I also tried NFS mounting and didn't encounter database locked issue.

@p0n1 commented on GitHub (Oct 26, 2023): > Nope, that's a valid way to do it! But I recommend this instead so you don't have to hardcode those file paths: > > ```yaml > - ./archivebox/data:/data > - /mnt/BAlink/archivebox/archive:/data/archive > ``` This is working. When the `index.sqlite3` created successfully in local, I just moved all files into `/mnt/smb/archivebox` and redeployed the container. The ArchiveBox could operate normally. No more `database is locked` issues. Not sure why. BTW, I also tried NFS mounting and didn't encounter database locked issue.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/ArchiveBox#3608

No description provided.

Rows
Columns