[GH-ISSUE #253] Switch SQLite to WAL mode #204

New issue

Closed

opened 2026-02-25 21:31:25 +03:00 by kerem · 2 comments

kerem commented

2026-02-25 21:31:25 +03:00

Owner

Originally created by @Mirodin on GitHub (Dec 12, 2020).
Original GitHub issue: https://github.com/ciur/papermerge/issues/253

Originally assigned to: @ciur on GitHub.

Is your feature request related to a problem? Please describe.

When I migrated my documents to Papermerge I stumbled upon a bottlenecks in SQLite when simultaneously adding documents and browsing the webinterface: a bunch of "database is locked" exceptions popped up in my logs. As I highly welcom SQLite as first class citizen I dug a little in the database settings and found that it was using the default rollback option, which causes a write operation to lock the database for any read operation whilst performing the write.

Describe the solution you'd like

I propose a switch to SQLite's Write-Ahead Logging as this allows simultaneus writes (i.e. worker indexing document) and reading (browsing the interface). I tried this here for my own system and just executing PRAGMA journal_mode = wal; drastically increased performance of the webinterface and stopped "database is locked" errors from popping up in the log file. As this is not a breaking change, implementation would be quite easy.

Originally created by @Mirodin on GitHub (Dec 12, 2020). Original GitHub issue: https://github.com/ciur/papermerge/issues/253 Originally assigned to: @ciur on GitHub. **Is your feature request related to a problem? Please describe.** When I migrated my documents to Papermerge I stumbled upon a bottlenecks in SQLite when simultaneously adding documents and browsing the webinterface: a bunch of "database is locked" exceptions popped up in my logs. As I highly welcom SQLite as first class citizen I dug a little in the database settings and found that it was using the default rollback option, which causes a write operation to lock the database for any read operation whilst performing the write. **Describe the solution you'd like** I propose a switch to SQLite's [Write-Ahead Logging](https://sqlite.org/wal.html) as this allows simultaneus writes (i.e. worker indexing document) and reading (browsing the interface). I tried this here for my own system and just executing `PRAGMA journal_mode = wal;` drastically increased performance of the webinterface and stopped "database is locked" errors from popping up in the log file. As this is not a breaking change, implementation would be quite easy.

kerem

2026-02-25 21:31:25 +03:00

closed this issue
added the
enhancement

feature request

no-activity
labels

kerem commented

2026-02-25 21:31:26 +03:00

Author

Owner

@ciur commented on GitHub (Dec 14, 2020):

@Mirodin, nice idea, thanks! I will definitely add that config!

@ciur commented on GitHub (Dec 14, 2020): @Mirodin, nice idea, thanks! I will definitely add that config!

kerem commented

2026-02-25 21:31:26 +03:00

Author

Owner

@ciur commented on GitHub (Feb 22, 2021):

@Mirodin,
I tried to enable WAL mode with following code placed in papermerge.core.signals module:

from django.db.backends.signals import connection_created

@receiver(connection_created)
def enable_sqlite_wal_mode(sender, connection, **kwargs):
    if connection.vendor == 'sqlite':
        cursor = connection.cursor()
        cursor.execute('PRAGMA journal_mode=wal;')

Although code executes Ok, i still receive django.db.utils.OperationalError: database is locked when uploading two ore more file in same time (i.e. papermerge browse viewer -> upload -> select 2 or more files -> exception in logs).
From Django documentation:

If you’re getting this error [...for sqlite db...], you can solve it by:

    Switching to another database backend.
   At a certain point SQLite becomes too “lite” for real-world applications,
   and these sorts of concurrency errors indicate you’ve reached that point.

if you know a solution which works, I am happy to go ahead with it.
Related to WAL mode for SQLite I found this opened django ticket.

@ciur commented on GitHub (Feb 22, 2021): @Mirodin, I tried to enable WAL mode with following code placed in papermerge.core.signals module: ``` from django.db.backends.signals import connection_created @receiver(connection_created) def enable_sqlite_wal_mode(sender, connection, **kwargs): if connection.vendor == 'sqlite': cursor = connection.cursor() cursor.execute('PRAGMA journal_mode=wal;') ``` Although code executes Ok, i still receive ```django.db.utils.OperationalError: database is locked``` when uploading two ore more file in same time (i.e. papermerge browse viewer -> upload -> select 2 or more files -> exception in logs). From [Django documentation](https://docs.djangoproject.com/en/3.1/ref/databases/#sqlite-notes): ``` If you’re getting this error [...for sqlite db...], you can solve it by: Switching to another database backend. At a certain point SQLite becomes too “lite” for real-world applications, and these sorts of concurrency errors indicate you’ve reached that point. ``` if you know a solution which works, I am happy to go ahead with it. Related to WAL mode for SQLite I found [this opened django ticket.](https://code.djangoproject.com/ticket/24018)