starred/karakeep

Fork 0

mirror of https://github.com/karakeep-app/karakeep.git synced 2026-04-25 07:56:05 +03:00

[GH-ISSUE #1690] Slow works of web interface #1054

New issue

Open

opened 2026-03-02 11:54:41 +03:00 by kerem · 6 comments

kerem commented

2026-03-02 11:54:41 +03:00

Owner

Originally created by @igortau on GitHub (Jun 30, 2025).
Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/1690

Describe the Bug

the web interface took a very long time to open after the update. that is, you have to wait probably half a minute or longer for the data to appear.

Steps to Reproduce

Open web browser and write karakeep container address

Expected Behaviour

the web interface took a very long time to open after the update. that is, you have to wait probably half a minute or longer for the data to appear.

Screenshots or Additional Context

Massage from watchtower
Watchtower updates on ea4afa62ac9a
Found new ghcr.io/karakeep-app/karakeep:latest image (c5fcb1a653c6)
Stopping /Karakeep-WEB (c0927c25acb1) with SIGTERM
Creating /Karakeep-WEB
Removing image ed31f8f95786

Log from karakeep web
✓ Starting...
✓ Ready in 4.1s
2025-06-30T09:34:10.921Z info: Workers version: nightly
2025-06-30T09:34:11.393Z info: [crawler] Loading adblocker ...
2025-06-30T09:34:11.507Z info: [Crawler] Connecting to existing browser instance: http://chrome:9222
2025-06-30T09:34:11.575Z info: [Crawler] Successfully resolved IP address, new address: http://172.29.12.5:9222/
2025-06-30T09:34:14.067Z info: Starting crawler worker ...
2025-06-30T09:34:14.068Z info: Starting inference worker ...
2025-06-30T09:34:14.069Z info: Starting search indexing worker ...
2025-06-30T09:34:14.070Z info: Starting tidy assets worker ...
2025-06-30T09:34:14.070Z info: Starting video worker ...
2025-06-30T09:34:14.071Z info: Starting feed worker ...
2025-06-30T09:34:14.072Z info: Starting asset preprocessing worker ...
2025-06-30T09:34:14.073Z info: Starting webhook worker ...
2025-06-30T09:34:14.073Z info: Starting rule engine worker ...
[next-auth][warn][NEXTAUTH_URL]
https://next-auth.js.org/warnings#nextauth_url
<-- GET /api/assets/38e309e0-2e2f-4149-bf6d-b8c446e6502e
--> GET /api/assets/38e309e0-2e2f-4149-bf6d-b8c446e6502e 200 35ms
<-- GET /api/assets/f95f5743-6e4a-4268-96b2-ebc060fcf6de
<-- GET /api/assets/760b39a6-971b-4082-8c80-aa343034c65b
<-- GET /api/assets/513dd28a-b0da-4977-93a0-cb40dc763504
<-- GET /api/assets/13b1d727-b539-4040-8726-d92292b202dd
<-- GET /api/assets/ff0f6fb5-9f0c-490f-b09c-9b28101694ba
--> GET /api/assets/f95f5743-6e4a-4268-96b2-ebc060fcf6de 200 36s
--> GET /api/assets/760b39a6-971b-4082-8c80-aa343034c65b 200 36s
--> GET /api/assets/513dd28a-b0da-4977-93a0-cb40dc763504 200 36s
--> GET /api/assets/13b1d727-b539-4040-8726-d92292b202dd 200 37s
--> GET /api/assets/ff0f6fb5-9f0c-490f-b09c-9b28101694ba 200 291ms
<-- GET /api/assets/d6a1daa0-fb06-47f4-8bb3-bc47bffb8bc9
--> GET /api/assets/d6a1daa0-fb06-47f4-8bb3-bc47bffb8bc9 200 221ms
<-- GET /api/assets/5db25bc1-68ca-4d29-8278-b600ff3cb5ed
--> GET /api/assets/5db25bc1-68ca-4d29-8278-b600ff3cb5ed 200 31ms
<-- GET /api/assets/e0f1111e-1cbf-4106-b0cd-dc63e23a97e1
<-- GET /api/assets/1e2989e6-86bb-400a-b168-153b1ed11a46
--> GET /api/assets/e0f1111e-1cbf-4106-b0cd-dc63e23a97e1 200 588ms
<-- GET /api/assets/831878e8-1c52-47c4-9de2-5936ade75293
--> GET /api/assets/1e2989e6-86bb-400a-b168-153b1ed11a46 200 603ms
<-- GET /api/assets/e442a966-1589-4e9e-942b-414ab101f865
<-- GET /api/assets/c07c3570-0523-4da2-bc11-9cf27eb92fd3
--> GET /api/assets/831878e8-1c52-47c4-9de2-5936ade75293 200 45ms
<-- GET /api/assets/a6082c41-bd2d-4e76-90d7-f7e718c10bb4
--> GET /api/assets/e442a966-1589-4e9e-942b-414ab101f865 200 44ms
<-- GET /api/assets/8fc3e61e-964f-444e-8f1b-47bd5ffa4ad0
--> GET /api/assets/c07c3570-0523-4da2-bc11-9cf27eb92fd3 200 53ms
--> GET /api/assets/a6082c41-bd2d-4e76-90d7-f7e718c10bb4 200 35ms
--> GET /api/assets/8fc3e61e-964f-444e-8f1b-47bd5ffa4ad0 200 114ms
<-- GET /api/assets/5a864fb4-6cfe-450d-b775-472130f00060
<-- GET /api/assets/2fb50598-556f-4219-8755-495f808a5984
<-- GET /api/assets/dbbcce8c-3624-417a-9062-79f58d62c713
<-- GET /api/assets/301cd8b5-3044-4955-8fc2-172d5ba63f26
--> GET /api/assets/5a864fb4-6cfe-450d-b775-472130f00060 200 461ms
--> GET /api/assets/2fb50598-556f-4219-8755-495f808a5984 200 470ms
--> GET /api/assets/dbbcce8c-3624-417a-9062-79f58d62c713 200 474ms
--> GET /api/assets/301cd8b5-3044-4955-8fc2-172d5ba63f26 200 496ms
<-- GET /api/assets/5cb6b8c3-106e-4bf6-9941-6f2f1db07d83
<-- GET /api/assets/842d7db9-9a47-4374-b7a8-f20a5a362b9a
<-- GET /api/assets/b675ffe4-bb46-4d1d-a062-1f87f871c01a
<-- GET /api/assets/28110704-0a4d-4d96-80bf-2bb624f5d3cf
--> GET /api/assets/842d7db9-9a47-4374-b7a8-f20a5a362b9a 200 68ms
--> GET /api/assets/b675ffe4-bb46-4d1d-a062-1f87f871c01a 200 49ms
--> GET /api/assets/28110704-0a4d-4d96-80bf-2bb624f5d3cf 200 46ms
--> GET /api/assets/5cb6b8c3-106e-4bf6-9941-6f2f1db07d83 200 86ms
2025-06-30T10:00:00.391Z info: [feed] Scheduling feed refreshing jobs ...
<-- GET /api/assets/38e309e0-2e2f-4149-bf6d-b8c446e6502e
--> GET /api/assets/38e309e0-2e2f-4149-bf6d-b8c446e6502e 200 226ms
<-- GET /api/assets/5cb6b8c3-106e-4bf6-9941-6f2f1db07d83
--> GET /api/assets/5cb6b8c3-106e-4bf6-9941-6f2f1db07d83 200 336ms
<-- GET /api/assets/842d7db9-9a47-4374-b7a8-f20a5a362b9a
--> GET /api/assets/842d7db9-9a47-4374-b7a8-f20a5a362b9a 200 23ms
<-- GET /api/assets/28110704-0a4d-4d96-80bf-2bb624f5d3cf
--> GET /api/assets/28110704-0a4d-4d96-80bf-2bb624f5d3cf 200 21ms
<-- GET /api/assets/b675ffe4-bb46-4d1d-a062-1f87f871c01a
--> GET /api/assets/b675ffe4-bb46-4d1d-a062-1f87f871c01a 200 20ms
<-- GET /api/assets/174c29e1-8620-48ad-9be5-114f703d5fb6
--> GET /api/assets/174c29e1-8620-48ad-9be5-114f703d5fb6 200 10ms
<-- GET /api/assets/86a7829e-0c91-4a9a-9e7b-97c91ba74dd9
--> GET /api/assets/86a7829e-0c91-4a9a-9e7b-97c91ba74dd9 200 116ms
<-- GET /api/assets/edb1696a-25cf-40f3-a260-69c604a89106
--> GET /api/assets/edb1696a-25cf-40f3-a260-69c604a89106 200 18ms
<-- GET /api/assets/d85e95e9-62fe-4373-9c9d-2d3dafc9368e
--> GET /api/assets/d85e95e9-62fe-4373-9c9d-2d3dafc9368e 200 39ms
<-- GET /api/assets/658146cc-867b-438c-bc73-7a0181dc499c
--> GET /api/assets/658146cc-867b-438c-bc73-7a0181dc499c 200 11ms
<-- GET /api/assets/329f4796-bee2-4ded-aaa8-e46f2cd6d19f
--> GET /api/assets/329f4796-bee2-4ded-aaa8-e46f2cd6d19f 200 70ms
<-- GET /api/assets/da56ebe3-029d-495f-bd0d-2fb5e8861234
--> GET /api/assets/da56ebe3-029d-495f-bd0d-2fb5e8861234 200 253ms
<-- GET /api/assets/3a2e07b9-8693-4c02-bc60-d7c32d064e7a
--> GET /api/assets/3a2e07b9-8693-4c02-bc60-d7c32d064e7a 200 38ms
<-- GET /api/assets/88155a9d-1fd4-4c7b-b36f-9f7bcd3943fb
--> GET /api/assets/88155a9d-1fd4-4c7b-b36f-9f7bcd3943fb 200 41ms
<-- GET /api/assets/3f193c01-822e-45e8-9f74-df75bdd38c73
<-- GET /api/assets/5691561a-c452-42e8-bdfe-7c89c9a06087
<-- GET /api/assets/90138225-28f8-40b8-9d76-7cda4f767d3c
<-- GET /api/assets/31268154-90e5-4944-ab4b-d94f4a936f24
--> GET /api/assets/3f193c01-822e-45e8-9f74-df75bdd38c73 200 619ms
--> GET /api/assets/5691561a-c452-42e8-bdfe-7c89c9a06087 200 68s
--> GET /api/assets/90138225-28f8-40b8-9d76-7cda4f767d3c 200 68s

Device Details

Chrome, Firefox, android app

Exact Karakeep Version

Karakeep vnightly 0.25

Have you checked the troubleshooting guide?

I have checked the troubleshooting guide and I haven't found a solution to my problem

Originally created by @igortau on GitHub (Jun 30, 2025). Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/1690 ### Describe the Bug the web interface took a very long time to open after the update. that is, you have to wait probably half a minute or longer for the data to appear. ### Steps to Reproduce Open web browser and write karakeep container address ### Expected Behaviour the web interface took a very long time to open after the update. that is, you have to wait probably half a minute or longer for the data to appear. ### Screenshots or Additional Context Massage from watchtower Watchtower updates on ea4afa62ac9a Found new ghcr.io/karakeep-app/karakeep:latest image (c5fcb1a653c6) Stopping /Karakeep-WEB (c0927c25acb1) with SIGTERM Creating /Karakeep-WEB Removing image ed31f8f95786 Log from karakeep web ✓ Starting... ✓ Ready in 4.1s 2025-06-30T09:34:10.921Z info: Workers version: nightly 2025-06-30T09:34:11.393Z info: [crawler] Loading adblocker ... 2025-06-30T09:34:11.507Z info: [Crawler] Connecting to existing browser instance: http://chrome:9222 2025-06-30T09:34:11.575Z info: [Crawler] Successfully resolved IP address, new address: http://172.29.12.5:9222/ 2025-06-30T09:34:14.067Z info: Starting crawler worker ... 2025-06-30T09:34:14.068Z info: Starting inference worker ... 2025-06-30T09:34:14.069Z info: Starting search indexing worker ... 2025-06-30T09:34:14.070Z info: Starting tidy assets worker ... 2025-06-30T09:34:14.070Z info: Starting video worker ... 2025-06-30T09:34:14.071Z info: Starting feed worker ... 2025-06-30T09:34:14.072Z info: Starting asset preprocessing worker ... 2025-06-30T09:34:14.073Z info: Starting webhook worker ... 2025-06-30T09:34:14.073Z info: Starting rule engine worker ... [next-auth][warn][NEXTAUTH_URL] https://next-auth.js.org/warnings#nextauth_url <-- GET /api/assets/38e309e0-2e2f-4149-bf6d-b8c446e6502e --> GET /api/assets/38e309e0-2e2f-4149-bf6d-b8c446e6502e 200 35ms <-- GET /api/assets/f95f5743-6e4a-4268-96b2-ebc060fcf6de <-- GET /api/assets/760b39a6-971b-4082-8c80-aa343034c65b <-- GET /api/assets/513dd28a-b0da-4977-93a0-cb40dc763504 <-- GET /api/assets/13b1d727-b539-4040-8726-d92292b202dd <-- GET /api/assets/ff0f6fb5-9f0c-490f-b09c-9b28101694ba --> GET /api/assets/f95f5743-6e4a-4268-96b2-ebc060fcf6de 200 36s --> GET /api/assets/760b39a6-971b-4082-8c80-aa343034c65b 200 36s --> GET /api/assets/513dd28a-b0da-4977-93a0-cb40dc763504 200 36s --> GET /api/assets/13b1d727-b539-4040-8726-d92292b202dd 200 37s --> GET /api/assets/ff0f6fb5-9f0c-490f-b09c-9b28101694ba 200 291ms <-- GET /api/assets/d6a1daa0-fb06-47f4-8bb3-bc47bffb8bc9 --> GET /api/assets/d6a1daa0-fb06-47f4-8bb3-bc47bffb8bc9 200 221ms <-- GET /api/assets/5db25bc1-68ca-4d29-8278-b600ff3cb5ed --> GET /api/assets/5db25bc1-68ca-4d29-8278-b600ff3cb5ed 200 31ms <-- GET /api/assets/e0f1111e-1cbf-4106-b0cd-dc63e23a97e1 <-- GET /api/assets/1e2989e6-86bb-400a-b168-153b1ed11a46 --> GET /api/assets/e0f1111e-1cbf-4106-b0cd-dc63e23a97e1 200 588ms <-- GET /api/assets/831878e8-1c52-47c4-9de2-5936ade75293 --> GET /api/assets/1e2989e6-86bb-400a-b168-153b1ed11a46 200 603ms <-- GET /api/assets/e442a966-1589-4e9e-942b-414ab101f865 <-- GET /api/assets/c07c3570-0523-4da2-bc11-9cf27eb92fd3 --> GET /api/assets/831878e8-1c52-47c4-9de2-5936ade75293 200 45ms <-- GET /api/assets/a6082c41-bd2d-4e76-90d7-f7e718c10bb4 --> GET /api/assets/e442a966-1589-4e9e-942b-414ab101f865 200 44ms <-- GET /api/assets/8fc3e61e-964f-444e-8f1b-47bd5ffa4ad0 --> GET /api/assets/c07c3570-0523-4da2-bc11-9cf27eb92fd3 200 53ms --> GET /api/assets/a6082c41-bd2d-4e76-90d7-f7e718c10bb4 200 35ms --> GET /api/assets/8fc3e61e-964f-444e-8f1b-47bd5ffa4ad0 200 114ms <-- GET /api/assets/5a864fb4-6cfe-450d-b775-472130f00060 <-- GET /api/assets/2fb50598-556f-4219-8755-495f808a5984 <-- GET /api/assets/dbbcce8c-3624-417a-9062-79f58d62c713 <-- GET /api/assets/301cd8b5-3044-4955-8fc2-172d5ba63f26 --> GET /api/assets/5a864fb4-6cfe-450d-b775-472130f00060 200 461ms --> GET /api/assets/2fb50598-556f-4219-8755-495f808a5984 200 470ms --> GET /api/assets/dbbcce8c-3624-417a-9062-79f58d62c713 200 474ms --> GET /api/assets/301cd8b5-3044-4955-8fc2-172d5ba63f26 200 496ms <-- GET /api/assets/5cb6b8c3-106e-4bf6-9941-6f2f1db07d83 <-- GET /api/assets/842d7db9-9a47-4374-b7a8-f20a5a362b9a <-- GET /api/assets/b675ffe4-bb46-4d1d-a062-1f87f871c01a <-- GET /api/assets/28110704-0a4d-4d96-80bf-2bb624f5d3cf --> GET /api/assets/842d7db9-9a47-4374-b7a8-f20a5a362b9a 200 68ms --> GET /api/assets/b675ffe4-bb46-4d1d-a062-1f87f871c01a 200 49ms --> GET /api/assets/28110704-0a4d-4d96-80bf-2bb624f5d3cf 200 46ms --> GET /api/assets/5cb6b8c3-106e-4bf6-9941-6f2f1db07d83 200 86ms 2025-06-30T10:00:00.391Z info: [feed] Scheduling feed refreshing jobs ... <-- GET /api/assets/38e309e0-2e2f-4149-bf6d-b8c446e6502e --> GET /api/assets/38e309e0-2e2f-4149-bf6d-b8c446e6502e 200 226ms <-- GET /api/assets/5cb6b8c3-106e-4bf6-9941-6f2f1db07d83 --> GET /api/assets/5cb6b8c3-106e-4bf6-9941-6f2f1db07d83 200 336ms <-- GET /api/assets/842d7db9-9a47-4374-b7a8-f20a5a362b9a --> GET /api/assets/842d7db9-9a47-4374-b7a8-f20a5a362b9a 200 23ms <-- GET /api/assets/28110704-0a4d-4d96-80bf-2bb624f5d3cf --> GET /api/assets/28110704-0a4d-4d96-80bf-2bb624f5d3cf 200 21ms <-- GET /api/assets/b675ffe4-bb46-4d1d-a062-1f87f871c01a --> GET /api/assets/b675ffe4-bb46-4d1d-a062-1f87f871c01a 200 20ms <-- GET /api/assets/174c29e1-8620-48ad-9be5-114f703d5fb6 --> GET /api/assets/174c29e1-8620-48ad-9be5-114f703d5fb6 200 10ms <-- GET /api/assets/86a7829e-0c91-4a9a-9e7b-97c91ba74dd9 --> GET /api/assets/86a7829e-0c91-4a9a-9e7b-97c91ba74dd9 200 116ms <-- GET /api/assets/edb1696a-25cf-40f3-a260-69c604a89106 --> GET /api/assets/edb1696a-25cf-40f3-a260-69c604a89106 200 18ms <-- GET /api/assets/d85e95e9-62fe-4373-9c9d-2d3dafc9368e --> GET /api/assets/d85e95e9-62fe-4373-9c9d-2d3dafc9368e 200 39ms <-- GET /api/assets/658146cc-867b-438c-bc73-7a0181dc499c --> GET /api/assets/658146cc-867b-438c-bc73-7a0181dc499c 200 11ms <-- GET /api/assets/329f4796-bee2-4ded-aaa8-e46f2cd6d19f --> GET /api/assets/329f4796-bee2-4ded-aaa8-e46f2cd6d19f 200 70ms <-- GET /api/assets/da56ebe3-029d-495f-bd0d-2fb5e8861234 --> GET /api/assets/da56ebe3-029d-495f-bd0d-2fb5e8861234 200 253ms <-- GET /api/assets/3a2e07b9-8693-4c02-bc60-d7c32d064e7a --> GET /api/assets/3a2e07b9-8693-4c02-bc60-d7c32d064e7a 200 38ms <-- GET /api/assets/88155a9d-1fd4-4c7b-b36f-9f7bcd3943fb --> GET /api/assets/88155a9d-1fd4-4c7b-b36f-9f7bcd3943fb 200 41ms <-- GET /api/assets/3f193c01-822e-45e8-9f74-df75bdd38c73 <-- GET /api/assets/5691561a-c452-42e8-bdfe-7c89c9a06087 <-- GET /api/assets/90138225-28f8-40b8-9d76-7cda4f767d3c <-- GET /api/assets/31268154-90e5-4944-ab4b-d94f4a936f24 --> GET /api/assets/3f193c01-822e-45e8-9f74-df75bdd38c73 200 619ms --> GET /api/assets/5691561a-c452-42e8-bdfe-7c89c9a06087 200 68s --> GET /api/assets/90138225-28f8-40b8-9d76-7cda4f767d3c 200 68s ### Device Details Chrome, Firefox, android app ### Exact Karakeep Version Karakeep vnightly 0.25 ### Have you checked the troubleshooting guide? - [x] I have checked the troubleshooting guide and I haven't found a solution to my problem

kerem added the

bug

status/untriaged

labels

2026-03-02 11:54:41 +03:00

kerem commented

2026-03-02 11:54:41 +03:00

Author

Owner

@igortau commented on GitHub (Jul 1, 2025):

I am currently using Karakeep in my setup, but I am encountering performance problems related to the use of SQLite as the database backend. Since Karakeep relies exclusively on SQLite, I am facing scalability and concurrency limitations, which impact overall system responsiveness.
Here are some specific issues I have observed:
• SQLite’s single-writer limitation causes slowdowns and frequent "database is locked" errors during concurrent operations.
• Some database queries take several seconds, resulting in UI lags and delays.
• Attempts to switch to PostgreSQL were not possible because Karakeep does not support other database backends.
Below are some excerpts from my logs that illustrate the problem:

2025-06-23 14:05:12 - ERROR - SQLite busy: database is locked
2025-06-23 14:06:03 - WARNING - Slow query detected: SELECT * FROM notes WHERE user_id=123; Duration: 3500 ms
2025-06-23 14:07:45 - INFO - Starting database vacuum to optimize performance

For reference, I am using the Karakeep container image from your official repository:
https://github.com/karakeep-app/karakeep/pkgs/container/karakeep
Could you please advise on potential ways to improve performance or plans for supporting more scalable databases such as PostgreSQL in the future?
Thank you very much for your time and support.

@igortau commented on GitHub (Jul 1, 2025): I am currently using Karakeep in my setup, but I am encountering performance problems related to the use of SQLite as the database backend. Since Karakeep relies exclusively on SQLite, I am facing scalability and concurrency limitations, which impact overall system responsiveness. Here are some specific issues I have observed: • SQLite’s single-writer limitation causes slowdowns and frequent "database is locked" errors during concurrent operations. • Some database queries take several seconds, resulting in UI lags and delays. • Attempts to switch to PostgreSQL were not possible because Karakeep does not support other database backends. Below are some excerpts from my logs that illustrate the problem: 2025-06-23 14:05:12 - ERROR - SQLite busy: database is locked 2025-06-23 14:06:03 - WARNING - Slow query detected: SELECT * FROM notes WHERE user_id=123; Duration: 3500 ms 2025-06-23 14:07:45 - INFO - Starting database vacuum to optimize performance For reference, I am using the Karakeep container image from your official repository: https://github.com/karakeep-app/karakeep/pkgs/container/karakeep Could you please advise on potential ways to improve performance or plans for supporting more scalable databases such as PostgreSQL in the future? Thank you very much for your time and support.

kerem commented

2026-03-02 11:54:41 +03:00

Author

Owner

@MohamedBassem commented on GitHub (Jul 1, 2025):

@igortau are you by any chance running sqlite on a network attached filesystem? Because it's unlikely you're hitting sqlite limitations unless you have a massive dataset. How large is your sqlite file?

@MohamedBassem commented on GitHub (Jul 1, 2025): @igortau are you by any chance running sqlite on a network attached filesystem? Because it's unlikely you're hitting sqlite limitations unless you have a massive dataset. How large is your sqlite file?

kerem commented

2026-03-02 11:54:41 +03:00

Author

Owner

@igortau commented on GitHub (Jul 1, 2025):

Thank you for your quick response!
Yes, I’m running Karakeep inside a container on a QNAP NAS (model TS-673A), and the SQLite database is stored on a shared volume managed by the NAS. The system is using QTS (not ZFS, likely ext4), and the volume is mounted into the container using Docker’s -v bind mount.
The SQLite file is currently about 25 MB — not huge — but I’m seeing the following issues:
• The UI occasionally becomes unresponsive for 3–5 seconds when creating or editing notes
• I frequently see database is locked errors in the logs
• Some simple queries (e.g. SELECT * FROM notes WHERE user_id = ?) take between 2 to 4 seconds

Here’s a typical log excerpt:
2025-06-23 14:05:12 - ERROR - SQLite busy: database is locked
2025-06-23 14:06:03 - WARNING - Slow query detected: SELECT * FROM notes WHERE user_id=123; Duration: 3500 ms

After your reply, I realized that the problem might be caused by SQLite being used on a networked filesystem (even if it's local to the NAS). I understand that SQLite doesn’t work well with concurrent access over shared or network-attached volumes.
Would you recommend storing the SQLite file on a local disk within the container’s writable layer, or is there a better workaround? I'm also considering switching to a different setup if you plan to support PostgreSQL or another backend in the future.
Thanks again — I really like Karakeep and appreciate the project!

@igortau commented on GitHub (Jul 1, 2025): Thank you for your quick response! Yes, I’m running Karakeep inside a container on a QNAP NAS (model TS-673A), and the SQLite database is stored on a shared volume managed by the NAS. The system is using QTS (not ZFS, likely ext4), and the volume is mounted into the container using Docker’s -v bind mount. The SQLite file is currently about 25 MB — not huge — but I’m seeing the following issues: • The UI occasionally becomes unresponsive for 3–5 seconds when creating or editing notes • I frequently see database is locked errors in the logs • Some simple queries (e.g. SELECT * FROM notes WHERE user_id = ?) take between 2 to 4 seconds Here’s a typical log excerpt: 2025-06-23 14:05:12 - ERROR - SQLite busy: database is locked 2025-06-23 14:06:03 - WARNING - Slow query detected: SELECT * FROM notes WHERE user_id=123; Duration: 3500 ms After your reply, I realized that the problem might be caused by SQLite being used on a networked filesystem (even if it's local to the NAS). I understand that SQLite doesn’t work well with concurrent access over shared or network-attached volumes. Would you recommend storing the SQLite file on a local disk within the container’s writable layer, or is there a better workaround? I'm also considering switching to a different setup if you plan to support PostgreSQL or another backend in the future. Thanks again — I really like Karakeep and appreciate the project!

kerem commented

2026-03-02 11:54:42 +03:00

Author

Owner

@igortau commented on GitHub (Jul 4, 2025):

Hi again,

Thanks for your support earlier!

After updating to the latest Karakeep version, the app starts up fine, but I'm seeing recurring issues related to the crawler worker and Playwright browser.

Karakeep runs inside a container on my QNAP NAS (TS-673A). The chrome browser instance is available at http://chrome:9222, and Karakeep tries to connect to it as shown in logs:

2025-07-04T13:49:48.699Z error: [Crawler] Failed to connect to the browser instance, will retry in 5 secs: browserType.connectOverCDP: connect ECONNREFUSED 172.29.12.4:9222

This happens repeatedly, then Karakeep receives SIGTERM and restarts services:

2025-07-04T13:50:16.970Z info: Received SIGTERM, shutting down ...
...
Running db migration script
...
2025-07-04T13:51:45.688Z info: Workers version: nightly

After that, it reconnects and starts again — until the next browser disconnect.

Additional context:

SQLite file is ~25 MB, stored on NAS shared volume (QTS, ext4)

Everything runs in Docker (with Portainer on NAS)

No heavy traffic or datasets involved

Chrome container might be restarting or crashing silently

Do you have any recommendations for stabilizing the connection to the Playwright browser? Should the browser run in the same container or network namespace?

@igortau commented on GitHub (Jul 4, 2025): Hi again, Thanks for your support earlier! After updating to the latest Karakeep version, the app starts up fine, but I'm seeing recurring issues related to the crawler worker and Playwright browser. Karakeep runs inside a container on my QNAP NAS (TS-673A). The chrome browser instance is available at http://chrome:9222, and Karakeep tries to connect to it as shown in logs: 2025-07-04T13:49:48.699Z error: [Crawler] Failed to connect to the browser instance, will retry in 5 secs: browserType.connectOverCDP: connect ECONNREFUSED 172.29.12.4:9222 This happens repeatedly, then Karakeep receives SIGTERM and restarts services: 2025-07-04T13:50:16.970Z info: Received SIGTERM, shutting down ... ... Running db migration script ... 2025-07-04T13:51:45.688Z info: Workers version: nightly After that, it reconnects and starts again — until the next browser disconnect. --- Additional context: SQLite file is ~25 MB, stored on NAS shared volume (QTS, ext4) Everything runs in Docker (with Portainer on NAS) No heavy traffic or datasets involved Chrome container might be restarting or crashing silently --- Do you have any recommendations for stabilizing the connection to the Playwright browser? Should the browser run in the same container or network namespace?

kerem commented

2026-03-02 11:54:42 +03:00

Author

Owner

@igortau commented on GitHub (Jul 5, 2025):

I've observed an issue with the karakeep-web container repeatedly polling the backend API exposed by karakeep-meili.

When interacting with tasks (e.g., /tasks/53963 and /tasks/53964), the karakeep-web container starts sending a very high number of repeated GET requests per second to the Meili backend.

Here’s a sample log from the karakeep-meili container:
[2025-06-30 23:52:08 +0000] [18] [INFO] 172.22.0.5:34900 - "GET /api/tasks/53964 HTTP/1.1" 200 [2025-06-30 23:52:08 +0000] [18] [INFO] 172.22.0.5:34900 - "GET /api/tasks/53964 HTTP/1.1" 200 [2025-06-30 23:52:08 +0000] [18] [INFO] 172.22.0.5:34900 - "GET /api/tasks/53964 HTTP/1.1" 200

These requests continue even after the task is finished (e.g., deleting an index). The polling doesn't seem to stop or slow down, which causes unnecessary load on the backend.

My setup:

QNAP NAS (TS-673A) with NVIDIA RTX 3090 GPU
Running karakeep-web, karakeep-meili, and karakeep-chrome in Docker via Portainer
Ollama (GPU) is integrated and working correctly

Could you consider adding exponential backoff or stop polling once the task is complete? Or possibly switch to WebSocket-based updates?

Let me know if I can provide more logs or details.

Thanks a lot!

@igortau commented on GitHub (Jul 5, 2025): I've observed an issue with the `karakeep-web` container repeatedly polling the backend API exposed by `karakeep-meili`. When interacting with tasks (e.g., `/tasks/53963` and `/tasks/53964`), the `karakeep-web` container starts sending a very high number of repeated GET requests per second to the Meili backend. Here’s a sample log from the `karakeep-meili` container: [2025-06-30 23:52:08 +0000] [18] [INFO] 172.22.0.5:34900 - "GET /api/tasks/53964 HTTP/1.1" 200 [2025-06-30 23:52:08 +0000] [18] [INFO] 172.22.0.5:34900 - "GET /api/tasks/53964 HTTP/1.1" 200 [2025-06-30 23:52:08 +0000] [18] [INFO] 172.22.0.5:34900 - "GET /api/tasks/53964 HTTP/1.1" 200 These requests continue even after the task is finished (e.g., deleting an index). The polling doesn't seem to stop or slow down, which causes unnecessary load on the backend. My setup: - QNAP NAS (TS-673A) with **NVIDIA RTX 3090** GPU - Running `karakeep-web`, `karakeep-meili`, and `karakeep-chrome` in Docker via Portainer - Ollama (GPU) is integrated and working correctly Could you consider adding exponential backoff or stop polling once the task is complete? Or possibly switch to WebSocket-based updates? Let me know if I can provide more logs or details. Thanks a lot!

kerem commented

2026-03-02 11:54:42 +03:00

Author

Owner

@rtmvc commented on GitHub (Jul 6, 2025):

Hello,

First, many thanks for this cool project I've just discovered. I hope it will help me preserve my 30k bookmarks after Pocket shutdown as well as my "hoarding" addiction.

I don't want shamefully plug into the discussion, but I'm running into the same kind of issues. I'm running Karakeep 0.25 on Podman using a Raspberry Pi4 with SQLite stored locally on an external HDD.
It's currently importing all my Pocket bookmarks, and I'm experiencing the same kind of slowness on the UI and keep having lots of SQLite busy, SQLite error errors in the logs constantly:

 2025-07-06T08:26:17.022Z info: Workers version: 0.25.0
 2025-07-06T08:26:17.058Z info: [crawler] Loading adblocker ...
 2025-07-06T08:26:17.163Z info: [Crawler] Connecting to existing browser instance: http://karakeep-chrome:9222
 2025-07-06T08:26:17.179Z info: [Crawler] Successfully resolved IP address, new address: http://127.0.0.1:9222/
 2025-07-06T08:26:17.552Z info: Starting crawler worker ...
 2025-07-06T08:26:17.555Z info: Starting inference worker ...
 2025-07-06T08:26:17.557Z info: Starting search indexing worker ...
 2025-07-06T08:26:17.558Z info: Starting tidy assets worker ...
 2025-07-06T08:26:17.559Z info: Starting video worker ...
 2025-07-06T08:26:17.562Z info: Starting feed worker ...
 2025-07-06T08:26:17.564Z info: Starting asset preprocessing worker ...
 2025-07-06T08:26:17.566Z info: Starting webhook worker ...
 2025-07-06T08:26:17.567Z info: Starting rule engine worker ...
 2025-07-06T08:26:28.940Z info: [webhook][2477] Starting a webhook job for bookmark with id "euhj6p4adha3wq3xr5m2cp3k for operation "created"
 2025-07-06T08:26:28.946Z info: [webhook][2477] Completed successfully
 2025-07-06T08:26:28.972Z error: [Crawler][205] Crawling job failed: SqliteError: database is locked
 SqliteError: database is locked
     at Database.prepare (/app/apps/workers/node_modules/.pnpm/better-sqlite3@11.3.0/node_modules/better-sqlite3/lib/methods/wrappers.js:5:21)
     at BetterSQLiteSession.prepareQuery (/app/apps/workers/node_modules/.pnpm/drizzle-orm@0.38.3_better-sqlite3@11.3.0/node_modules/src/better-sqlite3/session.ts:50:28)
     at QueryPromise._prepare (/app/apps/workers/node_modules/.pnpm/drizzle-orm@0.38.3_better-sqlite3@11.3.0/node_modules/src/sqlite-core/query-builders/query.ts:147:78)
     at QueryPromise.executeRaw (/app/apps/workers/node_modules/.pnpm/drizzle-orm@0.38.3_better-sqlite3@11.3.0/node_modules/src/sqlite-core/query-builders/query.ts:191:16)
     at QueryPromise.execute (/app/apps/workers/node_modules/.pnpm/drizzle-orm@0.38.3_better-sqlite3@11.3.0/node_modules/src/sqlite-core/query-builders/query.ts:197:15)
     at QueryPromise.then (/app/apps/workers/node_modules/.pnpm/drizzle-orm@0.38.3_better-sqlite3@11.3.0/node_modules/src/query-promise.ts:31:15)
     at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
 2025-07-06T08:26:41.716Z info: [ruleEngine][1828] Completed successfully
...
 /app/apps/workers/node_modules/.pnpm/drizzle-orm@0.33.0_better-sqlite3@11.3.0/node_modules/src/better-sqlite3/session.ts:113
                 return this.stmt.run(...params);
                                  ^
 
 SqliteError: database is locked
     at PreparedQuery.run (/app/apps/workers/node_modules/.pnpm/drizzle-orm@0.33.0_better-sqlite3@11.3.0/node_modules/src/better-sqlite3/session.ts:113:20)
     at QueryPromise.run (/app/apps/workers/node_modules/.pnpm/drizzle-orm@0.33.0_better-sqlite3@11.3.0/node_modules/src/sqlite-core/query-builders/update.ts:284:26)
     at QueryPromise.execute (/app/apps/workers/node_modules/.pnpm/drizzle-orm@0.33.0_better-sqlite3@11.3.0/node_modules/src/sqlite-core/query-builders/update.ts:300:53)
     at QueryPromise.then (/app/apps/workers/node_modules/.pnpm/drizzle-orm@0.33.0_better-sqlite3@11.3.0/node_modules/src/query-promise.ts:31:15)
     at process.processTicksAndRejections (node:internal/process/task_queues:105:5) {
   code: 'SQLITE_BUSY'
 }
 
 Node.js v22.16.0

Are there any possible settings, like timeouts, concurrency that could favor this ?
Thanks in advance.

@rtmvc commented on GitHub (Jul 6, 2025): Hello, First, many thanks for this cool project I've just discovered. I hope it will help me preserve my 30k bookmarks after Pocket shutdown as well as my "hoarding" addiction. I don't want shamefully plug into the discussion, but I'm running into the same kind of issues. I'm running Karakeep 0.25 on Podman using a Raspberry Pi4 with SQLite stored locally on an external HDD. It's currently importing all my Pocket bookmarks, and I'm experiencing the same kind of slowness on the UI and keep having lots of `SQLite busy`, `SQLite error` errors in the logs constantly: ``` 2025-07-06T08:26:17.022Z info: Workers version: 0.25.0 2025-07-06T08:26:17.058Z info: [crawler] Loading adblocker ... 2025-07-06T08:26:17.163Z info: [Crawler] Connecting to existing browser instance: http://karakeep-chrome:9222 2025-07-06T08:26:17.179Z info: [Crawler] Successfully resolved IP address, new address: http://127.0.0.1:9222/ 2025-07-06T08:26:17.552Z info: Starting crawler worker ... 2025-07-06T08:26:17.555Z info: Starting inference worker ... 2025-07-06T08:26:17.557Z info: Starting search indexing worker ... 2025-07-06T08:26:17.558Z info: Starting tidy assets worker ... 2025-07-06T08:26:17.559Z info: Starting video worker ... 2025-07-06T08:26:17.562Z info: Starting feed worker ... 2025-07-06T08:26:17.564Z info: Starting asset preprocessing worker ... 2025-07-06T08:26:17.566Z info: Starting webhook worker ... 2025-07-06T08:26:17.567Z info: Starting rule engine worker ... 2025-07-06T08:26:28.940Z info: [webhook][2477] Starting a webhook job for bookmark with id "euhj6p4adha3wq3xr5m2cp3k for operation "created" 2025-07-06T08:26:28.946Z info: [webhook][2477] Completed successfully 2025-07-06T08:26:28.972Z error: [Crawler][205] Crawling job failed: SqliteError: database is locked SqliteError: database is locked at Database.prepare (/app/apps/workers/node_modules/.pnpm/better-sqlite3@11.3.0/node_modules/better-sqlite3/lib/methods/wrappers.js:5:21) at BetterSQLiteSession.prepareQuery (/app/apps/workers/node_modules/.pnpm/drizzle-orm@0.38.3_better-sqlite3@11.3.0/node_modules/src/better-sqlite3/session.ts:50:28) at QueryPromise._prepare (/app/apps/workers/node_modules/.pnpm/drizzle-orm@0.38.3_better-sqlite3@11.3.0/node_modules/src/sqlite-core/query-builders/query.ts:147:78) at QueryPromise.executeRaw (/app/apps/workers/node_modules/.pnpm/drizzle-orm@0.38.3_better-sqlite3@11.3.0/node_modules/src/sqlite-core/query-builders/query.ts:191:16) at QueryPromise.execute (/app/apps/workers/node_modules/.pnpm/drizzle-orm@0.38.3_better-sqlite3@11.3.0/node_modules/src/sqlite-core/query-builders/query.ts:197:15) at QueryPromise.then (/app/apps/workers/node_modules/.pnpm/drizzle-orm@0.38.3_better-sqlite3@11.3.0/node_modules/src/query-promise.ts:31:15) at process.processTicksAndRejections (node:internal/process/task_queues:105:5) 2025-07-06T08:26:41.716Z info: [ruleEngine][1828] Completed successfully ... /app/apps/workers/node_modules/.pnpm/drizzle-orm@0.33.0_better-sqlite3@11.3.0/node_modules/src/better-sqlite3/session.ts:113 return this.stmt.run(...params); ^ SqliteError: database is locked at PreparedQuery.run (/app/apps/workers/node_modules/.pnpm/drizzle-orm@0.33.0_better-sqlite3@11.3.0/node_modules/src/better-sqlite3/session.ts:113:20) at QueryPromise.run (/app/apps/workers/node_modules/.pnpm/drizzle-orm@0.33.0_better-sqlite3@11.3.0/node_modules/src/sqlite-core/query-builders/update.ts:284:26) at QueryPromise.execute (/app/apps/workers/node_modules/.pnpm/drizzle-orm@0.33.0_better-sqlite3@11.3.0/node_modules/src/sqlite-core/query-builders/update.ts:300:53) at QueryPromise.then (/app/apps/workers/node_modules/.pnpm/drizzle-orm@0.33.0_better-sqlite3@11.3.0/node_modules/src/query-promise.ts:31:15) at process.processTicksAndRejections (node:internal/process/task_queues:105:5) { code: 'SQLITE_BUSY' } Node.js v22.16.0 ``` Are there any possible settings, like timeouts, concurrency that could favor this ? Thanks in advance.

kerem referenced this issue

2026-03-02 11:59:40 +03:00

[PR #1785] [MERGED] feat(mobile): add context aware sharing option in mobile app #1899

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/karakeep#1054

No description provided.

Rows
Columns