starred/karakeep

Fork 0

mirror of https://github.com/karakeep-app/karakeep.git synced 2026-04-25 07:56:05 +03:00

[GH-ISSUE #811] Jobs Stuck in Queue and Never Completing #531

New issue

Closed

opened 2026-03-02 11:50:36 +03:00 by kerem · 9 comments

kerem commented

2026-03-02 11:50:36 +03:00

Owner

Originally created by @shaankhosla on GitHub (Jan 2, 2025).
Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/811

Describe the Bug

I just set up Hoarder and added some links, and after 12 hours I don't think a single job has completed. The jobs page looks like this:

On the homepage the cards are empty:

The logs for the hoarder container have this:

s [TRPCError]: UNAUTHORIZED
at /app/apps/web/.next/server/chunks/1479.js:1:7806
at async h.middlewares (/app/apps/web/.next/server/chunks/6667.js:4:33569)
at async a (/app/apps/web/.next/server/chunks/6667.js:4:32963)
at async t (/app/apps/web/.next/server/chunks/6667.js:4:32336)
at async a (/app/apps/web/.next/server/chunks/6667.js:4:32963)
at async a (/app/apps/web/.next/server/chunks/6667.js:4:32963)
at async a (/app/apps/web/.next/server/chunks/6667.js:4:32963)
at async t (/app/apps/web/.next/server/chunks/6667.js:4:33302)
at async /app/apps/web/.next/server/app/api/trpc/[trpc]/route.js:1:4380
at async Promise.all (index 0) {
code: 'UNAUTHORIZED',
[cause]: undefined
}
s [TRPCError]: UNAUTHORIZED
at /app/apps/web/.next/server/chunks/1479.js:1:7806
at async h.middlewares (/app/apps/web/.next/server/chunks/6667.js:4:33569)
at async a (/app/apps/web/.next/server/chunks/6667.js:4:32963)
at async t (/app/apps/web/.next/server/chunks/6667.js:4:32336)
at async a (/app/apps/web/.next/server/chunks/6667.js:4:32963)
at async a (/app/apps/web/.next/server/chunks/6667.js:4:32963)
at async a (/app/apps/web/.next/server/chunks/6667.js:4:32963)
at async t (/app/apps/web/.next/server/chunks/6667.js:4:33302)
at async /app/apps/web/.next/server/app/api/trpc/[trpc]/route.js:1:4380
at async Promise.all (index 0) {
code: 'UNAUTHORIZED',
[cause]: undefined
}

Steps to Reproduce

This is my configuration:

.env:
DISABLE_PASSWORD_AUTH=true
DATA_DIR=/data
NEXTAUTH_URL=https://bookmarks.example.com
NEXTAUTH_SECRET=
OAUTH_WELLKNOWN_URL=https://auth.example.com/.well-known/openid-configuration
OAUTH_CLIENT_SECRET=
OAUTH_CLIENT_ID=bookmarks
MEILI_ADDR=http://meilisearch:7700
MEILI_MASTER_KEY=
BROWSER_WEB_URL=http://chrome:9222
OPENAI_API_KEY:

docker-compose:
hoarder:
container_name: hoarder
image: ghcr.io/hoarder-app/hoarder:release
restart: unless-stopped
volumes:
- ${HOME_DIRECTORY}/hoarder:/data
ports:
- 3001:3000
env_file:
- hoarder.env
networks:
- hoarder
chrome:
container_name: chrome
image: gcr.io/zenika-hub/alpine-chrome:123
restart: unless-stopped
command:
- --no-sandbox
- --disable-gpu
- --disable-dev-shm-usage
- --remote-debugging-address=0.0.0.0
- --remote-debugging-port=9222
- --hide-scrollbars
networks:
- hoarder
meilisearch:
container_name: meilisearch
image: getmeili/meilisearch:v1.11.1
restart: unless-stopped
env_file:
- hoarder.env
environment:
MEILI_NO_ANALYTICS: "true"
volumes:
- ${HOME_DIRECTORY}/meilisearch:/meili_data
networks:
- hoarder

networks:
hoarder:
external: false
driver: bridge
name: hoarder

Expected Behaviour

Items to flow through the queue

Screenshots or Additional Context

No response

Device Details

No response

Exact Hoarder Version

v0.20.0

Have you checked the troubleshooting guide?

I have checked the troubleshooting guide and I haven't found a solution to my problem

Originally created by @shaankhosla on GitHub (Jan 2, 2025). Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/811 ### Describe the Bug I just set up Hoarder and added some links, and after 12 hours I don't think a single job has completed. The jobs page looks like this: <img width="670" alt="image" src="https://github.com/user-attachments/assets/6b533912-ceed-4c66-8d68-f1583e71922f" /> On the homepage the cards are empty: <img width="1267" alt="image" src="https://github.com/user-attachments/assets/a6f1cd31-8251-4d93-bfe3-a2327b1857ed" /> The logs for the hoarder container have this: s [TRPCError]: UNAUTHORIZED at /app/apps/web/.next/server/chunks/1479.js:1:7806 at async h.middlewares (/app/apps/web/.next/server/chunks/6667.js:4:33569) at async a (/app/apps/web/.next/server/chunks/6667.js:4:32963) at async t (/app/apps/web/.next/server/chunks/6667.js:4:32336) at async a (/app/apps/web/.next/server/chunks/6667.js:4:32963) at async a (/app/apps/web/.next/server/chunks/6667.js:4:32963) at async a (/app/apps/web/.next/server/chunks/6667.js:4:32963) at async t (/app/apps/web/.next/server/chunks/6667.js:4:33302) at async /app/apps/web/.next/server/app/api/trpc/[trpc]/route.js:1:4380 at async Promise.all (index 0) { code: 'UNAUTHORIZED', [cause]: undefined } s [TRPCError]: UNAUTHORIZED at /app/apps/web/.next/server/chunks/1479.js:1:7806 at async h.middlewares (/app/apps/web/.next/server/chunks/6667.js:4:33569) at async a (/app/apps/web/.next/server/chunks/6667.js:4:32963) at async t (/app/apps/web/.next/server/chunks/6667.js:4:32336) at async a (/app/apps/web/.next/server/chunks/6667.js:4:32963) at async a (/app/apps/web/.next/server/chunks/6667.js:4:32963) at async a (/app/apps/web/.next/server/chunks/6667.js:4:32963) at async t (/app/apps/web/.next/server/chunks/6667.js:4:33302) at async /app/apps/web/.next/server/app/api/trpc/[trpc]/route.js:1:4380 at async Promise.all (index 0) { code: 'UNAUTHORIZED', [cause]: undefined } ### Steps to Reproduce This is my configuration: .env: DISABLE_PASSWORD_AUTH=true DATA_DIR=/data NEXTAUTH_URL=https://bookmarks.example.com NEXTAUTH_SECRET=<redacted> OAUTH_WELLKNOWN_URL=https://auth.example.com/.well-known/openid-configuration OAUTH_CLIENT_SECRET=<redacted> OAUTH_CLIENT_ID=bookmarks MEILI_ADDR=http://meilisearch:7700 MEILI_MASTER_KEY=<redacted> BROWSER_WEB_URL=http://chrome:9222 OPENAI_API_KEY: <redacted> docker-compose: hoarder: container_name: hoarder image: ghcr.io/hoarder-app/hoarder:release restart: unless-stopped volumes: - ${HOME_DIRECTORY}/hoarder:/data ports: - 3001:3000 env_file: - hoarder.env networks: - hoarder chrome: container_name: chrome image: gcr.io/zenika-hub/alpine-chrome:123 restart: unless-stopped command: - --no-sandbox - --disable-gpu - --disable-dev-shm-usage - --remote-debugging-address=0.0.0.0 - --remote-debugging-port=9222 - --hide-scrollbars networks: - hoarder meilisearch: container_name: meilisearch image: getmeili/meilisearch:v1.11.1 restart: unless-stopped env_file: - hoarder.env environment: MEILI_NO_ANALYTICS: "true" volumes: - ${HOME_DIRECTORY}/meilisearch:/meili_data networks: - hoarder networks: hoarder: external: false driver: bridge name: hoarder ### Expected Behaviour Items to flow through the queue ### Screenshots or Additional Context _No response_ ### Device Details _No response_ ### Exact Hoarder Version v0.20.0 ### Have you checked the troubleshooting guide? - [X] I have checked the troubleshooting guide and I haven't found a solution to my problem

kerem

2026-03-02 11:50:36 +03:00

closed this issue
added the
question
label

kerem commented

2026-03-02 11:50:37 +03:00

Author

Owner

@shaankhosla commented on GitHub (Jan 2, 2025):

Also when I search it says "No bookmarks found" every time.

@shaankhosla commented on GitHub (Jan 2, 2025): Also when I search it says "No bookmarks found" every time.

kerem commented

2026-03-02 11:50:37 +03:00

Author

Owner

@MohamedBassem commented on GitHub (Jan 2, 2025):

Can you restart the container and share the logs that gets logged after the container startup?

Search not working is expected, because the indexing jobs seems to be stuck as well.

@MohamedBassem commented on GitHub (Jan 2, 2025): Can you restart the container and share the logs that gets logged after the container startup? Search not working is expected, because the indexing jobs seems to be stuck as well.

kerem commented

2026-03-02 11:50:37 +03:00

Author

Owner

@shaankhosla commented on GitHub (Jan 2, 2025):

Can you restart the container and share the logs that gets logged after the container startup?

Search not working is expected, because the indexing jobs seems to be stuck as well.

s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service init-db-migration: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
Running db migration script
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service init-db-migration successfully started
s6-rc: info: service svc-workers: starting
s6-rc: info: service svc-web: starting
s6-rc: info: service svc-workers successfully started
s6-rc: info: service svc-web successfully started
s6-rc: info: service legacy-services: starting
s6-rc: info: service legacy-services successfully started
▲ Next.js 14.2.15

Local: http://localhost:3000
Network: http://0.0.0.0:3000

✓ Starting...
✓ Ready in 243ms

@hoarder/workers@0.1.0 start:prod /app/apps/workers
tsx index.ts

@shaankhosla commented on GitHub (Jan 2, 2025): > Can you restart the container and share the logs that gets logged after the container startup? > > Search not working is expected, because the indexing jobs seems to be stuck as well. s6-rc: info: service s6rc-oneshot-runner: starting s6-rc: info: service s6rc-oneshot-runner successfully started s6-rc: info: service fix-attrs: starting s6-rc: info: service init-db-migration: starting s6-rc: info: service fix-attrs successfully started s6-rc: info: service legacy-cont-init: starting Running db migration script s6-rc: info: service legacy-cont-init successfully started s6-rc: info: service init-db-migration successfully started s6-rc: info: service svc-workers: starting s6-rc: info: service svc-web: starting s6-rc: info: service svc-workers successfully started s6-rc: info: service svc-web successfully started s6-rc: info: service legacy-services: starting s6-rc: info: service legacy-services successfully started ▲ Next.js 14.2.15 - Local: http://localhost:3000 - Network: http://0.0.0.0:3000 ✓ Starting... ✓ Ready in 243ms > @hoarder/workers@0.1.0 start:prod /app/apps/workers > tsx index.ts 2025-01-02T17:55:00.514Z info: Workers version: 0.20.0 2025-01-02T17:55:00.522Z info: [crawler] Loading adblocker ... (node:121) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead. (Use `node --trace-deprecation ...` to show where the warning was created)

kerem commented

2026-03-02 11:50:37 +03:00

Author

Owner

@MohamedBassem commented on GitHub (Jan 2, 2025):

Ah, the worker seems to be stuck in downloading the adblocker at least until you captured the logs. It should timeout after a minute or two. Can you share the logs after the adblocker step is done?

@MohamedBassem commented on GitHub (Jan 2, 2025): Ah, the worker seems to be stuck in downloading the adblocker at least until you captured the logs. It should timeout after a minute or two. Can you share the logs after the adblocker step is done?

kerem commented

2026-03-02 11:50:37 +03:00

Author

Owner

@MohamedBassem commented on GitHub (Jan 2, 2025):

If you live in a place where network access is restricted (e.g to github) you might benefit from disabling adblocker step completely with CRAWLER_ENABLE_ADBLOCKER=false

@MohamedBassem commented on GitHub (Jan 2, 2025): If you live in a place where network access is restricted (e.g to github) you might benefit from disabling adblocker step completely with CRAWLER_ENABLE_ADBLOCKER=false

kerem commented

2026-03-02 11:50:37 +03:00

Author

Owner

@shaankhosla commented on GitHub (Jan 2, 2025):

Ah, the worker seems to be stuck in downloading the adblocker at least until you captured the logs. It should timeout after a minute or two. Can you share the logs after the adblocker step is done?

Ah okay. It's been ~8 minutes since I restarted it and it still just says this:

Local: http://localhost:3000
Network: http://0.0.0.0:3000

✓ Starting...
✓ Ready in 243ms

@hoarder/workers@0.1.0 start:prod /app/apps/workers
tsx index.ts

2025-01-02T17:55:00.514Z info: Workers version: 0.20.0
2025-01-02T17:55:00.522Z info: [crawler] Loading adblocker ...
(node:121) [DEP0040] DeprecationWarning: The punycode module is deprecated. Please use a userland alternative instead.
(Use node --trace-deprecation ... to show where the warning was created)
(node:69) [DEP0040] DeprecationWarning: The punycode module is deprecated. Please use a userland alternative instead.
(Use node --trace-deprecation ... to show where the warning was created)

@shaankhosla commented on GitHub (Jan 2, 2025): > Ah, the worker seems to be stuck in downloading the adblocker at least until you captured the logs. It should timeout after a minute or two. Can you share the logs after the adblocker step is done? Ah okay. It's been ~8 minutes since I restarted it and it still just says this: s6-rc: info: service s6rc-oneshot-runner: starting s6-rc: info: service s6rc-oneshot-runner successfully started s6-rc: info: service fix-attrs: starting s6-rc: info: service init-db-migration: starting s6-rc: info: service fix-attrs successfully started s6-rc: info: service legacy-cont-init: starting Running db migration script s6-rc: info: service legacy-cont-init successfully started s6-rc: info: service init-db-migration successfully started s6-rc: info: service svc-workers: starting s6-rc: info: service svc-web: starting s6-rc: info: service svc-workers successfully started s6-rc: info: service svc-web successfully started s6-rc: info: service legacy-services: starting s6-rc: info: service legacy-services successfully started ▲ Next.js 14.2.15 - Local: http://localhost:3000 - Network: http://0.0.0.0:3000 ✓ Starting... ✓ Ready in 243ms > @hoarder/workers@0.1.0 start:prod /app/apps/workers > tsx index.ts 2025-01-02T17:55:00.514Z info: Workers version: 0.20.0 2025-01-02T17:55:00.522Z info: [crawler] Loading adblocker ... (node:121) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead. (Use `node --trace-deprecation ...` to show where the warning was created) (node:69) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead. (Use `node --trace-deprecation ...` to show where the warning was created)

kerem commented

2026-03-02 11:50:37 +03:00

Author

Owner

@MohamedBassem commented on GitHub (Jan 2, 2025):

can you disable adblocker with the setting above and try again?

@MohamedBassem commented on GitHub (Jan 2, 2025): can you disable adblocker with the setting above and try again?

kerem commented

2026-03-02 11:50:38 +03:00

Author

Owner

@shaankhosla commented on GitHub (Jan 2, 2025):

Just did, it's working :)

@shaankhosla commented on GitHub (Jan 2, 2025): Just did, it's working :) <img width="675" alt="image" src="https://github.com/user-attachments/assets/b5ea62e0-58f8-4ef6-9e6c-92a78699d0f8" />

kerem commented

2026-03-02 11:50:38 +03:00

Author

Owner

@shaankhosla commented on GitHub (Jan 2, 2025):

Thanks!

@shaankhosla commented on GitHub (Jan 2, 2025): Thanks!

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/karakeep#531

No description provided.

Rows
Columns