starred/karakeep

Fork 0

mirror of https://github.com/karakeep-app/karakeep.git synced 2026-04-25 07:56:05 +03:00

[GH-ISSUE #975] Error `Did not receive done or success response in stream` when using Ollama on a slow machine with a generous timeout #645

New issue

Closed

opened 2026-03-02 11:51:35 +03:00 by kerem · 3 comments

kerem commented

2026-03-02 11:51:35 +03:00

Owner

Originally created by @buresdv on GitHub (Feb 3, 2025).
Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/975

Describe the Bug

I'm using Hoarder with a local installation of Ollama, both running through Docker Compose. My server is pretty slow, so it takes a long time to generate the tags.

Because I don't care about the speed with which the tags are generated, I opted for the 14b DeepSeek model, which should provide satisfactory tagging. I used to use the smaller model, which worked, but the results were not as relevant. However, when I switched the model in the env variable from 7b to 14b, the problem started happening. The model also works correctly when interacted with through Ollama's web UI.

However, no usable tags are generated, and the following errors show up in the logs:

2025-02-03T09:28:36.077Z info: [inference][426] Starting an inference job for bookmark with id "w5rqvyx59b6l0jy546hjuls6"
2025-02-03T09:28:46.626Z warn: Got an exception from ollama, will still attempt to deserialize the response we got so far: Error: Did not receive done or success response in stream.
2025-02-03T09:28:46.626Z error: [inference][426] inference job failed: Error: [inference][426] The model ignored our prompt and didn't respond with the expected JSON: {"issues":[{"code":"invalid_type","expected":"array","received":"undefined","path":["tags"],"message":"Required"}],"name":"ZodError"}. Here's a sneak peak from the response: {}

These unusable tags then get applied to the bookmark:

Steps to Reproduce

Set up Hoarder and connect it to a local instance Ollama
Save a bookmark
Wait for the unusable tags to get applied

Expected Behaviour

Relevant tags get applied to the bookmark

Screenshots or Additional Context

Compose files

Hoarder

version: "3.8"
services:
  web:
    image: ghcr.io/hoarder-app/hoarder:${HOARDER_VERSION:-release}
    restart: unless-stopped
    volumes:
      - /home/david/SSD/Externí/DockerContainery/Hoarder/app:/data
    ports:
      - 3075:3000
    env_file:
      - stack.env
    environment:
      MEILI_ADDR: http://meilisearch:7700
      BROWSER_WEB_URL: http://chrome:9222
      # OPENAI_API_KEY: ...
      DATA_DIR: /data
  chrome:
    image: gcr.io/zenika-hub/alpine-chrome:123
    restart: unless-stopped
    command:
      - --no-sandbox
      - --disable-gpu
      - --disable-dev-shm-usage
      - --remote-debugging-address=0.0.0.0
      - --remote-debugging-port=9222
      - --hide-scrollbars
  meilisearch:
    image: getmeili/meilisearch:v1.11.1
    restart: unless-stopped
    env_file:
      - stack.env
    environment:
      MEILI_NO_ANALYTICS: "true"
    volumes:
      - /home/david/SSD/Externí/DockerContainery/Hoarder/search:/meili_data

volumes:
  meilisearch:
  data:

Relevant environment variables

OLLAMA_BASE_URL: http://192.168.88.231:11434
INFERENCE_TEXT_MODEL: deepseek-r1:14b
INFERENCE_IMAGE_MODEL: llama3.2-vision:11b
INFERENCE_JOB_TIMEOUT_SEC: 1200

Ollama

services:
  ollama:
    volumes:
      - /home/david/SSD/Externí/DockerContainery/ollama/models:/root/.ollama
    container_name: ollama
    pull_policy: always
    tty: true
    restart: unless-stopped
    image: ollama/ollama:${OLLAMA_DOCKER_TAG-rocm}
    ports:
      - 11434:11434
    devices:
      - /dev/kfd:/dev/kfd
      - /dev/dri:/dev/dri
    environment:
      - 'HSA_OVERRIDE_GFX_VERSION=${HSA_OVERRIDE_GFX_VERSION-11.0.0}'

  open-webui:
    build:
      context: .
      args:
        OLLAMA_BASE_URL: '/ollama'
      dockerfile: Dockerfile
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    volumes:
      - /home/david/SSD/Externí/DockerContainery/ollama/ui:/app/backend/data
    depends_on:
      - ollama
    ports:
      - 6907:8080
    environment:
      # - 'OLLAMA_BASE_URL=http://ollama:11434'
      - 'WEBUI_SECRET_KEY=(key)'
    extra_hosts:
      - host.docker.internal:host-gateway
    restart: unless-stopped

volumes:
  ollama: {}
  open-webui: {}

Logs

Hoarder web UI
Ollama

Ollama UI

Installed models:

Device Details

No response

Exact Hoarder Version

v0.22.0

Have you checked the troubleshooting guide?

I have checked the troubleshooting guide and I haven't found a solution to my problem

Originally created by @buresdv on GitHub (Feb 3, 2025). Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/975 ### Describe the Bug I'm using Hoarder with a local installation of Ollama, both running through Docker Compose. My server is pretty slow, so it takes a long time to generate the tags. Because I don't care about the speed with which the tags are generated, I opted for the 14b DeepSeek model, which should provide satisfactory tagging. I used to use the smaller model, which worked, but the results were not as relevant. However, when I switched the model in the env variable from 7b to 14b, the problem started happening. The model also works correctly when interacted with through Ollama's web UI. However, no usable tags are generated, and the following errors show up in the logs: ``` 2025-02-03T09:28:36.077Z info: [inference][426] Starting an inference job for bookmark with id "w5rqvyx59b6l0jy546hjuls6" 2025-02-03T09:28:46.626Z warn: Got an exception from ollama, will still attempt to deserialize the response we got so far: Error: Did not receive done or success response in stream. 2025-02-03T09:28:46.626Z error: [inference][426] inference job failed: Error: [inference][426] The model ignored our prompt and didn't respond with the expected JSON: {"issues":[{"code":"invalid_type","expected":"array","received":"undefined","path":["tags"],"message":"Required"}],"name":"ZodError"}. Here's a sneak peak from the response: {} ``` These unusable tags then get applied to the bookmark: <img width="398" alt="Image" src="https://github.com/user-attachments/assets/309949d8-de64-43c3-8e9e-c4d1a3379af3" /> ### Steps to Reproduce 1. Set up Hoarder and connect it to a local instance Ollama 2. Save a bookmark 3. Wait for the unusable tags to get applied ### Expected Behaviour Relevant tags get applied to the bookmark ### Screenshots or Additional Context ### Compose files **Hoarder** ```yml version: "3.8" services: web: image: ghcr.io/hoarder-app/hoarder:${HOARDER_VERSION:-release} restart: unless-stopped volumes: - /home/david/SSD/Externí/DockerContainery/Hoarder/app:/data ports: - 3075:3000 env_file: - stack.env environment: MEILI_ADDR: http://meilisearch:7700 BROWSER_WEB_URL: http://chrome:9222 # OPENAI_API_KEY: ... DATA_DIR: /data chrome: image: gcr.io/zenika-hub/alpine-chrome:123 restart: unless-stopped command: - --no-sandbox - --disable-gpu - --disable-dev-shm-usage - --remote-debugging-address=0.0.0.0 - --remote-debugging-port=9222 - --hide-scrollbars meilisearch: image: getmeili/meilisearch:v1.11.1 restart: unless-stopped env_file: - stack.env environment: MEILI_NO_ANALYTICS: "true" volumes: - /home/david/SSD/Externí/DockerContainery/Hoarder/search:/meili_data volumes: meilisearch: data: ``` _Relevant environment variables_ ``` OLLAMA_BASE_URL: http://192.168.88.231:11434 INFERENCE_TEXT_MODEL: deepseek-r1:14b INFERENCE_IMAGE_MODEL: llama3.2-vision:11b INFERENCE_JOB_TIMEOUT_SEC: 1200 ``` **Ollama** ```yml services: ollama: volumes: - /home/david/SSD/Externí/DockerContainery/ollama/models:/root/.ollama container_name: ollama pull_policy: always tty: true restart: unless-stopped image: ollama/ollama:${OLLAMA_DOCKER_TAG-rocm} ports: - 11434:11434 devices: - /dev/kfd:/dev/kfd - /dev/dri:/dev/dri environment: - 'HSA_OVERRIDE_GFX_VERSION=${HSA_OVERRIDE_GFX_VERSION-11.0.0}' open-webui: build: context: . args: OLLAMA_BASE_URL: '/ollama' dockerfile: Dockerfile image: ghcr.io/open-webui/open-webui:main container_name: open-webui volumes: - /home/david/SSD/Externí/DockerContainery/ollama/ui:/app/backend/data depends_on: - ollama ports: - 6907:8080 environment: # - 'OLLAMA_BASE_URL=http://ollama:11434' - 'WEBUI_SECRET_KEY=(key)' extra_hosts: - host.docker.internal:host-gateway restart: unless-stopped volumes: ollama: {} open-webui: {} ``` ### Logs [Hoarder web UI](https://pastebin.com/hCJKytL8) [Ollama](https://pastebin.com/8Jq6NH4Z) ### Ollama UI **Installed models:** <img width="1017" alt="Image" src="https://github.com/user-attachments/assets/54fe4b9a-14f3-45bf-9871-7171ac518384" /> ### Device Details _No response_ ### Exact Hoarder Version v0.22.0 ### Have you checked the troubleshooting guide? - [x] I have checked the troubleshooting guide and I haven't found a solution to my problem

kerem

2026-03-02 11:51:35 +03:00

closed this issue
added the
question
label

kerem commented

2026-03-02 11:51:36 +03:00

Author

Owner

@MohamedBassem commented on GitHub (Feb 9, 2025):

@buresdv this unfortunately is a model issue, some models don't respect the prompt and either returns tags in the wrong format or random tags. According to the logs, we retried multiple times, we either got:

2025-02-03T09:28:36.060Z error: [inference][426] inference job failed: Error: [inference][426] The model ignored our prompt and didn't respond with the expected JSON: {"issues":[{"code":"invalid_type","expected":"array","received":"undefined","path":["tags"],"message":"Required"}],"name":"ZodError"}. Here's a sneak peak from the response: {}

Which as you can see, the model just returned an empty JSON {}.

2025-02-03T09:29:03.250Z warn: Got an exception from ollama, will still attempt to deserialize the response we got so far: Error: Did not receive done or success response in stream.
2025-02-03T09:29:03.250Z info: [inference][426] Inferring tag for bookmark "w5rqvyx59b6l0jy546hjuls6" used NaN tokens and inferred: tag1,tag2

In other times, the model returned tag1, tag2 which is what you're seeing in the UI.

So your only options are:

Attempting to increase context length of the model with INFERENCE_CONTEXT_LENGTH.
Choose another model.
Using custom prompts to try to convince the model to do what you want it to do.

@MohamedBassem commented on GitHub (Feb 9, 2025): @buresdv this unfortunately is a model issue, some models don't respect the prompt and either returns tags in the wrong format or random tags. According to the logs, we retried multiple times, we either got: > 2025-02-03T09:28:36.060Z error: [inference][426] inference job failed: Error: [inference][426] The model ignored our prompt and didn't respond with the expected JSON: {"issues":[{"code":"invalid_type","expected":"array","received":"undefined","path":["tags"],"message":"Required"}],"name":"ZodError"}. Here's a sneak peak from the response: {} Which as you can see, the model just returned an empty JSON `{}`. > 2025-02-03T09:29:03.250Z warn: Got an exception from ollama, will still attempt to deserialize the response we got so far: Error: Did not receive done or success response in stream. 2025-02-03T09:29:03.250Z info: [inference][426] Inferring tag for bookmark "w5rqvyx59b6l0jy546hjuls6" used NaN tokens and inferred: tag1,tag2 In other times, the model returned `tag1`, `tag2` which is what you're seeing in the UI. So your only options are: 1. Attempting to increase context length of the model with `INFERENCE_CONTEXT_LENGTH`. 2. Choose another model. 3. Using custom prompts to try to convince the model to do what you want it to do.

kerem commented

2026-03-02 11:51:37 +03:00

Author

Owner

@buresdv commented on GitHub (Feb 12, 2025):

Thank you for the response, it makes sense. I increased the inference context length and it helped somewhat, but I'm still having this problem more often than not.

I’ll try different models to see what will happen, but it's difficult because my server is very slow 🥲

As an aside, is there a way to regenerate the tags without having to delete and save the item again? That would be a really handy feature in cases where the tags are either pretty bad, or not existing.

@buresdv commented on GitHub (Feb 12, 2025): Thank you for the response, it makes sense. I increased the inference context length and it helped somewhat, but I'm still having this problem more often than not. I’ll try different models to see what will happen, but it's difficult because my server is very slow 🥲 As an aside, is there a way to regenerate the tags without having to delete and save the item again? That would be a really handy feature in cases where the tags are either pretty bad, or not existing.

kerem commented

2026-03-02 11:51:37 +03:00

Author

Owner

@MohamedBassem commented on GitHub (Feb 16, 2025):

@buresdv You can do that from the admin panel via Regenerate AI Tags for Failed Bookmarks Only or Regenerate AI Tags for All Bookmarks, this works in bulk. For links, you can also click the refresh button which recrawls and then tags.

Closing this PR for now, feel free to reoepn it you still have questions.

@MohamedBassem commented on GitHub (Feb 16, 2025): @buresdv You can do that from the admin panel via `Regenerate AI Tags for Failed Bookmarks Only` or `Regenerate AI Tags for All Bookmarks`, this works in bulk. For links, you can also click the `refresh` button which recrawls and then tags. Closing this PR for now, feel free to reoepn it you still have questions.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/karakeep#645

No description provided.

Rows
Columns

[GH-ISSUE #975] Error Did not receive done or success response in stream when using Ollama on a slow machine with a generous timeout #645

Describe the Bug

Steps to Reproduce

Expected Behaviour

Screenshots or Additional Context

Compose files

Logs

Ollama UI

Device Details

Exact Hoarder Version

Have you checked the troubleshooting guide?

[GH-ISSUE #975] Error `Did not receive done or success response in stream` when using Ollama on a slow machine with a generous timeout #645