[GH-ISSUE #1354] Ollama integration seems to request wrong model even though .env is set #865

New issue

Closed

opened 2026-03-02 11:53:22 +03:00 by kerem · 0 comments

kerem commented

2026-03-02 11:53:22 +03:00

Owner

Originally created by @xcutepoison on GitHub (May 3, 2025).
Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/1354

Describe the Bug

Even when setting the environment variable for running AI tagging locally via ollama with the corresponding models defined, karakeep still resorts to requesting the default AI model (gpt-4.1-mini) -> ResponseError: model "gpt-4.1-mini" not found, try pulling it first

Log dump:

2025-05-03T18:55:11.214Z info: [inference][21] Starting an inference job for bookmark with id "yeazn23b6nmoahf0pnjppp7c"
2025-05-03T18:55:11.217Z error: [inference][21] inference job failed: ResponseError: model "gpt-4.1-mini" not found, try pulling it first
ResponseError: model "gpt-4.1-mini" not found, try pulling it first
    at checkOk (/app/apps/workers/node_modules/.pnpm/ollama@0.5.14/node_modules/ollama/dist/browser.cjs:77:9)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async post (/app/apps/workers/node_modules/.pnpm/ollama@0.5.14/node_modules/ollama/dist/browser.cjs:141:3)
    at async Ollama.processStreamableRequest (/app/apps/workers/node_modules/.pnpm/ollama@0.5.14/node_modules/ollama/dist/browser.cjs:252:25)
    at async OllamaInferenceClient.runModel (/app/apps/workers/node_modules/.pnpm/@karakeep+shared@file+packages+shared_better-sqlite3@11.3.0/node_modules/@karakeep/shared/inference.ts:2:4500)
    at async OllamaInferenceClient.inferFromText (/app/apps/workers/node_modules/.pnpm/@karakeep+shared@file+packages+shared_better-sqlite3@11.3.0/node_modules/@karakeep/shared/inference.ts:2:5585)
    at async inferTagsFromText (/app/apps/workers/openaiWorker.ts:6:3133)
    at async inferTags (/app/apps/workers/openaiWorker.ts:6:3410)
    at async Object.runOpenAI [as run] (/app/apps/workers/openaiWorker.ts:6:7110)
    at async Runner.runOnce (/app/apps/workers/node_modules/.pnpm/liteque@0.3.2_better-sqlite3@11.3.0/node_modules/liteque/dist/runner.js:2:2656)
2025-05-03T19:00:00.951Z info: [feed] Scheduling feed refreshing jobs ...
ResponseError: model "gpt-4.1-mini" not found, try pulling it first
    at P (/app/apps/web/.next/server/chunks/3258.js:19:51244)
    ... 8 lines matching cause stack trace ...
    at async g (/app/apps/web/.next/server/chunks/3258.js:7:68) {
  cause: R [ResponseError]: model "gpt-4.1-mini" not found, try pulling it first
      at P (/app/apps/web/.next/server/chunks/3258.js:19:51244)
      at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
      at async L (/app/apps/web/.next/server/chunks/3258.js:19:52399)
      at async z.processStreamableRequest (/app/apps/web/.next/server/chunks/3258.js:19:53793)
      at async eB.runModel (/app/apps/web/.next/server/chunks/6815.js:1:31114)
      at async eB.inferFromText (/app/apps/web/.next/server/chunks/6815.js:1:31849)
      at async /app/apps/web/.next/server/chunks/6815.js:12:200
      at async h.middlewares (/app/apps/web/.next/server/chunks/3258.js:4:46337)
      at async g (/app/apps/web/.next/server/chunks/3258.js:7:68)
      at async g (/app/apps/web/.next/server/chunks/3258.js:7:68) {
    error: 'model "gpt-4.1-mini" not found, try pulling it first',
    status_code: 404
  },
  code: 'INTERNAL_SERVER_ERROR',
  name: 'TRPCError'
}
ResponseError: model "gpt-4.1-mini" not found, try pulling it first
    at P (/app/apps/web/.next/server/chunks/3258.js:19:51244)
    ... 8 lines matching cause stack trace ...
    at async g (/app/apps/web/.next/server/chunks/3258.js:7:68) {
  cause: R [ResponseError]: model "gpt-4.1-mini" not found, try pulling it first
      at P (/app/apps/web/.next/server/chunks/3258.js:19:51244)
      at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
      at async L (/app/apps/web/.next/server/chunks/3258.js:19:52399)
      at async z.processStreamableRequest (/app/apps/web/.next/server/chunks/3258.js:19:53793)
      at async eB.runModel (/app/apps/web/.next/server/chunks/6815.js:1:31114)
      at async eB.inferFromText (/app/apps/web/.next/server/chunks/6815.js:1:31849)
      at async /app/apps/web/.next/server/chunks/6815.js:12:200
      at async h.middlewares (/app/apps/web/.next/server/chunks/3258.js:4:46337)
      at async g (/app/apps/web/.next/server/chunks/3258.js:7:68)
      at async g (/app/apps/web/.next/server/chunks/3258.js:7:68) {
    error: 'model "gpt-4.1-mini" not found, try pulling it first',
    status_code: 404
  },
  code: 'INTERNAL_SERVER_ERROR',
  name: 'TRPCError'
}
ResponseError: model "gpt-4.1-mini" not found, try pulling it first
    at P (/app/apps/web/.next/server/chunks/3258.js:19:51244)
    ... 8 lines matching cause stack trace ...
    at async g (/app/apps/web/.next/server/chunks/3258.js:7:68) {
  cause: R [ResponseError]: model "gpt-4.1-mini" not found, try pulling it first
      at P (/app/apps/web/.next/server/chunks/3258.js:19:51244)
      at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
      at async L (/app/apps/web/.next/server/chunks/3258.js:19:52399)
      at async z.processStreamableRequest (/app/apps/web/.next/server/chunks/3258.js:19:53793)
      at async eB.runModel (/app/apps/web/.next/server/chunks/6815.js:1:31114)
      at async eB.inferFromText (/app/apps/web/.next/server/chunks/6815.js:1:31849)
      at async /app/apps/web/.next/server/chunks/6815.js:12:200
      at async h.middlewares (/app/apps/web/.next/server/chunks/3258.js:4:46337)
      at async g (/app/apps/web/.next/server/chunks/3258.js:7:68)
      at async g (/app/apps/web/.next/server/chunks/3258.js:7:68) {
    error: 'model "gpt-4.1-mini" not found, try pulling it first',
    status_code: 404
  },
  code: 'INTERNAL_SERVER_ERROR',
  name: 'TRPCError'
}

Making an API request manually works too (from another host, unfortunately couldn't run curl in the container) via
curl http://ms-docker:11434/api/generate -d '{"model": "phi3:mini-4k","prompt":"Why is the sky blue?"}'

Steps to Reproduce

This log is being dumped every time you invoke an AI API call, so for example when saving a new link (generating the tags) or requesting an AI summary via the WebUI

Docker Compose snippet:

  karakeep-web:
    image: ghcr.io/karakeep-app/karakeep:${KARAKEEP_VERSION:-release}
    restart: unless-stopped
    volumes:
      # By default, the data is stored in a docker volume called "data".
      # If you want to mount a custom directory, change the volume mapping to:
      # - /path/to/your/directory:/data
      - kara-data:/data
    ports:
      - 3004:3000
    env_file:
      - stack.env
    environment:
      MEILI_ADDR: http://karakeep-meilisearch:7700
      BROWSER_WEB_URL: http://karakeep-chrome:9222
      OLLAMA_BASE_URL: http://192.168.178.62:11434
      INTERFERENCE_TEXT_MODEL: "phi3:mini-4k"
      INTERFERENCE_IMAGE_MODEL: "llava:7b"
      
      # You almost never want to change the value of the DATA_DIR variable.
      # If you want to mount a custom directory, change the volume mapping above instead.
      DATA_DIR: /data # DON'T CHANGE THIS

karakeep version: 0.24.1

Expected Behaviour

Karakeep should be requesting the model defined in the "INTERFERENCE_TEXT_MODEL" environment variable.

Screenshots or Additional Context

No response

Device Details

No response

Exact Karakeep Version

v0.24.1

Have you checked the troubleshooting guide?

I have checked the troubleshooting guide and I haven't found a solution to my problem

Originally created by @xcutepoison on GitHub (May 3, 2025). Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/1354 ### Describe the Bug Even when setting the environment variable for running AI tagging locally via ollama with the corresponding models defined, karakeep still resorts to requesting the default AI model (gpt-4.1-mini) -> ResponseError: model "gpt-4.1-mini" not found, try pulling it first Log dump: ``` 2025-05-03T18:55:11.214Z info: [inference][21] Starting an inference job for bookmark with id "yeazn23b6nmoahf0pnjppp7c" 2025-05-03T18:55:11.217Z error: [inference][21] inference job failed: ResponseError: model "gpt-4.1-mini" not found, try pulling it first ResponseError: model "gpt-4.1-mini" not found, try pulling it first at checkOk (/app/apps/workers/node_modules/.pnpm/ollama@0.5.14/node_modules/ollama/dist/browser.cjs:77:9) at process.processTicksAndRejections (node:internal/process/task_queues:105:5) at async post (/app/apps/workers/node_modules/.pnpm/ollama@0.5.14/node_modules/ollama/dist/browser.cjs:141:3) at async Ollama.processStreamableRequest (/app/apps/workers/node_modules/.pnpm/ollama@0.5.14/node_modules/ollama/dist/browser.cjs:252:25) at async OllamaInferenceClient.runModel (/app/apps/workers/node_modules/.pnpm/@karakeep+shared@file+packages+shared_better-sqlite3@11.3.0/node_modules/@karakeep/shared/inference.ts:2:4500) at async OllamaInferenceClient.inferFromText (/app/apps/workers/node_modules/.pnpm/@karakeep+shared@file+packages+shared_better-sqlite3@11.3.0/node_modules/@karakeep/shared/inference.ts:2:5585) at async inferTagsFromText (/app/apps/workers/openaiWorker.ts:6:3133) at async inferTags (/app/apps/workers/openaiWorker.ts:6:3410) at async Object.runOpenAI [as run] (/app/apps/workers/openaiWorker.ts:6:7110) at async Runner.runOnce (/app/apps/workers/node_modules/.pnpm/liteque@0.3.2_better-sqlite3@11.3.0/node_modules/liteque/dist/runner.js:2:2656) 2025-05-03T19:00:00.951Z info: [feed] Scheduling feed refreshing jobs ... ResponseError: model "gpt-4.1-mini" not found, try pulling it first at P (/app/apps/web/.next/server/chunks/3258.js:19:51244) ... 8 lines matching cause stack trace ... at async g (/app/apps/web/.next/server/chunks/3258.js:7:68) { cause: R [ResponseError]: model "gpt-4.1-mini" not found, try pulling it first at P (/app/apps/web/.next/server/chunks/3258.js:19:51244) at process.processTicksAndRejections (node:internal/process/task_queues:105:5) at async L (/app/apps/web/.next/server/chunks/3258.js:19:52399) at async z.processStreamableRequest (/app/apps/web/.next/server/chunks/3258.js:19:53793) at async eB.runModel (/app/apps/web/.next/server/chunks/6815.js:1:31114) at async eB.inferFromText (/app/apps/web/.next/server/chunks/6815.js:1:31849) at async /app/apps/web/.next/server/chunks/6815.js:12:200 at async h.middlewares (/app/apps/web/.next/server/chunks/3258.js:4:46337) at async g (/app/apps/web/.next/server/chunks/3258.js:7:68) at async g (/app/apps/web/.next/server/chunks/3258.js:7:68) { error: 'model "gpt-4.1-mini" not found, try pulling it first', status_code: 404 }, code: 'INTERNAL_SERVER_ERROR', name: 'TRPCError' } ResponseError: model "gpt-4.1-mini" not found, try pulling it first at P (/app/apps/web/.next/server/chunks/3258.js:19:51244) ... 8 lines matching cause stack trace ... at async g (/app/apps/web/.next/server/chunks/3258.js:7:68) { cause: R [ResponseError]: model "gpt-4.1-mini" not found, try pulling it first at P (/app/apps/web/.next/server/chunks/3258.js:19:51244) at process.processTicksAndRejections (node:internal/process/task_queues:105:5) at async L (/app/apps/web/.next/server/chunks/3258.js:19:52399) at async z.processStreamableRequest (/app/apps/web/.next/server/chunks/3258.js:19:53793) at async eB.runModel (/app/apps/web/.next/server/chunks/6815.js:1:31114) at async eB.inferFromText (/app/apps/web/.next/server/chunks/6815.js:1:31849) at async /app/apps/web/.next/server/chunks/6815.js:12:200 at async h.middlewares (/app/apps/web/.next/server/chunks/3258.js:4:46337) at async g (/app/apps/web/.next/server/chunks/3258.js:7:68) at async g (/app/apps/web/.next/server/chunks/3258.js:7:68) { error: 'model "gpt-4.1-mini" not found, try pulling it first', status_code: 404 }, code: 'INTERNAL_SERVER_ERROR', name: 'TRPCError' } ResponseError: model "gpt-4.1-mini" not found, try pulling it first at P (/app/apps/web/.next/server/chunks/3258.js:19:51244) ... 8 lines matching cause stack trace ... at async g (/app/apps/web/.next/server/chunks/3258.js:7:68) { cause: R [ResponseError]: model "gpt-4.1-mini" not found, try pulling it first at P (/app/apps/web/.next/server/chunks/3258.js:19:51244) at process.processTicksAndRejections (node:internal/process/task_queues:105:5) at async L (/app/apps/web/.next/server/chunks/3258.js:19:52399) at async z.processStreamableRequest (/app/apps/web/.next/server/chunks/3258.js:19:53793) at async eB.runModel (/app/apps/web/.next/server/chunks/6815.js:1:31114) at async eB.inferFromText (/app/apps/web/.next/server/chunks/6815.js:1:31849) at async /app/apps/web/.next/server/chunks/6815.js:12:200 at async h.middlewares (/app/apps/web/.next/server/chunks/3258.js:4:46337) at async g (/app/apps/web/.next/server/chunks/3258.js:7:68) at async g (/app/apps/web/.next/server/chunks/3258.js:7:68) { error: 'model "gpt-4.1-mini" not found, try pulling it first', status_code: 404 }, code: 'INTERNAL_SERVER_ERROR', name: 'TRPCError' } ``` Making an API request manually works too (from another host, unfortunately couldn't run curl in the container) via `curl http://ms-docker:11434/api/generate -d '{"model": "phi3:mini-4k","prompt":"Why is the sky blue?"}'` ### Steps to Reproduce This log is being dumped every time you invoke an AI API call, so for example when saving a new link (generating the tags) or requesting an AI summary via the WebUI Docker Compose snippet: ``` karakeep-web: image: ghcr.io/karakeep-app/karakeep:${KARAKEEP_VERSION:-release} restart: unless-stopped volumes: # By default, the data is stored in a docker volume called "data". # If you want to mount a custom directory, change the volume mapping to: # - /path/to/your/directory:/data - kara-data:/data ports: - 3004:3000 env_file: - stack.env environment: MEILI_ADDR: http://karakeep-meilisearch:7700 BROWSER_WEB_URL: http://karakeep-chrome:9222 OLLAMA_BASE_URL: http://192.168.178.62:11434 INTERFERENCE_TEXT_MODEL: "phi3:mini-4k" INTERFERENCE_IMAGE_MODEL: "llava:7b" # You almost never want to change the value of the DATA_DIR variable. # If you want to mount a custom directory, change the volume mapping above instead. DATA_DIR: /data # DON'T CHANGE THIS ``` karakeep version: 0.24.1 ### Expected Behaviour Karakeep should be requesting the model defined in the "INTERFERENCE_TEXT_MODEL" environment variable. ### Screenshots or Additional Context _No response_ ### Device Details _No response_ ### Exact Karakeep Version v0.24.1 ### Have you checked the troubleshooting guide? - [x] I have checked the troubleshooting guide and I haven't found a solution to my problem

kerem

2026-03-02 11:53:22 +03:00

closed this issue
added the
bug

status/untriaged
labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/karakeep#865

No description provided.

Rows
Columns