[GH-ISSUE #3] [Feature Request] Add support for Ollama #1

Closed
opened 2026-03-02 11:45:25 +03:00 by kerem · 4 comments
Owner

Originally created by @MohamedBassem on GitHub (Mar 26, 2024).
Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/3

This issue to track supporting ollama as an alternative to openai.

Originally created by @MohamedBassem on GitHub (Mar 26, 2024). Original GitHub issue: https://github.com/karakeep-app/karakeep/issues/3 This issue to track supporting ollama as an alternative to openai.
kerem 2026-03-02 11:45:25 +03:00
Author
Owner

@MohamedBassem commented on GitHub (Mar 27, 2024):

Just landed the ollama support, it'll be included in the next release (happing later today)!

<!-- gh-comment-id:2022591694 --> @MohamedBassem commented on GitHub (Mar 27, 2024): Just landed the ollama support, it'll be included in the next release (happing later today)!
Author
Owner

@dillfrescott commented on GitHub (Mar 27, 2024):

Yes!!!!

<!-- gh-comment-id:2022659894 --> @dillfrescott commented on GitHub (Mar 27, 2024): Yes!!!!
Author
Owner

@MohamedBassem commented on GitHub (Mar 27, 2024):

Ollama support is now available in the 0.10.0 release! 🎉 You can check the new configuration arguments here!

Basically add the following to the env file (make sure you remove the OPENAI_API_KEY otherwise it takes precedence):

OLLAMA_BASE_URL=<ollama_addr>
INFERENCE_TEXT_MODEL=mistral
INFERENCE_IMAGE_MODEL=llava

Make sure that those two models are pulled in ollama with ollama pull mistral and ollama pull llava.

<!-- gh-comment-id:2022678668 --> @MohamedBassem commented on GitHub (Mar 27, 2024): Ollama support is now available in the `0.10.0` release! 🎉 You can check the new configuration arguments [here](https://docs.hoarder.app/configuration#inference-configs-for-automatic-tagging)! Basically add the following to the env file (make sure you remove the `OPENAI_API_KEY` otherwise it takes precedence): ``` OLLAMA_BASE_URL=<ollama_addr> INFERENCE_TEXT_MODEL=mistral INFERENCE_IMAGE_MODEL=llava ``` Make sure that those two models are pulled in ollama with `ollama pull mistral` and `ollama pull llava`.
Author
Owner

@dillfrescott commented on GitHub (Mar 27, 2024):

Id recommend the openchat 0106 model for text, that thing is impressive for a 7b model!

<!-- gh-comment-id:2023751188 --> @dillfrescott commented on GitHub (Mar 27, 2024): Id recommend the openchat 0106 model for text, that thing is impressive for a 7b model!
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/karakeep#1
No description provided.