starred/karakeep

Fork 0

mirror of https://github.com/karakeep-app/karakeep.git synced 2026-04-25 16:06:04 +03:00

[PR #2315] [CLOSED] refactor(inference): modular multi-provider AI architecture #2067

New issue

Closed

opened 2026-03-02 12:00:25 +03:00 by kerem · 0 comments

kerem commented

2026-03-02 12:00:25 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/karakeep-app/karakeep/pull/2315
Author: @camjac251
Created: 12/29/2025
Status: ❌ Closed

Base: main ← Head: feat/multi-provider-inference

📝 Commits (10+)

0f5bf36 release(docs): release the 0.27 docs
5d33616 fix(docs): fix links to point to lowercase docs
8129953 docs: establish cloudflare redirects after the new lowercase docs
8570b31 fix: migrate systemd config to the new worker path on update. fixes #1889
c09d9d4 fix: fix 5xx on invalid api key
d844284 fix: fix migration failing when no user settings are set. fixes #1919 (#1920)
2375976 feat(extension): add current tab title while saving from extension (#1930)
3747fb0 fix: Fix ranking of search results. fixes #1922
61435eb feat(search): add title search qualifier (#1940)
0e83ad2 refactor: strongly type the search plugin interface

📊 Changes

977 files changed (+89019 additions, -16705 deletions)

View changed files

📝 .dockerignore (+3 -2)
📝 .github/workflows/ci.yml (+8 -0)
📝 .github/workflows/claude.yml (+6 -5)
📝 .github/workflows/cli.yml (+9 -4)
📝 .github/workflows/docker.yml (+124 -24)
📝 .github/workflows/mcp.yml (+9 -4)
➖ .github/workflows/opencode.yml (+0 -29)
📝 .github/workflows/sdk.yml (+9 -4)
📝 .gitignore (+3 -0)
📝 .prettierignore (+3 -0)
➖ AGENTS.md (+0 -1)
➕ AGENTS.md (+70 -0)
📝 CLAUDE.md (+1 -1)
➖ GEMINI.md (+0 -69)
➕ GEMINI.md (+1 -0)
📝 README.md (+19 -7)
📝 apps/browser-extension/manifest.json (+1 -1)
📝 apps/browser-extension/package.json (+7 -5)
📝 apps/browser-extension/src/BookmarkSavedPage.tsx (+14 -1)
➕ apps/browser-extension/src/CustomHeadersPage.tsx (+155 -0)

...and 80 more files

📄 Description

Summary

Replace monolithic inference module with provider-specific clients supporting OpenAI, Anthropic Claude, Google Gemini, and Ollama. Each provider now has native SDK integration with structured output support via JSON schema.

New Providers

Anthropic: Claude 4.5 with structured outputs (beta), vision support
Google Gemini: 2.5/3.x with JSON schema, batch embeddings
OpenAI: Chat Completions + Responses API for GPT-5/o-series reasoning

Architecture Changes

Add packages/shared/inference/ with modular provider clients
Add InferenceClientFactory and EmbeddingClientFactory for provider selection
Separate inference and embedding providers (Anthropic lacks embeddings API)

New Configuration

INFERENCE_PROVIDER - explicit provider selection (auto-detects if unset)
ANTHROPIC_API_KEY, ANTHROPIC_BASE_URL
GEMINI_API_KEY, GEMINI_BASE_URL
EMBEDDING_PROVIDER - separate embedding configuration
OPENAI_USE_RESPONSES_API, OPENAI_REASONING_EFFORT for GPT-5

Breaking Changes

Remove INFERENCE_USE_MAX_COMPLETION_TOKENS (now auto-detected)
Remove INFERENCE_SUPPORTS_STRUCTURED_OUTPUT (use INFERENCE_OUTPUT_SCHEMA)

Other

Add ProviderIndicator component showing active provider/models in settings
Upgrade zod 3.24.2 -> 3.25.0 for Anthropic SDK compatibility
Add 106 unit tests and 19 live API integration tests
Add pnpm inference:live-test script for manual API testing

Test plan

All 106 unit tests pass
All 19 live API integration tests pass (OpenAI, Anthropic, Google)
Typecheck passes (22 packages)
Lint passes (19 packages)

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/karakeep-app/karakeep/pull/2315 **Author:** [@camjac251](https://github.com/camjac251) **Created:** 12/29/2025 **Status:** ❌ Closed **Base:** `main` ← **Head:** `feat/multi-provider-inference` --- ### 📝 Commits (10+) - [`0f5bf36`](https://github.com/karakeep-app/karakeep/commit/0f5bf36b6f1040dbf22b9367934c079f59ebca0d) release(docs): release the 0.27 docs - [`5d33616`](https://github.com/karakeep-app/karakeep/commit/5d3361656972bb6de2961edbd4b2d3f2d0981b50) fix(docs): fix links to point to lowercase docs - [`8129953`](https://github.com/karakeep-app/karakeep/commit/8129953d4b48afba9b011957db7201b52a7092ec) docs: establish cloudflare redirects after the new lowercase docs - [`8570b31`](https://github.com/karakeep-app/karakeep/commit/8570b319c6f62455aca31546163e8aa97aa6d2d5) fix: migrate systemd config to the new worker path on update. fixes #1889 - [`c09d9d4`](https://github.com/karakeep-app/karakeep/commit/c09d9d4936026e560a01b1034db8e18002f7a8d3) fix: fix 5xx on invalid api key - [`d844284`](https://github.com/karakeep-app/karakeep/commit/d844284ac8a53e1eb69ea3f7f209f4ff2e24a7b8) fix: fix migration failing when no user settings are set. fixes #1919 (#1920) - [`2375976`](https://github.com/karakeep-app/karakeep/commit/2375976f37822a114c39df67243a72c6897bf74d) feat(extension): add current tab title while saving from extension (#1930) - [`3747fb0`](https://github.com/karakeep-app/karakeep/commit/3747fb095f41d1cc56d62abb642e9b3662d40f96) fix: Fix ranking of search results. fixes #1922 - [`61435eb`](https://github.com/karakeep-app/karakeep/commit/61435eb9d6191594b59cab82ee0301f7530494f2) feat(search): add title search qualifier (#1940) - [`0e83ad2`](https://github.com/karakeep-app/karakeep/commit/0e83ad2a029666290c09384758ff831ec4c16dab) refactor: strongly type the search plugin interface ### 📊 Changes **977 files changed** (+89019 additions, -16705 deletions) <details> <summary>View changed files</summary> 📝 `.dockerignore` (+3 -2) 📝 `.github/workflows/ci.yml` (+8 -0) 📝 `.github/workflows/claude.yml` (+6 -5) 📝 `.github/workflows/cli.yml` (+9 -4) 📝 `.github/workflows/docker.yml` (+124 -24) 📝 `.github/workflows/mcp.yml` (+9 -4) ➖ `.github/workflows/opencode.yml` (+0 -29) 📝 `.github/workflows/sdk.yml` (+9 -4) 📝 `.gitignore` (+3 -0) 📝 `.prettierignore` (+3 -0) ➖ `AGENTS.md` (+0 -1) ➕ `AGENTS.md` (+70 -0) 📝 `CLAUDE.md` (+1 -1) ➖ `GEMINI.md` (+0 -69) ➕ `GEMINI.md` (+1 -0) 📝 `README.md` (+19 -7) 📝 `apps/browser-extension/manifest.json` (+1 -1) 📝 `apps/browser-extension/package.json` (+7 -5) 📝 `apps/browser-extension/src/BookmarkSavedPage.tsx` (+14 -1) ➕ `apps/browser-extension/src/CustomHeadersPage.tsx` (+155 -0) _...and 80 more files_ </details> ### 📄 Description ## Summary Replace monolithic inference module with provider-specific clients supporting OpenAI, Anthropic Claude, Google Gemini, and Ollama. Each provider now has native SDK integration with structured output support via JSON schema. ### New Providers - **Anthropic**: Claude 4.5 with structured outputs (beta), vision support - **Google Gemini**: 2.5/3.x with JSON schema, batch embeddings - **OpenAI**: Chat Completions + Responses API for GPT-5/o-series reasoning ### Architecture Changes - Add `packages/shared/inference/` with modular provider clients - Add `InferenceClientFactory` and `EmbeddingClientFactory` for provider selection - Separate inference and embedding providers (Anthropic lacks embeddings API) ### New Configuration - `INFERENCE_PROVIDER` - explicit provider selection (auto-detects if unset) - `ANTHROPIC_API_KEY`, `ANTHROPIC_BASE_URL` - `GEMINI_API_KEY`, `GEMINI_BASE_URL` - `EMBEDDING_PROVIDER` - separate embedding configuration - `OPENAI_USE_RESPONSES_API`, `OPENAI_REASONING_EFFORT` for GPT-5 ### Breaking Changes - Remove `INFERENCE_USE_MAX_COMPLETION_TOKENS` (now auto-detected) - Remove `INFERENCE_SUPPORTS_STRUCTURED_OUTPUT` (use `INFERENCE_OUTPUT_SCHEMA`) ### Other - Add `ProviderIndicator` component showing active provider/models in settings - Upgrade zod 3.24.2 -> 3.25.0 for Anthropic SDK compatibility - Add 106 unit tests and 19 live API integration tests - Add `pnpm inference:live-test` script for manual API testing ## Test plan - [x] All 106 unit tests pass - [x] All 19 live API integration tests pass (OpenAI, Anthropic, Google) - [x] Typecheck passes (22 packages) - [x] Lint passes (19 packages) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

kerem

2026-03-02 12:00:25 +03:00

closed this issue
added the
pull-request
label

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/karakeep#2067

No description provided.

Rows
Columns