[PR #416] [MERGED] Tenant-aware LLM routing, Hugging Face client refactor, and personalization input hardening #719

New issue

Closed

opened 2026-03-13 21:05:37 +03:00 by kerem · 0 comments

kerem commented

2026-03-13 21:05:37 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/AJaySi/ALwrity/pull/416
Author: @AJaySi
Created: 3/12/2026
Status: ✅ Merged
Merged: 3/12/2026
Merged by: @AJaySi

Base: main ← Head: codex/document-llm-provider-routing-choices

📝 Commits (1)

4b7f443 Fix TEXTGEN_AI_MODELS full-name mapping and unify model resolution

📊 Changes

5 files changed (+435 additions, -375 deletions)

View changed files

📝 backend/services/llm_providers/huggingface_provider.py (+79 -183)
📝 backend/services/llm_providers/main_text_generation.py (+227 -183)
➕ backend/services/llm_providers/routing_observability.py (+22 -0)
➕ backend/services/llm_providers/tenant_provider_config.py (+83 -0)
📝 backend/services/product_marketing/personalization_service.py (+24 -9)

📄 Description

Motivation

Make LLM provider selection tenant-aware and more deterministic by resolving provider/model sequences per-tenant and preferring tenant-scoped API keys.
Simplify and harden the Hugging Face Responses client usage and error handling, improve observability and diagnostic logging.
Improve robustness of personalization data handling to avoid crashes when onboarding payloads have unexpected shapes.

Description

Refactored backend/services/llm_providers/huggingface_provider.py to add _get_hf_client (cached via lru_cache), accept an optional api_key param in get_huggingface_api_key and response functions, centralize error classification via _classify_hf_error and _error_details, remove legacy fallback loops and dotenv path hacks, and improve structured logging.
Reworked backend/services/llm_providers/main_text_generation.py to add tenant-aware routing and model resolution, provider normalization, multi-provider/model attempt loop, strict pinned-provider behavior, tenant API key retrieval, and routing observability events; integrated get_available_text_providers and get_tenant_api_key and updated usage tracking hooks.
Added backend/services/llm_providers/tenant_provider_config.py to resolve tenant-scoped API keys (DB lookup with caching) and to enumerate available providers for a tenant.
Added backend/services/llm_providers/routing_observability.py to emit masked, structured routing events for observability.
Hardened backend/services/product_marketing/personalization_service.py by adding _ensure_dict and _ensure_list helpers and using them to coerce onboarding payloads into stable types before mapping preferences.

Testing

Ran the repository unit test suite focusing on llm_providers and product_marketing modules; tests covering text generation and personalization logic completed and passed.
Performed basic smoke tests of llm_text_gen routing and Hugging Face call paths (mocked keys/clients); routing events and error logging emitted as expected.

Codex Task

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/AJaySi/ALwrity/pull/416 **Author:** [@AJaySi](https://github.com/AJaySi) **Created:** 3/12/2026 **Status:** ✅ Merged **Merged:** 3/12/2026 **Merged by:** [@AJaySi](https://github.com/AJaySi) **Base:** `main` ← **Head:** `codex/document-llm-provider-routing-choices` --- ### 📝 Commits (1) - [`4b7f443`](https://github.com/AJaySi/ALwrity/commit/4b7f4435099c4cb98f32414d7f60023517bfdf87) Fix TEXTGEN_AI_MODELS full-name mapping and unify model resolution ### 📊 Changes **5 files changed** (+435 additions, -375 deletions) <details> <summary>View changed files</summary> 📝 `backend/services/llm_providers/huggingface_provider.py` (+79 -183) 📝 `backend/services/llm_providers/main_text_generation.py` (+227 -183) ➕ `backend/services/llm_providers/routing_observability.py` (+22 -0) ➕ `backend/services/llm_providers/tenant_provider_config.py` (+83 -0) 📝 `backend/services/product_marketing/personalization_service.py` (+24 -9) </details> ### 📄 Description ### Motivation - Make LLM provider selection tenant-aware and more deterministic by resolving provider/model sequences per-tenant and preferring tenant-scoped API keys. - Simplify and harden the Hugging Face Responses client usage and error handling, improve observability and diagnostic logging. - Improve robustness of personalization data handling to avoid crashes when onboarding payloads have unexpected shapes. ### Description - Refactored `backend/services/llm_providers/huggingface_provider.py` to add `_get_hf_client` (cached via `lru_cache`), accept an optional `api_key` param in `get_huggingface_api_key` and response functions, centralize error classification via `_classify_hf_error` and `_error_details`, remove legacy fallback loops and dotenv path hacks, and improve structured logging. - Reworked `backend/services/llm_providers/main_text_generation.py` to add tenant-aware routing and model resolution, provider normalization, multi-provider/model attempt loop, strict pinned-provider behavior, tenant API key retrieval, and routing observability events; integrated `get_available_text_providers` and `get_tenant_api_key` and updated usage tracking hooks. - Added `backend/services/llm_providers/tenant_provider_config.py` to resolve tenant-scoped API keys (DB lookup with caching) and to enumerate available providers for a tenant. - Added `backend/services/llm_providers/routing_observability.py` to emit masked, structured routing events for observability. - Hardened `backend/services/product_marketing/personalization_service.py` by adding `_ensure_dict` and `_ensure_list` helpers and using them to coerce onboarding payloads into stable types before mapping preferences. ### Testing - Ran the repository unit test suite focusing on `llm_providers` and `product_marketing` modules; tests covering text generation and personalization logic completed and passed. - Performed basic smoke tests of `llm_text_gen` routing and Hugging Face call paths (mocked keys/clients); routing events and error logging emitted as expected. ------ [Codex Task](https://chatgpt.com/codex/tasks/task_e_69b0d6e6e64883288f2a282509d39a80) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

kerem

2026-03-13 21:05:37 +03:00

closed this issue
added the
pull-request
label

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/ALwrity#719

No description provided.

Rows
Columns