[PR #421] [MERGED] Add standardized structured routing logs for LLM text requests #723

New issue

Closed

opened 2026-03-13 21:05:53 +03:00 by kerem · 0 comments

kerem commented

2026-03-13 21:05:53 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/AJaySi/ALwrity/pull/421
Author: @AJaySi
Created: 3/12/2026
Status: ✅ Merged
Merged: 3/12/2026
Merged by: @AJaySi

Base: main ← Head: codex/add-structured-logging-to-text-generation

📝 Commits (1)

d01d4af Add standardized structured routing logs for text generation

📊 Changes

3 files changed (+146 additions, -7 deletions)

View changed files

📝 backend/services/llm_providers/huggingface_provider.py (+54 -3)
📝 backend/services/llm_providers/main_text_generation.py (+46 -4)
📝 backend/utils/logger_utils.py (+46 -0)

📄 Description

Motivation

Provide a single, consistent structured logging schema for LLM routing decisions so AI facades (text/image/video/audio) can emit compatible telemetry.
Surface provider/model selection, fallback behavior and a tenant context in logs while avoiding leaking raw user identifiers by hashing them.

Description

Add emit_routing_event helper in backend/utils/logger_utils.py that emits a JSON payload with fields including flow_type/route_intent, provider_selected, model_selected, preferred_provider, fallback_count, fallback_models_tried, and a hashed tenant_user_id (stable SHA256 prefix).
Update llm_text_gen in backend/services/llm_providers/main_text_generation.py to initialize routing state (flow_type, preferred_provider, fallback_count, fallback_models_tried) and call emit_routing_event for the primary selection and when a single fallback is attempted, and to pass tenant_user_id into provider calls.
Update backend/services/llm_providers/huggingface_provider.py to accept tenant_user_id on both huggingface_text_response and huggingface_structured_json_response, emit per-attempt routing events for each candidate model (primary and fallback attempts), and include the retry/no-response_format fallback path in emitted events.
Keep existing logger behavior via get_service_logger and allow emit_routing_event to accept extra payload fields and a log level for flexibility.

Testing

Verified syntax by running python -m py_compile backend/utils/logger_utils.py backend/services/llm_providers/main_text_generation.py backend/services/llm_providers/huggingface_provider.py, which completed successfully with no syntax errors.

Codex Task

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/AJaySi/ALwrity/pull/421 **Author:** [@AJaySi](https://github.com/AJaySi) **Created:** 3/12/2026 **Status:** ✅ Merged **Merged:** 3/12/2026 **Merged by:** [@AJaySi](https://github.com/AJaySi) **Base:** `main` ← **Head:** `codex/add-structured-logging-to-text-generation` --- ### 📝 Commits (1) - [`d01d4af`](https://github.com/AJaySi/ALwrity/commit/d01d4af62fb073dd1998f4d97ad27986c5be6b8b) Add standardized structured routing logs for text generation ### 📊 Changes **3 files changed** (+146 additions, -7 deletions) <details> <summary>View changed files</summary> 📝 `backend/services/llm_providers/huggingface_provider.py` (+54 -3) 📝 `backend/services/llm_providers/main_text_generation.py` (+46 -4) 📝 `backend/utils/logger_utils.py` (+46 -0) </details> ### 📄 Description ### Motivation - Provide a single, consistent structured logging schema for LLM routing decisions so AI facades (text/image/video/audio) can emit compatible telemetry. - Surface provider/model selection, fallback behavior and a tenant context in logs while avoiding leaking raw user identifiers by hashing them. ### Description - Add `emit_routing_event` helper in `backend/utils/logger_utils.py` that emits a JSON payload with fields including `flow_type/route_intent`, `provider_selected`, `model_selected`, `preferred_provider`, `fallback_count`, `fallback_models_tried`, and a hashed `tenant_user_id` (stable SHA256 prefix). - Update `llm_text_gen` in `backend/services/llm_providers/main_text_generation.py` to initialize routing state (`flow_type`, `preferred_provider`, `fallback_count`, `fallback_models_tried`) and call `emit_routing_event` for the primary selection and when a single fallback is attempted, and to pass `tenant_user_id` into provider calls. - Update `backend/services/llm_providers/huggingface_provider.py` to accept `tenant_user_id` on both `huggingface_text_response` and `huggingface_structured_json_response`, emit per-attempt routing events for each candidate model (primary and fallback attempts), and include the retry/no-`response_format` fallback path in emitted events. - Keep existing logger behavior via `get_service_logger` and allow `emit_routing_event` to accept `extra` payload fields and a log `level` for flexibility. ### Testing - Verified syntax by running `python -m py_compile backend/utils/logger_utils.py backend/services/llm_providers/main_text_generation.py backend/services/llm_providers/huggingface_provider.py`, which completed successfully with no syntax errors. ------ [Codex Task](https://chatgpt.com/codex/tasks/task_e_69b0db6e6dec8328ad18f8a7472114b7) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

kerem

2026-03-13 21:05:53 +03:00

closed this issue
added the
pull-request
label

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/ALwrity#723

No description provided.

Rows
Columns