[PR #391] [MERGED] Add provenance-based plan quality metrics and classification to Today Workflow #695

New issue

Closed

opened 2026-03-13 21:04:16 +03:00 by kerem · 0 comments

kerem commented

2026-03-13 21:04:16 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/AJaySi/ALwrity/pull/391
Author: @AJaySi
Created: 3/6/2026
Status: ✅ Merged
Merged: 3/8/2026
Merged by: @AJaySi

Base: main ← Head: codex/add-second-quality-dimension-to-plans

📝 Commits (1)

84babd0 Add workflow provenance quality metrics and classification

📊 Changes

2 files changed (+97 additions, -1 deletions)

View changed files

📝 backend/api/today_workflow.py (+4 -0)
📝 backend/services/today_workflow_service.py (+93 -1)

📄 Description

Motivation

Provide a second quality dimension that quantifies how much of a generated daily plan originated from autonomous agents vs LLM/backfill/fallback so the frontend can avoid overclaiming AI personalization when fallback dominates.
Persist and surface these provenance-aware metrics on plan creation and in the /api/today-workflow response to enable transparency and threshold-based classification.

Description

Introduced provenance constants and normalization in today_workflow_service.py and ensure each task's metadata.provenance is one of agent_proposal, llm_backfill, or controlled_fallback via _sanitize_task and backfill paths.
Added _compute_plan_quality to aggregate metrics (agent-origin ratio/percent, agent-origin task count, number of pillars with agent-origin tasks, fallback ratio/count/percent, total task count) and classification logic that uses workflow_config.min_agent_origin_ratio with a default of 0.35 to return either AI-personalized or guided baseline.
Marked backfilled coverage tasks and controlled fallbacks with provenance and attach a quality block onto generated plan payloads (generate_agent_enhanced_plan) so plan_json persisted on creation contains the quality metrics.
Backfilled existing plans missing a plan_json.quality in get_or_create_daily_workflow_plan to avoid API inconsistencies.
Exposed the persisted metrics in the API by adding data.plan.quality to the /api/today-workflow response (reads plan.plan_json.quality).

Testing

Compiled the modified files with python -m py_compile backend/services/today_workflow_service.py backend/api/today_workflow.py and the check succeeded.
Verified changes were staged and committed (git commit) for the two modified files: backend/services/today_workflow_service.py and backend/api/today_workflow.py.

Codex Task

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/AJaySi/ALwrity/pull/391 **Author:** [@AJaySi](https://github.com/AJaySi) **Created:** 3/6/2026 **Status:** ✅ Merged **Merged:** 3/8/2026 **Merged by:** [@AJaySi](https://github.com/AJaySi) **Base:** `main` ← **Head:** `codex/add-second-quality-dimension-to-plans` --- ### 📝 Commits (1) - [`84babd0`](https://github.com/AJaySi/ALwrity/commit/84babd0407ea907f67bc92790f57ddcbebacdd29) Add workflow provenance quality metrics and classification ### 📊 Changes **2 files changed** (+97 additions, -1 deletions) <details> <summary>View changed files</summary> 📝 `backend/api/today_workflow.py` (+4 -0) 📝 `backend/services/today_workflow_service.py` (+93 -1) </details> ### 📄 Description ### Motivation - Provide a second quality dimension that quantifies how much of a generated daily plan originated from autonomous agents vs LLM/backfill/fallback so the frontend can avoid overclaiming AI personalization when fallback dominates. - Persist and surface these provenance-aware metrics on plan creation and in the `/api/today-workflow` response to enable transparency and threshold-based classification. ### Description - Introduced provenance constants and normalization in `today_workflow_service.py` and ensure each task's `metadata.provenance` is one of `agent_proposal`, `llm_backfill`, or `controlled_fallback` via `_sanitize_task` and backfill paths. - Added `_compute_plan_quality` to aggregate metrics (agent-origin ratio/percent, agent-origin task count, number of pillars with agent-origin tasks, fallback ratio/count/percent, total task count) and classification logic that uses `workflow_config.min_agent_origin_ratio` with a default of `0.35` to return either `AI-personalized` or `guided baseline`. - Marked backfilled coverage tasks and controlled fallbacks with provenance and attach a `quality` block onto generated plan payloads (`generate_agent_enhanced_plan`) so `plan_json` persisted on creation contains the quality metrics. - Backfilled existing plans missing a `plan_json.quality` in `get_or_create_daily_workflow_plan` to avoid API inconsistencies. - Exposed the persisted metrics in the API by adding `data.plan.quality` to the `/api/today-workflow` response (reads `plan.plan_json.quality`). ### Testing - Compiled the modified files with `python -m py_compile backend/services/today_workflow_service.py backend/api/today_workflow.py` and the check succeeded. - Verified changes were staged and committed (`git commit`) for the two modified files: `backend/services/today_workflow_service.py` and `backend/api/today_workflow.py`. ------ [Codex Task](https://chatgpt.com/codex/tasks/task_e_69aaf8d4b3b48328bece52fac8f87eaf) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

kerem

2026-03-13 21:04:16 +03:00

closed this issue
added the
pull-request
label

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/ALwrity#695

No description provided.

Rows
Columns