[PR #128] [CLOSED] feat: add Skill Forge — on-demand skill profiling, analysis, and opti… #133

New issue

Closed

opened 2026-02-27 08:09:24 +03:00 by kerem · 0 comments

kerem commented

2026-02-27 08:09:24 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/KeygraphHQ/shannon/pull/128
Author: @Hrishikesh332
Created: 2/12/2026
Status: ❌ Closed

Base: main ← Head: feat/skill-forge

📝 Commits (1)

8bc0a6d feat: add Skill Forge — on-demand skill profiling, analysis, and optimization

📊 Changes

15 files changed (+3030 additions, -2 deletions)

View changed files

➕ .cursor/skills/optimize-all-skills/SKILL.md (+63 -0)
➕ configs/forge-config.yaml (+63 -0)
➕ mcp-server/src/forge/analyzer.ts (+273 -0)
➕ mcp-server/src/forge/core.ts (+318 -0)
➕ mcp-server/src/forge/db.ts (+258 -0)
➕ mcp-server/src/forge/optimizer.ts (+248 -0)
➕ mcp-server/src/forge/profiler.ts (+271 -0)
➕ mcp-server/src/forge/types.ts (+227 -0)
➕ mcp-server/src/forge/validator.ts (+336 -0)
➕ mcp-server/src/forge/versioner.ts (+259 -0)
📝 mcp-server/src/index.ts (+17 -1)
➕ mcp-server/src/tools/forge-optimize.ts (+92 -0)
➕ mcp-server/src/tools/forge-status.ts (+87 -0)
➕ mcp-server/src/tools/optimize-skills.ts (+434 -0)
📝 mcp-server/src/types/tool-responses.ts (+84 -1)

📄 Description

…mization

Introduces the Skill Forge system: an on-demand optimization pipeline that profiles MCP tool and agent phase executions, detects slow/failing/token-heavy skills via configurable thresholds, generates optimized versions, validates them via structural + semantic A/B comparison, and manages version history with rollback support.

New modules (mcp-server/src/forge/):

types.ts: shared interfaces across all forge modules
db.ts: JSON-file persistent store for execution_log, skill_stats, versions
profiler.ts: profileTool() wrapper, logAgentPhaseExecution(), ToolCallTracker
analyzer.ts: 6 configurable flagging rules (slow, high_failure, token_heavy, cost_outlier, degrading_trend, high token ratio)
optimizer.ts: heuristic prompt optimizer + subagent prompt builder for v2
validator.ts: A/B comparison (JSON structural, markdown heading, semantic)
versioner.ts: checkpoint, promote, rollback, prune with SHA-256 hashing
core.ts: orchestrator — single-invocation pipeline, no scheduling

New MCP tools:

forge_optimize: triggers full PROFILE→ANALYZE→GENERATE→VALIDATE→PROMOTE cycle
forge_status: returns profiler stats, candidates, version history
optimize_skills: static SKILL.md quality checker against create-skill criteria

Config: configs/forge-config.yaml with all thresholds, DB path, A/B settings.
Gateway skill: .cursor/skills/optimize-all-skills/SKILL.md

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/KeygraphHQ/shannon/pull/128 **Author:** [@Hrishikesh332](https://github.com/Hrishikesh332) **Created:** 2/12/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `feat/skill-forge` --- ### 📝 Commits (1) - [`8bc0a6d`](https://github.com/KeygraphHQ/shannon/commit/8bc0a6d5992ffa8946e636f4544d06f049065a22) feat: add Skill Forge — on-demand skill profiling, analysis, and optimization ### 📊 Changes **15 files changed** (+3030 additions, -2 deletions) <details> <summary>View changed files</summary> ➕ `.cursor/skills/optimize-all-skills/SKILL.md` (+63 -0) ➕ `configs/forge-config.yaml` (+63 -0) ➕ `mcp-server/src/forge/analyzer.ts` (+273 -0) ➕ `mcp-server/src/forge/core.ts` (+318 -0) ➕ `mcp-server/src/forge/db.ts` (+258 -0) ➕ `mcp-server/src/forge/optimizer.ts` (+248 -0) ➕ `mcp-server/src/forge/profiler.ts` (+271 -0) ➕ `mcp-server/src/forge/types.ts` (+227 -0) ➕ `mcp-server/src/forge/validator.ts` (+336 -0) ➕ `mcp-server/src/forge/versioner.ts` (+259 -0) 📝 `mcp-server/src/index.ts` (+17 -1) ➕ `mcp-server/src/tools/forge-optimize.ts` (+92 -0) ➕ `mcp-server/src/tools/forge-status.ts` (+87 -0) ➕ `mcp-server/src/tools/optimize-skills.ts` (+434 -0) 📝 `mcp-server/src/types/tool-responses.ts` (+84 -1) </details> ### 📄 Description …mization Introduces the Skill Forge system: an on-demand optimization pipeline that profiles MCP tool and agent phase executions, detects slow/failing/token-heavy skills via configurable thresholds, generates optimized versions, validates them via structural + semantic A/B comparison, and manages version history with rollback support. New modules (mcp-server/src/forge/): - types.ts: shared interfaces across all forge modules - db.ts: JSON-file persistent store for execution_log, skill_stats, versions - profiler.ts: profileTool() wrapper, logAgentPhaseExecution(), ToolCallTracker - analyzer.ts: 6 configurable flagging rules (slow, high_failure, token_heavy, cost_outlier, degrading_trend, high token ratio) - optimizer.ts: heuristic prompt optimizer + subagent prompt builder for v2 - validator.ts: A/B comparison (JSON structural, markdown heading, semantic) - versioner.ts: checkpoint, promote, rollback, prune with SHA-256 hashing - core.ts: orchestrator — single-invocation pipeline, no scheduling New MCP tools: - forge_optimize: triggers full PROFILE→ANALYZE→GENERATE→VALIDATE→PROMOTE cycle - forge_status: returns profiler stats, candidates, version history - optimize_skills: static SKILL.md quality checker against create-skill criteria Config: configs/forge-config.yaml with all thresholds, DB path, A/B settings. Gateway skill: .cursor/skills/optimize-all-skills/SKILL.md --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>