[PR #76] Fix false positive billing error detection in agent messages #103

New issue

Open

opened 2026-02-27 07:20:26 +03:00 by kerem · 0 comments

kerem commented

2026-02-27 07:20:26 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/KeygraphHQ/shannon/pull/76
Author: @moscowchill
Created: 2/7/2026
Status: 🔄 Open

Base: main ← Head: fix/billing-detection-false-positive

📝 Commits (1)

750dea9 Fix false positive billing error detection in agent messages

📊 Changes

1 file changed (+18 additions, -1 deletions)

View changed files

📝 src/ai/message-handlers.ts (+18 -1)

📄 Description

Summary

detectApiError() was checking full assistant message content including serialized tool_use JSON
When agents wrote security reports containing terms like "usage limit" or "rate limit", the billing error patterns matched on report content inside Write tool calls, killing the agent with a false BillingError
Added extractTextOnlyContent() that filters out tool_use blocks so only the agent's actual text responses are checked for billing patterns

Test plan

Verified agent no longer crashes with false billing error when writing security reports containing "limit"/"cap" keywords
Confirmed pipeline progresses through Phase 3 report writing without false positive
Unit test for extractTextOnlyContent filtering tool_use blocks

🤖 Generated with Claude Code

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/KeygraphHQ/shannon/pull/76 **Author:** [@moscowchill](https://github.com/moscowchill) **Created:** 2/7/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `fix/billing-detection-false-positive` --- ### 📝 Commits (1) - [`750dea9`](https://github.com/KeygraphHQ/shannon/commit/750dea9ff877470d7b52fe7d469ed2db62873580) Fix false positive billing error detection in agent messages ### 📊 Changes **1 file changed** (+18 additions, -1 deletions) <details> <summary>View changed files</summary> 📝 `src/ai/message-handlers.ts` (+18 -1) </details> ### 📄 Description ## Summary - `detectApiError()` was checking full assistant message content including serialized tool_use JSON - When agents wrote security reports containing terms like "usage limit" or "rate limit", the billing error patterns matched on report content inside Write tool calls, killing the agent with a false `BillingError` - Added `extractTextOnlyContent()` that filters out tool_use blocks so only the agent's actual text responses are checked for billing patterns ## Test plan - [x] Verified agent no longer crashes with false billing error when writing security reports containing "limit"/"cap" keywords - [x] Confirmed pipeline progresses through Phase 3 report writing without false positive - [x] Unit test for `extractTextOnlyContent` filtering tool_use blocks 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>