[PR #140] [CLOSED] Fix completion promise false positives #158

New issue

Closed

opened 2026-02-27 10:22:27 +03:00 by kerem · 0 comments

kerem commented

2026-02-27 10:22:27 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/mikeyobrien/ralph-orchestrator/pull/140
Author: @douglaz
Created: 1/31/2026
Status: ❌ Closed

Base: main ← Head: fix/completion-promise-detection

📝 Commits (2)

d397c34 Fix completion promise false positives
4036544 Clarify and harden completion detection

📊 Changes

2 files changed (+97 additions, -2 deletions)

View changed files

📝 crates/ralph-cli/src/loop_runner.rs (+88 -2)
📝 crates/ralph-core/src/event_parser.rs (+9 -0)

📄 Description

Fixes https://github.com/mikeyobrien/ralph-orchestrator/issues/139

Problem

Some backends (notably Codex) echo the full prompt back in their transcript. The prompt itself includes Ralph instructions that mention the completion promise token (e.g. LOOP_COMPLETE).

Ralph was checking completion by doing a substring match against the raw backend output. That caused false positives:

A normal iteration that emits direction.set could terminate immediately because the echoed prompt contained LOOP_COMPLETE.
--completion-promise "" effectively enabled immediate completion because str::contains("") is always true.

Solution

Treat empty/whitespace-only completion promise as disabled in EventParser::contains_promise.
Strip a single echoed prompt instance from backend output for parsing only, but only when it looks like a transcript echo (at start of output or preceded by a common user marker). This keeps the behavior conservative and avoids stripping legitimate assistant text.

Tests

cargo test -p ralph-core event_parser --quiet
cargo test -p ralph-cli loop_runner --quiet

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/mikeyobrien/ralph-orchestrator/pull/140 **Author:** [@douglaz](https://github.com/douglaz) **Created:** 1/31/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `fix/completion-promise-detection` --- ### 📝 Commits (2) - [`d397c34`](https://github.com/mikeyobrien/ralph-orchestrator/commit/d397c34558bc23ef0ce98763b71dd099d1bb5cf3) Fix completion promise false positives - [`4036544`](https://github.com/mikeyobrien/ralph-orchestrator/commit/403654450fb8adc346c6e4af3ba45dc2c9388e2a) Clarify and harden completion detection ### 📊 Changes **2 files changed** (+97 additions, -2 deletions) <details> <summary>View changed files</summary> 📝 `crates/ralph-cli/src/loop_runner.rs` (+88 -2) 📝 `crates/ralph-core/src/event_parser.rs` (+9 -0) </details> ### 📄 Description Fixes https://github.com/mikeyobrien/ralph-orchestrator/issues/139 ## Problem Some backends (notably Codex) echo the full prompt back in their transcript. The prompt itself includes Ralph instructions that mention the completion promise token (e.g. `LOOP_COMPLETE`). Ralph was checking completion by doing a substring match against the raw backend output. That caused false positives: - A normal iteration that emits `direction.set` could terminate immediately because the echoed prompt contained `LOOP_COMPLETE`. - `--completion-promise ""` effectively enabled immediate completion because `str::contains("")` is always true. ## Solution - Treat empty/whitespace-only completion promise as disabled in `EventParser::contains_promise`. - Strip a single echoed prompt instance from backend output *for parsing only*, but only when it looks like a transcript echo (at start of output or preceded by a common `user` marker). This keeps the behavior conservative and avoids stripping legitimate assistant text. ## Tests - `cargo test -p ralph-core event_parser --quiet` - `cargo test -p ralph-cli loop_runner --quiet` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>