[PR #6] [MERGED] feat: add completion marker detection and loop prevention #84

New issue

Closed

opened 2026-02-27 10:22:10 +03:00 by kerem · 0 comments

kerem commented

2026-02-27 10:22:10 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/mikeyobrien/ralph-orchestrator/pull/6
Author: @terrylica
Created: 12/16/2025
Status: ✅ Merged
Merged: 12/16/2025
Merged by: @mikeyobrien

Base: main ← Head: feat/completion-detection-loop-prevention

📝 Commits (2)

1aad2e7 feat: add completion marker detection and loop prevention
a3d4c9d docs: add ASCII architecture diagrams and holistic documentation updates

📊 Changes

16 files changed (+1356 additions, -76 deletions)

View changed files

➕ docs/advanced/loop-detection.md (+282 -0)
📝 docs/advanced/monitoring.md (+61 -29)
📝 docs/advanced/security.md (+98 -14)
📝 docs/changelog.md (+41 -1)
📝 docs/faq.md (+66 -4)
📝 docs/glossary.md (+7 -1)
📝 docs/guide/overview.md (+65 -12)
📝 docs/quick-start.md (+28 -2)
📝 docs/troubleshooting.md (+183 -9)
📝 mkdocs.yml (+1 -0)
📝 pyproject.toml (+1 -0)
📝 src/ralph_orchestrator/orchestrator.py (+39 -3)
📝 src/ralph_orchestrator/safety.py (+42 -1)
➕ tests/test_completion_detection.py (+146 -0)
➕ tests/test_loop_detection.py (+204 -0)
📝 uv.lock (+92 -0)

📄 Description

Summary

This PR re-enables task completion detection and adds loop prevention to improve orchestration reliability.

Background

Completion marker detection was removed in commit f0c0ce48 during the .ralph/ → .agent/ directory standardization refactor. This caused the orchestrator to run until hard limits (iterations/runtime/cost) instead of stopping when tasks were marked complete.

Changes

Completion Marker Detection

Re-enables - [x] TASK_COMPLETE checkbox marker detection
Orchestrator checks prompt file before each iteration
Supports both - [x] TASK_COMPLETE and [x] TASK_COMPLETE formats
Immediately exits loop when marker is found

Loop Detection (new feature)

Uses rapidfuzz for fast fuzzy string matching
Compares current output against last 5 outputs (sliding window)
Uses 90% similarity threshold to detect repetitive patterns
Prevents infinite loops from runaway agents
Gracefully skips if rapidfuzz is not installed

Documentation

New advanced guide: docs/advanced/loop-detection.md
Updated changelog with new features
Updated glossary with "Completion Marker" and "Loop Detection" entries

Tests

23 new tests covering both features
Tests for edge cases: empty files, whitespace, case sensitivity, window behavior

Files Changed

File	Change
`pyproject.toml`	Added rapidfuzz dependency
`src/ralph_orchestrator/orchestrator.py`	Completion marker check + loop detection integration
`src/ralph_orchestrator/safety.py`	Loop detection method + reset() update
`docs/changelog.md`	New feature entries
`docs/glossary.md`	New terminology
`docs/advanced/loop-detection.md`	New guide
`tests/test_completion_detection.py`	9 tests
`tests/test_loop_detection.py`	14 tests

Test Plan

All 23 new tests pass
962 existing tests pass (3 pre-existing failures unrelated to this PR)
Completion marker correctly stops orchestration
Loop detection triggers at 90% similarity
Window correctly tracks last 5 outputs
Reset clears loop history

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/mikeyobrien/ralph-orchestrator/pull/6 **Author:** [@terrylica](https://github.com/terrylica) **Created:** 12/16/2025 **Status:** ✅ Merged **Merged:** 12/16/2025 **Merged by:** [@mikeyobrien](https://github.com/mikeyobrien) **Base:** `main` ← **Head:** `feat/completion-detection-loop-prevention` --- ### 📝 Commits (2) - [`1aad2e7`](https://github.com/mikeyobrien/ralph-orchestrator/commit/1aad2e7c56b23e2eae3dcbede9741c404f5f82ad) feat: add completion marker detection and loop prevention - [`a3d4c9d`](https://github.com/mikeyobrien/ralph-orchestrator/commit/a3d4c9ddcbdc05bce58a51d709584a919799985f) docs: add ASCII architecture diagrams and holistic documentation updates ### 📊 Changes **16 files changed** (+1356 additions, -76 deletions) <details> <summary>View changed files</summary> ➕ `docs/advanced/loop-detection.md` (+282 -0) 📝 `docs/advanced/monitoring.md` (+61 -29) 📝 `docs/advanced/security.md` (+98 -14) 📝 `docs/changelog.md` (+41 -1) 📝 `docs/faq.md` (+66 -4) 📝 `docs/glossary.md` (+7 -1) 📝 `docs/guide/overview.md` (+65 -12) 📝 `docs/quick-start.md` (+28 -2) 📝 `docs/troubleshooting.md` (+183 -9) 📝 `mkdocs.yml` (+1 -0) 📝 `pyproject.toml` (+1 -0) 📝 `src/ralph_orchestrator/orchestrator.py` (+39 -3) 📝 `src/ralph_orchestrator/safety.py` (+42 -1) ➕ `tests/test_completion_detection.py` (+146 -0) ➕ `tests/test_loop_detection.py` (+204 -0) 📝 `uv.lock` (+92 -0) </details> ### 📄 Description ## Summary This PR re-enables task completion detection and adds loop prevention to improve orchestration reliability. ### Background Completion marker detection was [removed in commit f0c0ce48](https://github.com/mikeyobrien/ralph-orchestrator/commit/f0c0ce48) during the `.ralph/` → `.agent/` directory standardization refactor. This caused the orchestrator to run until hard limits (iterations/runtime/cost) instead of stopping when tasks were marked complete. ### Changes **Completion Marker Detection** - Re-enables `- [x] TASK_COMPLETE` checkbox marker detection - Orchestrator checks prompt file before each iteration - Supports both `- [x] TASK_COMPLETE` and `[x] TASK_COMPLETE` formats - Immediately exits loop when marker is found **Loop Detection** (new feature) - Uses [rapidfuzz](https://pypi.org/project/RapidFuzz/) for fast fuzzy string matching - Compares current output against last 5 outputs (sliding window) - Uses 90% similarity threshold to detect repetitive patterns - Prevents infinite loops from runaway agents - Gracefully skips if rapidfuzz is not installed **Documentation** - New advanced guide: `docs/advanced/loop-detection.md` - Updated changelog with new features - Updated glossary with "Completion Marker" and "Loop Detection" entries **Tests** - 23 new tests covering both features - Tests for edge cases: empty files, whitespace, case sensitivity, window behavior ### Files Changed | File | Change | |------|--------| | `pyproject.toml` | Added rapidfuzz dependency | | `src/ralph_orchestrator/orchestrator.py` | Completion marker check + loop detection integration | | `src/ralph_orchestrator/safety.py` | Loop detection method + reset() update | | `docs/changelog.md` | New feature entries | | `docs/glossary.md` | New terminology | | `docs/advanced/loop-detection.md` | New guide | | `tests/test_completion_detection.py` | 9 tests | | `tests/test_loop_detection.py` | 14 tests | ## Test Plan - [x] All 23 new tests pass - [x] 962 existing tests pass (3 pre-existing failures unrelated to this PR) - [x] Completion marker correctly stops orchestration - [x] Loop detection triggers at 90% similarity - [x] Window correctly tracks last 5 outputs - [x] Reset clears loop history --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>