[PR #111] [MERGED] feat: multi-loop concurrency via git worktrees #140

New issue

Closed

opened 2026-02-27 10:22:23 +03:00 by kerem · 0 comments

kerem commented

2026-02-27 10:22:23 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/mikeyobrien/ralph-orchestrator/pull/111
Author: @mikeyobrien
Created: 1/25/2026
Status: ✅ Merged
Merged: 1/25/2026
Merged by: @mikeyobrien

Base: main ← Head: multi-loop-concurrency

📝 Commits (4)

19d56aa feat: add multi-loop concurrency via git worktrees
87f10c9 fix(loops): fix 4 bugs in multi-loop concurrency commands
f0f1426 docs(readme): add multi-loop concurrency documentation
12d53df docs: expand parallel loops documentation comprehensively

📊 Changes

24 files changed (+6578 additions, -34 deletions)

View changed files

📝 AGENTS.md (+64 -0)
📝 Cargo.lock (+1 -0)
📝 Cargo.toml (+1 -1)
📝 README.md (+274 -0)
➕ crates/ralph-cli/presets/merge-loop.yml (+315 -0)
📝 crates/ralph-cli/src/loop_runner.rs (+236 -11)
➕ crates/ralph-cli/src/loops.rs (+775 -0)
📝 crates/ralph-cli/src/main.rs (+153 -2)
📝 crates/ralph-cli/src/presets.rs (+25 -2)
📝 crates/ralph-core/Cargo.toml (+4 -0)
📝 crates/ralph-core/src/event_logger.rs (+17 -0)
📝 crates/ralph-core/src/event_loop/mod.rs (+108 -5)
➕ crates/ralph-core/src/file_lock.rs (+493 -0)
📝 crates/ralph-core/src/lib.rs (+21 -0)
➕ crates/ralph-core/src/loop_completion.rs (+239 -0)
➕ crates/ralph-core/src/loop_context.rs (+471 -0)
➕ crates/ralph-core/src/loop_history.rs (+589 -0)
➕ crates/ralph-core/src/loop_lock.rs (+443 -0)
➕ crates/ralph-core/src/loop_registry.rs (+580 -0)
📝 crates/ralph-core/src/memory_store.rs (+37 -3)

...and 4 more files

📄 Description

Summary

Adds support for running multiple Ralph orchestration loops in parallel using git worktrees for filesystem isolation. This enables working on multiple independent tasks simultaneously without conflicts.

Key Features

Automatic worktree spawning — When a loop starts while another is running, it automatically spawns into .worktrees/<loop-id>/
Lock-based coordination — Primary loop holds .ralph/loop.lock, additional loops detect this and spawn worktrees
Shared memories — .agent/memories.md is symlinked to main repo, enabling cross-loop learning
Isolated state — Events, tasks, and scratchpad are per-loop
Queue-based auto-merge — Completed worktree loops queue for merge; primary loop processes queue on completion
AI-powered conflict resolution — Merge-ralph uses hat collection (merger, resolver, tester, cleaner) to handle merges

New CLI Commands

ralph loops              # List all loops with status
ralph loops logs <id>    # View loop output (--follow for streaming)
ralph loops history <id> # Show event history
ralph loops diff <id>    # Show changes from merge-base
ralph loops attach <id>  # Open shell in worktree
ralph loops retry <id>   # Re-run merge for failed loop
ralph loops stop <id>    # Terminate running loop
ralph loops discard <id> # Abandon and cleanup
ralph loops prune        # Clean up stale loops

New `ralph run` Options

Option	Description
`--exclusive`	Wait for primary loop slot instead of spawning worktree
`--no-auto-merge`	Skip automatic merge after worktree loop completes

Architecture

Primary Loop (holds .ralph/loop.lock)
├── Runs in main workspace
├── Processes merge queue on completion
└── Spawns merge-ralph for queued loops

Worktree Loops (.worktrees/<loop-id>/)
├── Isolated filesystem via git worktree  
├── Symlinked memories → main repo
├── Queue for merge on completion
└── Exit cleanly (no spawn)

Files Changed

Area	Files	Purpose
Core	`loop_lock.rs`, `loop_registry.rs`, `merge_queue.rs`, `worktree.rs`, `loop_context.rs`, `loop_completion.rs`, `loop_history.rs`, `file_lock.rs`	Lock coordination, registry, merge queue, worktree management
CLI	`loops.rs`, `loop_runner.rs`, `main.rs`, `presets.rs`	Loop commands, queue processing, merge preset
Presets	`merge-loop.yml`	Hat collection for AI-powered merge workflow
Docs	`README.md`, `AGENTS.md`	Comprehensive user and developer documentation

Test plan

Manual E2E test: Two parallel loops modifying same file → successful merge
Lock coordination: Second loop spawns to worktree when lock held
Queue-based merge: Worktree queues, primary processes on completion
Conflict resolution: AI resolves conflicts and runs tests
All unit tests pass (cargo test)
CI pipeline validation

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/mikeyobrien/ralph-orchestrator/pull/111 **Author:** [@mikeyobrien](https://github.com/mikeyobrien) **Created:** 1/25/2026 **Status:** ✅ Merged **Merged:** 1/25/2026 **Merged by:** [@mikeyobrien](https://github.com/mikeyobrien) **Base:** `main` ← **Head:** `multi-loop-concurrency` --- ### 📝 Commits (4) - [`19d56aa`](https://github.com/mikeyobrien/ralph-orchestrator/commit/19d56aa0076709c1b19f74839de45e49c118475c) feat: add multi-loop concurrency via git worktrees - [`87f10c9`](https://github.com/mikeyobrien/ralph-orchestrator/commit/87f10c90aa36ddb9e922bc8c52856d74f72d2d22) fix(loops): fix 4 bugs in multi-loop concurrency commands - [`f0f1426`](https://github.com/mikeyobrien/ralph-orchestrator/commit/f0f1426cb457097d1488f138e4b12296641693ee) docs(readme): add multi-loop concurrency documentation - [`12d53df`](https://github.com/mikeyobrien/ralph-orchestrator/commit/12d53df7fef9e3aa99e499f50e1a9b8ab78f9cbc) docs: expand parallel loops documentation comprehensively ### 📊 Changes **24 files changed** (+6578 additions, -34 deletions) <details> <summary>View changed files</summary> 📝 `AGENTS.md` (+64 -0) 📝 `Cargo.lock` (+1 -0) 📝 `Cargo.toml` (+1 -1) 📝 `README.md` (+274 -0) ➕ `crates/ralph-cli/presets/merge-loop.yml` (+315 -0) 📝 `crates/ralph-cli/src/loop_runner.rs` (+236 -11) ➕ `crates/ralph-cli/src/loops.rs` (+775 -0) 📝 `crates/ralph-cli/src/main.rs` (+153 -2) 📝 `crates/ralph-cli/src/presets.rs` (+25 -2) 📝 `crates/ralph-core/Cargo.toml` (+4 -0) 📝 `crates/ralph-core/src/event_logger.rs` (+17 -0) 📝 `crates/ralph-core/src/event_loop/mod.rs` (+108 -5) ➕ `crates/ralph-core/src/file_lock.rs` (+493 -0) 📝 `crates/ralph-core/src/lib.rs` (+21 -0) ➕ `crates/ralph-core/src/loop_completion.rs` (+239 -0) ➕ `crates/ralph-core/src/loop_context.rs` (+471 -0) ➕ `crates/ralph-core/src/loop_history.rs` (+589 -0) ➕ `crates/ralph-core/src/loop_lock.rs` (+443 -0) ➕ `crates/ralph-core/src/loop_registry.rs` (+580 -0) 📝 `crates/ralph-core/src/memory_store.rs` (+37 -3) _...and 4 more files_ </details> ### 📄 Description ## Summary Adds support for running multiple Ralph orchestration loops in parallel using git worktrees for filesystem isolation. This enables working on multiple independent tasks simultaneously without conflicts. ### Key Features - **Automatic worktree spawning** — When a loop starts while another is running, it automatically spawns into `.worktrees/<loop-id>/` - **Lock-based coordination** — Primary loop holds `.ralph/loop.lock`, additional loops detect this and spawn worktrees - **Shared memories** — `.agent/memories.md` is symlinked to main repo, enabling cross-loop learning - **Isolated state** — Events, tasks, and scratchpad are per-loop - **Queue-based auto-merge** — Completed worktree loops queue for merge; primary loop processes queue on completion - **AI-powered conflict resolution** — Merge-ralph uses hat collection (merger, resolver, tester, cleaner) to handle merges ### New CLI Commands ```bash ralph loops # List all loops with status ralph loops logs <id> # View loop output (--follow for streaming) ralph loops history <id> # Show event history ralph loops diff <id> # Show changes from merge-base ralph loops attach <id> # Open shell in worktree ralph loops retry <id> # Re-run merge for failed loop ralph loops stop <id> # Terminate running loop ralph loops discard <id> # Abandon and cleanup ralph loops prune # Clean up stale loops ``` ### New `ralph run` Options | Option | Description | |--------|-------------| | `--exclusive` | Wait for primary loop slot instead of spawning worktree | | `--no-auto-merge` | Skip automatic merge after worktree loop completes | ### Architecture ``` Primary Loop (holds .ralph/loop.lock) ├── Runs in main workspace ├── Processes merge queue on completion └── Spawns merge-ralph for queued loops Worktree Loops (.worktrees/<loop-id>/) ├── Isolated filesystem via git worktree ├── Symlinked memories → main repo ├── Queue for merge on completion └── Exit cleanly (no spawn) ``` ### Files Changed | Area | Files | Purpose | |------|-------|---------| | Core | `loop_lock.rs`, `loop_registry.rs`, `merge_queue.rs`, `worktree.rs`, `loop_context.rs`, `loop_completion.rs`, `loop_history.rs`, `file_lock.rs` | Lock coordination, registry, merge queue, worktree management | | CLI | `loops.rs`, `loop_runner.rs`, `main.rs`, `presets.rs` | Loop commands, queue processing, merge preset | | Presets | `merge-loop.yml` | Hat collection for AI-powered merge workflow | | Docs | `README.md`, `AGENTS.md` | Comprehensive user and developer documentation | ## Test plan - [x] Manual E2E test: Two parallel loops modifying same file → successful merge - [x] Lock coordination: Second loop spawns to worktree when lock held - [x] Queue-based merge: Worktree queues, primary processes on completion - [x] Conflict resolution: AI resolves conflicts and runs tests - [x] All unit tests pass (`cargo test`) - [ ] CI pipeline validation --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>