[PR #200] [MERGED] feat(rpc): subprocess TUI via JSON-RPC stdin/stdout protocol #195

New issue

Closed

opened 2026-02-27 10:22:40 +03:00 by kerem · 0 comments

kerem commented

2026-02-27 10:22:40 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/mikeyobrien/ralph-orchestrator/pull/200
Author: @mikeyobrien
Created: 2/26/2026
Status: ✅ Merged
Merged: 2/26/2026
Merged by: @mikeyobrien

Base: main ← Head: fix/rpc-review-sweep

📝 Commits (7)

8e193f7 feat(proto): add JSON-RPC protocol types for stdin/stdout communication
f94b51e feat(adapters): add JsonRpcStreamHandler for JSON-lines output
1ed626a feat(cli): add RPC stdin command reader and dispatcher
c212d4f feat(cli): wire JSON-RPC mode into orchestration loop
4209377 feat(tui): convert TUI to JSON-RPC client mode
376ee87 feat(cli): make subprocess TUI the default mode
5593036 fix(rpc): harden JSON-RPC protocol layer

📊 Changes

22 files changed (+4933 additions, -96 deletions)

View changed files

📝 Cargo.lock (+21 -0)
📝 Cargo.toml (+2 -0)
➕ crates/ralph-adapters/src/json_rpc_handler.rs (+348 -0)
📝 crates/ralph-adapters/src/lib.rs (+2 -0)
📝 crates/ralph-adapters/src/pi_stream.rs (+20 -4)
📝 crates/ralph-adapters/src/pty_executor.rs (+120 -4)
📝 crates/ralph-adapters/src/stream_handler.rs (+15 -1)
📝 crates/ralph-api/src/runtime/dispatch.rs (+27 -0)
📝 crates/ralph-cli/src/loop_runner.rs (+353 -48)
📝 crates/ralph-cli/src/main.rs (+370 -22)
➕ crates/ralph-cli/src/rpc_stdin.rs (+536 -0)
➕ crates/ralph-proto/src/json_rpc.rs (+1036 -0)
📝 crates/ralph-proto/src/lib.rs (+5 -0)
📝 crates/ralph-tui/Cargo.toml (+7 -0)
📝 crates/ralph-tui/src/app.rs (+55 -6)
📝 crates/ralph-tui/src/lib.rs (+217 -11)
➕ crates/ralph-tui/src/rpc_bridge.rs (+506 -0)
➕ crates/ralph-tui/src/rpc_client.rs (+327 -0)
➕ crates/ralph-tui/src/rpc_source.rs (+499 -0)
➕ crates/ralph-tui/src/rpc_writer.rs (+199 -0)

...and 2 more files

📄 Description

Summary

Decouples the TUI from the orchestration process by introducing a JSON-RPC protocol over stdin/stdout. The TUI now runs as a subprocess that observes and controls the loop through typed JSON messages instead of sharing memory with it.

Motivation

The in-process TUI couples the display layer to the orchestration loop — a panic in rendering code kills the running agent, the TUI can't attach to an already-running loop, and non-terminal frontends (IDE plugins, web dashboards, scripts) have no way to observe or steer execution.

This PR introduces a JSON-lines IPC protocol that makes the TUI a regular client:

Crash isolation — TUI failure no longer takes down the orchestration loop
Attach/detach — frontends can connect to running loops without being part of the process
Frontend flexibility — anything that speaks JSON lines can observe or control Ralph
Remote observation — the TUI can also connect to a ralph-api server over HTTP/WebSocket for remote attach

What's new

Protocol (`ralph-proto`)

json_rpc.rs — Wire format for the IPC layer:

9 command types (stdin → Ralph): Prompt, Guidance, Steer, FollowUp, Abort, GetState, GetIterations, SetHat, ExtensionUiResponse
13 event types (Ralph → stdout): LoopStarted, IterationStart/End, TextDelta, ToolCallStart/End, Error, HatChanged, TaskStatusChanged, TaskCountsUpdated, GuidanceAck, LoopTerminated, Response, OrchestrationEvent
Supporting types for state snapshots, task counts, termination reasons
Full serde roundtrip test coverage for every variant

Event producer (`ralph-adapters`)

JsonRpcStreamHandler — Implements StreamHandler to emit RpcEvent JSON lines on stdout. Tracks tool call durations, iteration context, and token/cost metrics from SessionResult.

Command consumer (`ralph-cli`)

rpc_stdin.rs — RpcDispatcher reads commands from stdin, routes them to the appropriate channels (interrupt, guidance, state queries), and emits correlated responses. Runs as a background tokio task alongside the loop.

loop_runner.rs — Wired with --rpc flag support. Spawns stdin reader + stdout emitter tasks, adds an EventBus observer that maps internal events to RpcEvent::OrchestrationEvent, and maintains shared state (atomics + mutexes) for live GetState responses including hat, completion status, and accumulated cost.

TUI as subprocess client (`ralph-tui`)

rpc_source.rs — Reads JSON-line events from subprocess stdout, translates them into TuiState mutations (iteration buffers, tool call rendering, task counts, completion)
rpc_writer.rs — Writes typed RpcCommand messages to subprocess stdin (guidance, steer, abort, state queries)
rpc_bridge.rs — Alternative path: connects to ralph-api over HTTP/WebSocket for remote TUI attach
rpc_client.rs — HTTP + WebSocket client for the ralph-api RPC v1 protocol
text_renderer.rs — Extracted shared text→Lines rendering (markdown via termimad, ANSI preservation, control character sanitization)
state_mutations.rs — Shared TuiState mutation helpers used by both subprocess and remote modes

CLI (`ralph-cli/main.rs`)

Subprocess TUI is the default mode — ralph run spawns itself as a child with --rpc and wraps it in the TUI
--legacy-tui escape hatch for in-process mode during migration
--rpc flag for headless JSON-lines mode (IDE integrations)
ralph tui subcommand for remote attach to a running ralph-api server
Global args (-c, -H) forwarded to the subprocess

Test plan

All pre-commit hooks pass (fmt, clippy, full test suite)
cargo check --all — zero warnings from project crates
Serde roundtrip tests for all protocol types
Unit tests for dispatcher routing, stream handler output, TUI state mutations, RPC writer serialization
22 files changed, +4,933 / -96

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/mikeyobrien/ralph-orchestrator/pull/200 **Author:** [@mikeyobrien](https://github.com/mikeyobrien) **Created:** 2/26/2026 **Status:** ✅ Merged **Merged:** 2/26/2026 **Merged by:** [@mikeyobrien](https://github.com/mikeyobrien) **Base:** `main` ← **Head:** `fix/rpc-review-sweep` --- ### 📝 Commits (7) - [`8e193f7`](https://github.com/mikeyobrien/ralph-orchestrator/commit/8e193f72603a310a06c1e654427c66d13b737d64) feat(proto): add JSON-RPC protocol types for stdin/stdout communication - [`f94b51e`](https://github.com/mikeyobrien/ralph-orchestrator/commit/f94b51e3f6e385f9bec76279d62c8162b7fdc182) feat(adapters): add JsonRpcStreamHandler for JSON-lines output - [`1ed626a`](https://github.com/mikeyobrien/ralph-orchestrator/commit/1ed626a71e184e837f888b2a2b9501caf3f05e96) feat(cli): add RPC stdin command reader and dispatcher - [`c212d4f`](https://github.com/mikeyobrien/ralph-orchestrator/commit/c212d4f84e6612fc129c618e4245da9544920fbb) feat(cli): wire JSON-RPC mode into orchestration loop - [`4209377`](https://github.com/mikeyobrien/ralph-orchestrator/commit/4209377ded3d02082a69c2328c556e84171e5859) feat(tui): convert TUI to JSON-RPC client mode - [`376ee87`](https://github.com/mikeyobrien/ralph-orchestrator/commit/376ee873d3419b8b43f7f6c1ee8e68f3bdf48dd3) feat(cli): make subprocess TUI the default mode - [`5593036`](https://github.com/mikeyobrien/ralph-orchestrator/commit/55930364f14025de12e8dadf8776926f19b374d3) fix(rpc): harden JSON-RPC protocol layer ### 📊 Changes **22 files changed** (+4933 additions, -96 deletions) <details> <summary>View changed files</summary> 📝 `Cargo.lock` (+21 -0) 📝 `Cargo.toml` (+2 -0) ➕ `crates/ralph-adapters/src/json_rpc_handler.rs` (+348 -0) 📝 `crates/ralph-adapters/src/lib.rs` (+2 -0) 📝 `crates/ralph-adapters/src/pi_stream.rs` (+20 -4) 📝 `crates/ralph-adapters/src/pty_executor.rs` (+120 -4) 📝 `crates/ralph-adapters/src/stream_handler.rs` (+15 -1) 📝 `crates/ralph-api/src/runtime/dispatch.rs` (+27 -0) 📝 `crates/ralph-cli/src/loop_runner.rs` (+353 -48) 📝 `crates/ralph-cli/src/main.rs` (+370 -22) ➕ `crates/ralph-cli/src/rpc_stdin.rs` (+536 -0) ➕ `crates/ralph-proto/src/json_rpc.rs` (+1036 -0) 📝 `crates/ralph-proto/src/lib.rs` (+5 -0) 📝 `crates/ralph-tui/Cargo.toml` (+7 -0) 📝 `crates/ralph-tui/src/app.rs` (+55 -6) 📝 `crates/ralph-tui/src/lib.rs` (+217 -11) ➕ `crates/ralph-tui/src/rpc_bridge.rs` (+506 -0) ➕ `crates/ralph-tui/src/rpc_client.rs` (+327 -0) ➕ `crates/ralph-tui/src/rpc_source.rs` (+499 -0) ➕ `crates/ralph-tui/src/rpc_writer.rs` (+199 -0) _...and 2 more files_ </details> ### 📄 Description ## Summary Decouples the TUI from the orchestration process by introducing a JSON-RPC protocol over stdin/stdout. The TUI now runs as a subprocess that observes and controls the loop through typed JSON messages instead of sharing memory with it. ## Motivation The in-process TUI couples the display layer to the orchestration loop — a panic in rendering code kills the running agent, the TUI can't attach to an already-running loop, and non-terminal frontends (IDE plugins, web dashboards, scripts) have no way to observe or steer execution. This PR introduces a JSON-lines IPC protocol that makes the TUI a regular client: - **Crash isolation** — TUI failure no longer takes down the orchestration loop - **Attach/detach** — frontends can connect to running loops without being part of the process - **Frontend flexibility** — anything that speaks JSON lines can observe or control Ralph - **Remote observation** — the TUI can also connect to a `ralph-api` server over HTTP/WebSocket for remote attach ## What's new ### Protocol (`ralph-proto`) `json_rpc.rs` — Wire format for the IPC layer: - **9 command types** (stdin → Ralph): `Prompt`, `Guidance`, `Steer`, `FollowUp`, `Abort`, `GetState`, `GetIterations`, `SetHat`, `ExtensionUiResponse` - **13 event types** (Ralph → stdout): `LoopStarted`, `IterationStart/End`, `TextDelta`, `ToolCallStart/End`, `Error`, `HatChanged`, `TaskStatusChanged`, `TaskCountsUpdated`, `GuidanceAck`, `LoopTerminated`, `Response`, `OrchestrationEvent` - Supporting types for state snapshots, task counts, termination reasons - Full serde roundtrip test coverage for every variant ### Event producer (`ralph-adapters`) `JsonRpcStreamHandler` — Implements `StreamHandler` to emit `RpcEvent` JSON lines on stdout. Tracks tool call durations, iteration context, and token/cost metrics from `SessionResult`. ### Command consumer (`ralph-cli`) `rpc_stdin.rs` — `RpcDispatcher` reads commands from stdin, routes them to the appropriate channels (interrupt, guidance, state queries), and emits correlated responses. Runs as a background tokio task alongside the loop. `loop_runner.rs` — Wired with `--rpc` flag support. Spawns stdin reader + stdout emitter tasks, adds an `EventBus` observer that maps internal events to `RpcEvent::OrchestrationEvent`, and maintains shared state (atomics + mutexes) for live `GetState` responses including hat, completion status, and accumulated cost. ### TUI as subprocess client (`ralph-tui`) - `rpc_source.rs` — Reads JSON-line events from subprocess stdout, translates them into `TuiState` mutations (iteration buffers, tool call rendering, task counts, completion) - `rpc_writer.rs` — Writes typed `RpcCommand` messages to subprocess stdin (guidance, steer, abort, state queries) - `rpc_bridge.rs` — Alternative path: connects to `ralph-api` over HTTP/WebSocket for remote TUI attach - `rpc_client.rs` — HTTP + WebSocket client for the `ralph-api` RPC v1 protocol - `text_renderer.rs` — Extracted shared text→Lines rendering (markdown via termimad, ANSI preservation, control character sanitization) - `state_mutations.rs` — Shared TuiState mutation helpers used by both subprocess and remote modes ### CLI (`ralph-cli/main.rs`) - Subprocess TUI is the **default mode** — `ralph run` spawns itself as a child with `--rpc` and wraps it in the TUI - `--legacy-tui` escape hatch for in-process mode during migration - `--rpc` flag for headless JSON-lines mode (IDE integrations) - `ralph tui` subcommand for remote attach to a running `ralph-api` server - Global args (`-c`, `-H`) forwarded to the subprocess ## Test plan - All pre-commit hooks pass (fmt, clippy, full test suite) - `cargo check --all` — zero warnings from project crates - Serde roundtrip tests for all protocol types - Unit tests for dispatcher routing, stream handler output, TUI state mutations, RPC writer serialization - 22 files changed, +4,933 / -96 --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>