[GH-ISSUE #123] Loop done when tasks are incomplete

kerem commented

2026-02-27 10:21:56 +03:00

Owner

Originally created by @nartc on GitHub (Jan 27, 2026).
Original GitHub issue: https://github.com/mikeyobrien/ralph-orchestrator/issues/123

Hi, thank you for this first of all. I've been using last week and it's been really great.

However, sometimes I notice that the loop looks like it's done (i.e: saying LOOP COMPLETED) but it only completed like 3-4 tasks out of a phase with 15-16 tasks. Rerun ralph run -P path/to/PROMPT.md repeatedly makes it works on more tasks but it never fully completes everything. Some phases it does complete everything, but some phases it doesn't.

My flow is:

Plan normally with plan mode
Finalize everything into a PRD.md
ralph task PRD.md to break it into tasks -> PROMPT.md
ralph run -P path/to/PROMPT.md

The consequence, when the issue above happens, is that it destroys my claude limit and I'm just guessing here, probably due to it re-reads the PROMPT, process all of the tasks files, and checking / generating specs for those tasks.

Any idea what could go wrong here? Any suggestions?

Originally created by @nartc on GitHub (Jan 27, 2026). Original GitHub issue: https://github.com/mikeyobrien/ralph-orchestrator/issues/123 Hi, thank you for this first of all. I've been using last week and it's been really great. However, sometimes I notice that the loop looks like it's done (i.e: saying LOOP COMPLETED) but it only completed like 3-4 tasks out of a phase with 15-16 tasks. Rerun `ralph run -P path/to/PROMPT.md` repeatedly makes it works on more tasks but it never fully completes everything. Some phases it does complete everything, but some phases it doesn't. My flow is: - Plan normally with plan mode - Finalize everything into a `PRD.md` - `ralph task PRD.md` to break it into tasks -> `PROMPT.md` - `ralph run -P path/to/PROMPT.md` The consequence, when the issue above happens, is that it destroys my claude limit and I'm just guessing here, probably due to it re-reads the PROMPT, process all of the tasks files, and checking / generating specs for those tasks. Any idea what could go wrong here? Any suggestions?

kerem closed this issue

2026-02-27 10:21:56 +03:00

kerem commented

2026-02-27 10:21:57 +03:00

Author

Owner

@nartc commented on GitHub (Jan 27, 2026):

Here's an example of PROMPT.md

and after it says "LOOP COMPLETED", many tasks file have: status: pending still

@nartc commented on GitHub (Jan 27, 2026): Here's an example of PROMPT.md <img width="857" height="1139" alt="Image" src="https://github.com/user-attachments/assets/9f91482a-d902-4c95-aa15-95b67044a84e" /> and after it says "LOOP COMPLETED", many tasks file have: `status: pending` still <img width="513" height="201" alt="Image" src="https://github.com/user-attachments/assets/07328128-d22c-4fb3-9844-a27745df5a1f" />

kerem commented

2026-02-27 10:21:57 +03:00

Author

Owner

@mikeyobrien commented on GitHub (Jan 27, 2026):

Are you using a hat collection? If not I would try https://github.com/mikeyobrien/ralph-orchestrator/blob/main/presets/confession-loop.yml

I've noticed this occasionally happening too, working on prompt tuning to minimize this as much as possible.

I'd try skipping the ralph task step and prompting with ralph run -p "Implement the design in <path-to-file>" -c builtin:confession-loop

@mikeyobrien commented on GitHub (Jan 27, 2026): Are you using a hat collection? If not I would try https://github.com/mikeyobrien/ralph-orchestrator/blob/main/presets/confession-loop.yml I've noticed this occasionally happening too, working on prompt tuning to minimize this as much as possible. I'd try skipping the `ralph task` step and prompting with `ralph run -p "Implement the design in <path-to-file>" -c builtin:confession-loop`

kerem commented

2026-02-27 10:21:57 +03:00

Author

Owner

@nartc commented on GitHub (Jan 27, 2026):

I think I'm using hats with ralph init --preset spec-driven --agent claude. I'll look at confession loop

@nartc commented on GitHub (Jan 27, 2026): I _think_ I'm using hats with `ralph init --preset spec-driven --agent claude`. I'll look at confession loop

kerem commented

2026-02-27 10:21:57 +03:00

Author

Owner

@mikeyobrien commented on GitHub (Jan 27, 2026):

Models have gotten a lot smarter and with that I've notice how even minor changes in a prompt can have unintended consequences...

It's going to take help from the entire community to get a set of solid hat collections and workflows to build on top of. The current presets are examples of what is technically possible but YMMV.

@mikeyobrien commented on GitHub (Jan 27, 2026): Models have gotten a lot smarter and with that I've notice how even minor changes in a prompt can have unintended consequences... It's going to take help from the entire community to get a set of solid hat collections and workflows to build on top of. The current presets are examples of what is technically possible but YMMV.

kerem commented

2026-02-27 10:21:57 +03:00

Author

Owner

@nartc commented on GitHub (Jan 27, 2026):

Yeah makes sense. I noticed that this only starts happening when I started polluting the tasks/ directory with previous phase' tasks files.

tasks/
  web-ui/
    PROMPT.md
    task-01....md
    ...
  enhancement/
    PROMPT.md
    task-01....md
    ...
  ...

@nartc commented on GitHub (Jan 27, 2026): Yeah makes sense. I noticed that this only starts happening when I started polluting the `tasks/` directory with previous phase' tasks files. ``` tasks/ web-ui/ PROMPT.md task-01....md ... enhancement/ PROMPT.md task-01....md ... ... ```

kerem commented

2026-02-27 10:21:57 +03:00

Author

Owner

@mikeyobrien commented on GitHub (Jan 27, 2026):

My hunch is the first task decomposition might not include the entire set of tasks necessary to complete the overall objective.

Might need to add more explicit instructions in the meta loop to empower the agent to add any missing tasks it comes across proactively.

@mikeyobrien commented on GitHub (Jan 27, 2026): My hunch is the first task decomposition might not include the entire set of tasks necessary to complete the overall objective. Might need to add more explicit instructions in the meta loop to empower the agent to add any missing tasks it comes across proactively.

kerem commented

2026-02-27 10:21:57 +03:00

Author

Owner

@Skinner927 commented on GitHub (Jan 28, 2026):

I'm also having this issue with v2.2.5.

I did not ever have this issue on v2.1.33.

Here's what I've noticed when running ralph run:

.agent/memories.md is always missing (though I don't see it in old ralph either)
.agent/scratchpad.md is never created or written to.
.agent/tasks.jsonl is missing, but .agent/tasks.jsonl.lock is present.
.ralph/history.jsonl is missing, but .ralph/history.jsonl.lock is present.

Here is my ralph.yml showing my hats. It's a modified version of the spec-driven preset.
I have truncated the instructions and left only the lines that instruct how to emit events.
These were always at the end of every instruction.

ralph.yml (click to expand)

# Specification-First Development
cli:
  backend: "claude"

event_loop:
  starting_event: "spec.start"  # Ralph publishes this after coordination
  max_iterations: 999                   # Maximum orchestration loops
  max_runtime_seconds: 57600            # 16 hours max runtime

hats:
  spec_writer:
    name: "📋 Spec Writer"
    description: "Creates precise, unambiguous specifications with examples."
    triggers: ["spec.start", "spec.rejected"]
    publishes: ["spec.ready"]
    instructions: |
      ...

      Publish spec.ready when complete.

  spec_reviewer:
    name: "🔎 Spec Critic"
    description: "Reviews spec for completeness. Could someone implement from this?"
    triggers: ["spec.ready"]
    publishes: ["spec.approved", "spec.rejected"]
    instructions: |
      ...

      If issues found: publish spec.rejected with specific feedback
      If solid: publish spec.approved

  implementer:
    name: "⚙️ Implementer"
    description: "Implements EXACTLY what the spec says. No creative interpretation."
    triggers: ["spec.approved", "spec.violated"]
    publishes: ["implementation.done"]
    instructions: |
      ...

      Publish implementation.done when complete.

  test_verifier:
    name: "🧪 Test Verifier"
    description: Verifies the tests properly test the new features
    triggers: ["implementation.done"]
    publishes: ["tests.verified", "spec.violated"]
    default_publishes: "tests.verified"
    instructions: |
      ...

      If all checks and tests pass: publish tests.verified
      If violations or errors: publish spec.violated with details

  verifier:
    name: "✅ Spec Verifier"
    description: "Verifies implementation matches spec exactly."
    triggers: ["tests.verified"]
    publishes: ["spec.complete", "spec.violated"]
    default_publishes: "spec.complete"
    instructions: |
      ...

      If all pass: publish spec.complete
      If violations: publish spec.violated with details

  design_updater:
    name: "🧭️ Design Updater"
    description: "Updates the design plan with the latest implementation details"
    triggers: ["spec.complete"]
    publishes: ["design.updated"]
    default_publishes: "design.updated"
    instructions: |
      ...

      Publish design.updated when complete.

  committer:
    name: "💾️ Git Committer"
    description: "Commits Changes"
    triggers: ["design.updated"]
    publishes: ["task.complete"]
    default_publishes: "task.complete"
    instructions: |
      ...

      When you are done, output LOOP_COMPLETE

Here's the summary of what I can see happened.

Iteration 16 worked as expected and emitted tests.verified
Iteration 17 correctly started as the verifier (spec verifier) handling the tests.verified event.
- but in the middle switched to Ralph Coordinator and re-emitted the tests.verified event.
- You can see it made up a false flow tests.verified → Ralph → SpecVerifier.
- It then realized the error of its ways and put on the Spec Verifier hat.
- It then finished as Spec Verifier as it should and emitted spec.complete.
Iteration 18 now sees two events: spec.complete and tests.verified.
- So the coordinator in 18 emitted the event before it realized it messed up.
- It gets really confused by the 2 events, claims 3 times it should delegate to the Design Updater, gives up and just screams LOOP_COMPLETE, something neither the test_verifier or verifier should have been able to do .

console output (click to expand)

===============================================================================
 ITERATION 16 | ? test_verifier | 31m 57s elapsed | 16/999
===============================================================================
2026-01-28T17:45:27.625833Z  INFO ralph_core::event_loop: Memory injection check: enabled=true, inject=Auto, workspace_root="/home/user/src/project"
2026-01-28T17:45:27.625850Z  INFO ralph_core::event_loop: Looking for memories at: "/home/user/src/project/.agent/memories.md" (exists: false)
2026-01-28T17:45:27.625854Z  INFO ralph_core::event_loop: Successfully loaded 0 memories from store
2026-01-28T17:45:27.625855Z  INFO ralph_core::event_loop: Memory store is empty - no memories to inject
⚙ [Read] /home/user/src/project/specs/step5-rbac-spec.md
⚙ [Bash] ralph tools task list
I'm handling the implementation.done event for Step 5 RBAC. According to the workflow, after implementation.done, I need to trigger the Test Verifier hat to verify tests. Let me emit the tests.verified request to invoke that verification process.

First, let me run the tests and check coverage to verify that the implementation tests are high quality.

[... it does its thing ...]

All criteria pass. Emitting tests.verified.
⚙ [Bash] ralph emit "tests.verified" "All 20 ACs covered with unit an...
Tests verified. The RBAC implementation has comprehensive test coverage with all 20 acceptance criteria tested through both unit tests and API integration tests. Coverage is 82.4% for the rbac package, exceeding the 80% target.

Duration: 80337ms | Est. cost: $0.5535 | Turns: 13

===============================================================================
 ITERATION 17 | ? verifier | 33m 20s elapsed | 17/999
===============================================================================
2026-01-28T17:46:50.510744Z  INFO ralph_core::event_loop: Memory injection check: enabled=true, inject=Auto, workspace_root="/home/user/src/project"
2026-01-28T17:46:50.510783Z  INFO ralph_core::event_loop: Looking for memories at: "/home/user/src/project/.agent/memories.md" (exists: false)
2026-01-28T17:46:50.510794Z  INFO ralph_core::event_loop: Successfully loaded 0 memories from store
2026-01-28T17:46:50.510803Z  INFO ralph_core::event_loop: Memory store is empty - no memories to inject
I'm handling the tests.verified event. Let me first understand the context by reading the spec and checking what's been implemented.
⚙ [Read] /home/user/src/project/specs/step5-rbac-spec.md
⚙ [Bash] ralph tools task list 2>/dev/null || echo "No tasks"
Now let me check the current implementation status to verify the tests and see what's been done.
⚙ [Bash] go test ./internal/rbac/... ./internal/server/... -v -count=...
All tests pass. Now let me verify the implementation against the spec by checking the acceptance criteria coverage. Let me also check test coverage.
⚙ [Bash] go test -cover ./internal/rbac/... 2>&1
⚙ [Bash] go build ./... 2>&1
The tests.verified event confirms:
- All 20 ACs covered with unit and integration tests
- RBAC coverage 82.4% (exceeds 80% requirement)
- All tests pass
- Build succeeds

As Ralph (coordinator), I need to now delegate to the Spec Verifier to verify that the implementation matches the spec exactly. Let me emit the appropriate event.
⚙ [Bash] ralph emit "tests.verified" "All 20 ACs covered, RBAC covera...
The event was emitted, but I realize I already received the tests.verified event as the pending event for this iteration. According to the workflow, after tests.verified, the Spec Verifier should verify the implementation. Let me look at the workflow - when I receive tests.verified, I (as Ralph) should route to the 
Spec Verifier hat.

Looking at the flow:
tests.verified → Ralph → SpecVerifier (via spec.complete or spec.violated)

Wait - let me re-read the hat table. Actually, the Spec Verifier triggers on tests.verified and publishes spec.complete or spec.violated. As Ralph, I'm receiving this event and need to delegate by wearing the Spec Verifier hat.

Let me verify the implementation matches the spec as the Spec Verifier. I'll check each acceptance criterion manually.
⚙ [Read] /home/user/src/project/internal/rbac/rbac.go
⚙ [Read] /home/user/src/project/internal/server/rbac.go
Let me also check the groups.go for the RBAC integration and the migration file.
⚙ [Read] /home/user/src/project/internal/server/groups.go
⚙ [Read] /home/user/src/project/db/migrations/006_create_role_assignments.sql
Now let me look at the test files to confirm all ACs are tested properly.
⚙ [Grep] AC-[0-9]+
All 20 acceptance criteria from the RBAC spec have corresponding test coverage. Let me verify the implementation is complete against the spec by checking key aspects:

Verification Checklist as Spec Verifier:

1. ✅ Migration 006_create_role_assignments.sql - Matches spec exactly (UNIQUE constraint, CASCADE deletes, indexes)

2. ✅ Role Definitions - Admin, Operator, Viewer with correct permission mappings

3. ✅ Permission Types - view, operate, admin with correct role mappings

4. ✅ Hierarchical Inheritance - Uses traversal_ids for permission checking

5. ✅ RBAC Service - All required methods implemented:
   - AssignRole ✅
   - RevokeRole ✅
   - ListUserRoles ✅
   - ListGroupRoles ✅
   - HasPermission ✅
   - HasPermissionOnGroup ✅
   - GetUserRolesForGroup ✅
   - ListAccessibleRootIDs ✅

6. ✅ API Endpoints:
   - GET /api/v1/groups/{path}/roles ✅
   - POST /api/v1/groups/{path}/roles ✅
   - DELETE /api/v1/groups/{path}/roles/{role}/{user_id} ✅
   - GET /api/v1/users/{user_id}/roles ✅

7. ✅ Authorization Middleware - System admin bypass + RBAC checks on all group endpoints

8. ✅ All 20 Acceptance Criteria tested - AC-1 through AC-20

9. ✅ Test coverage - 82.4% (exceeds 80% requirement)

10. ✅ Build succeeds

11. ✅ All tests pass

The implementation matches the spec exactly. Let me emit the spec.complete event.
⚙ [Bash] ralph emit "spec.complete" "Step 5 RBAC verified: all 20 ACs...
The spec.complete event has been emitted. According to the workflow, this will trigger the Design Updater hat in the next iteration to update the design plan with the latest implementation details.

Duration: 62583ms | Est. cost: $0.4692 | Turns: 13

===============================================================================
 ITERATION 18 | ? design_updater | 34m 26s elapsed | 18/999
===============================================================================
2026-01-28T17:47:55.843745Z  INFO ralph_core::event_loop: Memory injection check: enabled=true, inject=Auto, workspace_root="/home/user/src/project"
2026-01-28T17:47:55.843800Z  INFO ralph_core::event_loop: Looking for memories at: "/home/user/src/project/.agent/memories.md" (exists: false)
2026-01-28T17:47:55.843819Z  INFO ralph_core::event_loop: Successfully loaded 0 memories from store
2026-01-28T17:47:55.843840Z  INFO ralph_core::event_loop: Memory store is empty - no memories to inject
⚙ [Read] /tmp/.tmpMqD2OM
I'm Ralph - the coordinator. Let me analyze the pending events and determine the next action.

Pending Events:
1. spec.complete - Step 5 RBAC verified: all 20 ACs pass, 82.4% coverage
2. tests.verified - All 20 ACs covered, RBAC coverage 82.4%, tests pass, build succeeds

According to the workflow, when I receive spec.complete, I should delegate to the 🧭️ Design Updater to update the design plan with the latest implementation details.

Let me first check the current task status and then emit the appropriate event.
⚙ [Bash] ralph tools task list
No tasks are pending. Now let me check the design plan location and the spec to understand the current state.
⚙ [Read] /home/user/src/project/.sop/planning/design/detailed-design.md
⚙ [Glob] specs/step5*.md
Now I have context. The spec.complete event indicates Step 5 RBAC has been verified. According to the workflow, I need to emit spec.complete event to trigger the 🧭️ Design Updater to update the design plan.

Looking at the flow: spec.complete → DesignUpdater → design.updated → GitCommitter → task.complete

Since I've received spec.complete and tests.verified events, I should delegate to the Design Updater to update the detailed design document with Step 5 RBAC implementation details.

Let me read the Step 5 RBAC spec first to understand what was implemented, so the Design Updater has proper context:
⚙ [Read] /home/user/src/project/specs/step5-rbac-spec.md
I now have the full context. The spec.complete event indicates Step 5 RBAC is verified with all 20 ACs passing and 82.4% coverage. 

According to the workflow diagram, when spec.complete is received, I should delegate to the 🧭️ Design Updater to update the design document. This is done by emitting spec.complete which triggers the Design Updater.

Let me emit the spec.complete event to delegate to the Design Updater:
⚙ [Bash] ralph emit "spec.complete" "Step 5 RBAC verified: all 20 ACs...
Event emitted. According to the instructions, I must stop working after publishing an event because a new iteration will start with fresh context.

LOOP_COMPLETE

Duration: 31874ms | Est. cost: $0.3715 | Turns: 7
2026-01-28T17:48:30.627249Z  INFO ralph_core::event_loop: Completion confirmed on consecutive iterations - terminating confirmations=2
2026-01-28T17:48:30.627313Z  INFO ralph::loop_runner: All done! LOOP_COMPLETE detected.
2026-01-28T17:48:30.627378Z  INFO ralph_core::event_loop: Wrapping up: completed. 18 iterations in 35m 0s. reason=completed iterations=18 duration=35m 0s

+----------------------------------------------------------+
| ? Loop terminated: Completion promise detected
+----------------------------------------------------------+
|   Iterations:  18
|   Elapsed:     2100.8s
+----------------------------------------------------------+

I also happened to record the session, but this doesn't look very useful?

recorded-session.jsonl

{"ts":1769620409782,"event":"_meta.loop_start","data":{"max_iterations":999,"prompt_file":"PROMPT.md","ux_mode":"cli"}}
{"ts":1769620423491,"event":"bus.publish","data":{"payload":"Starting Tarpon Remote Agent Management System implementation workflow","source":null,"target":null,"topic":"spec.start"}}
{"ts":1769620598603,"event":"bus.publish","data":{"payload":"Step 4: Groups with Hierarchy specification complete","source":null,"target":null,"topic":"spec.ready"}}
{"ts":1769620635693,"event":"bus.publish","data":{"payload":"Step 4: Groups with Hierarchy - spec is complete and implementable","source":null,"target":null,"topic":"spec.approved"}}
{"ts":1769620657934,"event":"bus.publish","data":{"payload":"Step 4: Groups with Hierarchy - ready for implementation","source":null,"target":null,"topic":"spec.approved"}}
{"ts":1769620680305,"event":"bus.publish","data":{"payload":"Step 4: Groups with Hierarchy - ready for implementation","source":null,"target":null,"topic":"spec.approved"}}
{"ts":1769621234853,"event":"bus.publish","data":{"payload":"Step 4: Groups with Hierarchy - All acceptance criteria implemented, tests passing with 86.6% coverage on group package","source":null,"target":null,"topic":"implementation.done"}}
{"ts":1769621280760,"event":"bus.publish","data":{"payload":"Step 4: Groups with Hierarchy - All 41 group tests pass (37 unit + 4 API), coverage 86.6%, all acceptance criteria verified","source":null,"target":null,"topic":"tests.verified"}}
{"ts":1769621352997,"event":"bus.publish","data":{"payload":"Step 4: Groups with Hierarchy - All 17 acceptance criteria verified. Implementation matches spec exactly: database migration, group service (Create, GetByID, GetByPath, ListRoots, ListChildren, Update, Delete, GetAncestors), 7 API endpoints, validation, error handling, cascade updates on slug rename. 41 tests passing, 86.6% coverage.","source":null,"target":null,"topic":"spec.complete"}}
{"ts":1769621408940,"event":"bus.publish","data":{"payload":"Step 4 Groups with Hierarchy added to detailed-design.md sections 9.18-9.23","source":null,"target":null,"topic":"design.updated"}}
{"ts":1769621448847,"event":"bus.publish","data":{"payload":"RECOVERY: Previous iteration did not publish an event. Review the scratchpad and either dispatch the next task or complete the loop.","source":null,"target":null,"topic":"task.resume"}}
{"ts":1769621488975,"event":"bus.publish","data":{"payload":"Step 5: Role-Based Access Control - Implement RBAC system with role assignments, hierarchical permission inheritance, and authorization middleware","source":null,"target":null,"topic":"spec.start"}}
{"ts":1769621619328,"event":"bus.publish","data":{"payload":"Step 5: RBAC specification complete - includes role_assignments table, hierarchical permission inheritance via traversal_ids, admin/operator/viewer roles, and authorization middleware","source":null,"target":null,"topic":"spec.ready"}}
{"ts":1769621645590,"event":"bus.publish","data":{"payload":"Step 5 RBAC spec is complete, unambiguous, and implementable. 20 acceptance criteria defined with Given-When-Then format. Ready for implementation.","source":null,"target":null,"topic":"spec.approved"}}
{"ts":1769621661213,"event":"bus.publish","data":{"payload":"Step 5 RBAC: 20 acceptance criteria, ready for implementation","source":null,"target":null,"topic":"spec.approved"}}
{"ts":1769622327625,"event":"bus.publish","data":{"payload":"Step 5 RBAC implementation complete. Migration, rbac service, endpoints, and tests all passing. RBAC coverage 82.4%, server coverage 73.7%.","source":null,"target":null,"topic":"implementation.done"}}
{"ts":1769622410510,"event":"bus.publish","data":{"payload":"All 20 ACs covered with unit and integration tests. RBAC coverage 82.4%, all tests pass.","source":null,"target":null,"topic":"tests.verified"}}
{"ts":1769622475843,"event":"bus.publish","data":{"payload":"All 20 ACs covered, RBAC coverage 82.4%, tests pass, build succeeds","source":null,"target":null,"topic":"tests.verified"}}
{"ts":1769622475843,"event":"bus.publish","data":{"payload":"Step 5 RBAC verified: all 20 ACs pass, 82.4% coverage, migration/service/endpoints match spec","source":null,"target":null,"topic":"spec.complete"}}
{"ts":1769622510627,"event":"bus.publish","data":{"payload":"## Reason\ncompleted\n\n## Status\nAll tasks completed successfully.\n\n## Summary\n- Iterations: 18\n- Duration: 35m 0s\n- Exit code: 0","source":null,"target":null,"topic":"loop.terminate"}}

So, any idea what might be causing this?

@Skinner927 commented on GitHub (Jan 28, 2026): I'm also having this issue with v2.2.5. I did not ever have this issue on v2.1.33. Here's what I've noticed when running `ralph run`: - `.agent/memories.md` is always missing (though I don't see it in old ralph either) - `.agent/scratchpad.md` is never created or written to. - `.agent/tasks.jsonl` is missing, but `.agent/tasks.jsonl.lock` is present. - `.ralph/history.jsonl` is missing, but `.ralph/history.jsonl.lock` is present. Here is my `ralph.yml` showing my hats. It's a modified version of the spec-driven preset. I have truncated the instructions and left only the lines that instruct how to emit events. These were always at the end of every instruction. <details> <summary>ralph.yml (click to expand)</summary> ```yaml # Specification-First Development cli: backend: "claude" event_loop: starting_event: "spec.start" # Ralph publishes this after coordination max_iterations: 999 # Maximum orchestration loops max_runtime_seconds: 57600 # 16 hours max runtime hats: spec_writer: name: "📋 Spec Writer" description: "Creates precise, unambiguous specifications with examples." triggers: ["spec.start", "spec.rejected"] publishes: ["spec.ready"] instructions: | ... Publish spec.ready when complete. spec_reviewer: name: "🔎 Spec Critic" description: "Reviews spec for completeness. Could someone implement from this?" triggers: ["spec.ready"] publishes: ["spec.approved", "spec.rejected"] instructions: | ... If issues found: publish spec.rejected with specific feedback If solid: publish spec.approved implementer: name: "⚙️ Implementer" description: "Implements EXACTLY what the spec says. No creative interpretation." triggers: ["spec.approved", "spec.violated"] publishes: ["implementation.done"] instructions: | ... Publish implementation.done when complete. test_verifier: name: "🧪 Test Verifier" description: Verifies the tests properly test the new features triggers: ["implementation.done"] publishes: ["tests.verified", "spec.violated"] default_publishes: "tests.verified" instructions: | ... If all checks and tests pass: publish tests.verified If violations or errors: publish spec.violated with details verifier: name: "✅ Spec Verifier" description: "Verifies implementation matches spec exactly." triggers: ["tests.verified"] publishes: ["spec.complete", "spec.violated"] default_publishes: "spec.complete" instructions: | ... If all pass: publish spec.complete If violations: publish spec.violated with details design_updater: name: "🧭️ Design Updater" description: "Updates the design plan with the latest implementation details" triggers: ["spec.complete"] publishes: ["design.updated"] default_publishes: "design.updated" instructions: | ... Publish design.updated when complete. committer: name: "💾️ Git Committer" description: "Commits Changes" triggers: ["design.updated"] publishes: ["task.complete"] default_publishes: "task.complete" instructions: | ... When you are done, output LOOP_COMPLETE ``` </details> Here's the summary of what I can see happened. - Iteration 16 worked as expected and emitted `tests.verified` - Iteration 17 correctly started as the verifier (spec verifier) handling the `tests.verified` event. - but in the middle switched to Ralph Coordinator and re-emitted the `tests.verified` event. - You can see it made up a false flow `tests.verified → Ralph → SpecVerifier`. - It then realized the error of its ways and put on the Spec Verifier hat. - It then finished as Spec Verifier as it should and emitted `spec.complete`. - Iteration 18 now sees two events: `spec.complete` and `tests.verified`. - So the coordinator in 18 emitted the event before it realized it messed up. - It gets really confused by the 2 events, claims 3 times it should delegate to the Design Updater, gives up and just screams `LOOP_COMPLETE`, something neither the test_verifier or verifier should have been able to do . <details> <summary>console output (click to expand)</summary> ``` =============================================================================== ITERATION 16 | ? test_verifier | 31m 57s elapsed | 16/999 =============================================================================== 2026-01-28T17:45:27.625833Z INFO ralph_core::event_loop: Memory injection check: enabled=true, inject=Auto, workspace_root="/home/user/src/project" 2026-01-28T17:45:27.625850Z INFO ralph_core::event_loop: Looking for memories at: "/home/user/src/project/.agent/memories.md" (exists: false) 2026-01-28T17:45:27.625854Z INFO ralph_core::event_loop: Successfully loaded 0 memories from store 2026-01-28T17:45:27.625855Z INFO ralph_core::event_loop: Memory store is empty - no memories to inject ⚙ [Read] /home/user/src/project/specs/step5-rbac-spec.md ⚙ [Bash] ralph tools task list I'm handling the implementation.done event for Step 5 RBAC. According to the workflow, after implementation.done, I need to trigger the Test Verifier hat to verify tests. Let me emit the tests.verified request to invoke that verification process. First, let me run the tests and check coverage to verify that the implementation tests are high quality. [... it does its thing ...] All criteria pass. Emitting tests.verified. ⚙ [Bash] ralph emit "tests.verified" "All 20 ACs covered with unit an... Tests verified. The RBAC implementation has comprehensive test coverage with all 20 acceptance criteria tested through both unit tests and API integration tests. Coverage is 82.4% for the rbac package, exceeding the 80% target. Duration: 80337ms | Est. cost: $0.5535 | Turns: 13 =============================================================================== ITERATION 17 | ? verifier | 33m 20s elapsed | 17/999 =============================================================================== 2026-01-28T17:46:50.510744Z INFO ralph_core::event_loop: Memory injection check: enabled=true, inject=Auto, workspace_root="/home/user/src/project" 2026-01-28T17:46:50.510783Z INFO ralph_core::event_loop: Looking for memories at: "/home/user/src/project/.agent/memories.md" (exists: false) 2026-01-28T17:46:50.510794Z INFO ralph_core::event_loop: Successfully loaded 0 memories from store 2026-01-28T17:46:50.510803Z INFO ralph_core::event_loop: Memory store is empty - no memories to inject I'm handling the tests.verified event. Let me first understand the context by reading the spec and checking what's been implemented. ⚙ [Read] /home/user/src/project/specs/step5-rbac-spec.md ⚙ [Bash] ralph tools task list 2>/dev/null || echo "No tasks" Now let me check the current implementation status to verify the tests and see what's been done. ⚙ [Bash] go test ./internal/rbac/... ./internal/server/... -v -count=... All tests pass. Now let me verify the implementation against the spec by checking the acceptance criteria coverage. Let me also check test coverage. ⚙ [Bash] go test -cover ./internal/rbac/... 2>&1 ⚙ [Bash] go build ./... 2>&1 The tests.verified event confirms: - All 20 ACs covered with unit and integration tests - RBAC coverage 82.4% (exceeds 80% requirement) - All tests pass - Build succeeds As Ralph (coordinator), I need to now delegate to the Spec Verifier to verify that the implementation matches the spec exactly. Let me emit the appropriate event. ⚙ [Bash] ralph emit "tests.verified" "All 20 ACs covered, RBAC covera... The event was emitted, but I realize I already received the tests.verified event as the pending event for this iteration. According to the workflow, after tests.verified, the Spec Verifier should verify the implementation. Let me look at the workflow - when I receive tests.verified, I (as Ralph) should route to the Spec Verifier hat. Looking at the flow: tests.verified → Ralph → SpecVerifier (via spec.complete or spec.violated) Wait - let me re-read the hat table. Actually, the Spec Verifier triggers on tests.verified and publishes spec.complete or spec.violated. As Ralph, I'm receiving this event and need to delegate by wearing the Spec Verifier hat. Let me verify the implementation matches the spec as the Spec Verifier. I'll check each acceptance criterion manually. ⚙ [Read] /home/user/src/project/internal/rbac/rbac.go ⚙ [Read] /home/user/src/project/internal/server/rbac.go Let me also check the groups.go for the RBAC integration and the migration file. ⚙ [Read] /home/user/src/project/internal/server/groups.go ⚙ [Read] /home/user/src/project/db/migrations/006_create_role_assignments.sql Now let me look at the test files to confirm all ACs are tested properly. ⚙ [Grep] AC-[0-9]+ All 20 acceptance criteria from the RBAC spec have corresponding test coverage. Let me verify the implementation is complete against the spec by checking key aspects: Verification Checklist as Spec Verifier: 1. ✅ Migration 006_create_role_assignments.sql - Matches spec exactly (UNIQUE constraint, CASCADE deletes, indexes) 2. ✅ Role Definitions - Admin, Operator, Viewer with correct permission mappings 3. ✅ Permission Types - view, operate, admin with correct role mappings 4. ✅ Hierarchical Inheritance - Uses traversal_ids for permission checking 5. ✅ RBAC Service - All required methods implemented: - AssignRole ✅ - RevokeRole ✅ - ListUserRoles ✅ - ListGroupRoles ✅ - HasPermission ✅ - HasPermissionOnGroup ✅ - GetUserRolesForGroup ✅ - ListAccessibleRootIDs ✅ 6. ✅ API Endpoints: - GET /api/v1/groups/{path}/roles ✅ - POST /api/v1/groups/{path}/roles ✅ - DELETE /api/v1/groups/{path}/roles/{role}/{user_id} ✅ - GET /api/v1/users/{user_id}/roles ✅ 7. ✅ Authorization Middleware - System admin bypass + RBAC checks on all group endpoints 8. ✅ All 20 Acceptance Criteria tested - AC-1 through AC-20 9. ✅ Test coverage - 82.4% (exceeds 80% requirement) 10. ✅ Build succeeds 11. ✅ All tests pass The implementation matches the spec exactly. Let me emit the spec.complete event. ⚙ [Bash] ralph emit "spec.complete" "Step 5 RBAC verified: all 20 ACs... The spec.complete event has been emitted. According to the workflow, this will trigger the Design Updater hat in the next iteration to update the design plan with the latest implementation details. Duration: 62583ms | Est. cost: $0.4692 | Turns: 13 =============================================================================== ITERATION 18 | ? design_updater | 34m 26s elapsed | 18/999 =============================================================================== 2026-01-28T17:47:55.843745Z INFO ralph_core::event_loop: Memory injection check: enabled=true, inject=Auto, workspace_root="/home/user/src/project" 2026-01-28T17:47:55.843800Z INFO ralph_core::event_loop: Looking for memories at: "/home/user/src/project/.agent/memories.md" (exists: false) 2026-01-28T17:47:55.843819Z INFO ralph_core::event_loop: Successfully loaded 0 memories from store 2026-01-28T17:47:55.843840Z INFO ralph_core::event_loop: Memory store is empty - no memories to inject ⚙ [Read] /tmp/.tmpMqD2OM I'm Ralph - the coordinator. Let me analyze the pending events and determine the next action. Pending Events: 1. spec.complete - Step 5 RBAC verified: all 20 ACs pass, 82.4% coverage 2. tests.verified - All 20 ACs covered, RBAC coverage 82.4%, tests pass, build succeeds According to the workflow, when I receive spec.complete, I should delegate to the 🧭️ Design Updater to update the design plan with the latest implementation details. Let me first check the current task status and then emit the appropriate event. ⚙ [Bash] ralph tools task list No tasks are pending. Now let me check the design plan location and the spec to understand the current state. ⚙ [Read] /home/user/src/project/.sop/planning/design/detailed-design.md ⚙ [Glob] specs/step5*.md Now I have context. The spec.complete event indicates Step 5 RBAC has been verified. According to the workflow, I need to emit spec.complete event to trigger the 🧭️ Design Updater to update the design plan. Looking at the flow: spec.complete → DesignUpdater → design.updated → GitCommitter → task.complete Since I've received spec.complete and tests.verified events, I should delegate to the Design Updater to update the detailed design document with Step 5 RBAC implementation details. Let me read the Step 5 RBAC spec first to understand what was implemented, so the Design Updater has proper context: ⚙ [Read] /home/user/src/project/specs/step5-rbac-spec.md I now have the full context. The spec.complete event indicates Step 5 RBAC is verified with all 20 ACs passing and 82.4% coverage. According to the workflow diagram, when spec.complete is received, I should delegate to the 🧭️ Design Updater to update the design document. This is done by emitting spec.complete which triggers the Design Updater. Let me emit the spec.complete event to delegate to the Design Updater: ⚙ [Bash] ralph emit "spec.complete" "Step 5 RBAC verified: all 20 ACs... Event emitted. According to the instructions, I must stop working after publishing an event because a new iteration will start with fresh context. LOOP_COMPLETE Duration: 31874ms | Est. cost: $0.3715 | Turns: 7 2026-01-28T17:48:30.627249Z INFO ralph_core::event_loop: Completion confirmed on consecutive iterations - terminating confirmations=2 2026-01-28T17:48:30.627313Z INFO ralph::loop_runner: All done! LOOP_COMPLETE detected. 2026-01-28T17:48:30.627378Z INFO ralph_core::event_loop: Wrapping up: completed. 18 iterations in 35m 0s. reason=completed iterations=18 duration=35m 0s +----------------------------------------------------------+ | ? Loop terminated: Completion promise detected +----------------------------------------------------------+ | Iterations: 18 | Elapsed: 2100.8s +----------------------------------------------------------+ ``` </details> I also happened to record the session, but this doesn't look very useful? <details><summary>recorded-session.jsonl</summary> ``` {"ts":1769620409782,"event":"_meta.loop_start","data":{"max_iterations":999,"prompt_file":"PROMPT.md","ux_mode":"cli"}} {"ts":1769620423491,"event":"bus.publish","data":{"payload":"Starting Tarpon Remote Agent Management System implementation workflow","source":null,"target":null,"topic":"spec.start"}} {"ts":1769620598603,"event":"bus.publish","data":{"payload":"Step 4: Groups with Hierarchy specification complete","source":null,"target":null,"topic":"spec.ready"}} {"ts":1769620635693,"event":"bus.publish","data":{"payload":"Step 4: Groups with Hierarchy - spec is complete and implementable","source":null,"target":null,"topic":"spec.approved"}} {"ts":1769620657934,"event":"bus.publish","data":{"payload":"Step 4: Groups with Hierarchy - ready for implementation","source":null,"target":null,"topic":"spec.approved"}} {"ts":1769620680305,"event":"bus.publish","data":{"payload":"Step 4: Groups with Hierarchy - ready for implementation","source":null,"target":null,"topic":"spec.approved"}} {"ts":1769621234853,"event":"bus.publish","data":{"payload":"Step 4: Groups with Hierarchy - All acceptance criteria implemented, tests passing with 86.6% coverage on group package","source":null,"target":null,"topic":"implementation.done"}} {"ts":1769621280760,"event":"bus.publish","data":{"payload":"Step 4: Groups with Hierarchy - All 41 group tests pass (37 unit + 4 API), coverage 86.6%, all acceptance criteria verified","source":null,"target":null,"topic":"tests.verified"}} {"ts":1769621352997,"event":"bus.publish","data":{"payload":"Step 4: Groups with Hierarchy - All 17 acceptance criteria verified. Implementation matches spec exactly: database migration, group service (Create, GetByID, GetByPath, ListRoots, ListChildren, Update, Delete, GetAncestors), 7 API endpoints, validation, error handling, cascade updates on slug rename. 41 tests passing, 86.6% coverage.","source":null,"target":null,"topic":"spec.complete"}} {"ts":1769621408940,"event":"bus.publish","data":{"payload":"Step 4 Groups with Hierarchy added to detailed-design.md sections 9.18-9.23","source":null,"target":null,"topic":"design.updated"}} {"ts":1769621448847,"event":"bus.publish","data":{"payload":"RECOVERY: Previous iteration did not publish an event. Review the scratchpad and either dispatch the next task or complete the loop.","source":null,"target":null,"topic":"task.resume"}} {"ts":1769621488975,"event":"bus.publish","data":{"payload":"Step 5: Role-Based Access Control - Implement RBAC system with role assignments, hierarchical permission inheritance, and authorization middleware","source":null,"target":null,"topic":"spec.start"}} {"ts":1769621619328,"event":"bus.publish","data":{"payload":"Step 5: RBAC specification complete - includes role_assignments table, hierarchical permission inheritance via traversal_ids, admin/operator/viewer roles, and authorization middleware","source":null,"target":null,"topic":"spec.ready"}} {"ts":1769621645590,"event":"bus.publish","data":{"payload":"Step 5 RBAC spec is complete, unambiguous, and implementable. 20 acceptance criteria defined with Given-When-Then format. Ready for implementation.","source":null,"target":null,"topic":"spec.approved"}} {"ts":1769621661213,"event":"bus.publish","data":{"payload":"Step 5 RBAC: 20 acceptance criteria, ready for implementation","source":null,"target":null,"topic":"spec.approved"}} {"ts":1769622327625,"event":"bus.publish","data":{"payload":"Step 5 RBAC implementation complete. Migration, rbac service, endpoints, and tests all passing. RBAC coverage 82.4%, server coverage 73.7%.","source":null,"target":null,"topic":"implementation.done"}} {"ts":1769622410510,"event":"bus.publish","data":{"payload":"All 20 ACs covered with unit and integration tests. RBAC coverage 82.4%, all tests pass.","source":null,"target":null,"topic":"tests.verified"}} {"ts":1769622475843,"event":"bus.publish","data":{"payload":"All 20 ACs covered, RBAC coverage 82.4%, tests pass, build succeeds","source":null,"target":null,"topic":"tests.verified"}} {"ts":1769622475843,"event":"bus.publish","data":{"payload":"Step 5 RBAC verified: all 20 ACs pass, 82.4% coverage, migration/service/endpoints match spec","source":null,"target":null,"topic":"spec.complete"}} {"ts":1769622510627,"event":"bus.publish","data":{"payload":"## Reason\ncompleted\n\n## Status\nAll tasks completed successfully.\n\n## Summary\n- Iterations: 18\n- Duration: 35m 0s\n- Exit code: 0","source":null,"target":null,"topic":"loop.terminate"}} ``` </details> So, any idea what might be causing this?

kerem commented

2026-02-27 10:21:57 +03:00

Author

Owner

@mikeyobrien commented on GitHub (Jan 28, 2026):

I took a deep look at this last night, I think this should stop happening with the latest meta-prompt. I'll be releasing it later today, though the changes are now on main if you want to check it out.

@mikeyobrien commented on GitHub (Jan 28, 2026): I took a deep look at this last night, I _think_ this should stop happening with the latest meta-prompt. I'll be releasing it later today, though the changes are now on main if you want to check it out.

kerem commented

2026-02-27 10:21:57 +03:00

Author

Owner

@Skinner927 commented on GitHub (Jan 29, 2026):

I'll definitely give it a shot. Thanks for looking into this, seriously.

Some extra datapoints if it helps:

v2.2.5 did the same thing 3 more times today, only got 1 or 2 steps in and got confused. I even tried simplifying the hats and removing all plugins.

v2.1.3 has been running for the last 5+ hours, no problems, with all the hats and plugins as before.

@Skinner927 commented on GitHub (Jan 29, 2026): I'll definitely give it a shot. Thanks for looking into this, seriously. Some extra datapoints if it helps: v2.2.5 did the same thing 3 more times today, only got 1 or 2 steps in and got confused. I even tried simplifying the hats and removing all plugins. v2.1.3 has been running for the last 5+ hours, no problems, with all the hats and plugins as before.

kerem commented

2026-02-27 10:21:57 +03:00

Author

Owner

@mikeyobrien commented on GitHub (Jan 29, 2026):

Thank you for the data points. v2.3.0 is now out. Hopefully this fixes things!

@mikeyobrien commented on GitHub (Jan 29, 2026): Thank you for the data points. v2.3.0 is now out. Hopefully this fixes things!

kerem commented

2026-02-27 10:21:57 +03:00

Author

Owner

@mikeyobrien commented on GitHub (Jan 29, 2026):

Looks like the regression was introduced with the migration to memory/tasks. I've re-added scratchpad and updated the prompt such that:

Long-term (project-scope -> Memories
Short-term (prompt-scope) -> Scratchpad/Context files

@mikeyobrien commented on GitHub (Jan 29, 2026): Looks like the regression was introduced with the migration to memory/tasks. I've re-added scratchpad and updated the prompt such that: Long-term (project-scope -> Memories Short-term (prompt-scope) -> Scratchpad/Context files

kerem commented

2026-02-27 10:21:57 +03:00

Author

Owner

@Skinner927 commented on GitHub (Jan 30, 2026):

I'm going to try the godnight10061/ralph-orchestrator#3 branch shortly, but I'm not sure it'll fix the issue. It looks like it's only checking for incomplete task/* files? When you run a spec loop, the specs are generated as the first hat in the loop, so there's no spec files to check. Basically, at the end of every loop, all spec files will be completed, the rest haven't been written.

I'm not well versed in this magic however, so I'll still give it a test in hopefully an hour.

@Skinner927 commented on GitHub (Jan 30, 2026): I'm going to try the godnight10061/ralph-orchestrator#3 branch shortly, but I'm not sure it'll fix the issue. It looks like it's only checking for incomplete `task/*` files? When you run a spec loop, the specs are generated as the first hat in the loop, so there's no spec files to check. Basically, at the end of every loop, all spec files will be completed, the rest haven't been written. I'm not well versed in this magic however, so I'll still give it a test in hopefully an hour.

kerem commented

2026-02-27 10:21:57 +03:00

Author

Owner

@mikeyobrien commented on GitHub (Jan 30, 2026):

Haven't run into a premature exit all day today.

🤞

@mikeyobrien commented on GitHub (Jan 30, 2026): Haven't run into a premature exit all day today. 🤞

kerem commented

2026-02-27 10:21:57 +03:00

Author

Owner

@nartc commented on GitHub (Feb 2, 2026):

I haven't run into a premature exit since last Saturday. I think we can close this issue down and if folks run into it again, they can open new one with new info.

@nartc commented on GitHub (Feb 2, 2026): I haven't run into a premature exit since last Saturday. I think we can close this issue down and if folks run into it again, they can open new one with new info.

Rows
Columns

[GH-ISSUE #123] Loop done when tasks are incomplete #47