Skip to content

Ralph Loop v2: codified autonomous software-building harness - #10

Merged
adamdaum merged 12 commits into
mainfrom
feat/ralph-v2-harness
Jul 5, 2026
Merged

Ralph Loop v2: codified autonomous software-building harness#10
adamdaum merged 12 commits into
mainfrom
feat/ralph-v2-harness

Conversation

@adamdaum

@adamdaum adamdaum commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Ground-up v2 of the Ralph technique: replaces the 55-line ralph.sh with a cross-platform TypeScript runtime (ralph-loop, bin ralph) in an npm-workspaces monorepo, alongside the slimmed create-ralph-loop scaffolder. Supervised-by-exception autonomous building with real, mechanical guardrails.

Highlights

Guardrailed loop — each iteration is a harness-driven state machine: git checkpoint → coder implements ONE feature → mechanical gates → independent verifier → accept-commit or hard-revert.

  • Mechanical gates run by the harness (not the agent): baseline-relative typecheck/test/build (only NEW failures block), diff-size bound, and a featureIntegrity gate making features.json harness-owned (no more self-grading).
  • Independent, fail-closed verifier in a fresh context on a cheaper model; unparseable/ambiguous → treated as failure. Completion is "all features verified" — the vestigial <promise>COMPLETE</promise> sentinel is gone.
  • Checkpoint + auto-revert; bounded retries then block.
  • Cost/iteration/wall-clock budgets + webhook/desktop notifications; per-role token/cost telemetry in .ralph/run-state.json; structured .ralph/progress.jsonl.
  • Periodic strong-model replan (validated <ralph-plan-update> ops, DAG-checked, verified features protected) and periodic gardener pass against entropy/"AI slop".

Multi-model routing & local LLMs — roles → (adapter, model, permission tier) in ralph.config.json. Default: Fable for planning, Codex for building, Haiku for cheap fail-closed verify. Adapters: claude, codex, aider (local via ollama/…).

Parallel-ready schema — features.json v2 with depends_on/status/attempts/verification/dormant lease; DAG-aware selection; planner prompt rewritten to minimize inter-feature dependencies; devServer.portRange groundwork.

Windows-native — no bash in the happy path. Fixed a real cross-spawn bug where shell:true exit code 1 (e.g. failing npm test) was misreported as ENOENT on Windows.

CLI

ralph run|plan|dev|doctor|status|migrate|export — including ralph migrate to upgrade v1 projects (feature_list.json + ralph.sh → v2).

Testing

  • 130 tests: unit suites per module + 8 real-git mock-adapter end-to-end scenarios (happy path, verifier pass/fail, gate-failure retry+block, features.json tamper revert, dependency ordering, replan-blocks-a-feature, agent give-up).
  • Live scaffold → ralph doctor passes end to end (detects installed codex/claude CLIs).
  • CI matrix added (ubuntu + windows).

Deferred

Multi-framework greenfield template packs (old issue #1). The runtime is already framework-agnostic (configurable devServer.command/gates; adopt-mode detects Vite/Remix/Astro/Next); shipping untested per-framework packs was left as a scoped follow-up.

Design provenance: OpenAI harness-engineering, hermes-agent/Nightwire fail-closed verification, Anthropic multi-agent coordination guidance, Aider Architect/Editor + RouteLLM routing.

🤖 Generated with Claude Code

adamdaum and others added 12 commits July 3, 2026 14:37
@adamdaum @claude
chore: convert to npm workspaces monorepo (Phase 0)
5aa4e14 
Split into packages/create-ralph-loop (scaffolder, bumped to 2.0.0, chalk
dropped, eta added) and packages/ralph (new ralph-loop runtime, bin: ralph,
CommonJS). Root becomes a private workspaces root with shared tsx/vitest/tsc
tooling. No user-facing behavior change yet.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@adamdaum @claude
feat(ralph): shared foundation — config/features/adapter/event schema…
c196d02 
…s + util

zod schemas for ralph.config.json (Fable/Codex/Haiku default routing) and
features.json v2 (status/depends_on/attempts/verification/lease). Adapter and
event type contracts. Cross-platform proc runner (cross-spawn, no execa/ESM),
git helpers, paths, logger, and the Gate context contract.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@adamdaum @claude
feat(ralph): add spawnDetached to proc runner for dev-server lifecycle
fbbd338 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@adamdaum @claude
feat(ralph): runner adapters + mechanical gates
5ca8d7d 
Adapters: claude (-p --output-format json, usage/cost parse), codex (exec
--json, sandbox tiers), mock (scripted turns for tests), registry. Gates:
baseline-relative command gates (tsc/test/build), featureIntegrity (hash),
diffSize. 43 unit tests.

Also fix a Windows cross-spawn bug: shell:true misreported exit code 1 as
ENOENT, which would break gate commands like `npm test`. runShell now uses
Node's native spawn for the shell path; cross-spawn stays for argv spawns.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@adamdaum @claude
feat(ralph): feature store/DAG, dev-server manager, telemetry, prompts
113aae0 
- features: FeatureStore (harness-only transitions, deterministic serialization
  + integrity hash), DAG validation/selection with depends_on gating, v1->v2
  migration
- devserver: cross-platform TS dev-server lifecycle (spawnDetached + fetch
  readiness + tree-kill), replacing the bash dev-up/down scripts
- events: progress.jsonl EventLog + resumable run-state store with per-role
  usage accounting
- prompts: eta template rendering (override-aware) + fail-closed <ralph-result>/
  <ralph-verdict>/<ralph-plan-update> parsing; v2 prompt assets (coding injects
  the feature, no self-grading; verifier; replanner; dependency-minimized init;
  prd; gardener)

108 unit tests, full strict build green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@adamdaum @claude
feat(ralph): guardrailed run loop + CLI (Phases 1-2 core)
dc4f62f 
The orchestrator: iteration state machine (checkpoint -> coder -> stage ->
mechanical gates -> independent fail-closed verifier -> accept-commit or
hard-revert), DAG-eligible feature selection, bounded retries then block,
per-role token/cost accounting, cost/iteration/time budgets, stall detection,
and webhook/desktop notifications. CLI: run, dev up|down|status, doctor,
status, migrate (v1 feature_list.json + bash scripts -> v2).

Completion is now "all features verified" — the vestigial <promise>COMPLETE
</promise> sentinel is gone. Success is no longer self-graded: the harness owns
features.json (integrity gate) and an independent verifier confirms every pass.

7 end-to-end scenarios (real git, mock adapters) cover happy path, verifier
pass/fail, gate-failure retry+block, features.json tamper revert, dependency
ordering, and agent give-up. 115 tests total, strict build green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@adamdaum @claude
feat(ralph): Phase 3-4 — replanner, gardener, model routing, local LL…
e944029 
…Ms, exports

- replanner: strong-model periodic review emitting validated <ralph-plan-update>
  ops (reprioritize/block/unblock/split/prune/add_dependency); verified features
  protected; DAG re-validated before apply. Wired into the loop on a cadence.
- gardener: periodic entropy/"AI slop" cleanup pass, gated like a coding turn.
- aider adapter: local-LLM path (ollama/openrouter models) behind the adapter
  interface; role-based routing table already honored end to end.
- CLI: `ralph plan` (planner generates PRD/app_spec/features), `ralph export
  --format eval-jsonl` (verification records for offline eval).
- config: garden cadence + devServer.portRange groundwork for parallel tracks.
- JSON Schema generation (schema/ralph.config.schema.json) for editor validation.
- Tests: replanner ops, aider parsing, replan e2e scenario. 130 tests total.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@adamdaum @claude
feat(create-ralph-loop): thin v2 template + eta + adopt de-duplication
b2a3320 
Emit projects for the ralph-loop v2 runtime: ralph.config.json (with $schema,
Fable/Codex/Haiku routing, gates/budgets/replan/verify), features.json v2 seed
(dependency-minimized INFRA/UI/FEAT), AGENTS.md as a docs/ table-of-contents,
docs/ knowledge-base skeleton. Delete the shipped bash loop/dev scripts and old
prompt files (the runtime owns these now) and the duplicated adopt-mode script
string literals from cli.ts. Templating moves from a naive {{var}} regex to eta.
Detection feeds the rendered config (devServer command/port, gates.test). Adopt
merges ralph-loop into devDependencies and points at `ralph migrate`. 5 smoke
tests validate the output against the runtime's zod schemas.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@adamdaum @claude
ci: GitHub Actions build+test matrix (ubuntu + windows); rewrite root…
cc2be33 
… README for v2

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@adamdaum @claude
docs: per-package READMEs + LICENSE, and a v2 CHANGELOG
9353401 
Add npm-landing READMEs for ralph-loop (runtime) and create-ralph-loop
(scaffolder), copy LICENSE into each package for clean publishing, and add a
root CHANGELOG documenting the v1→v2 rewrite and migration.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@adamdaum @claude
docs: add CONTRIBUTING with module map + adapter/gate recipes
fda36e8 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@adamdaum @claude
fix(ralph): close parent log fd in spawnDetached (Windows file lock)
b543d17 
spawnDetached passed the log-file descriptor to the child but never closed the
parent's copy. On Windows that keeps dev-server.log locked for the life of the
harness, so deleting/cleaning .ralph fails with ENOTEMPTY (red on windows CI).
Close the parent fd after spawn; the child keeps its own inherited handle.

Also harden two tests: retry temp-dir removal in the dev-server test (residual
child-handle release lag on Windows) and recreate src/ in the e2e coder fixture
(git clean -fd removes it on revert, which was masking retries as ENOENT throws).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@adamdaum
adamdaum merged commit d0567ec into main Jul 5, 2026
2 checks passed
@adamdaum
adamdaum deleted the feat/ralph-v2-harness branch July 5, 2026 21:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@adamdaum