Skip to content

Spec validation before loop start #7

Description

@adamdaum

Summary

The loop starts executing immediately without verifying that the PRD, app_spec, and feature_list are well-formed and consistent. A malformed spec can burn through expensive AI iterations before anyone notices.

Current Behavior

  • ralph.sh passes the coding prompt to Claude without validating inputs
  • If feature_list.json has invalid JSON, missing fields, or no features, the loop may error mid-execution or produce nonsensical results
  • No check that app_spec.txt exists or has the expected sections
  • No check that referenced tech stack components are compatible

Proposal

Add a validation step to ralph.sh (or as a standalone validate.sh / CLI command) that runs before the loop:

Checks to implement:

  1. feature_list.json

    • Valid JSON
    • All required fields present (id, priority, category, description, steps, passes)
    • No duplicate feature IDs
    • No gaps in priority ordering
    • depends_on references exist (if dependency resolution is implemented)
  2. app_spec.txt

    • File exists and is non-empty
    • Contains expected sections (Tech Stack, Data Models, API Routes, etc.)
  3. PRD.md

    • File exists
    • Contains expected sections
  4. Cross-file consistency

    • Features in feature_list reference concepts described in app_spec
    • Tech stack in app_spec matches PRD requirements

Output:

Validating specs...
  ✓ feature_list.json: 10 features, valid schema
  ✓ app_spec.txt: all sections present
  ✓ PRD.md: exists, 2,400 words
  ⚠ Warning: feature API-003 references "Redis" but app_spec tech stack does not include it
  ✓ Validation passed with 1 warning

Considerations

  • Validation should be non-blocking by default (warnings) with a --strict flag for hard failures
  • Could be integrated into init.sh as well, running after spec generation
  • Keep validation fast — it should add seconds, not minutes

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions