feat: Composable eval construction from registry graders

[spboyer]

Migrated from spboyer/waza#389

Summary

Design how users construct eval.yaml from a catalog of shared, versioned grader definitions.

Concept

Users should be able to:

1. Browse available graders (waza registry search 'factual')
2. Add a grader to their eval (waza registry add github.com/waza-evals/fact@v1.0)
3. Reference it in eval.yaml with local config overrides
4. Run evals that mix local and remote graders seamlessly

Design Questions

• What does waza registry search look like? Metadata format?
• How do remote graders interact with local graders in the same eval?
• Can you override config on a remote grader (e.g., change threshold)?
• How does waza init scaffold with registry graders?

Parent: #385

[spboyer]

Validation update on main:

• Repro attempt: go run ./cmd/waza registry search factual returns unknown command "registry" for "waza".
• So composable eval construction from a grader registry is not currently implemented in CLI UX.

Additional information needed to validate implementation details:

1. Exact command surface (registry search/add/list/pull and expected output format).
2. Local override precedence when mixing remote grader refs with inline/local config.
3. Locking/version behavior for reproducible runs (pinning + lockfile expectations).
4. Minimal v1 acceptance test scenario (end-to-end user flow).}

[spboyer]

Validation update:

• Reproduced on current main (4d32191c): there is no implemented registry search/add workflow for composing evals from shared grader definitions.

Additional information needed to validate implementation direction:

1. Desired metadata schema returned by registry search.
2. Precedence rules when local overrides conflict with remote defaults.
3. UX expectations for waza init scaffolding with registry graders.

[spboyer]

Validation & Design Status

Issue Status: Design Spike Validation

Current Understanding

Design for users to construct eval.yaml from a catalog of shared, versioned grader definitions with registry search, add, and reference functionality.

Status & Dependencies

1. Related PR docs: Eval & Grader Registry design doc (#13) #337: The "Eval & Grader Registry design doc" PR appears to cover this scope. Is that the authoritative design?
2. Prerequisite: Does this depend on feat: Go-module-style grader/eval references #15 (Go-module-style references) being designed first?
3. API Spec: What does the registry metadata format look like? (e.g., what's in the grader manifest?)
4. User Workflow: Confirm the intended workflow is: search → add → override config → run?
5. Offline Support: Should local vs. remote graders have different caching strategies?

Design Questions Status

Are the "Design Questions" in the issue description still open, or have they been addressed in design doc (#337)?

Please clarify: Is this blocked on #337 completion, or running in parallel?

[spboyer]

Revalidated on current \ (92c2868): \ still returns , so the registry UX is not implemented yet.\n\nAdditional details still needed to validate implementation scope:\n1. Exact command contract for \ (input/output shape).\n2. Precedence when remote defaults conflict with local overrides.\n3. Minimum end-to-end acceptance flow for v1 (search -> add -> run).