feat: Phase 1 Go-module-style grader refs

[Shayne Boyer (spboyer)]

Closes #15

Summary

Phase 1 implementation of Go-module-style grader references in eval.yaml, per docs/research/waza-eval-registry-design.md.

Adds a ref: field to grader entries so an eval can pull a shared, versioned grader preset from a Git repo instead of duplicating the config inline:

graders:
  - ref: github.com/waza-evals/fact#factuality@v1.0.0
    weight: 2.0
    config:
      threshold: 0.9

What's included

• Schema — GraderConfig.Ref field; when set, type and config validation are deferred to after ref resolution so the remote preset defines them (internal/models/spec.go).
• Ref parser — canonical host/owner/repo[/path][#export]@version parser. Phase 1 accepts only github.com host + exact semver tags or 40-char commit SHAs. Floating selectors (branches, ranges, latest) are rejected up front (internal/registry/ref.go).
• Lockfile — waza.lock (YAML, schema_version=1) pins each ref to a commit SHA and sha256 content digest. Sorted for stable diffs (internal/registry/lockfile.go).
• Resolver — direct GitHub raw content fetches (no git archive / full clone). Content-addressed disk cache at ~/.waza/cache/{host}/{owner}/{repo}/{sha}/. Digest verification on every load (internal/registry/resolver.go). Fetcher is an interface, so tests use httptest.
• Manifest — minimal waza.registry.yaml reader for #export-style refs (internal/registry/manifest.go).
• Expansion — merges remote preset with local overrides (scalar override, config deep-merge, list replace) and round-trips through existing GraderConfig.UnmarshalYAML so all strong-typed grader-kind validation runs unchanged (internal/registry/expand.go).
• waza get [eval.yaml] — resolves refs and writes/updates the lockfile. --verify mode checks the lock without modifying it (cmd/waza/cmd_get.go).
• waza run integration — auto-resolves refs on first run if no lockfile exists, then writes the lock. If the lock exists, refs are verified strictly (missing entry or digest mismatch fails the run) (cmd/waza/cmd_run.go).

Policy decisions

• Permissive first run. The design doc §8 says "fail if no lock." First-time UX is painful with that rule, so this compromise auto-resolves on first run when every ref is exact-pinned (which ParseRef already enforces) and writes the lock. Once the lock exists, verification is strict.
• Phase 1 scope. github.com host + grader-preset artifact only. Non-github hosts, eval-suite/mock-server artifacts, and floating version selectors are deferred to later phases.
• Digest = sha256 of the preset YAML file bytes, sha256:<hex> format.

Tests

• internal/models/spec_test.go — parsing a ref: grader entry without inline type/config.
• internal/registry/ref_test.go — parser accept/reject matrix (tags, SHAs, floating refs, wrong host, path+export conflict).
• internal/registry/lockfile_test.go — round-trip, upsert-replace, missing-file, bad-schema.
• internal/registry/resolver_test.go — httptest-backed GitHub API + raw content servers; covers fresh resolve → cache → verify, ErrRefNotInLock, ErrDigestMismatch, and subpath refs that skip the manifest.
• internal/registry/expand_test.go — merge rules for scalar override, config deep-merge, default name, missing-type guard.

Verify

go fmt ./...   # clean
go test ./...  # all pass
golangci-lint run  # 0 issues

Draft until follow-up items land:

• End-to-end integration test hitting waza run against a fixture eval + real cache
• Docs page under site/src/content/docs/reference/ for the ref: syntax and waza get
• README update pointing at waza get in the Commands section

[Copilot]

Pull request overview

Implements Phase 1 of “Go-module-style” remote grader preset references for eval.yaml, including parsing ref: strings, resolving remote YAML presets via GitHub raw-content fetches, caching + digest verification, and pinning resolutions in a waza.lock lockfile. This integrates into both waza get (explicit resolution) and waza run (auto-resolve on first run when no lock exists, strict verify once the lock exists).

Changes:

• Add ref: support to GraderConfig YAML parsing, deferring type/config validation until after remote resolution.
• Introduce registry/ref parsing, manifest reading, lockfile format, resolver/cache, and preset expansion (remote base + local overrides).
• Add waza get command and integrate resolver/lock policy into waza run.

Show a summary per file

File	Description
internal/registry/spec_resolve.go	Walks an EvalSpec and expands all ref: graders via resolver + lockfile, persisting lock updates when requested.
internal/registry/resolver.go	Fetches pinned preset YAML from GitHub, caches it, verifies sha256 digest, and updates lock entries.
internal/registry/resolver_test.go	Adds httptest-backed coverage for resolve/update/verify flows, cache usage, and digest mismatch behavior.
internal/registry/ref.go	Parses canonical host/owner/repo[/path][#export]@version refs with Phase 1 constraints (github.com + exact tag/SHA).
internal/registry/ref_test.go	Valid/invalid matrix tests for ref parsing and round-tripping.
internal/registry/manifest.go	Parses waza.registry.yaml and maps #export to preset file paths.
internal/registry/lockfile.go	Defines waza.lock schema and load/save/upsert behavior.
internal/registry/lockfile_test.go	Covers lockfile round-trip, sort stability, upsert behavior, and schema validation.
internal/registry/expand.go	Merges remote preset YAML with local overrides and re-validates via existing GraderConfig.UnmarshalYAML.
internal/registry/expand_test.go	Tests merge rules, scalar overrides, deep-merge semantics, and required remote type.
internal/models/spec.go	Adds GraderConfig.Ref and updates YAML decoding to preserve override config for ref-based graders.
internal/models/spec_test.go	Tests parsing a ref-based grader entry without inline type.
cmd/waza/root.go	Registers the new get subcommand.
cmd/waza/cmd_run.go	Resolves remote grader refs at run start; permissive first run (writes lock), strict verification after lock exists.
cmd/waza/cmd_get.go	Implements waza get [eval.yaml] to resolve refs and write/update waza.lock, plus --verify mode.

Review details

• Files reviewed: 15/15 changed files
• Comments generated: 4
• Review effort level: Low

[Shayne Boyer (spboyer)]

Closing as superseded by #494 for issue #15 consolidation. #494 is the canonical branch for this issue and will carry follow-up fixes.