feat: Go-module-style grader/eval references

[spboyer]

Migrated from spboyer/waza#387

Summary

Design how eval.yaml can reference remote graders via Go-module-style paths.

Concept

graders:
  - ref: github.com/waza-evals/fact@v1.0
  - ref: github.com/myorg/custom-grader@v2.1
    config:
      threshold: 0.8

Design Questions

• How does waza get resolve and cache remote graders?
• Version pinning semantics (exact, range, latest)?
• Offline/air-gapped fallback from local cache?
• How to handle transitive dependencies (grader A depends on dataset B)?
• Authentication for private repos?

Parent: #385

[spboyer]

Originally by Richard Park (@richardpark-msft) on 2026-02-24:

I'd actually been considering if we need to continue to use the sandbox for Python (and JS). They're weak sandboxes, but something we could think about if we're going to allow externally sourced graders: https://github.com/spboyer/waza/issues/183

[spboyer]

Originally by Richard Park (@richardpark-msft) on 2026-02-24:

Another option we can consider if supporting venv's and package.json's, letting users import potential graders from their respective package registries.

[spboyer]

Validation update on main:

• Repro attempt with an eval containing graders: - ref: github.com/waza-evals/fact@v1.0 fails schema/load (field ref not found in type models.rawGraderConfig).
• This confirms Go-module-style grader refs are not yet supported.

Additional information needed to validate implementation details:

1. Reference grammar and supported version selectors (exact/range/latest).
2. Cache location + offline behavior expectations.
3. Auth model for private refs.
4. Dependency model (whether transitive refs are supported in v1).}

[spboyer]

Validation update:

• Reproduced on current main (4d32191c) by checking command/schema surface:
  • No waza registry command implementation is present.
  • Eval schema currently models local grader configs and does not expose remote ref style grader references.

Additional information needed to move from spike to implementation-ready work:

1. Preferred versioning semantics (exact-only vs ranges).
2. Cache/offline policy requirements (TTL, pinning, lockfile expectations).
3. Auth model for private registries/repos.