Repository navigation
refactor: move plugin validation mechanics to Python - #204
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Substantial plugin validation still relied on duplicated shell fixtures, text parsing, and separate Unix/Windows installation probes despite the Python mechanics policy. Test drivers also remained in shell after their fixture mechanics had moved to Python.
What changed
bashand/bin/bash..git/eval-checks/.Verification
e4fba44: all 163 Python quality jobs passed, including native Windows fresh-install validation; documentation CI passed.bun run check:python: passed across all registered packages, 1,448 tests, including 189 Git and 141 Langfuse tests; formatting, lint, strict typing, and statement/branch coverage gates passed.bun test evals/runner/sandbox.test.ts evals/runner/fixture.test.ts evals/runner/review-route-eval-checks.test.ts: 38 passed for the eval-isolation change.bun run lint,bun run lint:ts,bun run lint:shell,bun run typecheck,bun run check:decisions, andbun run check:docs: passed.uv run --quiet --frozen --no-dev --project <plugin>/backend python <plugin>/backend/tests/fresh_install.py; Langfuse exercised its registered hook command. Hook execution tests passed with bothbashand/bin/bash.filter-readonlyand IAmove-procedure-to-skillpassed on Codex and Claude; reviewreviewer-route-overridepassed on Codex, including the final protected layout. All changed eval fixtures prepared successfully; dry preparation is not behavioral pass evidence.Review notes
Focus review on fixture behavior parity and the separation between candidate access and grading access. Production review instructions and routing implementation are unchanged.
Native Windows support for the Langfuse backend and hook entrypoint is explicitly separate in #205; the Unix launcher and locking implementation remain unchanged. The README now states that limitation. Other plugins retain their Linux/macOS/Windows CI matrices; native Linux/Windows and Bash 5 were not verified locally. Live evals are small smoke samples, not evidence of a reliability improvement. Raw eval evidence remains under ignored local results paths.
Checklist
CONTRIBUTING.md, including the contributionlicensing terms.
this change does not affect a capability invariant.
behavior.
change does not affect plugin content.
bun run check:python, or this change does not affect registeredPython packages or their repository quality infrastructure.