Harden Turntable review workflow - #1
Merged
Merged
Conversation
Luigi1010101
marked this pull request as ready for review
July 9, 2026 22:42
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Why
The original skill could conflate confidence with verification status, omit required evidence fields, overstate the research basis, and encourage verification without sufficiently explicit operational boundaries. It also lacked a repeatable way to compare Turntable with an ordinary review.
Validation
python3 evals/benchmark.py checkpython3 -m unittest discover -s evals -p 'test_*.py'(8 tests)turntable/turntabledirectory$turntablecopies match exactlyStatus
Turntable remains explicitly experimental. The benchmark provides a way to test superiority claims; this change does not claim those results in advance.
Note
Low Risk
Overview
This PR hardens the Turntable review workflow by restructuring its contract to require separate severity, evidence status, and confidence fields, along with atomic findings, explicit review scope, reproducible receipts, documented limitations, and a final hallucination gate. It tightens operational safety by adding read-only defaults and explicit authorization boundaries for live-system verification, while correcting the README and research notes to present Turntable as research-informed rather than independently proven. Installation is made repeatable without creating a nested skill directory, and a new CI workflow plus an
evals/suite (benchmark, cases, and tests) are introduced to validate the skill end-to-end.Written by Gitzilla for commit bed7529. This will update automatically on new runs. Configure in the Gitzilla dashboard.