Skip to content

Harden Turntable review workflow - #1

Merged
Luigi1010101 merged 1 commit into
mainfrom
agent/improve-turntable-skill
Jul 9, 2026
Merged

Harden Turntable review workflow#1
Luigi1010101 merged 1 commit into
mainfrom
agent/improve-turntable-skill

Conversation

@Luigi1010101

@Luigi1010101 Luigi1010101 commented Jul 9, 2026

Copy link
Copy Markdown
Owner

Summary

  • separate severity, evidence status, and confidence in the Turntable review contract
  • require atomic findings, explicit review scope, reproducible receipts, limitations, and a final hallucination gate
  • add read-only defaults and authorization boundaries for live-system verification
  • correct research and README claims so Turntable is presented as research-informed, not independently proven
  • make installation repeatable without creating a nested skill directory
  • add six hidden-oracle benchmark cases, paired baseline scoring, eight tests, and pinned GitHub Actions validation

Why

The original skill could conflate confidence with verification status, omit required evidence fields, overstate the research basis, and encourage verification without sufficiently explicit operational boundaries. It also lacked a repeatable way to compare Turntable with an ordinary review.

Validation

  • Codex skill validator passed for the repository and installed copies
  • python3 evals/benchmark.py check
  • python3 -m unittest discover -s evals -p 'test_*.py' (8 tests)
  • Python compilation checks for the benchmark and tests
  • repeated-install test confirmed no nested turntable/turntable directory
  • behavioral checks covered a verified cross-tenant defect, a clean control, and an unauthorized live-action request
  • repository and installed $turntable copies match exactly

Status

Turntable remains explicitly experimental. The benchmark provides a way to test superiority claims; this change does not claim those results in advance.

Note

Low Risk

Overview
This PR hardens the Turntable review workflow by restructuring its contract to require separate severity, evidence status, and confidence fields, along with atomic findings, explicit review scope, reproducible receipts, documented limitations, and a final hallucination gate. It tightens operational safety by adding read-only defaults and explicit authorization boundaries for live-system verification, while correcting the README and research notes to present Turntable as research-informed rather than independently proven. Installation is made repeatable without creating a nested skill directory, and a new CI workflow plus an evals/ suite (benchmark, cases, and tests) are introduced to validate the skill end-to-end.

Written by Gitzilla for commit bed7529. This will update automatically on new runs. Configure in the Gitzilla dashboard.

@Luigi1010101
Luigi1010101 marked this pull request as ready for review July 9, 2026 22:42
@Luigi1010101
Luigi1010101 merged commit 90b86ff into main Jul 9, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant