Repository navigation
What a proposal does: behaviour before and after - #18
Merged
Merged
Conversation
Agent proposals now open on a behaviour tab instead of the code diff.
Compare behaviour runs every example on the base commit and with the
proposal's files, in the browser, and compares every output cycle by cycle
(the extra cycles after each example included). Each difference is put in
words (lib/compare.ts): a shift of the same behaviour reads "led now
changes 1 cycle earlier", anything else "busy at cycle 40 was 1, now 0
(different in 3 cycles)"; with no difference it says the change only
touches cases the examples don't try.
For editors it also compares how well each version is tested: examples
passing, coverage and each state machine's moves added or removed, from
POST /agent/{id}/compare (design::quality runs the gate on both trees).
Checked in the browser with two test proposals added to the local database
(no model call): counting to N-2 was described cycle by cycle with the
first example going from passes to fails; starting the counter at 1 was
described as the LED changing 1 cycle earlier.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A non-expert reviewing an AI proposal sees a code diff they can't read, plus "all examples pass". Passing examples don't say what else changed. They should approve a behaviour change, not a code diff.
What
lib/compare.ts):lednow changes 1 cycle earlier";busyat cycle 40 was 1, now 0 (different in 3 cycles)";POST /agent/{id}/compare(design::qualityruns the gate on both trees).Checks
cargo test --workspace(122), clippy, fmt; web typecheck, lint, build.lednow changes 1 cycle earlier".🤖 Generated with Claude Code