Repository navigation
Record the first runs of scenarios 1 to 10 under Claude Code - #27
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
The host coverage log in
tests/scenarios.mdnow records a first run of scenarios 1–10 under Claude Code, so every scenario has a row.Why
The log was empty for scenarios 1–10. Each was run once with the skill and once without (baseline), on a workspace built to the scenario's setup, with
--setting-sources projectso a user-level install could not shadow the copy under test.Each Partial or Fail row links an issue that describes what the run showed and the fix or decision it needs. Scenario 5's issue is a decision, not a bug: its invariant conflicts with update mode applying the README standard.
Checks
node --test tests/repository.test.mjspassesnode --test tests/evals/*.test.mjspassesCHANGELOG.mdupdated — not needed; the log records runs, it changes no behaviourSKILL.mdis changed everywhere it appears inreferences/— no rule changedClaims this adds to the documentation
Each row is a claim about one run. Evidence: the workspaces, the before copies, and both agents' output were kept locally for the run; they are not committed, and the scenario text is enough to rebuild each setup. Scenario 7's with-metadata variant was not run and the row says so.