v1.2.1 - v1.5.0: Tier 2 regression harness, full scenario coverage, second curriculum - #3
Merged
Conversation
Every prior scenario (01-05) and the 1.2.1 real-install smoke test picked the existing bundled python-basics track at Track Setup Round 0. This scenario instead picks "Build a custom track" and scripts the full Round 1-4 interview, targeting curriculum generation completeness, axis-level mapping from the interview answers, and confirm-before-save behavior -- the last major untested path per issue #2. Live validation run in progress; fixes to follow based on what it finds, same pattern as scenarios 01-05. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019FNUmdkni7KChUL6s7hFvB
Scenario 06 (custom-track creation, Track Setup Rounds 1-4) passed 8/8 on its first live run -- the first scenario in this harness's history not to need a fix first. The generated curriculum (curricula/rust-cli-grep.md) had a real, smoke-tested verify command, correct axis mapping from the interview answers, and confirm-before- save was verified by checking the file didn't exist until after the learner confirmed. The run's one soft finding: SKILL.md's Track Setup interview never explicitly directs Sage to derive the `consequence` axis field -- only mastery (Round 3) and intent (Round 4) have a stated mapping. Sage got it right by judgment this run, not by following an instruction. Added an explicit rule for deriving consequence from Round 2's stated project stakes. Pre-release checklist bumped from "all five" to "all six" scenarios. Version bumped to 1.3.0. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019FNUmdkni7KChUL6s7hFvB
Scenario 04 only ever tested declining the recalibration offer; the accept branch (write axis_overrides, confirm the change) had never been live-tested. New scenario 07 triggers low_hint_streak >= 3, accepts the bump, then drives a second streak past threshold to confirm a later offer is measured against the newly-applied level. 9/9 assertions passed on the first live run. Two real spec gaps surfaced along the way and got fixed in SKILL.md: - The accept branch never said whether the triggering streak resets, unlike decline. Left alone, an unreset streak could immediately re-fire the same offer against the level Sage just applied. - The phase-transition question's 5th option (the recalibration offer) didn't say whether picking it also means "start Phase N+1" or is a standalone choice. Clarified: it means both. One soft finding (Rule 10 topic-key derivation ambiguity for coordinate-noun bullets) was left unfixed -- orthogonal to axis recalibration, recorded in CHANGELOG for a future pass instead of guessed at here. Pre-release checklist bumped from "all six" to "all seven" scenarios. Version bumped to 1.4.0. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds curricula/rust-cli.md -- a Rust CLI track building toward Ferrogrep, deliberately calibrated opposite to python-basics (mastery: medium, consequence: high, intent: output vs. low, low, growth) so switching between them exercises an actual posture change, not just a second copy of the same shape. New scenario 08 checks what /sage-tracks and /sage-switch have never had a chance to get wrong: progress isolation, hint-streak isolation, and axis-posture isolation between two concurrently-tracked courses, plus /tracks status accuracy across both. 8/8 PASS on the first live run (see #2's "out of scope for this pass" list). One real spec gap surfaced while drafting the scenario and got fixed in SKILL.md: Track Management gave /tracks and /switch one line each, thin enough that status vocabulary and switch semantics (resume vs. initialize, never re-onboard) were only inferable by chaining together Progress Rule 9 and Track Setup's gating language. Added an explicit paragraph. Toolchain limitation disclosed in CHANGELOG rather than hidden: the live-run machine had no Rust toolchain, so the one rust-cli exercise was verified by manual trace instead of a real rustc run. Pre-release checklist bumped from "all seven" to "all eight" scenarios. Version bumped to 1.5.0. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hi @luxsolari! 👋
Your private repo does not have access to Sourcery.
Please upgrade to continue using Sourcery ✨
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #2 and its two named follow-ups. Covers everything on
main-yet2nosince it last merged intomainvia #1:tier1-checks.yml), plus the real-install smoke test.rust-cli.md, deliberately opposite axis calibration frompython-basics) + scenario 08:/sage-switch//sage-tracksacross two real tracks. 8/8 first-run pass. Fixed a thin spec gap in Track Management (status vocabulary, switch resume/init semantics).Every scenario run found something real and fixed it in
SKILL.mdbefore merging — same discipline as the original harness rollout. Full detail per release inCHANGELOG.md.Test plan
python3 tests/test_check_progress_schema.py— 7/7 pass (Tier 1, also runs in CI)CHANGELOG.mdcurricula/python-basics.mdandcurricula/rust-cli.mdboth confirmed unchanged by cross-track switching (scenario 08)🤖 Generated with Claude Code