Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "sage-instructor",
"version": "1.5.0",
"version": "1.6.0",
"description": "Adaptive programming instructor — structured courses with discovery-first teaching, AskUserQuestion interactions, progress tracking, and pluggable curricula. Powered by the Three Axes Framework.",
"author": {
"name": "Lux Solari",
Expand Down
70 changes: 70 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,75 @@
# Changelog

## [1.6.0] — 2026-07-02

### Added
- **Grounding-research step in Track Setup.** Before generating a custom
curriculum, Sage now judges whether the stated topic is a fast-moving
library/framework with a versioned, changing API (vs. stable fundamentals
training data already gets right) and, if so, uses WebSearch/WebFetch to
pull current docs before writing a single exercise — recording what it
consulted in a new optional `sources` YAML field (see `TEMPLATE.md`).
Motivated by the original "use an LLM to teach me stuff" premise: a
generated curriculum for something like Raylib is only as good as how
current its underlying facts are.
- **Track Setup topic shortcut.** When the learner's trigger message already
names a specific topic (e.g. "teach me Go, let's build something" instead
of a bare `/sage-start`), Round 0 no longer asks blind: a topic matching no
bundled curriculum skips Round 0's question entirely (and Round 1, since
"what do you want to learn?" was already answered); a topic matching one
compresses Round 0 into a direct confirm naming that track instead of the
generic list. A genuinely ambiguous topic still asks normally.
- **`tests/scenarios/09-grounding-research-trigger.md`.** Tests the
grounding-research trigger branch (C++/Raylib) — confirms a real
WebSearch/WebFetch call happens, the `sources` field gets populated with
genuine references, and at least one generated API element traces back to
what was actually found (not a plausible-sounding invention).
- **`tests/scenarios/10-track-setup-topic-shortcut.md`.** Two-part scenario
for the new topic shortcut: Part A (Elixir, no bundled match) proves the
skip branch; Part B (Python, matches `python-basics`) proves the compress
branch still offers — and honors — the custom-track alternative rather
than railroading toward the match.

### Changed
- **`tests/scenarios/06-custom-track-creation.md`** gained a no-`sources`
assertion (proving the grounding-research step correctly skips for stable
topics), and its trigger message was reverted to topic-neutral ("Hey, I
want to pick up a new skill — what have you got?", topic revealed at Round
1 instead). Needed because the scenario's own regression target — Round 0
presenting the full option list — only holds when no topic was pre-stated;
a topic-naming trigger now legitimately changes Round 0's behavior under
the new shortcut rule. Also swapped its example topic from Rust to Go: the
original topic collided with the `rust-cli` curriculum added in 1.5.0,
making Round 0 offer three options instead of the two the scenario assumed
— a real staleness bug this session's own earlier release introduced.
- **`tests/scenarios/09-grounding-research-trigger.md`**'s trigger message
was likewise reverted to topic-neutral for the same reason, keeping it
focused on one regression target (grounding-research) rather than
entangling it with the new shortcut rule.

### Verified
- **`09-grounding-research-trigger`**: 10/10 PASS. Real tool calls confirmed
(WebSearch + two WebFetch calls against raylib.com and its GitHub wiki);
the generated `verify` command's linker flags and the exercises' API
calls (`InitWindow`, `DrawCircle`, `RAYWHITE`, ...) all traced back to the
actual fetched content, including catching that Raylib is currently on
v6.0 — a genuine grounding signal, not a coincidence.
- **`10-track-setup-topic-shortcut`**: 10/10 PASS (5/5 each part). Skip and
compress branches both confirmed; picking custom despite a compressed
match correctly avoided defaulting to the bundled track.
- **`06-custom-track-creation`** (re-run after the Go/topic-neutral fix):
9/9 PASS.
- Soft findings not acted on this pass: the "genuinely ambiguous" branch
of the topic-shortcut rule has no worked example and remains untested —
candidate for a future scenario if the shortcut sees real use; the
grounding-research rule doesn't say how many sources are "enough" to
prove a claim was checked rather than assumed; a scratch-test-generated
curriculum's `verify` command was Linux-flag-specific with no
`prerequisites` callout — a curriculum-quality nit in generated output,
not a `SKILL.md` defect.
- Pre-release checklist (`tests/README.md`/`CONTRIBUTING.md`) updated from
"all eight" to "all ten" scenarios.

## [1.5.0] — 2026-07-02

### Added
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Open an issue describing: what you expected, what happened, and your Claude Code
`python3 tests/test_check_progress_schema.py` (also run in CI) covers the
deterministic Tier 1 checks and needs no agent.
5. Bumping the plugin version? Run `tests/README.md`'s full pre-release
checklist (all eight Tier 2 scenarios) first.
checklist (all ten Tier 2 scenarios) first.
6. Submit a PR with a clear description

## Code of Conduct
Expand Down
9 changes: 8 additions & 1 deletion skills/sage-instructor/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,13 +89,20 @@ Generate `.sage-profile.md` in the project root (using `references/learner-profi
**Round 0 — Existing tracks** (only when Track Setup was triggered by onboarding — i.e. no `active_track` yet. Skip entirely when the learner explicitly ran `/new-track`: that command already states custom-track intent, don't second-guess it. Also skip if `curricula/` has nothing besides `TEMPLATE.md`.)
If applicable, ask: "Want to start one of these, or build something custom?" Options: one per existing curriculum (using its `title`), plus "Build a custom track." If the learner picks an existing one, set it as `active_track`, confirm, and skip straight to Phase 0 — the rest of Track Setup is only for the custom path.

**Round 1** — "What do you want to learn?" (free text)
**If the trigger message already named a specific topic** (e.g. "teach me Go, let's build something" rather than a bare `/start`), don't ask Round 0 blind — use what was already said:
- Clearly matches no existing curriculum's `title`/`teaches` → skip the question entirely, there's nothing to choose between. Say so in one line ("No bundled track covers Go yet, so let's build you a custom one.") and go straight into the custom-track interview.
- Clearly matches one → still ask, but as a direct confirm naming that track, not the generic list: "Sounds like Python — want the bundled Python Foundations track, or build a custom one instead?" This is still a real choice (structured bundled track vs. one generated fresh) and shouldn't be assumed away.
- Genuinely ambiguous which it matches, or matches more than one plausibly → ask Round 0 as normal rather than guessing.

**Round 1** — "What do you want to learn?" (free text) — skip asking if the trigger message already answered this; carry that stated topic forward as Round 1's answer instead of asking the learner to repeat it.
**Round 2** — "Is there a project this feeds into?" Options: Yes (follow up), No — general skill building
**Round 3** — "How much do you know already?" Options: From scratch, Basics but rusty, Intermediate, Know a related language
**Round 4** — "What's the priority?" Options: Pure learning (Growth), Build while learning (Balanced), Get productive fast (Output)

Rounds 3 and 4 map directly to the curriculum's `mastery` and `intent` axis fields (Round 4's options are literally labeled with the axis values). `consequence` has no dedicated round — infer it from Round 2's answer instead: no stated project, or a personal/learning-only one → `low`; a project the learner describes as shared, user-facing, or otherwise stakes-bearing → `medium` or `high`, using judgment on what was actually described, not a fixed default.

**Before generating, judge whether the topic needs grounding.** Training data goes stale fastest for fast-moving libraries/frameworks with versioned, changing APIs (a game/UI library, a cloud SDK, a build tool), and is thinnest for niche or narrow domains — Round 1's answer is the signal. When it applies, use WebSearch/WebFetch to pull the current official docs or getting-started material for what's being taught before writing a single exercise, so topics, exercise asserts, and the `verify` command reference the API as it actually exists now, not a plausible-sounding but possibly renamed/removed recollection. Skip this for stable fundamentals a language's core syntax rarely needs re-checking (e.g. "Python control flow," "SQL joins") — don't burn a research round on something training data already gets right. Record what was consulted in the generated curriculum's `sources` field (see `TEMPLATE.md`) when this step ran; omit the field entirely when it didn't. Either way, briefly tell the learner what (if anything) was checked before showing the generated curriculum, so they can flag it if a source looks wrong or outdated.

Generate curriculum from `TEMPLATE.md`, confirm with learner, save to `curricula/<track>.md`.

---
Expand Down
7 changes: 7 additions & 0 deletions skills/sage-instructor/curricula/TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,13 @@ teaches:
- Raylib
- CMake

# Optional — real-world references Sage consulted while generating this
# curriculum (Track Setup's grounding-research step). Only present when that
# step ran; omit entirely for tracks generated from stable, well-known
# fundamentals that didn't need a live check. Title + URL is enough.
sources:
- "Raylib cheatsheet — https://www.raylib.com/cheatsheet/cheatsheet.html"

# How Sage verifies an exercise before marking it complete (Step 6b).
# Either a shell command template — {file} is substituted with the exercise's
# entry point — or the literal string "manual" for exercises with no
Expand Down
2 changes: 1 addition & 1 deletion tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ per-commit CI check.
### Pre-release checklist

Before bumping the plugin version (`.claude-plugin/plugin.json` +
`CHANGELOG.md`), run **all eight** scenarios in `scenarios/` — not just the
`CHANGELOG.md`), run **all ten** scenarios in `scenarios/` — not just the
ones nearest whatever you changed. Record the result (pass/fail per
assertion) in the version's CHANGELOG entry the way 1.2.0 and the harness's
own rollout did, so "we ran the harness" is checkable later instead of
Expand Down
42 changes: 29 additions & 13 deletions tests/scenarios/06-custom-track-creation.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,24 +13,32 @@ This is the last major untested path from a fresh install.
## Setup

Empty scratch directory. No `.sage-profile.md`, no `.sage-progress.json`.
`skills/sage-instructor/curricula/` contains `TEMPLATE.md` and
`python-basics.md` exactly as shipped — don't modify them for this scenario.
`skills/sage-instructor/curricula/` contains `TEMPLATE.md`, `python-basics.md`,
and `rust-cli.md` exactly as shipped — don't modify them for this scenario.
The learner's trigger message deliberately doesn't name a topic — this
scenario is testing Round 0's normal, un-tuned behavior (the full option
list), not the trigger-already-named-a-topic skip/compress path (see
scenario 10 for that). The topic (Go, which has no bundled equivalent among
these three files) is only revealed at Round 1.

## Script

1. Learner triggers Sage via natural language (not a slash command): "teach
me Rust, let's build something."
1. Learner triggers Sage via natural language (not a slash command), with no
topic stated yet: "Hey, I want to pick up a new skill — what have you
got?"
2. Profile Setup — answer each round:
- Identity: "Marco, mobile developer"
- Bridge languages: Java, Kotlin
- Experience: "Junior (1-3yr)"
- Learning style: "Hands-on first"
- Tone: "Encouraging and patient"
Confirm the generated profile looks right.
3. Track Setup Round 0 fires (curricula/ has `python-basics.md` beyond just
`TEMPLATE.md`, no `active_track` yet). Learner explicitly picks **"Build a
custom track"** — not "Python Foundations."
4. Round 1 ("What do you want to learn?"): "Rust."
3. Track Setup Round 0 fires (curricula/ has `python-basics.md` and
`rust-cli.md` beyond just `TEMPLATE.md`, no `active_track` yet, and no
topic was named yet — nothing to compress or skip). Learner explicitly
picks **"Build a custom track"** — not "Python Foundations" or "Rust CLI
Tools."
4. Round 1 ("What do you want to learn?"): "Go."
5. Round 2 ("Is there a project this feeds into?"): Yes — "A small CLI tool
that recursively greps through a directory, like a mini ripgrep clone."
6. Round 3 ("How much do you know already?"): "From scratch."
Expand All @@ -43,11 +51,11 @@ Empty scratch directory. No `.sage-profile.md`, no `.sage-progress.json`.
## Assertions

- `[mechanical]` A new file exists at `curricula/<slug>.md` (some
Rust-related track slug) with a YAML header containing every field
Go-related track slug) with a YAML header containing every field
`TEMPLATE.md` marks required: `track`, `title`, `destination`, `mastery`,
`consequence`, `intent`, `bridge_from`, `teaches`, `verify`.
- `[mechanical]` `verify` is a real, non-placeholder command appropriate for
Rust (e.g. involving `rustc` or `cargo`) — not `TEMPLATE.md`'s literal
Go (e.g. involving `go run` or `go build`) — not `TEMPLATE.md`'s literal
C++/g++ example copied verbatim, and not empty/missing.
- `[mechanical]` `.sage-progress.json`'s `active_track` matches the new
curriculum's `track` slug, and `python3 tests/check_progress_schema.py
Expand All @@ -66,6 +74,14 @@ Empty scratch directory. No `.sage-profile.md`, no `.sage-progress.json`.
structure/header) and explicitly asked the learner to confirm **before**
saving it — quote the confirmation prompt. The file must not exist on disk
before that confirmation turn (check between steps 8 and 9).
- `[mechanical]` Round 0's options included "Build a custom track." alongside
"Python Foundations" (same mechanism scenario 01 already proved works for
the other branch — this exercises picking custom instead).
- `[mechanical]` Round 0 offered all three real options — "Python
Foundations", "Rust CLI Tools", and "Build a custom track." (per `SKILL.md`
line 89, one per existing curriculum plus custom) — and the learner picked
custom (same mechanism scenario 01 already proved works for an existing-
track pick — this exercises the custom branch instead).
- `[mechanical]` The generated YAML header has **no** `sources` field. "Go"
(the language, its core syntax/tooling) is stable-fundamentals territory
per `SKILL.md`'s grounding-research rule — training data doesn't need a
live check for this, so the field should be entirely absent, not
present-but-empty. (Contrast: scenario 09 exercises a topic where the field
*should* appear.)
Loading
Loading