Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
187 lines (130 loc) · 12.6 KB

CONTRIBUTING.md

File metadata and controls

187 lines (130 loc) · 12.6 KB
Raw

Contributing to idstack

Quick start

  1. Fork the repo and clone your fork
  2. Create a branch: git checkout -b my-skill
  3. Add your skill (see below)
  4. Run bin/idstack-gen-skills to generate the skill files from your template
  5. Run ./setup to register it
  6. Run the test suite (see Testing) to verify
  7. Submit a PR

How skills work

One directory, one source file, one generated file per target:

skills/your-skill/
├── SKILL.md.tmpl  # Source — this is the file you edit
└── SKILL.md       # Generated by bin/idstack-gen-skills — do not edit

That's the entire skill. No backend, no config files, no registration step beyond ./setup. Edit the .tmpl and regenerate — a hand-edit to SKILL.md is overwritten on the next bin/idstack-gen-skills run, and the smoke test fails on the drift in the meantime.

SKILL.md structure

1. YAML frontmatter:

---
name: your-skill
description: |
  What this skill does, in 2-3 lines. (idstack)
allowed-tools:
  - Bash
  - Read
  - Write
  - Edit
  - Glob
  - Grep
  - AskUserQuestion
---

The bare name field becomes the slash-command suffix: name: course-import is invoked as /idstack:course-import. The installed plugin handles the idstack: namespace; do not prefix the name yourself.

2. {{PREAMBLE}} placeholder — this is replaced by bin/idstack-gen-skills with the shared preamble (update check, manifest check, context recovery).

3. Skill implementation — the rest of the file is Markdown that defines the skill's workflow, decision trees, and outputs.

4. {{MANIFEST_SCHEMA}} placeholder — replaced with templates/manifest-schema.md.

5. {{IDSTACK_RESOLVE}} placeholder — replaced with templates/snippets/idstack-resolve.sh. Put this at the top of every bash block that calls $_IDSTACK/bin/..., not just the first. Bash blocks run in separate shells, so a value derived in an earlier block isn't there in a later one. Skipping it is how learn and course-export ended up pointing bin/ calls at a nonexistent directory for marketplace installs.

6. Timeline logging — at the end, a section that logs the session to .idstack/timeline.jsonl with skill-specific fields.

Template system

Skills use a template system to share common preamble code. Edit {skill}/SKILL.md.tmpl, not SKILL.md directly. The generated SKILL.md files have an <!-- AUTO-GENERATED --> header.

After editing a .tmpl file, templates/preamble.md, templates/manifest-schema.md, or templates/snippets/idstack-resolve.sh, regenerate:

bin/idstack-gen-skills            # write the regenerated files
bin/idstack-gen-skills --dry-run  # check only — exits non-zero if anything is stale

Commit the regenerated files in the same commit as the template change. The smoke test includes a freshness check that fails if any SKILL.md is stale.

Embedded Python must run on 3.9

Any Python embedded in templates/preamble.md or a skill body has to parse on Python 3.9 — that is what macOS ships, and it is the interpreter most designers will hit. An f-string reusing a quote inside its replacement field is legal from 3.12 (PEP 701) and a SyntaxError before it, which is exactly how context recovery stayed silently broken for several releases. test/test-preamble-python.sh runs every embedded block on both 3.9 and 3.12.

Shared shell libraries

Logic used by more than one script — or worth a unit test — belongs in bin/lib/, sourced by its callers rather than copy-pasted:

  • bin/lib/version-classify.sh — version comparison, shared by setup and bin/idstack-doctor
  • bin/lib/plugin-status.sh — parses claude plugin list down to idstack's own entry

Point the test at the shipped file, never at a copy of the logic. The version classifier drifted across three PRs while a mirrored copy inside its own test kept passing green.

The dual-output contract

Every skill that produces findings writes both:

  1. JSON section in .idstack/project.json — system state for downstream skills, the pipeline orchestrator, and bin/idstack-status.
  2. HTML report at .idstack/exports/<course-slug>/<skill>.html — the branded, self-contained human view. Follow the visual contract in templates/report.html.tmpl and the content contract in templates/report-format.md: observation → evidence → why-it-matters → suggestion, with severity (critical|warning|info) and evidence tier (T1–T5) on every finding. The skill writes the relative path back into its section's report_path field, and copies templates/assets/idstack.css into the course folder so the deliverable is self-contained when zipped.

Phrase recommendations as suggestions ("consider…"), not directives. Cite every recommendation with [DomainCode-N] [Tier]; uncited claims belong in Limitations or Notes, not Findings.

Manifest-write rules

  • Validate JSON on read. If malformed, report the error and stop. Never silently overwrite.
  • Own your section only. Read the full manifest, modify only the section your skill owns, preserve everything else.
  • Update the updated timestamp on every write.
  • Use bin/idstack-manifest-merge for the write path. It's section-scoped, atomic (tempfile + rename), preserves foreign sections, and validates against the canonical schema in templates/manifest-schema.md. Inline full-manifest Edit is the deprecated fallback.
  • One documented exception. needs-analysis and course-import keep the Read-modify-Write path because each writes several co-owned sections in one pass, which whole-section merge cannot express. Both say so inline. Don't convert them, and don't copy the pattern into a skill that owns a single section.
  • Standalone runs need bin/idstack-migrate --init first. On a missing manifest, plain idstack-migrate is a no-op, so the merge that follows exits 4 and the skill's results are silently never persisted. --init creates a canonical manifest with every section at its default.

Evidence standards

Every recommendation must cite its evidence tier:

Tier Meaning
T1 Meta-analyses, RCTs
T2 Quasi-experimental with controls
T3 Systematic reviews of mixed evidence
T4 Observational, no comparison group
T5 Expert opinion, theoretical frameworks

Use domain codes from evidence/references.md (e.g., [Alignment-14] [T1]). Stronger evidence takes precedence when tiers conflict.

Interaction pattern

  • Ask one structured question at a time, using the AskUserQuestion tool. The preamble's "Interaction Conventions" section defines this protocol.
  • Never batch multiple questions
  • Let users work through the workflow at their own pace
  • Write next-step text in the namespaced form, /idstack:foo. A bare /foo is not a command Claude Code resolves, and smoke-test fails on one.

Testing

bin/idstack-gen-skills --dry-run  # Generated files up to date? Run this first.
./setup                           # Register your new skill with Claude Code
./test/smoke-test.sh              # Verify the install

Every suite below runs in CI (.github/workflows/test.yml) on every push and pull request. Run whichever ones your change touches locally; CI runs them all:

Suite Covers
test/smoke-test.sh Install, SKILL.md freshness, YAML frontmatter, version agreement across VERSION / plugin.json / CHANGELOG.md, canonical manifest section names, /idstack: namespacing, resolve-snippet lockstep, schema migrations, bash -n on every script
test/integration-test.sh Behavioral tests across the bin/ scripts; also asserts the suite leaves your working tree untouched
test/test-setup.sh ./setup behavior — flag parsing, scope selection, all three legacy-cleanup shapes, failure handling. Runs against a repo copy with a fake $HOME and a stub claude
test/test-doctor.sh bin/idstack-doctor — every PROBLEM/WARNING branch and the exit contract. Runs against a repo copy with a pinned PATH, so "claude not found" means genuinely not found
test/test-status.sh bin/idstack-status, including the --readiness export gate. Each threshold is probed at its own boundary with the other two held passing
test/test-manifest-merge.sh bin/idstack-manifest-merge
test/test-version-classifier.sh bin/lib/version-classify.sh
test/test-plugin-status.sh bin/lib/plugin-status.sh
test/test-preamble-python.sh The preamble's embedded Python, on 3.9 and 3.12
test/test-extension.sh Chrome extension packaging and static analysis, plus eight node unit suites under test/test-*.js
test/test-rendered-landing.js Renders docs/index.html in headless Chrome across 11 widths and asserts the rendered outcome: no horizontal scroll, 44px touch targets, breakpoint column counts, sticky nav. Catches regressions the text suite cannot see (selector lists, @container, nested or print-only media queries, a <style> inside an HTML comment). Skips loudly with no browser
test/test-responsive-landing.js Responsive and mobile-ergonomics invariants for docs/index.html — fluid tokens, notch-safe gutters, breakpoint-scoped rules, touch targets. Runs on node, via smoke-test.sh
python3 test/check-evidence-cards.py . Verifies landing page evidence card study counts and tier ranges against evidence/references.md
python3 test/check-doc-accuracy.py . Validates version agreement, manifest schema version, binary/flag references, link targets, and surface accuracy across docs
test/mutation-test.sh Reintroduces each known defect into a throwaway copy and asserts the guarding test fails. Add a mutation here whenever you fix a bug — it is what proves your new test would have caught it

The CI matrix is ubuntu on Python 3.9 and 3.12, plus macOS on 3.12 for BSD grep/sed/awk differences. mutation-test.sh runs once, pinned to 3.9. Do not skip hooks or push with a red suite.

Writing a new assertion

test/test-helper.sh is sourced by every bash suite that runs assertions itself and owns the counters and the assertion (test-extension.sh delegates to node and asserts nothing of its own; the node suites cannot source it and accumulate their own problems instead):

. "$(dirname "$0")/test-helper.sh"

check "description"            "some-command"              # must exit 0
check "rejects a bad payload"  "$MERGE --section bogus"  3  # must exit 3
check "names the failing skill" "$STATUS --readiness"    0  "INCOMPLETE"

Arguments 3 and 4 are optional: an expected exit code, and an ERE the output must match. On failure the first five lines of output are printed. Do not add a private PASS=0/check() to a suite — smoke-test asserts none exist, and a mutation proves that guard works. Nine suites each had their own copy once and they drifted: two spelled it assert, and one swallowed failure output entirely, so CI reported a bare FAIL with nothing to act on.

Two suites keep a differently-shaped wrapper because their assertion is not "run a command": check_version compares a version string to a classification, check_listing feeds a claude plugin list listing to plugin_is_enabled. Both are named so they cannot shadow check.

Then test it manually in Claude Code: /idstack:your-skill.

What makes a good PR

  • Focused scope — one skill per PR
  • Evidence-backed — recommendations cite research, not opinions
  • Tested — the suite passes, manual test in Claude Code works, and a bug fix ships with both the test that catches it and a mutation in test/mutation-test.sh that proves the test works
  • Regenerated — bin/idstack-gen-skills --dry-run is clean, with generated files committed alongside the template change
  • Standalone — skill works without a manifest (ask questions as fallback)
  • Pipeline-aware — if upstream data exists in the manifest, use it to enrich recommendations

Looking for something to work on?

Two places, on purpose. TODOS.md is the long-lived feature backlog — deferred work with the reasoning and dependencies attached. Open issues hold work that came out of a specific code review or PR triage, so it stays next to the evidence that produced it; TODOS.md links the current ones. ROADMAP.md is the user-facing view.

Read DESIGN.md before changing anything visual; it is the source of truth for the report stylesheet and the landing page.

Questions or feedback?

Fill out this form — no GitHub account needed. You can also open an issue.