Skip to content

ci: gate apps/** and catalog/skill-packs.json#752

Merged
aaronjmars merged 1 commit into
mainfrom
ci-gaps-apps-and-skill-packs
Jul 20, 2026
Merged

ci: gate apps/** and catalog/skill-packs.json#752
aaronjmars merged 1 commit into
mainfrom
ci-gaps-apps-and-skill-packs

Conversation

@aaronjmars

Copy link
Copy Markdown
Collaborator

Summary

Two previously ungated surfaces in this repo:

apps/** had zero CI. Dependabot opens grouped monthly npm PRs against apps/dashboard, apps/mcp-server, and apps/webhook (.github/dependabot.yml) — nothing has ever verified those bumps leave an app compiling. New ci-apps workflow, one job per app so a red X names the broken surface:

  • dashboard — typecheck (new npm run typecheck), the 156 existing unit tests, and next build as three separate steps. Kept separate on purpose: the TS7/Next16 pin comment in dependabot.yml records a real breakage where tsc --noEmit passed but next build crashed.
  • cli — typecheck. Its tsconfig.json compiles ../dashboard/lib/**/*.ts and points typeRoots there (no TypeScript dep of its own), so the job installs the dashboard first. apps/cli/README.md documents the same.
  • mcp-servernpm run build (its dist/index.js is the published bin, not just a type-check).
  • webhooknode --check + wrangler deploy --dry-run (bundles the Worker, no Cloudflare credentials needed, so it's fork-PR-safe).

Also added the missing apps/cli entry to dependabot.yml — it was the only app with no dependency updates configured at all.

catalog/skill-packs.json had no validation. It's the one file in the repo routinely hand-edited by outside contributors — every community pack listing PR touches it directly, per the publishing checklist in docs/community-skill-packs.md. New scripts/validate-skill-packs.mjs + ci-skill-packs gate checks:

  • Registry shape: parseable JSON, repo as strict owner/repo, non-empty unique skills[] (kebab-case slugs), trust_level enum, capabilities restricted to the locked taxonomy — read directly out of ALLOWED_CAPABILITIES in bin/install-skill-pack so the two can never drift (same spirit as ci-capabilities-parity), secrets_required as UPPER_SNAKE names.
  • README parity: every registry entry has a matching row in the Community Packs table and vice versa, skill counts agree, --path flags agree with the registry's path (a mismatch here hands readers a copy-paste command that installs the wrong subtree), handles a monorepo publishing multiple packs from different subdirectories, and checks the "N community skill packs" Proof-of-work counter.
  • A security-relevant check: a self-declared trust_level: trusted that isn't in skills/security/trusted-sources.txt is rejected — --list prints its badge from the registry, but the installer decides the real scan bypass from the trusted-sources file, so an unbacked claim would advertise "security scan skipped" without ever earning it.

Confirmed the gate catches a live case: open PR #723's current diff would fail it — the README's pack counter says 10 while the registry it adds would total 13.

30 test cases added in scripts/tests/test_validate_skill_packs.sh, wired into ci-tests.yml alongside the existing suites.

Docs updated to match: .github/CONTRIBUTING.md gate table, .github/PULL_REQUEST_TEMPLATE.md checkboxes, docs/community-skill-packs.md, apps/cli/README.md.

Test plan

  • node scripts/validate-skill-packs.mjs passes against the committed registry
  • bash scripts/tests/test_validate_skill_packs.sh — 30/30 pass
  • Confirmed the gate fails against PR Community skill pack listing: sparkleware (proof-of-loadout, aeon-pulse, eth-gas-watch) #723's registry+README diff (fetched via gh api) with the correct error
  • apps/dashboard: npm run typecheck, npm test (156/156), npm run build all pass
  • apps/cli: npm run typecheck passes (after installing apps/dashboard)
  • apps/mcp-server: npm run typecheck and npm run build pass
  • apps/webhook: node --check src/worker.js and wrangler deploy --dry-run pass
  • All existing scripts/tests/*.sh/*.py suites still pass (pre-existing test_cron_due.sh failure is BSD-vs-GNU date on macOS, unrelated to this change, passes in CI's Ubuntu runners)
  • CI green on this PR (workflows added in this PR, so this is the first real run)

Two previously ungated surfaces:

- apps/** had zero CI despite Dependabot opening monthly grouped PRs
  against dashboard, mcp-server, and webhook — nothing ever verified
  those bumps left an app compiling. New ci-apps workflow: dashboard
  (typecheck + its 156 unit tests + `next build`, kept separate per the
  documented TS7/Next16 incompatibility), cli (typecheck, borrowing the
  dashboard's TypeScript since it has none of its own), mcp-server
  (build), webhook (syntax check + `wrangler deploy --dry-run`, no
  Cloudflare creds needed). Also added the missing apps/cli entry to
  dependabot.yml.

- catalog/skill-packs.json is the one file routinely hand-edited by
  outside contributors (every community pack listing), with nothing
  validating it. New scripts/validate-skill-packs.mjs + ci-skill-packs
  gate: registry shape (JSON, owner/repo format, non-empty unique
  skills[], trust_level enum, capabilities against the locked taxonomy
  read out of bin/install-skill-pack, secrets_required shape) and
  README parity (every entry has a matching table row and vice versa,
  skill counts agree, --path flags agree, handles monorepos publishing
  multiple packs, checks the "N community skill packs" counter). Also
  catches a self-declared trust_level: trusted that isn't backed by
  trusted-sources.txt, which is what the installer actually checks.
  30 test cases in scripts/tests/test_validate_skill_packs.sh, wired
  into ci-tests.

Docs (CONTRIBUTING.md, PULL_REQUEST_TEMPLATE.md, community-skill-packs.md,
apps/cli/README.md) updated to match.
@aaronjmars
aaronjmars merged commit 2c81481 into main Jul 20, 2026
7 checks passed
@aaronjmars
aaronjmars deleted the ci-gaps-apps-and-skill-packs branch July 20, 2026 16:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant