From f05c0401cb49371b9d80dde1ee494faae0f9039f Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Mon, 13 Jul 2026 21:49:39 +0100 Subject: [PATCH 01/27] Scaffold offworld marketplace, core plugin, shared rules and docs Add the org-wide Claude Code resource for Offworld Labs: - .claude-plugin/marketplace.json declaring the "offworld" marketplace with one plugin, core (./plugins/core). - plugins/core: plugin.json (v0.1.0) plus empty skills/commands/hooks/agents dirs, and one real skill (pr-description) to prove the pipeline end-to-end. - rules/: security and code-style stubs meant to be symlinked into each repo's .claude/rules/. - docs/: architecture, contracts, decisions, runbooks stubs; contracts noted as the source of truth for cross-service interfaces. - templates/: drop-in .claude/settings.json (registers the marketplace and enables core) and a starter CLAUDE.md. - README.md: install, adoption, and contribution process. Validated with `claude plugin validate` and confirmed core@offworld installs and the core:pr-description skill loads. Co-Authored-By: Claude Opus 4.8 (1M context) --- .claude-plugin/marketplace.json | 14 +++++ README.md | 64 +++++++++++++++++++++ docs/architecture.md | 10 ++++ docs/contracts/README.md | 14 +++++ docs/decisions/README.md | 10 ++++ docs/runbooks/README.md | 10 ++++ plugins/core/.claude-plugin/plugin.json | 8 +++ plugins/core/agents/.gitkeep | 0 plugins/core/commands/.gitkeep | 0 plugins/core/hooks/.gitkeep | 0 plugins/core/skills/.gitkeep | 0 plugins/core/skills/pr-description/SKILL.md | 24 ++++++++ rules/code-style.md | 13 +++++ rules/security.md | 13 +++++ templates/CLAUDE.md | 32 +++++++++++ templates/settings.json | 13 +++++ 16 files changed, 225 insertions(+) create mode 100644 .claude-plugin/marketplace.json create mode 100644 README.md create mode 100644 docs/architecture.md create mode 100644 docs/contracts/README.md create mode 100644 docs/decisions/README.md create mode 100644 docs/runbooks/README.md create mode 100644 plugins/core/.claude-plugin/plugin.json create mode 100644 plugins/core/agents/.gitkeep create mode 100644 plugins/core/commands/.gitkeep create mode 100644 plugins/core/hooks/.gitkeep create mode 100644 plugins/core/skills/.gitkeep create mode 100644 plugins/core/skills/pr-description/SKILL.md create mode 100644 rules/code-style.md create mode 100644 rules/security.md create mode 100644 templates/CLAUDE.md create mode 100644 templates/settings.json diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json new file mode 100644 index 0000000..b591e47 --- /dev/null +++ b/.claude-plugin/marketplace.json @@ -0,0 +1,14 @@ +{ + "name": "offworld", + "owner": { + "name": "Offworld Labs" + }, + "description": "Org-wide Claude Code plugins for Offworld Labs.", + "plugins": [ + { + "name": "core", + "source": "./plugins/core", + "description": "Core Offworld Labs skills, commands, agents, and hooks shared across all repos." + } + ] +} diff --git a/README.md b/README.md new file mode 100644 index 0000000..098dc20 --- /dev/null +++ b/README.md @@ -0,0 +1,64 @@ +# claude-shared + +Offworld Labs' org-wide Claude Code resource: a **plugin marketplace** (`offworld`) +plus **shared reference docs** used across every repo in the organisation. + +- `plugins/core` — the `core` plugin (skills, commands, agents, hooks). +- `rules/` — shared rules symlinked into each repo's `.claude/rules/`. +- `docs/` — on-demand org-wide docs (architecture, contracts, decisions, runbooks). +- `templates/` — drop-in `settings.json` and `CLAUDE.md` for new repos. + +## Install + +In any repo, add the marketplace and install the plugin: + +``` +/plugin marketplace add offworldlabs/claude-shared +/plugin install core@offworld +``` + +## Adopting in a consuming repo + +For zero-setup adoption, copy the template settings into the repo so everyone +who trusts the folder gets the marketplace and `core` plugin automatically: + +```bash +mkdir -p .claude +cp path/to/claude-shared/templates/settings.json .claude/settings.json +``` + +`.claude/settings.json` registers the `offworld` marketplace via +`extraKnownMarketplaces` and enables `core@offworld` via `enabledPlugins`, so +opening the repo prompts installation with no manual `/plugin` commands. + +Symlink the shared rules into the repo so they stay in sync with this repo: + +```bash +mkdir -p .claude/rules +ln -s ../../path/to/claude-shared/rules/security.md .claude/rules/security.md +ln -s ../../path/to/claude-shared/rules/code-style.md .claude/rules/code-style.md +``` + +Point the repo's `CLAUDE.md` at this repo's `docs/` for org-wide context instead +of duplicating it (see `templates/CLAUDE.md`). + +## Contributing + +Skills graduate from personal experimentation into the shared `core` plugin +through review: + +1. **Develop personally first.** Iterate on the skill in your own `~/.claude/` + until it works. +2. **Never merge a skill you haven't run.** A skill that hasn't been exercised + end-to-end does not go in. +3. **Test locally before opening the PR.** Add this checkout as a local + marketplace and reload: + ``` + /plugin marketplace add ./path/to/claude-shared + /plugin install core@offworld + /reload-plugins + ``` +4. **Open a PR** moving the skill into `plugins/core/skills//SKILL.md`. +5. **Bump the plugin version** in `plugins/core/.claude-plugin/plugin.json` on + every merged change. The version is the update signal for the entire org — + without a bump, no one receives the change. diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..a4133c2 --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,10 @@ +# Architecture + +Org-wide architecture overview for Offworld Labs. + +This document describes how services fit together across the organisation: the +high-level system map, major components and their responsibilities, and the data +flows between them. Individual repos should describe their *local* architecture +in their own `CLAUDE.md` and link here for the cross-service picture. + +> Stub — fill in the org-wide system diagram and component responsibilities. diff --git a/docs/contracts/README.md b/docs/contracts/README.md new file mode 100644 index 0000000..ee75b29 --- /dev/null +++ b/docs/contracts/README.md @@ -0,0 +1,14 @@ +# Contracts + +Cross-service interface definitions for Offworld Labs. + +The files in this directory are the **source of truth** for interfaces shared +between services: API schemas, event/message formats, shared data structures, +and versioning expectations. When two services communicate, the contract lives +here — not duplicated in each repo. + +**Consuming repos should point to these files from their `CLAUDE.md`** rather +than copying contract definitions locally. Duplicated contracts drift; a single +referenced source does not. + +> Stub — add one file per cross-service contract. diff --git a/docs/decisions/README.md b/docs/decisions/README.md new file mode 100644 index 0000000..ebff38d --- /dev/null +++ b/docs/decisions/README.md @@ -0,0 +1,10 @@ +# Decisions + +Architecture Decision Records (ADRs) for Offworld Labs. + +This directory captures significant, org-wide technical decisions: the context, +the options considered, the decision made, and its consequences. One record per +decision, kept append-only so the reasoning behind past choices stays +discoverable. + +> Stub — add one dated ADR file per significant decision. diff --git a/docs/runbooks/README.md b/docs/runbooks/README.md new file mode 100644 index 0000000..a3fc546 --- /dev/null +++ b/docs/runbooks/README.md @@ -0,0 +1,10 @@ +# Runbooks + +Operational runbooks for Offworld Labs. + +This directory holds step-by-step operational procedures: deployments, incident +response, rollbacks, and recovery playbooks. Each runbook should be concrete +enough to follow under pressure — exact commands, expected output, and +escalation paths. + +> Stub — add one runbook file per operational procedure. diff --git a/plugins/core/.claude-plugin/plugin.json b/plugins/core/.claude-plugin/plugin.json new file mode 100644 index 0000000..e8f499b --- /dev/null +++ b/plugins/core/.claude-plugin/plugin.json @@ -0,0 +1,8 @@ +{ + "name": "core", + "version": "0.1.0", + "description": "Core Offworld Labs skills, commands, agents, and hooks shared across all repos.", + "author": { + "name": "Offworld Labs" + } +} diff --git a/plugins/core/agents/.gitkeep b/plugins/core/agents/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/plugins/core/commands/.gitkeep b/plugins/core/commands/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/plugins/core/hooks/.gitkeep b/plugins/core/hooks/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/plugins/core/skills/.gitkeep b/plugins/core/skills/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/plugins/core/skills/pr-description/SKILL.md b/plugins/core/skills/pr-description/SKILL.md new file mode 100644 index 0000000..cfe6ae8 --- /dev/null +++ b/plugins/core/skills/pr-description/SKILL.md @@ -0,0 +1,24 @@ +--- +name: pr-description +description: Use when opening a pull request or when the user asks to write, draft, or generate a PR description, PR summary, or PR body. Reads the branch diff and produces a review-ready description. +--- + +# PR Description + +Write a clear, review-ready pull request description from the current branch's changes. + +## Steps + +1. Determine the base branch (usually `main`) and gather the diff: + - `git merge-base HEAD main` to find the fork point. + - `git diff ...HEAD --stat` for the file-level overview. + - `git diff ...HEAD` for the full change, and `git log ..HEAD` for commit context. +2. Read the changes closely enough to explain *what* changed and *why*, not just which files moved. +3. Produce the description with these sections: + - **Summary** — 1-3 sentences on what this PR does and the motivation behind it. + - **Changes** — bulleted list of the notable changes, grouped by area or concern. + - **Test coverage** — what tests were added or updated, what was run to verify, and any gaps. + - **Review notes** — anything reviewers should scrutinise: risky areas, trade-offs, follow-ups, migrations, or intentionally deferred work. +4. Keep it concise and factual. Do not invent testing that wasn't done — if coverage is thin, say so under Review notes. + +Output the description as Markdown ready to paste into the PR body. diff --git a/rules/code-style.md b/rules/code-style.md new file mode 100644 index 0000000..b17d9b6 --- /dev/null +++ b/rules/code-style.md @@ -0,0 +1,13 @@ + + +# Code Style Rules + +- TODO: Write minimal, self-documenting code; prefer clarity over cleverness. +- TODO: Match the existing patterns and conventions of the file you are editing. +- TODO: Cover all business logic with tests before marking work complete. diff --git a/rules/security.md b/rules/security.md new file mode 100644 index 0000000..483c6e7 --- /dev/null +++ b/rules/security.md @@ -0,0 +1,13 @@ + + +# Security Rules + +- TODO: Never commit secrets, credentials, or API keys — use environment variables or a secrets manager. +- TODO: Validate and sanitise all external input at trust boundaries. +- TODO: Keep dependencies patched and pinned; review new dependencies before adding them. diff --git a/templates/CLAUDE.md b/templates/CLAUDE.md new file mode 100644 index 0000000..dfa96b1 --- /dev/null +++ b/templates/CLAUDE.md @@ -0,0 +1,32 @@ + + +# CLAUDE.md + +## Project Overview + + + +## Build & Test Commands + + + +```bash +# install +# build +# test +# lint +``` + +## Local Architecture + + + +## Org-Wide Context + +For shared architecture, cross-service contracts, decisions, and runbooks, see +the Offworld Labs shared docs: https://github.com/offworldlabs/claude-shared/tree/main/docs diff --git a/templates/settings.json b/templates/settings.json new file mode 100644 index 0000000..3ada541 --- /dev/null +++ b/templates/settings.json @@ -0,0 +1,13 @@ +{ + "extraKnownMarketplaces": { + "offworld": { + "source": { + "source": "github", + "repo": "offworldlabs/claude-shared" + } + } + }, + "enabledPlugins": { + "core@offworld": true + } +} From c5482121119da0a764e41a419a178cffdbdbe88a Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Tue, 14 Jul 2026 09:54:07 +0100 Subject: [PATCH 02/27] Add corrected GitHub Actions workflow templates and runbook New repos that set up Claude PR review via /install-github-app hit a recurring issue: the generated workflows grant read-only tokens, so reviews run green but never post a comment. Ship the fix as reusable org assets: - templates/github-workflows/{claude-code-review,claude}.yml: drop-in workflows with pull-requests/issues set to write. - docs/runbooks/github-actions-claude-review.md: symptom, cause, fix, and the must-land-on-default-branch guard. - README: adoption section pointing new repos at the templates instead of the broken scaffold. Co-Authored-By: Claude Opus 4.8 (1M context) --- README.md | 16 ++++ docs/runbooks/github-actions-claude-review.md | 88 +++++++++++++++++++ .../github-workflows/claude-code-review.yml | 49 +++++++++++ templates/github-workflows/claude.yml | 55 ++++++++++++ 4 files changed, 208 insertions(+) create mode 100644 docs/runbooks/github-actions-claude-review.md create mode 100644 templates/github-workflows/claude-code-review.yml create mode 100644 templates/github-workflows/claude.yml diff --git a/README.md b/README.md index 098dc20..94775bb 100644 --- a/README.md +++ b/README.md @@ -42,6 +42,22 @@ ln -s ../../path/to/claude-shared/rules/code-style.md .claude/rules/code-style.m Point the repo's `CLAUDE.md` at this repo's `docs/` for org-wide context instead of duplicating it (see `templates/CLAUDE.md`). +### GitHub Actions (Claude PR review) + +Don't rely on the scaffold `/install-github-app` generates — it grants the +workflows read-only tokens, so the review runs green but never posts a comment. +Copy the pre-corrected workflows from `templates/github-workflows/` instead: + +```bash +mkdir -p .github/workflows +cp path/to/claude-shared/templates/github-workflows/claude-code-review.yml .github/workflows/ +cp path/to/claude-shared/templates/github-workflows/claude.yml .github/workflows/ +``` + +These must be committed to the repo's **default branch** before Claude will run +on PRs (a self-modification guard in `claude-code-action`). See +`docs/runbooks/github-actions-claude-review.md` for the full explanation. + ## Contributing Skills graduate from personal experimentation into the shared `core` plugin diff --git a/docs/runbooks/github-actions-claude-review.md b/docs/runbooks/github-actions-claude-review.md new file mode 100644 index 0000000..e73c152 --- /dev/null +++ b/docs/runbooks/github-actions-claude-review.md @@ -0,0 +1,88 @@ +# Runbook: Claude Code review workflow runs green but posts no comment + +## Symptom + +You set up the Claude GitHub Actions in a new repo (via `/install-github-app`). +The "Claude Code Review" job shows a green check on every PR, but **no review +comment ever appears**. + +## Cause + +The scaffold that `/install-github-app` generates grants the workflow a +**read-only** token: + +```yaml +permissions: + contents: read + pull-requests: read # can read the diff, cannot post + issues: read + id-token: write +``` + +The review runs and analyses the diff, then is **denied** when it tries to post +the comment. The job still exits green because a failed post is not a job +failure. Confirm in the run log: + +```bash +gh run view --log | grep -iE 'permission_denials_count|PullRequests:' +# "permission_denials_count": 1 ← the denied action was posting the comment +# PullRequests: read ← the read-only grant +``` + +> Note: "green check, no comment" is genuinely ambiguous between this and "clean +> code, nothing to flag." Always read the run log rather than assuming. + +## Fix + +Grant write access in **both** workflow files: + +```yaml +permissions: + contents: read + pull-requests: write + issues: write + id-token: write +``` + +The fastest path in a new repo is to skip the broken scaffold entirely and copy +the known-good templates from this repo: + +```bash +mkdir -p .github/workflows +cp path/to/claude-shared/templates/github-workflows/claude-code-review.yml .github/workflows/ +cp path/to/claude-shared/templates/github-workflows/claude.yml .github/workflows/ +``` + +## Critical gotcha: the fix must land on the default branch first + +`claude-code-action` refuses to run when the workflow file on a PR branch +differs from the copy on the default branch — a security guard so a PR can't +rewrite the review workflow to exfiltrate secrets. Log message: + +``` +Workflow validation failed. The workflow file must exist and have identical +content to the version on the repository's default branch. +``` + +Consequences: + +- You **cannot** validate a workflow change on the PR that makes the change. + Editing the workflow on a feature branch makes the review go *silent* on that + PR. +- Merge the workflow change to `main` (default branch) first. It takes effect on + the **next** normal PR opened afterward. +- A PR opened *before* the change, whose workflow now differs from the updated + default, is also skipped until rebased onto the new default. + +## Verify + +After the fix is on `main`, open a normal PR (one that does **not** touch +`.github/workflows/`) and confirm a `claude[bot]` comment appears. + +## Security note + +`pull-requests: write` on a `pull_request` trigger is sensitive only if the repo +accepts **fork** PRs — untrusted fork code could request the write token. For +private, internal-only org repos with no forks the risk is negligible. If you +expect external forks, use `pull_request_target` with author restrictions +instead. diff --git a/templates/github-workflows/claude-code-review.yml b/templates/github-workflows/claude-code-review.yml new file mode 100644 index 0000000..a2ed2cf --- /dev/null +++ b/templates/github-workflows/claude-code-review.yml @@ -0,0 +1,49 @@ +# Drop-in replacement for the workflow that `/install-github-app` generates. +# The generated scaffold ships pull-requests/issues as `read`, so the review +# runs but is denied when it tries to post — the job goes green with no comment. +# This copy grants `write` so comments post. Copy to .github/workflows/ in a new +# repo (see docs/runbooks/github-actions-claude-review.md). Requires the +# CLAUDE_CODE_OAUTH_TOKEN secret and the Claude GitHub App installed on the repo. +name: Claude Code Review + +on: + pull_request: + types: [opened, synchronize, ready_for_review, reopened] + # Optional: Only run on specific file changes + # paths: + # - "src/**/*.ts" + # - "src/**/*.tsx" + # - "src/**/*.js" + # - "src/**/*.jsx" + +jobs: + claude-review: + # Optional: Filter by PR author + # if: | + # github.event.pull_request.user.login == 'external-contributor' || + # github.event.pull_request.user.login == 'new-developer' || + # github.event.pull_request.author_association == 'FIRST_TIME_CONTRIBUTOR' + + runs-on: ubuntu-latest + permissions: + contents: read + pull-requests: write + issues: write + id-token: write + + steps: + - name: Checkout repository + uses: actions/checkout@v4 + with: + fetch-depth: 1 + + - name: Run Claude Code Review + id: claude-review + uses: anthropics/claude-code-action@v1 + with: + claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }} + plugin_marketplaces: 'https://github.com/anthropics/claude-code.git' + plugins: 'code-review@claude-code-plugins' + prompt: '/code-review:code-review ${{ github.repository }}/pull/${{ github.event.pull_request.number }}' + # See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md + # or https://code.claude.com/docs/en/cli-reference for available options diff --git a/templates/github-workflows/claude.yml b/templates/github-workflows/claude.yml new file mode 100644 index 0000000..33c456d --- /dev/null +++ b/templates/github-workflows/claude.yml @@ -0,0 +1,55 @@ +# Drop-in replacement for the workflow that `/install-github-app` generates. +# The generated scaffold ships pull-requests/issues as `read`, so @claude +# responses are denied when posting. This copy grants `write`. Copy to +# .github/workflows/ in a new repo (see +# docs/runbooks/github-actions-claude-review.md). Requires the +# CLAUDE_CODE_OAUTH_TOKEN secret and the Claude GitHub App installed on the repo. +name: Claude Code + +on: + issue_comment: + types: [created] + pull_request_review_comment: + types: [created] + issues: + types: [opened, assigned] + pull_request_review: + types: [submitted] + +jobs: + claude: + if: | + (github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) || + (github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude')) || + (github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) || + (github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude'))) + runs-on: ubuntu-latest + permissions: + contents: read + pull-requests: write + issues: write + id-token: write + actions: read # Required for Claude to read CI results on PRs + steps: + - name: Checkout repository + uses: actions/checkout@v4 + with: + fetch-depth: 1 + + - name: Run Claude Code + id: claude + uses: anthropics/claude-code-action@v1 + with: + claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }} + + # This is an optional setting that allows Claude to read CI results on PRs + additional_permissions: | + actions: read + + # Optional: Give a custom prompt to Claude. If this is not specified, Claude will perform the instructions specified in the comment that tagged it. + # prompt: 'Update the pull request description to include a summary of changes.' + + # Optional: Add claude_args to customize behavior and configuration + # See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md + # or https://code.claude.com/docs/en/cli-reference for available options + # claude_args: '--allowed-tools Bash(gh pr *)' From 10877453267389af1ba88624282c163998ccfc6a Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Tue, 14 Jul 2026 17:01:16 +0100 Subject: [PATCH 03/27] Design spec: core:setup-repo one-command new-repo setup Approved design for a core plugin skill that scaffolds a new org repo to standard (Claude enablement + CI + stack tooling), delivered as bundled plugin assets. Captures locked decisions (self-contained plugin, Python ruff py312, ts-frontend/ts-backend variants), the repo restructure folding PR #1 templates into the plugin, the setup procedure, testing, and phasing. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../2026-07-14-new-repo-setup-skill-design.md | 171 ++++++++++++++++++ 1 file changed, 171 insertions(+) create mode 100644 docs/superpowers/specs/2026-07-14-new-repo-setup-skill-design.md diff --git a/docs/superpowers/specs/2026-07-14-new-repo-setup-skill-design.md b/docs/superpowers/specs/2026-07-14-new-repo-setup-skill-design.md new file mode 100644 index 0000000..36ea553 --- /dev/null +++ b/docs/superpowers/specs/2026-07-14-new-repo-setup-skill-design.md @@ -0,0 +1,171 @@ +# Design: `core:setup-repo` — one-command new-repo setup + +**Date:** 2026-07-14 +**Status:** Approved (pending spec review) +**Repo:** offworldlabs/claude-shared + +## Problem + +Setting up a new Offworld Labs repo to org standards (Claude Code enablement, CI, +stack tooling) is manual, undocumented, and inconsistent — the sibling-repo scan +found ruff configured in only 3 of ~10 Python repos, no shared formatter/test +conventions in TS, and no repo with pre-commit, `.editorconfig`, or version +pinning. `claude-shared` should be the single place that encodes "how we set up a +repo," and the setup should be driven **by Claude Code, not a human checklist**: +open a blank repo, start Claude, say *"set this repo up per `claude-shared`,"* and +it happens. + +## Goals + +- A `core:setup-repo` skill that scaffolds a new repo to org standard in one shot. +- Covers three categories: **Claude Code enablement**, **CI/CD workflows**, + **stack scaffolding**. (Repo governance — CODEOWNERS, branch protection — is + explicitly out of scope.) +- Works offline in any repo where `core` is installed; deterministic; idempotent; + never silently clobbers existing files. + +## Non-goals + +- Governance/hygiene files (CODEOWNERS, branch protection, PR/issue templates). +- C++ scaffolding (blah2-arm, retina-spectrum) — deferred to a later phase. +- Human-readable setup checklists as the primary interface (the skill is the + interface; docs are supporting reference). + +## Decisions (locked) + +### Delivery model — self-contained plugin +The skill and **all** template assets are bundled inside the plugin and copied +into the target repo via `${CLAUDE_PLUGIN_ROOT}`. No runtime fetch. The plugin +`version` bump remains the org-wide update signal. Rules are **copied** into +repos (not symlinked), because symlinking into the plugin cache is fragile. + +### Stack defaults (from the sibling-repo scan) +- **Python** (dominant, ~10 repos): pip + `requirements.txt`; **pytest**; **ruff** + with `target-version = "py312"`, `line-length = 120`, `select = ["E","F","W"]`, + `format.quote-style = "double"`. +- **TypeScript**: npm; two variants — + - `ts-frontend`: TypeScript + Vite + Vitest + ESLint 9 (flat) + typescript-eslint + React 18. + - `ts-backend`: TypeScript + tsx + Vitest + ESLint 9 (flat) + typescript-eslint. +- Greenfield additions (nothing in the org has them today): `.editorconfig`, + optional pre-commit. + +### Prerequisite +Devs install `core@offworld` at **user scope** once, so the skill is available in +a brand-new repo that has no `.claude/settings.json` yet. The skill then writes +the repo's own `.claude/settings.json`, making the repo self-enabling thereafter. + +## Repository restructure + +The plugin becomes the single source of truth for scaffolded files. The +`templates/` directory added in PR #1 is **folded into the plugin** (PR #1 is +unmerged, so we restructure rather than duplicate). `docs/` stays at the repo root +as the human/agent knowledge hub. Final layout: + +``` +plugins/core/ + .claude-plugin/plugin.json # version bump = update signal + skills/ + pr-description/SKILL.md # existing + setup-repo/ + SKILL.md # the ordered procedure + assets/ + claude/ + settings.json # extraKnownMarketplaces + enabledPlugins + CLAUDE.md # starter template (<200-line ceiling note) + rules/ + security.md + code-style.md + ci/ + claude-code-review.yml # write-permission fix baked in + claude.yml + ci-python.yml # ruff check + ruff format --check + pytest + ci-node.yml # eslint + tsc + vitest + stack/ + python/ + pyproject.toml # ruff py312/120/EFW + format; pytest config + requirements.txt + requirements-dev.txt # ruff, pytest + gitignore # Python (shipped without leading dot; skill renames) + tests/.gitkeep + ts-frontend/ + package.json # vite, react, vitest, eslint9, typescript-eslint + tsconfig.json + eslint.config.js + vitest.config.ts + gitignore # Node + ts-backend/ + package.json # tsx, tsc, vitest, eslint9, typescript-eslint + tsconfig.json + eslint.config.js + vitest.config.ts + gitignore # Node + editorconfig # shared across all repos +docs/ # stays at root (architecture, contracts, decisions, runbooks) +rules/ # REMOVED at root — canonical copy now lives in plugin assets +templates/ # REMOVED at root — folded into plugin assets +``` + +Note: asset files that must ship as dotfiles (`.gitignore`, `.editorconfig`) are +stored **without** the leading dot in the plugin (so they aren't hidden/ignored in +the marketplace repo) and the skill renames them on copy. `claude-shared`'s own +`.github/workflows/` (the live Claude review workflows) are unchanged and separate +from the `ci/` asset templates. + +## The `setup-repo` procedure + +Ordered, idempotent, **never silently clobbers**: + +1. **Confirm target.** Is this a git repo? If not, offer `git init`. Note whether + it's empty. +2. **Determine stack.** Detect from existing files (`pyproject.toml`/`requirements*` + → python; `package.json`/`tsconfig.json` → ts) or ask: + `python` / `ts-frontend` / `ts-backend` / `none`. +3. **Claude enablement (always).** Write `.claude/settings.json`; create `CLAUDE.md` + from template **only if absent**; copy rules → `.claude/rules/`. +4. **CI.** Copy the Claude workflows + the stack's CI into `.github/workflows/`. + Remind about the default-branch guard and the `CLAUDE_CODE_OAUTH_TOKEN` secret + (cross-reference `docs/runbooks/github-actions-claude-review.md`). +5. **Stack scaffolding.** Copy the chosen stack assets, merging/skipping existing + files (e.g. don't overwrite an existing `package.json` — report and let the dev + reconcile). +6. **Shared.** Copy `.editorconfig` and the stack `.gitignore`. +7. **Report.** List every file written/skipped, then a manual follow-up checklist: + add the `CLAUDE_CODE_OAUTH_TOKEN` secret, install deps, commit workflows to the + default branch, optional branch protection. + +### Idempotency & updates +Re-running on an existing repo brings config up to the current standard: for any +file that already exists and differs, show the diff and ask before overwriting. +A clean re-run (no changes) reports "already up to standard." + +### Error handling +- Not a git repo → offer `git init`, else abort with guidance. +- Existing files → merge or skip with an explicit report; never blind-overwrite. +- No `gh` auth / can't set secret → surface as a manual step, don't fail the run. + +## Testing + +- **Asset validity:** each bundled asset is well-formed — `pyproject.toml` parses, + `package.json` is valid JSON, workflow/CI YAML is valid, `claude plugin validate` + passes for the plugin. +- **Procedure:** a script scaffolds into a throwaway temp git repo and asserts the + expected files land, then runs `ruff check` / `ruff format --check` / `pytest` + (python) and `eslint` / `tsc --noEmit` / `vitest run` (ts) against the generated + config to prove the defaults are internally consistent and pass on an empty repo. + +## Phasing + +Each phase is its own plan → implementation cycle. + +- **Phase 1** — `setup-repo` skill + Claude enablement + Claude workflows + + **Python stack** + fold PR #1 `templates/` into the plugin. Covers ~10/14 repos + and delivers the end-to-end vision for the dominant stack. +- **Phase 2** — `ts-frontend` + `ts-backend` stacks + `ci-node.yml`. +- **Phase 3 (optional, later)** — pre-commit, C++ (blah2-arm, retina-spectrum). + +## Open follow-ups (not blocking) + +- Whether the skill should also offer to run the first install (`pip install`, + `npm ci`) or leave it to the dev — lean toward reporting the command, not running it. +- Whether `CLAUDE.md` should be generated with repo-specific detail the skill + infers vs. left as the template stub — start with the stub, iterate later. From e28311c1a3ed57eb8e221f531b3bbe4db22b46d7 Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Tue, 14 Jul 2026 17:11:05 +0100 Subject: [PATCH 04/27] Spec: skill runs first install and fleshes out CLAUDE.md interactively Resolve the two open follow-ups per review: the setup skill runs the stack's first install, and writes CLAUDE.md as a stub then prompts the dev to flesh it out from a short description plus the scaffolded stack. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../2026-07-14-new-repo-setup-skill-design.md | 25 +++++++++++++------ 1 file changed, 17 insertions(+), 8 deletions(-) diff --git a/docs/superpowers/specs/2026-07-14-new-repo-setup-skill-design.md b/docs/superpowers/specs/2026-07-14-new-repo-setup-skill-design.md index 36ea553..c500491 100644 --- a/docs/superpowers/specs/2026-07-14-new-repo-setup-skill-design.md +++ b/docs/superpowers/specs/2026-07-14-new-repo-setup-skill-design.md @@ -129,9 +129,17 @@ Ordered, idempotent, **never silently clobbers**: files (e.g. don't overwrite an existing `package.json` — report and let the dev reconcile). 6. **Shared.** Copy `.editorconfig` and the stack `.gitignore`. -7. **Report.** List every file written/skipped, then a manual follow-up checklist: - add the `CLAUDE_CODE_OAUTH_TOKEN` secret, install deps, commit workflows to the - default branch, optional branch protection. +7. **Install.** Run the stack's first install (`pip install -r requirements.txt + -r requirements-dev.txt` for python; `npm install` for ts), reporting the + command and its result. Skip with a note if the toolchain is unavailable. +8. **Flesh out `CLAUDE.md`.** After writing the stub, prompt the dev for a + one-or-two-line description of what the repo does, then fill in the template's + Project Overview / Build & Test / Local Architecture sections from their answer + plus what was just scaffolded (stack, test/lint commands). Leave the stub as-is + if they skip. +9. **Report.** List every file written/skipped, then a manual follow-up checklist: + add the `CLAUDE_CODE_OAUTH_TOKEN` secret, commit workflows to the default + branch, optional branch protection. ### Idempotency & updates Re-running on an existing repo brings config up to the current standard: for any @@ -163,9 +171,10 @@ Each phase is its own plan → implementation cycle. - **Phase 2** — `ts-frontend` + `ts-backend` stacks + `ci-node.yml`. - **Phase 3 (optional, later)** — pre-commit, C++ (blah2-arm, retina-spectrum). -## Open follow-ups (not blocking) +## Resolved follow-ups -- Whether the skill should also offer to run the first install (`pip install`, - `npm ci`) or leave it to the dev — lean toward reporting the command, not running it. -- Whether `CLAUDE.md` should be generated with repo-specific detail the skill - infers vs. left as the template stub — start with the stub, iterate later. +- **First install:** the skill runs it (step 7), reporting the command and result; + skips gracefully if the toolchain is unavailable. +- **`CLAUDE.md`:** written as a stub, then the skill prompts the dev for a short + description and fleshes out the template sections from that plus the scaffolded + stack (step 8). Skipping leaves the stub. From a3780e1703b2da5b87f571ea0a965be6d58927b1 Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Tue, 14 Jul 2026 17:15:21 +0100 Subject: [PATCH 05/27] Plan: core:setup-repo Phase 1 implementation Task-by-task TDD plan for Phase 1: fold PR #1 templates into plugin assets, add Python stack + CI + editorconfig assets, a deterministic scaffold engine with an end-to-end test, and the setup-repo SKILL.md procedure. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../plans/2026-07-14-setup-repo-phase-1.md | 601 ++++++++++++++++++ 1 file changed, 601 insertions(+) create mode 100644 docs/superpowers/plans/2026-07-14-setup-repo-phase-1.md diff --git a/docs/superpowers/plans/2026-07-14-setup-repo-phase-1.md b/docs/superpowers/plans/2026-07-14-setup-repo-phase-1.md new file mode 100644 index 0000000..e64d844 --- /dev/null +++ b/docs/superpowers/plans/2026-07-14-setup-repo-phase-1.md @@ -0,0 +1,601 @@ +# `core:setup-repo` Phase 1 Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Ship a `core:setup-repo` skill that scaffolds a new org repo to standard (Claude enablement + Claude workflows + Python stack), driven by Claude Code, with all files bundled as plugin assets. + +**Architecture:** The plugin is the single source of truth. Static template files live under `plugins/core/skills/setup-repo/assets/`. A deterministic bash engine (`scaffold-repo.sh`) copies them into a target repo without clobbering existing files; the `SKILL.md` procedure orchestrates the engine plus the interactive parts (stack choice, first install, fleshing out `CLAUDE.md`). PR #1's root `templates/` and `rules/` are folded into the plugin. + +**Tech Stack:** Claude Code plugin (marketplace `offworld`), bash (engine + test harness), Python tooling defaults (ruff, pytest) as scaffolded assets. + +## Global Constraints + +- Marketplace name: `offworld`; plugin: `core`; skill namespace: `core:setup-repo`. +- Only `plugin.json`/`marketplace.json` live in `.claude-plugin/`; components at plugin root. +- Delivery model: self-contained plugin; skill references assets via `${CLAUDE_PLUGIN_ROOT}`. No runtime fetch. +- Python defaults (verbatim): ruff `target-version = "py312"`, `line-length = 120`, `[tool.ruff.lint] select = ["E", "F", "W"]`, `[tool.ruff.format] quote-style = "double"`; pytest `testpaths = ["tests"]`; pip + `requirements.txt` / `requirements-dev.txt`. +- Dotfile assets are stored WITHOUT the leading dot (`gitignore`, `editorconfig`) so they aren't hidden/ignored in the marketplace repo; the engine renames them on copy. +- Idempotent, never silently clobber: existing files are skipped and reported, never overwritten. +- CI workflows carry the write-permission fix (`pull-requests: write`, `issues: write`); they only take effect once on the target repo's default branch. +- Working branch for execution: `feat/offworld-marketplace-scaffold` (continues PR #1 into the Phase 1 deliverable). Phase 2 (ts-frontend/ts-backend) and Phase 3 (pre-commit, C++) are out of scope here. + +--- + +### Task 1: Restructure — fold PR #1 templates into plugin assets + +**Files:** +- Move: `templates/settings.json` → `plugins/core/skills/setup-repo/assets/claude/settings.json` +- Move: `templates/CLAUDE.md` → `plugins/core/skills/setup-repo/assets/claude/CLAUDE.md` +- Move: `templates/github-workflows/claude-code-review.yml` → `plugins/core/skills/setup-repo/assets/ci/claude-code-review.yml` +- Move: `templates/github-workflows/claude.yml` → `plugins/core/skills/setup-repo/assets/ci/claude.yml` +- Move: `rules/security.md` → `plugins/core/skills/setup-repo/assets/rules/security.md` +- Move: `rules/code-style.md` → `plugins/core/skills/setup-repo/assets/rules/code-style.md` +- Delete: root `templates/`, root `rules/` +- Modify: `README.md` (adoption section) + +**Interfaces:** +- Produces: the `assets/claude/`, `assets/ci/`, `assets/rules/` directories that Task 4's engine copies from. + +- [ ] **Step 1: Move the files with git mv** + +```bash +cd "$(git rev-parse --show-toplevel)" +mkdir -p plugins/core/skills/setup-repo/assets/{claude,ci,rules} +git mv templates/settings.json plugins/core/skills/setup-repo/assets/claude/settings.json +git mv templates/CLAUDE.md plugins/core/skills/setup-repo/assets/claude/CLAUDE.md +git mv templates/github-workflows/claude-code-review.yml plugins/core/skills/setup-repo/assets/ci/claude-code-review.yml +git mv templates/github-workflows/claude.yml plugins/core/skills/setup-repo/assets/ci/claude.yml +git mv rules/security.md plugins/core/skills/setup-repo/assets/rules/security.md +git mv rules/code-style.md plugins/core/skills/setup-repo/assets/rules/code-style.md +rmdir templates/github-workflows templates rules 2>/dev/null || true +``` + +- [ ] **Step 2: Update the rules-file symlink comment** + +The two rules files still contain a comment saying they get "symlinked into each repo's `.claude/rules/`". The model is now copy, not symlink. In BOTH `plugins/core/skills/setup-repo/assets/rules/security.md` and `.../code-style.md`, replace the comment block's line: + +``` + These files are symlinked into each consuming repo's .claude/rules/ so every +``` + +with: + +``` + The setup-repo skill copies these into each consuming repo's .claude/rules/ so every +``` + +- [ ] **Step 3: Rewrite the README adoption section** + +In `README.md`, replace the entire "## Adopting in a consuming repo" section (down to, but not including, "## Contributing") with: + +```markdown +## Adopting in a consuming repo + +Adoption is driven by Claude Code, not manual copying. Once, per machine, install +the plugin at user scope: + +``` +/plugin marketplace add offworldlabs/claude-shared +/plugin install core@offworld +``` + +Then, in any new repo, start Claude Code and ask: + +> set this repo up per `claude-shared` + +Claude invokes the `core:setup-repo` skill, which writes `.claude/settings.json` +(registering the marketplace and enabling `core`), a `CLAUDE.md`, the shared rules, +the Claude review workflows, and your stack's tooling — then installs deps and +helps you flesh out `CLAUDE.md`. See `docs/runbooks/github-actions-claude-review.md` +for the one manual follow-up (the `CLAUDE_CODE_OAUTH_TOKEN` secret). +``` + +- [ ] **Step 4: Validate the plugin still loads** + +Run: `claude plugin validate .` +Expected: `Validation passed` (no error). The moved workflow/settings files are now plugin assets, not components, so they don't affect validation. + +- [ ] **Step 5: Verify old paths are gone and new ones exist** + +Run: +```bash +test ! -d templates && test ! -d rules && echo "root dirs removed" +ls plugins/core/skills/setup-repo/assets/claude/settings.json \ + plugins/core/skills/setup-repo/assets/ci/claude.yml \ + plugins/core/skills/setup-repo/assets/rules/security.md +``` +Expected: prints "root dirs removed" and lists the three files with no error. + +- [ ] **Step 6: Commit** + +```bash +git add -A +git commit -m "Fold root templates and rules into setup-repo plugin assets" +``` + +--- + +### Task 2: Python stack assets + +**Files:** +- Create: `plugins/core/skills/setup-repo/assets/stack/python/pyproject.toml` +- Create: `plugins/core/skills/setup-repo/assets/stack/python/requirements.txt` +- Create: `plugins/core/skills/setup-repo/assets/stack/python/requirements-dev.txt` +- Create: `plugins/core/skills/setup-repo/assets/stack/python/gitignore` +- Create: `plugins/core/skills/setup-repo/assets/stack/python/tests/.gitkeep` + +**Interfaces:** +- Produces: the `assets/stack/python/` tree Task 4's engine copies when `stack == python`. + +- [ ] **Step 1: Write a failing validation check** + +Create `tests/setup-repo/test-python-assets.sh`: + +```bash +#!/usr/bin/env bash +set -euo pipefail +ROOT="$(git rev-parse --show-toplevel)" +PY="$ROOT/plugins/core/skills/setup-repo/assets/stack/python" + +python3 - "$PY/pyproject.toml" <<'EOF' +import sys, tomllib +data = tomllib.load(open(sys.argv[1], "rb")) +ruff = data["tool"]["ruff"] +assert ruff["target-version"] == "py312", ruff +assert ruff["line-length"] == 120, ruff +assert data["tool"]["ruff"]["lint"]["select"] == ["E", "F", "W"], ruff +assert data["tool"]["ruff"]["format"]["quote-style"] == "double", ruff +assert data["tool"]["pytest"]["ini_options"]["testpaths"] == ["tests"], data +print("pyproject.toml OK") +EOF + +grep -q "ruff" "$PY/requirements-dev.txt" +grep -q "pytest" "$PY/requirements-dev.txt" +test -f "$PY/gitignore" && grep -q "__pycache__" "$PY/gitignore" +test -f "$PY/tests/.gitkeep" +echo "python assets OK" +``` + +- [ ] **Step 2: Run it to verify it fails** + +Run: `bash tests/setup-repo/test-python-assets.sh` +Expected: FAIL (the `assets/stack/python/*` files don't exist yet — `tomllib.load` raises `FileNotFoundError`). + +- [ ] **Step 3: Create the Python assets** + +`plugins/core/skills/setup-repo/assets/stack/python/pyproject.toml`: +```toml +[tool.ruff] +target-version = "py312" +line-length = 120 + +[tool.ruff.lint] +select = ["E", "F", "W"] + +[tool.ruff.format] +quote-style = "double" + +[tool.pytest.ini_options] +testpaths = ["tests"] +``` + +`plugins/core/skills/setup-repo/assets/stack/python/requirements.txt`: +``` +# Runtime dependencies for this project. Add them below, pinned where practical. +``` + +`plugins/core/skills/setup-repo/assets/stack/python/requirements-dev.txt`: +``` +ruff>=0.8.0 +pytest>=8.0.0 +``` + +`plugins/core/skills/setup-repo/assets/stack/python/gitignore`: +``` +__pycache__/ +*.py[cod] +.pytest_cache/ +.ruff_cache/ +.venv/ +venv/ +*.egg-info/ +build/ +dist/ +.env +``` + +`plugins/core/skills/setup-repo/assets/stack/python/tests/.gitkeep`: (empty file) + +- [ ] **Step 4: Run the check to verify it passes** + +Run: `bash tests/setup-repo/test-python-assets.sh` +Expected: prints `pyproject.toml OK` then `python assets OK`. + +- [ ] **Step 5: Commit** + +```bash +git add plugins/core/skills/setup-repo/assets/stack/python tests/setup-repo/test-python-assets.sh +git commit -m "Add Python stack assets (ruff py312, pytest) with validation" +``` + +--- + +### Task 3: CI and editorconfig assets + +**Files:** +- Create: `plugins/core/skills/setup-repo/assets/ci/ci-python.yml` +- Create: `plugins/core/skills/setup-repo/assets/editorconfig` + +**Interfaces:** +- Produces: `assets/ci/ci-python.yml` (copied to `.github/workflows/ci.yml` for python) and `assets/editorconfig` (copied to `.editorconfig`), both used by Task 4's engine. + +- [ ] **Step 1: Write a failing validation check** + +Create `tests/setup-repo/test-ci-assets.sh`: + +```bash +#!/usr/bin/env bash +set -euo pipefail +ROOT="$(git rev-parse --show-toplevel)" +CI="$ROOT/plugins/core/skills/setup-repo/assets/ci/ci-python.yml" +EC="$ROOT/plugins/core/skills/setup-repo/assets/editorconfig" + +python3 - "$CI" <<'EOF' +import sys +try: + import yaml +except ModuleNotFoundError: + print("pyyaml missing; skipping YAML parse"); sys.exit(0) +doc = yaml.safe_load(open(sys.argv[1])) +jobs = doc["jobs"]["lint-and-test"]["steps"] +names = [s.get("name", "") for s in jobs] +assert any("Ruff" in n for n in names), names +assert any("Pytest" in n or "pytest" in n for n in names), names +print("ci-python.yml OK") +EOF + +grep -q "root = true" "$EC" +grep -q "indent_size = 4" "$EC" # python +grep -q "indent_size = 2" "$EC" # js/ts/yaml +echo "ci assets OK" +``` + +- [ ] **Step 2: Run it to verify it fails** + +Run: `bash tests/setup-repo/test-ci-assets.sh` +Expected: FAIL (files don't exist; `open(sys.argv[1])` raises `FileNotFoundError`). + +- [ ] **Step 3: Create the CI and editorconfig assets** + +`plugins/core/skills/setup-repo/assets/ci/ci-python.yml`: +```yaml +name: CI + +on: + push: + branches: [main] + pull_request: + +jobs: + lint-and-test: + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: "3.12" + + - name: Install dependencies + run: | + python -m pip install --upgrade pip + pip install -r requirements.txt -r requirements-dev.txt + + - name: Ruff lint + run: ruff check . + + - name: Ruff format check + run: ruff format --check . + + - name: Pytest + run: pytest +``` + +`plugins/core/skills/setup-repo/assets/editorconfig`: +``` +root = true + +[*] +charset = utf-8 +end_of_line = lf +insert_final_newline = true +trim_trailing_whitespace = true +indent_style = space + +[*.py] +indent_size = 4 + +[*.{js,jsx,ts,tsx,json,yml,yaml}] +indent_size = 2 + +[*.md] +trim_trailing_whitespace = false +``` + +- [ ] **Step 4: Run the check to verify it passes** + +Run: `bash tests/setup-repo/test-ci-assets.sh` +Expected: prints `ci-python.yml OK` (or the pyyaml-missing skip line) then `ci assets OK`. + +- [ ] **Step 5: Commit** + +```bash +git add plugins/core/skills/setup-repo/assets/ci/ci-python.yml plugins/core/skills/setup-repo/assets/editorconfig tests/setup-repo/test-ci-assets.sh +git commit -m "Add Python CI workflow and shared editorconfig assets" +``` + +--- + +### Task 4: Deterministic scaffold engine + end-to-end test + +**Files:** +- Create: `plugins/core/skills/setup-repo/scripts/scaffold-repo.sh` +- Create: `tests/setup-repo/test-scaffold.sh` + +**Interfaces:** +- Consumes: all `assets/**` from Tasks 1-3. +- Produces: `scaffold-repo.sh ` where `` is `python` or `none`; copies assets into ``, renaming `gitignore`→`.gitignore` and `editorconfig`→`.editorconfig`, skipping (not overwriting) existing files, and printing `WRITTEN:` / `SKIPPED:` reports. Exit code 2 on unknown stack. Task 5's SKILL.md calls this. + +- [ ] **Step 1: Write the failing end-to-end test** + +Create `tests/setup-repo/test-scaffold.sh`: + +```bash +#!/usr/bin/env bash +set -euo pipefail +ROOT="$(git rev-parse --show-toplevel)" +ENGINE="$ROOT/plugins/core/skills/setup-repo/scripts/scaffold-repo.sh" +TMP="$(mktemp -d)" +trap 'rm -rf "$TMP"' EXIT +git -C "$TMP" init -q + +bash "$ENGINE" "$TMP" python + +for f in .claude/settings.json CLAUDE.md .claude/rules/security.md \ + .claude/rules/code-style.md .github/workflows/claude.yml \ + .github/workflows/claude-code-review.yml .github/workflows/ci.yml \ + .editorconfig pyproject.toml requirements.txt requirements-dev.txt \ + .gitignore tests/.gitkeep; do + test -e "$TMP/$f" || { echo "MISSING: $f" >&2; exit 1; } +done + +python3 -c "import json; json.load(open('$TMP/.claude/settings.json'))" +python3 -c "import tomllib; tomllib.load(open('$TMP/pyproject.toml','rb'))" + +# non-clobber: second run skips everything and reports it +out="$(bash "$ENGINE" "$TMP" python)" +echo "$out" | grep -q "SKIPPED" || { echo "expected SKIPPED report" >&2; exit 1; } +echo "$out" | grep -q "pyproject.toml" || { echo "expected skipped file listed" >&2; exit 1; } + +# scaffolded python defaults are internally consistent +mkdir -p "$TMP/src" +cat > "$TMP/src/example.py" <<'PY' +def add(a: int, b: int) -> int: + return a + b +PY +cat > "$TMP/tests/test_example.py" <<'PY' +from src.example import add + + +def test_add() -> None: + assert add(2, 3) == 5 +PY +if command -v ruff >/dev/null 2>&1; then + ( cd "$TMP" && ruff format . >/dev/null && ruff check . && ruff format --check . ) + echo "ruff OK" +else + echo "ruff not installed; skipped" +fi +if command -v pytest >/dev/null 2>&1; then + ( cd "$TMP" && PYTHONPATH=. pytest -q ) + echo "pytest OK" +else + echo "pytest not installed; skipped" +fi + +# unknown stack exits 2 +if bash "$ENGINE" "$TMP" bogus 2>/dev/null; then + echo "expected non-zero exit for unknown stack" >&2; exit 1 +fi +echo "ALL CHECKS PASSED" +``` + +- [ ] **Step 2: Run it to verify it fails** + +Run: `bash tests/setup-repo/test-scaffold.sh` +Expected: FAIL (`scaffold-repo.sh` does not exist — `bash "$ENGINE"` errors "No such file or directory"). + +- [ ] **Step 3: Write the engine** + +`plugins/core/skills/setup-repo/scripts/scaffold-repo.sh`: +```bash +#!/usr/bin/env bash +# Deterministic file-scaffolding engine for the core:setup-repo skill. +# Copies bundled assets into a target repo without clobbering existing files. +# Usage: scaffold-repo.sh +# : python | none (ts-frontend / ts-backend land in Phase 2) +set -euo pipefail + +TARGET="${1:?target dir required}" +STACK="${2:-none}" +ASSETS="$(cd "$(dirname "${BASH_SOURCE[0]}")/../assets" && pwd)" + +written=() +skipped=() + +copy() { # copy + local src="$1" dest="$2" + mkdir -p "$(dirname "$dest")" + if [[ -e "$dest" ]]; then + skipped+=("$dest") + else + cp "$src" "$dest" + written+=("$dest") + fi +} + +# Claude enablement (always) +copy "$ASSETS/claude/settings.json" "$TARGET/.claude/settings.json" +copy "$ASSETS/claude/CLAUDE.md" "$TARGET/CLAUDE.md" +copy "$ASSETS/rules/security.md" "$TARGET/.claude/rules/security.md" +copy "$ASSETS/rules/code-style.md" "$TARGET/.claude/rules/code-style.md" + +# Claude review workflows (always) +copy "$ASSETS/ci/claude-code-review.yml" "$TARGET/.github/workflows/claude-code-review.yml" +copy "$ASSETS/ci/claude.yml" "$TARGET/.github/workflows/claude.yml" + +# Shared +copy "$ASSETS/editorconfig" "$TARGET/.editorconfig" + +# Stack +case "$STACK" in + python) + copy "$ASSETS/stack/python/pyproject.toml" "$TARGET/pyproject.toml" + copy "$ASSETS/stack/python/requirements.txt" "$TARGET/requirements.txt" + copy "$ASSETS/stack/python/requirements-dev.txt" "$TARGET/requirements-dev.txt" + copy "$ASSETS/stack/python/gitignore" "$TARGET/.gitignore" + copy "$ASSETS/stack/python/tests/.gitkeep" "$TARGET/tests/.gitkeep" + copy "$ASSETS/ci/ci-python.yml" "$TARGET/.github/workflows/ci.yml" + ;; + none) ;; + *) echo "unknown stack: $STACK" >&2; exit 2 ;; +esac + +echo "WRITTEN:" +printf ' %s\n' "${written[@]:-(none)}" +echo "SKIPPED (already present, left untouched):" +printf ' %s\n' "${skipped[@]:-(none)}" +``` + +- [ ] **Step 4: Make it executable and run the test** + +Run: +```bash +chmod +x plugins/core/skills/setup-repo/scripts/scaffold-repo.sh +bash tests/setup-repo/test-scaffold.sh +``` +Expected: ends with `ALL CHECKS PASSED` (with `ruff OK` / `pytest OK` if those tools are installed, else the "skipped" lines). + +- [ ] **Step 5: Commit** + +```bash +git add plugins/core/skills/setup-repo/scripts/scaffold-repo.sh tests/setup-repo/test-scaffold.sh +git commit -m "Add deterministic scaffold engine with end-to-end test" +``` + +--- + +### Task 5: The `setup-repo` SKILL.md procedure + +**Files:** +- Create: `plugins/core/skills/setup-repo/SKILL.md` + +**Interfaces:** +- Consumes: `scripts/scaffold-repo.sh` (Task 4) via `${CLAUDE_PLUGIN_ROOT}/skills/setup-repo/scripts/scaffold-repo.sh`. +- Produces: the model-invocable `core:setup-repo` skill. + +- [ ] **Step 1: Write the SKILL.md** + +`plugins/core/skills/setup-repo/SKILL.md`: +```markdown +--- +name: setup-repo +description: Use when setting up a new or blank Offworld Labs repository, or when the user asks to "set up this repo per claude-shared", scaffold a repo to org standards, or add the standard Claude/CI/tooling setup. Scaffolds Claude Code enablement, review workflows, and stack tooling. +--- + +# Set up a repo per claude-shared + +Scaffold the current repository to Offworld Labs standards: Claude Code +enablement, the Claude review workflows, and the chosen stack's tooling. The +mechanical file copying is done by the bundled engine; you handle the +interactive parts and the report. + +`ENGINE="${CLAUDE_PLUGIN_ROOT}/skills/setup-repo/scripts/scaffold-repo.sh"` + +## Procedure + +1. **Confirm the target.** Ensure the working directory is a git repo + (`git rev-parse --is-inside-work-tree`). If it is not, offer to run + `git init`; abort if the user declines. + +2. **Determine the stack.** Detect from existing files: `pyproject.toml` or + `requirements*.txt` → `python`. If ambiguous or empty, ask the user to choose + `python` or `none` (ts-frontend / ts-backend arrive in a later phase). + +3. **Scaffold the files.** Run the engine, which never overwrites existing files: + `bash "$ENGINE" . ` + Relay its `WRITTEN` / `SKIPPED` output to the user. + +4. **Install dependencies.** For `python`, run + `pip install -r requirements.txt -r requirements-dev.txt` + (prefer an active virtualenv). Report the command and result; if the + toolchain is unavailable, skip and note it rather than failing. + +5. **Flesh out CLAUDE.md.** The stub was just written. Ask the user for a + one-or-two-line description of what this repo does, then fill in the + `Project Overview`, `Build & Test Commands`, and `Local Architecture` + sections from their answer plus what was scaffolded (stack, `ruff check .`, + `ruff format --check .`, `pytest`). If they skip, leave the stub as-is. Keep + CLAUDE.md under the 200-line ceiling noted in the template. + +6. **Report and follow-ups.** Summarise files written vs skipped, then list the + manual steps: add the `CLAUDE_CODE_OAUTH_TOKEN` repo secret, and commit the + workflows to the default branch before Claude review runs (see + `docs/runbooks/github-actions-claude-review.md` for why). Do not commit on the + user's behalf unless asked. +``` + +- [ ] **Step 2: Validate the plugin and skill frontmatter** + +Run: `claude plugin validate ./plugins/core` +Expected: `Validation passed` (the `setup-repo` skill frontmatter parses; no YAML errors). + +- [ ] **Step 3: Confirm the skill is discoverable** + +Run: +```bash +claude plugin validate . +claude -p "List available skills whose name contains 'setup-repo'. Answer with just the namespaced skill name(s), or 'none'." --allowedTools "" 2>&1 | tail -3 +``` +Expected: `Validation passed`, and the model prints `core:setup-repo` (requires the plugin be installed/loadable; if it prints `none`, run `/reload-plugins` in an interactive session or reinstall `core@offworld` from the local marketplace first). + +- [ ] **Step 4: Commit** + +```bash +git add plugins/core/skills/setup-repo/SKILL.md +git commit -m "Add setup-repo skill procedure" +``` + +--- + +## Self-Review + +**Spec coverage:** +- Delivery model (self-contained plugin, `${CLAUDE_PLUGIN_ROOT}`) → Tasks 1, 4, 5. +- Python defaults (ruff py312/120/EFW/double, pytest) → Task 2, verified in Tasks 2 & 4. +- Repo restructure (fold `templates/`+`rules/`, keep `docs/`) → Task 1. +- Rules copied not symlinked → Task 1 Step 2 + engine (Task 4). +- CI incl. write-permission workflows + `ci-python.yml` → Tasks 1, 3, 4. +- `.editorconfig` → Task 3. +- Procedure steps 1-9 (confirm, detect, enable, CI, stack, shared, install, flesh CLAUDE.md, report) → Task 5 (mechanical copying delegated to the engine, Task 4). +- Idempotent / non-clobber → engine `copy()` + Task 4 test. +- Testing (asset validity, temp-repo scaffold, ruff/pytest) → Tasks 2, 3, 4. +- Prerequisite (core at user scope) + adoption flow → Task 1 README rewrite. +- Out of scope confirmed: ts stacks (Phase 2), C++/pre-commit (Phase 3), governance. + +**Placeholder scan:** none — every file's full contents and every command are inline. + +**Type/name consistency:** engine signature `scaffold-repo.sh ` with stacks `python|none`, dotfile renames `gitignore`→`.gitignore` / `editorconfig`→`.editorconfig`, and asset paths are identical across Tasks 1-5 and the tests. +``` From 1117882bd37053da950cfa3197491b378ea08861 Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Tue, 14 Jul 2026 17:33:27 +0100 Subject: [PATCH 06/27] Fold root templates and rules into setup-repo plugin assets --- README.md | 43 +++++-------------- .../assets/ci}/claude-code-review.yml | 0 .../skills/setup-repo/assets/ci}/claude.yml | 0 .../setup-repo/assets/claude}/CLAUDE.md | 0 .../setup-repo/assets/claude}/settings.json | 0 .../setup-repo/assets/rules}/code-style.md | 2 +- .../setup-repo/assets/rules}/security.md | 2 +- 7 files changed, 13 insertions(+), 34 deletions(-) rename {templates/github-workflows => plugins/core/skills/setup-repo/assets/ci}/claude-code-review.yml (100%) rename {templates/github-workflows => plugins/core/skills/setup-repo/assets/ci}/claude.yml (100%) rename {templates => plugins/core/skills/setup-repo/assets/claude}/CLAUDE.md (100%) rename {templates => plugins/core/skills/setup-repo/assets/claude}/settings.json (100%) rename {rules => plugins/core/skills/setup-repo/assets/rules}/code-style.md (85%) rename {rules => plugins/core/skills/setup-repo/assets/rules}/security.md (86%) diff --git a/README.md b/README.md index 94775bb..e723a55 100644 --- a/README.md +++ b/README.md @@ -19,44 +19,23 @@ In any repo, add the marketplace and install the plugin: ## Adopting in a consuming repo -For zero-setup adoption, copy the template settings into the repo so everyone -who trusts the folder gets the marketplace and `core` plugin automatically: +Adoption is driven by Claude Code, not manual copying. Once, per machine, install +the plugin at user scope: -```bash -mkdir -p .claude -cp path/to/claude-shared/templates/settings.json .claude/settings.json ``` - -`.claude/settings.json` registers the `offworld` marketplace via -`extraKnownMarketplaces` and enables `core@offworld` via `enabledPlugins`, so -opening the repo prompts installation with no manual `/plugin` commands. - -Symlink the shared rules into the repo so they stay in sync with this repo: - -```bash -mkdir -p .claude/rules -ln -s ../../path/to/claude-shared/rules/security.md .claude/rules/security.md -ln -s ../../path/to/claude-shared/rules/code-style.md .claude/rules/code-style.md +/plugin marketplace add offworldlabs/claude-shared +/plugin install core@offworld ``` -Point the repo's `CLAUDE.md` at this repo's `docs/` for org-wide context instead -of duplicating it (see `templates/CLAUDE.md`). - -### GitHub Actions (Claude PR review) +Then, in any new repo, start Claude Code and ask: -Don't rely on the scaffold `/install-github-app` generates — it grants the -workflows read-only tokens, so the review runs green but never posts a comment. -Copy the pre-corrected workflows from `templates/github-workflows/` instead: - -```bash -mkdir -p .github/workflows -cp path/to/claude-shared/templates/github-workflows/claude-code-review.yml .github/workflows/ -cp path/to/claude-shared/templates/github-workflows/claude.yml .github/workflows/ -``` +> set this repo up per `claude-shared` -These must be committed to the repo's **default branch** before Claude will run -on PRs (a self-modification guard in `claude-code-action`). See -`docs/runbooks/github-actions-claude-review.md` for the full explanation. +Claude invokes the `core:setup-repo` skill, which writes `.claude/settings.json` +(registering the marketplace and enabling `core`), a `CLAUDE.md`, the shared rules, +the Claude review workflows, and your stack's tooling — then installs deps and +helps you flesh out `CLAUDE.md`. See `docs/runbooks/github-actions-claude-review.md` +for the one manual follow-up (the `CLAUDE_CODE_OAUTH_TOKEN` secret). ## Contributing diff --git a/templates/github-workflows/claude-code-review.yml b/plugins/core/skills/setup-repo/assets/ci/claude-code-review.yml similarity index 100% rename from templates/github-workflows/claude-code-review.yml rename to plugins/core/skills/setup-repo/assets/ci/claude-code-review.yml diff --git a/templates/github-workflows/claude.yml b/plugins/core/skills/setup-repo/assets/ci/claude.yml similarity index 100% rename from templates/github-workflows/claude.yml rename to plugins/core/skills/setup-repo/assets/ci/claude.yml diff --git a/templates/CLAUDE.md b/plugins/core/skills/setup-repo/assets/claude/CLAUDE.md similarity index 100% rename from templates/CLAUDE.md rename to plugins/core/skills/setup-repo/assets/claude/CLAUDE.md diff --git a/templates/settings.json b/plugins/core/skills/setup-repo/assets/claude/settings.json similarity index 100% rename from templates/settings.json rename to plugins/core/skills/setup-repo/assets/claude/settings.json diff --git a/rules/code-style.md b/plugins/core/skills/setup-repo/assets/rules/code-style.md similarity index 85% rename from rules/code-style.md rename to plugins/core/skills/setup-repo/assets/rules/code-style.md index b17d9b6..116653c 100644 --- a/rules/code-style.md +++ b/plugins/core/skills/setup-repo/assets/rules/code-style.md @@ -1,6 +1,6 @@ diff --git a/plugins/core/skills/setup-repo/assets/rules/security.md b/plugins/core/skills/setup-repo/assets/rules/security.md index d70bfe9..c22a3d5 100644 --- a/plugins/core/skills/setup-repo/assets/rules/security.md +++ b/plugins/core/skills/setup-repo/assets/rules/security.md @@ -2,7 +2,7 @@ Shared security rules for Offworld Labs repos. The setup-repo skill copies these into each consuming repo's .claude/rules/ so every repo enforces the same baseline. Edit here; the change propagates to every - repo that symlinks this file. Replace the TODO placeholders below with real + repo that copies this file. Replace the TODO placeholders below with real imperatives before relying on them. --> From 5eba352501f037b8f8e7a352fe86ad7ffb9c5323 Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Tue, 14 Jul 2026 17:39:26 +0100 Subject: [PATCH 09/27] Add Python stack assets (ruff py312, pytest) with validation --- .../setup-repo/assets/stack/python/gitignore | 10 +++++++++ .../assets/stack/python/pyproject.toml | 12 ++++++++++ .../assets/stack/python/requirements-dev.txt | 2 ++ .../assets/stack/python/requirements.txt | 1 + .../assets/stack/python/tests/.gitkeep | 0 tests/setup-repo/test-python-assets.sh | 22 +++++++++++++++++++ 6 files changed, 47 insertions(+) create mode 100644 plugins/core/skills/setup-repo/assets/stack/python/gitignore create mode 100644 plugins/core/skills/setup-repo/assets/stack/python/pyproject.toml create mode 100644 plugins/core/skills/setup-repo/assets/stack/python/requirements-dev.txt create mode 100644 plugins/core/skills/setup-repo/assets/stack/python/requirements.txt create mode 100644 plugins/core/skills/setup-repo/assets/stack/python/tests/.gitkeep create mode 100755 tests/setup-repo/test-python-assets.sh diff --git a/plugins/core/skills/setup-repo/assets/stack/python/gitignore b/plugins/core/skills/setup-repo/assets/stack/python/gitignore new file mode 100644 index 0000000..f375b27 --- /dev/null +++ b/plugins/core/skills/setup-repo/assets/stack/python/gitignore @@ -0,0 +1,10 @@ +__pycache__/ +*.py[cod] +.pytest_cache/ +.ruff_cache/ +.venv/ +venv/ +*.egg-info/ +build/ +dist/ +.env diff --git a/plugins/core/skills/setup-repo/assets/stack/python/pyproject.toml b/plugins/core/skills/setup-repo/assets/stack/python/pyproject.toml new file mode 100644 index 0000000..4393d70 --- /dev/null +++ b/plugins/core/skills/setup-repo/assets/stack/python/pyproject.toml @@ -0,0 +1,12 @@ +[tool.ruff] +target-version = "py312" +line-length = 120 + +[tool.ruff.lint] +select = ["E", "F", "W"] + +[tool.ruff.format] +quote-style = "double" + +[tool.pytest.ini_options] +testpaths = ["tests"] diff --git a/plugins/core/skills/setup-repo/assets/stack/python/requirements-dev.txt b/plugins/core/skills/setup-repo/assets/stack/python/requirements-dev.txt new file mode 100644 index 0000000..9490314 --- /dev/null +++ b/plugins/core/skills/setup-repo/assets/stack/python/requirements-dev.txt @@ -0,0 +1,2 @@ +ruff>=0.8.0 +pytest>=8.0.0 diff --git a/plugins/core/skills/setup-repo/assets/stack/python/requirements.txt b/plugins/core/skills/setup-repo/assets/stack/python/requirements.txt new file mode 100644 index 0000000..aa4e3b2 --- /dev/null +++ b/plugins/core/skills/setup-repo/assets/stack/python/requirements.txt @@ -0,0 +1 @@ +# Runtime dependencies for this project. Add them below, pinned where practical. diff --git a/plugins/core/skills/setup-repo/assets/stack/python/tests/.gitkeep b/plugins/core/skills/setup-repo/assets/stack/python/tests/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/tests/setup-repo/test-python-assets.sh b/tests/setup-repo/test-python-assets.sh new file mode 100755 index 0000000..af764f2 --- /dev/null +++ b/tests/setup-repo/test-python-assets.sh @@ -0,0 +1,22 @@ +#!/usr/bin/env bash +set -euo pipefail +ROOT="$(git rev-parse --show-toplevel)" +PY="$ROOT/plugins/core/skills/setup-repo/assets/stack/python" + +python3 - "$PY/pyproject.toml" <<'EOF' +import sys, tomllib +data = tomllib.load(open(sys.argv[1], "rb")) +ruff = data["tool"]["ruff"] +assert ruff["target-version"] == "py312", ruff +assert ruff["line-length"] == 120, ruff +assert data["tool"]["ruff"]["lint"]["select"] == ["E", "F", "W"], ruff +assert data["tool"]["ruff"]["format"]["quote-style"] == "double", ruff +assert data["tool"]["pytest"]["ini_options"]["testpaths"] == ["tests"], data +print("pyproject.toml OK") +EOF + +grep -q "ruff" "$PY/requirements-dev.txt" +grep -q "pytest" "$PY/requirements-dev.txt" +test -f "$PY/gitignore" && grep -q "__pycache__" "$PY/gitignore" +test -f "$PY/tests/.gitkeep" +echo "python assets OK" From 2e97e2c1f33ff2af15fbe4116fe3cfe1454ccb95 Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Tue, 14 Jul 2026 17:42:14 +0100 Subject: [PATCH 10/27] Tighten requirements-dev version assertions in python assets test --- tests/setup-repo/test-python-assets.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/setup-repo/test-python-assets.sh b/tests/setup-repo/test-python-assets.sh index af764f2..d734dfa 100755 --- a/tests/setup-repo/test-python-assets.sh +++ b/tests/setup-repo/test-python-assets.sh @@ -15,8 +15,8 @@ assert data["tool"]["pytest"]["ini_options"]["testpaths"] == ["tests"], data print("pyproject.toml OK") EOF -grep -q "ruff" "$PY/requirements-dev.txt" -grep -q "pytest" "$PY/requirements-dev.txt" +grep -qE 'ruff>=0\.8' "$PY/requirements-dev.txt" +grep -qE 'pytest>=8' "$PY/requirements-dev.txt" test -f "$PY/gitignore" && grep -q "__pycache__" "$PY/gitignore" test -f "$PY/tests/.gitkeep" echo "python assets OK" From 4526cce1c25787445d103dfed1a910188ae5e136 Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Tue, 14 Jul 2026 17:43:43 +0100 Subject: [PATCH 11/27] Add Python CI workflow and shared editorconfig assets --- .../skills/setup-repo/assets/ci/ci-python.yml | 32 +++++++++++++++++++ .../skills/setup-repo/assets/editorconfig | 17 ++++++++++ tests/setup-repo/test-ci-assets.sh | 24 ++++++++++++++ 3 files changed, 73 insertions(+) create mode 100644 plugins/core/skills/setup-repo/assets/ci/ci-python.yml create mode 100644 plugins/core/skills/setup-repo/assets/editorconfig create mode 100644 tests/setup-repo/test-ci-assets.sh diff --git a/plugins/core/skills/setup-repo/assets/ci/ci-python.yml b/plugins/core/skills/setup-repo/assets/ci/ci-python.yml new file mode 100644 index 0000000..10dac2b --- /dev/null +++ b/plugins/core/skills/setup-repo/assets/ci/ci-python.yml @@ -0,0 +1,32 @@ +name: CI + +on: + push: + branches: [main] + pull_request: + +jobs: + lint-and-test: + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: "3.12" + + - name: Install dependencies + run: | + python -m pip install --upgrade pip + pip install -r requirements.txt -r requirements-dev.txt + + - name: Ruff lint + run: ruff check . + + - name: Ruff format check + run: ruff format --check . + + - name: Pytest + run: pytest diff --git a/plugins/core/skills/setup-repo/assets/editorconfig b/plugins/core/skills/setup-repo/assets/editorconfig new file mode 100644 index 0000000..664ce3c --- /dev/null +++ b/plugins/core/skills/setup-repo/assets/editorconfig @@ -0,0 +1,17 @@ +root = true + +[*] +charset = utf-8 +end_of_line = lf +insert_final_newline = true +trim_trailing_whitespace = true +indent_style = space + +[*.py] +indent_size = 4 + +[*.{js,jsx,ts,tsx,json,yml,yaml}] +indent_size = 2 + +[*.md] +trim_trailing_whitespace = false diff --git a/tests/setup-repo/test-ci-assets.sh b/tests/setup-repo/test-ci-assets.sh new file mode 100644 index 0000000..f665ca4 --- /dev/null +++ b/tests/setup-repo/test-ci-assets.sh @@ -0,0 +1,24 @@ +#!/usr/bin/env bash +set -euo pipefail +ROOT="$(git rev-parse --show-toplevel)" +CI="$ROOT/plugins/core/skills/setup-repo/assets/ci/ci-python.yml" +EC="$ROOT/plugins/core/skills/setup-repo/assets/editorconfig" + +python3 - "$CI" <<'EOF' +import sys +try: + import yaml +except ModuleNotFoundError: + print("pyyaml missing; skipping YAML parse"); sys.exit(0) +doc = yaml.safe_load(open(sys.argv[1])) +jobs = doc["jobs"]["lint-and-test"]["steps"] +names = [s.get("name", "") for s in jobs] +assert any("Ruff" in n for n in names), names +assert any("Pytest" in n or "pytest" in n for n in names), names +print("ci-python.yml OK") +EOF + +grep -q "root = true" "$EC" +grep -q "indent_size = 4" "$EC" # python +grep -q "indent_size = 2" "$EC" # js/ts/yaml +echo "ci assets OK" From 39b193eb66649b9ea748e38da5c841c0897b742a Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Tue, 14 Jul 2026 17:46:49 +0100 Subject: [PATCH 12/27] Harden CI asset test: assert run commands, triggers, python version --- tests/setup-repo/test-ci-assets.sh | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/tests/setup-repo/test-ci-assets.sh b/tests/setup-repo/test-ci-assets.sh index f665ca4..df213e9 100644 --- a/tests/setup-repo/test-ci-assets.sh +++ b/tests/setup-repo/test-ci-assets.sh @@ -11,10 +11,19 @@ try: except ModuleNotFoundError: print("pyyaml missing; skipping YAML parse"); sys.exit(0) doc = yaml.safe_load(open(sys.argv[1])) -jobs = doc["jobs"]["lint-and-test"]["steps"] -names = [s.get("name", "") for s in jobs] -assert any("Ruff" in n for n in names), names -assert any("Pytest" in n or "pytest" in n for n in names), names +# GitHub Actions `on:` is parsed as the boolean True by PyYAML (YAML 1.1), so accept either key +on = doc.get("on", doc.get(True)) +assert on is not None, doc +assert on["push"]["branches"] == ["main"], on +assert "pull_request" in on, on +steps = doc["jobs"]["lint-and-test"]["steps"] +runs = "\n".join(s.get("run", "") for s in steps) +assert "ruff check ." in runs, runs +assert "ruff format --check ." in runs, runs +assert "pytest" in runs, runs +assert "requirements.txt" in runs and "requirements-dev.txt" in runs, runs +setup_py = [s for s in steps if str(s.get("uses", "")).startswith("actions/setup-python")] +assert setup_py and setup_py[0]["with"]["python-version"] == "3.12", setup_py print("ci-python.yml OK") EOF From 4c66c5e848b48e0fd49a2c2277e945044a3b8321 Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Tue, 14 Jul 2026 17:48:42 +0100 Subject: [PATCH 13/27] Add deterministic scaffold engine with end-to-end test --- .../setup-repo/scripts/scaffold-repo.sh | 56 ++++++++++++++++++ tests/setup-repo/test-scaffold.sh | 57 +++++++++++++++++++ 2 files changed, 113 insertions(+) create mode 100755 plugins/core/skills/setup-repo/scripts/scaffold-repo.sh create mode 100755 tests/setup-repo/test-scaffold.sh diff --git a/plugins/core/skills/setup-repo/scripts/scaffold-repo.sh b/plugins/core/skills/setup-repo/scripts/scaffold-repo.sh new file mode 100755 index 0000000..34342c3 --- /dev/null +++ b/plugins/core/skills/setup-repo/scripts/scaffold-repo.sh @@ -0,0 +1,56 @@ +#!/usr/bin/env bash +# Deterministic file-scaffolding engine for the core:setup-repo skill. +# Copies bundled assets into a target repo without clobbering existing files. +# Usage: scaffold-repo.sh +# : python | none (ts-frontend / ts-backend land in Phase 2) +set -euo pipefail + +TARGET="${1:?target dir required}" +STACK="${2:-none}" +ASSETS="$(cd "$(dirname "${BASH_SOURCE[0]}")/../assets" && pwd)" + +written=() +skipped=() + +copy() { # copy + local src="$1" dest="$2" + mkdir -p "$(dirname "$dest")" + if [[ -e "$dest" ]]; then + skipped+=("$dest") + else + cp "$src" "$dest" + written+=("$dest") + fi +} + +# Claude enablement (always) +copy "$ASSETS/claude/settings.json" "$TARGET/.claude/settings.json" +copy "$ASSETS/claude/CLAUDE.md" "$TARGET/CLAUDE.md" +copy "$ASSETS/rules/security.md" "$TARGET/.claude/rules/security.md" +copy "$ASSETS/rules/code-style.md" "$TARGET/.claude/rules/code-style.md" + +# Claude review workflows (always) +copy "$ASSETS/ci/claude-code-review.yml" "$TARGET/.github/workflows/claude-code-review.yml" +copy "$ASSETS/ci/claude.yml" "$TARGET/.github/workflows/claude.yml" + +# Shared +copy "$ASSETS/editorconfig" "$TARGET/.editorconfig" + +# Stack +case "$STACK" in + python) + copy "$ASSETS/stack/python/pyproject.toml" "$TARGET/pyproject.toml" + copy "$ASSETS/stack/python/requirements.txt" "$TARGET/requirements.txt" + copy "$ASSETS/stack/python/requirements-dev.txt" "$TARGET/requirements-dev.txt" + copy "$ASSETS/stack/python/gitignore" "$TARGET/.gitignore" + copy "$ASSETS/stack/python/tests/.gitkeep" "$TARGET/tests/.gitkeep" + copy "$ASSETS/ci/ci-python.yml" "$TARGET/.github/workflows/ci.yml" + ;; + none) ;; + *) echo "unknown stack: $STACK" >&2; exit 2 ;; +esac + +echo "WRITTEN:" +printf ' %s\n' "${written[@]:-(none)}" +echo "SKIPPED (already present, left untouched):" +printf ' %s\n' "${skipped[@]:-(none)}" diff --git a/tests/setup-repo/test-scaffold.sh b/tests/setup-repo/test-scaffold.sh new file mode 100755 index 0000000..31c844c --- /dev/null +++ b/tests/setup-repo/test-scaffold.sh @@ -0,0 +1,57 @@ +#!/usr/bin/env bash +set -euo pipefail +ROOT="$(git rev-parse --show-toplevel)" +ENGINE="$ROOT/plugins/core/skills/setup-repo/scripts/scaffold-repo.sh" +TMP="$(mktemp -d)" +trap 'rm -rf "$TMP"' EXIT +git -C "$TMP" init -q + +bash "$ENGINE" "$TMP" python + +for f in .claude/settings.json CLAUDE.md .claude/rules/security.md \ + .claude/rules/code-style.md .github/workflows/claude.yml \ + .github/workflows/claude-code-review.yml .github/workflows/ci.yml \ + .editorconfig pyproject.toml requirements.txt requirements-dev.txt \ + .gitignore tests/.gitkeep; do + test -e "$TMP/$f" || { echo "MISSING: $f" >&2; exit 1; } +done + +python3 -c "import json; json.load(open('$TMP/.claude/settings.json'))" +python3 -c "import tomllib; tomllib.load(open('$TMP/pyproject.toml','rb'))" + +# non-clobber: second run skips everything and reports it +out="$(bash "$ENGINE" "$TMP" python)" +echo "$out" | grep -q "SKIPPED" || { echo "expected SKIPPED report" >&2; exit 1; } +echo "$out" | grep -q "pyproject.toml" || { echo "expected skipped file listed" >&2; exit 1; } + +# scaffolded python defaults are internally consistent +mkdir -p "$TMP/src" +cat > "$TMP/src/example.py" <<'PY' +def add(a: int, b: int) -> int: + return a + b +PY +cat > "$TMP/tests/test_example.py" <<'PY' +from src.example import add + + +def test_add() -> None: + assert add(2, 3) == 5 +PY +if command -v ruff >/dev/null 2>&1; then + ( cd "$TMP" && ruff format . >/dev/null && ruff check . && ruff format --check . ) + echo "ruff OK" +else + echo "ruff not installed; skipped" +fi +if command -v pytest >/dev/null 2>&1; then + ( cd "$TMP" && PYTHONPATH=. pytest -q ) + echo "pytest OK" +else + echo "pytest not installed; skipped" +fi + +# unknown stack exits 2 +if bash "$ENGINE" "$TMP" bogus 2>/dev/null; then + echo "expected non-zero exit for unknown stack" >&2; exit 1 +fi +echo "ALL CHECKS PASSED" From b13797945cd33a93d56cafad297d08a3ba8c76d8 Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Tue, 14 Jul 2026 17:54:23 +0100 Subject: [PATCH 14/27] Validate stack before copying; cover none-stack and exit-code in test --- .../setup-repo/scripts/scaffold-repo.sh | 6 +++- tests/setup-repo/test-scaffold.sh | 31 ++++++++++++++++--- 2 files changed, 31 insertions(+), 6 deletions(-) diff --git a/plugins/core/skills/setup-repo/scripts/scaffold-repo.sh b/plugins/core/skills/setup-repo/scripts/scaffold-repo.sh index 34342c3..f0c7452 100755 --- a/plugins/core/skills/setup-repo/scripts/scaffold-repo.sh +++ b/plugins/core/skills/setup-repo/scripts/scaffold-repo.sh @@ -9,6 +9,11 @@ TARGET="${1:?target dir required}" STACK="${2:-none}" ASSETS="$(cd "$(dirname "${BASH_SOURCE[0]}")/../assets" && pwd)" +case "$STACK" in + python|none) ;; + *) echo "unknown stack: $STACK" >&2; exit 2 ;; +esac + written=() skipped=() @@ -47,7 +52,6 @@ case "$STACK" in copy "$ASSETS/ci/ci-python.yml" "$TARGET/.github/workflows/ci.yml" ;; none) ;; - *) echo "unknown stack: $STACK" >&2; exit 2 ;; esac echo "WRITTEN:" diff --git a/tests/setup-repo/test-scaffold.sh b/tests/setup-repo/test-scaffold.sh index 31c844c..4b3847e 100755 --- a/tests/setup-repo/test-scaffold.sh +++ b/tests/setup-repo/test-scaffold.sh @@ -19,10 +19,28 @@ done python3 -c "import json; json.load(open('$TMP/.claude/settings.json'))" python3 -c "import tomllib; tomllib.load(open('$TMP/pyproject.toml','rb'))" -# non-clobber: second run skips everything and reports it +# non-clobber: second run skips everything and reports it, leaving files byte-identical +before="$(shasum "$TMP/pyproject.toml")" out="$(bash "$ENGINE" "$TMP" python)" echo "$out" | grep -q "SKIPPED" || { echo "expected SKIPPED report" >&2; exit 1; } echo "$out" | grep -q "pyproject.toml" || { echo "expected skipped file listed" >&2; exit 1; } +after="$(shasum "$TMP/pyproject.toml")" +[ "$before" = "$after" ] || { echo "skipped file was modified" >&2; exit 1; } + +# none stack: writes the always-files, omits all python-only files +TMP_NONE="$(mktemp -d)" +trap 'rm -rf "$TMP" "$TMP_NONE"' EXIT +git -C "$TMP_NONE" init -q +bash "$ENGINE" "$TMP_NONE" none +for f in .claude/settings.json CLAUDE.md .claude/rules/security.md \ + .claude/rules/code-style.md .github/workflows/claude.yml \ + .github/workflows/claude-code-review.yml .editorconfig; do + test -e "$TMP_NONE/$f" || { echo "MISSING (none stack): $f" >&2; exit 1; } +done +for f in pyproject.toml requirements.txt requirements-dev.txt .gitignore \ + tests/.gitkeep .github/workflows/ci.yml; do + test -e "$TMP_NONE/$f" && { echo "UNEXPECTED (none stack): $f" >&2; exit 1; } +done # scaffolded python defaults are internally consistent mkdir -p "$TMP/src" @@ -50,8 +68,11 @@ else echo "pytest not installed; skipped" fi -# unknown stack exits 2 -if bash "$ENGINE" "$TMP" bogus 2>/dev/null; then - echo "expected non-zero exit for unknown stack" >&2; exit 1 -fi +# unknown stack exits 2 and writes nothing +BOGUS="$(mktemp -d)" +trap 'rm -rf "$TMP" "$TMP_NONE" "$BOGUS"' EXIT +git -C "$BOGUS" init -q +set +e; bash "$ENGINE" "$BOGUS" bogus >/dev/null 2>&1; rc=$?; set -e +[ "$rc" -eq 2 ] || { echo "expected exit 2 for unknown stack, got $rc" >&2; exit 1; } +[ -z "$(ls -A "$BOGUS" | grep -v '^.git$' || true)" ] || { echo "unknown stack wrote files" >&2; exit 1; } echo "ALL CHECKS PASSED" From 67bc5ab16c6a583b3ea9b9c6dfdf0fb5255258ba Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Tue, 14 Jul 2026 17:56:33 +0100 Subject: [PATCH 15/27] Add setup-repo skill procedure --- plugins/core/skills/setup-repo/SKILL.md | 45 +++++++++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100644 plugins/core/skills/setup-repo/SKILL.md diff --git a/plugins/core/skills/setup-repo/SKILL.md b/plugins/core/skills/setup-repo/SKILL.md new file mode 100644 index 0000000..84d96b8 --- /dev/null +++ b/plugins/core/skills/setup-repo/SKILL.md @@ -0,0 +1,45 @@ +--- +name: setup-repo +description: Use when setting up a new or blank Offworld Labs repository, or when the user asks to "set up this repo per claude-shared", scaffold a repo to org standards, or add the standard Claude/CI/tooling setup. Scaffolds Claude Code enablement, review workflows, and stack tooling. +--- + +# Set up a repo per claude-shared + +Scaffold the current repository to Offworld Labs standards: Claude Code +enablement, the Claude review workflows, and the chosen stack's tooling. The +mechanical file copying is done by the bundled engine; you handle the +interactive parts and the report. + +`ENGINE="${CLAUDE_PLUGIN_ROOT}/skills/setup-repo/scripts/scaffold-repo.sh"` + +## Procedure + +1. **Confirm the target.** Ensure the working directory is a git repo + (`git rev-parse --is-inside-work-tree`). If it is not, offer to run + `git init`; abort if the user declines. + +2. **Determine the stack.** Detect from existing files: `pyproject.toml` or + `requirements*.txt` → `python`. If ambiguous or empty, ask the user to choose + `python` or `none` (ts-frontend / ts-backend arrive in a later phase). + +3. **Scaffold the files.** Run the engine, which never overwrites existing files: + `bash "$ENGINE" . ` + Relay its `WRITTEN` / `SKIPPED` output to the user. + +4. **Install dependencies.** For `python`, run + `pip install -r requirements.txt -r requirements-dev.txt` + (prefer an active virtualenv). Report the command and result; if the + toolchain is unavailable, skip and note it rather than failing. + +5. **Flesh out CLAUDE.md.** The stub was just written. Ask the user for a + one-or-two-line description of what this repo does, then fill in the + `Project Overview`, `Build & Test Commands`, and `Local Architecture` + sections from their answer plus what was scaffolded (stack, `ruff check .`, + `ruff format --check .`, `pytest`). If they skip, leave the stub as-is. Keep + CLAUDE.md under the 200-line ceiling noted in the template. + +6. **Report and follow-ups.** Summarise files written vs skipped, then list the + manual steps: add the `CLAUDE_CODE_OAUTH_TOKEN` repo secret, and commit the + workflows to the default branch before Claude review runs (see + `docs/runbooks/github-actions-claude-review.md` for why). Do not commit on the + user's behalf unless asked. From 2a2add0ee231b2e8adaefa22654bf1dd2b19378b Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Tue, 14 Jul 2026 18:02:56 +0100 Subject: [PATCH 16/27] Address final review: fix runbook path, dedup README, normalize test bits, tidy skill description --- README.md | 13 +++---------- docs/runbooks/github-actions-claude-review.md | 14 ++++++++++---- plugins/core/skills/setup-repo/SKILL.md | 2 +- tests/setup-repo/test-ci-assets.sh | 0 4 files changed, 14 insertions(+), 15 deletions(-) mode change 100644 => 100755 tests/setup-repo/test-ci-assets.sh diff --git a/README.md b/README.md index 0181d73..bac9642 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ plus **shared reference docs** used across every repo in the organisation. ## Install -In any repo, add the marketplace and install the plugin: +Install once per machine, at user scope: ``` /plugin marketplace add offworldlabs/claude-shared @@ -17,15 +17,8 @@ In any repo, add the marketplace and install the plugin: ## Adopting in a consuming repo -Adoption is driven by Claude Code, not manual copying. Once, per machine, install -the plugin at user scope: - -``` -/plugin marketplace add offworldlabs/claude-shared -/plugin install core@offworld -``` - -Then, in any new repo, start Claude Code and ask: +Adoption is driven by Claude Code, not manual copying. With `core` installed +(see Install above), open any new repo, start Claude Code, and ask: > set this repo up per `claude-shared` diff --git a/docs/runbooks/github-actions-claude-review.md b/docs/runbooks/github-actions-claude-review.md index e73c152..8baf1d1 100644 --- a/docs/runbooks/github-actions-claude-review.md +++ b/docs/runbooks/github-actions-claude-review.md @@ -44,13 +44,19 @@ permissions: id-token: write ``` -The fastest path in a new repo is to skip the broken scaffold entirely and copy -the known-good templates from this repo: +The fastest path in a new repo is to skip the broken scaffold entirely and run +the `core:setup-repo` skill, which installs the corrected workflows. Ask +Claude: + +> set this repo up per `claude-shared` + +If you need to copy the workflows manually, they live in the `core` plugin's +assets and copy as-is into `.github/workflows/` (no leading-dot rename needed): ```bash mkdir -p .github/workflows -cp path/to/claude-shared/templates/github-workflows/claude-code-review.yml .github/workflows/ -cp path/to/claude-shared/templates/github-workflows/claude.yml .github/workflows/ +cp path/to/claude-shared/plugins/core/skills/setup-repo/assets/ci/claude-code-review.yml .github/workflows/ +cp path/to/claude-shared/plugins/core/skills/setup-repo/assets/ci/claude.yml .github/workflows/ ``` ## Critical gotcha: the fix must land on the default branch first diff --git a/plugins/core/skills/setup-repo/SKILL.md b/plugins/core/skills/setup-repo/SKILL.md index 84d96b8..b53dc41 100644 --- a/plugins/core/skills/setup-repo/SKILL.md +++ b/plugins/core/skills/setup-repo/SKILL.md @@ -1,6 +1,6 @@ --- name: setup-repo -description: Use when setting up a new or blank Offworld Labs repository, or when the user asks to "set up this repo per claude-shared", scaffold a repo to org standards, or add the standard Claude/CI/tooling setup. Scaffolds Claude Code enablement, review workflows, and stack tooling. +description: Use when setting up a new or blank Offworld Labs repository, or when the user asks to "set up this repo per claude-shared", scaffold a repo to org standards, or add the standard Claude/CI/tooling setup. Scaffolds Claude Code enablement, the Claude review workflows, and stack-specific tooling for the repo. --- # Set up a repo per claude-shared diff --git a/tests/setup-repo/test-ci-assets.sh b/tests/setup-repo/test-ci-assets.sh old mode 100644 new mode 100755 From 50593d53c9f9fe930350cef75ade7071ca17e03c Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Wed, 15 Jul 2026 16:41:31 +0100 Subject: [PATCH 17/27] Tidy PR: remove doc stubs, drop superpowers dir, document docs/ subdirs - Remove the four placeholder docs stubs (architecture.md and the contracts/ decisions/runbooks README stubs); keep contracts/ and decisions/ as reserved dirs via .gitkeep, runbooks/ keeps its real runbook. - Remove docs/superpowers/ (spec + plan planning artifacts) and add .gitignore ignoring docs/superpowers/ and the .superpowers/ SDD scratch dir. - Add a README "Documentation" section describing each docs/ subdirectory. Co-Authored-By: Claude Opus 4.8 (1M context) --- .gitignore | 3 + README.md | 19 +- docs/architecture.md | 10 - docs/contracts/.gitkeep | 0 docs/contracts/README.md | 14 - docs/decisions/.gitkeep | 0 docs/decisions/README.md | 10 - docs/runbooks/README.md | 10 - .../plans/2026-07-14-setup-repo-phase-1.md | 601 ------------------ .../2026-07-14-new-repo-setup-skill-design.md | 180 ------ 10 files changed, 21 insertions(+), 826 deletions(-) create mode 100644 .gitignore delete mode 100644 docs/architecture.md create mode 100644 docs/contracts/.gitkeep delete mode 100644 docs/contracts/README.md create mode 100644 docs/decisions/.gitkeep delete mode 100644 docs/decisions/README.md delete mode 100644 docs/runbooks/README.md delete mode 100644 docs/superpowers/plans/2026-07-14-setup-repo-phase-1.md delete mode 100644 docs/superpowers/specs/2026-07-14-new-repo-setup-skill-design.md diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..5000dfc --- /dev/null +++ b/.gitignore @@ -0,0 +1,3 @@ +# Superpowers planning artifacts and subagent-driven-development scratch +docs/superpowers/ +.superpowers/ diff --git a/README.md b/README.md index bac9642..1994dea 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ Offworld Labs' org-wide Claude Code resource: a **plugin marketplace** (`offworl plus **shared reference docs** used across every repo in the organisation. - `plugins/core` — the `core` plugin; its `setup-repo` skill bundles the shared rules, `.claude/settings.json`, `CLAUDE.md`, and CI workflow templates used to scaffold new repos. -- `docs/` — on-demand org-wide docs (architecture, contracts, decisions, runbooks). +- `docs/` — on-demand org-wide reference docs (see [Documentation](#documentation)). ## Install @@ -28,6 +28,23 @@ the Claude review workflows, and your stack's tooling — then installs deps and helps you flesh out `CLAUDE.md`. See `docs/runbooks/github-actions-claude-review.md` for the one manual follow-up (the `CLAUDE_CODE_OAUTH_TOKEN` secret). +## Documentation + +Org-wide reference docs live under `docs/`. A consuming repo's `CLAUDE.md` should +point at these rather than duplicating them, so there's one source of truth. Each +subdirectory: + +- **`docs/contracts/`** — the source of truth for cross-service interfaces: API + schemas, event/message formats, and shared data structures. When two services + communicate, the contract lives here and consuming repos reference it instead of + copying it, so it can't drift. +- **`docs/decisions/`** — Architecture Decision Records (ADRs). One append-only + record per significant, org-wide technical decision: its context, the options + considered, the decision made, and the consequences. +- **`docs/runbooks/`** — operational procedures: deployments, incident response, + rollbacks, and recovery playbooks. Concrete enough to follow under pressure — + exact commands, expected output, and escalation paths. + ## Contributing Skills graduate from personal experimentation into the shared `core` plugin diff --git a/docs/architecture.md b/docs/architecture.md deleted file mode 100644 index a4133c2..0000000 --- a/docs/architecture.md +++ /dev/null @@ -1,10 +0,0 @@ -# Architecture - -Org-wide architecture overview for Offworld Labs. - -This document describes how services fit together across the organisation: the -high-level system map, major components and their responsibilities, and the data -flows between them. Individual repos should describe their *local* architecture -in their own `CLAUDE.md` and link here for the cross-service picture. - -> Stub — fill in the org-wide system diagram and component responsibilities. diff --git a/docs/contracts/.gitkeep b/docs/contracts/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/docs/contracts/README.md b/docs/contracts/README.md deleted file mode 100644 index ee75b29..0000000 --- a/docs/contracts/README.md +++ /dev/null @@ -1,14 +0,0 @@ -# Contracts - -Cross-service interface definitions for Offworld Labs. - -The files in this directory are the **source of truth** for interfaces shared -between services: API schemas, event/message formats, shared data structures, -and versioning expectations. When two services communicate, the contract lives -here — not duplicated in each repo. - -**Consuming repos should point to these files from their `CLAUDE.md`** rather -than copying contract definitions locally. Duplicated contracts drift; a single -referenced source does not. - -> Stub — add one file per cross-service contract. diff --git a/docs/decisions/.gitkeep b/docs/decisions/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/docs/decisions/README.md b/docs/decisions/README.md deleted file mode 100644 index ebff38d..0000000 --- a/docs/decisions/README.md +++ /dev/null @@ -1,10 +0,0 @@ -# Decisions - -Architecture Decision Records (ADRs) for Offworld Labs. - -This directory captures significant, org-wide technical decisions: the context, -the options considered, the decision made, and its consequences. One record per -decision, kept append-only so the reasoning behind past choices stays -discoverable. - -> Stub — add one dated ADR file per significant decision. diff --git a/docs/runbooks/README.md b/docs/runbooks/README.md deleted file mode 100644 index a3fc546..0000000 --- a/docs/runbooks/README.md +++ /dev/null @@ -1,10 +0,0 @@ -# Runbooks - -Operational runbooks for Offworld Labs. - -This directory holds step-by-step operational procedures: deployments, incident -response, rollbacks, and recovery playbooks. Each runbook should be concrete -enough to follow under pressure — exact commands, expected output, and -escalation paths. - -> Stub — add one runbook file per operational procedure. diff --git a/docs/superpowers/plans/2026-07-14-setup-repo-phase-1.md b/docs/superpowers/plans/2026-07-14-setup-repo-phase-1.md deleted file mode 100644 index e64d844..0000000 --- a/docs/superpowers/plans/2026-07-14-setup-repo-phase-1.md +++ /dev/null @@ -1,601 +0,0 @@ -# `core:setup-repo` Phase 1 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Ship a `core:setup-repo` skill that scaffolds a new org repo to standard (Claude enablement + Claude workflows + Python stack), driven by Claude Code, with all files bundled as plugin assets. - -**Architecture:** The plugin is the single source of truth. Static template files live under `plugins/core/skills/setup-repo/assets/`. A deterministic bash engine (`scaffold-repo.sh`) copies them into a target repo without clobbering existing files; the `SKILL.md` procedure orchestrates the engine plus the interactive parts (stack choice, first install, fleshing out `CLAUDE.md`). PR #1's root `templates/` and `rules/` are folded into the plugin. - -**Tech Stack:** Claude Code plugin (marketplace `offworld`), bash (engine + test harness), Python tooling defaults (ruff, pytest) as scaffolded assets. - -## Global Constraints - -- Marketplace name: `offworld`; plugin: `core`; skill namespace: `core:setup-repo`. -- Only `plugin.json`/`marketplace.json` live in `.claude-plugin/`; components at plugin root. -- Delivery model: self-contained plugin; skill references assets via `${CLAUDE_PLUGIN_ROOT}`. No runtime fetch. -- Python defaults (verbatim): ruff `target-version = "py312"`, `line-length = 120`, `[tool.ruff.lint] select = ["E", "F", "W"]`, `[tool.ruff.format] quote-style = "double"`; pytest `testpaths = ["tests"]`; pip + `requirements.txt` / `requirements-dev.txt`. -- Dotfile assets are stored WITHOUT the leading dot (`gitignore`, `editorconfig`) so they aren't hidden/ignored in the marketplace repo; the engine renames them on copy. -- Idempotent, never silently clobber: existing files are skipped and reported, never overwritten. -- CI workflows carry the write-permission fix (`pull-requests: write`, `issues: write`); they only take effect once on the target repo's default branch. -- Working branch for execution: `feat/offworld-marketplace-scaffold` (continues PR #1 into the Phase 1 deliverable). Phase 2 (ts-frontend/ts-backend) and Phase 3 (pre-commit, C++) are out of scope here. - ---- - -### Task 1: Restructure — fold PR #1 templates into plugin assets - -**Files:** -- Move: `templates/settings.json` → `plugins/core/skills/setup-repo/assets/claude/settings.json` -- Move: `templates/CLAUDE.md` → `plugins/core/skills/setup-repo/assets/claude/CLAUDE.md` -- Move: `templates/github-workflows/claude-code-review.yml` → `plugins/core/skills/setup-repo/assets/ci/claude-code-review.yml` -- Move: `templates/github-workflows/claude.yml` → `plugins/core/skills/setup-repo/assets/ci/claude.yml` -- Move: `rules/security.md` → `plugins/core/skills/setup-repo/assets/rules/security.md` -- Move: `rules/code-style.md` → `plugins/core/skills/setup-repo/assets/rules/code-style.md` -- Delete: root `templates/`, root `rules/` -- Modify: `README.md` (adoption section) - -**Interfaces:** -- Produces: the `assets/claude/`, `assets/ci/`, `assets/rules/` directories that Task 4's engine copies from. - -- [ ] **Step 1: Move the files with git mv** - -```bash -cd "$(git rev-parse --show-toplevel)" -mkdir -p plugins/core/skills/setup-repo/assets/{claude,ci,rules} -git mv templates/settings.json plugins/core/skills/setup-repo/assets/claude/settings.json -git mv templates/CLAUDE.md plugins/core/skills/setup-repo/assets/claude/CLAUDE.md -git mv templates/github-workflows/claude-code-review.yml plugins/core/skills/setup-repo/assets/ci/claude-code-review.yml -git mv templates/github-workflows/claude.yml plugins/core/skills/setup-repo/assets/ci/claude.yml -git mv rules/security.md plugins/core/skills/setup-repo/assets/rules/security.md -git mv rules/code-style.md plugins/core/skills/setup-repo/assets/rules/code-style.md -rmdir templates/github-workflows templates rules 2>/dev/null || true -``` - -- [ ] **Step 2: Update the rules-file symlink comment** - -The two rules files still contain a comment saying they get "symlinked into each repo's `.claude/rules/`". The model is now copy, not symlink. In BOTH `plugins/core/skills/setup-repo/assets/rules/security.md` and `.../code-style.md`, replace the comment block's line: - -``` - These files are symlinked into each consuming repo's .claude/rules/ so every -``` - -with: - -``` - The setup-repo skill copies these into each consuming repo's .claude/rules/ so every -``` - -- [ ] **Step 3: Rewrite the README adoption section** - -In `README.md`, replace the entire "## Adopting in a consuming repo" section (down to, but not including, "## Contributing") with: - -```markdown -## Adopting in a consuming repo - -Adoption is driven by Claude Code, not manual copying. Once, per machine, install -the plugin at user scope: - -``` -/plugin marketplace add offworldlabs/claude-shared -/plugin install core@offworld -``` - -Then, in any new repo, start Claude Code and ask: - -> set this repo up per `claude-shared` - -Claude invokes the `core:setup-repo` skill, which writes `.claude/settings.json` -(registering the marketplace and enabling `core`), a `CLAUDE.md`, the shared rules, -the Claude review workflows, and your stack's tooling — then installs deps and -helps you flesh out `CLAUDE.md`. See `docs/runbooks/github-actions-claude-review.md` -for the one manual follow-up (the `CLAUDE_CODE_OAUTH_TOKEN` secret). -``` - -- [ ] **Step 4: Validate the plugin still loads** - -Run: `claude plugin validate .` -Expected: `Validation passed` (no error). The moved workflow/settings files are now plugin assets, not components, so they don't affect validation. - -- [ ] **Step 5: Verify old paths are gone and new ones exist** - -Run: -```bash -test ! -d templates && test ! -d rules && echo "root dirs removed" -ls plugins/core/skills/setup-repo/assets/claude/settings.json \ - plugins/core/skills/setup-repo/assets/ci/claude.yml \ - plugins/core/skills/setup-repo/assets/rules/security.md -``` -Expected: prints "root dirs removed" and lists the three files with no error. - -- [ ] **Step 6: Commit** - -```bash -git add -A -git commit -m "Fold root templates and rules into setup-repo plugin assets" -``` - ---- - -### Task 2: Python stack assets - -**Files:** -- Create: `plugins/core/skills/setup-repo/assets/stack/python/pyproject.toml` -- Create: `plugins/core/skills/setup-repo/assets/stack/python/requirements.txt` -- Create: `plugins/core/skills/setup-repo/assets/stack/python/requirements-dev.txt` -- Create: `plugins/core/skills/setup-repo/assets/stack/python/gitignore` -- Create: `plugins/core/skills/setup-repo/assets/stack/python/tests/.gitkeep` - -**Interfaces:** -- Produces: the `assets/stack/python/` tree Task 4's engine copies when `stack == python`. - -- [ ] **Step 1: Write a failing validation check** - -Create `tests/setup-repo/test-python-assets.sh`: - -```bash -#!/usr/bin/env bash -set -euo pipefail -ROOT="$(git rev-parse --show-toplevel)" -PY="$ROOT/plugins/core/skills/setup-repo/assets/stack/python" - -python3 - "$PY/pyproject.toml" <<'EOF' -import sys, tomllib -data = tomllib.load(open(sys.argv[1], "rb")) -ruff = data["tool"]["ruff"] -assert ruff["target-version"] == "py312", ruff -assert ruff["line-length"] == 120, ruff -assert data["tool"]["ruff"]["lint"]["select"] == ["E", "F", "W"], ruff -assert data["tool"]["ruff"]["format"]["quote-style"] == "double", ruff -assert data["tool"]["pytest"]["ini_options"]["testpaths"] == ["tests"], data -print("pyproject.toml OK") -EOF - -grep -q "ruff" "$PY/requirements-dev.txt" -grep -q "pytest" "$PY/requirements-dev.txt" -test -f "$PY/gitignore" && grep -q "__pycache__" "$PY/gitignore" -test -f "$PY/tests/.gitkeep" -echo "python assets OK" -``` - -- [ ] **Step 2: Run it to verify it fails** - -Run: `bash tests/setup-repo/test-python-assets.sh` -Expected: FAIL (the `assets/stack/python/*` files don't exist yet — `tomllib.load` raises `FileNotFoundError`). - -- [ ] **Step 3: Create the Python assets** - -`plugins/core/skills/setup-repo/assets/stack/python/pyproject.toml`: -```toml -[tool.ruff] -target-version = "py312" -line-length = 120 - -[tool.ruff.lint] -select = ["E", "F", "W"] - -[tool.ruff.format] -quote-style = "double" - -[tool.pytest.ini_options] -testpaths = ["tests"] -``` - -`plugins/core/skills/setup-repo/assets/stack/python/requirements.txt`: -``` -# Runtime dependencies for this project. Add them below, pinned where practical. -``` - -`plugins/core/skills/setup-repo/assets/stack/python/requirements-dev.txt`: -``` -ruff>=0.8.0 -pytest>=8.0.0 -``` - -`plugins/core/skills/setup-repo/assets/stack/python/gitignore`: -``` -__pycache__/ -*.py[cod] -.pytest_cache/ -.ruff_cache/ -.venv/ -venv/ -*.egg-info/ -build/ -dist/ -.env -``` - -`plugins/core/skills/setup-repo/assets/stack/python/tests/.gitkeep`: (empty file) - -- [ ] **Step 4: Run the check to verify it passes** - -Run: `bash tests/setup-repo/test-python-assets.sh` -Expected: prints `pyproject.toml OK` then `python assets OK`. - -- [ ] **Step 5: Commit** - -```bash -git add plugins/core/skills/setup-repo/assets/stack/python tests/setup-repo/test-python-assets.sh -git commit -m "Add Python stack assets (ruff py312, pytest) with validation" -``` - ---- - -### Task 3: CI and editorconfig assets - -**Files:** -- Create: `plugins/core/skills/setup-repo/assets/ci/ci-python.yml` -- Create: `plugins/core/skills/setup-repo/assets/editorconfig` - -**Interfaces:** -- Produces: `assets/ci/ci-python.yml` (copied to `.github/workflows/ci.yml` for python) and `assets/editorconfig` (copied to `.editorconfig`), both used by Task 4's engine. - -- [ ] **Step 1: Write a failing validation check** - -Create `tests/setup-repo/test-ci-assets.sh`: - -```bash -#!/usr/bin/env bash -set -euo pipefail -ROOT="$(git rev-parse --show-toplevel)" -CI="$ROOT/plugins/core/skills/setup-repo/assets/ci/ci-python.yml" -EC="$ROOT/plugins/core/skills/setup-repo/assets/editorconfig" - -python3 - "$CI" <<'EOF' -import sys -try: - import yaml -except ModuleNotFoundError: - print("pyyaml missing; skipping YAML parse"); sys.exit(0) -doc = yaml.safe_load(open(sys.argv[1])) -jobs = doc["jobs"]["lint-and-test"]["steps"] -names = [s.get("name", "") for s in jobs] -assert any("Ruff" in n for n in names), names -assert any("Pytest" in n or "pytest" in n for n in names), names -print("ci-python.yml OK") -EOF - -grep -q "root = true" "$EC" -grep -q "indent_size = 4" "$EC" # python -grep -q "indent_size = 2" "$EC" # js/ts/yaml -echo "ci assets OK" -``` - -- [ ] **Step 2: Run it to verify it fails** - -Run: `bash tests/setup-repo/test-ci-assets.sh` -Expected: FAIL (files don't exist; `open(sys.argv[1])` raises `FileNotFoundError`). - -- [ ] **Step 3: Create the CI and editorconfig assets** - -`plugins/core/skills/setup-repo/assets/ci/ci-python.yml`: -```yaml -name: CI - -on: - push: - branches: [main] - pull_request: - -jobs: - lint-and-test: - runs-on: ubuntu-latest - steps: - - name: Checkout - uses: actions/checkout@v4 - - - name: Set up Python - uses: actions/setup-python@v5 - with: - python-version: "3.12" - - - name: Install dependencies - run: | - python -m pip install --upgrade pip - pip install -r requirements.txt -r requirements-dev.txt - - - name: Ruff lint - run: ruff check . - - - name: Ruff format check - run: ruff format --check . - - - name: Pytest - run: pytest -``` - -`plugins/core/skills/setup-repo/assets/editorconfig`: -``` -root = true - -[*] -charset = utf-8 -end_of_line = lf -insert_final_newline = true -trim_trailing_whitespace = true -indent_style = space - -[*.py] -indent_size = 4 - -[*.{js,jsx,ts,tsx,json,yml,yaml}] -indent_size = 2 - -[*.md] -trim_trailing_whitespace = false -``` - -- [ ] **Step 4: Run the check to verify it passes** - -Run: `bash tests/setup-repo/test-ci-assets.sh` -Expected: prints `ci-python.yml OK` (or the pyyaml-missing skip line) then `ci assets OK`. - -- [ ] **Step 5: Commit** - -```bash -git add plugins/core/skills/setup-repo/assets/ci/ci-python.yml plugins/core/skills/setup-repo/assets/editorconfig tests/setup-repo/test-ci-assets.sh -git commit -m "Add Python CI workflow and shared editorconfig assets" -``` - ---- - -### Task 4: Deterministic scaffold engine + end-to-end test - -**Files:** -- Create: `plugins/core/skills/setup-repo/scripts/scaffold-repo.sh` -- Create: `tests/setup-repo/test-scaffold.sh` - -**Interfaces:** -- Consumes: all `assets/**` from Tasks 1-3. -- Produces: `scaffold-repo.sh ` where `` is `python` or `none`; copies assets into ``, renaming `gitignore`→`.gitignore` and `editorconfig`→`.editorconfig`, skipping (not overwriting) existing files, and printing `WRITTEN:` / `SKIPPED:` reports. Exit code 2 on unknown stack. Task 5's SKILL.md calls this. - -- [ ] **Step 1: Write the failing end-to-end test** - -Create `tests/setup-repo/test-scaffold.sh`: - -```bash -#!/usr/bin/env bash -set -euo pipefail -ROOT="$(git rev-parse --show-toplevel)" -ENGINE="$ROOT/plugins/core/skills/setup-repo/scripts/scaffold-repo.sh" -TMP="$(mktemp -d)" -trap 'rm -rf "$TMP"' EXIT -git -C "$TMP" init -q - -bash "$ENGINE" "$TMP" python - -for f in .claude/settings.json CLAUDE.md .claude/rules/security.md \ - .claude/rules/code-style.md .github/workflows/claude.yml \ - .github/workflows/claude-code-review.yml .github/workflows/ci.yml \ - .editorconfig pyproject.toml requirements.txt requirements-dev.txt \ - .gitignore tests/.gitkeep; do - test -e "$TMP/$f" || { echo "MISSING: $f" >&2; exit 1; } -done - -python3 -c "import json; json.load(open('$TMP/.claude/settings.json'))" -python3 -c "import tomllib; tomllib.load(open('$TMP/pyproject.toml','rb'))" - -# non-clobber: second run skips everything and reports it -out="$(bash "$ENGINE" "$TMP" python)" -echo "$out" | grep -q "SKIPPED" || { echo "expected SKIPPED report" >&2; exit 1; } -echo "$out" | grep -q "pyproject.toml" || { echo "expected skipped file listed" >&2; exit 1; } - -# scaffolded python defaults are internally consistent -mkdir -p "$TMP/src" -cat > "$TMP/src/example.py" <<'PY' -def add(a: int, b: int) -> int: - return a + b -PY -cat > "$TMP/tests/test_example.py" <<'PY' -from src.example import add - - -def test_add() -> None: - assert add(2, 3) == 5 -PY -if command -v ruff >/dev/null 2>&1; then - ( cd "$TMP" && ruff format . >/dev/null && ruff check . && ruff format --check . ) - echo "ruff OK" -else - echo "ruff not installed; skipped" -fi -if command -v pytest >/dev/null 2>&1; then - ( cd "$TMP" && PYTHONPATH=. pytest -q ) - echo "pytest OK" -else - echo "pytest not installed; skipped" -fi - -# unknown stack exits 2 -if bash "$ENGINE" "$TMP" bogus 2>/dev/null; then - echo "expected non-zero exit for unknown stack" >&2; exit 1 -fi -echo "ALL CHECKS PASSED" -``` - -- [ ] **Step 2: Run it to verify it fails** - -Run: `bash tests/setup-repo/test-scaffold.sh` -Expected: FAIL (`scaffold-repo.sh` does not exist — `bash "$ENGINE"` errors "No such file or directory"). - -- [ ] **Step 3: Write the engine** - -`plugins/core/skills/setup-repo/scripts/scaffold-repo.sh`: -```bash -#!/usr/bin/env bash -# Deterministic file-scaffolding engine for the core:setup-repo skill. -# Copies bundled assets into a target repo without clobbering existing files. -# Usage: scaffold-repo.sh -# : python | none (ts-frontend / ts-backend land in Phase 2) -set -euo pipefail - -TARGET="${1:?target dir required}" -STACK="${2:-none}" -ASSETS="$(cd "$(dirname "${BASH_SOURCE[0]}")/../assets" && pwd)" - -written=() -skipped=() - -copy() { # copy - local src="$1" dest="$2" - mkdir -p "$(dirname "$dest")" - if [[ -e "$dest" ]]; then - skipped+=("$dest") - else - cp "$src" "$dest" - written+=("$dest") - fi -} - -# Claude enablement (always) -copy "$ASSETS/claude/settings.json" "$TARGET/.claude/settings.json" -copy "$ASSETS/claude/CLAUDE.md" "$TARGET/CLAUDE.md" -copy "$ASSETS/rules/security.md" "$TARGET/.claude/rules/security.md" -copy "$ASSETS/rules/code-style.md" "$TARGET/.claude/rules/code-style.md" - -# Claude review workflows (always) -copy "$ASSETS/ci/claude-code-review.yml" "$TARGET/.github/workflows/claude-code-review.yml" -copy "$ASSETS/ci/claude.yml" "$TARGET/.github/workflows/claude.yml" - -# Shared -copy "$ASSETS/editorconfig" "$TARGET/.editorconfig" - -# Stack -case "$STACK" in - python) - copy "$ASSETS/stack/python/pyproject.toml" "$TARGET/pyproject.toml" - copy "$ASSETS/stack/python/requirements.txt" "$TARGET/requirements.txt" - copy "$ASSETS/stack/python/requirements-dev.txt" "$TARGET/requirements-dev.txt" - copy "$ASSETS/stack/python/gitignore" "$TARGET/.gitignore" - copy "$ASSETS/stack/python/tests/.gitkeep" "$TARGET/tests/.gitkeep" - copy "$ASSETS/ci/ci-python.yml" "$TARGET/.github/workflows/ci.yml" - ;; - none) ;; - *) echo "unknown stack: $STACK" >&2; exit 2 ;; -esac - -echo "WRITTEN:" -printf ' %s\n' "${written[@]:-(none)}" -echo "SKIPPED (already present, left untouched):" -printf ' %s\n' "${skipped[@]:-(none)}" -``` - -- [ ] **Step 4: Make it executable and run the test** - -Run: -```bash -chmod +x plugins/core/skills/setup-repo/scripts/scaffold-repo.sh -bash tests/setup-repo/test-scaffold.sh -``` -Expected: ends with `ALL CHECKS PASSED` (with `ruff OK` / `pytest OK` if those tools are installed, else the "skipped" lines). - -- [ ] **Step 5: Commit** - -```bash -git add plugins/core/skills/setup-repo/scripts/scaffold-repo.sh tests/setup-repo/test-scaffold.sh -git commit -m "Add deterministic scaffold engine with end-to-end test" -``` - ---- - -### Task 5: The `setup-repo` SKILL.md procedure - -**Files:** -- Create: `plugins/core/skills/setup-repo/SKILL.md` - -**Interfaces:** -- Consumes: `scripts/scaffold-repo.sh` (Task 4) via `${CLAUDE_PLUGIN_ROOT}/skills/setup-repo/scripts/scaffold-repo.sh`. -- Produces: the model-invocable `core:setup-repo` skill. - -- [ ] **Step 1: Write the SKILL.md** - -`plugins/core/skills/setup-repo/SKILL.md`: -```markdown ---- -name: setup-repo -description: Use when setting up a new or blank Offworld Labs repository, or when the user asks to "set up this repo per claude-shared", scaffold a repo to org standards, or add the standard Claude/CI/tooling setup. Scaffolds Claude Code enablement, review workflows, and stack tooling. ---- - -# Set up a repo per claude-shared - -Scaffold the current repository to Offworld Labs standards: Claude Code -enablement, the Claude review workflows, and the chosen stack's tooling. The -mechanical file copying is done by the bundled engine; you handle the -interactive parts and the report. - -`ENGINE="${CLAUDE_PLUGIN_ROOT}/skills/setup-repo/scripts/scaffold-repo.sh"` - -## Procedure - -1. **Confirm the target.** Ensure the working directory is a git repo - (`git rev-parse --is-inside-work-tree`). If it is not, offer to run - `git init`; abort if the user declines. - -2. **Determine the stack.** Detect from existing files: `pyproject.toml` or - `requirements*.txt` → `python`. If ambiguous or empty, ask the user to choose - `python` or `none` (ts-frontend / ts-backend arrive in a later phase). - -3. **Scaffold the files.** Run the engine, which never overwrites existing files: - `bash "$ENGINE" . ` - Relay its `WRITTEN` / `SKIPPED` output to the user. - -4. **Install dependencies.** For `python`, run - `pip install -r requirements.txt -r requirements-dev.txt` - (prefer an active virtualenv). Report the command and result; if the - toolchain is unavailable, skip and note it rather than failing. - -5. **Flesh out CLAUDE.md.** The stub was just written. Ask the user for a - one-or-two-line description of what this repo does, then fill in the - `Project Overview`, `Build & Test Commands`, and `Local Architecture` - sections from their answer plus what was scaffolded (stack, `ruff check .`, - `ruff format --check .`, `pytest`). If they skip, leave the stub as-is. Keep - CLAUDE.md under the 200-line ceiling noted in the template. - -6. **Report and follow-ups.** Summarise files written vs skipped, then list the - manual steps: add the `CLAUDE_CODE_OAUTH_TOKEN` repo secret, and commit the - workflows to the default branch before Claude review runs (see - `docs/runbooks/github-actions-claude-review.md` for why). Do not commit on the - user's behalf unless asked. -``` - -- [ ] **Step 2: Validate the plugin and skill frontmatter** - -Run: `claude plugin validate ./plugins/core` -Expected: `Validation passed` (the `setup-repo` skill frontmatter parses; no YAML errors). - -- [ ] **Step 3: Confirm the skill is discoverable** - -Run: -```bash -claude plugin validate . -claude -p "List available skills whose name contains 'setup-repo'. Answer with just the namespaced skill name(s), or 'none'." --allowedTools "" 2>&1 | tail -3 -``` -Expected: `Validation passed`, and the model prints `core:setup-repo` (requires the plugin be installed/loadable; if it prints `none`, run `/reload-plugins` in an interactive session or reinstall `core@offworld` from the local marketplace first). - -- [ ] **Step 4: Commit** - -```bash -git add plugins/core/skills/setup-repo/SKILL.md -git commit -m "Add setup-repo skill procedure" -``` - ---- - -## Self-Review - -**Spec coverage:** -- Delivery model (self-contained plugin, `${CLAUDE_PLUGIN_ROOT}`) → Tasks 1, 4, 5. -- Python defaults (ruff py312/120/EFW/double, pytest) → Task 2, verified in Tasks 2 & 4. -- Repo restructure (fold `templates/`+`rules/`, keep `docs/`) → Task 1. -- Rules copied not symlinked → Task 1 Step 2 + engine (Task 4). -- CI incl. write-permission workflows + `ci-python.yml` → Tasks 1, 3, 4. -- `.editorconfig` → Task 3. -- Procedure steps 1-9 (confirm, detect, enable, CI, stack, shared, install, flesh CLAUDE.md, report) → Task 5 (mechanical copying delegated to the engine, Task 4). -- Idempotent / non-clobber → engine `copy()` + Task 4 test. -- Testing (asset validity, temp-repo scaffold, ruff/pytest) → Tasks 2, 3, 4. -- Prerequisite (core at user scope) + adoption flow → Task 1 README rewrite. -- Out of scope confirmed: ts stacks (Phase 2), C++/pre-commit (Phase 3), governance. - -**Placeholder scan:** none — every file's full contents and every command are inline. - -**Type/name consistency:** engine signature `scaffold-repo.sh ` with stacks `python|none`, dotfile renames `gitignore`→`.gitignore` / `editorconfig`→`.editorconfig`, and asset paths are identical across Tasks 1-5 and the tests. -``` diff --git a/docs/superpowers/specs/2026-07-14-new-repo-setup-skill-design.md b/docs/superpowers/specs/2026-07-14-new-repo-setup-skill-design.md deleted file mode 100644 index c500491..0000000 --- a/docs/superpowers/specs/2026-07-14-new-repo-setup-skill-design.md +++ /dev/null @@ -1,180 +0,0 @@ -# Design: `core:setup-repo` — one-command new-repo setup - -**Date:** 2026-07-14 -**Status:** Approved (pending spec review) -**Repo:** offworldlabs/claude-shared - -## Problem - -Setting up a new Offworld Labs repo to org standards (Claude Code enablement, CI, -stack tooling) is manual, undocumented, and inconsistent — the sibling-repo scan -found ruff configured in only 3 of ~10 Python repos, no shared formatter/test -conventions in TS, and no repo with pre-commit, `.editorconfig`, or version -pinning. `claude-shared` should be the single place that encodes "how we set up a -repo," and the setup should be driven **by Claude Code, not a human checklist**: -open a blank repo, start Claude, say *"set this repo up per `claude-shared`,"* and -it happens. - -## Goals - -- A `core:setup-repo` skill that scaffolds a new repo to org standard in one shot. -- Covers three categories: **Claude Code enablement**, **CI/CD workflows**, - **stack scaffolding**. (Repo governance — CODEOWNERS, branch protection — is - explicitly out of scope.) -- Works offline in any repo where `core` is installed; deterministic; idempotent; - never silently clobbers existing files. - -## Non-goals - -- Governance/hygiene files (CODEOWNERS, branch protection, PR/issue templates). -- C++ scaffolding (blah2-arm, retina-spectrum) — deferred to a later phase. -- Human-readable setup checklists as the primary interface (the skill is the - interface; docs are supporting reference). - -## Decisions (locked) - -### Delivery model — self-contained plugin -The skill and **all** template assets are bundled inside the plugin and copied -into the target repo via `${CLAUDE_PLUGIN_ROOT}`. No runtime fetch. The plugin -`version` bump remains the org-wide update signal. Rules are **copied** into -repos (not symlinked), because symlinking into the plugin cache is fragile. - -### Stack defaults (from the sibling-repo scan) -- **Python** (dominant, ~10 repos): pip + `requirements.txt`; **pytest**; **ruff** - with `target-version = "py312"`, `line-length = 120`, `select = ["E","F","W"]`, - `format.quote-style = "double"`. -- **TypeScript**: npm; two variants — - - `ts-frontend`: TypeScript + Vite + Vitest + ESLint 9 (flat) + typescript-eslint + React 18. - - `ts-backend`: TypeScript + tsx + Vitest + ESLint 9 (flat) + typescript-eslint. -- Greenfield additions (nothing in the org has them today): `.editorconfig`, - optional pre-commit. - -### Prerequisite -Devs install `core@offworld` at **user scope** once, so the skill is available in -a brand-new repo that has no `.claude/settings.json` yet. The skill then writes -the repo's own `.claude/settings.json`, making the repo self-enabling thereafter. - -## Repository restructure - -The plugin becomes the single source of truth for scaffolded files. The -`templates/` directory added in PR #1 is **folded into the plugin** (PR #1 is -unmerged, so we restructure rather than duplicate). `docs/` stays at the repo root -as the human/agent knowledge hub. Final layout: - -``` -plugins/core/ - .claude-plugin/plugin.json # version bump = update signal - skills/ - pr-description/SKILL.md # existing - setup-repo/ - SKILL.md # the ordered procedure - assets/ - claude/ - settings.json # extraKnownMarketplaces + enabledPlugins - CLAUDE.md # starter template (<200-line ceiling note) - rules/ - security.md - code-style.md - ci/ - claude-code-review.yml # write-permission fix baked in - claude.yml - ci-python.yml # ruff check + ruff format --check + pytest - ci-node.yml # eslint + tsc + vitest - stack/ - python/ - pyproject.toml # ruff py312/120/EFW + format; pytest config - requirements.txt - requirements-dev.txt # ruff, pytest - gitignore # Python (shipped without leading dot; skill renames) - tests/.gitkeep - ts-frontend/ - package.json # vite, react, vitest, eslint9, typescript-eslint - tsconfig.json - eslint.config.js - vitest.config.ts - gitignore # Node - ts-backend/ - package.json # tsx, tsc, vitest, eslint9, typescript-eslint - tsconfig.json - eslint.config.js - vitest.config.ts - gitignore # Node - editorconfig # shared across all repos -docs/ # stays at root (architecture, contracts, decisions, runbooks) -rules/ # REMOVED at root — canonical copy now lives in plugin assets -templates/ # REMOVED at root — folded into plugin assets -``` - -Note: asset files that must ship as dotfiles (`.gitignore`, `.editorconfig`) are -stored **without** the leading dot in the plugin (so they aren't hidden/ignored in -the marketplace repo) and the skill renames them on copy. `claude-shared`'s own -`.github/workflows/` (the live Claude review workflows) are unchanged and separate -from the `ci/` asset templates. - -## The `setup-repo` procedure - -Ordered, idempotent, **never silently clobbers**: - -1. **Confirm target.** Is this a git repo? If not, offer `git init`. Note whether - it's empty. -2. **Determine stack.** Detect from existing files (`pyproject.toml`/`requirements*` - → python; `package.json`/`tsconfig.json` → ts) or ask: - `python` / `ts-frontend` / `ts-backend` / `none`. -3. **Claude enablement (always).** Write `.claude/settings.json`; create `CLAUDE.md` - from template **only if absent**; copy rules → `.claude/rules/`. -4. **CI.** Copy the Claude workflows + the stack's CI into `.github/workflows/`. - Remind about the default-branch guard and the `CLAUDE_CODE_OAUTH_TOKEN` secret - (cross-reference `docs/runbooks/github-actions-claude-review.md`). -5. **Stack scaffolding.** Copy the chosen stack assets, merging/skipping existing - files (e.g. don't overwrite an existing `package.json` — report and let the dev - reconcile). -6. **Shared.** Copy `.editorconfig` and the stack `.gitignore`. -7. **Install.** Run the stack's first install (`pip install -r requirements.txt - -r requirements-dev.txt` for python; `npm install` for ts), reporting the - command and its result. Skip with a note if the toolchain is unavailable. -8. **Flesh out `CLAUDE.md`.** After writing the stub, prompt the dev for a - one-or-two-line description of what the repo does, then fill in the template's - Project Overview / Build & Test / Local Architecture sections from their answer - plus what was just scaffolded (stack, test/lint commands). Leave the stub as-is - if they skip. -9. **Report.** List every file written/skipped, then a manual follow-up checklist: - add the `CLAUDE_CODE_OAUTH_TOKEN` secret, commit workflows to the default - branch, optional branch protection. - -### Idempotency & updates -Re-running on an existing repo brings config up to the current standard: for any -file that already exists and differs, show the diff and ask before overwriting. -A clean re-run (no changes) reports "already up to standard." - -### Error handling -- Not a git repo → offer `git init`, else abort with guidance. -- Existing files → merge or skip with an explicit report; never blind-overwrite. -- No `gh` auth / can't set secret → surface as a manual step, don't fail the run. - -## Testing - -- **Asset validity:** each bundled asset is well-formed — `pyproject.toml` parses, - `package.json` is valid JSON, workflow/CI YAML is valid, `claude plugin validate` - passes for the plugin. -- **Procedure:** a script scaffolds into a throwaway temp git repo and asserts the - expected files land, then runs `ruff check` / `ruff format --check` / `pytest` - (python) and `eslint` / `tsc --noEmit` / `vitest run` (ts) against the generated - config to prove the defaults are internally consistent and pass on an empty repo. - -## Phasing - -Each phase is its own plan → implementation cycle. - -- **Phase 1** — `setup-repo` skill + Claude enablement + Claude workflows + - **Python stack** + fold PR #1 `templates/` into the plugin. Covers ~10/14 repos - and delivers the end-to-end vision for the dominant stack. -- **Phase 2** — `ts-frontend` + `ts-backend` stacks + `ci-node.yml`. -- **Phase 3 (optional, later)** — pre-commit, C++ (blah2-arm, retina-spectrum). - -## Resolved follow-ups - -- **First install:** the skill runs it (step 7), reporting the command and result; - skips gracefully if the toolchain is unavailable. -- **`CLAUDE.md`:** written as a stub, then the skill prompts the dev for a short - description and fleshes out the template sections from that plus the scaffolded - stack (step 8). Skipping leaves the stub. From cb2627545dbd3995e4df6d39505ffd5360b5b5db Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Wed, 15 Jul 2026 18:47:53 +0100 Subject: [PATCH 18/27] Reinstate docs/architecture.md with a surveyed RETINA architecture overview MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replace the removed stub with an evidence-based org-wide architecture doc, derived from surveying the offworldlabs repos: system tiers (edge Pi5 nodes → central server → web clients), the two-front signal chain (real SDR vs simulator), a component catalogue, deployment/fleet lifecycle (owl-os + Mender OTA), a repository map, and flagged open questions. Interfaces are cited from code/config; inferred/unverified links are marked as such. Reference it from the README Documentation section. Co-Authored-By: Claude Opus 4.8 (1M context) --- README.md | 9 +- docs/architecture.md | 232 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 239 insertions(+), 2 deletions(-) create mode 100644 docs/architecture.md diff --git a/README.md b/README.md index 1994dea..cde3e7b 100644 --- a/README.md +++ b/README.md @@ -31,8 +31,13 @@ for the one manual follow-up (the `CLAUDE_CODE_OAUTH_TOKEN` secret). ## Documentation Org-wide reference docs live under `docs/`. A consuming repo's `CLAUDE.md` should -point at these rather than duplicating them, so there's one source of truth. Each -subdirectory: +point at these rather than duplicating them, so there's one source of truth. + +- **`docs/architecture.md`** — the org-wide system architecture: the RETINA + passive-radar network's tiers, signal chain, components, and deployment/fleet + lifecycle. Start here to understand how the repos fit together. + +And each subdirectory: - **`docs/contracts/`** — the source of truth for cross-service interfaces: API schemas, event/message formats, and shared data structures. When two services diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..222a727 --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,232 @@ +# RETINA System Architecture + +RETINA — *Radio Echo Tracking by Inter-Node Analysis* — is an open-source +**distributed passive radar network**. Edge nodes listen for reflections of +existing broadcast transmitters (digital TV / FM illuminators of opportunity) +off aircraft, extract delay-Doppler detections locally, and forward them to a +central server that fuses detections from multiple nodes into tracked, +geolocated aircraft shown on live web maps. + +> **Status of this document.** Derived from a survey of the `offworldlabs` +> repositories as of 2026-07-15, with interfaces read from code and config. +> Ports, endpoints, and component roles are cited from source; links that could +> not be confirmed from the repos are marked **(inferred)** or **(unverified)**. +> Update it as the system evolves — it is the org-wide reference the individual +> repos should point at rather than re-describe. Open questions are collected in +> the last section. + +## 1. System context + +Three tiers plus external inputs and an out-of-band control plane: + +``` + External illuminators (broadcast TV/FM towers) ─ reflections ─┐ + ADS-B (readsb / adsb.lol / tar1090) ── ground-truth ──┐ │ + ▼ ▼ + ┌──────────────────────── EDGE RADAR NODE (Raspberry Pi 5) ───────────────┐ + │ owl-os (Mender-managed OS) + retina-node docker-compose stack │ + │ blah2 (C++ SDR DSP) → detections adsb2dd (truth) tar1090 (ADS-B) │ + │ retina-gui (node mgmt UI) [retina-spectrum: illuminator survey]│ + └──────────────────────────────┬──────────────────────────────────────────┘ + │ detections over TCP (per-node token) + ▼ + ┌──────────────────────── CENTRAL SERVER (cloud droplet) ─────────────────┐ + │ "Tower-Finder" monorepo = the RETINA server │ + │ FastAPI backend (TCP ingest + tracker + geolocator + analytics) │ + │ nginx + live-map SPA + admin dashboard → map/dash/api.retina.fm │ + │ tower-finder-service (illuminator site-survey utility, adjacent) │ + └──────────────────────────────┬──────────────────────────────────────────┘ + │ /ws/aircraft* WebSocket + REST + ▼ + Web clients (live map, dashboard) + + Control plane (out of band): hosted.mender.io ← fleet OTA for OS + app stack +``` + +- **Edge radar node** — a Raspberry Pi 5 running `owl-os` with the `retina-node` + Docker Compose stack. Captures IQ from an SDR (SDRplay RSPduo), computes + delay-Doppler detections, and forwards them to the central server. +- **Central server** — the `Tower-Finder` monorepo (the repo name is historical; + it is now the full RETINA server). Ingests detections from all nodes, runs + multi-target tracking and multi-node geolocation, and serves the live maps. +- **Web clients** — the live map (`map.retina.fm`) and admin dashboard + (`dash`/`admin.retina.fm`), served as static SPAs by the central server. +- **Control plane** — `hosted.mender.io` delivers OS and application updates to + the fleet over the air; it is deliberately separate from the data plane. + +## 2. Signal chain (data flow) + +There are **two ingestion fronts** feeding a common tracking → geolocation → +visualization tail: the real SDR front (production) and a simulator front (used +for hardware-free local testing). + +**A. Real SDR node (production)** + +``` +SDR (RSPduo) → blah2 C++ processor → blah2 Node API (:3000 /api/detection …) + ↑ ADS-B truth from adsb2dd (:49155 /api/dd) + ↑ adsb2dd polls a tar1090 /data/aircraft.json + → detections forwarded over TCP to the central server + → central tracker (Kalman + GNN) → central geolocator (Levenberg-Marquardt, multi-node) + → in-memory track state → /ws/aircraft* WebSocket → live map SPA +``` + +**B. Simulator front (hardware-free testing)** + +``` +synthetic-adsb (:5001 /data/aircraft.json, with injectable anomalies) + → adsb2dd (:49155 → .detection frames) → retina-tracker (:30100 TCP, via tracker-host bridge) + → retina-geolocator (batch: track JSONL → lat/lon/alt JSONL) + → tar1090 map (tar1090-node proxy serves enriched aircraft.json; readsb disabled) +``` + +**Ordering correction.** A common summary of the pipeline lists the geolocator +before the tracker. The code is unambiguous the other way: the **tracker runs +first** (it turns detections into tracks) and the **geolocator runs on the +tracker's output** (it solves each track's geographic position). The true order +is `detections → tracker → geolocator`. `retina-geolocator` consumes +`retina-tracker`'s JSONL track output, and `adsb2dd`'s synthetic detections are +explicitly "compatible with retina-tracker." + +**Two sources, not a sequence.** `blah2` (real SDR DSP) and `synthetic-adsb` +(simulator) are *alternative* front-ends for the same tail, not sequential +stages — the simulator stands in for the SDR + ADS-B hardware when testing +without a radio. + +## 3. Component catalogue + +### Edge / on-node +- **blah2-arm** (C++ + Node API) — the passive-radar DSP engine (Raspberry Pi 5 + fork of blah2). Captures 2-channel IQ, computes delay-Doppler maps and + in-processor tracks. REST API on `:3000` (`/api/detection`, `/api/map`, + `/api/tracker`, …); web UI on `:49152`. Can forward detections to the central + server. The core of the on-node stack. +- **adsb2dd** (Node/Express) — converts ADS-B aircraft positions into bistatic + delay-Doppler "truth" for a given rx/tx/frequency. Polls a tar1090 + `/data/aircraft.json`; serves `/api/dd` and `/api/synthetic-detections` on + `:49155`. +- **tar1090-node** (readsb + tar1090 + Node proxy) — ADS-B decode and map. A Node + proxy (`:3005`) serves an enriched `aircraft.json` (adds anomaly / Mach flags) + and disables readsb so synthetic data can drive the map; tar1090 renders on + `:8504`. +- **retina-spectrum** (C++) — standalone RF spectrum-survey tool to pick + illuminators; HTTP/SSE UI on `:3020`. Shares the single RSPduo with `blah2`, so + it runs *instead of* the radar stack (opt-in `spectrum` compose profile). +- **retina-gui** (Python/Flask) — per-node management / onboarding UI baked into + `owl-os` (systemd, `:80`, `owl.local`). Management plane, not data plane. + +### Central server +- **Tower-Finder** (Python FastAPI + React/Vite SPAs) — the RETINA central server. + One container (nginx + uvicorn) hosting: TCP detection ingest (`:3012`), the + multi-target **tracker** (Kalman + GNN) and node associator, the multi-node + **geolocator** (Levenberg-Marquardt), auth/admin/analytics, the live-map SPA, + and the admin dashboard. Exposes REST `/api/*` and `/ws/aircraft*` WebSocket + feeds behind `map`/`dash`/`api`/`testmap.retina.fm`. +- **tower-finder-service** (Python FastAPI) — the illuminator site-survey feature + extracted into a standalone microservice (2026-05-20). Given a lat/lon it ranks + nearby FM/VHF/UHF broadcast towers as candidate illuminators, querying external + databases (Maprad.io, FCC). Fronted by the monorepo's nginx at + `tower-finder.retina.fm`. Currently duplicates the tower code still present in + the monorepo (deduplication pending). + +### Tooling / simulation +- **retina-tracker** (Python) — the standalone tracker (Kalman/GNN) used in the + simulator pipeline and integration tests; TCP service on `:30100`. The central + server embeds the same tracking role. +- **retina-geolocator** (Python library) — LM delay/Doppler → lat/lon/alt/velocity + solver (single- and multi-node). A pip-installed library / batch tool, no + network service; consumed by the central server and offline scripts. +- **retina-simulation** (Python) — fleet load-test harness; streams detection + frames for 100–1000 synthetic nodes to a RETINA server over TCP (`:3012`). +- **synthetic-adsb** *(not a sibling repo here; referenced by integration + compose)* — simulates ADS-B on `:5001` with injectable anomalies; stands in for + real SDR + ADS-B in local testing. +- **tracker-host** *(not a sibling repo here; referenced by integration compose)* + — bridges `synthetic-adsb` to `retina-tracker`'s TCP port. +- **radar-replay** (Python/Flask) — records a live node's API to JSONL and replays + it through the same API (`:8090`) for offline debugging. +- **3lips-telemetry-solver** — empty skeleton in-repo; by name likely the intended + multi-node telemetry/geolocation solver, but **unimplemented here**. + +## 4. Deployment & fleet lifecycle + +**On each edge node:** +- **OS layer** — `owl-os`: a Mender-enabled Debian bookworm arm64 image for the + Pi 5, built with EDI. A/B-partitioned for safe rollback; ships Docker, the + SDRplay API, Chrony, Cloudflared, Avahi (`owl.local`), a WiFi captive portal, + and the Mender client. +- **Application layer** — the `retina-node` Docker Compose stack (images from + `ghcr.io/offworldlabs/*`): `config-merger` (runs once to merge + `default → user → forced` config into `config.yml` + `.env`), then `blah2`, + `blah2_web/api/host`, `tar1090`, `adsb2dd`, and optional `retina-spectrum`. A + node's data-plane target (central collector host + token, ADS-B source) is + selected by a network *profile* applied at "forced" precedence so it can't be + overridden by local edits. + +**Build → provision → update:** +1. **Build** — tagged CI builds produce Mender artifacts: `owl-os` (`os-v*`) builds + the full OS image + `.mender`; `retina-node` (`v*`) builds the compose bundle + into a `.mender` artifact plus the `config-merger` GHCR image. +2. **Provision** — flash the OS image → WiFi captive-portal onboarding → the node + registers as *pending* on `hosted.mender.io` → `node-infra/mender-auto-accept` + (a 30-second systemd timer on the central server) auto-approves nodes matching + an ID prefix → the `retina-node` stack is deployed via Mender OTA → + `config-merger` applies location/network config. +3. **Update** — push new `.mender` artifacts (app bundle and/or full A/B OS image) + through Mender; A/B partitioning + verified reboot gives safe rollback. + Switching the *data-plane* network is automated; switching the *OTA control* + plane is intentionally manual. + +**Central / cloud:** the `Tower-Finder` monorepo container + `tower-finder-service` +run on a DigitalOcean droplet, joined by a shared `retina-edge` Docker network and +fronted by Cloudflare; both deploy via `git reset --hard origin/main` + +`docker compose up -d --build` from GitHub Actions on push to `main`. The public +marketing site is a separate static repo (`landing-page-retina`). + +## 5. Repository map + +| Repo | Role | Stack | +| --- | --- | --- | +| `blah2-arm` | On-node SDR DSP engine + API | C++, Node | +| `adsb2dd` | ADS-B → delay-Doppler truth | Node/Express | +| `tar1090-node` | ADS-B decode + map + proxy | readsb, nginx, Node | +| `retina-spectrum` | Illuminator spectrum survey | C++ | +| `retina-gui` | Node management UI | Python/Flask | +| `Tower-Finder` | Central RETINA server (ingest, track, geolocate, maps) | Python/FastAPI, React/Vite | +| `tower-finder-service` | Illuminator site-survey microservice | Python/FastAPI | +| `retina-tracker` | Multi-target tracker (Kalman/GNN) | Python | +| `retina-geolocator` | LM delay/Doppler → position solver | Python (library) | +| `retina-simulation` | Fleet load-test harness | Python | +| `radar-replay` | Record/replay debug tool | Python/Flask | +| `retina-node` | On-device compose bundle + OTA packaging | Compose, Python | +| `owl-os` | Pi 5 OS image builder (Mender/EDI) | EDI, Ansible | +| `node-infra` | Central fleet automation (Mender auto-accept) | Python | +| `landing-page-retina` | Public marketing site | Static HTML | +| `3lips-telemetry-solver` | (empty) intended telemetry solver | — | + +## 6. Open questions / to reconcile + +These surfaced during the survey and are not yet confirmed from the repos: + +- **Detection-forwarding endpoint.** Nodes forward to a central collector + (`tracker.retnode.com:30050` in the node network profile), while the central + server's documented TCP ingest is `:3012`. The mapping between the two + (proxy / NAT / separate collector) is **unverified**. +- **How geolocated tracks reach the map.** `retina-geolocator` writes results to a + JSONL file and exposes no network interface; the map proxy reads `aircraft.json` + from a URL. The component that publishes solved tracks as the `aircraft.json` + the map serves is **(inferred)** — likely the central server's in-memory state / + WebSocket feed rather than the standalone geolocator. +- **`synthetic-adsb` and `tracker-host`** are referenced by the integration + compose but are not present as sibling repos here; the documented simulator + pipeline can't be built from these directories alone. +- **`3lips-telemetry-solver`** is an empty skeleton — the multi-node telemetry + solver role is unimplemented in-repo. +- **Duplicate tower code** in `Tower-Finder` and `tower-finder-service` pending + deduplication; `tower-finder-service`'s healthcheck hits `/api/health` despite + its README saying that endpoint was dropped on extraction. +- **Mixed default geographies** (San Francisco vs Adelaide) across + `retina-tracker`, the integration compose, and `blah2` defaults. +- Two similarly named repos exist: `landing-page-retina` (the actual static site) + and `landingpage` (an unconfigured GitHub Pages course template) — confirm which + publishes the public site. From 641ab97a48cf5def8ed46f6aeca7e15323c4f0fb Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Wed, 15 Jul 2026 19:42:38 +0100 Subject: [PATCH 19/27] Drop 3lips-telemetry-solver from architecture doc; name the integration compose The 3lips-telemetry-solver directory has been removed from the retina tree, so remove its component/repo-map/open-question references. Also name the file behind "the integration compose" explicitly: retina-tracker/docker-compose.integration-test.yml, which builds synthetic-adsb and tracker-host from local ./ build contexts. Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/architecture.md | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 222a727..0a82ea6 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -138,15 +138,14 @@ without a radio. network service; consumed by the central server and offline scripts. - **retina-simulation** (Python) — fleet load-test harness; streams detection frames for 100–1000 synthetic nodes to a RETINA server over TCP (`:3012`). -- **synthetic-adsb** *(not a sibling repo here; referenced by integration - compose)* — simulates ADS-B on `:5001` with injectable anomalies; stands in for - real SDR + ADS-B in local testing. -- **tracker-host** *(not a sibling repo here; referenced by integration compose)* - — bridges `synthetic-adsb` to `retina-tracker`'s TCP port. +- **synthetic-adsb** *(not present locally; built by `retina-tracker`'s + integration-test compose)* — simulates ADS-B on `:5001` with injectable + anomalies; stands in for real SDR + ADS-B in local testing. +- **tracker-host** *(not present locally; built by `retina-tracker`'s + integration-test compose)* — bridges `synthetic-adsb` to `retina-tracker`'s TCP + port. - **radar-replay** (Python/Flask) — records a live node's API to JSONL and replays it through the same API (`:8090`) for offline debugging. -- **3lips-telemetry-solver** — empty skeleton in-repo; by name likely the intended - multi-node telemetry/geolocation solver, but **unimplemented here**. ## 4. Deployment & fleet lifecycle @@ -202,7 +201,6 @@ marketing site is a separate static repo (`landing-page-retina`). | `owl-os` | Pi 5 OS image builder (Mender/EDI) | EDI, Ansible | | `node-infra` | Central fleet automation (Mender auto-accept) | Python | | `landing-page-retina` | Public marketing site | Static HTML | -| `3lips-telemetry-solver` | (empty) intended telemetry solver | — | ## 6. Open questions / to reconcile @@ -217,11 +215,11 @@ These surfaced during the survey and are not yet confirmed from the repos: from a URL. The component that publishes solved tracks as the `aircraft.json` the map serves is **(inferred)** — likely the central server's in-memory state / WebSocket feed rather than the standalone geolocator. -- **`synthetic-adsb` and `tracker-host`** are referenced by the integration - compose but are not present as sibling repos here; the documented simulator - pipeline can't be built from these directories alone. -- **`3lips-telemetry-solver`** is an empty skeleton — the multi-node telemetry - solver role is unimplemented in-repo. +- **`synthetic-adsb` and `tracker-host`** are built by `retina-tracker`'s + integration-test compose (`docker-compose.integration-test.yml`, via build + contexts `./synthetic-adsb` and `./tracker-host`) but are not present as sibling + repos here; the documented simulator pipeline can't be built from these + directories alone. - **Duplicate tower code** in `Tower-Finder` and `tower-finder-service` pending deduplication; `tower-finder-service`'s healthcheck hits `/api/health` despite its README saying that endpoint was dropped on extraction. From dd2306cf189559898ed73d7bd0750d625f280fba Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Wed, 15 Jul 2026 19:46:58 +0100 Subject: [PATCH 20/27] Clarify retina-tracker/geolocator are libraries vendored into central server retina-tracker is not deployed standalone: Tower-Finder includes it (and retina-geolocator, retina-custody, retina-simulation, retina-analytics) as libs/ git submodules pip-installed into the server image and imported by the backend. Its :30100 Dockerfile is integration-test only. Update the component catalogue, central-server entry, and repo map accordingly. Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/architecture.md | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 0a82ea6..1a6279d 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -121,7 +121,11 @@ without a radio. multi-target **tracker** (Kalman + GNN) and node associator, the multi-node **geolocator** (Levenberg-Marquardt), auth/admin/analytics, the live-map SPA, and the admin dashboard. Exposes REST `/api/*` and `/ws/aircraft*` WebSocket - feeds behind `map`/`dash`/`api`/`testmap.retina.fm`. + feeds behind `map`/`dash`/`api`/`testmap.retina.fm`. The tracking, geolocation, + and analytics algorithms are **vendored as git submodules under `libs/`** + (`retina-tracker`, `retina-geolocator`, `retina-custody`, `retina-simulation`, + `retina-analytics`) and pip-installed into the image — those repos run *inside* + this server, not as separate services. - **tower-finder-service** (Python FastAPI) — the illuminator site-survey feature extracted into a standalone microservice (2026-05-20). Given a lat/lon it ranks nearby FM/VHF/UHF broadcast towers as candidate illuminators, querying external @@ -130,12 +134,15 @@ without a radio. the monorepo (deduplication pending). ### Tooling / simulation -- **retina-tracker** (Python) — the standalone tracker (Kalman/GNN) used in the - simulator pipeline and integration tests; TCP service on `:30100`. The central - server embeds the same tracking role. +- **retina-tracker** (Python library) — the multi-target tracker (Kalman/GNN). In + production it is **not deployed standalone**: the central server vendors it as a + `libs/retina-tracker` git submodule and imports it directly (e.g. + `frame_processor`, `passive_radar`). Its own Dockerfile (a TCP service on + `:30100`) is used only by `retina-tracker`'s integration-test compose. - **retina-geolocator** (Python library) — LM delay/Doppler → lat/lon/alt/velocity - solver (single- and multi-node). A pip-installed library / batch tool, no - network service; consumed by the central server and offline scripts. + solver (single- and multi-node). No network service; vendored into the central + server as a `libs/` git submodule and also usable as a pip-installed batch tool + for offline scripts. - **retina-simulation** (Python) — fleet load-test harness; streams detection frames for 100–1000 synthetic nodes to a RETINA server over TCP (`:3012`). - **synthetic-adsb** *(not present locally; built by `retina-tracker`'s @@ -193,8 +200,8 @@ marketing site is a separate static repo (`landing-page-retina`). | `retina-gui` | Node management UI | Python/Flask | | `Tower-Finder` | Central RETINA server (ingest, track, geolocate, maps) | Python/FastAPI, React/Vite | | `tower-finder-service` | Illuminator site-survey microservice | Python/FastAPI | -| `retina-tracker` | Multi-target tracker (Kalman/GNN) | Python | -| `retina-geolocator` | LM delay/Doppler → position solver | Python (library) | +| `retina-tracker` | Multi-target tracker (Kalman/GNN) — library vendored into central server | Python | +| `retina-geolocator` | LM delay/Doppler → position solver — library vendored into central server | Python | | `retina-simulation` | Fleet load-test harness | Python | | `radar-replay` | Record/replay debug tool | Python/Flask | | `retina-node` | On-device compose bundle + OTA packaging | Compose, Python | From 07bed87b88f2f3743a63d46876a695d2ed2106fa Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Wed, 15 Jul 2026 19:53:26 +0100 Subject: [PATCH 21/27] Add retina-custody & retina-analytics; resolve simulator prod-usage question - Document the two remaining central-server library submodules: retina-custody (node identity, signature/hash-chain custody) and retina-analytics (inter-node association, node reputation/trust), with roles derived from the server's imports since the submodules aren't checked out locally. Add repo-map rows. - Add a "Are the simulators used in production?" subsection: synthetic-adsb and tracker-host are test-only (referenced solely by retina-tracker's integration harness, no prod deploy path); note the central server's separate internal _apply_synthetic_adsb + retina-simulation mechanism and the name-collision trap. Include a reusable verification method. Drop the now-answered open question. Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/architecture.md | 70 ++++++++++++++++++++++++++++++++++---------- 1 file changed, 54 insertions(+), 16 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 1a6279d..a3f47d5 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -88,10 +88,36 @@ is `detections → tracker → geolocator`. `retina-geolocator` consumes `retina-tracker`'s JSONL track output, and `adsb2dd`'s synthetic detections are explicitly "compatible with retina-tracker." -**Two sources, not a sequence.** `blah2` (real SDR DSP) and `synthetic-adsb` -(simulator) are *alternative* front-ends for the same tail, not sequential -stages — the simulator stands in for the SDR + ADS-B hardware when testing -without a radio. +**Two sources, not a sequence.** `blah2` (real SDR DSP) and the simulator front +are *alternatives* for the same tail, not sequential stages — the simulator +stands in for the SDR + ADS-B hardware when testing without a radio. + +### Are the simulators used in production? + +No — `synthetic-adsb` and `tracker-host` are a **test-only** harness. The evidence: +they appear **only** in `retina-tracker`'s integration-test compose and test +scripts (`docker-compose.integration-test.yml`, `test_anomaly_*.sh`, +`tests/test_integration_synthetic.py`), and in **none** of the production deploy +paths — not the `retina-node` edge stack, not `owl-os`, not `node-infra`, and not +the central server's build. They also aren't published as `ghcr.io/offworldlabs/*` +images that any prod compose pulls. + +Production *synthetic* data is a different mechanism, easy to confuse by name: +the central server has its own internal `_apply_synthetic_adsb` handling for +"synth" nodes in `backend/services/tcp_handler.py`, fed by the `retina-simulation` +load harness streaming to the ingest port — not by the `synthetic-adsb` container. +So a name match in the central server is *not* evidence the standalone repo is +deployed. + +**How to verify this yourself** for any component: `grep -rniE ''` across the +repos that actually deploy — `retina-node` and `owl-os` (edge), `Tower-Finder` +(central), `node-infra` (fleet) — over `*.yml`, `Dockerfile*`, `*.sh`, and CI +workflows; then classify each hit as a *test/integration* artifact (path or name +contains `test`/`integration`) or a *production* path (release workflow, prod +compose, systemd unit, the edge stack). If every hit is a test artifact and no +prod compose pulls a published image for it, it isn't in production. Watch for +name collisions — an internal function like `_apply_synthetic_adsb` is not the +`synthetic-adsb` repo. ## 3. Component catalogue @@ -143,14 +169,29 @@ without a radio. solver (single- and multi-node). No network service; vendored into the central server as a `libs/` git submodule and also usable as a pip-installed batch tool for offline scripts. +- **retina-custody** (Python library) — cryptographic chain-of-custody for node + data: node identity (`NodeIdentity`), signature verification (`SignatureVerifier`, + `SoftwareCryptoBackend`), and tamper-evident hash chains (`HashChainBuilder`/ + `Verifier`). Makes each node's detections authenticated and tamper-evident. + Vendored into the central server as a `libs/` submodule; imported by + `backend/core/state.py`. *(Role derived from the central server's imports — the + submodule isn't checked out locally.)* +- **retina-analytics** (Python library) — per-node analytics and trust: inter-node + detection association (`InterNodeAssociator`), node reputation and trust scoring + (`NodeReputation`, `TrustScoreState`, `AdsReportEntry`), coordinated by a + `NodeAnalyticsManager`. Vendored into the central server as a `libs/` submodule; + drives live state and the `/api/analytics` route. *(Role derived from the central + server's imports — the submodule isn't checked out locally.)* - **retina-simulation** (Python) — fleet load-test harness; streams detection frames for 100–1000 synthetic nodes to a RETINA server over TCP (`:3012`). -- **synthetic-adsb** *(not present locally; built by `retina-tracker`'s - integration-test compose)* — simulates ADS-B on `:5001` with injectable - anomalies; stands in for real SDR + ADS-B in local testing. -- **tracker-host** *(not present locally; built by `retina-tracker`'s - integration-test compose)* — bridges `synthetic-adsb` to `retina-tracker`'s TCP - port. +- **synthetic-adsb** *(test-only; not present locally)* — simulates ADS-B on + `:5001` with injectable anomalies; stands in for real SDR + ADS-B. Referenced + **only** by `retina-tracker`'s integration-test compose and test scripts — not + by any production deploy path (see "Are the simulators used in production?" + below). +- **tracker-host** *(test-only; not present locally)* — bridges `synthetic-adsb` + to `retina-tracker`'s TCP port; likewise referenced only by `retina-tracker`'s + integration harness. - **radar-replay** (Python/Flask) — records a live node's API to JSONL and replays it through the same API (`:8090`) for offline debugging. @@ -202,7 +243,9 @@ marketing site is a separate static repo (`landing-page-retina`). | `tower-finder-service` | Illuminator site-survey microservice | Python/FastAPI | | `retina-tracker` | Multi-target tracker (Kalman/GNN) — library vendored into central server | Python | | `retina-geolocator` | LM delay/Doppler → position solver — library vendored into central server | Python | -| `retina-simulation` | Fleet load-test harness | Python | +| `retina-custody` | Node identity + signature/hash-chain custody — library vendored into central server | Python | +| `retina-analytics` | Inter-node association + node reputation/trust — library vendored into central server | Python | +| `retina-simulation` | Fleet load-test harness — library vendored into central server | Python | | `radar-replay` | Record/replay debug tool | Python/Flask | | `retina-node` | On-device compose bundle + OTA packaging | Compose, Python | | `owl-os` | Pi 5 OS image builder (Mender/EDI) | EDI, Ansible | @@ -222,11 +265,6 @@ These surfaced during the survey and are not yet confirmed from the repos: from a URL. The component that publishes solved tracks as the `aircraft.json` the map serves is **(inferred)** — likely the central server's in-memory state / WebSocket feed rather than the standalone geolocator. -- **`synthetic-adsb` and `tracker-host`** are built by `retina-tracker`'s - integration-test compose (`docker-compose.integration-test.yml`, via build - contexts `./synthetic-adsb` and `./tracker-host`) but are not present as sibling - repos here; the documented simulator pipeline can't be built from these - directories alone. - **Duplicate tower code** in `Tower-Finder` and `tower-finder-service` pending deduplication; `tower-finder-service`'s healthcheck hits `/api/health` despite its README saying that endpoint was dropped on extraction. From b42343a250befe329206178331e66520228f93f3 Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Wed, 15 Jul 2026 19:58:37 +0100 Subject: [PATCH 22/27] Remove synthetic-adsb and tracker-host (test-only) from architecture doc They aren't part of the live system, so drop them: remove the simulator data-flow front and the "used in production?" subsection they anchored, and their component entries. Present the production signal chain (SDR -> blah2 -> central server tracker/geolocator libs -> map) as the single flow, keep the tracker-before- geolocator ordering note, and keep a brief testing note pointing at retina-simulation. Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/architecture.md | 71 ++++++++------------------------------------ 1 file changed, 12 insertions(+), 59 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index a3f47d5..e6e97a9 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -56,11 +56,8 @@ Three tiers plus external inputs and an out-of-band control plane: ## 2. Signal chain (data flow) -There are **two ingestion fronts** feeding a common tracking → geolocation → -visualization tail: the real SDR front (production) and a simulator front (used -for hardware-free local testing). - -**A. Real SDR node (production)** +The production flow runs from an SDR at the edge to the live map, with the +tracker and geolocator running as libraries **inside** the central server: ``` SDR (RSPduo) → blah2 C++ processor → blah2 Node API (:3000 /api/detection …) @@ -71,53 +68,17 @@ SDR (RSPduo) → blah2 C++ processor → blah2 Node API (:3000 /api/detection → in-memory track state → /ws/aircraft* WebSocket → live map SPA ``` -**B. Simulator front (hardware-free testing)** - -``` -synthetic-adsb (:5001 /data/aircraft.json, with injectable anomalies) - → adsb2dd (:49155 → .detection frames) → retina-tracker (:30100 TCP, via tracker-host bridge) - → retina-geolocator (batch: track JSONL → lat/lon/alt JSONL) - → tar1090 map (tar1090-node proxy serves enriched aircraft.json; readsb disabled) -``` - -**Ordering correction.** A common summary of the pipeline lists the geolocator -before the tracker. The code is unambiguous the other way: the **tracker runs -first** (it turns detections into tracks) and the **geolocator runs on the -tracker's output** (it solves each track's geographic position). The true order -is `detections → tracker → geolocator`. `retina-geolocator` consumes -`retina-tracker`'s JSONL track output, and `adsb2dd`'s synthetic detections are -explicitly "compatible with retina-tracker." - -**Two sources, not a sequence.** `blah2` (real SDR DSP) and the simulator front -are *alternatives* for the same tail, not sequential stages — the simulator -stands in for the SDR + ADS-B hardware when testing without a radio. - -### Are the simulators used in production? - -No — `synthetic-adsb` and `tracker-host` are a **test-only** harness. The evidence: -they appear **only** in `retina-tracker`'s integration-test compose and test -scripts (`docker-compose.integration-test.yml`, `test_anomaly_*.sh`, -`tests/test_integration_synthetic.py`), and in **none** of the production deploy -paths — not the `retina-node` edge stack, not `owl-os`, not `node-infra`, and not -the central server's build. They also aren't published as `ghcr.io/offworldlabs/*` -images that any prod compose pulls. - -Production *synthetic* data is a different mechanism, easy to confuse by name: -the central server has its own internal `_apply_synthetic_adsb` handling for -"synth" nodes in `backend/services/tcp_handler.py`, fed by the `retina-simulation` -load harness streaming to the ingest port — not by the `synthetic-adsb` container. -So a name match in the central server is *not* evidence the standalone repo is -deployed. +**Ordering: tracker before geolocator.** A common summary of the pipeline lists +the geolocator before the tracker; the code is unambiguous the other way. The +**tracker runs first** (it turns detections into tracks) and the **geolocator +runs on the tracker's output** (it solves each track's geographic position) — +`retina-geolocator` consumes `retina-tracker`'s track output. Both run as +libraries inside the central server (see §3), not as separate services. -**How to verify this yourself** for any component: `grep -rniE ''` across the -repos that actually deploy — `retina-node` and `owl-os` (edge), `Tower-Finder` -(central), `node-infra` (fleet) — over `*.yml`, `Dockerfile*`, `*.sh`, and CI -workflows; then classify each hit as a *test/integration* artifact (path or name -contains `test`/`integration`) or a *production* path (release workflow, prod -compose, systemd unit, the edge stack). If every hit is a test artifact and no -prod compose pulls a published image for it, it isn't in production. Watch for -name collisions — an internal function like `_apply_synthetic_adsb` is not the -`synthetic-adsb` repo. +**Testing & simulation.** The pipeline is exercised without radio hardware by the +`retina-simulation` load harness, which streams synthetic detections for many +nodes to the central server's ingest port, alongside the server's own +synthetic-node handling. ## 3. Component catalogue @@ -184,14 +145,6 @@ name collisions — an internal function like `_apply_synthetic_adsb` is not the server's imports — the submodule isn't checked out locally.)* - **retina-simulation** (Python) — fleet load-test harness; streams detection frames for 100–1000 synthetic nodes to a RETINA server over TCP (`:3012`). -- **synthetic-adsb** *(test-only; not present locally)* — simulates ADS-B on - `:5001` with injectable anomalies; stands in for real SDR + ADS-B. Referenced - **only** by `retina-tracker`'s integration-test compose and test scripts — not - by any production deploy path (see "Are the simulators used in production?" - below). -- **tracker-host** *(test-only; not present locally)* — bridges `synthetic-adsb` - to `retina-tracker`'s TCP port; likewise referenced only by `retina-tracker`'s - integration harness. - **radar-replay** (Python/Flask) — records a live node's API to JSONL and replays it through the same API (`:8090`) for offline debugging. From 2100e59450b4db1f4eafc224b852f1d0ecff371e Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Wed, 15 Jul 2026 20:20:19 +0100 Subject: [PATCH 23/27] Update landingpage -> landing-page-owl in architecture doc Reflect the repo rename (landingpage -> landing-page-owl, paralleling landing-page-retina): update the repo map and open-questions bullet. Tower-Finder is intentionally left as-is. Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/architecture.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index e6e97a9..042820f 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -203,7 +203,8 @@ marketing site is a separate static repo (`landing-page-retina`). | `retina-node` | On-device compose bundle + OTA packaging | Compose, Python | | `owl-os` | Pi 5 OS image builder (Mender/EDI) | EDI, Ansible | | `node-infra` | Central fleet automation (Mender auto-accept) | Python | -| `landing-page-retina` | Public marketing site | Static HTML | +| `landing-page-retina` | RETINA public marketing site | Static HTML | +| `landing-page-owl` | Owl product landing page (placeholder/template at last survey) | Static HTML | ## 6. Open questions / to reconcile @@ -223,6 +224,8 @@ These surfaced during the survey and are not yet confirmed from the repos: its README saying that endpoint was dropped on extraction. - **Mixed default geographies** (San Francisco vs Adelaide) across `retina-tracker`, the integration compose, and `blah2` defaults. -- Two similarly named repos exist: `landing-page-retina` (the actual static site) - and `landingpage` (an unconfigured GitHub Pages course template) — confirm which - publishes the public site. +- Two product landing pages exist: `landing-page-retina` and `landing-page-owl` + (the latter renamed from `landingpage`). At last survey `landing-page-owl` still + held an unconfigured GitHub Pages template rather than real content, and neither + repo had evidenced deploy config (CI / CNAME) — confirm `landing-page-owl` has + been populated and how both are published. From 511305d023fc344b19f6c18878efb304833852e5 Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Wed, 15 Jul 2026 21:13:21 +0100 Subject: [PATCH 24/27] Resolve live-map-feed question; refine detection-forwarding flag Confirmed from code that geolocated tracks reach the map via in-process geolocation -> in-memory state -> /ws/aircraft* WebSocket -> live-map SPA (not a file / tar1090 aircraft.json); document it and drop the inferred open question. Refine the detection-forwarding item: node profile forwards to :30050, server binds :3012, and no mapping exists in either repo, so it's infra-level -- with the concrete infra checks to confirm it. Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/architecture.md | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 042820f..6144e4a 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -80,6 +80,14 @@ libraries inside the central server (see §3), not as separate services. nodes to the central server's ingest port, alongside the server's own synthetic-node handling. +**Live map feed.** Geolocated tracks reach the map *inside* the central server, +not via a file or tar1090's `aircraft.json`. Geolocation runs in-process +(`_run_geolocation()` during frame processing, updating an in-memory geolocated- +aircraft store in `backend/core/state.py`), and that state is broadcast over the +`/ws/aircraft*` WebSocket endpoints (`backend/routes/streaming.py`) to the live-map +SPA (`frontend/src/components/map/hooks.ts`). The standalone `retina-geolocator`'s +JSONL output is the offline/batch path, not the live feed. + ## 3. Component catalogue ### Edge / on-node @@ -210,15 +218,14 @@ marketing site is a separate static repo (`landing-page-retina`). These surfaced during the survey and are not yet confirmed from the repos: -- **Detection-forwarding endpoint.** Nodes forward to a central collector - (`tracker.retnode.com:30050` in the node network profile), while the central - server's documented TCP ingest is `:3012`. The mapping between the two - (proxy / NAT / separate collector) is **unverified**. -- **How geolocated tracks reach the map.** `retina-geolocator` writes results to a - JSONL file and exposes no network interface; the map proxy reads `aircraft.json` - from a URL. The component that publishes solved tracks as the `aircraft.json` - the map serves is **(inferred)** — likely the central server's in-memory state / - WebSocket feed rather than the standalone geolocator. +- **Detection-forwarding port mapping.** Confirmed from code: nodes on the + `retina` network profile forward to `tracker.retnode.com:30050`, while the + central server binds `:3012` — and **no `30050`→`3012` mapping exists in either + repo** (no nginx `stream` block, no `30050` reference outside the node config), + so this hop is infrastructure, not code. Still to confirm at the infra level: + resolve `tracker.retnode.com` (`dig`), check what publishes/redirects `30050` on + the droplet (`docker compose ps` — the prod compose may publish `30050:3012` — + or an LB / `iptables` DNAT), and confirm a node connects end-to-end. - **Duplicate tower code** in `Tower-Finder` and `tower-finder-service` pending deduplication; `tower-finder-service`'s healthcheck hits `/api/health` despite its README saying that endpoint was dropped on extraction. From fce840f612a19b7de4ca44076657d19385723adc Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Wed, 15 Jul 2026 21:44:12 +0100 Subject: [PATCH 25/27] Record detection-forwarding finding: disabled on inspected node MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A live production node's merged blah2 config shows tracker_forward.enabled:false with base defaults (blah2_tracker:3012), not the retina profile's tracker.retnode.com:30050 (NXDOMAIN host, placeholder token) — so the :30050 path is an unapplied template on that node. Resolve the old :30050<->:3012 open question accordingly, add a config-gated caveat to the data-flow section, and leave "does any node forward in prod" as the remaining open item. Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/architecture.md | 28 ++++++++++++++++++++-------- 1 file changed, 20 insertions(+), 8 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 6144e4a..53fdfc0 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -68,6 +68,14 @@ SDR (RSPduo) → blah2 C++ processor → blah2 Node API (:3000 /api/detection → in-memory track state → /ws/aircraft* WebSocket → live map SPA ``` +**Caveat — the node→central forward is config-gated.** The "forwarded over TCP to +the central server" step depends on each node's merged `tracker_forward` config. +On the one production node inspected, forwarding was **disabled** (`enabled: false`, +base default `blah2_tracker:3012`, not the `retina` profile's +`tracker.retnode.com:30050`) — so this hop is not necessarily live fleet-wide (see +open questions). Such a node still runs the full local pipeline (`blah2` + ADS-B +truth) but doesn't feed the central server. + **Ordering: tracker before geolocator.** A common summary of the pipeline lists the geolocator before the tracker; the code is unambiguous the other way. The **tracker runs first** (it turns detections into tracks) and the **geolocator @@ -218,14 +226,18 @@ marketing site is a separate static repo (`landing-page-retina`). These surfaced during the survey and are not yet confirmed from the repos: -- **Detection-forwarding port mapping.** Confirmed from code: nodes on the - `retina` network profile forward to `tracker.retnode.com:30050`, while the - central server binds `:3012` — and **no `30050`→`3012` mapping exists in either - repo** (no nginx `stream` block, no `30050` reference outside the node config), - so this hop is infrastructure, not code. Still to confirm at the infra level: - resolve `tracker.retnode.com` (`dig`), check what publishes/redirects `30050` on - the droplet (`docker compose ps` — the prod compose may publish `30050:3012` — - or an LB / `iptables` DNAT), and confirm a node connects end-to-end. +- **Detection forwarding was disabled on the inspected node.** On a live + production node, `blah2`'s merged config shows + `tracker_forward: {enabled: false, host: blah2_tracker, port: 3012}` — it does + **not** forward to a central collector, and it carries the base defaults, not the + `retina` network profile's `tracker.retnode.com:30050`. That profile is a + template that isn't applied here (its host has no public DNS record — NXDOMAIN — + and its token is a placeholder), which is why the earlier `:30050`↔`:3012` + question had no answer in the repos: `:30050` is never used on this node. Still + open across the fleet: whether *any* production node currently forwards to the + central server, and to what real host/port — check other nodes' merged configs + and the central server's connected-node list. The `:3012` ingest is meanwhile + exercised by the `retina-simulation` harness. - **Duplicate tower code** in `Tower-Finder` and `tower-finder-service` pending deduplication; `tower-finder-service`'s healthcheck hits `/api/health` despite its README saying that endpoint was dropped on extraction. From b07f5287ca44b44441c382a2f5380ac8c3fb135b Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Wed, 15 Jul 2026 21:52:54 +0100 Subject: [PATCH 26/27] Remove open-questions section from architecture doc The surveyed gaps/inconsistencies now have tracking tickets, so drop the "Open questions / to reconcile" section and update the two references to it (the status note and the data-flow caveat) to point at tickets instead. Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/architecture.md | 37 +++++-------------------------------- 1 file changed, 5 insertions(+), 32 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 53fdfc0..1e9a9e3 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -12,8 +12,8 @@ geolocated aircraft shown on live web maps. > Ports, endpoints, and component roles are cited from source; links that could > not be confirmed from the repos are marked **(inferred)** or **(unverified)**. > Update it as the system evolves — it is the org-wide reference the individual -> repos should point at rather than re-describe. Open questions are collected in -> the last section. +> repos should point at rather than re-describe. Known gaps and inconsistencies +> found during the survey are tracked in separate tickets rather than listed here. ## 1. System context @@ -72,9 +72,9 @@ SDR (RSPduo) → blah2 C++ processor → blah2 Node API (:3000 /api/detection the central server" step depends on each node's merged `tracker_forward` config. On the one production node inspected, forwarding was **disabled** (`enabled: false`, base default `blah2_tracker:3012`, not the `retina` profile's -`tracker.retnode.com:30050`) — so this hop is not necessarily live fleet-wide (see -open questions). Such a node still runs the full local pipeline (`blah2` + ADS-B -truth) but doesn't feed the central server. +`tracker.retnode.com:30050`) — so this hop is not necessarily live fleet-wide. +Such a node still runs the full local pipeline (`blah2` + ADS-B truth) but doesn't +feed the central server. **Ordering: tracker before geolocator.** A common summary of the pipeline lists the geolocator before the tracker; the code is unambiguous the other way. The @@ -221,30 +221,3 @@ marketing site is a separate static repo (`landing-page-retina`). | `node-infra` | Central fleet automation (Mender auto-accept) | Python | | `landing-page-retina` | RETINA public marketing site | Static HTML | | `landing-page-owl` | Owl product landing page (placeholder/template at last survey) | Static HTML | - -## 6. Open questions / to reconcile - -These surfaced during the survey and are not yet confirmed from the repos: - -- **Detection forwarding was disabled on the inspected node.** On a live - production node, `blah2`'s merged config shows - `tracker_forward: {enabled: false, host: blah2_tracker, port: 3012}` — it does - **not** forward to a central collector, and it carries the base defaults, not the - `retina` network profile's `tracker.retnode.com:30050`. That profile is a - template that isn't applied here (its host has no public DNS record — NXDOMAIN — - and its token is a placeholder), which is why the earlier `:30050`↔`:3012` - question had no answer in the repos: `:30050` is never used on this node. Still - open across the fleet: whether *any* production node currently forwards to the - central server, and to what real host/port — check other nodes' merged configs - and the central server's connected-node list. The `:3012` ingest is meanwhile - exercised by the `retina-simulation` harness. -- **Duplicate tower code** in `Tower-Finder` and `tower-finder-service` pending - deduplication; `tower-finder-service`'s healthcheck hits `/api/health` despite - its README saying that endpoint was dropped on extraction. -- **Mixed default geographies** (San Francisco vs Adelaide) across - `retina-tracker`, the integration compose, and `blah2` defaults. -- Two product landing pages exist: `landing-page-retina` and `landing-page-owl` - (the latter renamed from `landingpage`). At last survey `landing-page-owl` still - held an unconfigured GitHub Pages template rather than real content, and neither - repo had evidenced deploy config (CI / CNAME) — confirm `landing-page-owl` has - been populated and how both are published. From b9c95ef832fe8794dd7f38b80669484695bbbbcf Mon Sep 17 00:00:00 2001 From: Jonny Spicer Date: Wed, 15 Jul 2026 21:58:33 +0100 Subject: [PATCH 27/27] Adopt uv over pip in the Python standard (drop-in, keeps requirements.txt) uv is a faster drop-in for pip that reads the same requirements.txt, so adopt it without abandoning the requirements.txt convention every org repo uses: - ci-python.yml: add astral-sh/setup-uv and install via `uv pip install --system -r requirements.txt -r requirements-dev.txt` (drop the pip self-upgrade). - setup-repo SKILL.md: install step now uses `uv venv && uv pip install ...`, with a pip fallback when uv is absent. - test-ci-assets.sh: assert the CI installs with uv and includes a setup-uv step. - Bump core plugin version 0.1.0 -> 0.2.0 (the org-wide update signal). Requirements.txt files, stack detection, and the other assets are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) --- plugins/core/.claude-plugin/plugin.json | 2 +- plugins/core/skills/setup-repo/SKILL.md | 12 ++++++++---- .../core/skills/setup-repo/assets/ci/ci-python.yml | 7 ++++--- tests/setup-repo/test-ci-assets.sh | 4 ++++ 4 files changed, 17 insertions(+), 8 deletions(-) diff --git a/plugins/core/.claude-plugin/plugin.json b/plugins/core/.claude-plugin/plugin.json index e8f499b..de115c2 100644 --- a/plugins/core/.claude-plugin/plugin.json +++ b/plugins/core/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "core", - "version": "0.1.0", + "version": "0.2.0", "description": "Core Offworld Labs skills, commands, agents, and hooks shared across all repos.", "author": { "name": "Offworld Labs" diff --git a/plugins/core/skills/setup-repo/SKILL.md b/plugins/core/skills/setup-repo/SKILL.md index b53dc41..4bc3bc4 100644 --- a/plugins/core/skills/setup-repo/SKILL.md +++ b/plugins/core/skills/setup-repo/SKILL.md @@ -26,10 +26,14 @@ interactive parts and the report. `bash "$ENGINE" . ` Relay its `WRITTEN` / `SKIPPED` output to the user. -4. **Install dependencies.** For `python`, run - `pip install -r requirements.txt -r requirements-dev.txt` - (prefer an active virtualenv). Report the command and result; if the - toolchain is unavailable, skip and note it rather than failing. +4. **Install dependencies.** For `python`, use `uv` (the org standard, a fast + drop-in for pip that reads the same `requirements.txt`): create a venv and + install with + `uv venv && uv pip install -r requirements.txt -r requirements-dev.txt`. + If `uv` isn't installed, fall back to + `pip install -r requirements.txt -r requirements-dev.txt` in an active + virtualenv. Report the command and result; if the toolchain is unavailable, + skip and note it rather than failing. 5. **Flesh out CLAUDE.md.** The stub was just written. Ask the user for a one-or-two-line description of what this repo does, then fill in the diff --git a/plugins/core/skills/setup-repo/assets/ci/ci-python.yml b/plugins/core/skills/setup-repo/assets/ci/ci-python.yml index 10dac2b..18424b7 100644 --- a/plugins/core/skills/setup-repo/assets/ci/ci-python.yml +++ b/plugins/core/skills/setup-repo/assets/ci/ci-python.yml @@ -17,10 +17,11 @@ jobs: with: python-version: "3.12" + - name: Install uv + uses: astral-sh/setup-uv@v5 + - name: Install dependencies - run: | - python -m pip install --upgrade pip - pip install -r requirements.txt -r requirements-dev.txt + run: uv pip install --system -r requirements.txt -r requirements-dev.txt - name: Ruff lint run: ruff check . diff --git a/tests/setup-repo/test-ci-assets.sh b/tests/setup-repo/test-ci-assets.sh index df213e9..36faf01 100755 --- a/tests/setup-repo/test-ci-assets.sh +++ b/tests/setup-repo/test-ci-assets.sh @@ -22,6 +22,10 @@ assert "ruff check ." in runs, runs assert "ruff format --check ." in runs, runs assert "pytest" in runs, runs assert "requirements.txt" in runs and "requirements-dev.txt" in runs, runs +# org standard: install with uv, not pip +assert "uv pip install" in runs, runs +uses = [str(s.get("uses", "")) for s in steps] +assert any(u.startswith("astral-sh/setup-uv") for u in uses), uses setup_py = [s for s in steps if str(s.get("uses", "")).startswith("actions/setup-python")] assert setup_py and setup_py[0]["with"]["python-version"] == "3.12", setup_py print("ci-python.yml OK")