MCP engine: deterministic workflow enforcement via tool scoping + hooks

.claude-plugin/plugin.json

@@ -1,8 +1,14 @@
 {
   "name": "devkit",
-  "version": "2.0.39",
+  "version": "2.1.0",
   "description": "A deterministic development harness for AI agents — YAML workflow engine, self-learning hooks, and multi-agent consensus",
   "author": {
     "name": "5uck1ess"
+  },
+  "mcpServers": {
+    "devkit-engine": {
+      "command": "${CLAUDE_PLUGIN_ROOT}/bin/devkit",
+      "args": ["mcp"]
+    }
   }
 }

.gitignore

@@ -1,3 +1,6 @@
+# Compiled binaries
+bin/
+
 # Credentials — never commit
 *.token
 *.secret
@@ -23,3 +26,4 @@ dist/
 # Temp
 *.log
 *.tmp
+.worktrees

CHANGELOG.md

@@ -1,5 +1,35 @@
 # Changelog
 
+## 2.1.0
+
+### MCP Engine — Deterministic Workflow Enforcement (PR #52)
+
+Replaces the broken subprocess-spawning engine with an MCP server that runs inside Claude Code. Step skipping is now structurally impossible.
+
+#### Added
+- **MCP server** (`src/mcp/`) — Go server exposes 4 tools: `devkit_start`, `devkit_advance`, `devkit_status`, `devkit_list`. Registered via `mcpServers` in plugin.json.
+- **PreToolUse guard hook** (`hooks/devkit-guard.sh`) — reads `session.json`, blocks Bash/Edit/Write/Read/Glob/Grep/Agent/WebFetch/WebSearch/NotebookEdit/Skill during command steps (exit 2).
+- **Stop guard hook** (`hooks/devkit-stop-guard.sh`) — blocks session end if workflow is incomplete.
+- **Condensed principles** (`skills/_principles.yml`) — ~120 tokens of DRY/YAGNI/clean-code/dont-reinvent/executing/scratchpad/stuck/test-gen rules injected per workflow step instead of loading full skill files.
+- **Hot session state** (`src/lib/state_json.go`) — atomic write to `$CLAUDE_PLUGIN_DATA/session.json` for fast hook reads (<50ms).
+- **Workflow YAML extensions** — `enforce` (hard/soft), `branch` (git branch per session), `principles` (per-workflow and per-step override).
+- **New MCP tools** — 6 integration tests covering lifecycle, loops with gates, principle injection, expect-failure, path traversal rejection.
+
+#### Changed
+- **Engine role** — CLI that spawned subprocesses → MCP server + state machine
+- **Claude runner** — `claude -p` subprocess (broken with OAuth) → Claude Code IS the runner
+- **Enforcement** — None (markdown honor system) → MCP tool scoping + PreToolUse exit 2
+- **Principle skills** — Loaded if Claude decided to → injected by engine per step
+- **Token usage** — ~50k+ for 8-step workflow → ~17k (~65% reduction)
+- **Skills and commands** — All 8 entry points (research, deep-research, autoloop, tri-review, tri-debug, tri-security, pr-ready, status) now use MCP tools instead of `ensure-engine.sh` + `devkit workflow run`
+
+#### Removed
+- `scripts/ensure-engine.sh` — no longer needed (binary ships in `bin/`, auto-PATH)
+- `scripts/install-engine.sh` — installed by plugin manifest
+
+#### Fixed
+- Engine can now run inside Claude Code (was impossible with OAuth tokens and `claude -p` subprocess)
+
 ## 2.0.34
 
 ### Deterministic Workflow Conversion (PRs #38–#45)

CONTRIBUTING.md

@@ -2,24 +2,24 @@
 
 ## Adding a Workflow
 
-Most command logic lives in YAML workflows executed by the Go engine. Only 8 slash commands remain as tab-completable entry points.
+Most command logic lives in YAML workflows executed by the Go engine. Only 6 slash commands remain as tab-completable entry points — everything else is context-activated via skills.
 
 1. Create `workflows/my-workflow.yml` with steps, model assignments, and loop/gate definitions
-2. Test with `devkit workflow run my-workflow "input"`
+2. Test with `devkit_start` MCP tool or `devkit workflow run my-workflow "input"` from terminal
 3. Optionally add a context-activated skill in `skills/` to auto-trigger it
 
 See `skills/creating-workflows/SKILL.md` for YAML schema reference.
 
 ### Adding a Slash Command (rare — only for top-level entry points)
 
-Only add a command if it needs tab-completion. Most workflows are invoked via `devkit workflow run` or context-activated skills.
+Only add a command if it needs tab-completion. Most workflows are invoked via MCP tools (`devkit_start`/`devkit_advance`) or context-activated skills.
 
 1. Create `commands/my-command.md` with YAML frontmatter:
    ```markdown
    ---
    description: What this command does.
    ---
-   Run `devkit workflow run my-workflow` to execute.
+   Use `devkit_start` MCP tool with workflow name to execute.
    ```
 
 The command name is derived from the filename: `commands/my-command.md` becomes `/devkit:my-command`.

README.md

@@ -1,6 +1,6 @@
 # Devkit
 
-A deterministic development harness for AI agents. The Go engine controls orchestration (loops, branches, gates, budgets). The agent handles creativity. Every change is measured, gated, and auditable.
+A deterministic development harness for AI agents. The MCP engine controls workflow execution (step ordering, gates, loops, branches). The agent handles creativity. Every step is enforced, measured, and auditable.
 
 Works with just Claude. Optionally adds Codex and Gemini for multi-agent consensus.
 
@@ -68,7 +68,7 @@ These handle concerns devkit doesn't — methodology, specialized reviews, and c
 
 ```bash
 brew install rtk       # Token optimization (60-90% savings on Bash output)
-brew install ast-grep  # AST-based repo mapping (devkit workflow run repo-map)
+brew install ast-grep  # AST-based repo mapping (used by onboard skill)
 ```
 
 ### Verify
@@ -91,31 +91,51 @@ This shows which CLIs are installed, which agents are available, and which comma
 # "research the best auth library for Node"
 
 # Slash commands for complex workflows:
-/devkit:pr-ready              # Full PR pipeline
 /tri:review                   # Multi-agent code review
-devkit workflow run self-lint "npm run lint"  # Fix all lint errors
+# Or just describe: "submit a PR", "ship this" → pr-ready skill auto-activates
 ```
 
 ---
 
+## How It Works
+
+Devkit runs as an **MCP server** inside Claude Code. When a workflow starts, the engine takes control:
+
+```
+devkit_start("research", "best Go testing frameworks")
+  → Engine creates session, returns Step 1 + condensed principles
+  → Claude executes the step using standard tools
+  → Claude calls devkit_advance(session_id)
+  → Engine validates, records output, returns Step 2
+  → ...repeat until WORKFLOW COMPLETE
+
+Enforcement (runs automatically):
+  PreToolUse hook → blocks out-of-step actions during command steps
+  Stop hook → prevents session end during active workflows
+```
+
+**Why MCP?** Claude can't skip steps because the engine controls what comes next. Claude can't call tools that aren't valid for the current step. The engine holds state — Claude doesn't self-report.
+
+---
+
 ## Commands
 
-8 tab-completable slash commands. All other workflows are context-activated via skills or invoked directly with `devkit workflow run <name>`.
+6 tab-completable slash commands. All other workflows are context-activated via skills (auto-triggered by natural language) or invoked via MCP tools.
 
 | Command | What it does |
 |---|---|
 | `/tri:review` | Code review from 1-3 agents, consolidated report |
 | `/tri:debug` | Independent root-cause analysis from each agent |
 | `/tri:security` | Security audit with severity-ranked consensus |
-| `/devkit:pr-ready` | Lint, test, security, changelog, create PR |
-| `/devkit:pr-monitor` | Watch CI, fetch reviewer comments, fix iteratively, push |
-| `/devkit:workflow` | Run user-defined YAML workflows |
+| `/devkit:workflow` | Run any YAML workflow by name |
 | `/devkit:status` | Health check |
 | `/devkit:setup-rules` | Install language-specific coding rules to `~/.claude/rules/` |
 
-### Workflows (via `devkit workflow run <name>`)
+Tasks like "ship this PR" or "submit a PR" auto-activate the `pr-ready` skill — no slash command needed.
 
-All 18 YAML workflows can be invoked directly. Skills auto-activate for common triggers (e.g., "research X", "fix this bug", "add a feature").
+### Workflows
+
+All 18 YAML workflows are invoked via the MCP engine. Skills auto-activate for common triggers (e.g., "research X", "fix this bug", "add a feature").
 
 | Workflow | What it does |
 |---|---|
@@ -155,26 +175,28 @@ Skills activate automatically based on context. No slash command needed.
 | "scrape this URL" | `scrape` |
 | "create an ADR" | `adr` |
 
-Coding principles (`clean-code`, `dry`, `yagni`, `dont-reinvent`, `executing`, `stuck`, `scratchpad`) load as reference when relevant.
+Coding principles (`clean-code`, `dry`, `yagni`, `dont-reinvent`, `executing`, `stuck`, `scratchpad`) are injected as condensed rules (~120 tokens) per workflow step — not loaded as full skill files.
 
 ---
 
 ## Hooks
 
-10 hooks across 4 lifecycle events. All installed automatically with the plugin.
+12 hooks across 4 lifecycle events. All installed automatically with the plugin.
 
 | Event | Hook | What it catches |
 |---|---|---|
 | PreToolUse | **safety-check** | `rm -rf /`, `DROP TABLE`, force push, editing secrets |
 | PreToolUse | **security-patterns** | `eval()`, XSS, shell injection, weak hashes, hardcoded secrets |
 | PreToolUse | **audit-trail** | Logs every command to `.devkit/audit.log` |
-| PreToolUse | **pr-gate** | Prompts to run `/devkit:pr-ready` before `gh pr create` |
+| PreToolUse | **pr-gate** | Prompts to run the pr-ready skill before `gh pr create` |
 | PreToolUse | **rtk-rewrite** | Compresses Bash output via RTK (no-op if not installed) |
+| PreToolUse | **devkit-guard** | Blocks out-of-step tools during workflow command steps |
 | PostToolUse | **post-validate** | Suppressed errors, leaked secrets, writes outside repo |
 | PostToolUse | **slop-detect** | AI code patterns — doc/code imbalance, restating comments |
 | PostToolUse | **lang-review** | Language-aware checks: Go, TypeScript, Rust, Python, Shell |
 | SubagentStop | **subagent-stop** | Verifies subagent work before accepting |
 | Stop | **stop-gate** | Merge conflicts, cross-domain test gaps, linter pass |
+| Stop | **devkit-stop-guard** | Blocks session end during active workflows |
 
 ---
 
@@ -211,78 +233,28 @@ Language-specific rules that auto-activate when Claude reads matching files. Ins
 
 ---
 
-## Go CLI Harness
-
-The compiled Go binary handles deterministic orchestration — the machine controls the loop, the agent is the body.
-
-### Build
-
-```bash
-cd src && make install
-```
-
-### What it does that plugins can't
-
-- **Exact iteration counts** — Go owns the loop, not the LLM
-- **Command steps** — run shell commands directly in workflows, $0 cost
-- **Loop gates** — shell command after each iteration, auto-revert on failure
-- **YAML workflows** — branching, loops, parallel dispatch, budget enforcement
-- **Triage-based skipping** — typo fix doesn't run a 14-step pipeline
-- **Crash recovery** — SQLite state survives crashes
-- **Hard budget caps** — stops at your dollar limit
-- **True parallel dispatch** — goroutines, not sequential prompts
-
-### Examples
-
-```bash
-# Run 50 improvement iterations, stop at $20
-devkit improve --metric "npm test" --iterations 50 --budget 20.00
-
-# Implement a feature with test verification
-devkit feature "add JWT auth" --target src/auth/ --test "npm test"
-
-# Multi-agent review
-devkit review
-
-# Run any YAML workflow
-devkit workflow feature "add JWT auth"
-
-# Check session history
-devkit status
-```
-
-### Testing
-
-```bash
-cd src && go test ./... -v
-```
-
-140+ tests across 6 packages. All use mock runners — no API calls needed.
-
----
-
 ## Architecture
 
 ```
-Workflow Engine (Go binary)
+MCP Server (bin/devkit mcp — auto-started by plugin)
+  ├── Tools: devkit_start, devkit_advance, devkit_status, devkit_list
+  ├── State: session.json (hot, <50ms reads) + SQLite (cold history)
   ├── Parse YAML → validate steps, branches, budget
-  ├── Create session + git branch
   ├── Walk steps:
-  │   ├── Command steps → shell execution (deterministic, $0)
-  │   ├── Prompt steps → LLM runner (Claude/Codex/Gemini)
+  │   ├── Command steps → engine executes shell directly ($0 cost)
+  │   ├── Prompt steps → Claude works, calls devkit_advance when done
   │   ├── Loop with gate → run, verify, keep or revert
   │   ├── Branch → case-insensitive substring match → goto
-  │   ├── Parallel → goroutines with mutex
-  │   └── Budget check every step
-  └── Commit, report, clean up
+  │   └── Parallel → Agent tool dispatch (Claude/Codex/Gemini)
+  └── Principles injected per step (~120 tokens, not full skill files)
 
-Multi-Agent (tri-* workflows)
-  ├── Claude  → native background agent (always)
-  ├── Codex   → plugin or CLI (optional)
-  └── Gemini  → plugin or CLI (optional)
+Enforcement:
+  ├── MCP tool scoping — Claude can only call devkit_advance to progress
+  ├── PreToolUse hook — exit 2 blocks tools during command steps
+  └── Stop hook — blocks session end during active workflows
 
-Self-Improvement (self-* workflows)
-  └── Loop: propose → measure → keep/revert → repeat
+Terminal fallback (devkit workflow run <name>):
+  └── Subprocess runners for Codex/Gemini CLI usage
 ```
 
 ---
@@ -291,18 +263,18 @@ Self-Improvement (self-* workflows)
 
 ```
 devkit/
-├── commands/          # 8 slash commands (tab-completable entry points)
-├── skills/            # 19 context-activated skills
+├── commands/          # 6 slash commands (tab-completable entry points)
+├── skills/            # 20 context-activated skills + _principles.yml
 ├── agents/            # 6 agents (reviewer, researcher, improver, ...)
-├── hooks/             # 10 hooks (safety, security, quality gates)
+├── hooks/             # 12 hooks (safety, security, quality gates, workflow enforcement)
 ├── workflows/         # 18 YAML workflow definitions
 ├── resources/rules/   # Language-specific coding rules
-├── presets/            # Reserved for future use
-├── src/               # Go CLI harness
+├── src/               # Go engine + MCP server
+│   ├── mcp/           # MCP server (tools, principles loader, session management)
 │   ├── engine/        # YAML workflow engine (parser, executor, tests)
-│   ├── runners/       # Claude, Codex, Gemini interfaces
-│   ├── loops/         # Improve, feature, bugfix, refactor, testgen
-│   ├── lib/           # DB, git, metrics, reporting
-│   └── cmd/           # CLI entry points
-└── .github/workflows/ # CI (build+test+vet) + auto-release
+│   ├── runners/       # Codex, Gemini interfaces (terminal fallback)
+│   ├── lib/           # DB, git, metrics, session state, reporting
+│   └── cmd/           # CLI entry points (including `devkit mcp`)
+├── bin/               # Auto-PATH binary (built by make install-plugin)
+└── .github/workflows/ # CI (build+test+vet) + auto-release (6 platforms)
 ```

ROADMAP.md

@@ -2,11 +2,12 @@
 
 ## Implemented
 
-- **8 slash commands** — Tab-completable entry points (tri-review, tri-debug, tri-security, pr-ready, pr-monitor, status, setup-rules, workflow); 16 former commands now context-activated via skills or invoked directly via `devkit workflow run`
+- **MCP engine** — Go server exposes `devkit_start`, `devkit_advance`, `devkit_status`, `devkit_list` tools inside Claude Code. Step ordering enforced via MCP tool scoping + PreToolUse hook exit 2. Session state in session.json (hot path, <50ms hook reads) + SQLite (cold history). ~65% token reduction vs old monolithic prompts.
+- **6 slash commands** — Tab-completable entry points for things that need explicit invocation (tri-review, tri-debug, tri-security, status, setup-rules, workflow); 18 former commands now context-activated via skills or invoked via MCP tools (pr-monitor folded into pr-ready workflow; pr-ready is now a natural-language skill)
 - **Deterministic workflow conversion** — All command logic moved from LLM-interpreted markdown to Go-engine-driven YAML workflows; ~3,600 lines of inline logic removed
-- **19 context-activated skills** — 9 auto-trigger workflows (test-gen, doc-gen, changelog, onboard, research, deep-research, scrape, autoloop, adr) + 6 coding principles (executing, clean-code, DRY, YAGNI, dont-reinvent, stuck) + 2 tools (gcli, creating-workflows) + 1 iteration memory (scratchpad) + 1 orchestration (mega-pr)
+- **20 context-activated skills** — 10 auto-trigger workflows (test-gen, doc-gen, changelog, onboard, research, deep-research, scrape, autoloop, adr, pr-ready) + 6 coding principles (executing, clean-code, DRY, YAGNI, dont-reinvent, stuck) + 2 tools (gcli, creating-workflows) + 1 iteration memory (scratchpad) + 1 orchestration (mega-pr)
 - **6 agents** — Scoped tool access, worktree isolation, model assignment
-- **10 hooks** — Safety (destructive command blocking, edit-time security patterns, PR gate), observability (audit trail, slop detection, post-validation, subagent verification, language-aware code review), optimization (RTK token compression)
+- **12 hooks** — Safety (destructive command blocking, edit-time security patterns, PR gate), observability (audit trail, slop detection, post-validation, subagent verification, language-aware code review), optimization (RTK token compression), workflow enforcement (devkit-guard, devkit-stop-guard)
 - **Graceful degradation** — tri:* commands work with 1-3 agents depending on installed CLIs
 - **Goal decomposition** — Task DAG with dependency ordering and parallel execution
 - **Concurrency limits** — Max 3 parallel agents in multi-agent commands