diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..8ba947e --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,71 @@ +name: CI + +on: + push: + branches: [main] + pull_request: + workflow_dispatch: + +# A queued run on the same ref is superseded by a newer push. +concurrency: + group: ci-${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +permissions: + contents: read + +jobs: + verify: + name: verify (node ${{ matrix.node }}) + runs-on: ubuntu-latest + timeout-minutes: 15 + strategy: + fail-fast: false + matrix: + # 22 is the floor declared in engines; 24 is the current line. + node: ['22', '24'] + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-node@v4 + with: + node-version: ${{ matrix.node }} + + # MaskShift has no runtime dependency tree, so there is nothing to install. + # Assert that stays true rather than trusting it. + - name: Assert zero runtime dependencies + run: node scripts/assert-no-dependencies.mjs + + - name: Syntax check + run: npm run check + + - name: Unit and integration tests + run: npm test + + - name: End-to-end smoke test + run: npm run smoke + + docs: + name: docs are in sync + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-node@v4 + with: + node-version: '22' + + # docs/ is generated from the runtime. Regenerating it here proves the committed + # copy matches the code, and that generation stays hermetic: the runner has no + # ~/.claude or ~/.codex skills, so a generator that scanned them would produce a + # different file here than on a developer machine, and this step would fail. + - name: Regenerate documentation + run: npm run docs + + - name: Fail if the committed docs differ + run: | + if ! git diff --exit-code --stat -- docs/; then + echo "::error::docs/ is stale. Run 'npm run docs' and commit the result." + exit 1 + fi diff --git a/CHANGELOG.md b/CHANGELOG.md index 4cae4cb..906def4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,13 @@ ## Unreleased +- **Model output now streams.** Every provider type streams a turn token by token instead of returning it whole: Server-Sent Events for `openai-responses`, `openai-compatible`, `anthropic` and `gemini`, newline-delimited JSON for `ollama`, all read with Node's built-in `fetch` body and `TextDecoder` — no dependency. `maskshift run` writes tokens straight to stdout (and stops repeating the finished answer under the closing rule when it just streamed it), and the interface renders the answer being written where it used to show a spinner, repainting on the ticker it already runs while busy. Tool-call arguments split across frames are reassembled before parsing. Streaming is skipped where it would mislead: the text tool protocol rewrites content after the turn, so it stays unary, and an endpoint that accepts `stream: true` but replies with one JSON body is detected from its content type and parsed normally rather than reporting an empty turn. Control it with `streaming` globally or per provider. +- **Model requests now survive a transient failure.** A single 429 or dropped connection used to end a run outright — `fetchJson` recorded the status code and no caller ever read it. Requests now retry HTTP 408/409/425/429/500/502/503/504 and network errors with exponential backoff and jitter, honouring `Retry-After`; 400/401/403/404/422 are never retried (the tool-protocol downgrade depends on a 400 surfacing at once), cancelling a run breaks out of the backoff instead of waiting it out, a stream is only retried before its first token so nothing is shown twice, and model-listing probes stay single-shot. Configure with `providerRetry`, or `retry` on one provider. +- Added `.github/workflows/ci.yml`: `npm run check`, `npm test` and `npm run smoke` on Node 22 and 24, plus a job that regenerates `docs/` and fails if the committed copy differs. Added `npm run deps` (`scripts/assert-no-dependencies.mjs`), which asserts the zero-dependency claim mechanically — no runtime dependency fields in `package.json`, and no bare import specifiers anywhere in `src/`, `bin/`, `scripts/` or `tests/` — and wired it into `npm run verify` and CI. +- **Fixed `npm run docs` publishing the generating machine's home directory.** `skillsDirs` includes `~/.claude/skills` and `~/.codex/skills` by default, so the generator documented whatever the person running it happened to have installed: `docs/SKILLS.md` listed 44 skills where the repository ships 36, and eight rows carried paths like `../../../root/.claude/skills/synced//docx`. Generation is now scoped to the bundled `skills/` directory, refuses to emit any path outside the repository, and drops the `generatedAt` timestamp from `docs/CAPABILITY-MANIFEST.json` so the output is reproducible and the CI drift check is possible at all. Corrected the README's skill count and badge to 36. +- Fixed `port_inspect` falling through to a bare `netstat -anp` without checking that netstat exists, which returned a shell "not found" with exit 127 instead of an error. It now reports which inspector ran and whether anything matched, and errors clearly when none is installed. The accompanying test asserted `code === 0` on an unused port, but `ss`, `lsof` and `netstat` all exit non-zero when a filter matches nothing — so the suite failed on any machine, and the failure aborted the scenario before `rsync_transfer` was reached, making the coverage gate blame two tools it had never run. Scenarios now record why they aborted and the gate reports that cause instead. +- Fixed `VERSION` being hardcoded as `1.0.0` in `src/core/utils.mjs` while `package.json` said `1.0.1`, so every banner, `--version`, `doctor` report, generated document and LSP `clientInfo` announced the wrong version. It is read from the manifest now, and the test that asserted the literal reads the manifest too. +- Removed the `force` option from repository indexing, including the `repo_index` tool schema and the `workspace index --force` flag. `index()` accepted it and `#runIndex` never declared a second parameter, so it was dropped on the floor at every call site — and since every pass is a full rescan there was never anything for it to force. - Published to npm as `maskshift`. `package.json` now carries `files`, `repository`, `homepage`, `bugs` and `author` metadata, and the README documents `npm install -g maskshift` / `npx maskshift` alongside the source-based install paths. - **Made the interface mouse-driven as well as keyboard-driven.** Added an SGR mouse decoder (`?1006`, with the legacy X10 encoding as a fallback) to the raw-mode input layer and `src/tui/regions.mjs`, a per-frame hit-test registry: each surface declares the cells it occupies while it paints, so a click resolves against exactly the frame the user was looking at. View tabs, rail tabs, the `TARGET`/`PERSONA`/mode chips, the session title, hint-rail keys, starter prompts, list rows, catalogue tabs and filters, scrollbars and every overlay row, field and button now respond to a click; the wheel scrolls whatever is under the pointer without stealing focus; a click outside an overlay dismisses it. Reporting is switched off whenever the alternate screen is left, so quitting or crashing cannot strand the terminal in tracking mode. `MASKSHIFT_MOUSE=click|hover|off`, the `ui.mouse` setting, `f2` and the `mouse.cycle` palette action all control it, since tracking suppresses the terminal's own text selection (most terminals still select on shift+drag). - Fixed the heist view dropping the composer's bottom rule: the transcript and composer were two stacked frames, and the lower one was trimmed to fit the region, so the panel rendered permanently unclosed. They are now one frame split by an internal seam, which also recovers three rows of transcript and removes the doubled rule through the middle of the view. diff --git a/README.md b/README.md index 840b343..d11b35f 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,7 @@ Node 22+ Runtime dependencies: none 149 tools - 44 skills + 36 skills GPL-3.0

@@ -42,8 +42,9 @@ runtime dependency tree, no HTTP server, no browser, no listening socket. | | | |---|---| | **149 native tools** | Filesystem, shell and process control, search and indexing, Git worktrees and checkpoints, LSP, browsers over CDP, containers and Kubernetes, SSH and rsync, databases, runtimes, images, PDF and Jupyter, web retrieval, plugins, automations, memory and orchestration. → [tool inventory](docs/TOOLS.md) | -| **44 bundled skills** | Loaded lazily by description, alongside skills imported from Claude, Codex, Copilot and workspace skill directories. → [skills](docs/SKILLS.md) | +| **36 bundled skills** | Loaded lazily by description, alongside skills imported from Claude, Codex, Copilot and workspace skill directories. → [skills](docs/SKILLS.md) | | **Lazy MCP fabric** | stdio and Streamable HTTP, stateless and legacy initialization, resources, prompts, qualified tools, imported configs, and the live official MCP Registry. Servers connect on demand, so the catalog never floods the context window. → [MCP config](docs/CONFIGURATION.md#mcp-definitions) | +| **Streaming and resilient** | Turns arrive token by token over SSE or NDJSON from every provider type, and a transient 429, 5xx or dropped connection is retried with jittered backoff instead of ending the run. → [streaming](docs/CONFIGURATION.md#streaming), [retries](docs/CONFIGURATION.md#retries) | | **Any model** | Ollama, OpenAI Responses, OpenAI-compatible servers, Anthropic, Gemini, OpenRouter, LM Studio and vLLM — with a text protocol that gives models *without* a native tool API the full tool surface. → [providers](docs/CONFIGURATION.md#providers) | | **Autonomous repo context** | Project instructions, manifests, repository tree, indexed code chunks (lexical plus optional semantic retrieval), stored memory, recent history and Git state. → [architecture](docs/ARCHITECTURE.md) | | **Parallel agents** | Independent sessions and optional isolated Git worktrees for delegated work. | diff --git a/docs/CAPABILITY-MANIFEST.json b/docs/CAPABILITY-MANIFEST.json index f366346..8a98cb7 100644 --- a/docs/CAPABILITY-MANIFEST.json +++ b/docs/CAPABILITY-MANIFEST.json @@ -1,8 +1,7 @@ { - "generatedAt": "2026-09-05T16:47:23.027Z", - "version": "1.0.0", + "version": "1.0.1", "nativeToolCount": 149, - "bundledSkillCount": 44, + "bundledSkillCount": 36, "curatedMcpCount": 8, "tools": [ { @@ -1358,8 +1357,7 @@ "name": "architecture", "description": "Design system architecture with explicit requirements, boundaries, data flows, tradeoffs, failure modes, and evolution paths." }, - "bodyChars": 615, - "updatedAt": "2026-09-03T13:19:11.718Z" + "bodyChars": 615 }, { "name": "backend-api", @@ -1371,8 +1369,7 @@ "name": "backend-api", "description": "Design and implement reliable local or network APIs with validation, streaming, cancellation, observability, and stable contracts." }, - "bodyChars": 535, - "updatedAt": "2026-09-03T13:19:11.718Z" + "bodyChars": 535 }, { "name": "browser-automation", @@ -1384,8 +1381,7 @@ "name": "browser-automation", "description": "Automate and verify browser workflows using accessibility-first locators, deterministic waits, screenshots, and failure artifacts." }, - "bodyChars": 450, - "updatedAt": "2026-09-03T13:19:11.718Z" + "bodyChars": 450 }, { "name": "build-repair", @@ -1397,8 +1393,7 @@ "name": "build-repair", "description": "Repair broken builds, type checks, packaging, and startup paths across environments with minimal, verifiable changes." }, - "bodyChars": 496, - "updatedAt": "2026-09-03T13:19:11.718Z" + "bodyChars": 496 }, { "name": "ci-cd", @@ -1410,8 +1405,7 @@ "name": "ci-cd", "description": "Create and repair CI/CD pipelines with caching, parallelism, reproducibility, artifacts, and actionable failures." }, - "bodyChars": 509, - "updatedAt": "2026-09-03T13:19:11.718Z" + "bodyChars": 509 }, { "name": "code-review", @@ -1423,8 +1417,7 @@ "name": "code-review", "description": "Review changes for real correctness, maintainability, regressions, missing tests, and integration defects with low-noise findings." }, - "bodyChars": 586, - "updatedAt": "2026-09-03T13:19:11.718Z" + "bodyChars": 586 }, { "name": "codebase-indexing", @@ -1436,8 +1429,7 @@ "name": "codebase-indexing", "description": "Build and maintain scalable repository indexes for lexical, structural, symbol, and semantic retrieval with bounded context." }, - "bodyChars": 611, - "updatedAt": "2026-09-03T13:19:11.718Z" + "bodyChars": 611 }, { "name": "cpp-qt", @@ -1449,8 +1441,7 @@ "name": "cpp-qt", "description": "Implement and debug modern C++ and Qt applications, including signals/slots, models, threading, CMake, rendering, and ownership." }, - "bodyChars": 588, - "updatedAt": "2026-09-03T13:19:11.718Z" + "bodyChars": 588 }, { "name": "data-analysis", @@ -1462,8 +1453,7 @@ "name": "data-analysis", "description": "Analyze datasets reproducibly, validate quality, calculate defensible metrics, and produce clear artifacts." }, - "bodyChars": 490, - "updatedAt": "2026-09-03T13:19:11.718Z" + "bodyChars": 490 }, { "name": "database-migrations", @@ -1475,8 +1465,7 @@ "name": "database-migrations", "description": "Evolve persistent schemas safely with compatible migrations, backfills, rollback strategy, and integrity checks." }, - "bodyChars": 527, - "updatedAt": "2026-09-03T13:19:11.718Z" + "bodyChars": 527 }, { "name": "debugging", @@ -1488,8 +1477,7 @@ "name": "debugging", "description": "Diagnose and repair software failures using evidence, reproduction, instrumentation, and regression tests rather than speculative edits." }, - "bodyChars": 549, - "updatedAt": "2026-09-03T13:19:11.721Z" + "bodyChars": 549 }, { "name": "dependency-upgrade", @@ -1501,8 +1489,7 @@ "name": "dependency-upgrade", "description": "Upgrade libraries and runtimes with compatibility research, focused changes, lockfile integrity, and regression verification." }, - "bodyChars": 520, - "updatedAt": "2026-09-03T13:19:11.721Z" + "bodyChars": 520 }, { "name": "docker-kubernetes", @@ -1514,8 +1501,7 @@ "name": "docker-kubernetes", "description": "Containerize and operate development services with reproducible images, health checks, persistent state, and clear networking." }, - "bodyChars": 520, - "updatedAt": "2026-09-03T13:19:11.721Z" + "bodyChars": 520 }, { "name": "documentation", @@ -1527,22 +1513,7 @@ "name": "documentation", "description": "Write accurate user, operator, API, and architecture documentation grounded in the implemented product." }, - "bodyChars": 530, - "updatedAt": "2026-09-03T13:19:11.721Z" - }, - { - "name": "docx", - "description": "Use this skill whenever the user wants to create, read, edit, or manipulate Word documents (.docx) or Word templates (.dotx). Triggers include: any mention of 'Word doc', 'word document', '.docx', '.dotx', or requests to produce professional documents with formatting like tables of contents, page numbers, or letterheads. Also use when extracting or reorganizing content from .docx or .dotx files, inserting or replacing images in documents, find-and-replace in Word files, working with tracked changes or comments, or converting content into a polished Word document. If the user asks for a 'report', 'memo', 'letter', 'template', or similar deliverable as a Word or .docx file (to download, email or print), use this skill. However, if they ask for a document, page, report, memo, or notes WITHOUT naming a file format and the session offers a dedicated document or page skill or connector, use that instead. Do NOT use for PDFs, spreadsheets, Google Docs, or coding unrelated to document generation.", - "path": "../../../root/.claude/skills/synced/a4481896-b89c-449e-a11d-7b2d047e6ce3_d06b37e9-9318-4502-ad2e-d8f06ef9935c/docx", - "file": "../../../root/.claude/skills/synced/a4481896-b89c-449e-a11d-7b2d047e6ce3_d06b37e9-9318-4502-ad2e-d8f06ef9935c/docx/SKILL.md", - "source": "local", - "meta": { - "name": "docx", - "description": "Use this skill whenever the user wants to create, read, edit, or manipulate Word documents (.docx) or Word templates (.dotx). Triggers include: any mention of 'Word doc', 'word document', '.docx', '.dotx', or requests to produce professional documents with formatting like tables of contents, page numbers, or letterheads. Also use when extracting or reorganizing content from .docx or .dotx files, inserting or replacing images in documents, find-and-replace in Word files, working with tracked changes or comments, or converting content into a polished Word document. If the user asks for a 'report', 'memo', 'letter', 'template', or similar deliverable as a Word or .docx file (to download, email or print), use this skill. However, if they ask for a document, page, report, memo, or notes WITHOUT naming a file format and the session offers a dedicated document or page skill or connector, use that instead. Do NOT use for PDFs, spreadsheets, Google Docs, or coding unrelated to document generation.", - "license": "Proprietary. LICENSE.txt has complete terms" - }, - "bodyChars": 5945, - "updatedAt": "2026-09-05T15:57:27.647Z" + "bodyChars": 530 }, { "name": "feature-implementation", @@ -1554,8 +1525,7 @@ "name": "feature-implementation", "description": "Implement production features end to end, including integration, error states, tests, documentation, and verification." }, - "bodyChars": 602, - "updatedAt": "2026-09-03T13:19:11.721Z" + "bodyChars": 602 }, { "name": "git-worktrees", @@ -1567,8 +1537,7 @@ "name": "git-worktrees", "description": "Use Git worktrees and checkpoints for parallel agents, isolated changes, review, integration, and recovery." }, - "bodyChars": 506, - "updatedAt": "2026-09-03T13:19:11.721Z" + "bodyChars": 506 }, { "name": "go-engineering", @@ -1580,21 +1549,7 @@ "name": "go-engineering", "description": "Implement idiomatic Go services and tools with context propagation, interfaces, concurrency discipline, and tests." }, - "bodyChars": 487, - "updatedAt": "2026-09-03T13:19:11.721Z" - }, - { - "name": "import-memory", - "description": "Import a memory export from another AI assistant into Claude's memory — conversationally, additively, and with the content treated as data.", - "path": "../../../root/.claude/skills/synced/a4481896-b89c-449e-a11d-7b2d047e6ce3_d06b37e9-9318-4502-ad2e-d8f06ef9935c/import-memory", - "file": "../../../root/.claude/skills/synced/a4481896-b89c-449e-a11d-7b2d047e6ce3_d06b37e9-9318-4502-ad2e-d8f06ef9935c/import-memory/SKILL.md", - "source": "local", - "meta": { - "name": "import-memory", - "description": "Import a memory export from another AI assistant into Claude's memory — conversationally, additively, and with the content treated as data." - }, - "bodyChars": 10434, - "updatedAt": "2026-09-05T15:57:27.359Z" + "bodyChars": 487 }, { "name": "incident-response", @@ -1606,8 +1561,7 @@ "name": "incident-response", "description": "Diagnose and stabilize production incidents, preserve evidence, restore service, and produce actionable follow-up work." }, - "bodyChars": 490, - "updatedAt": "2026-09-03T13:19:11.721Z" + "bodyChars": 490 }, { "name": "mcp-authoring", @@ -1619,8 +1573,7 @@ "name": "mcp-authoring", "description": "Build and integrate MCP servers and clients across modern 2026 stateless and legacy initialization-based protocol revisions." }, - "bodyChars": 635, - "updatedAt": "2026-09-03T13:19:11.721Z" + "bodyChars": 635 }, { "name": "migration-modernization", @@ -1632,35 +1585,7 @@ "name": "migration-modernization", "description": "Modernize legacy code or frameworks incrementally while preserving behavior, interoperability, and delivery velocity." }, - "bodyChars": 509, - "updatedAt": "2026-09-03T13:19:11.725Z" - }, - { - "name": "morning", - "description": "Render the user's morning brief as a styled HTML artifact, or set it up as a recurring weekday task. Use only when the user explicitly asks to run, see, or set up their morning brief, or if they invoke /morning by name. A question about their day, schedule, or calendar is not by itself a request for the brief; answer it directly instead.", - "path": "../../../root/.claude/skills/synced/a4481896-b89c-449e-a11d-7b2d047e6ce3_d06b37e9-9318-4502-ad2e-d8f06ef9935c/morning", - "file": "../../../root/.claude/skills/synced/a4481896-b89c-449e-a11d-7b2d047e6ce3_d06b37e9-9318-4502-ad2e-d8f06ef9935c/morning/SKILL.md", - "source": "local", - "meta": { - "name": "morning", - "description": "Render the user's morning brief as a styled HTML artifact, or set it up as a recurring weekday task. Use only when the user explicitly asks to run, see, or set up their morning brief, or if they invoke /morning by name. A question about their day, schedule, or calendar is not by itself a request for the brief; answer it directly instead." - }, - "bodyChars": 17571, - "updatedAt": "2026-09-05T15:57:27.657Z" - }, - { - "name": "pdf", - "description": "Use this skill whenever the user wants to do anything with PDF files. This includes reading or extracting text/tables from PDFs, combining or merging multiple PDFs into one, splitting PDFs apart, rotating pages, adding watermarks, creating new PDFs, filling PDF forms, encrypting/decrypting PDFs, extracting images, and OCR on scanned PDFs to make them searchable. If the user mentions a .pdf file or asks to produce one, use this skill.", - "path": "../../../root/.claude/skills/synced/a4481896-b89c-449e-a11d-7b2d047e6ce3_d06b37e9-9318-4502-ad2e-d8f06ef9935c/pdf", - "file": "../../../root/.claude/skills/synced/a4481896-b89c-449e-a11d-7b2d047e6ce3_d06b37e9-9318-4502-ad2e-d8f06ef9935c/pdf/SKILL.md", - "source": "local", - "meta": { - "name": "pdf", - "description": "Use this skill whenever the user wants to do anything with PDF files. This includes reading or extracting text/tables from PDFs, combining or merging multiple PDFs into one, splitting PDFs apart, rotating pages, adding watermarks, creating new PDFs, filling PDF forms, encrypting/decrypting PDFs, extracting images, and OCR on scanned PDFs to make them searchable. If the user mentions a .pdf file or asks to produce one, use this skill.", - "license": "Proprietary. LICENSE.txt has complete terms" - }, - "bodyChars": 7513, - "updatedAt": "2026-09-05T15:57:27.531Z" + "bodyChars": 509 }, { "name": "performance", @@ -1672,8 +1597,7 @@ "name": "performance", "description": "Profile and optimize latency, throughput, memory, context usage, and startup costs using measured bottlenecks." }, - "bodyChars": 579, - "updatedAt": "2026-09-03T13:19:11.725Z" + "bodyChars": 579 }, { "name": "plugin-authoring", @@ -1685,22 +1609,7 @@ "name": "plugin-authoring", "description": "Create modular MaskShift plugins that add tools, skills, providers, hooks, UI panels, or MCP catalog sources without coupling core policy." }, - "bodyChars": 513, - "updatedAt": "2026-09-03T13:19:11.725Z" - }, - { - "name": "pptx", - "description": "Use this skill any time a .pptx or .potx file is involved in any way — as input, output, or both. This includes: creating slide decks, pitch decks, or presentations as PowerPoint (.pptx) files; reading, parsing, or extracting text from any .pptx or .potx file (even if the extracted content will be used elsewhere, like in an email, summary, or creating a different type of slide deck); editing, modifying, or updating existing presentations; combining or splitting slide files; working with templates (.potx), layouts, speaker notes, or comments. Trigger whenever the user asks for a PowerPoint or .pptx file, or references a .pptx or .potx filename, regardless of what they plan to do with the content afterward. However, when the user asks for a deck, slides, a slide deck, or a presentation without naming a file format, default to using a dedicated slide-deck artifact type or a separate slides skill if this session offers one; otherwise, use this skill.", - "path": "../../../root/.claude/skills/synced/a4481896-b89c-449e-a11d-7b2d047e6ce3_d06b37e9-9318-4502-ad2e-d8f06ef9935c/pptx", - "file": "../../../root/.claude/skills/synced/a4481896-b89c-449e-a11d-7b2d047e6ce3_d06b37e9-9318-4502-ad2e-d8f06ef9935c/pptx/SKILL.md", - "source": "local", - "meta": { - "name": "pptx", - "description": "Use this skill any time a .pptx or .potx file is involved in any way — as input, output, or both. This includes: creating slide decks, pitch decks, or presentations as PowerPoint (.pptx) files; reading, parsing, or extracting text from any .pptx or .potx file (even if the extracted content will be used elsewhere, like in an email, summary, or creating a different type of slide deck); editing, modifying, or updating existing presentations; combining or splitting slide files; working with templates (.potx), layouts, speaker notes, or comments. Trigger whenever the user asks for a PowerPoint or .pptx file, or references a .pptx or .potx filename, regardless of what they plan to do with the content afterward. However, when the user asks for a deck, slides, a slide deck, or a presentation without naming a file format, default to using a dedicated slide-deck artifact type or a separate slides skill if this session offers one; otherwise, use this skill.", - "license": "Proprietary. LICENSE.txt has complete terms" - }, - "bodyChars": 20375, - "updatedAt": "2026-09-05T15:57:27.647Z" + "bodyChars": 513 }, { "name": "prompt-evals", @@ -1712,8 +1621,7 @@ "name": "prompt-evals", "description": "Create evaluations for agent prompts, tool routing, context retrieval, and coding outcomes using reproducible tasks and scoring." }, - "bodyChars": 634, - "updatedAt": "2026-09-03T13:19:11.725Z" + "bodyChars": 634 }, { "name": "python-engineering", @@ -1725,8 +1633,7 @@ "name": "python-engineering", "description": "Build maintainable Python applications and automation with typed boundaries, packaging, environments, tests, and robust subprocess handling." }, - "bodyChars": 512, - "updatedAt": "2026-09-03T13:19:11.725Z" + "bodyChars": 512 }, { "name": "refactoring", @@ -1738,8 +1645,7 @@ "name": "refactoring", "description": "Restructure code safely while preserving behavior, reducing duplication, clarifying ownership, and maintaining compatibility." }, - "bodyChars": 471, - "updatedAt": "2026-09-03T13:19:11.725Z" + "bodyChars": 471 }, { "name": "release", @@ -1751,8 +1657,7 @@ "name": "release", "description": "Prepare reproducible production releases with versioning, changelogs, packaging, smoke tests, and rollback notes." }, - "bodyChars": 543, - "updatedAt": "2026-09-03T13:19:11.725Z" + "bodyChars": 543 }, { "name": "repo-cartography", @@ -1764,8 +1669,7 @@ "name": "repo-cartography", "description": "Map unfamiliar repositories and identify the correct files, dependency paths, conventions, and verification commands before implementation." }, - "bodyChars": 610, - "updatedAt": "2026-09-03T13:19:11.725Z" + "bodyChars": 610 }, { "name": "rust-engineering", @@ -1777,8 +1681,7 @@ "name": "rust-engineering", "description": "Implement reliable Rust systems with ownership-aware design, errors, async cancellation, tests, and workspace conventions." }, - "bodyChars": 404, - "updatedAt": "2026-09-03T13:19:11.725Z" + "bodyChars": 404 }, { "name": "scientific-computing", @@ -1790,8 +1693,7 @@ "name": "scientific-computing", "description": "Implement numerically sound scientific and engineering software with units, validation, reproducibility, and performance awareness." }, - "bodyChars": 540, - "updatedAt": "2026-09-03T13:19:11.725Z" + "bodyChars": 540 }, { "name": "security-audit", @@ -1803,8 +1705,7 @@ "name": "security-audit", "description": "Audit code and dependencies for concrete exploitable weaknesses while preserving MaskShift’s intentionally permissive local execution philosophy." }, - "bodyChars": 683, - "updatedAt": "2026-09-03T13:19:11.729Z" + "bodyChars": 683 }, { "name": "self-improving-skills", @@ -1816,47 +1717,19 @@ "name": "self-improving-skills", "description": "Capture durable, non-obvious workflow knowledge as reusable skills and improve it from verified outcomes." }, - "bodyChars": 592, - "updatedAt": "2026-09-03T13:19:11.729Z" - }, - { - "name": "skill-creator", - "description": "Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.", - "path": "../../../root/.claude/skills/synced/a4481896-b89c-449e-a11d-7b2d047e6ce3_d06b37e9-9318-4502-ad2e-d8f06ef9935c/skill-creator", - "file": "../../../root/.claude/skills/synced/a4481896-b89c-449e-a11d-7b2d047e6ce3_d06b37e9-9318-4502-ad2e-d8f06ef9935c/skill-creator/SKILL.md", - "source": "local", - "meta": { - "name": "skill-creator", - "description": "Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy." - }, - "bodyChars": 32807, - "updatedAt": "2026-09-05T15:57:27.415Z" - }, - { - "name": "startup-hook-skill", - "description": "Creating and developing startup hooks for Claude Code on the web. Use when the user wants to set up a repository for Claude Code on the web, create a SessionStart hook to ensure their project can run tests and linters during web sessions.", - "path": "../../../root/.claude/skills/session-start-hook", - "file": "../../../root/.claude/skills/session-start-hook/SKILL.md", - "source": "local", - "meta": { - "name": "startup-hook-skill", - "description": "Creating and developing startup hooks for Claude Code on the web. Use when the user wants to set up a repository for Claude Code on the web, create a SessionStart hook to ensure their project can run tests and linters during web sessions." - }, - "bodyChars": 4618, - "updatedAt": "2026-09-03T13:19:11.772Z" + "bodyChars": 592 }, { "name": "terminal-phantom-ui", - "description": "Build MaskShift terminal interfaces in the Phantom Protocol design language — stencil frames with notched title tabs, a crimson-and-bone palette that degrades to 16 colours, and the ANSI width discipline that keeps a TUI aligned.", + "description": "Build MaskShift terminal interfaces in the Phantom Protocol design language — tokens over hex values, one marker gutter per pane, one filled chip per screen, and the ANSI width discipline that keeps a TUI aligned.", "path": "skills/terminal-phantom-ui", "file": "skills/terminal-phantom-ui/SKILL.md", "source": "bundled", "meta": { "name": "terminal-phantom-ui", - "description": "Build MaskShift terminal interfaces in the Phantom Protocol design language — stencil frames with notched title tabs, a crimson-and-bone palette that degrades to 16 colours, and the ANSI width discipline that keeps a TUI aligned." + "description": "Build MaskShift terminal interfaces in the Phantom Protocol design language — tokens over hex values, one marker gutter per pane, one filled chip per screen, and the ANSI width discipline that keeps a TUI aligned." }, - "bodyChars": 5346, - "updatedAt": "2026-09-05T16:47:15.135Z" + "bodyChars": 8377 }, { "name": "test-engineering", @@ -1868,8 +1741,7 @@ "name": "test-engineering", "description": "Design and implement high-signal unit, integration, end-to-end, property, and regression tests for changed behavior." }, - "bodyChars": 625, - "updatedAt": "2026-09-03T13:19:11.729Z" + "bodyChars": 625 }, { "name": "typescript-engineering", @@ -1881,8 +1753,7 @@ "name": "typescript-engineering", "description": "Build strict TypeScript applications with stable runtime validation, modular architecture, and browser/server correctness." }, - "bodyChars": 484, - "updatedAt": "2026-09-03T13:19:11.729Z" + "bodyChars": 484 }, { "name": "ui-visual-qa", @@ -1894,8 +1765,7 @@ "name": "ui-visual-qa", "description": "Inspect web user interfaces for clipping, overlap, responsiveness, focus, loading, empty, error, and interaction-state defects." }, - "bodyChars": 500, - "updatedAt": "2026-09-03T13:19:11.729Z" + "bodyChars": 500 }, { "name": "web-research", @@ -1907,22 +1777,7 @@ "name": "web-research", "description": "Research changing technical facts from primary sources and convert findings into implementation decisions with traceability." }, - "bodyChars": 567, - "updatedAt": "2026-09-03T13:19:11.729Z" - }, - { - "name": "xlsx", - "description": "Use this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like \\\"the xlsx in my downloads\\\") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.", - "path": "../../../root/.claude/skills/synced/a4481896-b89c-449e-a11d-7b2d047e6ce3_d06b37e9-9318-4502-ad2e-d8f06ef9935c/xlsx", - "file": "../../../root/.claude/skills/synced/a4481896-b89c-449e-a11d-7b2d047e6ce3_d06b37e9-9318-4502-ad2e-d8f06ef9935c/xlsx/SKILL.md", - "source": "local", - "meta": { - "name": "xlsx", - "description": "Use this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like \\\"the xlsx in my downloads\\\") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.", - "license": "Proprietary. LICENSE.txt has complete terms" - }, - "bodyChars": 7504, - "updatedAt": "2026-09-05T15:57:27.647Z" + "bodyChars": 567 } ], "mcpServers": [ diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index 7fb60b3..9cea29b 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -183,6 +183,64 @@ Supported `type` values are `ollama`, `openai-responses`, `openai-compatible`, ` An `anthropic` provider entry also accepts `promptCaching` (default `true`). When enabled, MaskShift marks the stable system-prompt block, the active tool schema list, and the conversation-so-far boundary with `cache_control: {"type": "ephemeral"}` breakpoints so a run's repeated turns reuse cached input tokens instead of rebilling them in full. Set `"promptCaching": false` on the provider entry if a proxy in front of the Anthropic-compatible endpoint rejects the `cache_control` field. +## Streaming + +`streaming` (default `true`) makes model output arrive token by token instead of appearing when +the turn is finished. `maskshift run` writes tokens straight to stdout, and the interface shows +the answer being written where it previously showed a spinner. Every provider type streams: +Server-Sent Events for `openai-responses`, `openai-compatible`, `anthropic` and `gemini`, +newline-delimited JSON for `ollama`. + +```json +{ + "streaming": false, + "providers": [ + { "id": "lab-ollama", "streaming": false } + ] +} +``` + +Set the top-level key to disable it everywhere, or `streaming: false` on one provider entry to +disable it for that endpoint alone. Streaming is skipped automatically in two cases: when a +model is being driven through the [text tool protocol](#providers) — the tool calls live +inside the prose and are rewritten once the turn completes — and when an endpoint accepts the +streaming request but answers with a single JSON body anyway, which MaskShift detects from the +response content type and parses normally rather than reporting an empty turn. + +`--json` output is unaffected: it is written once, when the run completes. + +## Retries + +`providerRetry` governs how a model request recovers from a transient failure. Without it a +single 429 or a dropped connection ends a run that may already have spent real money. + +| Field | Default | Meaning | +|---|---:|---| +| `providerRetry.attempts` | `3` | Total attempts, including the first. `1` disables retrying. | +| `providerRetry.baseMs` | `500` | First backoff window. Doubles per attempt. | +| `providerRetry.maxMs` | `30000` | Ceiling for a single backoff, and for a `Retry-After` the server asks for. | + +Retries apply to HTTP 408, 409, 425, 429, 500, 502, 503 and 504, and to network-level failures. +A `Retry-After` header is honoured when present; otherwise the wait is exponential with jitter +so parallel agents do not resynchronise on the same retry. Status codes that mean the request +itself is wrong — 400, 401, 403, 404, 422 — are never retried, and cancelling a run stops the +backoff immediately rather than waiting it out. + +A streaming request is only retried before its first token reaches the caller, so nothing is +ever shown twice. Model-listing probes (`maskshift doctor`, provider discovery) stay +single-shot so a dead endpoint is reported promptly instead of after three waits. + +Override per provider with a `retry` object on the provider entry: + +```json +{ + "providerRetry": { "attempts": 5, "baseMs": 1000 }, + "providers": [ + { "id": "flaky-proxy", "retry": { "attempts": 8, "maxMs": 60000 } } + ] +} +``` + ## Memory ranking `memory` controls how `memory_search`/`memory_list` rank and age persistent memories. diff --git a/docs/SKILLS.md b/docs/SKILLS.md index 0bf8e02..c06e102 100644 --- a/docs/SKILLS.md +++ b/docs/SKILLS.md @@ -1,27 +1,6 @@ # Bundled Skills -MaskShift ships with **44 skills**. Descriptions are indexed at startup; full skill bodies are loaded only after activation. - -## Where skills come from - -A skill is a directory containing a `SKILL.md` with YAML front matter and operational -instructions. MaskShift scans, in order: - -```text -/skills -~/.maskshift/skills -/.maskshift/skills -/.agents/skills -/.claude/skills -~/.codex/skills -~/.claude/skills -~/.copilot/skills -``` - -Skills written for Claude, Codex and Copilot therefore load without conversion. The agent can -also create and improve skills itself through the `skill_*` native tools. - -## Bundled +MaskShift ships with **36 skills**. Descriptions are indexed at startup; full skill bodies are loaded only after activation. | Skill | Description | Source | |---|---|---| @@ -39,19 +18,14 @@ also create and improve skills itself through the `skill_*` native tools. | `dependency-upgrade` | Upgrade libraries and runtimes with compatibility research, focused changes, lockfile integrity, and regression verification. | `skills/dependency-upgrade` | | `docker-kubernetes` | Containerize and operate development services with reproducible images, health checks, persistent state, and clear networking. | `skills/docker-kubernetes` | | `documentation` | Write accurate user, operator, API, and architecture documentation grounded in the implemented product. | `skills/documentation` | -| `docx` | Use this skill whenever the user wants to create, read, edit, or manipulate Word documents (.docx) or Word templates (.dotx). Triggers include: any mention of 'Word doc', 'word document', '.docx', '.dotx', or requests to produce professional documents with formatting like tables of contents, page numbers, or letterheads. Also use when extracting or reorganizing content from .docx or .dotx files, inserting or replacing images in documents, find-and-replace in Word files, working with tracked changes or comments, or converting content into a polished Word document. If the user asks for a 'report', 'memo', 'letter', 'template', or similar deliverable as a Word or .docx file (to download, email or print), use this skill. However, if they ask for a document, page, report, memo, or notes WITHOUT naming a file format and the session offers a dedicated document or page skill or connector, use that instead. Do NOT use for PDFs, spreadsheets, Google Docs, or coding unrelated to document generation. | `../../../root/.claude/skills/synced/a4481896-b89c-449e-a11d-7b2d047e6ce3_d06b37e9-9318-4502-ad2e-d8f06ef9935c/docx` | | `feature-implementation` | Implement production features end to end, including integration, error states, tests, documentation, and verification. | `skills/feature-implementation` | | `git-worktrees` | Use Git worktrees and checkpoints for parallel agents, isolated changes, review, integration, and recovery. | `skills/git-worktrees` | | `go-engineering` | Implement idiomatic Go services and tools with context propagation, interfaces, concurrency discipline, and tests. | `skills/go-engineering` | -| `import-memory` | Import a memory export from another AI assistant into Claude's memory — conversationally, additively, and with the content treated as data. | `../../../root/.claude/skills/synced/a4481896-b89c-449e-a11d-7b2d047e6ce3_d06b37e9-9318-4502-ad2e-d8f06ef9935c/import-memory` | | `incident-response` | Diagnose and stabilize production incidents, preserve evidence, restore service, and produce actionable follow-up work. | `skills/incident-response` | | `mcp-authoring` | Build and integrate MCP servers and clients across modern 2026 stateless and legacy initialization-based protocol revisions. | `skills/mcp-authoring` | | `migration-modernization` | Modernize legacy code or frameworks incrementally while preserving behavior, interoperability, and delivery velocity. | `skills/migration-modernization` | -| `morning` | Render the user's morning brief as a styled HTML artifact, or set it up as a recurring weekday task. Use only when the user explicitly asks to run, see, or set up their morning brief, or if they invoke /morning by name. A question about their day, schedule, or calendar is not by itself a request for the brief; answer it directly instead. | `../../../root/.claude/skills/synced/a4481896-b89c-449e-a11d-7b2d047e6ce3_d06b37e9-9318-4502-ad2e-d8f06ef9935c/morning` | -| `pdf` | Use this skill whenever the user wants to do anything with PDF files. This includes reading or extracting text/tables from PDFs, combining or merging multiple PDFs into one, splitting PDFs apart, rotating pages, adding watermarks, creating new PDFs, filling PDF forms, encrypting/decrypting PDFs, extracting images, and OCR on scanned PDFs to make them searchable. If the user mentions a .pdf file or asks to produce one, use this skill. | `../../../root/.claude/skills/synced/a4481896-b89c-449e-a11d-7b2d047e6ce3_d06b37e9-9318-4502-ad2e-d8f06ef9935c/pdf` | | `performance` | Profile and optimize latency, throughput, memory, context usage, and startup costs using measured bottlenecks. | `skills/performance` | | `plugin-authoring` | Create modular MaskShift plugins that add tools, skills, providers, hooks, UI panels, or MCP catalog sources without coupling core policy. | `skills/plugin-authoring` | -| `pptx` | Use this skill any time a .pptx or .potx file is involved in any way — as input, output, or both. This includes: creating slide decks, pitch decks, or presentations as PowerPoint (.pptx) files; reading, parsing, or extracting text from any .pptx or .potx file (even if the extracted content will be used elsewhere, like in an email, summary, or creating a different type of slide deck); editing, modifying, or updating existing presentations; combining or splitting slide files; working with templates (.potx), layouts, speaker notes, or comments. Trigger whenever the user asks for a PowerPoint or .pptx file, or references a .pptx or .potx filename, regardless of what they plan to do with the content afterward. However, when the user asks for a deck, slides, a slide deck, or a presentation without naming a file format, default to using a dedicated slide-deck artifact type or a separate slides skill if this session offers one; otherwise, use this skill. | `../../../root/.claude/skills/synced/a4481896-b89c-449e-a11d-7b2d047e6ce3_d06b37e9-9318-4502-ad2e-d8f06ef9935c/pptx` | | `prompt-evals` | Create evaluations for agent prompts, tool routing, context retrieval, and coding outcomes using reproducible tasks and scoring. | `skills/prompt-evals` | | `python-engineering` | Build maintainable Python applications and automation with typed boundaries, packaging, environments, tests, and robust subprocess handling. | `skills/python-engineering` | | `refactoring` | Restructure code safely while preserving behavior, reducing duplication, clarifying ownership, and maintaining compatibility. | `skills/refactoring` | @@ -61,13 +35,10 @@ also create and improve skills itself through the `skill_*` native tools. | `scientific-computing` | Implement numerically sound scientific and engineering software with units, validation, reproducibility, and performance awareness. | `skills/scientific-computing` | | `security-audit` | Audit code and dependencies for concrete exploitable weaknesses while preserving MaskShift’s intentionally permissive local execution philosophy. | `skills/security-audit` | | `self-improving-skills` | Capture durable, non-obvious workflow knowledge as reusable skills and improve it from verified outcomes. | `skills/self-improving-skills` | -| `skill-creator` | Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | `../../../root/.claude/skills/synced/a4481896-b89c-449e-a11d-7b2d047e6ce3_d06b37e9-9318-4502-ad2e-d8f06ef9935c/skill-creator` | -| `startup-hook-skill` | Creating and developing startup hooks for Claude Code on the web. Use when the user wants to set up a repository for Claude Code on the web, create a SessionStart hook to ensure their project can run tests and linters during web sessions. | `../../../root/.claude/skills/session-start-hook` | | `terminal-phantom-ui` | Build MaskShift terminal interfaces in the Phantom Protocol design language — tokens over hex values, one marker gutter per pane, one filled chip per screen, and the ANSI width discipline that keeps a TUI aligned. | `skills/terminal-phantom-ui` | | `test-engineering` | Design and implement high-signal unit, integration, end-to-end, property, and regression tests for changed behavior. | `skills/test-engineering` | | `typescript-engineering` | Build strict TypeScript applications with stable runtime validation, modular architecture, and browser/server correctness. | `skills/typescript-engineering` | | `ui-visual-qa` | Inspect web user interfaces for clipping, overlap, responsiveness, focus, loading, empty, error, and interaction-state defects. | `skills/ui-visual-qa` | | `web-research` | Research changing technical facts from primary sources and convert findings into implementation decisions with traceability. | `skills/web-research` | -| `xlsx` | Use this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like \"the xlsx in my downloads\") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | `../../../root/.claude/skills/synced/a4481896-b89c-449e-a11d-7b2d047e6ce3_d06b37e9-9318-4502-ad2e-d8f06ef9935c/xlsx` | Workspace and user skill directories can extend this catalog without modifying the core distribution. diff --git a/docs/TOOLS.md b/docs/TOOLS.md index 320c9f7..2ad8de9 100644 --- a/docs/TOOLS.md +++ b/docs/TOOLS.md @@ -1,6 +1,6 @@ # Native Tool Inventory -Generated from the MaskShift 1.0.0 runtime. **149 native tools** are available before plugins or MCP servers add more capabilities. +Generated from the MaskShift 1.0.1 runtime. **149 native tools** are available before plugins or MCP servers add more capabilities. Only activated descriptors enter a model request; this document is the complete local catalog. diff --git a/maskshift.config.example.json b/maskshift.config.example.json index c13745a..1c341ad 100644 --- a/maskshift.config.example.json +++ b/maskshift.config.example.json @@ -39,6 +39,12 @@ ".github/copilot-instructions.md", ".cursorrules" ], + "streaming": true, + "providerRetry": { + "attempts": 3, + "baseMs": 500, + "maxMs": 30000 + }, "providers": [ { "id": "ollama", diff --git a/package.json b/package.json index 50f2dd5..066ab3c 100644 --- a/package.json +++ b/package.json @@ -27,10 +27,11 @@ "scripts": { "start": "node --no-warnings ./bin/maskshift.mjs", "dev": "node --no-warnings --watch ./bin/maskshift.mjs", + "deps": "node --no-warnings ./scripts/assert-no-dependencies.mjs", "check": "node --no-warnings ./scripts/check.mjs", "test": "node --no-warnings --test ./tests/*.test.mjs", "smoke": "node --no-warnings ./scripts/smoke-test.mjs", - "verify": "npm run check && npm test && npm run smoke", + "verify": "npm run deps && npm run check && npm test && npm run smoke", "docs": "node --no-warnings ./scripts/generate-docs.mjs", "capture": "node --no-warnings ./scripts/capture-tui.mjs", "doctor": "node --no-warnings ./bin/maskshift.mjs doctor", diff --git a/scripts/assert-no-dependencies.mjs b/scripts/assert-no-dependencies.mjs new file mode 100644 index 0000000..556ad19 --- /dev/null +++ b/scripts/assert-no-dependencies.mjs @@ -0,0 +1,74 @@ +// MaskShift's central claim is that it runs on Node's built-ins alone. That is easy to state +// and easy to break: one `import { x } from 'some-package'` merged into src/ turns the claim +// into a lie long before anyone notices a missing node_modules. This asserts it mechanically. +// +// Two checks: +// 1. package.json declares no runtime dependencies. +// 2. no module under src/, bin/, scripts/ or tests/ imports a bare specifier — every import +// is either `node:*` or a relative path. + +import fsp from 'node:fs/promises'; +import path from 'node:path'; +import { createRequire } from 'node:module'; +import { fileURLToPath } from 'node:url'; + +const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..'); +const SCANNED = ['src', 'bin', 'scripts', 'tests']; +const RUNTIME_FIELDS = ['dependencies', 'peerDependencies', 'optionalDependencies']; + +// Matched against statement positions rather than anywhere in the text: a specifier-shaped +// string inside a comment or an argument list is not an import, and scanning raw source for +// quotes reports those as dependencies. +// +// Negated classes match newlines, so a multi-line named import is covered by the first rule. +const PATTERNS = [ + // import … from 'x' / export … from 'x' + /^[ \t]*(?:import|export)\b[^'"]*?\bfrom\s*['"]([^'"]+)['"]/gm, + // import 'x' (side-effect only) + /^[ \t]*import\s*['"]([^'"]+)['"]/gm, + // import('x') / require('x'), skipping comment lines and member calls like foo.import(…) + /^(?![ \t]*(?:\/\/|\*))[^\n]*?(? ` - ${line}`).join('\n')}\n\n`); + process.stderr.write('MaskShift runs on Node built-ins only. Use a node: module or vendor the code.\n'); + process.exit(1); +} + +const devCount = Object.keys(manifest.devDependencies || {}).length; +console.log(`MaskShift dependency check PASS (${scanned} modules, 0 runtime dependencies, ${devCount} dev dependencies)`); diff --git a/scripts/generate-docs.mjs b/scripts/generate-docs.mjs index 435d73c..88bf347 100644 --- a/scripts/generate-docs.mjs +++ b/scripts/generate-docs.mjs @@ -3,15 +3,33 @@ import os from 'node:os'; import path from 'node:path'; import { fileURLToPath } from 'node:url'; import { createRuntime } from '../src/runtime.mjs'; +import { VERSION } from '../src/core/utils.mjs'; const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..'); +const bundledSkillsDir = path.join(root, 'skills'); const home = await fsp.mkdtemp(path.join(os.tmpdir(), 'maskshift-docs-')); -const runtime = await createRuntime({ workspacePath: root, configOverrides: { home, autoIndex: false } }); +// skillsDirs defaults include ~/.claude/skills and ~/.codex/skills, so a plain runtime here +// documents whatever the person running this happens to have installed — and writes their +// home directory into the published table. Scan the bundled directory and nothing else. +const runtime = await createRuntime({ + workspacePath: root, + configOverrides: { home, autoIndex: false, skillsDirs: [bundledSkillsDir] }, +}); function cell(value) { return String(value ?? '').replaceAll('|', '\\|').replaceAll('\n', ' '); } +// Nothing outside the repository may reach the generated docs. +function repoRelative(value, label) { + if (!value) return value; + const relative = path.relative(root, value); + if (relative.startsWith('..') || path.isAbsolute(relative)) { + throw new Error(`Refusing to document a path outside the repository (${label}): ${value}`); + } + return relative.replaceAll(path.sep, '/'); +} + try { const tools = runtime.toolRegistry.list({ includeSchema: false }); const grouped = new Map(); @@ -22,7 +40,7 @@ try { } const toolLines = [ '# Native Tool Inventory', '', - `Generated from the MaskShift 1.0.0 runtime. **${tools.length} native tools** are available before plugins or MCP servers add more capabilities.`, '', + `Generated from the MaskShift ${VERSION} runtime. **${tools.length} native tools** are available before plugins or MCP servers add more capabilities.`, '', 'Only activated descriptors enter a model request; this document is the complete local catalog.', '', ]; for (const [category, values] of [...grouped].sort(([a], [b]) => a.localeCompare(b))) { @@ -34,14 +52,17 @@ try { } await fsp.writeFile(path.join(root, 'docs', 'TOOLS.md'), `${toolLines.join('\n')}\n`); - const skills = runtime.skillManager.list(); + // Belt and braces: the config override above should make this a no-op, but a skill picked + // up from anywhere else must never be published as one MaskShift ships. + const skills = runtime.skillManager.list() + .filter((skill) => !skill.path || !path.relative(bundledSkillsDir, skill.path).startsWith('..')); const skillLines = [ '# Bundled Skills', '', `MaskShift ships with **${skills.length} skills**. Descriptions are indexed at startup; full skill bodies are loaded only after activation.`, '', '| Skill | Description | Source |', '|---|---|---|', ]; for (const skill of skills.sort((a, b) => a.name.localeCompare(b.name))) { - const relative = skill.path ? path.relative(root, skill.path).replaceAll(path.sep, '/') : skill.source; + const relative = skill.path ? repoRelative(skill.path, skill.name) : skill.source; skillLines.push(`| \`${cell(skill.name)}\` | ${cell(skill.description)} | \`${cell(relative)}\` |`); } skillLines.push('', 'Workspace and user skill directories can extend this catalog without modifying the core distribution.', ''); @@ -60,15 +81,20 @@ try { if (value && typeof value === 'object') return Object.fromEntries(Object.entries(value).map(([key, child]) => [key, portable(child)])); return portablePath(value); }; - const portableSkills = skills.map((skill) => ({ + // updatedAt is the SKILL.md mtime, which is checkout time on a fresh clone — it says + // nothing about the skill and makes the manifest differ on every machine. Dropped here + // for the same reason generatedAt is: this file is committed and diffed by CI. + const portableSkills = skills.map(({ updatedAt, ...skill }) => ({ ...skill, - path: skill.path ? path.relative(root, skill.path).replaceAll(path.sep, '/') : skill.source, - file: skill.file ? path.relative(root, skill.file).replaceAll(path.sep, '/') : undefined, + path: skill.path ? repoRelative(skill.path, skill.name) : skill.source, + file: skill.file ? repoRelative(skill.file, skill.name) : undefined, })); const mcpServers = portable(runtime.mcpManager.listServers()); const manifest = { - generatedAt: new Date().toISOString(), - version: '1.0.0', + // No generation timestamp: the manifest is checked into the repository and CI verifies it + // matches the code by regenerating and diffing. A field that changes on every run makes + // that check impossible, and says nothing the commit history does not already record. + version: VERSION, nativeToolCount: tools.length, bundledSkillCount: skills.length, curatedMcpCount: mcpServers.length, diff --git a/src/agent/engine.mjs b/src/agent/engine.mjs index e1ade98..f456114 100644 --- a/src/agent/engine.mjs +++ b/src/agent/engine.mjs @@ -215,6 +215,13 @@ export class AgentEngine { const system = this.promptBuilder.system({ workspaceContext, capabilityState, planState: entry.planState, run: currentRun, session: currentSession }); const tools = await this.capabilityController.descriptors(capabilityState); this.#event(run.id, 'model-turn', { step, tools: tools.map((tool) => tool.name), skillCount: capabilityState.skills.size }, scope); + // Token deltas go straight to the bus, never through #event: that persists a row per + // call, and a run streams thousands of tokens. + let streamedChars = 0; + const onDelta = entry.options.stream === false ? null : (text) => { + streamedChars += text.length; + this.eventBus.emit('run.assistant-delta', { step, text }, scope); + }; const response = await this.providerManager.complete({ modelRef: currentRun.model_id, messages: [{ role: 'system', content: system.text, blocks: system.blocks }, ...history], @@ -222,7 +229,9 @@ export class AgentEngine { signal, temperature: entry.options.temperature ?? 0.1, maxTokens: entry.options.maxTokens || 16_384, + onDelta, }); + if (streamedChars) this.eventBus.emit('run.assistant-delta-end', { step }, scope); usage.push(response.usage); costs.push(estimateUsageCost(this.config.get(), response.providerId, response.providerType, response.model, response.usage)); const assistantMessage = { @@ -233,7 +242,13 @@ export class AgentEngine { sessionId: session.id, role: 'assistant', content: response.content || '', meta: { runId: run.id, modelRef: response.modelRef, toolCalls: response.toolCalls || [], finishReason: response.finishReason, usage: response.usage }, }); - this.#event(run.id, 'assistant', { content: response.content || '', toolCalls: response.toolCalls || [], modelRef: response.modelRef, usage: response.usage }, scope); + this.#event(run.id, 'assistant', { + content: response.content || '', toolCalls: response.toolCalls || [], + modelRef: response.modelRef, usage: response.usage, + // Salvage mode rewrites the content after the fact, so what streamed is not what + // this message carries; the renderer must print it again in that case. + streamed: streamedChars > 0 && response.toolProtocol !== 'text-salvage', + }, scope); if (response.content) finalContent = response.content; // A text-protocol model wrote something call-shaped that would not parse. Correct it @@ -261,7 +276,7 @@ export class AgentEngine { this.#event(run.id, 'completed', { final: finalContent, steps: step, capabilities: meta.capabilities }, scope); await this.hooks?.run('Stop', { ...scope, workspacePath, status: 'completed', final: finalContent }); await this.hooks?.run('RunCompleted', { ...scope, workspacePath, status: 'completed', final: finalContent }); - if (run.workspace_id && this.config.get().autoIndex) void this.indexer.index(run.workspace_id, { force: true }).catch(() => {}); + if (run.workspace_id && this.config.get().autoIndex) void this.indexer.index(run.workspace_id).catch(() => {}); return completed; } diff --git a/src/agent/providers.mjs b/src/agent/providers.mjs index a3fba12..e20c0b1 100644 --- a/src/agent/providers.mjs +++ b/src/agent/providers.mjs @@ -31,27 +31,412 @@ function combineSignals(signal, timeoutMs) { }; } -async function fetchJson(url, options, { signal, timeoutMs = 180_000 } = {}) { - const combined = combineSignals(signal, timeoutMs); - let response; - try { - response = await fetch(url, { ...options, signal: combined.signal }); - } catch (error) { - combined.cleanup(); - throw new Error(`Model request failed: ${error.message}`); - } - combined.cleanup(); - const text = await response.text(); - const data = safeJsonParse(text, null); - if (!response.ok) { - const message = data?.error?.message || data?.message || truncate(text, 4000) || `HTTP ${response.status}`; - const error = new Error(`${response.status} ${response.statusText}: ${message}`); - error.status = response.status; - error.data = data; - throw error; - } - if (!data) throw new Error(`Provider returned invalid JSON: ${truncate(text, 2000)}`); - return data; +// Statuses worth a second attempt. 4xx that mean "your request is wrong" (400, 401, 403, +// 404, 422) are excluded deliberately: retrying them burns time and never succeeds, and the +// tool-protocol downgrade in #complete depends on a 400 surfacing immediately. +const RETRY_STATUS = new Set([408, 409, 425, 429, 500, 502, 503, 504]); +const DEFAULT_RETRY = { attempts: 3, baseMs: 500, maxMs: 30_000 }; + +function retryAfterMs(response) { + const header = response?.headers?.get?.('retry-after'); + if (!header) return null; + const seconds = Number(header); + if (Number.isFinite(seconds)) return Math.max(0, seconds * 1000); + const date = Date.parse(header); + return Number.isFinite(date) ? Math.max(0, date - Date.now()) : null; +} + +// Full jitter over the top half of the window: enough spread to break up a thundering herd +// without ever collapsing the wait to nearly zero. +function backoffMs(attempt, settings, suggestedMs) { + if (Number.isFinite(suggestedMs)) return Math.min(suggestedMs, settings.maxMs); + const ceiling = Math.min(settings.maxMs, settings.baseMs * 2 ** (attempt - 1)); + return Math.round(ceiling * (0.5 + Math.random() / 2)); +} + +// Not unref'd: a run waiting out a backoff is real work, and the process must not exit under it. +function sleep(ms, signal) { + return new Promise((resolve, reject) => { + if (signal?.aborted) { reject(signal.reason || new Error('Aborted')); return; } + const onAbort = () => { clearTimeout(timer); reject(signal.reason || new Error('Aborted')); }; + const timer = setTimeout(() => { signal?.removeEventListener('abort', onAbort); resolve(); }, ms); + signal?.addEventListener('abort', onAbort, { once: true }); + }); +} + +async function fetchJson(url, options, { signal, timeoutMs = 180_000, retry = null, onRetry = null } = {}) { + const settings = { ...DEFAULT_RETRY, ...(retry || {}) }; + // Callers opt in. Probes (model listings, health checks) keep single-shot behaviour. + const attempts = retry ? Math.max(1, Number(settings.attempts) || 1) : 1; + + for (let attempt = 1; ; attempt += 1) { + const combined = combineSignals(signal, timeoutMs); + let response = null; + let text = ''; + let transportError = null; + try { + response = await fetch(url, { ...options, signal: combined.signal }); + text = await response.text(); + } catch (error) { + transportError = error; + } finally { + combined.cleanup(); + } + + // The run was cancelled. Surface that, never retry into it. + if (signal?.aborted) throw signal.reason instanceof Error ? signal.reason : new Error('Model request aborted'); + + if (transportError) { + const error = new Error(`Model request failed: ${transportError.message}`); + if (attempt >= attempts) throw error; + const waitMs = backoffMs(attempt, settings, null); + onRetry?.({ attempt, attempts, waitMs, reason: error.message, status: null }); + await sleep(waitMs, signal); + continue; + } + + const data = safeJsonParse(text, null); + if (!response.ok) { + const message = data?.error?.message || data?.message || truncate(text, 4000) || `HTTP ${response.status}`; + const error = new Error(`${response.status} ${response.statusText}: ${message}`); + error.status = response.status; + error.data = data; + if (attempt >= attempts || !RETRY_STATUS.has(response.status)) throw error; + const waitMs = backoffMs(attempt, settings, retryAfterMs(response)); + onRetry?.({ attempt, attempts, waitMs, reason: truncate(error.message, 300), status: response.status }); + await sleep(waitMs, signal); + continue; + } + if (!data) throw new Error(`Provider returned invalid JSON: ${truncate(text, 2000)}`); + return data; + } +} + +// ------------------------------------------------------------------- streaming +// +// Providers stream over SSE (OpenAI, Anthropic, Gemini) or newline-delimited JSON (Ollama). +// Both framings are a few lines over the fetch body, so streaming costs no dependency. + +async function* decodedChunks(body, touch) { + const decoder = new TextDecoder(); + for await (const chunk of body) { + touch?.(); + const text = decoder.decode(chunk, { stream: true }); + if (text) yield text; + } + const tail = decoder.decode(); + if (tail) yield tail; +} + +async function* sseData(body, touch) { + let buffer = ''; + for await (const text of decodedChunks(body, touch)) { + buffer += text.replaceAll('\r\n', '\n'); + let index; + while ((index = buffer.indexOf('\n\n')) >= 0) { + const frame = buffer.slice(0, index); + buffer = buffer.slice(index + 2); + // Per the SSE spec a frame may carry several data: lines; providers send one JSON + // object, but joining keeps a pretty-printed payload intact. + const data = frame.split('\n').filter((line) => line.startsWith('data:')) + .map((line) => line.slice(5).trim()).join('\n'); + if (data && data !== '[DONE]') yield data; + } + } +} + +async function* ndjsonData(body, touch) { + let buffer = ''; + for await (const text of decodedChunks(body, touch)) { + buffer += text.replaceAll('\r\n', '\n'); + let index; + while ((index = buffer.indexOf('\n')) >= 0) { + const line = buffer.slice(0, index).trim(); + buffer = buffer.slice(index + 1); + if (line) yield line; + } + } + if (buffer.trim()) yield buffer.trim(); +} + +// Open a streaming request. Unlike fetchJson the timeout is an *idle* watchdog reset on every +// chunk, not a total budget: a long generation is normal, a silent socket is not. Retrying is +// only sound before the first token has been handed to the caller, so this retries while +// establishing the response and hands the body over untouched once headers are good. +async function openStream(url, options, { signal, timeoutMs = 300_000, retry = null, onRetry = null } = {}) { + const settings = { ...DEFAULT_RETRY, ...(retry || {}) }; + const attempts = retry ? Math.max(1, Number(settings.attempts) || 1) : 1; + + for (let attempt = 1; ; attempt += 1) { + const controller = new AbortController(); + const forward = () => controller.abort(signal.reason || new Error('Aborted')); + signal?.addEventListener('abort', forward, { once: true }); + let timer = null; + const touch = () => { + clearTimeout(timer); + timer = setTimeout(() => controller.abort(new Error(`Provider stream stalled for ${timeoutMs} ms`)), timeoutMs); + timer.unref(); + }; + const release = () => { clearTimeout(timer); signal?.removeEventListener('abort', forward); }; + touch(); + + let response = null; + let transportError = null; + try { + response = await fetch(url, { ...options, signal: controller.signal }); + } catch (error) { transportError = error; } + + if (signal?.aborted) { + release(); + throw signal.reason instanceof Error ? signal.reason : new Error('Model request aborted'); + } + if (!transportError && response.ok && response.body) { + // Not every endpoint honours `stream: true` — proxies and older local servers answer + // with one plain JSON body. Reading that as a stream finds zero frames and silently + // yields an empty turn, so detect it and hand the body back for the unary parser. + const contentType = response.headers.get('content-type') || ''; + if (!/event-stream|ndjson|jsonl/i.test(contentType)) { + const text = await response.text().catch(() => ''); + release(); + const data = safeJsonParse(text, null); + if (!data) throw new Error(`Provider ignored the streaming request and returned a non-JSON body: ${truncate(text, 2000)}`); + return { unary: data, release: () => {} }; + } + return { response, touch, release }; + } + + let error; + if (transportError) { + error = new Error(`Model request failed: ${transportError.message}`); + } else if (!response.body) { + release(); + throw new Error(`Provider returned an empty stream (HTTP ${response.status})`); + } else { + const text = await response.text().catch(() => ''); + const data = safeJsonParse(text, null); + const message = data?.error?.message || data?.message || truncate(text, 4000) || `HTTP ${response.status}`; + error = new Error(`${response.status} ${response.statusText}: ${message}`); + error.status = response.status; + error.data = data; + } + const suggested = transportError ? null : retryAfterMs(response); + release(); + + const retryable = transportError ? true : RETRY_STATUS.has(error.status); + if (attempt >= attempts || !retryable) throw error; + const waitMs = backoffMs(attempt, settings, suggested); + onRetry?.({ attempt, attempts, waitMs, reason: truncate(error.message, 300), status: error.status ?? null }); + await sleep(waitMs, signal); + } +} + +// Tool calls arrive as fragments keyed by index; concatenate then parse once at the end. +function assembleStreamedCalls(fragments) { + return [...fragments.entries()] + .sort((a, b) => a[0] - b[0]) + .map(([, call], position) => normalizeToolCall({ id: call.id, name: call.name, arguments: call.args || '{}' }, position)) + .filter((call) => call.name); +} + +// Unary response parsers. Shared by the non-streaming path and by the streaming path when an +// endpoint ignores `stream: true`, so one provider quirk is only ever described in one place. +function parseOpenAiBody(data) { + const choice = data.choices?.[0] || {}; + const message = choice.message || {}; + return { + content: typeof message.content === 'string' ? message.content : (message.content || []).map((item) => item.text || '').join(''), + toolCalls: (message.tool_calls || []).map(normalizeToolCall).filter((call) => call.name), + finishReason: choice.finish_reason || null, + usage: data.usage || null, + }; +} + +function parseResponsesBody(data) { + const output = data.output || []; + const content = output.filter((item) => item.type === 'message') + .flatMap((item) => item.content || []) + .filter((item) => item.type === 'output_text' || typeof item.text === 'string') + .map((item) => item.text || '').join('\n'); + return { + content: content || data.output_text || '', + toolCalls: output.filter((item) => item.type === 'function_call').map((item, index) => normalizeToolCall({ + id: item.call_id || item.id, name: item.name, arguments: item.arguments, + }, index)).filter((call) => call.name), + finishReason: data.status || null, + usage: data.usage || null, + responseId: data.id || null, + }; +} + +function parseOllamaBody(data) { + const message = data.message || {}; + return { + content: message.content || '', + toolCalls: (message.tool_calls || []).map(normalizeToolCall).filter((call) => call.name), + finishReason: data.done_reason || (data.done ? 'stop' : null), + usage: { + input_tokens: data.prompt_eval_count, + output_tokens: data.eval_count, + total_duration_ns: data.total_duration, + }, + }; +} + +function parseAnthropicBody(data) { + const blocks = data.content || []; + return { + content: blocks.filter((block) => block.type === 'text').map((block) => block.text).join('\n'), + toolCalls: blocks.filter((block) => block.type === 'tool_use').map((block, index) => normalizeToolCall(block, index)), + finishReason: data.stop_reason || null, + usage: data.usage || null, + }; +} + +function parseGeminiBody(data) { + const candidate = data.candidates?.[0] || {}; + const parts = candidate.content?.parts || []; + return { + content: parts.filter((part) => typeof part.text === 'string').map((part) => part.text).join('\n'), + toolCalls: parts.filter((part) => part.functionCall).map((part, index) => normalizeToolCall(part, index)), + finishReason: candidate.finishReason || null, + usage: data.usageMetadata || null, + }; +} + +async function readOpenAiStream(response, touch, onDelta) { + let content = ''; + let finishReason = null; + let usage = null; + const fragments = new Map(); + for await (const data of sseData(response.body, touch)) { + const chunk = safeJsonParse(data, null); + if (!chunk) continue; + if (chunk.usage) usage = chunk.usage; + const choice = chunk.choices?.[0]; + if (!choice) continue; + if (choice.finish_reason) finishReason = choice.finish_reason; + const delta = choice.delta || {}; + const text = typeof delta.content === 'string' ? delta.content : (delta.content || []).map((part) => part.text || '').join(''); + if (text) { content += text; onDelta?.(text); } + for (const call of delta.tool_calls || []) { + const index = call.index ?? fragments.size; + const entry = fragments.get(index) || { id: call.id, name: '', args: '' }; + if (call.id) entry.id = call.id; + if (call.function?.name) entry.name += call.function.name; + if (call.function?.arguments) entry.args += call.function.arguments; + fragments.set(index, entry); + } + } + return { content, toolCalls: assembleStreamedCalls(fragments), finishReason, usage }; +} + +async function readResponsesStream(response, touch, onDelta) { + let content = ''; + let finishReason = null; + let usage = null; + let responseId = null; + const fragments = new Map(); + for await (const data of sseData(response.body, touch)) { + const event = safeJsonParse(data, null); + if (!event) continue; + const index = event.output_index ?? fragments.size; + switch (event.type) { + case 'response.output_text.delta': + if (event.delta) { content += event.delta; onDelta?.(event.delta); } + break; + case 'response.output_item.added': + if (event.item?.type === 'function_call') { + fragments.set(index, { id: event.item.call_id || event.item.id, name: event.item.name || '', args: '' }); + } + break; + case 'response.function_call_arguments.delta': { + const entry = fragments.get(index) || { id: event.item_id, name: '', args: '' }; + entry.args += event.delta || ''; + fragments.set(index, entry); + break; + } + case 'response.completed': + case 'response.incomplete': + case 'response.failed': + responseId = event.response?.id || responseId; + usage = event.response?.usage || usage; + finishReason = event.response?.status || finishReason; + break; + default: break; + } + } + return { content, toolCalls: assembleStreamedCalls(fragments), finishReason, usage, responseId }; +} + +async function readAnthropicStream(response, touch, onDelta) { + let content = ''; + let finishReason = null; + const usage = {}; + const fragments = new Map(); + for await (const data of sseData(response.body, touch)) { + const event = safeJsonParse(data, null); + if (!event) continue; + if (event.type === 'message_start') Object.assign(usage, event.message?.usage || {}); + else if (event.type === 'content_block_start' && event.content_block?.type === 'tool_use') { + fragments.set(event.index, { id: event.content_block.id, name: event.content_block.name || '', args: '' }); + } else if (event.type === 'content_block_delta') { + if (event.delta?.type === 'text_delta' && event.delta.text) { + content += event.delta.text; + onDelta?.(event.delta.text); + } else if (event.delta?.type === 'input_json_delta') { + const entry = fragments.get(event.index); + if (entry) entry.args += event.delta.partial_json || ''; + } + } else if (event.type === 'message_delta') { + finishReason = event.delta?.stop_reason || finishReason; + Object.assign(usage, event.usage || {}); + } else if (event.type === 'error') { + throw new Error(event.error?.message || 'Anthropic stream error'); + } + } + return { content, toolCalls: assembleStreamedCalls(fragments), finishReason, usage }; +} + +async function readOllamaStream(response, touch, onDelta) { + let content = ''; + let finishReason = null; + let usage = null; + const toolCalls = []; + for await (const line of ndjsonData(response.body, touch)) { + const chunk = safeJsonParse(line, null); + if (!chunk) continue; + if (chunk.error) throw new Error(String(chunk.error)); + const text = chunk.message?.content || ''; + if (text) { content += text; onDelta?.(text); } + for (const call of chunk.message?.tool_calls || []) { + toolCalls.push(normalizeToolCall(call, toolCalls.length)); + } + if (chunk.done) { + finishReason = chunk.done_reason || 'stop'; + usage = { input_tokens: chunk.prompt_eval_count, output_tokens: chunk.eval_count, total_duration_ns: chunk.total_duration }; + } + } + return { content, toolCalls: toolCalls.filter((call) => call.name), finishReason, usage }; +} + +async function readGeminiStream(response, touch, onDelta) { + let content = ''; + let finishReason = null; + let usage = null; + const toolCalls = []; + for await (const data of sseData(response.body, touch)) { + const chunk = safeJsonParse(data, null); + if (!chunk) continue; + if (chunk.usageMetadata) usage = chunk.usageMetadata; + const candidate = chunk.candidates?.[0]; + if (!candidate) continue; + if (candidate.finishReason) finishReason = candidate.finishReason; + for (const part of candidate.content?.parts || []) { + if (typeof part.text === 'string' && part.text) { content += part.text; onDelta?.(part.text); } + if (part.functionCall) toolCalls.push(normalizeToolCall(part, toolCalls.length)); + } + } + return { content, toolCalls: toolCalls.filter((call) => call.name), finishReason, usage }; } function normalizeToolCall(call, index = 0) { @@ -196,6 +581,27 @@ export class ProviderManager { return provider.apiKey || (provider.apiKeyEnv ? process.env[provider.apiKeyEnv] : undefined); } + // Streaming is on unless the config or the individual provider opts out. + streamingEnabled(provider) { + if (provider.streaming === false) return false; + return this.config.get().streaming !== false; + } + + // Shared request options for every completion call: per-attempt timeout, retry policy, and + // an event so a retried run is visible rather than looking like a stall. + requestOptions(provider, { signal, timeoutMs }) { + const settings = { ...DEFAULT_RETRY, ...(this.config.get().providerRetry || {}), ...(provider.retry || {}) }; + return { + signal, + timeoutMs: provider.timeoutMs || timeoutMs, + retry: settings, + onRetry: (info) => { + this.logger.warn('Retrying model request', { provider: provider.id, ...info }); + this.eventBus.emit('model.request.retrying', { provider: provider.id, ...info }); + }, + }; + } + isConfigured(provider) { if (!provider.enabled) return false; if (provider.type === 'ollama') return true; @@ -322,13 +728,16 @@ export class ProviderManager { const outbound = useText ? toTextProtocolMessages(messages, tools) : messages; const wireTools = useText ? [] : tools; const { signal, temperature, maxTokens } = options; + // In text-protocol mode the tool calls live inside the prose and parseToolCalls rewrites + // the content below, so streaming it would show the user call syntax that then vanishes. + const onDelta = useText ? null : options.onDelta; let result; - if (resolved.provider.type === 'anthropic') result = await this.#anthropic(resolved, outbound, wireTools, { signal, temperature, maxTokens }); - else if (resolved.provider.type === 'openai-responses') result = await this.#openAiResponses(resolved, outbound, wireTools, { signal, temperature, maxTokens }); - else if (resolved.provider.type === 'ollama') result = await this.#ollama(resolved, outbound, wireTools, { signal, temperature }); - else if (resolved.provider.type === 'gemini') result = await this.#gemini(resolved, outbound, wireTools, { signal, temperature, maxTokens }); - else result = await this.#openAiCompatible(resolved, outbound, wireTools, { signal, temperature, maxTokens }); + if (resolved.provider.type === 'anthropic') result = await this.#anthropic(resolved, outbound, wireTools, { signal, temperature, maxTokens, onDelta }); + else if (resolved.provider.type === 'openai-responses') result = await this.#openAiResponses(resolved, outbound, wireTools, { signal, temperature, maxTokens, onDelta }); + else if (resolved.provider.type === 'ollama') result = await this.#ollama(resolved, outbound, wireTools, { signal, temperature, onDelta }); + else if (resolved.provider.type === 'gemini') result = await this.#gemini(resolved, outbound, wireTools, { signal, temperature, maxTokens, onDelta }); + else result = await this.#openAiCompatible(resolved, outbound, wireTools, { signal, temperature, maxTokens, onDelta }); result.toolProtocol = useText ? 'text' : 'native'; result.parseErrors = []; @@ -360,7 +769,7 @@ export class ProviderManager { return result; } - async complete({ modelRef, messages, tools = [], signal, temperature = 0.1, maxTokens = 16_384 }) { + async complete({ modelRef, messages, tools = [], signal, temperature = 0.1, maxTokens = 16_384, onDelta = null }) { const resolved = await this.resolveModel(modelRef); const started = Date.now(); const protocol = this.toolProtocolFor(resolved); @@ -368,7 +777,7 @@ export class ProviderManager { try { let result; try { - result = await this.#dispatch(resolved, messages, tools, { signal, temperature, maxTokens }, protocol); + result = await this.#dispatch(resolved, messages, tools, { signal, temperature, maxTokens, onDelta }, protocol); } catch (error) { // The endpoint rejected the tool schema outright: remember it and re-run in text mode // rather than surfacing a dead end to the user. @@ -380,16 +789,18 @@ export class ProviderManager { this.eventBus.emit('model.tool-protocol.downgraded', { provider: resolved.provider.id, model: resolved.model, reason: truncate(error.message, 300), }); - result = await this.#dispatch(resolved, messages, tools, { signal, temperature, maxTokens }, 'text'); + result = await this.#dispatch(resolved, messages, tools, { signal, temperature, maxTokens, onDelta }, 'text'); } result.modelRef = resolved.ref; result.providerId = resolved.provider.id; result.providerType = resolved.provider.type; result.model = resolved.model; result.durationMs = Date.now() - started; + result.streamed = Boolean(onDelta) && result.toolProtocol !== 'text'; this.eventBus.emit('model.request.completed', { provider: resolved.provider.id, model: resolved.model, durationMs: result.durationMs, toolCalls: result.toolCalls.length, usage: result.usage, toolProtocol: result.toolProtocol, + streamed: result.streamed, }); return result; } catch (error) { @@ -398,7 +809,7 @@ export class ProviderManager { } } - async #openAiResponses(resolved, messages, tools, { signal, temperature, maxTokens }) { + async #openAiResponses(resolved, messages, tools, { signal, temperature, maxTokens, onDelta }) { const provider = resolved.provider; const headers = { 'Content-Type': 'application/json', ...provider.headers }; const key = this.apiKey(provider); @@ -413,27 +824,19 @@ export class ProviderManager { ...(tools.length ? { tools: toResponsesTools(tools), tool_choice: 'auto', parallel_tool_calls: true } : {}), ...provider.requestDefaults, }; - const data = await fetchJson(`${provider.baseUrl.replace(/\/$/, '')}/responses`, { + const url = `${provider.baseUrl.replace(/\/$/, '')}/responses`; + const requestOptions = this.requestOptions(provider, { signal, timeoutMs: 300_000 }); + if (onDelta && this.streamingEnabled(provider)) { + const stream = await openStream(url, { method: 'POST', headers, body: JSON.stringify({ ...body, stream: true }) }, requestOptions); + try { return stream.unary ? parseResponsesBody(stream.unary) : await readResponsesStream(stream.response, stream.touch, onDelta); } + finally { stream.release(); } + } + return parseResponsesBody(await fetchJson(url, { method: 'POST', headers, body: JSON.stringify(body), - }, { signal, timeoutMs: provider.timeoutMs || 300_000 }); - const output = data.output || []; - const content = output.filter((item) => item.type === 'message') - .flatMap((item) => item.content || []) - .filter((item) => item.type === 'output_text' || typeof item.text === 'string') - .map((item) => item.text || '').join('\n'); - const toolCalls = output.filter((item) => item.type === 'function_call').map((item, index) => normalizeToolCall({ - id: item.call_id || item.id, name: item.name, arguments: item.arguments, - }, index)).filter((call) => call.name); - return { - content: content || data.output_text || '', - toolCalls, - finishReason: data.status || null, - usage: data.usage || null, - responseId: data.id || null, - }; + }, requestOptions)); } - async #openAiCompatible(resolved, messages, tools, { signal, temperature, maxTokens }) { + async #openAiCompatible(resolved, messages, tools, { signal, temperature, maxTokens, onDelta }) { const provider = resolved.provider; const headers = { 'Content-Type': 'application/json', ...provider.headers }; const key = this.apiKey(provider); @@ -446,20 +849,20 @@ export class ProviderManager { ...(tools.length ? { tools: toOpenAiTools(tools), tool_choice: 'auto', parallel_tool_calls: true } : {}), ...provider.requestDefaults, }; - const data = await fetchJson(`${provider.baseUrl.replace(/\/$/, '')}/chat/completions`, { + const url = `${provider.baseUrl.replace(/\/$/, '')}/chat/completions`; + const requestOptions = this.requestOptions(provider, { signal, timeoutMs: 300_000 }); + if (onDelta && this.streamingEnabled(provider)) { + const streamBody = { ...body, stream: true, stream_options: { include_usage: true } }; + const stream = await openStream(url, { method: 'POST', headers, body: JSON.stringify(streamBody) }, requestOptions); + try { return stream.unary ? parseOpenAiBody(stream.unary) : await readOpenAiStream(stream.response, stream.touch, onDelta); } + finally { stream.release(); } + } + return parseOpenAiBody(await fetchJson(url, { method: 'POST', headers, body: JSON.stringify(body), - }, { signal, timeoutMs: provider.timeoutMs || 300_000 }); - const choice = data.choices?.[0] || {}; - const message = choice.message || {}; - return { - content: typeof message.content === 'string' ? message.content : (message.content || []).map((item) => item.text || '').join(''), - toolCalls: (message.tool_calls || []).map(normalizeToolCall).filter((call) => call.name), - finishReason: choice.finish_reason || null, - usage: data.usage || null, - }; + }, requestOptions)); } - async #ollama(resolved, messages, tools, { signal, temperature }) { + async #ollama(resolved, messages, tools, { signal, temperature, onDelta }) { const provider = resolved.provider; const body = { model: resolved.model, @@ -473,23 +876,19 @@ export class ProviderManager { ...(tools.length ? { tools: toOpenAiTools(tools) } : {}), }; const headers = { 'Content-Type': 'application/json', ...provider.headers }; - const data = await fetchJson(`${provider.baseUrl.replace(/\/$/, '')}/api/chat`, { + const url = `${provider.baseUrl.replace(/\/$/, '')}/api/chat`; + const requestOptions = this.requestOptions(provider, { signal, timeoutMs: 600_000 }); + if (onDelta && this.streamingEnabled(provider)) { + const stream = await openStream(url, { method: 'POST', headers, body: JSON.stringify({ ...body, stream: true }) }, requestOptions); + try { return stream.unary ? parseOllamaBody(stream.unary) : await readOllamaStream(stream.response, stream.touch, onDelta); } + finally { stream.release(); } + } + return parseOllamaBody(await fetchJson(url, { method: 'POST', headers, body: JSON.stringify(body), - }, { signal, timeoutMs: provider.timeoutMs || 600_000 }); - const message = data.message || {}; - return { - content: message.content || '', - toolCalls: (message.tool_calls || []).map(normalizeToolCall).filter((call) => call.name), - finishReason: data.done_reason || (data.done ? 'stop' : null), - usage: { - input_tokens: data.prompt_eval_count, - output_tokens: data.eval_count, - total_duration_ns: data.total_duration, - }, - }; + }, requestOptions)); } - async #anthropic(resolved, messages, tools, { signal, temperature, maxTokens }) { + async #anthropic(resolved, messages, tools, { signal, temperature, maxTokens, onDelta }) { const provider = resolved.provider; const cachingEnabled = provider.promptCaching !== false; const converted = mergeAnthropicMessages(messages); @@ -517,26 +916,25 @@ export class ProviderManager { ...(toolDefs.length ? { tools: toolDefs } : {}), ...provider.requestDefaults, }; - const data = await fetchJson(`${provider.baseUrl.replace(/\/$/, '')}/messages`, { - method: 'POST', - headers: { - 'Content-Type': 'application/json', - 'x-api-key': this.apiKey(provider), - 'anthropic-version': provider.anthropicVersion || '2023-06-01', - ...provider.headers, - }, - body: JSON.stringify(body), - }, { signal, timeoutMs: provider.timeoutMs || 300_000 }); - const blocks = data.content || []; - return { - content: blocks.filter((block) => block.type === 'text').map((block) => block.text).join('\n'), - toolCalls: blocks.filter((block) => block.type === 'tool_use').map((block, index) => normalizeToolCall(block, index)), - finishReason: data.stop_reason || null, - usage: data.usage || null, + const url = `${provider.baseUrl.replace(/\/$/, '')}/messages`; + const headers = { + 'Content-Type': 'application/json', + 'x-api-key': this.apiKey(provider), + 'anthropic-version': provider.anthropicVersion || '2023-06-01', + ...provider.headers, }; + const requestOptions = this.requestOptions(provider, { signal, timeoutMs: 300_000 }); + if (onDelta && this.streamingEnabled(provider)) { + const stream = await openStream(url, { method: 'POST', headers, body: JSON.stringify({ ...body, stream: true }) }, requestOptions); + try { return stream.unary ? parseAnthropicBody(stream.unary) : await readAnthropicStream(stream.response, stream.touch, onDelta); } + finally { stream.release(); } + } + return parseAnthropicBody(await fetchJson(url, { + method: 'POST', headers, body: JSON.stringify(body), + }, requestOptions)); } - async #gemini(resolved, messages, tools, { signal, temperature, maxTokens }) { + async #gemini(resolved, messages, tools, { signal, temperature, maxTokens, onDelta }) { const provider = resolved.provider; const converted = toGemini(messages); const body = { @@ -549,16 +947,17 @@ export class ProviderManager { parameters: tool.inputSchema || { type: 'object', properties: {} }, })) }] } : {}), }; - const data = await fetchJson(`${provider.baseUrl.replace(/\/$/, '')}/models/${encodeURIComponent(resolved.model)}:generateContent?key=${encodeURIComponent(this.apiKey(provider))}`, { - method: 'POST', headers: { 'Content-Type': 'application/json', ...provider.headers }, body: JSON.stringify(body), - }, { signal, timeoutMs: provider.timeoutMs || 300_000 }); - const candidate = data.candidates?.[0] || {}; - const parts = candidate.content?.parts || []; - return { - content: parts.filter((part) => typeof part.text === 'string').map((part) => part.text).join('\n'), - toolCalls: parts.filter((part) => part.functionCall).map((part, index) => normalizeToolCall(part, index)), - finishReason: candidate.finishReason || null, - usage: data.usageMetadata || null, - }; + const base = `${provider.baseUrl.replace(/\/$/, '')}/models/${encodeURIComponent(resolved.model)}`; + const key = encodeURIComponent(this.apiKey(provider)); + const headers = { 'Content-Type': 'application/json', ...provider.headers }; + const requestOptions = this.requestOptions(provider, { signal, timeoutMs: 300_000 }); + if (onDelta && this.streamingEnabled(provider)) { + const stream = await openStream(`${base}:streamGenerateContent?alt=sse&key=${key}`, { method: 'POST', headers, body: JSON.stringify(body) }, requestOptions); + try { return stream.unary ? parseGeminiBody(stream.unary) : await readGeminiStream(stream.response, stream.touch, onDelta); } + finally { stream.release(); } + } + return parseGeminiBody(await fetchJson(`${base}:generateContent?key=${key}`, { + method: 'POST', headers, body: JSON.stringify(body), + }, requestOptions)); } } diff --git a/src/browser/manager.mjs b/src/browser/manager.mjs index 3fd22dc..7ad73cd 100644 --- a/src/browser/manager.mjs +++ b/src/browser/manager.mjs @@ -1,3 +1,4 @@ +import fs from 'node:fs'; import fsp from 'node:fs/promises'; import net from 'node:net'; import os from 'node:os'; @@ -12,6 +13,33 @@ const BROWSER_COMMANDS = process.platform === 'darwin' ? ['chrome.exe', 'msedge.exe', 'chromium.exe'] : ['chromium', 'chromium-browser', 'google-chrome', 'google-chrome-stable', 'microsoft-edge']; +// Chromium's sandbox needs unprivileged user namespaces. Root cannot use it at all, and a +// distro can withhold it from everyone — Ubuntu 23.10+ restricts it through AppArmor, which +// is what a default GitHub Actions runner does. Inferring this from the uid alone (the +// previous rule) gets the non-root-but-restricted case wrong, and Chromium's answer there is +// to abort with "No usable sandbox!" rather than to degrade. +export function sandboxUnavailable({ + platform = process.platform, + uid = typeof process.getuid === 'function' ? process.getuid() : null, + readFile = (file) => fs.readFileSync(file, 'utf8'), +} = {}) { + if (platform !== 'linux') return false; + if (uid === null || uid === 0) return true; + const restrictions = [ + // 1 means AppArmor blocks unprivileged userns for unconfined programs. + ['/proc/sys/kernel/apparmor_restrict_unprivileged_userns', (value) => value === '1'], + // 0 means the kernel disallows them outright. + ['/proc/sys/kernel/unprivileged_userns_clone', (value) => value === '0'], + ['/proc/sys/user/max_user_namespaces', (value) => value === '0'], + ]; + for (const [file, blocked] of restrictions) { + try { + if (blocked(String(readFile(file)).trim())) return true; + } catch { /* an absent knob means the restriction is not in force */ } + } + return false; +} + async function freePort() { return new Promise((resolve, reject) => { const server = net.createServer(); @@ -100,6 +128,15 @@ export class BrowserManager { const base = this.config.get().browser?.profilesDir || path.join(this.config.get().home, 'browser', 'profiles'); const userDataDir = absolutePath(path.join(base, profileName)); await ensureDir(userDataDir); + // Dropping Chromium's sandbox is a real reduction in isolation, so say so rather than + // letting it happen quietly. The alternative on a restricted host is not a safer + // browser, it is Chromium refusing to start at all. + const noSandbox = sandboxUnavailable(); + if (noSandbox) { + this.logger?.warn('Launching Chromium without its sandbox: this host does not permit unprivileged user namespaces', { + platform: process.platform, uid: typeof process.getuid === 'function' ? process.getuid() : null, + }); + } const args = [ `--remote-debugging-port=${port}`, '--remote-debugging-address=127.0.0.1', @@ -108,7 +145,7 @@ export class BrowserManager { '--disable-component-update', '--disable-sync', '--disable-features=Translate,OptimizationHints', '--window-size=1440,1000', ...(headless ? ['--headless=new', '--hide-scrollbars'] : []), - ...(process.platform === 'linux' && (typeof process.getuid !== 'function' || process.getuid() === 0) ? ['--no-sandbox'] : []), + ...(noSandbox ? ['--no-sandbox'] : []), ...(this.config.get().browser?.args || []), ...(extraArgs || []), url || 'about:blank', ]; const child = spawn(this.executable, args, { stdio: ['ignore', 'ignore', 'pipe'], detached: process.platform !== 'win32' }); diff --git a/src/cli/commands.mjs b/src/cli/commands.mjs index 4a7ceb4..2e2bade 100644 --- a/src/cli/commands.mjs +++ b/src/cli/commands.mjs @@ -57,7 +57,7 @@ const workspaceCommands = { await runtime.skillManager.setWorkspace(workspace.path); runtime.pluginManager.workspacePath = workspace.path; await runtime.pluginManager.scan({ activate: true }); - if (args.index !== false && args['no-index'] !== true) await runtime.indexer.index(workspace.id, { force: true }); + if (args.index !== false && args['no-index'] !== true) await runtime.indexer.index(workspace.id); if (ui.emit(workspace)) return; ui.ok(`Target locked: ${workspace.name}`); ui.fields([['id', workspace.id], ['path', workspace.path], ['git', workspace.meta?.gitRoot || 'not a repository']]); @@ -141,11 +141,11 @@ const workspaceCommands = { }, }, index: { - usage: 'workspace index [--force]', + usage: 'workspace index', summary: 'Build or rebuild the local context index', async run(context) { const workspace = await resolveWorkspace(context); - const stats = await context.runtime.indexer.index(workspace.id, { force: context.args.force !== false }); + const stats = await context.runtime.indexer.index(workspace.id); if (context.ui.emit(stats)) return; context.ui.ok('Index rebuilt'); context.ui.fields(Object.entries(stats).map(([key, value]) => [key, typeof value === 'object' ? JSON.stringify(value) : value])); diff --git a/src/cli/main.mjs b/src/cli/main.mjs index 7be7b12..b5df0e7 100644 --- a/src/cli/main.mjs +++ b/src/cli/main.mjs @@ -128,6 +128,8 @@ async function headlessRun(runtime, ui, args, positional) { }); const quiet = Boolean(args.quiet || ui.json); + let streaming = false; + let lastStreamedTurn = ''; const unsubscribe = quiet ? () => {} : runtime.eventBus.subscribe((event) => { if (event.sessionId !== session.id) return; const payload = event.payload || {}; @@ -138,8 +140,19 @@ async function headlessRun(runtime, ui, args, positional) { case 'run.model-turn': ui.line(ui.theme.paint(`${ui.marks.dot} turn ${String(payload.step).padStart(2, '0')} — ${payload.tools?.length ?? 0} tools active`, { fg: ui.theme.roles.border })); break; + case 'run.assistant-delta': + // First token of the turn: open a blank line, then write through unstyled — markdown + // cannot be rendered a fragment at a time. + if (!streaming) { ui.line(); streaming = true; lastStreamedTurn = ''; } + lastStreamedTurn += payload.text; + ui.writeRaw(payload.text); + break; + case 'run.assistant-delta-end': + if (streaming) { ui.writeRaw('\n'); streaming = false; } + break; case 'run.assistant': - if (payload.content?.trim()) { ui.line(); ui.markdown(payload.content); } + // Skip content the user already watched arrive. + if (!payload.streamed && payload.content?.trim()) { ui.line(); ui.markdown(payload.content); } for (const call of payload.toolCalls || []) { ui.line(ui.theme.paint(` ${ui.marks.caret} ${call.name}`, { fg: ui.theme.roles.tool }) + ui.theme.paint(` ${oneLine(JSON.stringify(call.args ?? {}), ui.width - call.name.length - 8)}`, { fg: ui.theme.roles.border })); @@ -178,8 +191,9 @@ async function headlessRun(runtime, ui, args, positional) { cost: completed.meta?.costEstimate || null, }, null, 2)); } else { + const alreadyShown = final?.content?.trim() && final.content.trim() === lastStreamedTurn.trim(); ui.rule(); - if (final?.content) ui.markdown(final.content); + if (final?.content && !alreadyShown) ui.markdown(final.content); ui.line(); const tone = completed.status === 'completed' ? 'ok' : completed.status === 'cancelled' ? 'warn' : 'fail'; ui.status(tone, `run ${completed.status}${completed.error ? `: ${completed.error}` : ''}`); diff --git a/src/cli/ui.mjs b/src/cli/ui.mjs index 93b34fd..d65c934 100644 --- a/src/cli/ui.mjs +++ b/src/cli/ui.mjs @@ -24,6 +24,8 @@ export class Ui { } write(text = '') { this.stream.write(`${text}\n`); } + // Unterminated write, for streaming model output token by token. + writeRaw(text = '') { if (!this.json) this.stream.write(text); } writeError(text = '') { this.errorStream.write(`${text}\n`); } emit(value) { diff --git a/src/core/config.mjs b/src/core/config.mjs index f31e421..33f678c 100644 --- a/src/core/config.mjs +++ b/src/core/config.mjs @@ -153,6 +153,14 @@ export function defaultConfig() { '.github/copilot-instructions.md', '.cursorrules', ], + // Stream model output token by token. Per-provider `streaming: false` overrides this. + streaming: true, + // Transient-failure policy for model requests. Per-provider `retry` overrides this. + providerRetry: { + attempts: 3, + baseMs: 500, + maxMs: 30_000, + }, providers: envProviderDefaults(), mcpServers: {}, hooks: {}, @@ -187,6 +195,7 @@ function mergeConfig(base, override) { merged.automations = { ...base.automations, ...(override?.automations || {}) }; merged.browser = { ...base.browser, ...(override?.browser || {}) }; merged.indexing = { ...base.indexing, ...(override?.indexing || {}) }; + merged.providerRetry = { ...base.providerRetry, ...(override?.providerRetry || {}) }; merged.memory = { ...base.memory, ...(override?.memory || {}) }; merged.pricing = { ...base.pricing, ...(override?.pricing || {}), diff --git a/src/core/utils.mjs b/src/core/utils.mjs index f08a257..65562ae 100644 --- a/src/core/utils.mjs +++ b/src/core/utils.mjs @@ -1,11 +1,14 @@ import crypto from 'node:crypto'; +import { createRequire } from 'node:module'; import fs from 'node:fs'; import fsp from 'node:fs/promises'; import os from 'node:os'; import path from 'node:path'; import { spawn } from 'node:child_process'; -export const VERSION = '1.0.0'; +// Read from package.json rather than restated here: the two drifted (1.0.0 vs 1.0.1) and +// every banner and `--version` printed the stale one. +export const VERSION = createRequire(import.meta.url)('../../package.json').version; export function id(prefix = 'id') { return `${prefix}_${Date.now().toString(36)}_${crypto.randomBytes(6).toString('hex')}`; diff --git a/src/indexer/repository-indexer.mjs b/src/indexer/repository-indexer.mjs index c336eac..7a66a11 100644 --- a/src/indexer/repository-indexer.mjs +++ b/src/indexer/repository-indexer.mjs @@ -135,9 +135,11 @@ export class RepositoryIndexer { return chunks; } - async index(workspaceId, { force = false } = {}) { + // Every pass is a full rescan, so there is nothing for a `force` option to force. It used to + // be accepted here and dropped on the floor by #runIndex, which declares no second parameter. + async index(workspaceId) { if (this.running.has(workspaceId)) return this.running.get(workspaceId); - const task = this.#runIndex(workspaceId, { force }).finally(() => this.running.delete(workspaceId)); + const task = this.#runIndex(workspaceId).finally(() => this.running.delete(workspaceId)); this.running.set(workspaceId, task); return task; } diff --git a/src/lsp/client.mjs b/src/lsp/client.mjs index 4b35439..1d61a4a 100644 --- a/src/lsp/client.mjs +++ b/src/lsp/client.mjs @@ -2,7 +2,7 @@ import { spawn } from 'node:child_process'; import fsp from 'node:fs/promises'; import path from 'node:path'; import { pathToFileURL, fileURLToPath } from 'node:url'; -import { id, truncate } from '../core/utils.mjs'; +import { id, truncate, VERSION } from '../core/utils.mjs'; function languageId(file) { const ext = path.extname(file).toLowerCase(); @@ -59,7 +59,7 @@ export class LanguageServerClient { }); const initialized = await this.request('initialize', { processId: process.pid, - clientInfo: { name: 'MaskShift', version: '1.0.0' }, + clientInfo: { name: 'MaskShift', version: VERSION }, locale: 'en', rootPath: this.root, rootUri: pathToFileURL(this.root).href, diff --git a/src/tools/platform-tools.mjs b/src/tools/platform-tools.mjs index 01a06fd..1598b84 100644 --- a/src/tools/platform-tools.mjs +++ b/src/tools/platform-tools.mjs @@ -255,7 +255,25 @@ export function registerPlatformTools(registry, { config }) { name: 'port_inspect', title: 'Inspect network ports', description: 'Inspect listening sockets and processes using ss, netstat, or lsof.', category: 'system', readOnly: true, inputSchema: { type: 'object', properties: { port: { type: 'integer', minimum: 1, maximum: 65535 }, protocol: { type: 'string', enum: ['tcp', 'udp', 'all'], default: 'all' } } }, - execute: async (args) => { const ss = await commandExists('ss'); const lsof = await commandExists('lsof'); const command = ss ? `${shellQuote(ss)} -lntup ${args.port ? `'( sport = :${Number(args.port)} )'` : ''}` : lsof ? `${shellQuote(lsof)} -nP -i${args.protocol === 'tcp' ? 'TCP' : args.protocol === 'udp' ? 'UDP' : ''}${args.port ? `:${Number(args.port)}` : ''}` : 'netstat -anp'; return runCommand(command, { timeoutMs: 20_000, maxOutputChars: 80_000 }); }, + // ss, lsof and netstat all exit non-zero when a filter simply matches nothing, so the + // exit code reports "no sockets", not failure. Only a missing inspector is an error. + execute: async (args) => { + const [ss, lsof, netstat] = await Promise.all([commandExists('ss'), commandExists('lsof'), commandExists('netstat')]); + const inspector = ss ? 'ss' : lsof ? 'lsof' : netstat ? 'netstat' : null; + if (!inspector) throw new Error('No socket inspector is installed. Install one of: ss (iproute2), lsof, netstat (net-tools).'); + const command = inspector === 'ss' + ? `${shellQuote(ss)} -lntup ${args.port ? `'( sport = :${Number(args.port)} )'` : ''}` + : inspector === 'lsof' + ? `${shellQuote(lsof)} -nP -i${args.protocol === 'tcp' ? 'TCP' : args.protocol === 'udp' ? 'UDP' : ''}${args.port ? `:${Number(args.port)}` : ''}` + : `${shellQuote(netstat)} -anp`; + const result = await runCommand(command, { timeoutMs: 20_000, maxOutputChars: 80_000 }); + // ss and netstat print their column headers whether or not the filter matched, so + // "stdout is non-empty" does not mean "found something". Count rows past the header + // instead. lsof prints nothing at all on no match, so the same arithmetic holds. + const headerRows = { ss: 1, lsof: 1, netstat: 2 }[inspector]; + const rows = result.stdout.split('\n').filter((line) => line.trim()); + return { ...result, inspector, rows: Math.max(0, rows.length - headerRows), matched: rows.length > headerRows }; + }, }); registry.register({ diff --git a/src/tools/search-tools.mjs b/src/tools/search-tools.mjs index 0f692a6..192a145 100644 --- a/src/tools/search-tools.mjs +++ b/src/tools/search-tools.mjs @@ -103,10 +103,10 @@ export function registerSearchTools(registry, { indexer, workspaceManager, confi description: 'Build or refresh MaskShift’s local SQLite FTS code index for structure-aware context retrieval.', category: 'search', risk: 'local-index', keywords: ['index', 'rag', 'codebase context', 'fts'], - inputSchema: { type: 'object', properties: { force: { type: 'boolean', default: true } } }, + inputSchema: { type: 'object', properties: {} }, execute: async (args, context) => { if (!context.workspaceId) throw new Error('Repository indexing requires a workspace'); - return indexer.index(context.workspaceId, { force: args.force !== false }); + return indexer.index(context.workspaceId); }, }); diff --git a/src/tui/app.mjs b/src/tui/app.mjs index b59199c..3253df3 100644 --- a/src/tui/app.mjs +++ b/src/tui/app.mjs @@ -26,6 +26,9 @@ import * as terminalView from './views/terminal.mjs'; const VIEWS = [chatView, filesView, arsenalView, networkView, modshopView, terminalView]; const EVENT_LIMIT = 400; +// How much of an in-flight answer stays on screen. The full turn lands in the transcript the +// moment it completes, so the live view only has to show the leading edge of the writing. +const STREAM_PREVIEW_LINES = 12; const TERMINAL_LIMIT = 2000; export class MaskShiftTui { @@ -76,6 +79,7 @@ export class MaskShiftTui { this.activeCapabilities = new Set(); this.subagents = 0; this.pendingCalls = new Map(); + this.streamingText = ''; this.tokenHistory = []; this.totals = { input: 0, output: 0, cost: 0 }; this.startedAt = null; @@ -259,12 +263,22 @@ export class MaskShiftTui { }); } if (this.busy && this.pendingCalls.size === 0) { + // Tokens arriving now: show the answer being written rather than a spinner label. The + // 120ms ticker already repaints while busy, so deltas only have to accumulate. + const streamed = this.streamingText.trim(); entries.push({ - render: (theme, width) => [fit( - gutter(theme, this.spinner.frame(theme), { tone: theme.roles.primary }) - + theme.paint(this.thinkingLabel || 'Thinking…', { fg: theme.roles.muted, italic: true }), - width, - )], + render: (theme, width) => (streamed + ? [ + fit(gutter(theme, this.spinner.frame(theme), { tone: theme.roles.primary }) + + theme.paint('MASKSHIFT', { fg: theme.roles.primary, bold: true }), width), + ...wrap(streamed, Math.max(8, width - 2)).slice(-STREAM_PREVIEW_LINES) + .map((piece) => fit(` ${theme.paint(piece, { fg: theme.roles.text })}`, width)), + ] + : [fit( + gutter(theme, this.spinner.frame(theme), { tone: theme.roles.primary }) + + theme.paint(this.thinkingLabel || 'Thinking…', { fg: theme.roles.muted, italic: true }), + width, + )]), }); } return entries; @@ -645,9 +659,14 @@ export class MaskShiftTui { this.startedAt = Date.now(); this.step = 0; this.pendingCalls.clear(); + this.streamingText = ''; this.thinkingLabel = 'STUDYING THE TARGET'; break; + case 'run.assistant-delta': + this.streamingText += payload.text || ''; + break; case 'run.model-turn': + this.streamingText = ''; this.step = payload.step || this.step + 1; this.thinkingLabel = `TURN ${String(this.step).padStart(2, '0')} — ${payload.tools?.length ?? 0} TOOLS ACTIVE`; this.activeCapabilities = new Set(payload.tools || []); @@ -659,6 +678,7 @@ export class MaskShiftTui { this.tokenHistory.push((payload.usage.outputTokens || payload.usage.output_tokens || 0)); if (this.tokenHistory.length > 120) this.tokenHistory.shift(); } + this.streamingText = ''; for (const call of payload.toolCalls || []) this.pendingCalls.set(call.id, { name: call.name, args: call.args }); this.messages = this.runtime.store.listMessages(this.sessionId, 1000); this.thinkingLabel = payload.toolCalls?.length ? 'EXECUTING TOOLS' : 'WRITING'; @@ -677,6 +697,7 @@ export class MaskShiftTui { case 'run.cancelled': case 'run.max-steps': { this.pendingCalls.clear(); + this.streamingText = ''; this.messages = this.runtime.store.listMessages(this.sessionId, 1000); const run = this.runId ? this.runtime.store.getRun(this.runId) : null; this.activeRun = null; @@ -1249,7 +1270,7 @@ export class MaskShiftTui { this.newSession({ silent: true }); await this.loadFileTree({ force: true }); await this.refreshGit(); - if (values.index) void this.runtime.indexer.index(workspace.id, { force: true }).catch(() => {}); + if (values.index) void this.runtime.indexer.index(workspace.id).catch(() => {}); this.toast(`Target locked: ${workspace.name}`, 'success'); }, }); @@ -1467,7 +1488,7 @@ export class MaskShiftTui { if (!this.workspaceId) return; this.toast('Indexing the target…', 'info'); try { - const stats = await this.runtime.indexer.index(this.workspaceId, { force: true }); + const stats = await this.runtime.indexer.index(this.workspaceId); this.toast(`Indexed ${stats.files ?? stats.chunks ?? 0} entries`, 'success'); } catch (error) { this.toast(error.message, 'error'); diff --git a/tests/all-tools.test.mjs b/tests/all-tools.test.mjs index bacd901..38c30dd 100644 --- a/tests/all-tools.test.mjs +++ b/tests/all-tools.test.mjs @@ -59,10 +59,25 @@ async function setup(t, overrides = {}) { return { project, runtime, workspace, context, call, optional }; } +// A scenario that throws part-way never reaches the tools listed after the failing line, so +// the coverage gate would blame those tools for an assertion that failed somewhere else. +// Record the real failure and let the gate report that instead. +const scenarioFailures = []; +function scenario(suite, name, body) { + return suite.test(name, async (t) => { + try { + await body(t); + } catch (error) { + scenarioFailures.push({ name, message: error.message }); + throw error; + } + }); +} + // Sequential scenarios keep PATH/fetch fixtures local to this test process. // No live cloud account, container, remote machine, or paid model is used. test('all native tools have executable verification', { timeout: 180_000 }, async (suite) => { - await suite.test('filesystem writes, reads, patches, moves and deletes', async (t) => { + await scenario(suite, 'filesystem writes, reads, patches, moves and deletes', async (t) => { const { call, optional, project } = await setup(t); await call('fs_mkdir', { path: 'nested' }, r => assert.equal(r.created, true)); await call('fs_write', { path: 'nested/a.txt', content: 'alpha\nbeta\n' }, r => assert.equal(r.size, 11)); @@ -79,7 +94,7 @@ test('all native tools have executable verification', { timeout: 180_000 }, asyn await assert.rejects(fsp.access(path.join(project, 'nested/b.txt')), { code: 'ENOENT' }); }); - await suite.test('shell commands and background process lifecycle', async (t) => { + await scenario(suite, 'shell commands and background process lifecycle', async (t) => { const { call, optional, project } = await setup(t); await call('shell_exec', { command: 'printf hello' }, r => { assert.equal(r.code, 0); assert.equal(r.stdout, 'hello'); }); await call('shell_exec_parallel', { commands: ['printf one', { command: 'printf two' }] }, r => assert.deepEqual(r.map(x => [x.code, x.stdout]), [[0, 'one'], [0, 'two']])); @@ -95,7 +110,7 @@ test('all native tools have executable verification', { timeout: 180_000 }, asyn await call('command_lookup', { commands: ['node', 'maskshift_nonexistent_command_123'] }, r => { assert.ok(r.node); assert.equal(r.maskshift_nonexistent_command_123, null); }); }); - await suite.test('search, project context and indexing', async (t) => { + await scenario(suite, 'search, project context and indexing', async (t) => { const { call, optional, project } = await setup(t); await fsp.writeFile(path.join(project, 'imports.js'), "import fs from 'node:fs';\n"); await optional('search_text', ['rg'], { query: 'velocity', glob: '*.js' }, r => assert.ok(r.matches.some(m => m.text.includes('function velocity')))); @@ -114,7 +129,7 @@ test('all native tools have executable verification', { timeout: 180_000 }, asyn await call('usage_report', {}, r => assert.equal(r.runsConsidered, 0)); }); - await suite.test('Git branch, commit, checkpoint and worktree lifecycle', async (t) => { + await scenario(suite, 'Git branch, commit, checkpoint and worktree lifecycle', async (t) => { const { call, optional, project } = await setup(t); await call('git_status', {}, r => assert.match(r.status, /^## /)); await call('git_log', {}, r => assert.equal(r.log[0].subject, 'initial')); @@ -134,7 +149,7 @@ test('all native tools have executable verification', { timeout: 180_000 }, asyn assert.match(await fsp.readFile(path.join(worktree.path, 'index.js'), 'utf8'), /verification/); }); - await suite.test('memory, skills and lazy capability state', async (t) => { + await scenario(suite, 'memory, skills and lazy capability state', async (t) => { const { call, runtime } = await setup(t); const memory = await call('memory_save', { title: 'fixture-memory', content: 'Remember the velocity formula' }); await call('memory_search', { query: 'velocity' }, r => assert.ok(r.some(m => m.id === memory.id))); @@ -157,7 +172,7 @@ test('all native tools have executable verification', { timeout: 180_000 }, asyn await call('plan_get', {}, r => assert.equal(r.steps[0].status, 'completed')); }); - await suite.test('SQLite, archives, runtime cells and environment', async (t) => { + await scenario(suite, 'SQLite, archives, runtime cells and environment', async (t) => { const { call, optional, project } = await setup(t); await call('sqlite_query', { database: 'data.sqlite', sql: 'CREATE TABLE items (id INTEGER, name TEXT)' }); await call('sqlite_query', { database: 'data.sqlite', sql: 'INSERT INTO items VALUES (?, ?)', parameters: [7, "it's a test"] }, r => assert.equal(r.changes, 1)); @@ -172,13 +187,21 @@ test('all native tools have executable verification', { timeout: 180_000 }, asyn await call('environment_set', { values: { MASKSHIFT_TEST_MARKER: 'fixture' } }); try { await call('environment_list', { filter: '^MASKSHIFT_TEST_MARKER$', includeValues: true }, r => assert.deepEqual(r, { MASKSHIFT_TEST_MARKER: 'fixture' })); } finally { await call('environment_set', { values: { MASKSHIFT_TEST_MARKER: null } }); } - await optional('port_inspect', ['ss', 'lsof', 'netstat'], { port: 1 }, r => assert.equal(r.code, 0)); + // Port 1 is unused everywhere. The exit code cannot be asserted — lsof exits 1 on no + // match while ss exits 0 — and neither can raw stdout, because ss and netstat print + // their headers regardless. Only the parsed result is stable across all three. + await optional('port_inspect', ['ss', 'lsof', 'netstat'], { port: 1 }, r => { + assert.ok(['ss', 'lsof', 'netstat'].includes(r.inspector)); + assert.equal(typeof r.stdout, 'string'); + assert.equal(r.rows, 0); + assert.equal(r.matched, false); + }); const source = path.join(project, 'index.js'); const transferred = await optional('rsync_transfer', ['rsync'], { source, destination: path.join(project, 'copy.js') }, r => assert.equal(r.code, 0)); if (transferred) assert.equal(await fsp.readFile(path.join(project, 'copy.js'), 'utf8'), await fsp.readFile(source, 'utf8')); }); - await suite.test('PDF extraction and notebook edits', async (t) => { + await scenario(suite, 'PDF extraction and notebook edits', async (t) => { const { call, optional, project } = await setup(t); // Minimal real PDF fixture; pdftotext is the only required PDF dependency. const stream = 'BT /F1 18 Tf 30 100 Td (MASKSHIFT_PDF_OK) Tj ET'; @@ -195,7 +218,7 @@ test('all native tools have executable verification', { timeout: 180_000 }, asyn await call('notebook_read', { path: 'fixture.ipynb' }, r => assert.equal(r.cells[0].source, '2+2')); }); - await suite.test('image OCR and description fallback', async (t) => { + await scenario(suite, 'image OCR and description fallback', async (t) => { const { call, project } = await setup(t); await fsp.writeFile(path.join(project, 'fixture.png'), makeFixturePng()); // No tesseract/vision model guaranteed on this host, so this exercises the graceful-degradation @@ -208,7 +231,7 @@ test('all native tools have executable verification', { timeout: 180_000 }, asyn }); }); - await suite.test('automation and plugin lifecycle in disposable home', async (t) => { + await scenario(suite, 'automation and plugin lifecycle in disposable home', async (t) => { const { call, project, runtime } = await setup(t); const automation = await call('automation_create', { name: 'verify', schedule: 'every 1h', enabled: false, action: { type: 'shell', command: 'printf AUTOMATION_OK > automated.txt' } }); const automationId = automation.id; @@ -237,7 +260,7 @@ test('all native tools have executable verification', { timeout: 180_000 }, asyn assert.equal(await runtime.toolRegistry.execute('installed_fixture_ping', {}), 'pong'); }); - await suite.test('HTTP fetch, download, search and registry fixtures', async (t) => { + await scenario(suite, 'HTTP fetch, download, search and registry fixtures', async (t) => { const { call, optional, project } = await setup(t); const server = await jsonServer(t, (req, res) => { if (req.url === '/file') { res.writeHead(200, { 'Content-Type': 'text/plain' }); return res.end('DOWNLOAD_OK'); } @@ -258,7 +281,7 @@ test('all native tools have executable verification', { timeout: 180_000 }, asyn await call('mcp_registry_install', { registryName: 'io.fixture/test-server' }, r => assert.match(JSON.stringify(r), /test-server/), 'HTTP response fixture'); }); - await suite.test('MCP tools over a real stdio fixture server', async (t) => { + await scenario(suite, 'MCP tools over a real stdio fixture server', async (t) => { const { call, context } = await setup(t); await call('mcp_add', { name: 'verification', definition: { transport: 'stdio', command: process.execPath, args: [path.join(here, 'fixtures-mcp-server.mjs'), 'modern'], enabled: true, lazy: true } }); await call('mcp_list', {}, r => assert.ok(r.some(s => s.name === 'verification'))); @@ -272,7 +295,7 @@ test('all native tools have executable verification', { timeout: 180_000 }, asyn assert.equal(context.capabilityState.mcpServers.has('verification'), false); }); - await suite.test('LSP tools over framed stdio with edits applied to files', async (t) => { + await scenario(suite, 'LSP tools over framed stdio with edits applied to files', async (t) => { const { call, runtime, project } = await setup(t); await call('lsp_discover', {}, r => assert.ok(r.some(s => s.id === 'typescript'))); runtime.lspManager.availability = [{ id: 'fixture', languages: ['javascript'], available: true, selected: { executable: process.execPath, args: [path.join(here, 'fixtures-lsp-server.mjs')] } }]; @@ -291,7 +314,7 @@ test('all native tools have executable verification', { timeout: 180_000 }, asyn assert.equal(runtime.lspManager.list().length, 0); }); - await suite.test('agent tools with a deterministic local model endpoint', async (t) => { + await scenario(suite, 'agent tools with a deterministic local model endpoint', async (t) => { const server = await jsonServer(t, (req, res) => respondJson(res, 200, { id: 'fixture', output: [{ type: 'message', role: 'assistant', content: [{ type: 'output_text', text: 'FIXTURE_AGENT_OK' }] }] })); const { call, runtime, workspace, context } = await setup(t, { defaultModel: 'fixture:local', providers: [{ id: 'fixture', type: 'openai-responses', baseUrl: server.url, apiKeyEnv: null, enabled: true, models: [{ id: 'local' }], timeoutMs: 5000 }] }); context.runId = null; // Manual CLI tools have no parent run. @@ -303,7 +326,7 @@ test('all native tools have executable verification', { timeout: 180_000 }, asyn assert.equal((await runtime.engine.waitForRun(run.id)).status, 'cancelled'); }); - await suite.test('external agent bridges execute a local echo CLI', async (t) => { + await scenario(suite, 'external agent bridges execute a local echo CLI', async (t) => { const { call } = await setup(t, { agentBridges: { claude: { enabled: false }, codex: { enabled: false }, opencode: { enabled: false }, copilot: { enabled: false }, hermes: { enabled: false }, aider: { enabled: false }, fixture: { command: 'node', args: ['-e', 'console.log(process.argv[1])', '{prompt}'] } } }); await call('agent_bridge_discover', {}, r => assert.ok(r.some(b => b.name === 'fixture' && b.available)), 'local CLI fixture'); await call('agent_bridge_help', { bridge: 'fixture' }, r => { assert.equal(r.code, 0); assert.match(r.stdout, /Usage/); }, 'local CLI fixture'); @@ -311,7 +334,7 @@ test('all native tools have executable verification', { timeout: 180_000 }, asyn await call('external_agent_run', { command: 'node', args: ['-e', 'console.log(process.argv[1])', '{prompt}'], prompt: 'CUSTOM_OK' }, r => { assert.equal(r.code, 0); assert.equal(r.stdout.trim(), 'CUSTOM_OK'); }, 'local CLI fixture'); }); - await suite.test('container, SSH, database CLI and service command adapters', async (t) => { + await scenario(suite, 'container, SSH, database CLI and service command adapters', async (t) => { const { call, optional, project } = await setup(t); const bin = path.join(project, 'fixture-bin'); await fsp.mkdir(bin); const source = `#!${process.execPath}\nimport path from 'node:path';\nconst argv = process.argv.slice(2);\nif (path.basename(process.argv[1]) === 'docker' && argv[0] === 'ps') console.log(JSON.stringify({ID:'fixture-container', Names:'fixture'}));\nelse console.log(JSON.stringify({command:path.basename(process.argv[1]),argv}));\n`; @@ -335,7 +358,7 @@ test('all native tools have executable verification', { timeout: 180_000 }, asyn await fixtureCall('system_service', { service: 'fixture.service', action: 'status', user: true }, r => assert.deepEqual(JSON.parse(r.stdout).argv, ['--user', 'status', 'fixture.service'])); }); - await suite.test('browser tool delegation contracts', async (t) => { + await scenario(suite, 'browser tool delegation contracts', async (t) => { const { call, runtime, workspace } = await setup(t); await call('browser_discover', {}, r => assert.equal(typeof r, 'object')); const cases = [ @@ -368,6 +391,11 @@ test('all native tools have executable verification', { timeout: 180_000 }, asyn await suite.test('coverage gate: every native tool has assertions or an explicit dependency skip', () => { const missing = inventory.filter(tool => !coverage.has(tool.name)).map(tool => tool.name); + if (scenarioFailures.length) { + assert.fail(`Coverage is unverifiable: ${scenarioFailures.length} scenario(s) aborted before finishing.\n` + + scenarioFailures.map(failure => ` - ${failure.name}: ${failure.message}`).join('\n') + + (missing.length ? `\nTools never reached: ${missing.join(', ')}` : '')); + } assert.deepEqual(missing, [], `Tools without successful assertions: ${missing.join(', ')}`); }); if (process.env.MASKSHIFT_TOOL_REPORT) { diff --git a/tests/browser-sandbox.test.mjs b/tests/browser-sandbox.test.mjs new file mode 100644 index 0000000..a68d0df --- /dev/null +++ b/tests/browser-sandbox.test.mjs @@ -0,0 +1,63 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { sandboxUnavailable } from '../src/browser/manager.mjs'; + +// Chromium aborts outright — "No usable sandbox!" — when its sandbox cannot start, so this +// predicate decides whether a launch works at all on a given host. It is untestable by +// observation (no single machine exhibits every case), which is exactly why it is worth +// pinning: the previous rule inferred it from the uid alone and got a default GitHub +// Actions runner wrong. +const missing = () => { throw Object.assign(new Error('ENOENT'), { code: 'ENOENT' }); }; +const knobs = (values) => (file) => (file in values ? values[file] : missing()); + +const APPARMOR = '/proc/sys/kernel/apparmor_restrict_unprivileged_userns'; +const USERNS_CLONE = '/proc/sys/kernel/unprivileged_userns_clone'; +const MAX_USERNS = '/proc/sys/user/max_user_namespaces'; + +test('Chromium sandbox availability detection', async (suite) => { + await suite.test('non-Linux hosts always keep the sandbox', () => { + for (const platform of ['darwin', 'win32']) { + assert.equal(sandboxUnavailable({ platform, uid: 0, readFile: missing }), false); + } + }); + + await suite.test('root cannot use the sandbox at all', () => { + assert.equal(sandboxUnavailable({ platform: 'linux', uid: 0, readFile: missing }), true); + }); + + await suite.test('a plain unprivileged user keeps the sandbox', () => { + assert.equal(sandboxUnavailable({ platform: 'linux', uid: 1001, readFile: missing }), false); + assert.equal(sandboxUnavailable({ + platform: 'linux', uid: 1001, readFile: knobs({ [APPARMOR]: '0\n', [MAX_USERNS]: '64230\n' }), + }), false); + }); + + // The case that broke CI: non-root on Ubuntu 23.10+, where AppArmor withholds + // unprivileged user namespaces from unconfined programs. + await suite.test('AppArmor restricting unprivileged userns disables the sandbox', () => { + assert.equal(sandboxUnavailable({ + platform: 'linux', uid: 1001, readFile: knobs({ [APPARMOR]: '1\n' }), + }), true); + }); + + await suite.test('a kernel with user namespaces switched off disables the sandbox', () => { + assert.equal(sandboxUnavailable({ + platform: 'linux', uid: 1001, readFile: knobs({ [USERNS_CLONE]: '0\n' }), + }), true); + assert.equal(sandboxUnavailable({ + platform: 'linux', uid: 1001, readFile: knobs({ [MAX_USERNS]: '0\n' }), + }), true); + }); + + await suite.test('an unreadable knob is treated as no restriction, not as a failure', () => { + assert.equal(sandboxUnavailable({ + platform: 'linux', + uid: 1001, + readFile: () => { throw Object.assign(new Error('EACCES'), { code: 'EACCES' }); }, + }), false); + }); + + await suite.test('a host that cannot report a uid is treated as unable to sandbox', () => { + assert.equal(sandboxUnavailable({ platform: 'linux', uid: null, readFile: missing }), true); + }); +}); diff --git a/tests/cli-agent.test.mjs b/tests/cli-agent.test.mjs index 78a42e3..fbc3459 100644 --- a/tests/cli-agent.test.mjs +++ b/tests/cli-agent.test.mjs @@ -2,6 +2,7 @@ import assert from 'node:assert/strict'; import fsp from 'node:fs/promises'; import path from 'node:path'; import test from 'node:test'; +import { createRequire } from 'node:module'; import { main } from '../src/cli/main.mjs'; import { createProject, jsonServer, readJsonBody, respondJson, runtimeForTest, waitFor } from './helpers.mjs'; @@ -157,7 +158,8 @@ test('the CLI exposes the full capability surface without a browser', async (t) assert.equal(configured.maxAgentSteps, 42); const doctorReport = JSON.parse((await cli(['doctor', ...base])).output); - assert.equal(doctorReport.version, '1.0.0'); + // Assert against the manifest, not a literal: the two drifted once already. + assert.equal(doctorReport.version, createRequire(import.meta.url)('../package.json').version); assert.ok(doctorReport.tools >= 140); assert.ok(doctorReport.commands.node); }); @@ -183,7 +185,7 @@ test('workspace search reads back what the indexer wrote', async (t) => { const project = await createProject(t); const runtime = await runtimeForTest(t, project); const workspace = await runtime.workspaceManager.open(project); - await runtime.indexer.index(workspace.id, { force: true }); + await runtime.indexer.index(workspace.id); const hits = await waitFor(async () => { const found = await runtime.indexer.search(workspace.id, 'velocity distance time', 5); return found.length ? found : null; diff --git a/tests/providers.test.mjs b/tests/providers.test.mjs new file mode 100644 index 0000000..402da8f --- /dev/null +++ b/tests/providers.test.mjs @@ -0,0 +1,115 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { createProject, runtimeForTest, jsonServer, respondJson } from './helpers.mjs'; + +// A provider fixture whose responses are scripted per attempt, so a test can assert exactly +// how many times MaskShift called the endpoint. +async function scriptedProvider(t, script, providerOverrides = {}) { + const state = { attempts: 0, retryEvents: [] }; + const server = await jsonServer(t, (request, response) => { + const step = script[Math.min(state.attempts, script.length - 1)]; + state.attempts += 1; + if (step.destroy) { request.destroy(); response.destroy(); return; } + respondJson(response, step.status, step.body ?? { error: { message: 'scripted failure' } }, step.headers || {}); + }); + const project = await createProject(t, { git: false }); + const runtime = await runtimeForTest(t, project, { + defaultModel: 'fixture:local', + // Keep backoff sub-millisecond: this asserts retry policy, not wall-clock patience. + providerRetry: { attempts: 3, baseMs: 1, maxMs: 4 }, + providers: [{ + id: 'fixture', name: 'Fixture', type: 'openai-responses', baseUrl: server.url, + apiKeyEnv: null, enabled: true, models: [{ id: 'local' }], timeoutMs: 5000, + ...providerOverrides, + }], + }); + runtime.eventBus.subscribe((event) => { if (event.type === 'model.request.retrying') state.retryEvents.push(event.payload); }); + return { state, runtime, server }; +} + +const OK_BODY = { id: 'fixture', output: [{ type: 'message', role: 'assistant', content: [{ type: 'output_text', text: 'RETRY_OK' }] }] }; +const ask = (runtime) => runtime.providerManager.complete({ + modelRef: 'fixture:local', messages: [{ role: 'user', content: 'ping' }], tools: [], maxTokens: 32, +}); + +test('provider retry', { timeout: 30_000 }, async (suite) => { + await suite.test('retries a 503 and returns the eventual success', async (t) => { + const { state, runtime } = await scriptedProvider(t, [ + { status: 503 }, + { status: 200, body: OK_BODY }, + ]); + const result = await ask(runtime); + assert.equal(result.content, 'RETRY_OK'); + assert.equal(state.attempts, 2); + }); + + await suite.test('retries a 429 and honours Retry-After', async (t) => { + const { state, runtime } = await scriptedProvider(t, [ + { status: 429, headers: { 'Retry-After': '0' } }, + { status: 200, body: OK_BODY }, + ]); + assert.equal((await ask(runtime)).content, 'RETRY_OK'); + assert.equal(state.attempts, 2); + assert.equal(state.retryEvents.length, 1); + assert.equal(state.retryEvents[0].status, 429); + assert.equal(state.retryEvents[0].waitMs, 0); + }); + + await suite.test('retries a dropped connection', async (t) => { + const { state, runtime } = await scriptedProvider(t, [ + { destroy: true }, + { status: 200, body: OK_BODY }, + ]); + assert.equal((await ask(runtime)).content, 'RETRY_OK'); + assert.equal(state.attempts, 2); + assert.equal(state.retryEvents[0].status, null); + }); + + await suite.test('never retries a 401', async (t) => { + const { state, runtime } = await scriptedProvider(t, [{ status: 401, body: { error: { message: 'bad key' } } }]); + await assert.rejects(ask(runtime), /401/); + assert.equal(state.attempts, 1); + assert.equal(state.retryEvents.length, 0); + }); + + // The tool-protocol downgrade reads a 400 body, so retrying one would delay the recovery + // it depends on by the full backoff budget. + await suite.test('never retries a 400', async (t) => { + const { state, runtime } = await scriptedProvider(t, [{ status: 400, body: { error: { message: 'malformed' } } }]); + await assert.rejects(ask(runtime), /400/); + assert.equal(state.attempts, 1); + }); + + await suite.test('gives up after the configured attempt budget', async (t) => { + const { state, runtime } = await scriptedProvider(t, [{ status: 500 }]); + await assert.rejects(ask(runtime), /500/); + assert.equal(state.attempts, 3); + assert.equal(state.retryEvents.length, 2); + }); + + await suite.test('a per-provider retry setting overrides the global default', async (t) => { + const { state, runtime } = await scriptedProvider(t, [{ status: 500 }], { retry: { attempts: 1 } }); + await assert.rejects(ask(runtime), /500/); + assert.equal(state.attempts, 1); + }); + + await suite.test('a cancelled run stops retrying instead of waiting out the backoff', async (t) => { + const { state, runtime } = await scriptedProvider(t, [{ status: 503 }], { retry: { attempts: 5, baseMs: 10_000, maxMs: 10_000 } }); + const controller = new AbortController(); + const pending = runtime.providerManager.complete({ + modelRef: 'fixture:local', messages: [{ role: 'user', content: 'ping' }], tools: [], maxTokens: 32, + signal: controller.signal, + }); + // Let the first attempt land and enter its backoff, then cancel. + await new Promise((resolve) => setTimeout(resolve, 150)); + controller.abort(new Error('Run cancelled')); + await assert.rejects(pending, /cancelled/i); + assert.equal(state.attempts, 1); + }); + + await suite.test('model listing probes stay single-shot', async (t) => { + const { state, runtime } = await scriptedProvider(t, [{ status: 503 }]); + await runtime.providerManager.discover('fixture', { force: true }).catch(() => {}); + assert.equal(state.attempts, 1); + }); +}); diff --git a/tests/runtime.test.mjs b/tests/runtime.test.mjs index f27a6e4..0f0fb33 100644 --- a/tests/runtime.test.mjs +++ b/tests/runtime.test.mjs @@ -27,7 +27,7 @@ test('runtime exposes maximal lazy capabilities and executes host tools', async assert.equal(shell.code, 0); assert.equal(shell.stdout, 'HOST_EXEC_OK'); - const index = await runtime.indexer.index(workspace.id, { force: true }); + const index = await runtime.indexer.index(workspace.id); assert.ok(index.indexedFiles >= 3); assert.ok((await runtime.indexer.search(workspace.id, 'velocity distance time')).some((hit) => hit.path === 'index.js')); }); diff --git a/tests/streaming.test.mjs b/tests/streaming.test.mjs new file mode 100644 index 0000000..1a96f6b --- /dev/null +++ b/tests/streaming.test.mjs @@ -0,0 +1,234 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { createProject, runtimeForTest, jsonServer, readJsonBody } from './helpers.mjs'; + +// Serve a scripted SSE (or NDJSON) body so the reader is exercised against real chunk +// boundaries rather than one tidy write. +function streamServer(t, frames, { contentType = 'text/event-stream', sse = true } = {}) { + const seen = { bodies: [], urls: [] }; + return jsonServer(t, async (request, response) => { + seen.urls.push(request.url); + seen.bodies.push(await readJsonBody(request).catch(() => null)); + response.writeHead(200, { 'Content-Type': contentType }); + for (const frame of frames) { + response.write(sse ? `data: ${JSON.stringify(frame)}\n\n` : `${JSON.stringify(frame)}\n`); + } + if (sse) response.write('data: [DONE]\n\n'); + response.end(); + }).then((server) => ({ ...server, seen })); +} + +async function providerRuntime(t, server, type, extra = {}) { + const project = await createProject(t, { git: false }); + return runtimeForTest(t, project, { + defaultModel: 'fixture:local', + providers: [{ + id: 'fixture', name: 'Fixture', type, baseUrl: server.url, apiKey: 'test-key', + apiKeyEnv: null, enabled: true, models: [{ id: 'local' }], timeoutMs: 5000, ...extra, + }], + }); +} + +function collect() { + const deltas = []; + return { deltas, onDelta: (text) => deltas.push(text) }; +} + +const ask = (runtime, onDelta, tools = []) => runtime.providerManager.complete({ + modelRef: 'fixture:local', messages: [{ role: 'user', content: 'ping' }], tools, maxTokens: 64, onDelta, +}); + +test('provider streaming', { timeout: 30_000 }, async (suite) => { + await suite.test('openai-compatible streams text and assembles split tool-call arguments', async (t) => { + const server = await streamServer(t, [ + { choices: [{ delta: { content: 'Hel' } }] }, + { choices: [{ delta: { content: 'lo' } }] }, + { choices: [{ delta: { tool_calls: [{ index: 0, id: 'call_1', function: { name: 'fs_read', arguments: '{"path":' } }] } }] }, + { choices: [{ delta: { tool_calls: [{ index: 0, function: { arguments: '"a.txt"}' } }] } }] }, + { choices: [{ delta: {}, finish_reason: 'tool_calls' }], usage: { prompt_tokens: 7, completion_tokens: 3 } }, + ]); + const runtime = await providerRuntime(t, server, 'openai-compatible'); + const { deltas, onDelta } = collect(); + const result = await ask(runtime, onDelta); + + assert.deepEqual(deltas, ['Hel', 'lo']); + assert.equal(result.content, 'Hello'); + assert.equal(result.finishReason, 'tool_calls'); + assert.deepEqual(result.usage, { prompt_tokens: 7, completion_tokens: 3 }); + assert.equal(result.streamed, true); + assert.deepEqual(result.toolCalls, [{ id: 'call_1', name: 'fs_read', args: { path: 'a.txt' } }]); + assert.equal(server.seen.bodies[0].stream, true); + }); + + await suite.test('openai-responses streams text and function-call arguments', async (t) => { + const server = await streamServer(t, [ + { type: 'response.output_text.delta', delta: 'one ' }, + { type: 'response.output_text.delta', delta: 'two' }, + { type: 'response.output_item.added', output_index: 1, item: { type: 'function_call', call_id: 'fc_1', name: 'shell_exec' } }, + { type: 'response.function_call_arguments.delta', output_index: 1, delta: '{"command":"ls"}' }, + { type: 'response.completed', response: { id: 'resp_1', status: 'completed', usage: { input_tokens: 4 } } }, + ]); + const runtime = await providerRuntime(t, server, 'openai-responses'); + const { deltas, onDelta } = collect(); + const result = await ask(runtime, onDelta); + + assert.deepEqual(deltas, ['one ', 'two']); + assert.equal(result.content, 'one two'); + assert.equal(result.finishReason, 'completed'); + assert.deepEqual(result.toolCalls, [{ id: 'fc_1', name: 'shell_exec', args: { command: 'ls' } }]); + }); + + await suite.test('anthropic streams text deltas and input_json_delta tool arguments', async (t) => { + const server = await streamServer(t, [ + { type: 'message_start', message: { usage: { input_tokens: 11 } } }, + { type: 'content_block_start', index: 0, content_block: { type: 'text' } }, + { type: 'content_block_delta', index: 0, delta: { type: 'text_delta', text: 'thinking' } }, + { type: 'content_block_start', index: 1, content_block: { type: 'tool_use', id: 'toolu_1', name: 'fs_write' } }, + { type: 'content_block_delta', index: 1, delta: { type: 'input_json_delta', partial_json: '{"path"' } }, + { type: 'content_block_delta', index: 1, delta: { type: 'input_json_delta', partial_json: ':"b.txt"}' } }, + { type: 'message_delta', delta: { stop_reason: 'tool_use' }, usage: { output_tokens: 9 } }, + ]); + const runtime = await providerRuntime(t, server, 'anthropic'); + const { deltas, onDelta } = collect(); + const result = await ask(runtime, onDelta); + + assert.deepEqual(deltas, ['thinking']); + assert.equal(result.content, 'thinking'); + assert.equal(result.finishReason, 'tool_use'); + assert.deepEqual(result.usage, { input_tokens: 11, output_tokens: 9 }); + assert.deepEqual(result.toolCalls, [{ id: 'toolu_1', name: 'fs_write', args: { path: 'b.txt' } }]); + }); + + await suite.test('ollama streams newline-delimited JSON', async (t) => { + const server = await streamServer(t, [ + { message: { content: 'aa' }, done: false }, + { message: { content: 'bb' }, done: false }, + { message: { content: '' }, done: true, done_reason: 'stop', prompt_eval_count: 5, eval_count: 2 }, + ], { contentType: 'application/x-ndjson', sse: false }); + const runtime = await providerRuntime(t, server, 'ollama'); + const { deltas, onDelta } = collect(); + const result = await ask(runtime, onDelta); + + assert.deepEqual(deltas, ['aa', 'bb']); + assert.equal(result.content, 'aabb'); + assert.equal(result.finishReason, 'stop'); + assert.equal(result.usage.input_tokens, 5); + }); + + await suite.test('gemini streams over the SSE endpoint', async (t) => { + const server = await streamServer(t, [ + { candidates: [{ content: { parts: [{ text: 'gem' }] } }] }, + { candidates: [{ content: { parts: [{ text: 'ini' }] }, finishReason: 'STOP' }], usageMetadata: { totalTokenCount: 12 } }, + ]); + const runtime = await providerRuntime(t, server, 'gemini'); + const { deltas, onDelta } = collect(); + const result = await ask(runtime, onDelta); + + assert.deepEqual(deltas, ['gem', 'ini']); + assert.equal(result.content, 'gemini'); + assert.equal(result.finishReason, 'STOP'); + // Streaming must hit :streamGenerateContent with alt=sse, not the unary endpoint. + assert.match(server.seen.urls[0], /:streamGenerateContent\?alt=sse/); + }); + + await suite.test('an SSE frame split across chunk boundaries is still parsed', async (t) => { + const chunks = ['data: {"choices":[{"del', 'ta":{"content":"split"}}]}\n', '\ndata: [DONE]\n\n']; + const server = await jsonServer(t, (request, response) => { + response.writeHead(200, { 'Content-Type': 'text/event-stream' }); + for (const chunk of chunks) response.write(chunk); + response.end(); + }); + const runtime = await providerRuntime(t, server, 'openai-compatible'); + const { deltas, onDelta } = collect(); + assert.equal((await ask(runtime, onDelta)).content, 'split'); + assert.deepEqual(deltas, ['split']); + }); + + await suite.test('no onDelta means the non-streaming path, byte for byte', async (t) => { + const seen = []; + const server = await jsonServer(t, async (request, response) => { + seen.push(await readJsonBody(request)); + const body = JSON.stringify({ choices: [{ message: { content: 'unary' }, finish_reason: 'stop' }] }); + response.writeHead(200, { 'Content-Type': 'application/json', 'Content-Length': Buffer.byteLength(body) }); + response.end(body); + }); + const runtime = await providerRuntime(t, server, 'openai-compatible'); + assert.equal((await ask(runtime, null)).content, 'unary'); + assert.equal(seen[0].stream, undefined); + }); + + await suite.test('streaming: false on a provider falls back to the unary endpoint', async (t) => { + const seen = []; + const server = await jsonServer(t, async (request, response) => { + seen.push(await readJsonBody(request)); + const body = JSON.stringify({ choices: [{ message: { content: 'unary' } }] }); + response.writeHead(200, { 'Content-Type': 'application/json', 'Content-Length': Buffer.byteLength(body) }); + response.end(body); + }); + const runtime = await providerRuntime(t, server, 'openai-compatible', { streaming: false }); + const { deltas, onDelta } = collect(); + assert.equal((await ask(runtime, onDelta)).content, 'unary'); + assert.equal(deltas.length, 0); + assert.equal(seen[0].stream, undefined); + }); + + // Proxies and older local servers accept `stream: true` and answer with one JSON body + // anyway. Reading that as a stream finds no frames and yields a silently empty turn. + await suite.test('an endpoint that ignores stream:true is parsed as a unary body', async (t) => { + const server = await jsonServer(t, (request, response) => { + const body = JSON.stringify({ + choices: [{ message: { content: 'ignored the stream flag', tool_calls: [{ id: 'c1', function: { name: 'fs_read', arguments: '{"path":"z"}' } }] }, finish_reason: 'stop' }], + }); + response.writeHead(200, { 'Content-Type': 'application/json', 'Content-Length': Buffer.byteLength(body) }); + response.end(body); + }); + const runtime = await providerRuntime(t, server, 'openai-compatible'); + const { deltas, onDelta } = collect(); + const result = await ask(runtime, onDelta); + + assert.equal(result.content, 'ignored the stream flag'); + assert.deepEqual(result.toolCalls, [{ id: 'c1', name: 'fs_read', args: { path: 'z' } }]); + assert.equal(result.finishReason, 'stop'); + assert.deepEqual(deltas, [], 'nothing streamed, so no deltas should be reported'); + }); + + await suite.test('a non-JSON body on the streaming path reports the endpoint, not an empty turn', async (t) => { + const server = await jsonServer(t, (request, response) => { + response.writeHead(200, { 'Content-Type': 'text/html' }); + response.end('502 from a proxy'); + }); + const runtime = await providerRuntime(t, server, 'openai-compatible'); + await assert.rejects(ask(runtime, collect().onDelta), /ignored the streaming request/); + }); + + await suite.test('a stream that fails before its first token is still retried', async (t) => { + let attempts = 0; + const server = await jsonServer(t, (request, response) => { + attempts += 1; + if (attempts === 1) { respondFailure(response); return; } + response.writeHead(200, { 'Content-Type': 'text/event-stream' }); + response.write(`data: ${JSON.stringify({ choices: [{ delta: { content: 'after-retry' } }] })}\n\n`); + response.write('data: [DONE]\n\n'); + response.end(); + }); + function respondFailure(response) { + const body = JSON.stringify({ error: { message: 'overloaded' } }); + response.writeHead(503, { 'Content-Type': 'application/json', 'Content-Length': Buffer.byteLength(body) }); + response.end(body); + } + const project = await createProject(t, { git: false }); + const runtime = await runtimeForTest(t, project, { + defaultModel: 'fixture:local', + providerRetry: { attempts: 3, baseMs: 1, maxMs: 4 }, + providers: [{ + id: 'fixture', type: 'openai-compatible', baseUrl: server.url, apiKeyEnv: null, + enabled: true, models: [{ id: 'local' }], timeoutMs: 5000, + }], + }); + const { deltas, onDelta } = collect(); + assert.equal((await ask(runtime, onDelta)).content, 'after-retry'); + assert.equal(attempts, 2); + // The retry happened before any token reached the caller, so nothing was shown twice. + assert.deepEqual(deltas, ['after-retry']); + }); +}); diff --git a/tests/tools.test.mjs b/tests/tools.test.mjs index 941e6e4..315fbf6 100644 --- a/tests/tools.test.mjs +++ b/tests/tools.test.mjs @@ -120,7 +120,7 @@ test('repository indexing blends embedding-based semantic search with lexical se }); const workspace = await runtime.workspaceManager.open(project); - const index = await runtime.indexer.index(workspace.id, { force: true }); + const index = await runtime.indexer.index(workspace.id); assert.ok(index.embeddedChunks >= 1); assert.ok(embedCalls.length >= 1); @@ -133,7 +133,7 @@ test('repository indexing blends embedding-based semantic search with lexical se // Reindexing without content changes must not re-embed unchanged chunks (content-hash reuse). const callsBeforeReindex = embedCalls.length; - await runtime.indexer.index(workspace.id, { force: true }); + await runtime.indexer.index(workspace.id); const statsAfter = runtime.indexer.stats(workspace.id); assert.equal(statsAfter.embeddedChunks, stats.embeddedChunks); assert.equal(embedCalls.length, callsBeforeReindex, 'no new embedding requests for unchanged content'); diff --git a/tests/tui.test.mjs b/tests/tui.test.mjs index d8be90a..ae076e5 100644 --- a/tests/tui.test.mjs +++ b/tests/tui.test.mjs @@ -124,6 +124,47 @@ test('every transcript row shares one left edge', async (t) => { assert.ok(rows.some((line) => line.includes('fs_read'))); }); +test('a streaming turn renders in place of the spinner without breaking the frame', async (t) => { + const project = await createProject(t); + const runtime = await runtimeForTest(t, project); + const app = new MaskShiftTui(runtime, { + workspacePath: project, output: new FakeTerminal(120, 32), headless: true, theme, + }); + await app.bootstrap(); + app.view = 'chat'; + app.activeRun = { id: 'run_fixture', status: 'running' }; + + // No tokens yet: the spinner label still holds the row. + assert.ok(app.snapshot().map(stripAnsi).some((line) => line.includes('Thinking'))); + + app.onRunEvent({ type: 'run.assistant-delta', sessionId: app.sessionId, payload: { text: 'Reading the renderer ' } }); + app.onRunEvent({ type: 'run.assistant-delta', sessionId: app.sessionId, payload: { text: 'and diffing the frame. '.repeat(40) } }); + app.screen.invalidate(); + + const frame = app.snapshot(); + const plain = frame.map(stripAnsi); + assert.ok(plain.some((line) => line.includes('Reading the renderer')), 'streamed text should be on screen'); + assert.ok(!plain.some((line) => line.includes('Thinking')), 'the spinner label gives way to the text'); + // The renderer's contract: every row is exactly the terminal width. + for (const line of frame) assert.equal(visibleWidth(line), 120); + + // A completed turn hands the content to the transcript and clears the live view. + app.onRunEvent({ type: 'run.assistant', sessionId: app.sessionId, payload: { content: 'done', toolCalls: [], streamed: true } }); + assert.equal(app.streamingText, ''); +}); + +test('deltas from a subagent never reach the parent transcript', async (t) => { + const project = await createProject(t); + const runtime = await runtimeForTest(t, project); + const app = new MaskShiftTui(runtime, { + workspacePath: project, output: new FakeTerminal(100, 24), headless: true, theme, + }); + await app.bootstrap(); + app.activeRun = { id: 'run_fixture', status: 'running' }; + app.onRunEvent({ type: 'run.assistant-delta', sessionId: 'ses_some_subagent', payload: { text: 'subagent chatter' } }); + assert.equal(app.streamingText, ''); +}); + test('the active view is named once per screen', async (t) => { const project = await createProject(t); const runtime = await runtimeForTest(t, project);