Add streaming support and retry logic for model providers

.github/workflows/ci.yml

@@ -0,0 +1,71 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+  workflow_dispatch:
+
+# A queued run on the same ref is superseded by a newer push.
+concurrency:
+  group: ci-${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+permissions:
+  contents: read
+
+jobs:
+  verify:
+    name: verify (node ${{ matrix.node }})
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    strategy:
+      fail-fast: false
+      matrix:
+        # 22 is the floor declared in engines; 24 is the current line.
+        node: ['22', '24']
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-node@v4
+        with:
+          node-version: ${{ matrix.node }}
+
+      # MaskShift has no runtime dependency tree, so there is nothing to install.
+      # Assert that stays true rather than trusting it.
+      - name: Assert zero runtime dependencies
+        run: node scripts/assert-no-dependencies.mjs
+
+      - name: Syntax check
+        run: npm run check
+
+      - name: Unit and integration tests
+        run: npm test
+
+      - name: End-to-end smoke test
+        run: npm run smoke
+
+  docs:
+    name: docs are in sync
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-node@v4
+        with:
+          node-version: '22'
+
+      # docs/ is generated from the runtime. Regenerating it here proves the committed
+      # copy matches the code, and that generation stays hermetic: the runner has no
+      # ~/.claude or ~/.codex skills, so a generator that scanned them would produce a
+      # different file here than on a developer machine, and this step would fail.
+      - name: Regenerate documentation
+        run: npm run docs
+
+      - name: Fail if the committed docs differ
+        run: |
+          if ! git diff --exit-code --stat -- docs/; then
+            echo "::error::docs/ is stale. Run 'npm run docs' and commit the result."
+            exit 1
+          fi

CHANGELOG.md

@@ -2,6 +2,13 @@
 
 ## Unreleased
 
+- **Model output now streams.** Every provider type streams a turn token by token instead of returning it whole: Server-Sent Events for `openai-responses`, `openai-compatible`, `anthropic` and `gemini`, newline-delimited JSON for `ollama`, all read with Node's built-in `fetch` body and `TextDecoder` — no dependency. `maskshift run` writes tokens straight to stdout (and stops repeating the finished answer under the closing rule when it just streamed it), and the interface renders the answer being written where it used to show a spinner, repainting on the ticker it already runs while busy. Tool-call arguments split across frames are reassembled before parsing. Streaming is skipped where it would mislead: the text tool protocol rewrites content after the turn, so it stays unary, and an endpoint that accepts `stream: true` but replies with one JSON body is detected from its content type and parsed normally rather than reporting an empty turn. Control it with `streaming` globally or per provider.
+- **Model requests now survive a transient failure.** A single 429 or dropped connection used to end a run outright — `fetchJson` recorded the status code and no caller ever read it. Requests now retry HTTP 408/409/425/429/500/502/503/504 and network errors with exponential backoff and jitter, honouring `Retry-After`; 400/401/403/404/422 are never retried (the tool-protocol downgrade depends on a 400 surfacing at once), cancelling a run breaks out of the backoff instead of waiting it out, a stream is only retried before its first token so nothing is shown twice, and model-listing probes stay single-shot. Configure with `providerRetry`, or `retry` on one provider.
+- Added `.github/workflows/ci.yml`: `npm run check`, `npm test` and `npm run smoke` on Node 22 and 24, plus a job that regenerates `docs/` and fails if the committed copy differs. Added `npm run deps` (`scripts/assert-no-dependencies.mjs`), which asserts the zero-dependency claim mechanically — no runtime dependency fields in `package.json`, and no bare import specifiers anywhere in `src/`, `bin/`, `scripts/` or `tests/` — and wired it into `npm run verify` and CI.
+- **Fixed `npm run docs` publishing the generating machine's home directory.** `skillsDirs` includes `~/.claude/skills` and `~/.codex/skills` by default, so the generator documented whatever the person running it happened to have installed: `docs/SKILLS.md` listed 44 skills where the repository ships 36, and eight rows carried paths like `../../../root/.claude/skills/synced/<uuid>/docx`. Generation is now scoped to the bundled `skills/` directory, refuses to emit any path outside the repository, and drops the `generatedAt` timestamp from `docs/CAPABILITY-MANIFEST.json` so the output is reproducible and the CI drift check is possible at all. Corrected the README's skill count and badge to 36.
+- Fixed `port_inspect` falling through to a bare `netstat -anp` without checking that netstat exists, which returned a shell "not found" with exit 127 instead of an error. It now reports which inspector ran and whether anything matched, and errors clearly when none is installed. The accompanying test asserted `code === 0` on an unused port, but `ss`, `lsof` and `netstat` all exit non-zero when a filter matches nothing — so the suite failed on any machine, and the failure aborted the scenario before `rsync_transfer` was reached, making the coverage gate blame two tools it had never run. Scenarios now record why they aborted and the gate reports that cause instead.
+- Fixed `VERSION` being hardcoded as `1.0.0` in `src/core/utils.mjs` while `package.json` said `1.0.1`, so every banner, `--version`, `doctor` report, generated document and LSP `clientInfo` announced the wrong version. It is read from the manifest now, and the test that asserted the literal reads the manifest too.
+- Removed the `force` option from repository indexing, including the `repo_index` tool schema and the `workspace index --force` flag. `index()` accepted it and `#runIndex` never declared a second parameter, so it was dropped on the floor at every call site — and since every pass is a full rescan there was never anything for it to force.
 - Published to npm as `maskshift`. `package.json` now carries `files`, `repository`, `homepage`, `bugs` and `author` metadata, and the README documents `npm install -g maskshift` / `npx maskshift` alongside the source-based install paths.
 - **Made the interface mouse-driven as well as keyboard-driven.** Added an SGR mouse decoder (`?1006`, with the legacy X10 encoding as a fallback) to the raw-mode input layer and `src/tui/regions.mjs`, a per-frame hit-test registry: each surface declares the cells it occupies while it paints, so a click resolves against exactly the frame the user was looking at. View tabs, rail tabs, the `TARGET`/`PERSONA`/mode chips, the session title, hint-rail keys, starter prompts, list rows, catalogue tabs and filters, scrollbars and every overlay row, field and button now respond to a click; the wheel scrolls whatever is under the pointer without stealing focus; a click outside an overlay dismisses it. Reporting is switched off whenever the alternate screen is left, so quitting or crashing cannot strand the terminal in tracking mode. `MASKSHIFT_MOUSE=click|hover|off`, the `ui.mouse` setting, `f2` and the `mouse.cycle` palette action all control it, since tracking suppresses the terminal's own text selection (most terminals still select on shift+drag).
 - Fixed the heist view dropping the composer's bottom rule: the transcript and composer were two stacked frames, and the lower one was trimmed to fit the region, so the panel rendered permanently unclosed. They are now one frame split by an internal seam, which also recovers three rows of transcript and removes the doubled rule through the middle of the view.

README.md

@@ -14,7 +14,7 @@
   <img alt="Node 22+" src="https://img.shields.io/badge/node-%E2%89%A522-3ecf8e?style=flat-square">
   <img alt="Runtime dependencies: none" src="https://img.shields.io/badge/runtime%20deps-0-4aa8ff?style=flat-square">
   <img alt="149 tools" src="https://img.shields.io/badge/tools-149-2bd9c0?style=flat-square">
-  <img alt="44 skills" src="https://img.shields.io/badge/skills-44-a78bfa?style=flat-square">
+  <img alt="36 skills" src="https://img.shields.io/badge/skills-36-a78bfa?style=flat-square">
   <img alt="GPL-3.0" src="https://img.shields.io/badge/license-GPL--3.0-e5384f?style=flat-square">
 </p>
 
@@ -42,8 +42,9 @@ runtime dependency tree, no HTTP server, no browser, no listening socket.
 | | |
 |---|---|
 | **149 native tools** | Filesystem, shell and process control, search and indexing, Git worktrees and checkpoints, LSP, browsers over CDP, containers and Kubernetes, SSH and rsync, databases, runtimes, images, PDF and Jupyter, web retrieval, plugins, automations, memory and orchestration. → [tool inventory](docs/TOOLS.md) |
-| **44 bundled skills** | Loaded lazily by description, alongside skills imported from Claude, Codex, Copilot and workspace skill directories. → [skills](docs/SKILLS.md) |
+| **36 bundled skills** | Loaded lazily by description, alongside skills imported from Claude, Codex, Copilot and workspace skill directories. → [skills](docs/SKILLS.md) |
 | **Lazy MCP fabric** | stdio and Streamable HTTP, stateless and legacy initialization, resources, prompts, qualified tools, imported configs, and the live official MCP Registry. Servers connect on demand, so the catalog never floods the context window. → [MCP config](docs/CONFIGURATION.md#mcp-definitions) |
+| **Streaming and resilient** | Turns arrive token by token over SSE or NDJSON from every provider type, and a transient 429, 5xx or dropped connection is retried with jittered backoff instead of ending the run. → [streaming](docs/CONFIGURATION.md#streaming), [retries](docs/CONFIGURATION.md#retries) |
 | **Any model** | Ollama, OpenAI Responses, OpenAI-compatible servers, Anthropic, Gemini, OpenRouter, LM Studio and vLLM — with a text protocol that gives models *without* a native tool API the full tool surface. → [providers](docs/CONFIGURATION.md#providers) |
 | **Autonomous repo context** | Project instructions, manifests, repository tree, indexed code chunks (lexical plus optional semantic retrieval), stored memory, recent history and Git state. → [architecture](docs/ARCHITECTURE.md) |
 | **Parallel agents** | Independent sessions and optional isolated Git worktrees for delegated work. |