Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@
各節はそのversion公開時点の変更記録であり、後続versionにより置換された仕様を含む。
現行runtime契約は[`docs/00_overview.md`](https://github.com/kitepon/Spotter/blob/main/docs/00_overview.md)から辿る。

## Unreleased

- Spotter自身が端末内runtime-errorの累計をBugHubへ送るopt-in CLIを追加。署名済み受領だけでackし、既定では通信しない。

## 1.8.1 — 2026-10-03

- Jev監査の大きなtool catalogを入力上限内のリクエストへ分け、全候補の判定とtoken usageを統合する。
Expand Down
22 changes: 18 additions & 4 deletions README.ja.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
project marker、catalog discoveryとhost-local tool DB、評価store、dashboard server、
diagnostics、installer、release packagingを所有します。
[dotagents](https://github.com/kitepon/dotagents)が所有するのは共有agent指示と、
Spotterの端末内runtime-error集計を有効化する任意のfactory-reporter設定です。
旧来の端末内runtime-error集計に使える任意のfactory-reporter設定です。実行時エラーの送信はSpotterが所有します。
Spotterのcatalogやhost統合はdotagentsの責務ではありません。MarkItDownは別区分の第三者CLIです。

Claude には「使えるツールがあるのに、使うべきタイミングで使わない」という構造的な弱点があります。記録すべき決定を memory / caveat MCP に残さない、docs lookup MCP を呼ばずに古い知識で応答する、ブラウザ自動化 MCP で確認せず UI 状態を推測する — **「分からないと自覚できない」から、ツールを取りに行けない**。
Expand Down Expand Up @@ -290,9 +290,10 @@ dashboardの2つのPowerShell actionを非対話・console非表示で起動す

## 端末内runtime error集計

factory diagnosticsとruntime error集計は既定OFFです。canonicalなdotagents factory reporter設定で
JSON booleanの`collection.enabled: true`が明示された場合だけ、固定codeの失敗を端末内へ集計します。
Spotterはreporting credentialもnetwork送信経路も持ちません。保存APIは固定templateとallow-list済み集計だけを受け付け、
factory diagnosticsとruntime error集計は既定OFFです。Spotter専用設定で
`collection.enabled: true`を明示した場合、固定codeの失敗を端末内へ集計します。この設定が無い端末では、
旧dotagents factory reporter設定を収集だけの互換入力として読みます。送信はSpotter専用設定の
`reporting.enabled: true`で別に明示した時だけ行い、既定では通信しません。保存APIは固定templateとallow-list済み集計だけを受け付け、
例外本文、stdout/stderr、stack、prompt、hook payload、finding、ファイル内容、絶対pathを保存しません。

daemonとCodex hookの収集境界は、bounded timeout付きのkill可能なchild process groupで実行します。
Expand All @@ -305,6 +306,19 @@ readbackを検証します。
受理後のlifecycle操作です。`spotter diagnostics logs`と`spotter diagnostics factory`はboundedな件数と
statusだけを返し、store/config pathやrecord本文を出しません。

指揮官の端末で送信を有効にする場合、所有者だけが読める通常ファイル(0600)を
`~/.config/spotter/runtime-errors.json`(Windowsは`%LOCALAPPDATA%\Spotter\runtime-errors-config.json`)
に置きます。内容は`{"schema_version":"1.0","collection":{"enabled":true},"reporting":{"enabled":true}}`です。
BugHub担当が別途、端末ごとの合鍵を`~/.config/bughub/product-credentials/spotter.json`
(Windowsは`%LOCALAPPDATA%\bughub\product-credentials\spotter.json`)へ置きます。
`spotter runtime-errors report`を実行し、継続時はlaunchd、systemd user timer、Windows Task Schedulerで
同じコマンドを1時間に1回起動します。未受領だけを送信し、署名を検証した200応答だけでackします。
配布物の`ops/runtime-reporting/`にlaunchdとsystemd user timerの雛形があります。
`REPLACE_WITH_ABSOLUTE_SPOTTER_PATH`を導入済みCLIの絶対pathへ替えてから登録します。
Windowsでは同じ利用者のTask Schedulerへ`spotter.cmd runtime-errors report`を1時間ごとに登録します。
422が返った版は再送を止め、修正版を導入すると再試行します。無効化は`reporting.enabled`を`false`にします。
401/403で拒まれた合鍵は入れ替わるまで再送を止めます。時計ずれはackせず、時刻修正後に再試行します。

Primary auditor backend policy: Claude hooks の auto selection は PATH に Codex CLI があれば Codex CLI、
なければ Haiku compatibility path。Codex native hooks の auto selection は Codex CLI です。
`SPOTTER_AUDITOR_BACKEND` の明示 override はどちらの host でも優先し、runtime failure では別 backend へ
Expand Down
40 changes: 31 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ This repository owns the complete Spotter product surface: auditor behavior,
Claude/Codex/Cursor/Grok hook adapters, project markers, catalog discovery and host-local
tool databases, evaluation storage, dashboard servers, diagnostics, installers,
and release packaging. [dotagents](https://github.com/kitepon/dotagents)
owns shared agent instructions and the optional factory-reporter configuration
that enables Spotter's local runtime-error aggregate; it does not own Spotter's
owns shared agent instructions and an optional legacy factory-reporter configuration
for local collection; Spotter owns its product runtime-error reporting; dotagents does not own Spotter's
catalog or host integration. MarkItDown is a separate third-party CLI.

Claude has a structural blind spot: **it can't reach for a tool it doesn't realize it needs**. It may skip a project memory MCP when a decision should be recorded, answer from stale memory instead of a docs-lookup MCP, or reason about UI state without a browser-automation MCP. The model can't always tell when it doesn't know — so the tool stays unused.
Expand Down Expand Up @@ -305,17 +305,18 @@ otherwise the Haiku-compatible path. Codex native hooks automatically select Cod
error aggregate は collection が既定OFFです。npm `latest`、tag / GitHub Release、
公開CI、registry由来installを確認済みです。

Spotter collects fixed-code runtime failures only when the canonical dotagents factory reporter
configuration contains the JSON boolean `collection.enabled: true`. Missing, malformed, and disabled
configuration all fail closed. Collection is local-only: Spotter has no reporting credential or network
transport code. The owner-private atomic store contains only fixed templates and allow-listed aggregates;
Spotter collects fixed-code runtime failures when its owner-private product config contains
`collection.enabled: true`. If that config is absent, the old canonical dotagents factory reporter
config remains a collection-only compatibility input. Malformed or disabled product config fails closed.
Reporting is independently opt-in with `reporting.enabled: true`; the default makes no network request.
The owner-private atomic store contains only fixed templates and allow-listed aggregates;
raw exceptions, stdout/stderr, stacks, prompts, hook payloads, findings, file contents, and absolute paths
are not accepted by its API.

Daemon and direct Codex-hook owner boundaries perform collection in a killable child-process group with
a bounded timeout. A blocked FIFO or descendant therefore cannot stall the hook or daemon. HTTP reporting
metadata, when present in the shared config, is accepted only when `new URL(value).href === value` and the
scheme is HTTP(S); Spotter still neither reads credentials nor sends the aggregate anywhere.
a bounded timeout. A blocked FIFO or descendant therefore cannot stall the hook or daemon. The product
report command reads only Spotter's opt-in config and the owner-private BugHub credential, never factory
reporting metadata. It sends only fixed aggregate fields to the configured owner-LAN BugHub endpoint.

On POSIX, every config/store read revalidates the current uid and exact `0600` file / `0700` directory
modes. Store mutations use a private SQLite `BEGIN IMMEDIATE` mutex that the OS releases on process crash;
Expand All @@ -330,6 +331,27 @@ and `spotter diagnostics factory` include only bounded store counts/status, neve
or record payload. Programmatic consumers can import `readRuntimeErrorSnapshot`,
`acknowledgeRuntimeErrors`, `resolveRuntimeError`, `reopenRuntimeError`, and `compactRuntimeErrors`.
Acknowledgement is monotonic, and compaction never removes an unacknowledged record.

To enable owner-LAN reporting on one terminal, create the following `0600` owner-owned regular file
at `~/.config/spotter/runtime-errors.json` (Windows: `%LOCALAPPDATA%\Spotter\runtime-errors-config.json`):

```json
{"schema_version":"1.0","collection":{"enabled":true},"reporting":{"enabled":true}}
```

BugHub's operator supplies a separate `0600` credential at
`~/.config/bughub/product-credentials/spotter.json` (Windows:
`%LOCALAPPDATA%\bughub\product-credentials\spotter.json`). Run `spotter runtime-errors report`
once, then schedule the same command hourly with launchd, a systemd user timer, or Windows Task Scheduler.
The package includes launchd and systemd templates in `ops/runtime-reporting/`; replace
`REPLACE_WITH_ABSOLUTE_SPOTTER_PATH` with the installed CLI path before registering them. On Windows,
register an hourly task for the installed `spotter.cmd runtime-errors report` under the same user account.
Only unacknowledged records are sent. A signed 200 response advances the cursor; timeout, 5xx, or an
unverified response leaves it pending. A 422 blocks further sends for that installed version until a
corrected Spotter version is installed; the block marker is under the private `~/.spotter` state directory
(Windows: `%LOCALAPPDATA%\Spotter`). A rejected 401/403 credential is paused until it is rotated;
clock-skew responses leave the cursor pending for a later attempt. Disable reporting by changing
`reporting.enabled` to `false`.
The Codex SessionStart hook refreshes `.spotter/tool-db.codex.json` in the background
without touching the Claude DB.
Codex CLI auditor child processes use a versioned product policy. The production selection is
Expand Down
6 changes: 6 additions & 0 deletions bin/spotter.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import { runCursorHookCommand } from '../src/cli/cursor-hook-cmd.mjs';
import { runGrokHookCommand } from '../src/cli/grok-hook-cmd.mjs';
import { runAuditorCommand } from '../src/cli/auditor-cmd.mjs';
import { runDiagnosticsCommand } from '../src/cli/diagnostics-cmd.mjs';
import { runRuntimeErrorReportCommand } from '../src/cli/runtime-error-report-cmd.mjs';
import { runEvaluationCommand } from '../src/cli/evaluation-cmd.mjs';
import { runDashboardCommand } from '../src/cli/dashboard-cmd.mjs';
import {
Expand Down Expand Up @@ -53,6 +54,7 @@ Usage:
spotter diagnostics factory emit a fixed-field read-only JSON diagnostic
spotter diagnostics runtime-errors [snapshot|ack|resolve|reopen|compact]
consume the local allow-listed aggregate store
spotter runtime-errors report send pending aggregates to the owner LAN BugHub (opt-in)
spotter evaluation report [--project PATH] [--from ISO] [--to ISO] [--host HOST]
[--tool-id ID] [--backend NAME] [--model NAME]
[--spotter-version VERSION] [--json]
Expand Down Expand Up @@ -151,6 +153,10 @@ async function main() {
}
await runDiagnosticsCommand({ argv: rest });
return;
case 'runtime-errors':
if (rest[0] !== 'report') throw Object.assign(new Error('usage: spotter runtime-errors report'), { exitCode: 2 });
await runRuntimeErrorReportCommand({ argv: rest.slice(1) });
return;
case 'evaluation':
await runEvaluationCommand({ argv: rest });
return;
Expand Down
15 changes: 12 additions & 3 deletions docs/02_spotter-claude-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ Public CLI:
- `spotter diagnostics factory`
- `spotter diagnostics runtime-errors [snapshot [--after-cursor <n>] [--limit <n>]
| ack <cursor> | resolve <fingerprint> | reopen <fingerprint> | compact]`
- `spotter runtime-errors report`
- `spotter evaluation report [--project <path>] [--from <ISO>] [--to <ISO>] [--host <host>]
[--tool-id <id>] [--backend <name>] [--model <name>] [--spotter-version <version>] [--json]`
- `spotter evaluation cases --outcome <outcome> [--project <path>] [--from <ISO>] [--to <ISO>]
Expand Down Expand Up @@ -316,9 +317,10 @@ existing Claude-facing `{pass, missing_tools, reason?}` shape.
- `spotter diagnostics logs` is read-only. It parses daemon log files and reports
`pass:false`, missing-tool counts, duration summaries, catalog-external drops,
role-collapse resets, Haiku failures, and handler errors without changing daemon behavior.
- Runtime error collection is a separate local projection gated only by the canonical dotagents
reporter config's JSON boolean `collection.enabled: true`. Missing/malformed/disabled config does
not create or touch the store. The store performs no network I/O and accepts only fixed Spotter
- Runtime error collection is a separate local projection gated by Spotter's owner-private product
config `collection.enabled: true`. If absent, the canonical dotagents reporter config is a
collection-only compatibility input. Malformed/disabled product config fails closed. The store
performs no network I/O and accepts only fixed Spotter
failure kinds; code, template, component, and severity come from a closed registry rather than
exception/provider/hook input. Fingerprints use the factory-v1 canonical SHA-256 sequence.
- The daemon owns transport and PID-state persistence observations. The daemon owns Claude primary
Expand All @@ -339,6 +341,13 @@ existing Claude-facing `{pass, missing_tools, reason?}` shape.
- `spotter diagnostics runtime-errors` is the read-only allow-listed snapshot. `diagnostics logs` and
`diagnostics factory` expose only bounded collection/store status and counts. Cursor acknowledgement
is monotonic; resolve/reopen advance sequence; compaction preserves all unacknowledged records.
- `spotter runtime-errors report` is an independent one-shot product sender. It requires Spotter's
product config to enable both collection and reporting. It reads the BugHub owner-private credential,
sends at most 500 pending records and 512 KiB to the fixed owner-LAN endpoint, and projects only the
contract's allow-listed fields. The request signs the exact body bytes with HMAC-SHA256. Only HTTP 200
with `accepted:true`, matching `report_id`, and a valid response signature advances the cursor. 422
blocks repeat sends by the same installed version. Rejected credentials pause until rotation; clock
skew does not acknowledge. Hourly OS scheduling is an operator opt-in.

## Evaluation Contract

Expand Down
11 changes: 11 additions & 0 deletions docs/open-issues.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,17 @@ Spotterの現在の未完事項だけを記録する。
- 解決した項目は本文へ残さず、CHANGELOGまたは完了計画へ移す。
- P0は現在のrollout判断、P1は次の運用窓、P2は既知だが未発生のplatform固有リスク。

## P0 — BugHubへの製品報告rollout

Spotter専用のopt-in送信CLIはdraft PR #5で実装済み。BugHubの受け口は本番へ配備済みで、各端末の合鍵は担当者側の承認待ち。
Spotterのreleaseと4端末への適用にはオーナー承認が必要。既存のauditor修正PR #4も
版付きreleaseと4端末への導入まで台帳上は未解決。

### 次の行動

署名試験、製品CI、レビューを通してdraft PRを用意する。BugHubの受け口と合鍵が届いたら、
オーナーの公開・適用承認後に4端末で送信を有効にし、端末・版・送信時刻・受領結果を報告する。

## P1 — Grok監査結果の会話内表示

Grok 1.0.41のnative `UserPromptSubmit` / `Stop` hookはstdoutを会話へ渡さない。
Expand Down
10 changes: 10 additions & 0 deletions ops/runtime-reporting/dev.spotter.runtime-errors.plist
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
<key>Label</key><string>dev.spotter.runtime-errors</string>
<key>ProgramArguments</key><array>
<string>REPLACE_WITH_ABSOLUTE_SPOTTER_PATH</string>
<string>runtime-errors</string><string>report</string>
</array>
<key>StartInterval</key><integer>3600</integer>
</dict></plist>
6 changes: 6 additions & 0 deletions ops/runtime-reporting/spotter-runtime-errors.service
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[Unit]
Description=Send pending Spotter runtime errors to owner-LAN BugHub

[Service]
Type=oneshot
ExecStart=REPLACE_WITH_ABSOLUTE_SPOTTER_PATH runtime-errors report
10 changes: 10 additions & 0 deletions ops/runtime-reporting/spotter-runtime-errors.timer
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[Unit]
Description=Report pending Spotter runtime errors hourly

[Timer]
OnStartupSec=5m
OnUnitActiveSec=1h
Unit=spotter-runtime-errors.service

[Install]
WantedBy=timers.target
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@
"src",
"scripts",
"ops/dashboard",
"ops/runtime-reporting",
"docs/11_dashboard-operations.md",
"README.md",
"LICENSE",
Expand Down
16 changes: 16 additions & 0 deletions src/cli/runtime-error-report-cmd.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
import { reportRuntimeErrors } from '../core/runtime-error-reporter.mjs';

export async function runRuntimeErrorReportCommand({ argv = [], write = (line) => process.stdout.write(line) } = {}) {
if (argv.length !== 0) {
const error = new Error('usage: spotter runtime-errors report');
error.exitCode = 2;
throw error;
}
let result;
try { result = await reportRuntimeErrors(); }
catch {
result = { status: 'report_unavailable' };
}
write(`${JSON.stringify(result)}\n`);
if (!['accepted', 'nothing_to_report', 'disabled'].includes(result.status)) process.exitCode = 1;
}
Loading
Loading