Skip to content

feat(monitoring): add production health checks with auto-issue filing - #191

Merged
ytnobody merged 1 commit into
developfrom
hermit/ytnobody/issue-190
Jul 28, 2026
Merged

feat(monitoring): add production health checks with auto-issue filing#191
ytnobody merged 1 commit into
developfrom
hermit/ytnobody/issue-190

Conversation

@ytnobody

Copy link
Copy Markdown
Owner

Summary

  • Adds internal/healthcheck and a new run_health_checks MCP tool (Issue feat(monitoring): add production health checks with auto-issue filing #190): runs each [[health_checks]] command (harness.toml) with a 30s timeout and returns {name, ok, output}[].
  • Failing checks with no existing open issue open a new issue labeled production-incident, deduped by an "[health-check: <name>]" title prefix. Checks that recover get a one-time "recovered at <time>" comment on the matching issue (never auto-closed).
  • Unconfigured projects (health_checks omitted/empty) see run_health_checks as a true no-op — existing behavior is unchanged.
  • Adds health_checks_since to .hermit/superintendent-state.json (get_loop_state/update_loop_state), and a new Superintendent-cycle step (both this repo's CLAUDE.md and the hermit init scaffolding template) that calls run_health_checks on a 5-minute cadence, mirroring how run_requirements_sweep is tracked hourly.
  • Adds mcp__hermit__run_health_checks to .claude/settings.json's permissions allowlist and updates the tools/list count assertions in cmd/hermit/serve_test.go (20 → 21 registered tools).

Assumption / deviation from the Issue text

The Issue body describes the config file as .hermit/config.yaml, but this codebase's actual (and only) MCP-server config file is harness.toml (e.g. [requirements].test_command, [[repos]], etc. — there is no .hermit/config.yaml anywhere in the repo). I implemented [[health_checks]] as a new array-of-tables section in harness.toml, following the exact same config-loading path as [requirements], rather than introducing a second, unprecedented config file. This matches the Issue's own instruction to "踏襲する" the existing test_command config pattern.

Test plan

  • go build ./...
  • go vet ./...
  • gofmt -l . (clean)
  • golangci-lint run ./... (0 issues)
  • go test ./... (all packages pass)
  • New tests cover: RunChecks (pass/fail/timeout/type-skip), Reconcile (new-issue-on-failure, dedup-on-still-failing, recovered-comment-once, no-op-on-passing-with-no-issue), the GitHub adapter (title-prefix matching, issue body contents, label, one-time recovered comment), the run_health_checks MCP tool end-to-end (no-op when unconfigured, per-check result shape, issue-opening, dedup, recovery comment, once-only), and harness.toml parsing of [[health_checks]] (including the omitted-section no-op case).

Closes #190

🤖 Generated with Claude Code

Adds a new internal/healthcheck package and run_health_checks MCP tool
(Issue #190): a project can declare [[health_checks]] command entries
in harness.toml, and the tool runs each one (30s timeout), returning
{name, ok, output} per check. Failing checks with no existing open
issue open a new "production-incident"-labeled issue (deduped by an
"[health-check: <name>]" title prefix); checks that recover get a
one-time "recovered at <time>" comment on the matching issue (never
auto-closed). Unconfigured projects see no change in behavior.

Wires health_checks_since into .hermit/superintendent-state.json
(get_loop_state/update_loop_state) and adds a new Superintendent-cycle
step (both HERMIT's own CLAUDE.md and the scaffolding template) that
calls run_health_checks on a 5-minute cadence, mirroring how
run_requirements_sweep is tracked hourly.

Assumption: the issue body referenced ".hermit/config.yaml" as the
existing test_command config file, but the actual config file in this
codebase is harness.toml ([requirements].test_command) — health_checks
follows the same harness.toml convention as [[repos]] etc.

Closes #190
@ytnobody

Copy link
Copy Markdown
Owner Author

Superintendent review (HIGH risk — manual review before merge)

evaluate_riskがHIGH判定(20+ファイル変更、500+行変更、.claude/settings.json/CLAUDE.md/cmd/hermit/main.goがhigh-riskパス該当)だったため、diffを実際に読んで内容を確認しました。

変更内容の要約:

  • internal/healthcheckパッケージを新設(RunChecks: コマンド実行+30秒タイムアウト+WaitDelayでグランドチャイルドプロセスによるタイムアウト無効化を防止/Reconcile: [health-check: <name>]タイトルプレフィックスでの重複排除起票と、回復時の一度きりrecovered atコメント、close無し)。設計はIssue #190の要件・受け入れ条件を満たしています。
  • run_health_checks MCPツールをinternal/mcp/tools.goに登録し、main.go/server.goで配線。githubClientインターフェースにAddLabelを追加。
  • internal/stateHealthChecksSinceを追加し、get_loop_state/update_loop_state経由でのみ読み書きする既存の設計原則を踏襲。
  • CLAUDE.md(本リポジトリ自身+scaffoldテンプレートCLAUDE.md.tmpl)のSuperintendentサイクルに5分間隔のヘルスチェックステップを追加、以降のステップ番号をリナンバー。
  • .claude/settings.jsonのpermissions.allowへの追記は、新規追加したmcp__hermit__run_health_checksのみで、他の変更なし。

懸念点・特記事項:

  • Issue本文は設定ファイルを.hermit/config.yamlとしていましたが、実際のこのリポジトリの設定はharness.tomlのため、[[health_checks]]をharness.tomlの新規array-of-tablesとして実装しています。これはIssueの「既存のtest_commandパターンを踏襲する」という指示に沿った合理的な判断で、PR説明にも明記されています。
  • テストはinternal/healthcheckのユニットテスト、MCPツールレベルの結合テスト、config parsingテストまで一通り揃っており、受け入れ条件の各項目(重複起票防止・recovered一度きり・close無し・未設定時no-op)がテストでカバーされています。
  • mergeStateStatus: CLEAN / mergeable: MERGEABLEで、developとの間に隠れた意味的コンフリクトを示す兆候はありません。
  • CIは全てpassしています(Security Scan / Lint / Test)。

結論: 内容を確認した限り、安全にマージできる状態に見えます(looks safe to merge pending approval)。ただしHIGH risk判定のため、このパスではマージを見送り、人間の承認を待ちます。

@ytnobody

Copy link
Copy Markdown
Owner Author

⚠️ HERMIT: HIGH risk detected.
Reasons: [20 or more files changed 500 or more lines changed .claude/settings.json is in a high-risk path CLAUDE.md is in a high-risk path cmd/hermit/main.go is in a high-risk path]

@ytnobody
ytnobody merged commit 8fa14e1 into develop Jul 28, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(monitoring): add production health checks with auto-issue filing

1 participant