Skip to content

feat: add review controls, agent usage, security digest - #8

Merged
kacigaya merged 2 commits into
mainfrom
feat/review-controls-health-digest
Nov 3, 2026
Merged

kacigaya merged 2 commits into
mainfrom
feat/review-controls-health-digest

Conversation

@kacigaya

@kacigaya kacigaya commented Nov 3, 2026 •

Copy link
Copy Markdown
Owner

Summary

This PR adds three of the four requested features. The fix priority queue (feature 1) shipped in #7 and is part of v0.5.0. It already sorts by severity, EPSS exploit probability, and patch availability from the Dependabot API.

Review controls

  • New endpoints: POST /api/v1/reviews/{id}/retry and POST /api/v1/reviews/{id}/cancel. Both require can_admin.
  • Retry accepts only the newest failed or cancelled review of an idle pull request. Other cases return 409.
  • A per-PR advisory lock serializes retries and webhook enqueues.
  • Cancel keeps the worker lease until the agent stops, so a retry cannot overlap the old run.
  • The review page shows a plain-language reason and a next step for failed and cancelled reviews. The raw error stays under Error details.
  • Close and draft events now store their own cancel reason. Before, they stored superseded.

Agent health and usage

  • The worker records the agent at start, the agent run time, and the token counts that each CLI reports:
    • Claude: modelUsage
    • Codex: turn.completed. Codex now runs with --json.
    • OpenCode: step_finish
  • When the CLI names the model, the review stores that model.
  • GET /agents returns health, the result of the latest model list run in the sandbox. This check does not test credentials.
  • GET /stats returns per-agent reviews, median agent time, tokens, and models.
  • The overview has a new Agent health and usage card. The review page shows agent time and tokens.

Weekly security digest

  • Admins turn it on in Settings with an HTTPS chat webhook. Slack and Mattermost work directly. Discord works with a URL that ends in /slack.
  • Every 7 days, the worker reads Dependabot alerts with a token limited to alerts and metadata. It reads at most 500 repositories. It posts new, fixed, and overdue alerts, with severity deadlines of 15, 30, 90, and 180 days.
  • The window starts where the last successful read started. A temporary read failure stops the send, and the worker retries every hour.
  • The database stores the webhook URL encrypted with AES-256-GCM. The key comes from the GitHub App private key through HKDF, so a database dump alone does not reveal the URL. The installation ID is bound to each value. Only the host stays in plain text, for display.
  • The API never returns the webhook URL. Errors never contain it.
  • After a GitHub App private key rotation, the settings page shows an error. Enter the URL again.
  • The worker resolves the host once and refuses any private, loopback, link-local, or reserved address. It connects to a checked address on port 443 and follows no redirects.
  • The new direct crates are tokio-rustls and webpki-roots. Both were already in Cargo.lock. OSV reports no advisories for them.

Publication record

  • The worker stores findings before it posts. It marks the posted findings and saves the GitHub review ID in one transaction.
  • A retry keeps the published rows of earlier attempts. Before posting, the worker checks its lease again, so a cancelled review does not post.

Migration

0004_usage_health_digest.sql adds nullable columns to reviews, agent_models, and installations. Run crocode migrate before starting the new API and worker.

Validation

  1. cargo fmt --all --check, cargo clippy --all-targets -- -D warnings
  2. cargo test --all -- --include-ignored on Postgres 17: 62 passed, including new tests for the retry and cancel rules, usage stats, digest claims, URL sealing, and URL privacy.
  3. node --test crates/server/src/osv.test.mjs: 6 passed.
  4. In web/: bun run lint, bun run typecheck, bun test (49 passed), bun run build, bun run build:docs, scripts/check-docs-export.ts, and scripts/smoke.ts. The smoke test covers agent health, the retry action, and digest settings.
  5. Live checks on this host:
    • Claude Code 2.1.310 and Codex 0.166.0 usage output matches the parsers.
    • A digest POST to a fake Slack hook passed DNS, the address check, and TLS, and returned the expected 404.

Known limits

  • I could not check the OpenCode step_finish token shape live, because the free tier hit its limit. The parser follows the documented event fields.
  • A cancel that lands between the final lease check and the GitHub post can still post. The same holds for a URL change between the re-check and the digest send. Both windows are milliseconds long.

- Retry failed or cancelled reviews and cancel queued or running ones
  from the dashboard. Admin only. A per-PR lock and the worker lease
  keep retries from racing pushes or overlapping a stopping run.
- Explain failed and cancelled reviews in plain words. Keep the raw
  error under Error details.
- Record agent time and the token usage that each CLI reports. Codex
  runs with --json. Show harness availability from the model list check.
- Add an opt-in weekly security digest. It posts new, fixed, and
  overdue Dependabot alerts to an HTTPS chat webhook. Delivery connects
  only to public addresses and never exposes the URL.
- Store findings before posting and mark them published with the
  GitHub review ID in one transaction.
Copilot AI balanced review requested due to automatic review settings November 3, 2026 00:26

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@kacigaya kacigaya added bug Something isn't working enhancement New feature or request labels Nov 3, 2026
Seal the URL with AES-256-GCM under an HKDF key derived from the
GitHub App private key. Bind the installation ID as associated data.
Keep only the host in plain text for display. A database dump alone
no longer reveals the URL. After an App key rotation the digest
reports an error and the admin enters the URL again.
@kacigaya
kacigaya merged commit 61153ac into main Nov 3, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants