Skip to content

feat: make captured traffic and monitoring signals trustworthy - #24

Merged
biancarosa merged 3 commits into
mainfrom
feat/trustworthy-signals
Sep 21, 2026
Merged

biancarosa merged 3 commits into
mainfrom
feat/trustworthy-signals

Conversation

@biancarosa

@biancarosa biancarosa commented Sep 20, 2026 •

Copy link
Copy Markdown
Collaborator

Netkit's first UX-roadmap deliverable makes captured traffic safe to inspect and its signals meaningful enough to support monitoring. Previously, history could retain credentials, a failed refresh could look like an empty capture, HTTP status was hidden behind a generic Error label, and /metrics returned a constant request count of zero.

This change sanitizes records before retention, separates transfer outcome from upstream/proxy status, defines timing and capture scope, and exports process-lifetime counters and histograms that survive history eviction and clearing. Dashboard fetch failures preserve the last good results with a stale-data warning. Drafts are tab-local and legacy persisted drafts are discarded without hydration.

Included

  • Recursive JSON/header redaction with additional --redact-fields, URL credential/query masking, bounded inspection capture, and explicit omission for opaque, incomplete or >64 KiB bodies. Forwarded traffic is unchanged.
  • Complete, interrupted, client-canceled, inspection-blocked, upstream-error, proxy-error and opaque-tunnel outcomes; bounded TLS/timeout failure reasons.
  • Real request/exchange counters, handler-lifetime and time-to-response-headers histograms, and a retained-record gauge. Metric dimensions are bounded and contain no URLs, hosts, paths or credentials.
  • Clear retained-buffer/time-window and HTTPS inspection/tunnel scope, honest timing labels, separate loading/empty/unavailable/stale states, and sanitized copy actions.
  • Regression fixtures and a dashboard safety/build CI job.

Validation

  • go test -race -tags=unit ./...
  • go test -count=1 -race -tags=e2e ./test/e2e (fresh race-enabled binary built for each suite; Makefile disables result caching)
  • golangci-lint run
  • Dashboard: npm run test:unit (6 tests), npx tsc --noEmit, npx eslint src, npm run build
  • Production dashboard exercised against a local proxy and synthetic upstream: original response reaches client, stored/admin data excludes synthetic secrets, 401 remains separate from Complete, details display sanitized fields, failed refresh preserves stale rows, recovery shows an empty capture, and a draft disappears after reload.
  • Clipboard content could not be read back through the in-app browser; unit/API tests verify the sanitized data supplied to copy actions.

Compatibility and limits

Captured non-JSON/partial/oversized bodies are now omitted; structured JSON is normalized and sensitive fields masked. The field/pattern policy is not a general secret detector: custom credentials need explicit fields/policy changes. No raw-capture reveal/export is provided. Original network forwarding is unchanged.

The constant netkit_proxy_status metric is removed; use the scraper's up signal for reachability. Existing success means transfer completion, including HTTP errors, and the legacy proxy_overhead_us field is explicitly described as other elapsed time (including body transfer), not CPU overhead. CONNECT measures establishment only; pre-inner-request inspection handshake failures are not individual exchange records. Metrics reset on process restart and do not yet provide host/route dimensions or alert rules.

Policy and API semantics are documented in docs/trustworthy-capture.md. This PR does not deploy or change the live instance.

@biancarosa
biancarosa merged commit ddaf3d8 into main Sep 21, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant