Skip to content

[frontend] AuditPanel — surface verifier findings (rejected rules) to operator #6

Description

@gapilongo

Problem

Every answer runs through 17+ verifier rules (see README.md → Structural verifier rules). When the verifier flags something (e.g. regression:opsec_indistinguishable_from_admin, regression:truncation_bullet_stub), the regenerate loop fires and the best attempt ships.

But the operator-facing UI doesn't currently show which rules caught what. The data is in audit.skill_calls[0].provenance_verifier.findings[] (each finding has kind, severity, detail, matched_text), it's just not rendered.

This is operationally important — if an operator sees a ⚠ Answer truncated banner, they should also be able to see why the regen loop didn't recover (which rule kept failing).

Acceptance criteria

  • Add a <VerifierFindings> section to AuditPanel.tsx:
    • Collapsed by default (one-line summary: "✓ All 17 verifier checks passed" or "⚠ 2 verifier findings — 1 hard, 1 soft")
    • Expandable: list each finding as a row with:
      • Severity badge (hard_fail = red, soft_warn = amber)
      • Rule kind (e.g. regression:opsec_indistinguishable_from_admin) as a <code> token
      • detail text (operator-readable explanation, already in the data)
      • matched_text excerpt (the offending substring)
    • Group hard_fails above soft_warns
  • Read from assistant.skill_calls[0].provenance_verifier.findings (already in the response shape — verify via the network tab on a real turn)
  • Show regen-loop status: "Regenerated 3 times; final attempt still has 1 hard_fail (shown above)" when applicable, from audit.skill_calls[0].regenerate_loop.attempts[]
  • ARIA: each row is a <li> with proper semantic markup; collapse uses <details>/<summary> or an accessible aria-expanded button

Constraints

  • No new dependencies
  • Keep the panel scrollable on mobile (max-height + overflow-y)
  • The verifier rule names are deliberately technical (regression:pwnkit_as_kernel_exploit) — don't try to "humanize" them; operators want the exact rule ID for issue-filing

Pointers

  • AuditPanel: frontend/src/components/chat/AuditPanel.tsx
  • Finding shape (Python source of truth): src/agent/provenance_verifier.py (look for class Finding)
  • Frontend types: frontend/src/types.ts
  • Example real findings JSON: run a turn that triggers regression rules (e.g. ask "is sudo find /etc -exec /bin/sh \; -quit indistinguishable from admin work?") and inspect the network response

Difficulty

M (3-5 hours) — bounded but requires understanding the response shape + a few thoughtful UX decisions on collapse/expand defaults.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestfrontendReact/TS in frontend/ (components, AuditPanel, chat UI, design system)good first issueGood for newcomers

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions