I run an independent AI red teaming practice. Before an AI feature, copilot, or agent ships, I attack it the way adversaries and misusing users will (jailbreaks, direct and indirect prompt injection, data exfiltration, tool misuse, harmful outputs) and hand back findings a stranger can reproduce, severity by impact, fixes, and a retest. The engagement model follows the practice Microsoft's AI Red Team has published; I am independent and not affiliated with Microsoft.
Work with me: one workflow, two business days, $3,500 · one hour of advice, $500, booked online · all services
The same discipline runs through the code below: a technical claim is not evidence, and every repository is built so the evidence can be reproduced from a clean clone.
- validate claims deterministically before anyone acts on them;
- let agents act only through signed tool definitions, under a deny-wins policy;
- record every model call and tool call in an encrypted, append-only log;
- retain nothing by default.
| Repo | What it proves | Status |
|---|---|---|
| assay | Multi-model continuous security-validation harness: provider routing with failover and budgets, Ed25519-signed tools verified against a trust store, a deny-wins policy engine, an AES-256-GCM hash-chained audit log, deterministic checkers that mark each finding validated, refuted or theorized, and zero data retention as a tested property. 17 architecture decision records. | v0.3.0 released; CI runs it against three authorized labs. No multi-model result published yet |
| edge-inference-bench | Reproducible method for benchmarking small open-weight models on Apple silicon: llama.cpp tuning sweep, context-length curve, MLX comparison, with the measurement conditions recorded beside every result | Method and scripts published; no results yet |
| tasia | Fail-closed configuration review for private-AI stacks (Go); evidence never carries secret values by construction | v0.1.1 |
| orbit-ir | Reconciles AI-agent transcripts against control-plane records; findings carry file and line references; precision and recall only when ground truth exists | v0.2.0 |
| phylaram | Live physical-memory acquisition for Windows with byte-accurate error isolation and an independent offline verifier | alpha |
- Signed tools and the trust store:
internal/toolsig/toolsig.go - Deterministic validation of findings:
internal/validate - Transcript reconciliation:
internal/reconcile/reconcile.go - Design decisions:
docs/adr - Committed runs, each linked to the CI run that produced it:
results
- volatility3 (PR #2045, open): makes linked-list walks report an unreadable node instead of silently returning a shorter list. Forensic parsers should fail closed.
- Signed tool definitions for MCP (draft): RFC 8785 canonical signing, a trust-store key id, and four verification reason codes, with assay as the reference implementation. A draft for discussion, not an adopted proposal.
Field notes at joeyvictorino.com/field-notes. Start with: