Open-source dashboards, scripts, and frameworks for measuring phishing simulation program effectiveness.
Most security awareness programs generate reports that look busy but don't answer the question a CFO or CISO actually asks: "Is this working, and is it worth the money?"
This repo gives you the measurement layer to answer that question — whether you're running GoPhish self-hosted or evaluating managed platforms.
Definition: Percentage of recipients who clicked a simulated phishing link in a campaign.
The raw number is almost meaningless. A 14% click rate at month 1 vs. a 14% click rate at month 12 are completely different problems. What matters is the trajectory — is it declining? How fast? Segment by department and tenure to find where it's stuck.
Good signal: Click rate declining 3–5% per quarter across the org.
Red flag: Flat or increasing click rate after 6+ months of training.
Definition: Percentage of recipients who reported a phishing simulation to security rather than just ignoring it or clicking.
This is the metric most programs ignore and the one that best predicts real-world resilience. An employee who reports a phish — even one they almost clicked — is doing exactly what you want. Employees who correctly ignore it but never report it are silent risk.
Target: Report rate should exceed click rate within 12 months of a mature program.
Formula: (Phish Reports / Total Recipients) × 100
Definition: Median time between phishing email delivery and a security report being filed.
Fast time-to-report is operationally valuable. If an employee reports within 15 minutes, your SOC has time to pull the email from inboxes before others click. If reports trickle in after 4 hours, the damage is done.
Target: Median < 30 minutes for mature programs.
Track: P50, P75, P90 — not just the mean (outliers skew it badly).
Definition: Percentage of employees who clicked in 2+ campaigns within a rolling 90-day window.
Single clicks happen. Repeated clicks identify employees who aren't retaining training — whether due to role, cognitive load, or disengagement. These individuals need a different intervention, not more click-through modules.
Formula: (Employees with 2+ clicks in 90 days / Total Employees) × 100
Action threshold: Any department above 8% repeat offenders warrants a targeted response.
Definition: Standard deviation of click rates across departments, normalized against company average.
This surfaces organizational risk concentration. A company-average 6% click rate looks fine until you see that Finance is at 18% and Engineering is at 2%. The aggregate hides the risk.
Use this for: Board reporting, targeted training investment, insurance risk assessments.
Visualization: Heat map by department × campaign type.
| Metric | Why It's Misleading |
|---|---|
| Training completion rate | Tells you who clicked "Next." Tells you nothing about retention or behavior change. |
| Quiz scores | Gameable, momentary. Correlation with real-world behavior is weak. |
| Email open rate | Not a security metric. Also increasingly unreliable due to email client pre-fetching. |
| Campaign volume | More simulations ≠ better outcomes. Quality and spacing matter more than frequency. |
| Average click rate (no segmentation) | Organizational averages hide the departments that need help. Always segment. |
/dashboards/ - Importable Grafana JSON dashboard + Looker Studio and Metabase setup guides
/scripts/ - Python scripts for GoPhish/SAT data export, ROI calculation, variance analysis
/docs/ - Benchmarks, methodology, executive reporting templates
/templates/ - Quarterly review and board deck markdown templates
requirements.txt - Python dependencies (Python 3.9+ required)
All five of the metrics above — click rate trend, report rate, time-to-report, repeat offenders, and department variance — are tracked automatically in HailBytes SAT, with pre-built dashboards and board-ready export.
This repo helps you build them yourself if you're self-hosting GoPhish or HailBytes SAT. The SAT export script pulls from the SAT API and outputs the same format as the GoPhish script, so all downstream analysis scripts work without modification.
If you reach the point where maintaining the measurement infrastructure costs more than the program itself, that's typically when teams move to a managed platform.
pip install -r requirements.txt
# ── GoPhish self-hosted ──────────────────────────────────────────────────────
export GOPHISH_API_KEY=your_key_here
python scripts/gophish-export.py --host https://localhost:3333 --output campaigns.json
# ── HailBytes SAT ────────────────────────────────────────────────────────────
# Find your API key in SAT admin UI → Settings → API
export SAT_API_KEY=your_key_here
python scripts/hailbytes-sat-export.py --host https://sat.yourcompany.com --output campaigns.json
# ── Downstream analysis (same commands for both sources) ─────────────────────
python scripts/calculate-roi.py --data campaigns.json
python scripts/department-variance.py --data campaigns.json
# Optional: adjust the repeat-offender rolling window (default 90 days)
python scripts/department-variance.py --data campaigns.json --window 60Grafana dashboard: Import dashboards/grafana-gophish.json into Grafana 9+. Requires the Infinity datasource plugin. Set the gophish_host and api_key template variables after import.
See /scripts/ for full documentation on each script.
PRs welcome. If you have industry benchmark data, GoPhish dashboard configs, or ROI calculation methodologies to share, open an issue or submit directly.
MIT