FraudGuard - Agentic Fraud Detection Backend

Submission-ready. Trained model (models/behavior.joblib) and submission file (data/submission_team.csv) are both generated. Latest performance:

Metric	Value
AUROC (eval set)	0.5607
Precision @ 0.5	0.261
Recall @ 0.5	0.106
Baseline gap	~5× above chance (baseline 0.511)

0. Important: Eval Label Restriction

fraud_labels_eval_HIDDEN.csv is RESTRICTED — used for testing and verdict generation ONLY. It must NEVER be used for training. The model is trained exclusively on fraud_labels_train.csv (400k rows).

1. Overview

FraudGuard is a multi-agent, real-time transaction fraud scoring system built for the GIBL AI/ML Hackathon 2026 Track B. Five specialized detection agents run in parallel on every transaction, a synthesis agent combines them with dynamic weights per transaction type, and the verdict (ALLOW, OTP_ONLY, BLOCK) is persisted with a full audit trail.

What it does

• Scores a transaction end-to-end through the live FastAPI service.
• Detects Nepali-banking fraud patterns such as structuring, impossible travel, device compromise, smurfing rings, SIM-swap, dormancy-break anomalies and money-mule graph structures.
• Uses Redis for live velocity/device/profile state, Neo4j for graph risk, PostgreSQL for verdict/audit persistence and a trained ML artifact for the behavior agent.
• Explains every decision with per-agent scores and human-readable reasons.

The five agents

Agent	Engine	Detects
Velocity	Redis sliding windows	bursts, smurfing fan-out, structuring amounts
Geo	Redis location state	impossible travel, long jumps, international risk
Device	Redis fingerprint cache	rooted/Tor/VPN devices, multi-account devices, locale mismatch
Graph	Neo4j + cached graph scores	proximity to known fraud seeds, mule rings
Behavior	LightGBM / BlendModel	behavioral anomalies and learned fraud patterns

More docs:

• Architecture — §2.0 has the basic end-to-end workflow (dashboard click → 5 agents → verdict → OTP interlock)
• API reference
• Guides
• Model evaluation

2. Setup & Run

Prerequisites

• Docker Desktop with Compose v2.
• Dataset files placed flat inside ./data/.
• A trained model at ./models/behavior.joblib.

If ./models/behavior.joblib is missing, the API still starts, but the behavior agent falls back to heuristic mode.

All normal runtime services run in containers:

• FastAPI API: http://localhost:8000
• PostgreSQL: localhost:5432
• Redis: localhost:6379
• Neo4j browser: http://localhost:7474
• Neo4j Bolt: localhost:7687

Windows PowerShell

From the project root:

# 1. Create configuration.
Copy-Item .env.example .env

# 2. Edit JWT_SECRET and ADMIN_PASSWORD.
notepad .env

# 3. Build and start containers.
docker compose up --build -d

# 4. Hydrate Redis caches and load the Neo4j graph from ./data.
docker compose run --rm api python scripts/load_data.py /data

# 5. Check health. Use curl.exe, not PowerShell's curl alias.
curl.exe http://localhost:8000/health

Expected health response:

{"api":"ok","postgres":"ok","redis":"ok"}

Open the dashboard:

Start-Process "http://localhost:8000/"

You can also paste this URL into your browser:

http://localhost:8000/

macOS / Linux

cp .env.example .env
# edit JWT_SECRET and ADMIN_PASSWORD in .env

docker compose up --build -d
docker compose run --rm api python scripts/load_data.py /data

curl -s http://localhost:8000/health
# {"api":"ok","postgres":"ok","redis":"ok"}

Open:

http://localhost:8000/

3. Quick Start

Browser dashboard

Go to:

http://localhost:8000/

Sign in with:

username: admin
password: value of ADMIN_PASSWORD in .env

Then use the form or presets to score a transaction. The dashboard is a single self-contained HTML page (no build step, no CDN) and is fully responsive — it works on a phone screen for judge demos.

Live OTP demo: enter your email (and optionally a Twilio-verified phone) in the contact fields, score a risky transaction, and the OTP panel appears — read the 6-digit code from your inbox, verify it in the UI, and watch the interlock release the transaction (codes are one-time use; a second verify is BLOCKED). The SIM-swap preset shows the escalation path instead.

3.1 Notifications: email primary, SMS secondary

Channel	Role	Latency	Config
SMTP email	Primary — always attempted	API responds in ms (code stored, delivery in background); the SMTP connection is pooled and pre-warmed at startup, so no per-send TLS handshake	EMAIL_FROM, EMAIL_PASSWORD (Gmail App Password)
Twilio SMS	Secondary — parallel, optional	best-effort, hard 8s timeout	SMS_ENABLED=true, TWILIO_*

Phone numbers are auto-normalized to E.164 (9819124921 → +9779819124921) — missing country codes were the cause of the original SMS outage (Twilio error 21211). Trial Twilio accounts can only text numbers verified in the Twilio console, so .env sets TWILIO_TO_OVERRIDE to redirect every SMS to the verified number during demos. Channel state is visible at GET /health ("email": "ok", "sms": "ok") and as chips in the dashboard header; per-challenge delivery is confirmed live in the OTP panel via GET /otp/status. Delivery failures never block or slow scoring — the verdict returns immediately and notifications are fire-and-forget. Tip: the first OTP email from a fresh sender usually lands in Spam; mark it "Not spam" once.

Score with PowerShell

$login = curl.exe -s http://localhost:8000/auth/login `
  -H "Content-Type: application/json" `
  -d '{"username":"admin","password":"ChangeMe123!"}' | ConvertFrom-Json

$token = $login.access_token

curl.exe -s http://localhost:8000/score `
  -H "Authorization: Bearer $token" `
  -H "Content-Type: application/json" `
  -d '{
    "txn_id":"TXN-20260703-A9F3C1AB",
    "timestamp":"2026-07-03 02:14:07.481",
    "account_id":"ACC-0048293",
    "counterparty_id":"MERCH-8812",
    "txn_type":"ESEWA_P2P",
    "amount_npr":49900,
    "channel":"MOBILE_APP",
    "device_id":"DEV-7F3A21",
    "ip_address":"103.10.28.4",
    "latitude":27.71,
    "longitude":85.32
  }'

If you changed ADMIN_PASSWORD in .env, use that password in the login request instead of ChangeMe123!.

Score with macOS / Linux

TOKEN=$(curl -s http://localhost:8000/auth/login \
  -H 'Content-Type: application/json' \
  -d '{"username":"admin","password":"ChangeMe123!"}' \
  | python -c 'import sys,json;print(json.load(sys.stdin)["access_token"])')

curl -s http://localhost:8000/score \
  -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' \
  -d '{
    "txn_id":"TXN-20260703-A9F3C1AB",
    "timestamp":"2026-07-03 02:14:07.481",
    "account_id":"ACC-0048293",
    "counterparty_id":"MERCH-8812",
    "txn_type":"ESEWA_P2P",
    "amount_npr":49900,
    "channel":"MOBILE_APP",
    "device_id":"DEV-7F3A21",
    "ip_address":"103.10.28.4",
    "latitude":27.71,
    "longitude":85.32
  }'

4. Validation & Training

Run the project sanity checks:

docker compose run --rm api python scripts/test_pipeline.py /data /models/behavior.joblib

Expected final line:

ALL 11 TESTS PASSED

Run the OTP + notification unit tests (offline — no Redis/SMTP/Twilio needed; covers the interlock state machine, one-time-code burn, brute-force lockout, SIM-swap escalation, E.164 normalization and the pooled SMTP sender):

python scripts/test_otp_notification.py

Expected final line:

ALL 17 TESTS PASSED

Train the model (already done — models/behavior.joblib exists with AUROC 0.5607):

# Train on fraud_labels_train.csv only (400k rows — eval labels RESTRICTED to test/verdict)
docker compose run --rm api python scripts/train_final.py /data /models/behavior.joblib

Score the training labels for threshold tuning:

docker compose run --rm api python scripts/batch_predict.py /data /models/behavior.joblib /data/fraud_labels_train.csv /data/scored_train.csv

Tune thresholds:

docker compose run --rm api python scripts/tune_thresholds.py /data/scored_train.csv /data/fraud_labels_train.csv 0.028

Copy the printed OTP_THRESHOLD and BLOCK_THRESHOLD values into .env, then restart the API:

docker compose restart api

Generate the submission file (already done — data/submission_team.csv exists):

docker compose run --rm api python scripts/batch_predict.py /data /models/behavior.joblib /data/fraud_labels_eval_HIDDEN.csv /data/submission_team.csv

Validate the submission:

docker compose run --rm api python scripts/validate_submission.py /data/submission_team.csv /data/fraud_labels_eval_HIDDEN.csv

Full workflow details are in docs/GUIDES.md.

5. Useful Docker Commands

# Show running containers
docker compose ps

# Follow API logs
docker compose logs -f api

# Restart API after changing .env or app code
docker compose restart api

# Stop containers without deleting database volumes
docker compose down

# Stop containers and delete database volumes
docker compose down -v

6. Local Python Run

Docker is recommended. If you want to run the API locally, keep Postgres, Redis and Neo4j running through Docker, then run:

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
$env:BEHAVIOR_MODEL="models/behavior.joblib"
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

For local Python, update .env store URLs to local ports:

POSTGRES_DSN=postgresql://fraud:fraudpass@localhost:5432/frauddb
REDIS_URL=redis://localhost:6379/0
NEO4J_URI=bolt://localhost:7687
BEHAVIOR_MODEL=models/behavior.joblib

Repository Layout

app/                FastAPI service: agents, API, security, data access
scripts/            Training, scoring, validation and analysis CLI tools
docs/               Technical documentation
demo/dashboard.html Browser demo UI
data/               Dataset files and generated artifacts, mounted at /data
models/             Trained model artifacts, mounted at /models