Revenue Operations Agent

An AI employee that protects the revenue number. It reads the sales pipeline, detects what changed since the last run, flags the deals and data that need attention, explains why in plain language, recommends the next action, and tracks every decision - and it never touches a record or messages a rep without a human gate.

Built by Akash Kumar. Full design doc and PRD available on request.

Quickstart

python3 -m venv venv
./venv/bin/pip install -r requirements.txt
./venv/bin/uvicorn app.api:app --port 8321
# open http://localhost:8321 and hit "Run review"

No API key required. The LLM layer picks a backend automatically:

Backend	When	What it is
api	ANTHROPIC_API_KEY is set	Anthropic API: claude-opus-4-8 reasoning, claude-haiku-4-5 fast tier
cli	claude CLI installed	Claude Code session (claude -p), Sonnet reasoning, Haiku fast
none	neither available	Deterministic rule-based text - the demo can never die

Force one with REVOPS_LLM=api|cli|none.

How it works: the triage funnel

snapshots (CSV now, CRM sync later)
  -> ingest + data contracts     (schema check, source quality score)
  -> snapshot store              (diff: slips, regressions, amount moves)
  -> rules screen                (deterministic, 100% of deals, playbook-driven)
  -> agent core                  (LLM reasons over the flagged subset only)
  -> calibrated scorer           (owns the probability; the LLM only explains it)
  -> governance                  (autonomy risk matrix, audit trail)
  -> human gate                  (approve / edit / reject with reason codes)
  -> action log                  (resolution tracked, not just detection)
  -> decision memory             (every human decision retrievable next run)

Key design choices:

• Change over time is a first-class signal. Two point-in-time snapshots ship in data/; half the findings (46-day close slip, stage regression, amount halved) are only visible as diffs.
• The LLM never invents the probability. CalibratedScorer is a heuristic today behind the exact interface a model trained on closed won/lost history will use (calibration checked with Brier score in production).
• Rules findings and model findings are visually distinct in the UI (DATA vs MODEL chips) - trust survives the first model mistake.
• Every definition lives in playbook.yaml, versioned: thresholds, quarter-end strictness, the autonomy risk matrix, nudge rate limits.
• Cost per run is measured, not estimated - the run summary reports LLM calls and dollars (about $0.58 for a 14-deal pipeline via the CLI backend).

Evals

./venv/bin/python -m evals.run_evals        # golden set + grounding
./venv/bin/python -m data.gen_synthetic     # generate the regression corpus
./venv/bin/python -m evals.run_synthetic    # regression: exact expected labels

1. Golden set - rules screen precision/recall per flag code against labeled deals (evals/golden.yaml). Currently 1.00 / 1.00.
2. Clean set - deals labeled healthy must not be flagged.
3. Grounding check - every date and dollar amount in an LLM-written reason must exist in the deal's actual input. Invented facts fail the build.
4. Synthetic regression corpus (data/gen_synthetic.py) - seeded deals with issues injected by construction, so expected flags are exact labels, not human guesses. Covers all 12 flag codes including multi-flag stacks; scales to any size (verified at 540 deals, 1.00/1.00 across the board).
5. Volume corpus (data/maven_to_snapshots.py) - 2,089 real-shaped open deals from the public Maven Analytics CRM dataset, converted into snapshot pairs with seeded churn. The deterministic screen covers all 2,089 deals in under 0.1s - the funnel economics hold at volume. The dataset's Won/Lost labels are also the training set the calibrated scorer graduates onto.

Online, the human approval rate in the audit log is the live accuracy metric.

Governance

• Autonomy is a risk matrix per action class (insight / field update / outbound nudge), not one dial. Only low-risk insights auto-clear, and only under the playbook amount limit.
• The human gate has three outcomes: approve, edit (the richest training signal), reject with a reason code (wrong_facts, right_facts_wrong_call, wrong_tone, wrong_timing).
• Everything lands in an append-only audit log: runs, decisions, failures.

Decision memory

Every human decision is joined back to the run context it was made in and retrieved by situation similarity (flag codes + stage + segment) on the next run, then injected into the agent's prompt. Deterministic Jaccard retrieval today; the interface is embedding-shaped so a vector store swaps in.

Honest limits

• Sample snapshots, not a live Salesforce/HubSpot sync.
• Heuristic scorer, not yet trained on real closed won/lost history.
• Activity signal is one column deep (no email/meeting stream).
• Single user, no auth. Actions are drafted and gated, never executed.

Repo map

playbook.yaml        semantic layer: definitions, autonomy matrix, quarter rules
data/                two point-in-time sample snapshots (one corrupt row on purpose)
app/models.py        Deal, ChangeEvent, Flag, Verdict
app/loader.py        ingest + data contracts
app/snapshots.py     snapshot store + diff
app/rules.py         deterministic rules screen
app/scoring.py       calibrated scorer interface + priority ranking
app/llm.py           pluggable LLM backends (api / cli / none)
app/agent.py         orchestration: the triage funnel
app/governance.py    audit trail, action log, reason codes
app/memory.py        decision memory retrieval
app/api.py           FastAPI backend
ui/index.html        triage inbox (keyboard-first)
evals/               golden set + evals harness