AceSAT — Adaptive Multi-Agent SAT Prep

A tireless, personalized SAT tutor for students who don't have access to one.

AceSAT is an AI agent, not a chatbot: it diagnoses a student's weak spots, builds a session-level study plan, actively decides what to teach next, tracks mastery over time, and reflects on progress — all without a human tutor in the loop.

Built solo for the AceSAT Education AI-Agent Hackathon.

---

Live Demo

• Frontend: https://acesat-multiagent.netlify.app/
• Backend API: https://acesat-backend-f9aqgxcpdmbxdcga.centralindia-01.azurewebsites.net
• API docs (Swagger): https://acesat-backend-f9aqgxcpdmbxdcga.centralindia-01.azurewebsites.net/docs
• Source: https://github.com/atifanawaz/acesat-multiagent

Try it as a guest — no account needed to start a diagnostic and see the adaptive loop in action.

---

Why this matters

Students in underserved public schools often lack access to quality SAT prep or personalized tutoring simply because of where they live. AceSAT closes that gap with a system that adapts to this specific student, in real time, at zero marginal cost per student.

---

What makes this an agent, not a chatbot

Most "AI tutor" projects wrap a single LLM call in a nice UI: ask a question, get an answer. AceSAT has a real multi-layer decision-making pipeline:

Planner Agent          — session-level: "what should THIS session accomplish?"
      ↓
Orchestrator           — turn-level: "which subject should the next question come from?"
      ↓
Subject Specialists    — topic-level: "which topic within this subject, and how hard?"
      ↓
Knowledge Graph        — enforces prerequisites (won't teach Geometry before Algebra is solid)
      ↓
Bayesian Knowledge Tracing — real probabilistic mastery estimation, updated after every answer
      ↓
Multi-Armed Bandit     — epsilon-greedy exploration so no topic gets starved
      ↓
Question Generator     — LLM-generated, validated, deduplicated, math-verified
      ↓
Reflection Agent       — a second, separate agent that narrates session progress
      ↓
Database               — every decision logged for real evaluation, not just a demo

The Planner sets one goal per session (e.g. "focus on Algebra, it's weakest") and the Orchestrator executes against that goal with turn-level flexibility — reacting when a different subject becomes the genuinely weakest one, and occasionally reviewing a solid-but-stale topic to guard against forgetting (a real spaced-repetition mechanism, not just a review reminder).

---

Core features

• Diagnostic assessment (student-selectable: 10 or 15 questions) that produces an evidence-based mastery prior, not a flat 30% guess
• Adaptive routing across Math / Reading / Language, with prerequisite-aware topic sequencing that stays responsive to the student's current mastery, not a stale session snapshot
• Bayesian Knowledge Tracing for real per-topic mastery estimation, kept consistent end-to-end — the live session, the database, and the dashboard always agree on the same numbers
• Forgetting-curve decay — mastery erodes slightly for topics left untouched too long, and a maintenance pass reviews them
• 4-tier difficulty progression (easy → medium → hard → very-hard) so a 95%-mastery student isn't served the same difficulty as a 76%-mastery one
• Question validation pipeline: rejects duplicate or malformed choices, repeated/near-duplicate questions (TF-IDF similarity), self-contradictory or leaked-reasoning explanations, and mathematically-impossible questions (symbolic verification via sympy) — with a graceful, always-verified static fallback if live generation fails twice
• SAT Score Predictor — an honest, explicitly-labeled heuristic estimate (not a fake-precision psychometric model)
• Session Reflection Agent — a genuinely separate LLM call that narrates what went well and what needs work, with a deterministic fallback if the API call fails
• Dashboard with mastery overview, predicted score, and session history
• Lightweight authentication (username/email/password, PBKDF2 hashing) with a guest mode that upgrades to a full account without losing progress
• Rate limiting on both question generation and auth endpoints — a sliding-window limiter keyed per-student for generation (protecting the expensive Groq calls) and per-IP for login/register (protecting against enumeration and brute-forcing)
• Structured JSON logging throughout the backend, so every request and decision is machine-parseable, not just readable in a dev terminal — the kind of thing that matters once this is actually deployed, not just demoed
• Real evaluation harness — routing accuracy, mastery improvement, latency, and generation-quality metrics computed from logged data, not eyeballed

---

Architecture

Backend: Python / FastAPI, SQLite (swappable to Postgres via DATABASE_URL) — deployed on Azure App Service (Linux, Python 3.12) Frontend: Single-page HTML/CSS/JavaScript, no framework — deployed on Netlify LLM: Groq API (Llama 3.3 70B) for question generation and session reflection

File structure

backend/
  main.py                FastAPI app, all endpoints
  db.py                  SQLAlchemy models, persistence layer
  models.py               Pydantic request/response schemas
  config.py                Every tunable constant, centralized and documented
  knowledge_graph.py        Topic DAG with prerequisites
  mastery_engine.py         Bayesian Knowledge Tracing + forgetting curve
  bandit.py                 Epsilon-greedy topic selection
  orchestrator.py            Subject-level routing (Planner-aware)
  specialist.py               Subject "teacher" agents
  planner.py                   Session-level goal setting
  diagnostic.py                 Prior estimation from diagnostic results
  llm_service.py                 Question generation + validation pipeline
  score_predictor.py              SAT score heuristic
  reflection_agent.py              Session reflection agent
  rate_limiter.py                   In-memory sliding-window rate limiting
  logging_config.py                  Structured JSON logging
  eval_harness.py                     Offline evaluation metrics
  requirements.txt
  tests/                                Automated test suite (see Testing below)
frontend/
  index.html              Complete frontend

---

Run it locally

cd backend
python -m venv .venv
source .venv/bin/activate       # Windows: .venv\Scripts\Activate.ps1
pip install -r requirements.txt

export GROQ_API_KEY="your_key_here"   # get one free at console.groq.com
uvicorn main:app --reload

Then open frontend/index.html in a browser (backend runs at http://localhost:8000).

All backend files above must be present — main.py imports every one of them directly or transitively; a missing file will crash the server at startup with ModuleNotFoundError.

---

Deployment notes

Deployed as two independent, statically-linked pieces:

• Backend runs on Azure App Service as a single-worker process (uvicorn main:app --host 0.0.0.0 --port 8000). It's deliberately single-worker: active tutoring sessions live in an in-process Python dictionary rather than a shared store, so multiple worker processes would each hold a different, incomplete view of session state. The GROQ_API_KEY is set as an Azure environment variable, never committed to source.
• Frontend is a static deploy on Netlify pointed at the Azure backend URL.
• Database is currently SQLite on the App Service instance — fine for a hackathon demo, but not guaranteed to persist across restarts/redeploys on a free-tier plan. db.py already supports switching to Postgres via DATABASE_URL for anything longer-lived.
• Rate limiting is in-memory and per-process, matching the single-worker deployment above — noted explicitly in rate_limiter.py as the first thing to revisit if this ever runs behind multiple workers.

---

Testing

A pytest suite covers the parts of the system where correctness matters most for an adaptive tutor — not just "does it run," but "does it stay consistent as state changes across a real session":

cd backend
pip install pytest
pytest tests/ -v

What's covered:

• Routing — the Orchestrator picks the lowest-mastery subject, switches when a different subject becomes weaker, and its stated reasoning always matches the exact mastery values used for the decision
• Mastery consistency — mastery updates after every answer, and the live session, the database, and the Dashboard all agree on the same numbers
• Question generation validation — every generated question's correct answer exists among its own options, choices are complete/unique/free of leaked reasoning, and math questions are checked against sympy where the phrasing allows it

Tests run offline and deterministically (no GROQ_API_KEY required, no network calls) by exercising the same static, hand-verified fallback question bank the live app falls back to if generation fails — so the suite is fast and reproducible in CI.

---

An honest note on rigor

A few design choices are worth stating plainly, in the same spirit as the rest of this project:

• The SAT score predictor is a labeled heuristic (linear mastery → score mapping), not a calibrated psychometric model. It says so directly in the UI.
• The diagnostic prior and Bayesian Knowledge Tracing parameters are standard, explainable defaults, not fitted to real student data. They're real, nameable ML/stats ideas (prior estimation, BKT) — just not claiming more precision than they have.
• sympy verification currently covers linear-equation-style algebra/geometry phrasing specifically — other question types rely on structural and explanation-consistency validation rather than full symbolic proof.

I think this honesty is a feature: a system that's clear about what it knows and doesn't know is more trustworthy than one that fakes confidence.

---

Evaluation

Run python eval_harness.py --json against logged usage data for:

• Routing accuracy (does the Orchestrator follow its own rules consistently?)
• Mastery improvement over sessions (descriptive, not causal — small samples are noisy, and I say so)
• Question generation quality (real rejection rate + reason breakdown, not a guess)
• Latency (p50/p95)

---

Built by

Built solo for the AceSAT Education AI-Agent Hackathon, August 2026.