Citationist

Citation Verification System for the TUM course "Foundations and Applications of Generative AI"

A web app that takes a piece of scientific writing — a thesis chapter, a paper section, or an AI-generated paragraph — and tells the user, claim by claim, whether the cited sources actually support what is written.

For each cited claim the user gets:

• A verdict: supported / partially supported / not supported / unclear
• The exact passage from the cited paper that drove the verdict
• A confidence score
• A one-sentence plain-language explanation
• "Possibly more relevant references" — Claude-reranked alternative papers for any claim whose cited source doesn't actually back it

Quick start — for a teammate setting this up fresh

# 1. Clone this repo
git clone <repo-url> GenAI
cd GenAI

# 2. Install Python dependencies
cd code/citationist
/Library/Frameworks/Python.framework/Versions/3.13/bin/python3 -m pip install -r requirements.txt
# (or: pip install -r requirements.txt, if your active Python is 3.13)

# 3. Get an Anthropic API key from https://console.anthropic.com/
#    Copy the template and paste your key in:
cp .env.example .env
# then open .env and replace the placeholder with your real key

# 4. Launch the server
./run.sh             # foreground, Ctrl-C to stop
./run.sh --bg        # background, logs to citationist.log

# 5. Open http://127.0.0.1:8000 in your browser

Try it:

• Click "Run sample" — runs against the bundled test paper with 1 deliberately fabricated reference; see what gets caught.
• Drop a PDF in the upload zone — your own thesis chapter or a paper from code/test_paper/local_pdfs/ (Riad's local set).
• Click "Paste text" — drop in any short paragraph with citations. Optionally expand "Add a bibliography" and paste a reference list in any format (BibTeX, APA, Vancouver, or Author, Year — Title one-liners).

Requirements:

• Python 3.13 — the launcher prefers the framework Python at /Library/Frameworks/Python.framework/Versions/3.13/bin/python3; fallback chain documented in run.sh
• code/citationist/.env with ANTHROPIC_API_KEY=… (template: .env.example)
• Optional: SEMANTIC_SCHOLAR_API_KEY to lift Semantic Scholar rate limits
• Optional for reference-health panel: code/sciwrite-lint/ cloned from upstream — without it, the Reference Health pane stays empty but Citation Health Score + claim cards still work fully

The eval harness (eval/run_eval.py) requires the same .env — running it once is the fastest way to confirm your setup works.

Repo layout

The single source of truth for what's where, why we built it that way, and which features from the proposal are done lives in code/citationist/ARCHITECTURE.md. Read it first.

For the project-wide source of truth (scope, schedule, team roles, working principles) see claude.md.

GenAI/
├── README.md                       # ← you are here (setup + quick start)
├── claude.md                       # project source of truth (scope, plan, team)
├── Group Files/
│   ├── Claim labelling.xlsx        # 52-row eval dataset, 8 domains
│   ├── tum_logo.png                # source asset for UI brand bar
│   ├── Updated proposal_Apr 29.pdf
│   ├── Gen AI Project.pdf
│   ├── GenAI_CourseProject_GradingCriteria.pdf
│   └── architecture.md             # (deprecated; superseded by code/citationist/ARCHITECTURE.md)
├── code/
│   ├── citationist/                # the app
│   │   ├── ARCHITECTURE.md         # engineering source of truth — read first
│   │   ├── server.py               # FastAPI HTTP layer + 7 endpoints
│   │   ├── claims.py               # .bib + .tex parser, OpenAlex resolver
│   │   ├── pdf_ingest.py           # PDF → claim-citation pairs (pypdf path)
│   │   ├── pdf_layout.py           # layout-aware PDF helpers
│   │   ├── verifier.py             # Claude tool-call: judge + rerank
│   │   ├── requirements.txt        # Python deps
│   │   ├── .env.example            # API-key template (copy to .env)
│   │   ├── run.sh                  # start/stop script
│   │   ├── static/                 # vanilla HTML/CSS/JS UI + TUM logo
│   │   └── eval/                   # labelled eval harness (56 pairs)
│   ├── test_paper/                 # bundled .tex/.bib sample
│   │   └── local_pdfs/             # PRIVATE — own papers (gitignored)
│   └── sciwrite-lint/              # third-party clone we wrap (gitignored)
└── .gitignore

Pipeline at a glance

[input PDF / .tex+.bib / pasted text]
        ↓
[1] Parse — pypdf or layout-aware pdfplumber for footnote-style theses
        ↓
[2] Claim detection — sentences carrying in-text citation markers
        ↓
[3] Reference resolution — OpenAlex → Semantic Scholar → CrossRef,
    by DOI then by title (six-stage chain with Jaccard guard)
        ↓
[4] Evidence retrieval — abstract via DOI, fallback to title search
        ↓
[5] Claude judges (claim, abstract) → verdict + confidence + evidence
        ↓
[6] If verdict ≠ supported: Claude reranks 10 OpenAlex candidates by
    claim-driven relevance, returns top 3 with one-sentence "why each fits"
        ↓
[7] Aggregate → Citation Health Score + Citation Integrity / Claim
    Support sub-scores + severity-sorted claim cards + dangling-citation
    + dangling-cross-reference banners

Differentiators from sciwrite-lint and other lint tools

See Group Files/architecture.md §5 — short list:

• Web UI for non-technical users (sciwrite-lint is CLI)
• AI-text stress-test mode (paste a paragraph, expose hallucinated citations live)
• Three-source DOI chain + claim-driven title-search fallback
• Layout-aware PDF parser with body↔footnote linkage
• APA-aware bibliography parser
• Claim-driven "stronger sources" recommender with Claude rerank — lint tools flag problems; Citationist also suggests fixes with one-sentence reasoning per pick
• Citation Integrity vs Claim Support sub-scores (separates retrieval failures from entailment failures)

Team

7-person team, see claude.md §9 for role assignments. Tutor: Emilija.