reportwise — Lab Report Explanations With Rules First

reportwise is a lab report assistant for patients and clinicians. It takes a lab report PDF through these steps to a validated, cited explanation and a follow-up chat:

extract → parse → flag → scrub → retrieve → explain → validate → chat.

Summary · Workflow · Run it · Configuration · Known problems · Glossary

Note

This README uses ASD-STE100 Simplified Technical English. The writing rules and the project vocabulary are in docs/ste-style-guide.md. Each term in the Glossary has only one meaning.

Warning

Do not use reportwise to make a medical decision. It gives information, not a diagnosis, and it never recommends a medicine. A doctor must review every result.

---

reportwise reads the lab values of a report and flags them with fixed rules before any model sees them. The LLM only explains the flags, and it gets a de-identified JSON block, not the report. A validator then makes the explanation agree with the rules, removes any medicine or dose, and keeps only citations to the curated knowledge base. If the LLM fails, a rules-only explanation with a clear label replaces it.

This README is the one location that explains all of reportwise. It gives these topics:

• the general design
• each component and its procedure, step by step
• the decision rules
• the data map
• the runbook
• the validation results and the known problems

If you are…	Read
A manager or reviewer	1, 3, 4, 12, 14
A developer who joins the project	All sections, in sequence. Keep 10 and 13 open while you work
An operator who runs reportwise	10, then the section for the component that you use

---

Table of contents

1. 🧭 Summary
2. 🏗️ How reportwise is built
   • 2.1 Components
   • 2.2 System context
   • 2.3 Repository layout
3. 🛡️ Design rules
4. 🔄 The end-to-end workflow
   • 4.1 Full flow
   • 4.2 The life cycle of one report
   • 4.3 Who does which step
5. 🔵 Ingestion and de-identification
6. 🟢 Rules and the knowledge base
7. 🟣 Explanation, chat and the profile model
8. ⚖️ The validator and the safety rules
9. 🗂️ Data and file map
10. ▶️ How to run reportwise
    • 10.1 Prerequisites · 10.2 Installation · 10.3 Run reportwise · 10.4 Environment variables
11. 🧩 How to extend reportwise
12. ✅ Validation results
13. ⚠️ Known problems
14. 📌 Key points
15. 📖 Glossary
16. 📄 License

---

1. Summary

The problem. A model that reads a whole lab report can invent an abnormal value or recommend a medicine. It can also obey an instruction in the report text. These questions are difficult:

• Which values are outside the range, and who decides that?
• Which text leaves the machine, and does it contain names or IDs?
• Can the output name a medicine or a dose?
• What does the user see when the model fails?

reportwise gives each of these questions its own component. Rules flag the values, a scrubber removes identifiers, one structured LLM call explains the flags, and a validator checks the result.

Item	Value
Input	A lab report PDF with a text layer (OCR optional)
Output	Results table with flags, explanation with cited sources, critical values, follow-up chat, JSON
Components	8: ingestion, de-identification, rules, knowledge base, LLM clients, analyzer and validator, chat, accounts
Providers	OpenAI-compatible chat or Gemini, both optional. --no-llm keeps all data on the machine
Offline mode	Synthetic reports, template explainer, golden-set evaluation, latency benchmark
Safety	Rule flags, one data block, medicine filter, source check, labelled fallback, hashed passwords
Tests	34 unit tests (pytest), no skips

flowchart LR
    IN["lab report PDF"] --> A["lab table + rule flags"] --> B["scrubbed data block"] --> C["one LLM call"] --> D["validator"] --> OUT["explanation + sources + chat"]

Loading

---

2. How reportwise is built

2.1 Components

Component	Module	Purpose
Settings	src/reportwise/config.py	Read and check environment variables. Hide keys in repr
PDF	src/reportwise/ingest/pdf.py	Text layer with pypdf, OCR fallback, a small PDF writer for synthetic reports
Lab parser	src/reportwise/ingest/lab_parser.py	Lab lines to LabResult, units, ranges, H/L marks
De-identification	src/reportwise/ingest/deid.py	Replace names, IDs, dates, phones, e-mails, addresses
Rules	src/reportwise/rules/	18 tests, aliases, units, typical ranges, critical limits, conversions
Knowledge base	src/reportwise/knowledge.py, src/reportwise/kb/*.md	19 curated files, 38 chunks, MedlinePlus source links, TF-IDF search
LLM clients	src/reportwise/llm/	OpenAI and Gemini JSON clients, retry and cache, template explainer
Schemas and validator	src/reportwise/schemas.py	Explanation, ChatReply, medicine pattern, validate_explanation()
Analyzer	src/reportwise/analyzer.py	The pipeline, the data block, the fallback
Chat	src/reportwise/chat.py	Follow-up questions, refusals, one history entry per question
Accounts	src/reportwise/auth.py	PBKDF2-SHA256 password hashes, no default account
Evaluation	src/reportwise/evaluate.py	Golden set, PHI leak count, stage latency benchmark
Profile model	src/reportwise/ml.py	Leak-free disease-from-profile model on the synthetic health table
CLI	src/reportwise/cli.py	The reportwise command
App	src/reportwise/app/streamlit_app.py	Optional Streamlit app with login and chat

The component map shows which module calls which module. An arrow points from the caller to the module that it uses.

flowchart TB
    subgraph ENTRY["Entry points"]
        CLI["cli.py<br/>reportwise command"]
        APP["app/streamlit_app.py<br/>login and chat"]
    end
    subgraph PIPE["Analysis pipeline"]
        AN["analyzer.py<br/>Analyzer, build_payload"]
        PDF["ingest/pdf.py<br/>extract_text"]
        LP["ingest/lab_parser.py<br/>parse_report"]
        DE["ingest/deid.py<br/>scrub"]
        RU["rules/<br/>flag_value, specs"]
        KB["knowledge.py<br/>KnowledgeBase"]
        LLM["llm/<br/>OpenAIJson, GeminiJson, RetryingLLM,<br/>TemplateExplainer"]
        SC["schemas.py<br/>validate_explanation"]
    end
    subgraph AFTER["After the analysis"]
        CH["chat.py<br/>ChatSession"]
        AU["auth.py<br/>UserStore"]
    end
    subgraph OFFLINE["Offline tools"]
        CFG["config.py<br/>Settings"]
        SYN["synthetic.py<br/>make_report"]
        EV["evaluate.py<br/>eval_golden, benchmark"]
        ML["ml.py<br/>evaluate_table"]
    end

    CLI --> CFG
    CLI --> AN
    CLI --> LLM
    CLI --> CH
    CLI --> AU
    CLI --> SYN
    CLI --> EV
    CLI --> ML
    APP -- "make_analyzer, render" --> CLI
    APP --> AU
    APP --> CH
    AN --> PDF
    AN --> LP
    AN --> DE
    AN --> KB
    AN --> LLM
    AN --> SC
    LP --> RU
    SC --> RU
    CH --> DE
    CH --> KB
    CH --> LLM
    CH --> SC
    EV --> AN
    EV --> SYN
    SYN --> PDF
    SYN --> RU

Loading

2.2 System context

flowchart TB
    U["patient or clinician"] --> UI["CLI / Streamlit app (login)"]
    UI --> AN["Analyzer"]
    AN --> KB["knowledge base (in the package)"]
    AN --> LLM["OpenAI / Gemini (optional) or template explainer"]
    AN --> C["cache/ (optional LLM reply cache)"]
    UI --> USERS["users.json (hashed passwords)"]

Loading

2.3 Repository layout

reportwise/
├── src/reportwise/
│   ├── config.py          settings
│   ├── analyzer.py        pipeline and fallback
│   ├── schemas.py         output schemas and validator
│   ├── chat.py            follow-up chat
│   ├── knowledge.py       knowledge base loader and search
│   ├── auth.py            user accounts
│   ├── evaluate.py        golden set and benchmark
│   ├── ml.py              leak-free profile model
│   ├── synthetic.py       synthetic reports
│   ├── cli.py             reportwise command
│   ├── ingest/            pdf.py, lab_parser.py, deid.py
│   ├── rules/             reference_ranges.json, flag rules
│   ├── kb/                19 Markdown knowledge files
│   ├── llm/               clients, retry, cache, template explainer
│   └── app/               streamlit_app.py
├── data/README.md         data rules and sources
├── docs/ste-style-guide.md
├── tests/                 pytest suite (offline)
└── .github/workflows/ci.yml

---

3. Design rules

3.1 Rules flag, the model explains

parse_line() flags each value with the report range, or with the default range when the report has none. The validator removes any finding for a test that the rules did not flag. It puts back the rule flag when the model gives a different one.

flowchart LR
    LINE[/"Lab line"/] --> PL["parse_line<br/>flag_value"]
    PL --> FLAGS[("Rule flags<br/>one for each test")]
    FLAGS --> PAY["Data block"]
    PAY --> LLM["LLM explains<br/>the flags"]
    LLM --> VAL["validate_explanation"]
    FLAGS --> VAL
    VAL --> OUT[/"Finding flag = rule flag.<br/>No finding for a test that the rules did not flag"/]

Loading

3.2 Data and instructions stay apart

The system prompt has the instructions. The user message has only one JSON block inside <report_data> tags. The system prompt says that the block is data. An instruction in the report text stays inside the block, and the validator still applies.

3.3 Identifiers do not leave the machine

The LLM gets the parsed lab table, the knowledge chunks and the scrubbed non-lab lines. The scrubber replaces names, IDs, dates, phone numbers, e-mail addresses and address lines. --no-llm sends nothing.

3.4 No medicine in any output

The validator removes each sentence that names a medicine, a dose or a prescription. It also removes a verb such as "take" with a medicine word. The chat refuses medicine questions before retrieval.

3.5 Every statement has a source

Each finding and habit must cite a retrieved knowledge chunk. The validator removes other citations and adds the chunk of the test when a finding has none.

3.6 A failure is labelled, not hidden

The LLM client has a timeout, retries with back-off and an optional cache. If the call fails or the reply fails the schema, the analyzer uses the rules-only explanation, sets used_fallback and records the reason in issues.

3.7 Real accounts

There is no built-in user. reportwise add-user stores a salted PBKDF2-SHA256 hash (310,000 iterations). Passwords must have 12 or more characters.

flowchart TD
    ADD[/"reportwise add-user name"/] --> UN{"User name has 3 to 32<br/>lower-case letters, digits, . _ -?"}
    UN -- "no" --> ERR[/"ValueError, exit code 2"/]
    UN -- "yes" --> PW{"Password has 12<br/>or more characters?"}
    PW -- "no" --> ERR
    PW -- "yes" --> HASH["hash_password: 16-byte random salt,<br/>PBKDF2-SHA256, 310,000 iterations"]
    HASH --> USERS[("users.json<br/>REPORTWISE_USERS_FILE")]
    LOGIN[/"App login form"/] --> ANY{"users.json has<br/>one or more users?"}
    ANY -- "no" --> NOACC[/"Error: run reportwise add-user first"/]
    ANY -- "yes" --> VER["UserStore.verify:<br/>constant-time hash compare"]
    USERS --> VER
    VER --> OK{"Hash matches?"}
    OK -- "yes" --> IN[/"Session user set,<br/>PDF upload shown"/]
    OK -- "no" --> NO[/"Wrong user name or password"/]

Loading

---

4. The end-to-end workflow

4.1 Full flow

flowchart TD
    P[/"Lab report PDF"/] --> X["extract_text: pypdf text layer"]
    X --> TL{"Text layer has 10 or more<br/>characters per page?"}
    TL -- "no, OCR off" --> OE[/"NeedsOCRError, exit code 3"/]
    TL -- "no, REPORTWISE_OCR=tesseract" --> O["OCR: pdf2image + pytesseract"]
    TL -- "yes" --> L["parse_report: test, value,<br/>unit, range, H/L"]
    O --> L
    RJ[("rules/reference_ranges.json<br/>18 tests")] --> L
    L --> ANY{"Lab results found?"}
    ANY -- "no" --> NE[/"ValueError, exit code 2"/]
    ANY -- "yes" --> F["Rule flags<br/>report range wins"]
    L --> N["Non-lab lines,<br/>1500 characters or less"] --> S["scrub: names, IDs, dates,<br/>phones, e-mails, addresses"]
    F --> K["KnowledgeBase.for_test:<br/>chunks of flagged tests + reference ranges"]
    KBF[("kb/*.md<br/>38 chunks")] --> K
    F --> B["build_payload: one JSON block<br/>in report_data tags"]
    S --> B
    K --> B
    B --> NL{"--no-llm?"}
    NL -- "yes" --> FB["fallback_explanation:<br/>rules only"]
    NL -- "no" --> M["One JSON LLM call<br/>timeout, retry, cache"]
    CA[("cache/<br/>optional reply cache")] <--> M
    M --> OK{"Call succeeds and reply<br/>passes the Explanation schema?"}
    OK -- "no" --> FB
    OK -- "yes" --> V["validate_explanation"]
    FB -- "used_fallback = true" --> V
    V --> OUT[/"Results table, critical values,<br/>explanation, sources, disclaimer"/]
    OUT --> CH["ChatSession.ask: refusals, retrieval,<br/>one history entry per question"]
    OUT --> HUMAN{{"HUMAN<br/>a doctor reviews every result"}}
    CH --> HUMAN

    classDef human fill:#fff3cd,stroke:#b8901f,color:#3d2f00,font-weight:bold
    class HUMAN human

Loading

4.2 The life cycle of one report

stateDiagram-v2
    state "PDF file" as Pdf
    state "Report text" as Text
    state "Lab results with rule flags" as Parsed
    state "Scrubbed notes" as Scrubbed
    state "Knowledge chunks retrieved" as Retrieved
    state "LLM explanation" as LlmExp
    state "Rules-only explanation" as Fallback
    state "Validated analysis" as Validated
    state "Chat session" as Chat
    [*] --> Pdf: user gives a report
    Pdf --> NotPdf: suffix is not .pdf
    Pdf --> NeedsOCRError: no text layer and OCR off
    Pdf --> Text: extract_text
    Text --> NoResults: no lab line parsed
    Text --> Parsed: parse_report
    Parsed --> Scrubbed: scrub non-lab lines
    Scrubbed --> Retrieved: kb.for_test
    Retrieved --> LlmExp: LLM reply passes the schema
    Retrieved --> Fallback: no LLM, LLM error or schema error
    LlmExp --> Validated: validate_explanation
    Fallback --> Validated: validate_explanation, used_fallback
    Validated --> Chat: chat command or app
    Validated --> [*]
    Chat --> [*]
    NotPdf --> [*]
    NeedsOCRError --> [*]
    NoResults --> [*]

Loading

1. extract_text() reads the text layer. An empty layer raises NeedsOCRError, unless OCR is on.
2. parse_report() reads each lab line and keeps the first line of each test.
3. Each result gets a flag from the report range or from the default range.
4. scrub() cleans the lines that are not lab lines.
5. The analyzer gets the knowledge chunks of each flagged test and the chunk about reference ranges.
6. The analyzer sends one data block to the LLM.
7. The reply must pass the Explanation schema.
8. validate_explanation() makes the reply agree with the flags and the sources.
9. The output shows the results, critical values, findings, habits, questions and sources.
10. The chat session keeps the analysis and answers follow-up questions.

4.3 Who does which step

sequenceDiagram
    autonumber
    actor U as User
    participant CLI as reportwise CLI
    participant AN as Analyzer
    participant PDF as ingest/pdf.py
    participant LP as lab_parser and rules
    participant DE as deid.scrub
    participant KB as KnowledgeBase
    participant LLM as RetryingLLM
    participant API as OpenAI or Gemini API
    participant VA as validate_explanation

    U->>CLI: reportwise analyze report.pdf
    CLI->>CLI: Settings.from_env, make_analyzer
    CLI->>AN: analyze_pdf(path)
    AN->>PDF: extract_text(path, ocr)
    PDF-->>AN: PdfText
    AN->>LP: parse_report(text)
    LP-->>AN: LabResult list with rule flags
    AN->>DE: scrub(non-lab lines)
    DE-->>AN: scrubbed notes and counts
    AN->>KB: for_test(each flagged test)
    KB-->>AN: knowledge chunks
    AN->>LLM: complete_json(SYSTEM, data block)
    LLM->>API: POST, timeout, retry with back-off
    API-->>LLM: JSON reply
    LLM-->>AN: dict, or LLMError after the last retry
    AN->>AN: Explanation schema, or fallback_explanation
    AN->>VA: explanation, rule flags, retrieved sources
    VA-->>AN: fixed explanation and issues
    AN-->>CLI: Analysis
    CLI-->>U: results, explanation, sources, disclaimer

Loading

---

5. Ingestion and de-identification

Purpose. Turn a PDF into a table of lab results and a scrubbed text.

flowchart TD
    F[/"File path"/] --> SUF{"Suffix is .pdf?"}
    SUF -- "no" --> E1[/"ValueError:<br/>only PDF files are accepted"/]
    SUF -- "yes" --> RD["PdfReader:<br/>extract_text of each page"]
    RD --> LEN{"10 or more characters<br/>per page on average?"}
    LEN -- "yes" --> T1[/"PdfText, used_ocr false"/]
    LEN -- "no" --> OCR{"REPORTWISE_OCR"}
    OCR -- "tesseract" --> TS["pdf2image at 300 dpi,<br/>pytesseract on each page"]
    TS --> T2[/"PdfText, used_ocr true"/]
    OCR -- "none" --> E2[/"NeedsOCRError, exit code 3"/]

Loading

Input	Output
A PDF report	LabResult list, scrubbed notes, replacement counts

Procedure

1. Refuse a file that is not a PDF.
2. Read the text layer. If it has fewer than 10 characters per page, use OCR or raise NeedsOCRError (exit code 3).
3. Match each line to the lab line pattern. Map the printed name to a test by the longest alias.
4. Read the value, the qualifier (<, >), the unit, the range (low - high, <limit, >limit) and the H/L mark.
5. Flag the value. If the report has no range, convert the default range to the unit of the report.
6. If the report mark (H/L) disagrees with the rule flag, set flag_mismatch.
7. Scrub the lines that are not lab lines (1500 characters or less).

The diagram shows how parse_line() reads and flags one line.

flowchart TD
    L[/"One report line"/] --> M{"Line matches<br/>the _LINE pattern?"}
    M -- "no" --> SKIP[/"Not a lab line"/]
    M -- "yes" --> T{"match_test: an alias<br/>at the start of the name?"}
    T -- "no" --> SKIP
    T -- "yes" --> R{"Range printed<br/>on the report?"}
    R -- "low - high, or limit" --> RR["range_source report"]
    R -- "no" --> CV{"No unit, or a unit<br/>with a known conversion?"}
    CV -- "yes" --> DR["Default range in the report unit<br/>range_source default"]
    CV -- "no" --> UNK["flag unknown<br/>range_source none"]
    RR --> FV["flag_value with the<br/>critical limits"]
    DR --> FV
    FV --> HL{"Report H/L mark differs<br/>from the rule flag?"}
    HL -- "yes" --> MM["flag_mismatch = true"]
    HL -- "no" --> OUT[/"LabResult"/]
    MM --> OUT
    UNK --> OUT
    OUT --> FIRST["parse_report keeps the<br/>first line of each test"]

Loading

The scrubber applies its rules in this order. Each rule counts its replacements.

flowchart LR
    IN[/"Non-lab lines"/] --> N["Labelled names<br/>[NAME]"]
    N --> A["Address lines<br/>[ADDRESS]"]
    A --> I["Labelled IDs<br/>[ID]"]
    I --> E["E-mail addresses<br/>[EMAIL]"]
    E --> D["Dates<br/>[DATE]"]
    D --> P["Phone numbers<br/>[PHONE]"]
    P --> OUT[/"Scrubbed notes and<br/>replacement counts"/]

Loading

Identifier	Rule	Placeholder
Name	After Patient Name:, Name:, Dr:, Physician:, Ordered by:, Reviewed by: and others	[NAME]
ID	After MRN, Patient ID, ID, Accession, SSN, Insurance	[ID]
Address	After Address:	[ADDRESS]
Date	03/04/2026, 2026-03-04, Mar 4, 2026, 4 March 2026	[DATE]
Phone	10-digit numbers with separators	[PHONE]
E-mail	name@domain.tld	[EMAIL]

---

6. Rules and the knowledge base

Purpose. Give each result a flag and give each flag a cited explanation source.

flowchart TD
    IN[/"value, low, high,<br/>critical low, critical high"/] --> NR{"No low and<br/>no high limit?"}
    NR -- "yes" --> UNK[/"unknown"/]
    NR -- "no" --> CL{"Below the<br/>critical low limit?"}
    CL -- "yes" --> FCL[/"critical_low"/]
    CL -- "no" --> CH{"Above the<br/>critical high limit?"}
    CH -- "yes" --> FCH[/"critical_high"/]
    CH -- "no" --> LO{"Below the low limit?"}
    LO -- "yes" --> FL[/"low"/]
    LO -- "no" --> HI{"Above the high limit?"}
    HI -- "yes" --> FH[/"high"/]
    HI -- "no" --> FN[/"normal"/]

Loading

Input	Output
LabResult	Flag, critical status, knowledge chunks

Flag	Rule
critical_low	Below the critical low limit (only some tests)
low	Below the low limit
normal	In the range
high	Above the high limit
critical_high	Above the critical high limit (only some tests)
unknown	No range and no known unit conversion

Test with critical limits	Critical low	Critical high	Unit
hemoglobin	7.0	20.0	g/dL
wbc	2.0	30.0	10^3/uL
platelets	50	1000	10^3/uL
glucose	54	400	mg/dL
sodium	125	155	mmol/L
potassium	3.0	6.0	mmol/L

The other 12 tests are hba1c, total cholesterol, ldl, hdl, triglycerides, creatinine, egfr, alt, ast, tsh, vitamin d and ferritin. Unit conversions exist for glucose, the four lipid tests, creatinine and hemoglobin.

The knowledge base has one Markdown file per test plus reference-ranges.md. Each file has a What it means section and an Everyday habits section. Each section is one chunk with an id such as ldl#1. The front matter gives the MedlinePlus page that the text is based on. For a free chat question, a TF-IDF search finds the 3 nearest chunks.

flowchart LR
    MD[/"kb/*.md in the package,<br/>or REPORTWISE_KB_DIR"/] --> FM["_parse: front matter<br/>id, test, title, source"]
    FM --> SP["One chunk for each<br/>level-2 heading"]
    SP --> CH[("38 chunks<br/>id = file id + section number")]
    CH --> TF["TfidfVectorizer<br/>1 and 2 word grams"]
    CH --> FT["for_test: exact test name<br/>for flagged tests"]
    TF --> SE["search: k = 3,<br/>score above 0 only"]
    FT --> OUT[/"Chunks for the data block"/]
    SE --> OUT2[/"Chunks for a free<br/>chat question"/]

Loading

---

7. Explanation, chat and the profile model

Purpose. Explain the flags, answer follow-up questions and keep the old disease model honest.

Input	Output
Results, chunks, scrubbed notes, questions	Explanation, chat Turn objects, profile model metrics

Procedure (explanation)

1. Build the data block: results, notes, chunks.
2. Call the LLM once with temperature 0.
3. Validate the reply with the Explanation schema and the validator.
4. If the call or the schema fails, use the rules-only explanation.

The LLM client for openai and gemini is a RetryingLLM. The diagram shows one call.

flowchart TD
    REQ[/"System prompt and data block"/] --> CK{"REPORTWISE_CACHE_DIR set and<br/>a file for the prompt hash exists?"}
    CK -- "yes" --> HIT[/"Cached JSON reply"/]
    CK -- "no" --> CALL["OpenAIJson or GeminiJson:<br/>POST with REPORTWISE_TIMEOUT_S"]
    CALL --> ERR{"HTTP, key or<br/>JSON error?"}
    ERR -- "no" --> SAVE["Write the cache file<br/>if a cache folder is set"]
    SAVE --> OUT[/"JSON object"/]
    ERR -- "yes" --> LEFT{"Retries left?<br/>REPORTWISE_MAX_RETRIES"}
    LEFT -- "yes" --> WAIT["Wait 1 s, 2 s, 4 s ..."]
    WAIT --> CALL
    LEFT -- "no" --> FAIL[/"LLMError: failed after<br/>all attempts"/]
    FAIL --> FB["Analyzer: rules-only explanation,<br/>reason in issues"]

Loading

Procedure (chat)

1. If the question describes an emergency, answer with the emergency text.
2. If the question asks for a medicine or a dose, refuse.
3. If the question is not about the report or lab tests, refuse.
4. Get the chunks of the tests named in the question, or search the knowledge base.
5. Call the LLM with the scrubbed question, the results, the last 4 turns and the chunks.
6. Keep the answer only if it cites a retrieved chunk. Remove medicine sentences.
7. Add exactly one turn to the history.

flowchart TD
    Q[/"Question"/] --> EMP{"Empty?"}
    EMP -- "yes" --> VE[/"ValueError"/]
    EMP -- "no" --> EM{"Emergency words?"}
    EM -- "yes" --> K1[/"emergency"/]
    EM -- "no" --> MED{"Medicine or<br/>dose request?"}
    MED -- "yes" --> K2[/"medication_refused"/]
    MED -- "no" --> SC{"Report word or<br/>test name?"}
    SC -- "no" --> K3[/"out_of_scope"/]
    SC -- "yes" --> NT{"Names a test<br/>in the report?"}
    NT -- "yes" --> FT["kb.for_test of<br/>each named test"]
    NT -- "no" --> SE["kb.search, k = 3"]
    FT --> PAY["Scrub the question. Data block:<br/>results, last 4 turns, chunks"]
    SE --> PAY
    PAY --> LLM{"LLM reply passes<br/>the ChatReply schema?"}
    LLM -- "no LLM, error<br/>or schema error" --> NC{"Any chunk?"}
    NC -- "no" --> K4
    NC -- "yes" --> FBK["First chunk<br/>as the answer"]
    LLM -- "yes" --> FIL["Remove medicine sentences.<br/>Keep retrieved sources only"]
    FBK --> FIL
    FIL --> SRC{"A source is left?"}
    SRC -- "no" --> K4[/"no_source"/]
    SRC -- "yes" --> K5[/"answer + disclaimer"/]
    K1 --> H["history: add one Turn"]
    K2 --> H
    K3 --> H
    K4 --> H
    K5 --> H

Loading

LLM	When	Behaviour
TemplateExplainer	REPORTWISE_LLM=fake (default)	Writes the explanation from the chunks in the data block. It does not reason
OpenAIJson	REPORTWISE_LLM=openai	response_format JSON, temperature 0, seed 7
GeminiJson	REPORTWISE_LLM=gemini	generateContent with a system instruction and JSON output. The key goes in a header

Profile model. ml.py predicts the disease from profile columns only. It refuses the symptom and recommendation columns and keeps classes with 20 or more rows. The split is stratified, and a Pipeline holds the one-hot encoding and the scaling. The output gives macro-F1 next to a majority baseline. --show-leak also trains with the Symptoms column, to show the leak.

flowchart LR
    SRC[/"--csv table, or<br/>make_health_table"/] --> CF{"check_features:<br/>a forbidden column?"}
    CF -- "yes" --> LE[/"LeakageError"/]
    CF -- "no, or allow_leaky" --> KEEP["Keep classes with 20 or more rows.<br/>Drop rows with a missing number"]
    KEEP --> SPL["train_test_split<br/>80 / 20, stratified"]
    SPL --> PIPE["Pipeline: StandardScaler +<br/>OneHotEncoder, RandomForest 200 trees"]
    SPL --> BASE["DummyClassifier<br/>most_frequent"]
    PIPE --> MET[/"accuracy, balanced accuracy,<br/>macro-F1"/]
    BASE --> BMET[/"baseline accuracy<br/>and macro-F1"/]

Loading

---

8. The validator and the safety rules

validate_explanation() runs on every explanation, from the LLM or from the fallback. The diagram shows its checks in sequence.

flowchart TD
    IN[/"Explanation, rule flags,<br/>retrieved chunk ids"/] --> EACH["For each finding"]
    EACH --> FL{"Rules flagged<br/>the test?"}
    FL -- "no" --> RM["Remove the finding,<br/>record an issue"]
    FL -- "yes" --> DUP{"Test seen before?"}
    DUP -- "yes" --> SKIP["Skip the second finding"]
    DUP -- "no" --> SAME{"Flag = rule flag?"}
    SAME -- "no" --> SET["Use the rule flag,<br/>record an issue"]
    SAME -- "yes" --> MED["remove_medication_sentences"]
    SET --> MED
    MED --> SRC["Keep retrieved sources only.<br/>None left: add the test chunk"]
    SRC --> MISS["Add a plain finding for each<br/>flagged test with no finding"]
    RM --> MISS
    SKIP --> MISS
    MISS --> HAB["Habits: remove medicine sentences,<br/>remove a habit with no valid source"]
    HAB --> SUM["Summary: remove medicine sentences.<br/>Questions: remove a medicine question"]
    SUM --> OUT[/"Fixed Explanation<br/>and the issues list"/]

Loading

Check	Action	Recorded in issues
Finding for a test that the rules did not flag	Remove the finding	Yes
Flag in the reply differs from the rule flag	Use the rule flag	Yes
Flagged test with no finding	Add a plain finding with the test's chunk	Yes
Source not in the retrieved chunks	Remove the source. Add the test's chunk if none is left	Yes
Sentence with a medicine, a dose, prescri… or "take/start/stop + medicine word"	Remove the sentence	Yes
Habit with no valid source	Remove the habit	Yes
Question for the doctor that names a medicine	Remove the question	No
LLM error or schema error	Rules-only explanation, used_fallback = true	Yes

Chat kind	When
emergency	Chest pain, "cannot breathe", fainted, suicidal words, severe bleeding
medication_refused	Dose, prescription, "which medicine", "should I take" and the medicine pattern
out_of_scope	No report word and no test name
no_source	The answer cites no retrieved chunk
answer	All other questions

CLI exit code	Meaning
0	Success
2	Input, configuration or LLM error
3	The PDF has no text layer and OCR is off

---

9. Data and file map

Path	Committed?	Contents
src/reportwise/rules/reference_ranges.json	Yes	18 tests, aliases, units, ranges, critical limits, conversions
src/reportwise/kb/*.md	Yes	19 knowledge files with source links
data/README.md	Yes	Data rules
data/synthetic/	No (git ignores it)	Synthetic PDFs and truth.jsonl
data/Merged_Health_Dataset.csv	No (git ignores it)	Optional synthetic health table
*.pdf	No (git ignores it)	Reports
cache/	No (git ignores it)	Optional LLM reply cache. Delete it after use with real reports
users.json	No (git ignores it)	Hashed passwords
.env	No (git ignores it)	Credentials

---

10. How to run reportwise

10.1 Prerequisites

Need	For
Python 3.11+	All components
An OpenAI or Gemini API key	REPORTWISE_LLM=openai or gemini
ocr extra, Tesseract and Poppler	Scanned reports
app extra	The Streamlit app

10.2 Installation

git clone https://github.com/KrishnaAnnavaram/reportwise.git
cd reportwise
python -m venv .venv
. .venv/bin/activate            # Windows: .venv\Scripts\activate
pip install -e ".[dev]"

10.3 Run reportwise

Offline demo:

reportwise demo --out demo_out

Offline, step by step:

reportwise synth --out data/synthetic --n 20
reportwise analyze data/synthetic/rep-0000.pdf
reportwise analyze data/synthetic/rep-0000.pdf --no-llm --json
reportwise eval --n 30
reportwise bench --runs 20
reportwise kb
reportwise ml-eval --show-leak

With an LLM and the web app:

export REPORTWISE_LLM=gemini      # or openai
export GEMINI_API_KEY=...         # Windows PowerShell: $env:GEMINI_API_KEY = "..."
reportwise analyze my-report.pdf
reportwise chat my-report.pdf
reportwise bench --pdf my-report.pdf --runs 10
reportwise add-user clinic.user
streamlit run src/reportwise/app/streamlit_app.py

Command	Result
reportwise analyze <pdf>	Results table, explanation, sources (--json, --no-llm)
reportwise chat <pdf>	Terminal chat about one report
reportwise synth	Synthetic PDF reports and truth
reportwise eval	Golden-set metrics on synthetic reports
reportwise bench	p50 and p95 latency for each stage
reportwise add-user	Adds or updates a user in users.json
reportwise ml-eval	Leak-free profile model metrics (--csv, --show-leak)
reportwise kb	Lists the knowledge chunks and sources
reportwise demo	The full offline workflow

The diagram shows the order of the commands and the files that connect them.

flowchart LR
    INS["pip install -e .[dev]"] --> DEMO["reportwise demo<br/>analysis, 2 chat questions,<br/>golden set, latency"]
    INS --> SYN["reportwise synth"]
    SYN --> DIR[("data/synthetic/<br/>rep-0000.pdf ..., truth.jsonl")]
    DIR --> AN["reportwise analyze<br/>reportwise chat"]
    INS --> EV["reportwise eval<br/>reportwise bench"]
    INS --> KB["reportwise kb<br/>reportwise ml-eval"]
    INS --> ADD["reportwise add-user"]
    ADD --> USERS[("users.json")]
    USERS --> APP["streamlit run<br/>src/reportwise/app/streamlit_app.py"]

Loading

10.4 Environment variables

Variable	Used by	Meaning
REPORTWISE_LLM	Analyzer, chat	fake (default), openai or gemini
REPORTWISE_MODEL	LLM	Default gpt-4o-mini (OpenAI) or gemini-2.5-flash (Gemini)
OPENAI_API_KEY	OpenAI client	Key
GEMINI_API_KEY	Gemini client	Key (GOOGLE_API_KEY is also read)
REPORTWISE_OPENAI_BASE_URL	OpenAI client	Default https://api.openai.com/v1
REPORTWISE_TIMEOUT_S	LLM	HTTP timeout. Default 30
REPORTWISE_MAX_RETRIES	LLM	Retries after the first call. Default 2
REPORTWISE_CACHE_DIR	LLM	Reply cache folder. Default: no cache
REPORTWISE_KB_DIR	Knowledge base	Your own Markdown folder. Default: the package files
REPORTWISE_OCR	PDF	none (default) or tesseract
REPORTWISE_USERS_FILE	App, add-user	Default users.json

Credentials are only in a local .env file. Git ignores this file. Do not print or commit credentials.

The diagram shows how make_llm() uses the settings to select the LLM client.

flowchart LR
    ENV[/"Environment variables"/] --> S["Settings.from_env"]
    S --> CHK{"REPORTWISE_LLM and<br/>REPORTWISE_OCR valid?"}
    CHK -- "no" --> ERR[/"ValueError, exit code 2"/]
    CHK -- "yes" --> NL{"--no-llm?"}
    NL -- "yes" --> NONE["No LLM:<br/>rules-only explanation"]
    NL -- "no" --> P{"REPORTWISE_LLM"}
    P -- "fake" --> T["TemplateExplainer"]
    P -- "openai" --> O["OpenAIJson<br/>OPENAI_API_KEY, base URL"]
    P -- "gemini" --> G["GeminiJson<br/>GEMINI_API_KEY or GOOGLE_API_KEY"]
    O --> KEY{"Key set?"}
    G --> KEY
    KEY -- "no" --> LERR[/"LLMError, exit code 2"/]
    KEY -- "yes" --> R["RetryingLLM<br/>retries and optional cache"]
    T --> A["Analyzer"]
    R --> A
    NONE --> A

Loading

---

11. How to extend reportwise

You want to…	Do this	Code change?
Use another model	Set REPORTWISE_MODEL	No
Use a local OpenAI-compatible server	Set REPORTWISE_LLM=openai and REPORTWISE_OPENAI_BASE_URL	No
Add a lab test	Add it to reference_ranges.json and add kb/<test>.md	No
Use your own knowledge files	Set REPORTWISE_KB_DIR	No
Add a medicine word to the filter	Add it to DRUG_WORDS in schemas.py	Small
Add a lab line format	Change _LINE in lab_parser.py and add a test	Small
Use an identity provider	Replace _login() in the app	Yes

---

12. Validation results

Validation	Result	Command
Unit tests	34 passed, 0 skipped (also in CI)	pytest -q
Golden set, 30 synthetic reports, 300 lab lines	test recall 1.000, value 1.000, unit 1.000, flag 1.000, PHI leaks 0, invented findings 0	reportwise eval --n 30
Latency, template explainer, no network (20 runs)	total p50 2.3 ms, p95 3.2 ms	reportwise demo
Profile model on the synthetic 14,512-row table (49 classes kept, 24 dropped)	leak-free: accuracy 0.292, macro-F1 0.180 (baseline 0.087 and 0.003)	reportwise ml-eval --csv ... --show-leak
Same table, with the leaky Symptoms column	accuracy 0.975, macro-F1 0.934	reportwise ml-eval --csv ... --show-leak

The diagram shows how eval_golden() gets the golden-set numbers.

flowchart LR
    N[/"n reports, seed 100"/] --> MR["make_report: known values,<br/>flags and fake identifiers"]
    MR --> AT["analyze_text with a RecordingLLM<br/>that keeps every prompt"]
    AT --> CMP["Compare the results with the truth:<br/>test, value, unit, flag"]
    AT --> INV["Count findings for tests that are<br/>normal or not in the report"]
    AT --> PHI["Search the recorded prompts for the<br/>fake name, MRN, date and phone"]
    CMP --> OUT[/"test_recall, value_accuracy,<br/>unit_accuracy, flag_accuracy"/]
    INV --> OUT2[/"invented_findings"/]
    PHI --> OUT3[/"phi_leaks"/]

Loading

The golden set comes from the same author as the parser. Thus it shows that the parts work together, not that the parser reads every real report layout. The latency numbers measure the real code path with the offline explainer. With a hosted LLM, the llm stage takes most of the time. Measure it with reportwise bench --pdf. The profile model results show that the old high scores came from the leaky columns. Profile columns alone predict the synthetic disease label poorly.

---

13. Known problems

Read these problems before you use reportwise in production.

#	Area	Problem	Impact and action
1	Results	No result with a hosted LLM or with real reports is in CI	Run reportwise eval and bench with your provider and de-identified reports
2	Parser	Only one line per result is read. Tables split over columns or lines are missed	Missed tests are not flagged. Check results against the report
3	Ranges	The default ranges are typical adult ranges. Age, sex and laboratory change them	A value can be flagged wrong when the report has no range
4	Scrubber	The rules can miss a name in free text	Review the scrubber, or use --no-llm for real reports
5	Medicine filter	The filter uses a fixed word list and patterns	A rare medicine name can pass. Extend DRUG_WORDS
6	Knowledge base	19 short files, written for this project	Clinicians must review the text before real use
7	OCR	OCR needs Tesseract and Poppler and is not tested in CI	Scanned reports can fail to parse
8	Accounts	Local accounts only, no rate limit, no audit log	Put the app behind an identity provider for real use
9	Use	This is not a medical device	Do not use it for diagnosis or treatment. A doctor must review every result

---

14. Key points

1. Rules make the flags. The LLM cannot add or change an abnormal result.
2. One data block, one call. Instructions and report data stay apart, and the model gets no identifiers.
3. No medicine in any output. The validator and the chat refusals enforce it.
4. Every statement cites a source. Each source is a reviewed knowledge chunk with a MedlinePlus link.
5. Failures are visible. A rules-only explanation replaces a failed LLM call and says so.
6. The old disease model was a leak. Without the symptom columns, macro-F1 falls from 0.934 to 0.180.

---

15. Glossary

Term	Meaning
Critical value	A result with the flag critical_low or critical_high
De-identification	The removal of names, IDs, dates, phones, e-mails and addresses
Disclaimer	The fixed text at the end of each output
Explanation	The validated summary, findings, habits and questions
Fallback	The rules-only explanation
Finding	The explanation of one flagged test
Flag	critical_low, low, normal, high, critical_high or unknown
Knowledge chunk	One section of a knowledge file, with an id such as ldl#1
Lab line	One report line with a test, a value, a unit and an optional range
Macro-F1	The mean F1 score over all classes
Payload	The JSON block inside <report_data> tags
PHI	Protected health information
Range source	report, default or none
Reference range	The low and high limits for a test
Result	The parsed LabResult of one test
Source	The id of a knowledge chunk that an answer cites
Test	A canonical lab test name
Validator	The code that makes the explanation agree with the rules

---

16. License

MIT © 2026 Krishna Annavaram