Optional AI review, and which models to use at what cost

README.md

@@ -4,7 +4,7 @@ Use accessible HTML to tag and update a PDF.
 
 [Iris](https://github.com/EqualifyEverything/equalify-iris) turns page images into accessible HTML. This tool takes that HTML and the original PDF, and gives back **the same PDF, tagged**: a structure tree a screen reader can follow, with form fields filled in if you give it values. The page looks exactly as it did.
 
-It runs offline. It makes no network or model calls.
+Tagging runs offline and makes no network or model calls. An optional `review` asks a Claude model to check the result.
 
 ## Install
 
@@ -25,6 +25,7 @@ The sources are TypeScript, run directly by Node. Node does not do that inside `
 iris-pdf tag --pdf in.pdf --pages pages.json --out out.pdf --report report.json
 iris-pdf fields --pdf in.pdf [--json]
 iris-pdf check --pdf out.pdf          # runs veraPDF's PDF/UA-1 check, if installed
+iris-pdf review --pdf out.pdf [--report review.json]   # optional AI review, below
 ```
 
 `tag` options:
@@ -76,28 +77,37 @@ Then two checks run, and if either fails nothing is written (exit 2):
 
 `--report` writes JSON: per page, where the text came from and how many words matched; the structure written; fields set and skipped; the check results; and every warning. Warnings name what could not be done, for example `unmatched_text` (page text missing from the HTML, kept as a paragraph), `missing_alt`, `field_not_in_html`, `unmatched_link`, `duplicate_text_layer`, `page_not_in_html` and `page_not_tagged` (the page is left as it was; a blank page needs no HTML and is not warned), `no_title`, `font_not_embedded` (a source font has no embedded program, which PDF/UA-1 requires; the source drawing is not changed), `source_marked_content` (the page drawing has marked-content ids left from an earlier tag tree), `alignment_incomplete` (the page and the HTML differ too much to match every word in time; the rest is kept as unmatched text).
 
+## Review
+
+`review` checks what `check` cannot: whether the tags say what the page says. For each page it sends a Claude model the page image and what a screen reader gets from the page: the structure, text, alt text, link targets and field names. It reports missing content, wrong reading order, wrong element types or heading levels, tables, alt text, link text, field names and language. It prints one finding per line, writes them to `--report` as JSON with the tokens used and an estimated cost, and exits 0. A page the model could not review is reported with its error, the other pages are kept, and the exit is 1. Nothing in the PDF is changed.
+
+**It sends page images and text to the model provider.** With `ANTHROPIC_API_KEY` set, it uses the Anthropic API. Otherwise it uses Amazon Bedrock through the AWS CLI, with your AWS credentials and region. Choose with `--provider anthropic|bedrock` and `--model <id>`. The default model is Opus 5.5, at about US$0.03 a page. See [docs/models.md](docs/models.md) for the models compared and their costs.
+
+## PDF/UA
+
 The output declares PDF/UA-1 only when it has a title, every page is tagged, every source font is embedded, and no page drawing has leftover marked content. The tests check each such claim with veraPDF.
 
 ## Refusals and exit codes
 
 | Exit | When |
 |---|---|
 | 0 | Done. |
-| 1 | Refused: `encrypted` (no or wrong password), `permissions_denied`, `too_many_pages` (over 25), `too_many_words` (over 4000 on a page), `already_tagged`, `xfa` (dynamic form), `signed`, `no_acroform_field`, `no_text_positions`, `strict`. |
+| 1 | Refused: `encrypted` (no or wrong password), `permissions_denied`, `too_many_pages` (over 25), `too_many_words` (over 4000 on a page), `already_tagged`, `xfa` (dynamic form), `signed`, `no_acroform_field`, `no_text_positions`, `strict`. From `review`: `review_failed` (the model or its API failed on a page). |
 | 2 | A check failed: `pixels_changed`, `text_lost`. |
-| 3 | Bad input: `unreadable`, `bad_pages`, `no_document_language`, `bad_value`, `field_not_settable`, `bad_arguments`. |
+| 3 | Bad input: `unreadable`, `bad_pages`, `no_document_language`, `bad_value`, `field_not_settable`, `bad_arguments`. From `review`: `not_tagged`, `no_readable_structure` (not tagged by this tool), `bad_structure` (nested over 64 levels), `no_credentials`. |
 
 Errors print one line: `iris-pdf: <code>: <message>`.
 
 ## Privacy
 
-Form values are personal data. They are never printed, logged, or put in the report or an error message; only field names are.
+Form values are personal data. They are never printed, logged, or put in the report or an error message; only field names are. `review` sends page images, which show any filled-in values, to the model provider. After `--flatten` the values are also tagged text, so they are sent as text too and can appear in its findings.
 
 ## Known limits
 
 - **The text exists twice** on a page that already had a text layer: the original, now an artifact, and ours. Screen readers use ours. Plain copy-and-paste tools may show the text doubled. The report warns `duplicate_text_layer`.
 - A table that continues onto the next page is tagged as two tables.
 - `check` needs veraPDF installed.
+- `review` reads only structure tagged by this tool. Its findings are a model's judgment: check them before acting on them. Text in the document can mislead the model, so no findings does not prove a document accessible.
 - A form with no fields (a flat form) cannot be filled.
 
 ## License

docs/models.md

@@ -0,0 +1,53 @@
+# Models and costs
+
+Where this project uses a Claude model, which one, and what it costs. `tag`, `fields` and `check` use no model and cost nothing to run.
+
+List prices, USD per million tokens (September 2026):
+
+| Model | Input | Output |
+|---|---|---|
+| Haiku 4.5 | 1 | 5 |
+| Sonnet 5 | 2 | 10 |
+| Opus 5.5 | 4 | 20 |
+| Opus 5 | 5 | 25 |
+| Fable 5.1 | 10 | 50 |
+
+These apply to the Anthropic API and to Bedrock's `global.` inference profiles. Bedrock's regional profiles (`us.`, `eu.`, …) cost 10% more. Our AWS organization allows only the `us.` profiles, so the figures below include that 10%.
+
+## `iris-pdf review`: Opus 5.5
+
+**Use Opus 5.5, the default.** If cost matters more than precision, `--model us.anthropic.claude-sonnet-5` (or `claude-sonnet-5`) costs about two thirds as much. Haiku 4.5 is not recommended.
+
+Measured on 2026-09-24 on Bedrock `us.` profiles, with the prompt in `src/review/review.ts`:
+
+| Model | Seeded defects found (of 8) | Findings on 3 clean pages | 25-page scanned report (ACIR): cost, per page, time |
+|---|---|---|---|
+| Opus 5.5 | 8 | 0 | $0.72, $0.029, 54 s |
+| Sonnet 5 | 8 | 0 | $0.47, $0.019, 82 s |
+| Haiku 4.5 | 8 | 4 | $0.14, $0.005, 35 s |
+
+- **The seeded defects** were fixture pages tagged from altered HTML: a heading tagged as a paragraph, a skipped heading level, swapped columns, a list and a table tagged as paragraphs, generic alt text, and Chinese text in a document declared English.
+- **On real documents** both Opus and Sonnet found real problems: links with no destination, captions that aren't on the page, and OCR noise tagged as text.
+  - Sonnet made more mistakes. Before the prompt said so, it misread `Art` (Article) as an artifact. It also writes about twice as many output tokens, which is why it is only about a third cheaper.
+  - Haiku reported running headers as missing content despite being told they are artifacts. Once it answered without calling the findings tool.
+- **Fable 5.1** was not available on Bedrock to this account when measured. At 2.5 times Opus 5.5's price, it isn't needed for this task.
+- **Tokens.** A page is about 4,000 input tokens (the image is most of it) and 300–850 output tokens.
+
+The review sends each page's image and its text to the model provider, so do not use it on documents that must not leave your machine.
+
+## The PR reviewer: Opus 5, moving to Opus 5.5
+
+`.github/workflows/code-review.yml` runs Claude Code on Bedrock. Its model is the repository variable `BEDROCK_REVIEW_MODEL`, and defaults to `us.anthropic.claude-opus-5`.
+
+Over its last 18 successful runs, a review cost $0.30 to $2.67, **$1.44 on average**, in 7 to 30 turns taking 1 to 10 minutes. Claude Code reports these figures at list price, so on the `us.` profile add about 10%. Each push to a PR is reviewed again, so a PR that takes several rounds costs several reviews.
+
+**Recommended: Opus 5.5.** It is newer, and its list price is 20% lower, so a review would cost about $1.15 on average. To switch:
+
+1. Add `us.anthropic.claude-opus-5-5` to the review role's Bedrock policy (`equalify-iris-gha-bedrock-review`, policy `bedrock-invoke-opus5`). It needs the inference profile ARN plus the `anthropic.claude-opus-5-5` foundation-model ARNs in its regions, as the policy has for Opus 5.
+2. Set the repository variable `BEDROCK_REVIEW_MODEL` to `us.anthropic.claude-opus-5-5`.
+
+Keep review on an Opus model. Its job is to find what the author missed, and that is where the cheaper models are weakest.
+
+## The maintainer agent: Opus 5.5
+
+The Iris PDF Maintainer runs in Claude Code on Opus 5.5, which suits design, multi-file changes and weighing review findings. Claude Code meters its sessions, not this repo, so there is no per-task figure here. As a guide, a working session reads a great deal of code, and most of that is cached input, which Opus 5.5 bills at $0.20 per million tokens. Broad searches it hands to subagents can run on Sonnet 5 or Haiku 4.5 for less.

src/cli.ts

@@ -1,10 +1,11 @@
 #!/usr/bin/env node
-// The iris-pdf command. Exit codes: 0 done, 1 refused, 2 verification
-// failed, 3 bad arguments or unreadable input.
+// The iris-pdf command. Exit codes: 0 done, 1 refused or the review
+// failed, 2 verification failed, 3 bad arguments or unreadable input.
 import { readFileSync, writeFileSync } from "node:fs";
 import { parseArgs } from "node:util";
 import { tag, fields, newReport, IrisPdfError, EXIT, VERSION, type TagOptions } from "./index.ts";
 import { checkPdfUa } from "./verify/pdfua.ts";
+import { review, plain, type Provider } from "./review/review.ts";
 
 const USAGE = `iris-pdf ${VERSION}
 
@@ -13,14 +14,15 @@ iris-pdf tag --pdf <in.pdf> --pages <pages.json> --out <out.pdf>
              [--ocr auto|off|required] [--verify pixels,text|off] [--verify-dpi 150]
              [--flatten] [--password <pw>] [--allow-signed] [--partial] [--strict]
 iris-pdf fields --pdf <in.pdf> [--json] [--password <pw>]
-iris-pdf check --pdf <in.pdf>`;
+iris-pdf check --pdf <in.pdf>
+iris-pdf review --pdf <tagged.pdf> [--report <review.json>] [--provider anthropic|bedrock] [--model <id>] [--password <pw>]`;
 
 const OPTIONS = {
   pdf: { type: "string" }, pages: { type: "string" }, values: { type: "string" }, out: { type: "string" },
   report: { type: "string" }, lang: { type: "string" }, title: { type: "string" }, ocr: { type: "string" },
   verify: { type: "string" }, "verify-dpi": { type: "string" }, flatten: { type: "boolean" },
   password: { type: "string" }, "allow-signed": { type: "boolean" }, partial: { type: "boolean" },
-  strict: { type: "boolean" }, json: { type: "boolean" }, help: { type: "boolean", short: "h" },
+  strict: { type: "boolean" }, provider: { type: "string" }, model: { type: "string" }, json: { type: "boolean" }, help: { type: "boolean", short: "h" },
 } as const;
 
 function badArgs(message: string): never {
@@ -44,7 +46,7 @@ function readPdf(path: string | undefined): Uint8Array {
   }
 }
 
-function main(argv: string[]): number {
+async function main(argv: string[]): Promise<number> {
   const [command, ...rest] = argv;
   let args;
   try {
@@ -70,6 +72,19 @@ function main(argv: string[]): number {
     return result.passed === false ? EXIT.verification : 0;
   }
 
+  if (command === "review") {
+    if (args.provider && args.provider !== "anthropic" && args.provider !== "bedrock") badArgs("--provider is anthropic or bedrock.");
+    const result = await review(readPdf(args.pdf), { provider: args.provider as Provider, model: args.model, password: args.password });
+    for (const { page, findings } of result.pages) for (const f of findings) console.log(`page ${page}\t${f.severity}\t${f.kind}\t${plain(f.element)}\t${plain(f.detail)}`);
+    const failed = result.pages.filter((p) => p.error);
+    for (const p of failed) console.error(`iris-pdf: review_failed: page ${p.page}: ${plain(p.error!)}`);
+    const n = result.pages.reduce((n, p) => n + p.findings.length, 0);
+    const usd = result.estimatedCostUsd === null ? "" : `, about US$${result.estimatedCostUsd.toFixed(4)}`;
+    console.error(`${n} finding${n === 1 ? "" : "s"} from ${result.model} (${result.usage.inputTokens} input, ${result.usage.outputTokens} output tokens${usd}).`);
+    if (args.report) writeFileSync(args.report, JSON.stringify(result, null, 2) + "\n");
+    return failed.length ? EXIT.refused : 0;
+  }
+
   if (command !== "tag") badArgs(`Unknown command "${command}".\n${USAGE}`);
   if (!args.pages || !args.out) badArgs("tag needs --pdf, --pages and --out.");
   const ocr = args.ocr ?? "auto";
@@ -101,7 +116,7 @@ function main(argv: string[]): number {
 }
 
 try {
-  process.exitCode = main(process.argv.slice(2));
+  process.exitCode = await main(process.argv.slice(2));
 } catch (e) {
   if (!(e instanceof IrisPdfError)) throw e;
   console.error(`iris-pdf: ${e.code}: ${e.message}`);

src/index.ts

@@ -1,11 +1,12 @@
 // The library API. tag() makes the tagged PDF; fields() lists a PDF's form
-// fields, which is what Iris asks for at upload time.
+// fields, which is what Iris asks for at upload time. review() is an optional AI review.
 import { openPdf, type OpenOptions } from "./pdf/document.ts";
 import { inventory, type Field } from "./pdf/widgets.ts";
 
 export { tag, type PagesInput, type TagOptions } from "./tag.ts";
 export { newReport, IrisPdfError, EXIT, VERSION, type Report, type Warning } from "./report.ts";
 export type { Field, FormValue } from "./pdf/widgets.ts";
+export { review, type ReviewOptions, type ReviewReport, type Finding } from "./review/review.ts";
 
 export function fields(pdf: Uint8Array, opts: OpenOptions = {}): Field[] {
   return inventory(openPdf(pdf, { ...opts, readOnly: true }).doc);