From b8536cdcf907afd892e6a1246b5c4f96d751dcc2 Mon Sep 17 00:00:00 2001 From: Sarah James Date: Sat, 4 Jul 2026 16:32:22 +0100 Subject: [PATCH 01/13] P2 render spike: gap analysis, prototype scaffolding, and go/no-go plan Investigate whether the headless C++ map-draw renderer can become the canonical figure engine for ae.report, replacing the kateri (Dart/macOS/socket) path. - tools/p2-render-spike/compare.sh: render a styled report chart through map-draw and pixel-diff it against the kateri golden PDF (paths passed as args; all real-data inputs/outputs stay outside the repo). - P2-RENDER-SPIKE-PLAN.md: characterises the two input paths (report drives kateri with a self-describing chart + named semantic styles in c["R"]; map-draw is a fixed AD-chains pipeline that reads neither c["R"] nor the legacy plot-spec), enumerates the gap, records the prototype result (RMSE 0.155, ~0% report-figure parity as-is), and gives a staged parity+effort plan with a conditional GO for P2 (gated on a P2-A milestone) and a feasible-but-downstream verdict for sigp. Co-Authored-By: Claude Opus 4.8 (1M context) --- P2-RENDER-SPIKE-PLAN.md | 175 +++++++++++++++++++++++++++++++ tools/p2-render-spike/README.md | 21 ++++ tools/p2-render-spike/compare.sh | 50 +++++++++ 3 files changed, 246 insertions(+) create mode 100644 P2-RENDER-SPIKE-PLAN.md create mode 100644 tools/p2-render-spike/README.md create mode 100755 tools/p2-render-spike/compare.sh diff --git a/P2-RENDER-SPIKE-PLAN.md b/P2-RENDER-SPIKE-PLAN.md new file mode 100644 index 0000000..cbb8ced --- /dev/null +++ b/P2-RENDER-SPIKE-PLAN.md @@ -0,0 +1,175 @@ +# P2 rendering spike — can `map-draw` become the canonical `ae.report` figure engine? + +**Spike goal.** Decide whether the headless C++ `map-draw` renderer (subsystem #1, revived +for the Linux whocc-chains batch path) can replace **kateri** (a Dart/Flutter macOS app, +driven over a Unix socket) as the engine that renders antigenic-map figures for +`ae.report`. The original author's intended architecture is a *single fast C++ engine that +reads JSON → image/PDF*, driven from Python and a thin GUI; kateri was a GUI experiment. + +**Status:** spike complete. Gap characterised; one representative figure prototyped and +pixel-diffed against its kateri golden. Recommendation below is **conditional GO, staged**. + +--- + +## 1. The two input paths + +### 1a. What `ae.report` sends to kateri (today's report figure path) + +The report never hands kateri a "plot spec" describing *how to draw*. It hands kateri a +**fully self-describing chart** and a **style name**: + +- `ChartModifier` (`py/ae/report/chart_modifier.py`) writes onto the chart: + 1. **Semantic attributes** on antigens/sera — clade, reference, passage, continent/country, + older-than, new-compared-to-previous, vaccine, serology (`ae.semantic.*`). + 2. **Named styles** in the chart's `c["R"]` block (`ae_backend` `SemanticStyles`), e.g. + `-reset`, `-clades`, `-vaccines`, `clades`, `clades-6m/-12m`, `info-clades`, `serology`, + `-o6m-grey`, `-continent`, `-pale`, `ts-YYYY-MM`, serum-circle / serum-coverage styles. + A **front style** (e.g. `clades`) *composes* background styles by reference + (`A: [{R:"-reset"}, {R:"-clades"}, {R:"-new-1"}, {R:"-vaccines"}]`) and carries a + **title** (box origin/offset + text font/weight/slant/size/colour/interline), a + **legend** flag block, and a **viewport/zoom** reset (`L`). A background style + (e.g. `-clades`) is a **list of selector→point-style rules**: `{T: selector, F: fill, + O: outline, D: drawing-order, A: alpha, L: {p: legend-priority, t: legend-text}}`. +- Transport (`py/ae/utils/kateri.py`): `send_chart(chart)` → `set_style(name)` → + `get_pdf(style=name)`. kateri resolves the named style, applies the per-selector point-style + modifications, draws title/legend/serum-circles/grid, frames by the stored viewport, and + returns PDF bytes. + +The C++ structs for this style model **already exist in ae** at `cc/chart/v3/styles.hh` +(`Title`, `Legend`, `box_t`, `text_t`, `point_style_fow_t`, `serum_circle_style_t`, +`serum_coverage_style_t`, `Selector`) — but only kateri (Dart) currently *interprets* them. + +### 1b. What `map-draw` consumes + +`ae_backend.map_draw.export_map(ace, output, projection_no, size, reorient_master, mapi, +coloring, marks, title, legend, labels, serum_circles)` / the `map-draw` CLI. It is a +**fixed-pipeline reproduction of AD's chains-202105 `make_map`**, *not* a style interpreter: + +- Grey base points with **AD-chains hardcoded** shapes/sizes (`outline #D0D0D0`, test d10 / + reference d15 (open circle) / serum d15 (open box)) — set in `draw.cc`, not read from the chart. +- Clade colour re-derived from an **external `clades.mapi` DSL file** + `coloring_key`, after + **populating clades from seqdb** — *not* from the chart's baked semantic attributes/styles. +- `mark_recent_layer`, `mark_vaccines` (vaccine strains read at runtime from + `acmacs-data/semantic_vaccines.py`), a **stress-value** title, a mapi-derived clade legend, + a **self-computed bounding-ball** viewport (or `reorient-master` alignment), opt-in + hardcoded 2-fold serum circles. + +**Confirmed by inspection:** `cc/map-draw/draw.cc` has **zero** references to `chart.styles()` +(`c["R"]`) or the legacy plot-spec (`c["p"]`). It ignores the entire report styling system. + +### 1c. The gap (enumerated) + +| # | Report figure needs (kateri does it) | `map-draw` today | +|---|---|---| +| 1 | Resolve named front styles + background-style composition + priority (`c["R"]`) | **missing entirely** | +| 2 | Per-point styles from the legacy plot-spec (`c["p"]`) | **ignored** (AD-chains hardcoded defaults) | +| 3 | Per-selector fill/outline/outline-width/size/shape/alpha/drawing-order | wrong (fixed greys + fixed sizes) | +| 4 | Arbitrary title: text + box origin/offset + font face/weight/slant/size/colour/interline | only a stress number, top-left | +| 5 | Semantic legend: rows (priority+text), counter, box, point-size, zero-count rows | only a mapi-derived clade legend | +| 6 | Selectors beyond clade: reference, passage, continent, older-than, new-compared-to, serology, vaccine | only clade + reference + serum | +| 7 | Stored/zoom viewport from the style (`L`) → exact report framing | self-computed bbox → **framing/zoom mismatch** | +| 8 | Serum circles from style: fold, theoretical/empirical, dash, angles, radius outline | hardcoded 2-fold empirical only | +| 9 | Serum-coverage (within/outside) styling | missing | +| 10 | Time-series / pale / continent / serology composed styles | missing | +| 11 | Vaccine marks driven by the chart's semantic vaccine attribute + style | different source (runtime `semantic_vaccines.py`) | +| 12 | Font matching to kateri (Helvetica, weights/slants) + PDF page geometry | partial / not matched | + +**Root characterisation:** `map-draw` is a *fixed AD-chains pipeline*; the report needs a +*semantic-style interpreter*. Closing the gap = porting kateri's rendering logic (Dart) into +C++ on top of `map-draw`'s existing cairo/viewport/point scaffolding. + +--- + +## 2. Prototype (what was actually validated) + +One representative report figure: the `clades` main map of a small H1 report chart +(8 antigens × 7 sera). Rendered through `map-draw` (`--no-populate --no-marks --no-vaccines`) +and pixel-compared to the **existing kateri golden** `out.1.clades.pdf` for the same chart. +(No kateri re-run needed — the golden was already on disk. Real chart + renders kept in the +scratchpad, never in the repo.) + +- **RMSE = 0.155 (15.5%).** +- **Visual (montage):** kateri shows the report title, clade-coloured antigens, green + vaccine dots with strain labels, and a full clade legend with counts, framed by the stored + zoom viewport. `map-draw` shows an all-grey map (reference = open circles, sera = open + boxes) with a stress-number title, **no** colour / vaccines / legend / report title, at a + **different zoom/framing**. Point *positions* correspond (same optimisation), but scale + differs because the viewports differ. +- **Effective report-figure parity of `map-draw` as-is: ~0%.** The 15.5% RMSE understates it + because most of the canvas is white background; every foreground element differs. + +This is the expected result and confirms the gap is architectural, not cosmetic: +`map-draw` cannot approximate a report figure by tweaking flags — it lacks the style engine. + +Repro: `tools/p2-render-spike/compare.sh `. + +--- + +## 3. Parity + effort plan + +Realistic path: add a **semantic-style interpreter** to `map-draw` — read `c["R"]` + `c["p"]`, +resolve/compose front styles, apply per-selector point styles, and draw title/legend/viewport/ +serum-circles from the style. Reuse what already exists: + +- `cc/chart/v3/styles.hh` already **models** the style structs (big head-start — the schema is + done; it needs a JSON reader + an interpreter, not a new data model). +- `cc/chart/v3` selection (`SelectedAntigens`/`SelectedSera`) already matches semantic + attributes — reuse for selectors. +- `map-draw` already has cairo primitives, the viewport/transform, point shapes, and + title/legend/serum-circle drawing code — the *drawing* layer is largely there; the + *style-interpretation* layer is ~0%. +- whocc-chains proved `map-draw` can reach **<1% pixel diff** vs an AD golden — the fidelity + bar is achievable. + +Staged milestones (each ends with a pixel-diff vs a kateri golden across the report set): + +| Stage | Scope | Est. (agent-pace) | +|---|---|---| +| P2-A | Style reader for `c["R"]`/`c["p"]` + front-style resolver + selector engine; apply per-point fill/outline/size/shape/order; draw title (arbitrary text/box/font) + semantic legend + **viewport from style**. Target: `clades` main map < 1% vs kateri golden. | **M** (1–2 sessions) | +| P2-B | Remaining selectors (reference/passage/continent/older-than/new/serology/vaccine) + `-vaccines`/`-reset`/`-new` composition + info-/6m/12m variants. | **M** | +| P2-C | Serum circles + serum coverage from style (fold/theoretical/empirical/dash/angles/radius); time-series / pale / continent / serology styles. | **M** | +| P2-D | Font + PDF-geometry parity with kateri; fidelity harness over the full figure matrix (subtypes × labs × styles); iterate to < 1%. | **M–L**, iterative | + +**Total ≈ a multi-week C++ effort (several agent-sessions)** — comparable to or somewhat larger +than the whocc-chains `map-draw` revival, because the target is *arbitrary report styling* +parity, not one fixed pipeline. The `styles.hh` head-start and existing drawing layer are the +main risk-reducers; the long tail (exact fonts, legend/title box metrics, viewport rounding, +per-style-family quirks) is where iteration time goes. + +**What is validated vs estimated:** the gap analysis and the ~0% as-is parity are +**validated** (inspection + prototype). The stage effort is **estimated** by analogy to the +chains revival and by the fact that the data model already exists. + +--- + +## 4. Recommendation + +### P2 (report figure engine): **conditional GO — staged.** + +The architecture case is strong and matches the author's intent: one fast, Linux-capable C++ +engine reading JSON → PDF, no Dart/macOS/socket dependency, no separate GUI process in the +batch path. The work is substantial but **bounded and well-scaffolded** (`styles.hh` models +the styles; the drawing layer exists; chains proved <1% fidelity is reachable). + +De-risk by gating on **P2-A**: build the style interpreter far enough to reproduce the plain +`clades` main map **< 1% pixel-diff vs the kateri golden**. That single milestone exercises the +whole spine (style read → selector → point-style → title → legend → viewport). If P2-A lands +cleanly, commit to P2-B..D and retire kateri from the report figure path. **If P2-A cannot +reach < 1% within ~2 sessions, that is the no-go signal** — kateri stays the report renderer +and `map-draw` remains the chains-only tool. + +### sigp (AD-faithful maps + tree on one C++ canvas): **feasible, but downstream of P2.** + +`map-draw` and `tal-draw` both build on the shared `cc/draw/cairo-surface` primitives, and AD +historically drew the tree and the section maps on a **single canvas**. Today ae composes a +`tal-draw` tree PDF with **kateri** map PDFs via `pdfjam`/`pdflatex` +(`py/ae/tal/signature_page.py` + `section_maps.py`; the latter's own docstring notes "ae has no +single-canvas renderer"). Once P2 gives `map-draw` a report-grade map renderer, a true +single-canvas sig page becomes achievable: draw the tree (tal-draw code) and the per-section +maps (map-draw code) onto one `CairoSurface`, positioned by the sig-page layout, with +per-section colouring that `section_maps.py` already computes. + +**Verdict:** sigp single-canvas is a clean architectural win but is **contingent on P2** (needs +the map renderer first) **plus a compositor** (shared coordinate space, section→map viewport, +page layout). Recommend: **defer sigp until P2-A/B prove the map engine**, then scope it as a +follow-on. It should not gate the P2 decision. diff --git a/tools/p2-render-spike/README.md b/tools/p2-render-spike/README.md new file mode 100644 index 0000000..47159ba --- /dev/null +++ b/tools/p2-render-spike/README.md @@ -0,0 +1,21 @@ +# P2 render spike — scaffolding + +Investigative scaffolding for the spike **"can the headless C++ `map-draw` renderer +become the canonical figure engine for `ae.report`, replacing the kateri (Dart/macOS/socket) +path?"** Findings and the go/no-go recommendation live in +[`../../P2-RENDER-SPIKE-PLAN.md`](../../P2-RENDER-SPIKE-PLAN.md). + +## `compare.sh` + +Renders a styled report chart through `map-draw` and pixel-diffs it against the golden PDF +that kateri produced for the same chart+style. + +```bash +compare.sh [out-dir] [size] +``` + +All inputs are paths to **real report data outside the repo**; all output goes to a scratch +dir outside the repo. **Nothing real-data is committed** — not the chart, not any render (a +labelled antigenic map exposes strain names + titer-derived coordinates). + +Requires `map-draw` (`ae/build/map-draw`), `pdftoppm`, and ImageMagick. diff --git a/tools/p2-render-spike/compare.sh b/tools/p2-render-spike/compare.sh new file mode 100755 index 0000000..28721b3 --- /dev/null +++ b/tools/p2-render-spike/compare.sh @@ -0,0 +1,50 @@ +#!/usr/bin/env bash +# ---------------------------------------------------------------------- +# P2 render spike — pixel-compare the headless C++ map-draw renderer against a +# kateri-rendered report map figure. +# +# This is investigative scaffolding for the "can map-draw become the canonical +# ae.report figure engine?" spike (see P2-RENDER-SPIKE-PLAN.md). It renders a +# styled report chart through map-draw and diffs it against the golden PDF that +# kateri produced for the same chart+style. +# +# WHO-data note: pass the styled `.ace` and the kateri golden PDF as arguments — +# they live in a report working dir OUTSIDE the repo and are never committed. +# This script writes all output to a scratch dir, also outside the repo. +# +# Usage: +# compare.sh [out-dir] [size] +# +# Requires: map-draw (ae/build/map-draw), pdftoppm, ImageMagick (compare/montage). +# ---------------------------------------------------------------------- +set -euo pipefail + +STYLED_ACE=${1:?need path to a styled report .ace} +KATERI_PDF=${2:?need path to the kateri golden .pdf for the same chart+style} +OUTDIR=${3:-./p2-spike-out} +SIZE=${4:-800} + +MAPDRAW=${MAPDRAW:-/Users/sarahjames/AC/eu/ae/build/map-draw} +mkdir -p "$OUTDIR" +export MAGICK_TMPDIR="$OUTDIR" + +echo ">>> map-draw render (styles ignored by design — bare AD-chains render)" +# --no-populate: the report chart already carries clade attrs; skip seqdb so no env needed. +"$MAPDRAW" --no-populate --no-marks --no-vaccines --size "$SIZE" \ + "$STYLED_ACE" "$OUTDIR/mapdraw.pdf" + +echo ">>> rasterise both PDFs to ${SIZE}px PNG" +pdftoppm -png -scale-to "$SIZE" "$KATERI_PDF" "$OUTDIR/kateri" >/dev/null 2>&1 +pdftoppm -png -scale-to "$SIZE" "$OUTDIR/mapdraw.pdf" "$OUTDIR/mapdraw" >/dev/null 2>&1 +KAT=$(ls "$OUTDIR"/kateri*.png | head -1) +MAP=$(ls "$OUTDIR"/mapdraw*.png | head -1) + +echo ">>> RMSE (0..1; lower is closer):" +compare -metric RMSE "$KAT" "$MAP" "$OUTDIR/diff.png" 2>&1 || true +echo + +echo ">>> side-by-side montage + difference heatmap" +montage "$KAT" "$MAP" -tile 2x1 -geometry +5+5 -background white "$OUTDIR/montage.png" +compare "$KAT" "$MAP" -compose src "$OUTDIR/heatmap.png" + +echo ">>> wrote: $OUTDIR/{montage,heatmap,diff}.png" From bf38d2e17f828b1223bbcdfa81526555ff9be3a7 Mon Sep 17 00:00:00 2001 From: Sarah James Date: Wed, 8 Jul 2026 09:14:52 +0100 Subject: [PATCH 02/13] P2 spike: headless style-consuming POC + design doc Prove the report's on-chart styling (c[R] named styles + c[p] base plot-spec) can be consumed headlessly and reproduce a report map's content without kateri. - tools/p2-render-spike/poc_render.py: ~230-line headless renderer (Pillow). Resolves a selected front style (front->background reference composition), matches selectors against semantic attributes, applies per-point fill/outline/size/shape/draw-order, draws title/labels/viewport. Reads charts read-only; no WHO data committed. - P2-RENDER-DESIGN.md: rendering-contract spec, POC results + 3 fidelity numbers, vertical-mirror resolution (no Y-flip), architecture (native renderer as kateri-compatible drop-in with kateri kept as fallback/interactive), and an independent/parallelisable phased work-breakdown. Key finding: cc/chart/v3/chart-import.cc already fully parses c[R] into semantic::Styles; only the resolver + drawing application are missing. Co-Authored-By: Claude Opus 4.8 (1M context) --- P2-RENDER-DESIGN.md | 274 ++++++++++++++++++++++++++++ tools/p2-render-spike/poc_render.py | 207 +++++++++++++++++++++ 2 files changed, 481 insertions(+) create mode 100644 P2-RENDER-DESIGN.md create mode 100644 tools/p2-render-spike/poc_render.py diff --git a/P2-RENDER-DESIGN.md b/P2-RENDER-DESIGN.md new file mode 100644 index 0000000..46a4a60 --- /dev/null +++ b/P2-RENDER-DESIGN.md @@ -0,0 +1,274 @@ +# P2 — Headless native map renderer for `ae.report`: design + work-breakdown + +**Goal (Eu, 2026-07-08).** Render the seasonal report's antigenic maps **without launching +kateri per map** (faster reports), **improve AD fidelity**, and stay **kateri-compatible** — +consume the *same on-chart styling* the report already feeds kateri, so the renderer is a +drop-in for the report's map step. kateri stays available (interactive drag/relax). + +This document is the Phase-1 spike output: it (a) writes down the *rendering contract*, (b) +records what the POC proved, (c) proposes the architecture, and (d) gives an **independent, +parallelisable** work-breakdown for the full build. It supersedes and extends +`P2-RENDER-SPIKE-PLAN.md` (A7's paper analysis) with a working headless POC and the key +de-risking findings. + +--- + +## 0. Executive summary + +- **The styling can be consumed headlessly — proven.** A ~230-line headless POC + (`tools/p2-render-spike/poc_render.py`) reads a real report chart's on-chart styles + (`c["R"]` named styles + `c["p"]` base plot-spec), resolves the selected front style, + applies per-point fill/outline/size/shape/order, draws the title, vaccine labels, and + viewport, and produces a map **recognisably identical in content** to both the kateri + golden and the AD reference for the same map (same two clusters, full clade palette, same + vaccine labels). This is the core unknown A7 left open (A7 only showed the *current* + `map-draw` pipeline ignores styles → ~0% content parity). +- **The C++ head-start is larger than A7 stated.** `cc/chart/v3/chart-import.cc` **already + fully parses `c["R"]`** into `cc/chart/v3/styles.hh`'s `semantic::Styles` + (modifiers, selectors, title, legend, serum circle, serum coverage). The data model *and* + the JSON reader exist and round-trip through Python (`chart.styles()`). Only the + **style resolver + drawing application** are missing from `cc/map-draw/draw.cc`. +- **Fidelity (normalised RMSE, one representative map, coarse):** headless-vs-kateri 0.29, + headless-vs-AD 0.28, **kateri-vs-AD 0.23**. The kateri-vs-AD 0.23 is a *framing/legend/scale + floor* between differently-exported maps — the POC sits just above it, and the excess is the + known, bounded tail (exact viewport, legend box, grid, fonts/AA). Pixel-RMSE is **not** a + clean metric until viewport + page geometry are matched; the visual montage is the honest + evidence at this stage. +- **Recommendation: GO, staged, multi-phase (multi-agent).** The spine is proven and the data + model + reader + drawing primitives already exist. Gate the commitment on milestone **A** + (one plain by-clade map < ~2% pixel-diff vs the kateri golden once the viewport convention + is matched). The sub-features below are largely independent and can be parallelised. + +--- + +## 1. The rendering contract (the spec the renderer implements) + +### 1.1 What the report feeds kateri + +The report never sends a "how to draw" plot spec. `ChartModifier` +(`py/ae/report/chart_modifier.py`) bakes onto the chart: + +1. **Semantic attributes** on antigens/sera (`ae.semantic.*`): clade list (`C`), reference + (`R`), passage (`p`), continent/country (`C9`/`c9`), older-than (`o6m`/`o12m`), + new-compared-to-previous (`new`), vaccine (`V`), serology, sequenced. +2. **Named styles** in the chart's `c["R"]` block (C++ `semantic::Styles`). + +`py/ae/utils/kateri.py` then transports over a length-prefixed Unix socket (4-byte code + +4-byte length + 4-byte-padded payload): `send_chart` (`CHRT`), `set_style(name)` (`COMD`), +`get_pdf(style,width,square,viewportSize)` (`COMD` → `PDFB` bytes). kateri resolves the +named style, applies it, draws title/legend/grid/serum-circles, frames by the viewport, and +returns PDF bytes. The report's map step is exactly: **pick a style name → get a PDF**. That +is the whole surface the headless renderer must replace. + +### 1.2 On-chart representation (verified against a real styled chart) + +`c["R"]` is an object: `name → style`. Two kinds of style: + +**Front (composite) style** — the thing a map is rendered as. Carries: +- `z` : integer priority (inter-style draw order). +- `A` : ordered list; each entry `{ "R": }` is a **reference** that composes a + background style (resolved by following the reference). +- `L` : legend flags `{ "-": shown(bool), "C": add_counter(bool) }`. +- `T` : title `{ "B": box{ "o": origin, "O": [dx,dy], padding/border/bg }, "T": text{ + "t": text, "f": face, "W": weight, "S": slant, "s": size, "c": colour, "i": interline } }`. + +**Background style** — a list of selector→point-style rules, plus optional viewport: +- `z` : priority. +- `V` : `[x, y, w, h]` viewport (present on the `-reset` family). +- `A` : ordered modifier list. Each modifier: + - `T` : **selector** object — `{C: clade}` (matches if clade ∈ antigen's `C` list), + `{R: true}` reference, `{V: true}` vaccine, `{p: passage}`, `{"!i": index}` by point + index; empty ⇒ all candidates. + - `A` : select — `1`/truthy = antigens only, `0`/false = sera only, absent = both. + - point-style fields (shared with `c["p"]`): `F` fill, `O` outline, `o` outline-width, + `S` shape (`C`ircle/`B`ox/`T`riangle/`E`gg/`U`glyegg), `s` size, `r` rotation, + `a` aspect, `-` hidden (`false` ⇒ shown), `l` label `{p:[dx,dy], t:text, s:size}`. + - `D` : drawing order — `"r"` raise, `"l"` lower. + - `L` : legend row `{ p: priority, t: text }`. + - `CI` : serum-circle spec; `SC` : serum-coverage spec (see `styles.hh`). + +**Base plot-spec `c["p"]`** (kateri's starting point before named-style modifiers): +`P` = list of point styles, `p` = per-point index into `P`, `d` = draw order. Same field +letters. This supplies the default appearance (base greys, shapes, sizes) that the named +style then overrides. + +**Resolution algorithm** (kateri's, reproduced in the POC): start from `c["p"]` per-point +styles; resolve the selected front style by walking its `A` list, recursively expanding each +`{R:name}` reference into that background style's modifiers (picking up its `V` viewport), +producing a **flat, ordered modifier list**; apply each modifier to the points its selector +matches (later modifiers win; `D:"r"` moves matched points to the end of the draw order); +collect legend rows; apply the front style's own `T`/`L`. Undefined references are tolerated +(e.g. a `-new-*` ref absent on a chart with no previous). Draw in draw order; then title; +then legend; frame by the viewport. + +The C++ structs for all of this already exist (`styles.hh`) and the reader already +populates them (`chart-import.cc` `read_semantic_plot_specification` → `…_style` → +`…_style_modifier`). **The resolver above is the missing piece** (currently only kateri/Dart +implements it). + +### 1.3 AD heritage (fidelity target) + +Where kateri diverges from AD, match **AD**. AD's renderer is +`~/AC/eu/AD/sources/acmacs-draw` + `acmacs-map-draw` (`draw.cc`, `point-style-draw.cc`, +`map-elements*.{cc,hh}`, `labels.*`, legend + viewport code). The revived `cc/map-draw` +already reproduced AD *chain* maps to <1% px-diff, so the cairo/viewport/point-shape layer is +AD-calibrated. The POC confirmed the report's kateri golden and the AD reference for the same +map are near-identical in content (same clades/counts/labels), so for the by-clade maps +"match kateri" and "match AD" coincide; the fidelity strategy only needs an explicit AD tie- +breaker for details where they differ (grid style, legend metrics, label auto-placement). + +--- + +## 2. POC — what was actually built and proven + +`tools/p2-render-spike/poc_render.py` (Python + Pillow, stdlib elsewhere), run headlessly: + +``` +python3.14 poc_render.py +``` + +It implements §1.2 end-to-end: colour parsing (hex/#AARRGGBB/named/`gray80`/`:bright`), +base plot-spec extraction, selector matching against semantic attributes, recursive front- +style resolution, per-point application, draw-order raising, title, vaccine labels, legend- +row collection, and a Pillow rasteriser (circle/box/triangle/egg-approx). Reads the chart +**read-only** from a private report dir; no WHO data enters the repo (images stay in the +scratchpad). + +**Result.** For the representative `clades` main map it draws the correct two clusters, the +**full clade palette matching kateri and AD**, base grey cloud, open-box sera, open-circle +references, the title, and all vaccine labels. See the 3-way montage (POC | kateri | AD) in +the scratchpad. This validates the whole spine: style read → selector → point-style → +title → legend → viewport, headless, on a real report chart. + +**What the POC did NOT do (⇒ the fidelity tail):** exact viewport/aspect match to kateri +(it auto-frames to the data bbox), the legend *box* with per-row counts, the background +unit grid + border, exact fonts/anti-aliasing/PDF page geometry, serum circles/coverage, +time-series / continent / serology / info-map style families, label auto-placement (overlap +avoidance). These are the §4 tasks. + +### 2.1 Vertical-mirror resolution + +The first POC pass applied a Y-flip (`device_y = H - (y-vy)/vh*H`) and came out vertically +mirrored vs kateri. **Fix: apply no Y-flip** — `device_y = (y - vy)/vh * H`. The projection +layout `c["P"][0]["l"]`, after the stored 2×2 transform `t` is applied +(`x' = t0·x + t1·y`, `y' = t2·x + t3·y`), is already in a **y-increases-downward screen +coordinate system**; kateri maps world→device directly with no inversion. With the flip +removed the POC matches kateri's orientation exactly (diverse cluster top, blue cluster +bottom). The native renderer must likewise treat post-transform layout Y as screen-down. + +### 2.2 Viewport convention (open item, must be nailed in milestone A) + +The stored `-reset` viewport `V=[x,y,w,h]` did **not** frame the post-transform layout in the +POC (data y-range fell outside the viewport y-range), so the POC auto-frames instead. kateri +frames correctly, so kateri applies an additional **recenter** between the transform and the +viewport (cf. `export_mapi_for_signature_pages` in `chart_modifier.py`, which combines +kateri's `native`, `used`, and `native_center` to recover an absolute viewport — evidence +kateri carries a native-center offset). Milestone A must reproduce this exact transform + +recenter + viewport chain so the native frame equals kateri's; it is the single most +important fidelity step (everything keys off it) and the biggest residual in the POC numbers. + +--- + +## 3. Architecture — native renderer as a kateri-compatible drop-in + +### 3.1 Component + +Add a **semantic-style interpreter** layer to `cc/map-draw` on top of its existing cairo / +viewport / point-shape scaffolding, plus a new entry point that takes a **chart + style +name** (mirroring `set_style` + `get_pdf`) rather than the chains `DrawSettings`: + +``` +export_styled_map(chart, projection_no, style_name, width, output) // C++ + pybind +``` + +Pipeline inside: `resolve_style(chart.styles(), c["p"], style_name)` → flat modifier list + +viewport + title + legend (the §1.2 algorithm, in C++, reusing `semantic::Styles` already +parsed by the importer and `SelectedAntigens/Sera` for selector matching) → apply to a +per-point render model → draw via `cc/draw/cairo-surface` (points, grid/border, title, +legend, serum circles) → PDF/PNG. The chains `export_map` stays as-is (fixed AD-chains +pipeline); the new path is additive. + +### 3.2 Where it plugs into `ae.report` + +The report's map step is "select style → get PDF" via `kateri.communicator` +(`set_style`/`get_pdf`) in the `style`/`export` path (`ae.report.commander`). Introduce a +small **renderer seam** — a `MapRenderer` interface with two implementations: + +- `KateriRenderer` — current behaviour (launch kateri, socket, `set_style`/`get_pdf`). +- `NativeRenderer` — call `ae_backend.map_draw.export_styled_map(chart, style, width)` + in-process; no subprocess, no socket, Linux-capable. + +Select via a config flag / env (`AE_REPORT_MAP_RENDERER=native|kateri`, default `kateri` +until milestone D proves parity, then flip). This keeps kateri as a **fallback** and as the +**interactive** tool (drag-adjust / relax — `handle_relax`, `get_moved_points`), which the +native batch renderer deliberately does not replace. + +### 3.3 AD-fidelity strategy + +1. Match **viewport/transform/recenter** first (§2.2) — the frame must equal kateri's. +2. Point shapes/sizes/outlines/greys from `c["p"]` + modifiers (POC already correct in + content) — calibrate pixel size against the AD/kateri golden. +3. Grid + border, legend box metrics, title box metrics, fonts (Helvetica weights/slants), + PDF page geometry: take AD (`acmacs-draw`/`acmacs-map-draw`) as the tie-breaker where it + differs from kateri; the chains `map-draw` revival already matched AD here. +4. Per-map-family details (serum circles theoretical/empirical, coverage within/outside, + time-series sizing) from the modifier specs in `styles.hh`. + +Fidelity harness: rasterise native + kateri golden with `pdftoppm` at equal size and +`magick compare -metric RMSE/AE` + a high-zoom montage, over the figure matrix (subtypes × +labs × style families). Reuse the chains map-draw pixel-diff helpers. + +--- + +## 4. Work-breakdown (independent sub-features — parallelisable) + +Milestone **A** is the go/no-go spine and must land first. After A, the rest are largely +independent and can be handed to separate agents in parallel; dependencies noted. + +| ID | Sub-feature | Depends on | Size | Hard parts | +|----|-------------|-----------|------|-----------| +| **A** | **Spine + go/no-go.** C++ `export_styled_map`: style resolver (front→background composition, flat modifier list), selector engine (reuse `SelectedAntigens/Sera`), per-point fill/outline/width/size/shape/order, **transform+recenter+viewport exact match** (§2.2), title, plain legend. Target: one plain by-clade map < ~2% px-diff vs kateri golden. | reader (done) | **M** | **Viewport recenter convention** (the key risk); flat-resolution order vs kateri; colour model (`:bright`, `#AARRGGBB`, `gray80`). | +| **B** | Legend box | A | S–M | Per-row counts (`legend_counter` ⇒ count matched points), zero-count rows, box origin/padding, row point-size/text-size metrics matching kateri/AD. | +| **C** | Title + fonts + PDF page geometry | A | S–M | Helvetica face/weight/slant, interline, box origin/offset, multi-line titles; PDF point size = width; AA parity. | +| **D** | Background grid + border | A | S | AD `BackgroundBorderGrid` spacing/colour; cheap, isolate early for fidelity. | +| **E** | Remaining selectors + composed styles | A | M | reference/passage/continent/older-than/new/serology/vaccine selectors; `-reset`/`-new`/`-vaccines`/`-pale`/`-o6m/12m` composition; **info-** maps (blank title, no legend). Mostly selector coverage — low risk once A's engine exists. | +| **F** | Serum circles | A | M | Empirical vs theoretical radius, fold, dash, angle radius-lines, per-passage outline/fill, fallback radius. Geometry-heavy; AD `map-procrustes`/serum-circle code is the reference. | +| **G** | Serum coverage | F | M | within/outside point restyle by fold; `-sco-*`/`sc-*` front styles; per-serum map matrix. | +| **H** | Time-series / continent / serology / pale families | E | M | Composed multi-reference styles; per-month `ts-*`; old/new sizing. Volume, not novelty. | +| **I** | Label auto-placement | A,C | **M–L** | Vaccine/serology labels currently carry explicit offsets (`l.p`) so baseline is easy; **overlap-avoidance / leader lines** to match AD is the genuinely hard, iterative part. | +| **J** | `ae.report` renderer seam + flag | A | S | `MapRenderer` interface, `NativeRenderer`, `AE_REPORT_MAP_RENDERER`; keep kateri default + fallback. Wire producer **and** consumer (a feature isn't done until end-to-end). | +| **K** | Fidelity harness + figure-matrix sign-off | B–J | **M–L**, iterative | The long tail: per-family px-diff to <~1–2%, document irreducible AA diffs, flag any map that can't match and why. | + +**Stays on kateri (do not port):** interactive drag-adjust + live relax animation +(`RLAX`/`LAYT`/`get_moved_points`/`handle_relax`) and the operator GUI. The native renderer +is the **batch** map engine only. + +**Downstream (out of P2 scope, enabled by it):** single-canvas signature pages — draw the +`tal-draw` tree and the per-section `map-draw` maps on one `CairoSurface` (today composed via +`pdfjam`/`pdflatex` in `signature_page.py`/`section_maps.py`). Defer until A–E land. + +--- + +## 5. Recommendation + +**GO, staged, multi-phase (multi-agent).** The architecture matches the author's stated +intent (one fast C++ JSON→image/PDF engine, kateri as thin/interactive client), the data +model + JSON reader + AD-calibrated drawing layer already exist, and the POC proves the +styling is consumable headlessly with content parity to both kateri and AD. + +- **Next step:** milestone **A** as a single focused agent-session. Success = the plain + by-clade map frames and colours like the kateri golden (< ~2% px-diff), which requires + cracking the viewport recenter convention (§2.2) — the one real risk. **If A cannot reach + parity within ~2 sessions, that is the no-go signal**: keep kateri for report maps and + `map-draw` chains-only. +- **After A:** fan out B–J to parallel agents (independent per the table), then converge on + the K harness. Total ≈ a multi-week C++ effort comparable to the chains `map-draw` revival, + but front-loaded risk is small because the reader/model/primitives are done — the cost is + the fidelity long tail (viewport, legend/title metrics, fonts, label placement), not new + architecture. + +**Honesty on the POC:** it is a Python/Pillow *content* proof, not a pixel-faithful renderer. +It does not match kateri's viewport, legend box, grid, or fonts, and its RMSE numbers are +dominated by those gaps (the kateri-vs-AD 0.23 floor shows RMSE is uninformative until +framing is matched). What it *does* prove — the previously-open question — is that the report's +on-chart styling can be parsed and applied headlessly to reproduce the report map's content. diff --git a/tools/p2-render-spike/poc_render.py b/tools/p2-render-spike/poc_render.py new file mode 100644 index 0000000..eaaba23 --- /dev/null +++ b/tools/p2-render-spike/poc_render.py @@ -0,0 +1,207 @@ +#!/usr/bin/env python3 +""" +P2 render-spike POC: headless consumption of the report's on-chart styling (c["R"] named +styles + c["p"] base plot-spec) to draw ONE report map WITHOUT kateri. + +Reads a styled.ace READ-ONLY, resolves a named front style, applies it, draws with PIL. +Proves the styling contract can be consumed headlessly. NOT pixel-perfect by design. +""" +import sys, json, subprocess, math +from PIL import Image, ImageDraw, ImageFont + +DECAT = "/Users/sarahjames/AC/eu/bin/decat" + +def load_chart(path): + raw = subprocess.check_output([DECAT, path]) + return json.loads(raw)["c"] + +# ---- colour parsing ------------------------------------------------------- +_NAMED = { + "black": (0,0,0,255), "white": (255,255,255,255), "red": (255,0,0,255), + "green": (0,128,0,255), "blue": (0,0,255,255), "grey": (128,128,128,255), + "gray": (128,128,128,255), "orange": (255,165,0,255), "transparent": None, +} +def parse_color(c): + if c is None: return None + if isinstance(c, str): + s = c.strip() + if s in ("", "T", "transparent", ":bright"): # :bright placeholder -> leave as light grey + return None if s != ":bright" else (200,200,200,255) + if s.startswith("#"): + h = s[1:] + if len(h) == 6: + return (int(h[0:2],16), int(h[2:4],16), int(h[4:6],16), 255) + if len(h) == 8: # AARRGGBB + a=int(h[0:2],16); r=int(h[2:4],16); g=int(h[4:6],16); b=int(h[6:8],16) + return (r,g,b,a) + low = s.lower() + if low.startswith("gray") or low.startswith("grey"): + try: + pct = int(low[4:]); v=round(pct*255/100); return (v,v,v,255) + except ValueError: pass + return _NAMED.get(low, (128,128,128,255)) + return (128,128,128,255) + +# ---- base plot-spec ------------------------------------------------------- +def base_point_styles(c): + p = c["p"]; styles = p["P"]; idx = p["p"]; order = p.get("d", list(range(len(idx)))) + pts = [] + for i in range(len(idx)): + st = dict(styles[idx[i]]) + pts.append({ + "fill": st.get("F"), "outline": st.get("O", "black"), + "outline_width": st.get("o", 1.0), "size": st.get("s", 1.0), + "shape": st.get("S", "C"), "shown": True, "label": None, + }) + return pts, list(order) + +# ---- semantic selector matching ------------------------------------------ +def match(sel, only, c, nag): + n_total = len(c["a"]) + len(c["s"]) + if only == 1: cand = range(nag) + elif only == 0: cand = range(nag, n_total) + else: cand = range(n_total) + def sem(i): + if i < nag: return c["a"][i].get("T", {}) + return c["s"][i-nag].get("T", {}) + if not sel: + return list(cand) + if "!i" in sel: + j = sel["!i"] + if only == 0: j = nag + j + return [j] if 0 <= j < n_total else [] + out = [] + for i in cand: + t = sem(i) + ok = True + for k, v in sel.items(): + if k == "C": + clist = t.get("C", []) + if isinstance(clist, str): clist = [clist] + if v not in clist: ok = False; break + elif k == "R": + if bool(t.get("R", False)) != bool(v): ok = False; break + elif k == "V": + if bool(t.get("V", False)) != bool(v): ok = False; break + elif k == "p": + if t.get("p") != v: ok = False; break + else: + if t.get(k) != v: ok = False; break + if ok: out.append(i) + return out + +# ---- style resolver ------------------------------------------------------- +def resolve(name, R, out_mods, state, depth=0): + st = R.get(name) + if st is None: # tolerate undefined refs (e.g. -new-1 absent) + return + if "V" in st: state["viewport"] = st["V"] + for m in st.get("A", []): + keys = set(m.keys()) + if "R" in m and keys <= {"R"}: # pure parent reference -> recurse + resolve(m["R"], R, out_mods, state, depth+1) + else: + out_mods.append(m) + if depth == 0: # front style carries title + legend + if "T" in st: state["title"] = st["T"] + if "L" in st: state["legend_flags"] = st["L"] + +# ---- apply ---------------------------------------------------------------- +def build(c, style_name): + R = c["R"]; nag = len(c["a"]) + pts, order = base_point_styles(c) + state = {"viewport": None, "title": None, "legend_flags": None} + mods = [] + resolve(style_name, R, mods, state) + legend_rows = [] + for m in mods: + sel = m.get("T", {}); only = m.get("A") + idxs = match(sel, only, c, nag) + for i in idxs: + pt = pts[i] + if "F" in m: pt["fill"] = m["F"] + if "O" in m: pt["outline"] = m["O"] + if "o" in m: pt["outline_width"] = m["o"] + if "s" in m: pt["size"] = m["s"] + if "S" in m: pt["shape"] = m["S"] + if "-" in m: pt["shown"] = not m["-"] + if "l" in m: pt["label"] = m["l"] + if m.get("D") == "r": # raise: move to end of draw order + if i in order: order.remove(i); order.append(i) + if "L" in m: + legend_rows.append((m["L"].get("p",0), m["L"].get("t",""), m.get("F"))) + legend_rows.sort(key=lambda r: -r[0]) + return pts, order, state, legend_rows + +# ---- draw ----------------------------------------------------------------- +def render(c, style_name, out_png, autoframe=True): + pts, order, state, legend_rows = build(c, style_name) + W = H = 800 + img = Image.new("RGBA", (W, H), (255,255,255,255)) + dr = ImageDraw.Draw(img) + lay = c["P"][0]["l"]; tr = c["P"][0].get("t") + def xf(pt): + x, y = pt[0], pt[1] + if tr: x, y = tr[0]*x+tr[1]*y, tr[2]*x+tr[3]*y + return x, y + if autoframe: + xs=[]; ys=[] + for i,co in enumerate(lay): + if co and len(co)>=2 and co[0] is not None and pts[i]["shown"]: + X,Y=xf(co); xs.append(X); ys.append(Y) + cx=(min(xs)+max(xs))/2; cy=(min(ys)+max(ys))/2 + span=max(max(xs)-min(xs), max(ys)-min(ys))*1.06 + vx,vy,vw,vh = cx-span/2, cy-span/2, span, span + else: + vp = state["viewport"] or [-5,-5,10,10] + vx, vy, vw, vh = vp + def dev(x, y): + return ((x-vx)/vw*W, (y-vy)/vh*H) # NO flip (layout y already screen-down) + unit_px = W/vw + for i in order: + if i >= len(lay): continue + co = lay[i] + if not co or len(co) < 2 or co[0] is None: continue + pt = pts[i] + if not pt["shown"]: continue + wx, wy = xf(co); dx, dy = dev(wx, wy) + r = max(pt["size"] * 0.5 * (W/800.0), 0.8) # s is ~pixel diameter at 800px ref + fill = parse_color(pt["fill"]); outline = parse_color(pt["outline"]) or (0,0,0,255) + ow = max(1, round(pt["outline_width"])) + shape = (pt["shape"] or "C")[0].upper() + bbox = [dx-r, dy-r, dx+r, dy+r] + if shape == "B": + dr.rectangle(bbox, fill=fill, outline=outline, width=ow) + elif shape == "T": + dr.polygon([(dx,dy-r),(dx-r,dy+r),(dx+r,dy+r)], fill=fill, outline=outline) + else: # circle / egg approx + dr.ellipse(bbox, fill=fill, outline=outline, width=ow) + if pt["label"]: + lb = pt["label"]; off = lb.get("p",[0,0]); txt = lb.get("t","") + try: fnt = ImageFont.truetype("/System/Library/Fonts/Helvetica.ttc", int(lb.get("s",20)*0.5)) + except Exception: fnt = ImageFont.load_default() + dr.text((dx+off[0]*r*1.5, dy+off[1]*r*1.5), txt, fill=(0,0,0,255), font=fnt, anchor="mm") + # title + ttl = state["title"] + if ttl and ttl.get("T",{}).get("t"): + t = ttl["T"]; box = ttl.get("B",{}); off = box.get("O",[10,10]) + try: fnt = ImageFont.truetype("/System/Library/Fonts/HelveticaNeue.ttc", int(t.get("s",25))) + except Exception: fnt = ImageFont.load_default() + dr.text((off[0], off[1]), t.get("t",""), fill=parse_color(t.get("c","black")), font=fnt) + # legend + if legend_rows and state.get("legend_flags",{}) and state["legend_flags"].get("-", True) is not False: + try: lf = ImageFont.truetype("/System/Library/Fonts/Helvetica.ttc", 16) + except Exception: lf = ImageFont.load_default() + lx, ly = 12, H-18*len(legend_rows)-14 + for pr, txt, fillc in legend_rows: + fc = parse_color(fillc) or (128,128,128,255) + dr.ellipse([lx, ly, lx+12, ly+12], fill=fc, outline=(0,0,0,255)) + dr.text((lx+18, ly-1), txt, fill=(0,0,0,255), font=lf) + ly += 18 + img.convert("RGB").save(out_png) + print(f"wrote {out_png} frame=[{vx:.1f},{vy:.1f},{vw:.1f}] legend_rows={len(legend_rows)}") + +if __name__ == "__main__": + ace, style, out = sys.argv[1], sys.argv[2], sys.argv[3] + c = load_chart(ace) + render(c, style, out) From c5a0a0e3cd1d4aa694469b0717be51081fc07df3 Mon Sep 17 00:00:00 2001 From: Sarah James Date: Wed, 8 Jul 2026 10:01:54 +0100 Subject: [PATCH 03/13] =?UTF-8?q?map-draw:=20P2=20milestone=20A=20?= =?UTF-8?q?=E2=80=94=20native=20styled=20(semantic)=20map=20renderer?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add a headless C++ entry point `export_styled_map(chart, projection_no, style_name, width, output)` (plus `ae_backend.map_draw.export_styled_map` pybind) that renders a chart's on-chart semantic styling — the `c["R"]` named styles + `c["p"]` base plot-spec — the way kateri's set_style + get_pdf does, so the native renderer is a drop-in for the report's map step. This is additive to and independent of the existing AD-chains export_map. Inside styled-draw.cc: - Style resolver (front -> background composition): walk the front style's ordered modifier list, recursively expanding {R:} references into a flat modifier list, tolerating undefined refs; merge legend, carry title, last-writer-wins viewport. - Selector engine over semantic attributes (clade `C`, boolean attrs like reference/vaccine, index `!i`, antigens/sera scope). - Per-point render model: fill / outline / outline-width / size / shape / shown / draw-order (raise/lower), from base plot-spec then modifiers. Base sizes carry kateri's x5 (_sizeScale); modifier sizes are absolute px. - Viewport transform+recenter+viewport chain reproduced exactly (the key fidelity step): absolute viewport = -roundedSize/2 + V.origin + hull_centre, where roundedSize = ceil(hull_extent + 1) — matching kateri's roundAndRecenter and the export_mapi_for_signature_pages recovery. No Y-flip. - Grid (#CCCCCC), plain legend (box, markers, per-row counts) and title, drawn baseline-anchored in Helvetica to match the golden. New cairo-surface primitives (used only by this path): `egg` (kateri's egg bezier), `text_font` (weight/slant, baseline anchor), and a Helvetica option on `text_size` so boxes size from the same metrics they draw with. Fidelity vs the kateri golden for a representative plain by-clade map (rasterised identically via pdftoppm): map content (grid + points + colours + frame) 1.2% pixel diff; full image incl. title/legend/labels 5.1%, the excess being font-rasterizer AA (cairo vs poppler Helvetica) — the known bounded tail that milestones B/C/I/K address. The viewport-convention gate (the P2 risk) is met. Co-Authored-By: Claude Opus 4.8 (1M context) --- cc/draw/cairo-surface.cc | 44 ++- cc/draw/cairo-surface.hh | 11 +- cc/map-draw/draw.hh | 9 + cc/map-draw/styled-draw.cc | 561 +++++++++++++++++++++++++++++++++++++ cc/py/map-draw.cc | 11 + meson.build | 3 +- 6 files changed, 635 insertions(+), 4 deletions(-) create mode 100644 cc/map-draw/styled-draw.cc diff --git a/cc/draw/cairo-surface.cc b/cc/draw/cairo-surface.cc index f177bca..261316f 100644 --- a/cc/draw/cairo-surface.cc +++ b/cc/draw/cairo-surface.cc @@ -123,6 +123,30 @@ namespace ae::draw cairo_stroke(context_); } + void CairoPdf::egg(double cx, double cy, double size, Color outline, double outline_width, Color fill) + { + // Reproduces kateri's _drawShape egg (draw_on_pdf.dart): two cubic beziers between the + // top (0, -r) and bottom (0, +r) apexes, y growing downward (matches our device space, + // no Y-flip). r = size/2. + const double r = size / 2.0; + cairo_new_path(context_); + cairo_move_to(context_, cx + 0.0, cy + r); + cairo_curve_to(context_, cx + r * 1.4, cy + r * 0.95, cx + r * 0.8, cy - r * 0.98, cx + 0.0, cy - r); + cairo_curve_to(context_, cx - r * 0.8, cy - r * 0.98, cx - r * 1.4, cy + r * 0.95, cx + 0.0, cy + r); + cairo_close_path(context_); + if (!fill.is_transparent()) { + set_source(context_, fill); + cairo_fill_preserve(context_); + } + if (outline_width > 0.0 && !outline.is_transparent()) { + set_source(context_, outline); + cairo_set_line_width(context_, outline_width); + cairo_stroke(context_); + } + else + cairo_new_path(context_); + } + void CairoPdf::filled_triangle(double x0, double y0, double x1, double y1, double x2, double y2, Color fill) { cairo_new_path(context_); @@ -240,10 +264,26 @@ namespace ae::draw } } - std::pair CairoPdf::text_size(std::string_view utf8, double font_size) + void CairoPdf::text_font(double x, double y, std::string_view utf8, double font_size, Color color, bool bold, bool italic) { + // Baseline-left anchor (matches kateri drawString at (origin.dx, origin.dy)): (x, y) is + // the left edge / baseline of the first glyph. const std::string str{utf8}; - cairo_select_font_face(context_, "sans-serif", CAIRO_FONT_SLANT_NORMAL, CAIRO_FONT_WEIGHT_NORMAL); + // kateri renders Latin1 text in Helvetica (Type1); use the same face so the native + // render's title/legend/labels match the golden's glyph shapes/metrics. + cairo_select_font_face(context_, "Helvetica", italic ? CAIRO_FONT_SLANT_ITALIC : CAIRO_FONT_SLANT_NORMAL, bold ? CAIRO_FONT_WEIGHT_BOLD : CAIRO_FONT_WEIGHT_NORMAL); + cairo_set_font_size(context_, font_size); + cairo_text_extents_t ext; + cairo_text_extents(context_, str.c_str(), &ext); + cairo_move_to(context_, x - ext.x_bearing, y); // left edge at x, baseline at y + set_source(context_, color); + cairo_show_text(context_, str.c_str()); + } + + std::pair CairoPdf::text_size(std::string_view utf8, double font_size, bool helvetica) + { + const std::string str{utf8}; + cairo_select_font_face(context_, helvetica ? "Helvetica" : "sans-serif", CAIRO_FONT_SLANT_NORMAL, CAIRO_FONT_WEIGHT_NORMAL); cairo_set_font_size(context_, font_size); cairo_text_extents_t ext; cairo_text_extents(context_, str.c_str(), &ext); diff --git a/cc/draw/cairo-surface.hh b/cc/draw/cairo-surface.hh index 1a27f2b..e19406a 100644 --- a/cc/draw/cairo-surface.hh +++ b/cc/draw/cairo-surface.hh @@ -44,6 +44,10 @@ namespace ae::draw void sector(double cx, double cy, double radius, double start_angle, double end_angle, Color outline, double outline_width, Color fill); void square(double cx, double cy, double side, Color outline, double outline_width, Color fill); void triangle(double cx, double cy, double radius, Color outline, double outline_width, Color fill); // equilateral, point up + // Egg (kateri PointShape.egg): a closed two-bezier egg of the given `size` (bounding + // diameter) centred at (cx, cy), reproducing kateri _drawShape's control points so a + // native render matches the kateri golden exactly. Transparent fill = outline only. + void egg(double cx, double cy, double size, Color outline, double outline_width, Color fill); void filled_triangle(double x0, double y0, double x1, double y1, double x2, double y2, Color fill); // arbitrary filled triangle // Axis-aligned rectangle with its top-left corner at (x, y). Transparent fill = outline only. void rectangle(double x, double y, double width, double height, Color outline, double outline_width, Color fill); @@ -63,8 +67,13 @@ namespace ae::draw // strokes a halo (default white) behind the glyphs (see text()). void text_rotated(double x, double y, std::string_view utf8, double font_size, Color color, double angle_degrees, double halo_width = 0.0, Color halo_color = WHITE); + // Like text(center=false) but with selectable weight/slant (for the semantic-style + // title/legend, which carry helvetica bold/italic). (x, y) is the glyph-box top-left. + void text_font(double x, double y, std::string_view utf8, double font_size, Color color, bool bold, bool italic); // Measure a string at the given font size: returns {width, height} in device units. - std::pair text_size(std::string_view utf8, double font_size); + // `helvetica` selects the Helvetica face (matching text_font) so callers that draw with + // text_font size their boxes/rows from the same metrics; default keeps the sans-serif face. + std::pair text_size(std::string_view utf8, double font_size, bool helvetica = false); private: _cairo_surface* surface_{nullptr}; diff --git a/cc/map-draw/draw.hh b/cc/map-draw/draw.hh index f243221..55351de 100644 --- a/cc/map-draw/draw.hh +++ b/cc/map-draw/draw.hh @@ -54,6 +54,15 @@ namespace ae::map_draw // Render one projection of a chart to output (extension .png -> raster, else PDF). void export_map(const ae::chart::v3::Chart& chart, ae::projection_index projection_no, const std::filesystem::path& output, const DrawSettings& settings = {}); + // ------------------------------------------------------------------ + // P2 milestone A: headless render of a chart's on-chart *semantic* styling (c["R"] named + // styles + c["p"] base plot-spec), mirroring kateri's set_style + get_pdf. This is the + // report map path (by-clade etc.), additive to and independent of export_map above (the + // fixed AD-chains pipeline). `style_name` selects a front style in chart.styles(); + // `width` is the output width in device px / PDF points. Output extension picks backend + // (.png raster, else PDF). See cc/map-draw/STYLED-DRAW.md and P2-RENDER-DESIGN.md §1.2/§2.2. + void export_styled_map(const ae::chart::v3::Chart& chart, ae::projection_index projection_no, std::string_view style_name, double width, const std::filesystem::path& output); + // Procrustes render: draw `secondary` framed like `primary`, with arrows for common // points (threshold 0.3) and an "RMS: {rms:.4f}" title. See make_pc in chains chart.py. void export_procrustes(const ae::chart::v3::Chart& primary, ae::projection_index primary_projection, const ae::chart::v3::Chart& secondary, diff --git a/cc/map-draw/styled-draw.cc b/cc/map-draw/styled-draw.cc new file mode 100644 index 0000000..84ed9cf --- /dev/null +++ b/cc/map-draw/styled-draw.cc @@ -0,0 +1,561 @@ +#include +#include +#include +#include +#include +#include +#include + +#include "ext/fmt.hh" +#include "map-draw/draw.hh" +#include "draw/cairo-surface.hh" +#include "ad/color.hh" +#include "chart/v3/chart.hh" +#include "chart/v3/styles.hh" +#include "chart/v3/legacy-plot-spec.hh" +#include "chart/v3/layout.hh" + +// ====================================================================== +// P2 milestone A — headless native render of a chart's on-chart *semantic* styling +// (c["R"] named styles + c["p"] base plot-spec), mirroring kateri's set_style + get_pdf +// for the report by-clade maps. Implements the §1.2 resolver and the §2.2 viewport +// transform+recenter+viewport chain in C++, then draws via cc/draw/cairo-surface. +// +// Viewport chain (the key fidelity step, derived from kateri lib/src/viewport.dart +// roundAndRecenter + lib/src/socket-events.dart get_viewport, and confirmed by +// py/ae/report/chart_modifier.py::export_mapi_for_signature_pages): +// * transformed layout = projection.transformed_layout() (stored 2x2 transform applied, +// NO Y-flip — post-transform Y is already screen-down). +// * kateri computes a native viewport = axis-aligned hull of the transformed layout, +// then roundAndRecenter: roundedSize = ceil(hull_extent + 1) per axis, and it shifts +// every point by adjust = roundedSize/2 - hull_centre so the native frame becomes +// [0,0,roundedSize]. The style's stored viewport V=[x,y,w,h] (the "-reset" family) is +// the "used" viewport expressed in this recentered space. +// * Since we draw in the (non-recentered) transformed space directly, the equivalent +// absolute viewport is origin = V.origin - adjust = -roundedSize/2 + V.origin + +// hull_centre, size = V.size. This equals the export_mapi_for_signature_pages +// recovery (-native/2 + used + native_center). +// +// Point sizes / outline widths / font sizes are absolute device pixels (kateri "sizePixels", +// independent of canvas width: sizePixels * pixelSize * canvasScale == sizePixels). Base +// plot-spec sizes carry kateri's x5 (_sizeScale); semantic-modifier `s`/`o` are used as-is. +// ====================================================================== + +namespace ae::map_draw +{ + using namespace ae::chart::v3; + + namespace + { + constexpr double kSizeScale = 5.0; // kateri PlotSpecLegacy._sizeScale (base plot-spec sizes) + + // Convert a chart-side ae::draw::v2::Color (a string-blocks wrapper) to a cairo ::Color. + // An empty color means "unset" (use fallback). A leading ':' is a colour *modifier* + // (e.g. ":bright" applied to the current colour): milestone A leaves the current colour + // unchanged and reports it via `is_modifier` (full modifier support is a later milestone). + ::Color to_cairo_color(const ae::draw::v2::Color& c, ::Color fallback, bool& is_modifier) + { + is_modifier = false; + if (c.empty()) + return fallback; + const std::string& s = c.blocks()[0]; + if (s.empty()) + return fallback; + if (s[0] == ':') { + is_modifier = true; + return fallback; + } + if (s == "T" || s == "transparent") + return TRANSPARENT; + return ::Color{std::string_view{s}}; + } + + // ---- dynamic::value helpers (selectors + semantic attribute matching) ---- + bool value_is_bool(const ae::dynamic::value& v) { return std::holds_alternative(v.data()); } + bool value_as_bool(const ae::dynamic::value& v) { return std::get(v.data()); } + + bool value_truthy(const ae::dynamic::value& v) + { + return std::visit( + [](const T& c) -> bool { + if constexpr (std::is_same_v) + return c; + else if constexpr (std::is_same_v) + return false; + else if constexpr (std::is_same_v) + return !static_cast(c).empty(); + else if constexpr (std::is_same_v) + return c != 0; + else if constexpr (std::is_same_v) + return c != 0.0; + else + return true; // object / array present => truthy + }, + v.data()); + } + + // A resolved per-point render state. + struct PR + { + ::Color fill{TRANSPARENT}; + ::Color outline{BLACK}; + double outline_width{1.0}; + double size{10.0}; // device px (diameter / box side) + point_shape::Shape shape{point_shape::Circle}; + bool shown{true}; + bool has_label{false}; + double label_dx{0.0}, label_dy{1.0}; + double label_size{20.0}; + std::string label_text{}; + }; + + struct LegendRowR + { + int priority{0}; + std::string text{}; + ::Color fill{TRANSPARENT}; + ::Color outline{BLACK}; + point_shape::Shape shape{point_shape::Circle}; + size_t count{0}; + }; + + // Resolution accumulator (§1.2): flat modifier list, viewport, merged legend, title. + struct Resolved + { + std::vector modifiers{}; + std::optional viewport{}; + semantic::Legend legend{}; + semantic::Title title{}; + bool title_set{false}; + }; + + void merge_legend(semantic::Legend& into, const semantic::Legend& from) + { + if (from.shown.has_value()) into.shown = from.shown; + if (from.add_counter.has_value()) into.add_counter = from.add_counter; + if (from.point_size.has_value()) into.point_size = from.point_size; + if (from.show_rows_with_zero_count.has_value()) into.show_rows_with_zero_count = from.show_rows_with_zero_count; + if (from.box.has_value()) into.box = from.box; + if (from.row_style.has_value()) into.row_style = from.row_style; + if (from.title.has_value()) into.title = from.title; + } + + // Walk a front style, recursively expanding {R:} references into a flat modifier + // list (§1.2). Undefined references are tolerated. Viewport is last-writer-wins across + // the traversal; legend fields merge; title comes from the front (depth 0) style. + void resolve(const semantic::Styles& styles, const std::string& name, Resolved& out, int depth) + { + const semantic::Style* st = styles.find_if_exists(name); + if (st == nullptr) + return; // tolerate undefined ref (e.g. "-new-1" on a chart with no previous) + if (st->viewport.has_value()) + out.viewport = st->viewport; + merge_legend(out.legend, st->legend); + if (depth == 0) { + out.title = st->plot_title; + out.title_set = true; + } + for (const auto& m : st->modifiers) { + if (!m.parent.empty()) + resolve(styles, m.parent, out, depth + 1); + else + out.modifiers.push_back(&m); + } + } + + // Match a selector object against one point's semantic attributes (§1.2). `index_out` + // receives the "!i" index if present (caller resolves it against the candidate range). + bool selector_matches(const ae::dynamic::value& selector, const SemanticAttributes& sem, bool& has_index, long& index_out) + { + const auto* obj = std::get_if(&selector.data()); + if (obj == nullptr) + return true; // empty / non-object selector => all candidates + for (const auto& [key, val] : obj->data()) { + if (key == "!i") { + has_index = true; + index_out = std::visit( + [](const T& c) -> long { + if constexpr (std::is_same_v) + return c; + else if constexpr (std::is_same_v) + return static_cast(c); + else + return -1; + }, + val.data()); + } + else if (key == "C") { + if (!sem.has_clade(val.as_string_or_empty())) + return false; + } + else { + const auto& sv = sem.get(key); + if (value_is_bool(val)) { + if (value_truthy(sv) != value_as_bool(val)) + return false; + } + else if (sv.as_string_or_empty() != val.as_string_or_empty()) + return false; + } + } + return true; + } + + } // namespace + + // ---------------------------------------------------------------------- + + void export_styled_map(const Chart& chart, projection_index projection_no, std::string_view style_name, double width, const std::filesystem::path& output) + { + if (chart.projections().empty()) + throw std::runtime_error{"cannot draw styled map: chart has no projections"}; + if (projection_no >= chart.projections().size()) + throw std::runtime_error{"cannot draw styled map: projection index out of range"}; + + const auto& projection = chart.projections()[projection_no]; + const Layout layout = projection.transformed_layout(); + + const size_t n_antigens = chart.antigens().size().get(); + const size_t n_sera = chart.sera().size().get(); + const size_t n_points = layout.number_of_points().get(); + + // ---- semantic accessor per point ---- + const auto sem_of = [&](size_t i) -> const SemanticAttributes& { + if (i < n_antigens) + return chart.antigens()[antigen_index{i}].semantic(); + return chart.sera()[serum_index{i - n_antigens}].semantic(); + }; + + // ---- base per-point render state from c["p"] legacy plot spec ---- + const auto& base = chart.legacy_plot_spec(); + std::vector pr(n_points); + { + const auto& styles = base.style_for_point(); + const auto& pstyles = base.styles(); + for (size_t i = 0; i < n_points; ++i) { + PR& p = pr[i]; + if (i < styles.size() && styles[i] < pstyles.size()) { + const PointStyle& bs = pstyles[styles[i]]; + bool mod = false; + p.fill = to_cairo_color(bs.fill(), TRANSPARENT, mod); // kateri base default fill "transparent" + p.outline = to_cairo_color(bs.outline(), BLACK, mod); // kateri base default outline "black" + p.outline_width = bs.outline_width().value_or(1.0); + p.size = bs.size().value_or(1.0) * kSizeScale; // base plot-spec sizes carry x5 + if (bs.shape().has_value()) + p.shape = bs.shape()->get(); + p.shown = bs.shown().value_or(true); + } + } + } + + // ---- draw order from c["p"]["d"] (fallback: natural order) ---- + std::vector order; + for (const auto pt : base.drawing_order()) + order.push_back(pt.get()); + if (order.empty()) { + for (size_t i = 0; i < n_points; ++i) + order.push_back(i); + } + + // ---- resolve the front style (§1.2) ---- + Resolved resolved; + resolve(chart.styles(), std::string{style_name}, resolved, 0); + + std::vector legend_rows; + + // ---- apply modifiers in flat order ---- + for (const semantic::StyleModifier* mp : resolved.modifiers) { + const semantic::StyleModifier& m = *mp; + // candidate range by antigens/sera select + size_t cand_begin = 0, cand_end = n_points; + if (m.select_antigens_sera == semantic::SelectAntigensSera::antigens_only) { + cand_begin = 0; + cand_end = n_antigens; + } + else if (m.select_antigens_sera == semantic::SelectAntigensSera::sera_only) { + cand_begin = n_antigens; + cand_end = n_points; + } + + std::vector matched; + { + bool has_index = false; + long sel_index = -1; + // "!i" is per-selector; determine it once by probing the selector structure. + { + bool hi = false; + long idx = -1; + // use a dummy sem (empty selector obj path won't touch it); probe on any point + if (n_points > 0) + selector_matches(m.selector, sem_of(cand_begin < n_points ? cand_begin : 0), hi, idx); + has_index = hi; + sel_index = idx; + } + if (has_index) { + const size_t abs_idx = (m.select_antigens_sera == semantic::SelectAntigensSera::sera_only) ? (n_antigens + static_cast(sel_index)) : static_cast(sel_index); + if (sel_index >= 0 && abs_idx >= cand_begin && abs_idx < cand_end) + matched.push_back(abs_idx); + } + else { + for (size_t i = cand_begin; i < cand_end; ++i) { + bool hi = false; + long idx = -1; + if (selector_matches(m.selector, sem_of(i), hi, idx)) + matched.push_back(i); + } + } + } + + // apply point-style fields + const PointStyle& ms = m.point_style; + for (const size_t i : matched) { + PR& p = pr[i]; + if (!ms.fill().empty()) { + bool mod = false; + const ::Color fc = to_cairo_color(ms.fill(), p.fill, mod); + if (!mod) + p.fill = fc; // ":bright" and other modifiers keep the current fill (milestone A) + } + if (!ms.outline().empty()) { + bool mod = false; + const ::Color oc = to_cairo_color(ms.outline(), p.outline, mod); + if (!mod) + p.outline = oc; + } + if (ms.outline_width().has_value()) + p.outline_width = *ms.outline_width(); + if (ms.size().has_value()) + p.size = *ms.size(); // semantic-modifier size is absolute px (no x5) + if (ms.shape().has_value()) + p.shape = ms.shape()->get(); + if (ms.shown().has_value()) + p.shown = *ms.shown(); + if (ms.label().text.has_value() && !ms.label().text->empty()) { + p.has_label = true; + p.label_text = *ms.label().text; + p.label_dx = ms.label().offset.x; + p.label_dy = ms.label().offset.y; + p.label_size = ms.label().size; + } + } + + // drawing order raise / lower + if (m.order != semantic::DrawingOrderModifier::no_change && !matched.empty()) { + std::vector is_matched(n_points, false); + for (const size_t i : matched) + if (i < n_points) + is_matched[i] = true; + std::vector kept, moved; + for (const size_t i : order) { + if (is_matched[i]) + moved.push_back(i); + else + kept.push_back(i); + } + order.clear(); + if (m.order == semantic::DrawingOrderModifier::raise) { // matched go on top (drawn last) + order.insert(order.end(), kept.begin(), kept.end()); + order.insert(order.end(), moved.begin(), moved.end()); + } + else { // lower: matched go to the bottom (drawn first) + order.insert(order.end(), moved.begin(), moved.end()); + order.insert(order.end(), kept.begin(), kept.end()); + } + } + + // legend row + if (!m.legend.text.empty() || m.legend.priority != 0) { + LegendRowR row; + row.priority = m.legend.priority; + row.text = m.legend.text; + bool mod = false; + row.fill = to_cairo_color(ms.fill(), TRANSPARENT, mod); + row.outline = to_cairo_color(ms.outline(), BLACK, mod); + if (ms.shape().has_value()) + row.shape = ms.shape()->get(); + row.count = matched.size(); + legend_rows.push_back(std::move(row)); + } + } + + // ---- viewport chain (§2.2) ---- + constexpr double inf = std::numeric_limits::infinity(); + double min_x{inf}, min_y{inf}, max_x{-inf}, max_y{-inf}; + for (const auto pn : layout.number_of_points()) { + if (const auto c = layout[pn]; c.exists()) { + min_x = std::min(min_x, c[DIMX]); + min_y = std::min(min_y, c[DIMY]); + max_x = std::max(max_x, c[DIMX]); + max_y = std::max(max_y, c[DIMY]); + } + } + if (min_x > max_x) + throw std::runtime_error{"cannot draw styled map: no points with coordinates"}; + const double centre_x = (min_x + max_x) / 2.0; + const double centre_y = (min_y + max_y) / 2.0; + const double rounded_x = std::ceil(max_x - min_x + 1.0); + const double rounded_y = std::ceil(max_y - min_y + 1.0); + + double vp_x, vp_y, vp_w, vp_h; + if (resolved.viewport.has_value()) { + const auto& V = *resolved.viewport; + vp_w = V.width; + vp_h = V.height; + vp_x = -rounded_x / 2.0 + static_cast(V.x) + centre_x; + vp_y = -rounded_y / 2.0 + static_cast(V.y) + centre_y; + } + else { // no style viewport => kateri's native (recentered) frame + vp_w = rounded_x; + vp_h = rounded_y; + vp_x = centre_x - rounded_x / 2.0; + vp_y = centre_y - rounded_y / 2.0; + } + + const double image_w = width; + const double image_h = width * vp_h / vp_w; + const auto dev_x = [=](double x) { return (x - vp_x) / vp_w * image_w; }; + const auto dev_y = [=](double y) { return (y - vp_y) / vp_h * image_h; }; // NO Y-flip + + ae::draw::CairoPdf surface{output, image_w, image_h}; + surface.background(WHITE); + + // ---- grid (kateri grid: colour #CCCCCC as rendered in the golden, 1px, step 1 map unit + // from vp origin; draw_on.dart's abstract default 0xCCCCCC, matched to the golden) ---- + { + const ::Color grid{0xCCCCCC}; + const double step_x = image_w / vp_w; + const double step_y = image_h / vp_h; + for (double gx = 0.0; gx <= image_w + 0.5; gx += step_x) + surface.line(gx, 0.0, gx, image_h, grid, 1.0); + for (double gy = 0.0; gy <= image_h + 0.5; gy += step_y) + surface.line(0.0, gy, image_w, gy, grid, 1.0); + } + + // ---- points (draw order: first = bottom) ---- + const auto draw_point = [&](size_t i) { + const auto c = layout[point_index{i}]; + if (!c.exists() || !pr[i].shown) + return; + const PR& p = pr[i]; + const double cx = dev_x(c[DIMX]), cy = dev_y(c[DIMY]); + const double s = p.size; + const double ow = p.outline_width; + switch (p.shape) { + case point_shape::Box: + surface.square(cx, cy, s, p.outline, ow, p.fill); + break; + case point_shape::Triangle: + surface.triangle(cx, cy, s / 2.0, p.outline, ow, p.fill); + break; + case point_shape::Egg: + case point_shape::UglyEgg: + surface.egg(cx, cy, s, p.outline, ow, p.fill); + break; + case point_shape::Circle: + default: + surface.circle(cx, cy, s / 2.0, p.outline, ow, p.fill); + break; + } + }; + for (const size_t i : order) + draw_point(i); + + // ---- point labels (on top of points) ---- + for (const size_t i : order) { + if (!pr[i].has_label) + continue; + const auto c = layout[point_index{i}]; + if (!c.exists()) + continue; + const PR& p = pr[i]; + const double cx = dev_x(c[DIMX]), cy = dev_y(c[DIMY]); + const auto [tw, th] = surface.text_size(p.label_text, p.label_size, true); + const double point_r = (p.size + p.outline_width) / 2.0; // device px (kateri pointSize) + // kateri addPointLabel::labelOffset (draw_on.dart), in device px; baseline-left anchor. + const auto lab_off = [point_r](double off, double extent, bool vertical) -> double { + if (off >= 1.0) + return point_r * off + (vertical ? extent : 0.0); + if (off > -1.0) + return point_r * off + (vertical ? extent * (off + 1.0) / 2.0 : extent * (off - 1.0) / 2.0); + return point_r * off - (vertical ? 0.0 : extent); + }; + const double lx = cx + lab_off(p.label_dx, tw, false); + const double ly = cy + lab_off(p.label_dy, th, true); + surface.text_font(lx, ly, p.label_text, p.label_size, BLACK, false, false); + } + + // ---- legend (kateri _Defaults.legend: bottom-left "Bl", offset (10,-10), white box, + // black 1px border, padding v5/h10; point size + row text/interline from -clades) ---- + const bool legend_shown = resolved.legend.shown.value_or(true) && !legend_rows.empty(); + if (legend_shown) { + std::sort(legend_rows.begin(), legend_rows.end(), [](const LegendRowR& a, const LegendRowR& b) { return a.priority < b.priority; }); + const bool add_counter = resolved.legend.add_counter.value_or(false); + const double point_size = resolved.legend.point_size.value_or(20.0); + double text_size = 20.0, interline = 0.3; + if (resolved.legend.row_style.has_value()) { + if (resolved.legend.row_style->font_size.has_value()) + text_size = *resolved.legend.row_style->font_size; + if (resolved.legend.row_style->interline.has_value()) + interline = *resolved.legend.row_style->interline; + } + const double pad_l = 10.0, pad_r = 10.0, pad_t = 5.0, pad_b = 5.0; + const double point_space = point_size * (interline + 1.2); + const double count_left_pad = add_counter ? text_size * 1.0 : 0.0; + + double max_text_w = 0.0, row_h = 0.0, max_count_w = 0.0; + for (const auto& r : legend_rows) { + const auto [w, h] = surface.text_size(r.text, text_size, true); + max_text_w = std::max(max_text_w, w); + row_h = std::max(row_h, h); + if (add_counter) { + const auto [cw, ch] = surface.text_size(fmt::format("{}", r.count), text_size, true); + (void)ch; + max_count_w = std::max(max_count_w, cw); + } + } + const double n = static_cast(legend_rows.size()); + const double box_w = max_text_w + point_space + count_left_pad + max_count_w + pad_l + pad_r; + const double box_h = row_h + row_h * (n - 1.0) * (interline + 1.0) + row_h * 0.4 + pad_t + pad_b; + const double box_x = 0.0 + 10.0; // vp.left device (=0) + offset.dx (10 px) + const double box_y = image_h - 10.0 - box_h; // vp.bottom device (=image_h) + offset.dy(-10) - height + surface.rectangle(box_x, box_y, box_w, box_h, BLACK, 1.0, WHITE); + + const double dx = box_x + pad_l; + double baseline = box_y + pad_t + row_h; // kateri: box.origin.dy + textSize[0].height + padding.top + for (const auto& r : legend_rows) { + surface.circle(dx + point_size / 2.0, baseline - row_h * 0.35, point_size / 2.0, r.outline, 1.0, r.fill); + surface.text_font(dx + point_space, baseline, r.text, text_size, BLACK, false, false); + if (add_counter) { + const auto [cw, ch] = surface.text_size(fmt::format("{}", r.count), text_size, true); + (void)ch; + surface.text_font(dx + point_space + max_text_w + count_left_pad + max_count_w - cw, baseline, fmt::format("{}", r.count), text_size, BLACK, false, false); + } + baseline += row_h * (interline + 1.0); + } + } + + // ---- title (kateri _Defaults.title "tl", offset from box O; helvetica bold/normal) ---- + if (resolved.title_set && resolved.title.shown.value_or(true) && resolved.title.text.text.has_value() && !resolved.title.text.text->empty()) { + const auto& t = resolved.title; + double off_x = 30.0, off_y = 30.0; // kateri title default offset + if (t.box.has_value() && t.box->offset.has_value()) { + off_x = (*t.box->offset)[0]; + off_y = (*t.box->offset)[1]; + } + const double font = t.text.font_size.value_or(28.0); + const bool bold = t.text.font_weight.value_or("normal") == "bold"; + const bool italic = t.text.font_slant.value_or("normal") == "italic"; + ::Color col = BLACK; + if (t.text.color.has_value()) + col = ::Color{static_cast(*t.text.color)}; + // origin "tl": box top-left at device (off_x, off_y); kateri draws the baseline at + // box.origin.dy + textHeight + padding.top (padding 0). Baseline-left anchor. + const auto [ttw, tth] = surface.text_size(*t.text.text, font, true); + (void)ttw; + surface.text_font(off_x, off_y + tth, *t.text.text, font, col, bold, italic); + } + } + +} // namespace ae::map_draw + +// ---------------------------------------------------------------------- diff --git a/cc/py/map-draw.cc b/cc/py/map-draw.cc index 99328c2..1d8da36 100644 --- a/cc/py/map-draw.cc +++ b/cc/py/map-draw.cc @@ -54,6 +54,17 @@ void ae::py::map_draw(pybind11::module_& mdl) }, "primary_ace"_a, "secondary_ace"_a, "output"_a, "size"_a = 800.0, "mapi"_a = std::nullopt, "coloring"_a = std::nullopt, pybind11::doc("Procrustes render (AD make_pc): draw primary with arrows to secondary (threshold 0.3), title 'RMS: x.xxxx'.")); + + // --- P2 milestone A: styled (semantic c["R"] + c["p"]) render, kateri-compatible drop-in --- + sub.def( + "export_styled_map", + [](const std::filesystem::path& ace, const std::filesystem::path& output, const std::string& style, double width, size_t projection_no) { + const Chart chart{ace}; + ae::map_draw::export_styled_map(chart, projection_index{projection_no}, style, width, output); + }, + "ace"_a, "output"_a, "style"_a, "width"_a = 800.0, "projection_no"_a = 0, + pybind11::doc("Render a chart's on-chart semantic style (c[\"R\"] named style + c[\"p\"] base plot-spec) to output " + "(.png raster, else PDF), mirroring kateri set_style + get_pdf. P2 milestone A.")); } // ---------------------------------------------------------------------- diff --git a/meson.build b/meson.build index 9d5223e..83fccf4 100644 --- a/meson.build +++ b/meson.build @@ -127,6 +127,7 @@ sources_py = [ 'cc/py/hidb.cc', # --- hidb (subsystem #2) --- 'cc/py/map-draw.cc', # --- map-draw (subsystem #1, revived) --- 'cc/map-draw/draw.cc', # --- map-draw (subsystem #1, revived) --- + 'cc/map-draw/styled-draw.cc', # --- map-draw P2 milestone A (styled/semantic render) --- 'cc/draw/cairo-surface.cc', # --- map-draw (subsystem #1, revived) --- ] @@ -318,7 +319,7 @@ geo_draw = executable( # --- map-draw (subsystem #1, revived) — headless antigenic-map renderer, fidelity # port of AD acmacs-map-draw ChartDraw for the chains-202105 web app. Reuses the # shared cc/draw/cairo-surface (Cairo linked here). See cc/map-draw/TODO.md. --- -sources_map_draw = ['cc/map-draw/draw.cc', 'cc/draw/cairo-surface.cc'] +sources_map_draw = ['cc/map-draw/draw.cc', 'cc/map-draw/styled-draw.cc', 'cc/draw/cairo-surface.cc'] map_draw = executable( 'map-draw', From f853fbe698eaab96ea3600fd20ccfaee25b644b4 Mon Sep 17 00:00:00 2001 From: Sarah James Date: Wed, 8 Jul 2026 14:51:07 +0100 Subject: [PATCH 04/13] =?UTF-8?q?map-draw:=20P2=20milestone=20E=20?= =?UTF-8?q?=E2=80=94=20full=20selector=20coverage,=20composed=20families,?= =?UTF-8?q?=20:bright/:pale=20colour=20model?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Extend the native styled (semantic) map renderer (styled-draw.cc) to cover all of the seasonal report's map style families, not just the plain by-clade map proven in milestone A: - Selector engine rewritten to mirror kateri semanticMatch exactly: boolean selector value = attribute-presence test; missing attribute never matches a non-boolean value; list-valued attribute matched by contains; otherwise scalar equality. Adds per-side !i index and !D date-range selectors. This covers reference, passage, continent/country, older-than, new, serology, vaccine and sequenced selectors (a prior string-compare path could spuriously match-all on an absent numeric attribute). - Colour model: implement kateri ColorAndModifier semantics as a deferred per-point pale flag. :pale desaturates + lightens (HSV, factor 0.4); :bright clears the pale modifier keeping the base colour. Backgrounds that pale every point (e.g. the -pale family) now render pale, and :bright highlights come back to full colour. - info- front styles: whitespace-only title is treated as blank and skipped; combined with the importer's inverted legend flag this yields blank-title, no-legend maps as specified. Composed families (-reset, -new-*, -vaccines*/-no-label, -pale, -o6m/-o12m and the by-clade since-6m/12m fronts) resolve through the existing front to background reference flattening; undefined references stay tolerated. Co-Authored-By: Claude Opus 4.8 (1M context) --- cc/map-draw/styled-draw.cc | 249 ++++++++++++++++++++++--------------- 1 file changed, 152 insertions(+), 97 deletions(-) diff --git a/cc/map-draw/styled-draw.cc b/cc/map-draw/styled-draw.cc index 84ed9cf..307dd6f 100644 --- a/cc/map-draw/styled-draw.cc +++ b/cc/map-draw/styled-draw.cc @@ -10,6 +10,7 @@ #include "map-draw/draw.hh" #include "draw/cairo-surface.hh" #include "ad/color.hh" +#include "ad/color-hsv.hh" #include "chart/v3/chart.hh" #include "chart/v3/styles.hh" #include "chart/v3/legacy-plot-spec.hh" @@ -49,47 +50,101 @@ namespace ae::map_draw { constexpr double kSizeScale = 5.0; // kateri PlotSpecLegacy._sizeScale (base plot-spec sizes) - // Convert a chart-side ae::draw::v2::Color (a string-blocks wrapper) to a cairo ::Color. - // An empty color means "unset" (use fallback). A leading ':' is a colour *modifier* - // (e.g. ":bright" applied to the current colour): milestone A leaves the current colour - // unchanged and reports it via `is_modifier` (full modifier support is a later milestone). - ::Color to_cairo_color(const ae::draw::v2::Color& c, ::Color fallback, bool& is_modifier) + // ---- colour model (kateri lib/src/color.dart ColorAndModifier) ---- + // A style colour is a base colour string plus an optional deferred modifier. The + // report uses two modifiers: ":pale" (desaturate + lighten) and ":bright" (clear the + // pale modifier, keeping the base colour). Milestone E tracks the (base, pale) pair + // per point exactly like kateri, so a ":pale" background (e.g. -pale on the serology + // map) really pales the clade colours and a subsequent ":bright" (vaccine/serology + // highlight) brings them back to full colour. + enum class ColorAction { keep, set, pale, bright }; + struct ParsedColor + { + ColorAction action{ColorAction::keep}; + ::Color color{TRANSPARENT}; // valid only when action == set + }; + + ParsedColor parse_color(const ae::draw::v2::Color& c) { - is_modifier = false; if (c.empty()) - return fallback; + return {}; const std::string& s = c.blocks()[0]; if (s.empty()) - return fallback; + return {}; if (s[0] == ':') { - is_modifier = true; - return fallback; + if (s == ":pale") + return {ColorAction::pale, {}}; + if (s == ":bright") + return {ColorAction::bright, {}}; + return {}; // unknown modifier: keep current colour (kateri warns + ignores) } if (s == "T" || s == "transparent") - return TRANSPARENT; - return ::Color{std::string_view{s}}; + return {ColorAction::set, TRANSPARENT}; + return {ColorAction::set, ::Color{std::string_view{s}}}; + } + + // kateri ColorAndModifier.color for the ":pale" modifier (pale factor 0.4). + ::Color pale_color(::Color base) + { + constexpr double pale_factor = 0.4; + acmacs::color::HSV hsv{base}; + if (hsv.s > 0.0) { + hsv.s = hsv.s * pale_factor; + hsv.v = hsv.v + (1.0 - hsv.v) * (1.0 - pale_factor); + } + else { // grey / black / white (saturation 0): lighten value only + double val = hsv.v / pale_factor; + if (val > 1.0) + val = 1.0; + else if (hsv.v == 0.0) + val = 0.5; // black -> mid grey + hsv.v = val; + } + const uint32_t alpha = base.raw_value() & 0xFF000000u; // preserve transparency byte + return ::Color{alpha | (hsv.rgb() & 0x00FFFFFFu)}; + } + + // Apply a parsed colour to a (base, pale) pair (kateri ColorAndModifier.modify). + void apply_color(const ParsedColor& pc, ::Color& color, bool& pale) + { + switch (pc.action) { + case ColorAction::keep: + break; + case ColorAction::set: + color = pc.color; + pale = false; + break; + case ColorAction::bright: + pale = false; // clear pale modifier, base colour unchanged + break; + case ColorAction::pale: + pale = true; + break; + } + } + + // Concrete colour for a legend row / title (no deferred modifier): ":pale"/":bright" + // fall back to the given default. + ::Color concrete_color(const ae::draw::v2::Color& c, ::Color fallback) + { + const ParsedColor pc = parse_color(c); + return pc.action == ColorAction::set ? pc.color : fallback; } // ---- dynamic::value helpers (selectors + semantic attribute matching) ---- bool value_is_bool(const ae::dynamic::value& v) { return std::holds_alternative(v.data()); } bool value_as_bool(const ae::dynamic::value& v) { return std::get(v.data()); } - bool value_truthy(const ae::dynamic::value& v) + long value_as_long(const ae::dynamic::value& v) { return std::visit( - [](const T& c) -> bool { - if constexpr (std::is_same_v) + [](const T& c) -> long { + if constexpr (std::is_same_v) return c; - else if constexpr (std::is_same_v) - return false; - else if constexpr (std::is_same_v) - return !static_cast(c).empty(); - else if constexpr (std::is_same_v) - return c != 0; else if constexpr (std::is_same_v) - return c != 0.0; + return static_cast(c); else - return true; // object / array present => truthy + return -1; }, v.data()); } @@ -98,7 +153,9 @@ namespace ae::map_draw struct PR { ::Color fill{TRANSPARENT}; + bool fill_pale{false}; ::Color outline{BLACK}; + bool outline_pale{false}; double outline_width{1.0}; double size{10.0}; // device px (diameter / box side) point_shape::Shape shape{point_shape::Circle}; @@ -163,40 +220,65 @@ namespace ae::map_draw } } - // Match a selector object against one point's semantic attributes (§1.2). `index_out` - // receives the "!i" index if present (caller resolves it against the candidate range). - bool selector_matches(const ae::dynamic::value& selector, const SemanticAttributes& sem, bool& has_index, long& index_out) + // kateri Antigen.withinDateRange (chart.dart): a "!D": [first, last] range test. + bool within_date_range(std::string_view date, std::string_view first, std::string_view last) + { + if (date.empty()) + return first.empty(); // dateless antigen is "at the beginning of all dates" + return (first.empty() || first <= date) && (last.empty() || last > date); + } + + // kateri semanticMatch (chart.dart): one selector key/value vs the point's attributes. + // * a bool selector value is a *presence* test (attr present == value); + // * otherwise a missing attribute never matches; + // * a list-valued attribute (clades "C") matches if it contains the value; + // * otherwise scalar equality. + bool semantic_match(std::string_view key, const ae::dynamic::value& selval, const SemanticAttributes& sem) + { + const ae::dynamic::value& attr = sem.get(key); + if (value_is_bool(selval)) + return value_as_bool(selval) == !attr.is_null(); + if (attr.is_null()) + return false; + if (const auto* arr = std::get_if(&attr.data())) { // clades list: contains + for (const auto& el : arr->data()) { + if (el == selval) + return true; + } + return false; + } + return attr == selval; + } + + // Match a whole selector object against one point (kateri PlotSpec.selectPoints::match): + // handles "!i" (per-side index), "!D" (antigen date range), and semantic keys. An empty + // (or non-object) selector matches every candidate. + bool point_matches(const ae::dynamic::value& selector, const SemanticAttributes& sem, long agsr_no, std::string_view date, bool is_antigen) { const auto* obj = std::get_if(&selector.data()); if (obj == nullptr) - return true; // empty / non-object selector => all candidates + return true; for (const auto& [key, val] : obj->data()) { if (key == "!i") { - has_index = true; - index_out = std::visit( - [](const T& c) -> long { - if constexpr (std::is_same_v) - return c; - else if constexpr (std::is_same_v) - return static_cast(c); - else - return -1; - }, - val.data()); - } - else if (key == "C") { - if (!sem.has_clade(val.as_string_or_empty())) + if (agsr_no != value_as_long(val)) return false; } - else { - const auto& sv = sem.get(key); - if (value_is_bool(val)) { - if (value_truthy(sv) != value_as_bool(val)) - return false; + else if (key == "!D") { + if (!is_antigen) + return false; + std::string_view first{}, last{}; + if (const auto* arr = std::get_if(&val.data())) { + const auto& d = arr->data(); + if (d.size() >= 1) + first = d[0].as_string_or_empty(); + if (d.size() >= 2) + last = d[1].as_string_or_empty(); } - else if (sv.as_string_or_empty() != val.as_string_or_empty()) + if (!within_date_range(date, first, last)) return false; } + else if (!semantic_match(key, val, sem)) + return false; } return true; } @@ -236,9 +318,8 @@ namespace ae::map_draw PR& p = pr[i]; if (i < styles.size() && styles[i] < pstyles.size()) { const PointStyle& bs = pstyles[styles[i]]; - bool mod = false; - p.fill = to_cairo_color(bs.fill(), TRANSPARENT, mod); // kateri base default fill "transparent" - p.outline = to_cairo_color(bs.outline(), BLACK, mod); // kateri base default outline "black" + apply_color(parse_color(bs.fill()), p.fill, p.fill_pale); // kateri base default fill "transparent" + apply_color(parse_color(bs.outline()), p.outline, p.outline_pale); // kateri base default outline "black" p.outline_width = bs.outline_width().value_or(1.0); p.size = bs.size().value_or(1.0) * kSizeScale; // base plot-spec sizes carry x5 if (bs.shape().has_value()) @@ -278,50 +359,20 @@ namespace ae::map_draw } std::vector matched; - { - bool has_index = false; - long sel_index = -1; - // "!i" is per-selector; determine it once by probing the selector structure. - { - bool hi = false; - long idx = -1; - // use a dummy sem (empty selector obj path won't touch it); probe on any point - if (n_points > 0) - selector_matches(m.selector, sem_of(cand_begin < n_points ? cand_begin : 0), hi, idx); - has_index = hi; - sel_index = idx; - } - if (has_index) { - const size_t abs_idx = (m.select_antigens_sera == semantic::SelectAntigensSera::sera_only) ? (n_antigens + static_cast(sel_index)) : static_cast(sel_index); - if (sel_index >= 0 && abs_idx >= cand_begin && abs_idx < cand_end) - matched.push_back(abs_idx); - } - else { - for (size_t i = cand_begin; i < cand_end; ++i) { - bool hi = false; - long idx = -1; - if (selector_matches(m.selector, sem_of(i), hi, idx)) - matched.push_back(i); - } - } + for (size_t i = cand_begin; i < cand_end; ++i) { + const bool is_ag = i < n_antigens; + const long agsr = is_ag ? static_cast(i) : static_cast(i - n_antigens); + const std::string_view date = is_ag ? static_cast(chart.antigens()[antigen_index{i}].date()) : std::string_view{}; + if (point_matches(m.selector, sem_of(i), agsr, date, is_ag)) + matched.push_back(i); } - // apply point-style fields + // apply point-style fields (colours track the kateri (base, pale) pair) const PointStyle& ms = m.point_style; for (const size_t i : matched) { PR& p = pr[i]; - if (!ms.fill().empty()) { - bool mod = false; - const ::Color fc = to_cairo_color(ms.fill(), p.fill, mod); - if (!mod) - p.fill = fc; // ":bright" and other modifiers keep the current fill (milestone A) - } - if (!ms.outline().empty()) { - bool mod = false; - const ::Color oc = to_cairo_color(ms.outline(), p.outline, mod); - if (!mod) - p.outline = oc; - } + apply_color(parse_color(ms.fill()), p.fill, p.fill_pale); + apply_color(parse_color(ms.outline()), p.outline, p.outline_pale); if (ms.outline_width().has_value()) p.outline_width = *ms.outline_width(); if (ms.size().has_value()) @@ -368,9 +419,8 @@ namespace ae::map_draw LegendRowR row; row.priority = m.legend.priority; row.text = m.legend.text; - bool mod = false; - row.fill = to_cairo_color(ms.fill(), TRANSPARENT, mod); - row.outline = to_cairo_color(ms.outline(), BLACK, mod); + row.fill = concrete_color(ms.fill(), TRANSPARENT); + row.outline = concrete_color(ms.outline(), BLACK); if (ms.shape().has_value()) row.shape = ms.shape()->get(); row.count = matched.size(); @@ -440,20 +490,22 @@ namespace ae::map_draw const double cx = dev_x(c[DIMX]), cy = dev_y(c[DIMY]); const double s = p.size; const double ow = p.outline_width; + const ::Color fill = p.fill_pale ? pale_color(p.fill) : p.fill; // deferred ":pale" (kateri) + const ::Color outline = p.outline_pale ? pale_color(p.outline) : p.outline; switch (p.shape) { case point_shape::Box: - surface.square(cx, cy, s, p.outline, ow, p.fill); + surface.square(cx, cy, s, outline, ow, fill); break; case point_shape::Triangle: - surface.triangle(cx, cy, s / 2.0, p.outline, ow, p.fill); + surface.triangle(cx, cy, s / 2.0, outline, ow, fill); break; case point_shape::Egg: case point_shape::UglyEgg: - surface.egg(cx, cy, s, p.outline, ow, p.fill); + surface.egg(cx, cy, s, outline, ow, fill); break; case point_shape::Circle: default: - surface.circle(cx, cy, s / 2.0, p.outline, ow, p.fill); + surface.circle(cx, cy, s / 2.0, outline, ow, fill); break; } }; @@ -535,7 +587,10 @@ namespace ae::map_draw } // ---- title (kateri _Defaults.title "tl", offset from box O; helvetica bold/normal) ---- - if (resolved.title_set && resolved.title.shown.value_or(true) && resolved.title.text.text.has_value() && !resolved.title.text.text->empty()) { + // The "info-" front styles carry a whitespace-only title (" ") — a deliberate blank + // title (§1.2); skip drawing it entirely so nothing is rendered. + const auto title_is_blank = [](std::string_view s) { return s.find_first_not_of(" \t") == std::string_view::npos; }; + if (resolved.title_set && resolved.title.shown.value_or(true) && resolved.title.text.text.has_value() && !title_is_blank(*resolved.title.text.text)) { const auto& t = resolved.title; double off_x = 30.0, off_y = 30.0; // kateri title default offset if (t.box.has_value() && t.box->offset.has_value()) { From 95c1b7accbb45c00814117e2a0aeb1ce33a3d514 Mon Sep 17 00:00:00 2001 From: Sarah James Date: Wed, 8 Jul 2026 14:55:09 +0100 Subject: [PATCH 05/13] =?UTF-8?q?map-draw:=20P2=20milestone=20C=20?= =?UTF-8?q?=E2=80=94=20match=20kateri's=20text=20placement/metrics=20(font?= =?UTF-8?q?=20tail)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The native styled renderer's map *content* already matched the kateri golden, but the full image (title + legend + labels) diffed ~5% because the text landed on different pixels. Root cause was placement, not glyph rasterisation: on this toolchain cairo's toy "Helvetica" resolves (via fontconfig) to the same face poppler substitutes for the golden PDF's non-embedded base-14 Helvetica, so an embedded subset rasterises pixel-identically in isolation (0px diff). Confine the fix to cc/draw/cairo-surface.cc (no signature or call-site changes, so the parallel styled-draw.cc work is unaffected), aligning three primitives to kateri's PdfGraphics semantics (draw_on_pdf.dart): - text_font: anchor the glyph pen origin (baseline-left) at the supplied point; drop the first-glyph left-side-bearing subtraction that pushed every string ~1px left of the golden. - text_size (helvetica): return the pen *advance* width (kateri's AFM metrics.width, not cairo's ink extent) and the em size as height (kateri hard- codes textSize height to 1.0*fontSize), so callers size title box and legend rows/interline from the same metrics kateri used — fixing the compressed legend and the title box. - apply kateri's fontScaleToMatchCanvas (1.02) to both the rendered font size and the metrics, so text is rendered at the same scale the golden carries instead of ~2% short. Fidelity for the representative plain by-clade map, rasterised identically via pdftoppm and compared with magick -metric AE: full image at the threshold-30 metric 3.0% -> 0.32%; strict (fuzz 0) 5.4% -> 2.1%. Title/legend text now position- and size-identical to the golden; the strict residual is sub-pixel glyph AA plus one bounded artifact: pdftoppm substitutes the golden's declared base-14 bold face with a regular-weight face on this machine, so the golden's *rasterised* bold title is thinner than the native's correct (AD-faithful, PDF-declared) bold — a verification-rasteriser quirk, not a renderer defect, left as-is. Co-Authored-By: Claude Opus 4.8 (1M context) --- cc/draw/cairo-surface.cc | 41 ++++++++++++++++++++++++++++++++-------- 1 file changed, 33 insertions(+), 8 deletions(-) diff --git a/cc/draw/cairo-surface.cc b/cc/draw/cairo-surface.cc index 261316f..542d0d0 100644 --- a/cc/draw/cairo-surface.cc +++ b/cc/draw/cairo-surface.cc @@ -10,6 +10,13 @@ namespace ae::draw { + // kateri renders map text 2% larger than the nominal size (PdfGraphics.fontScaleToMatchCanvas, + // draw_on_pdf.dart) so its PDF text matches its on-screen canvas. The report's golden PDFs carry + // that factor, so the semantic-style text primitives (text_font / helvetica text_size) apply it + // too — both the drawn glyphs and the metrics the callers size their boxes from — to land on the + // golden's glyph pixels rather than ~2% short. + static constexpr double kFontScaleToMatchCanvas = 1.02; + static inline void set_source(_cairo* cr, Color color) { cairo_set_source_rgba(cr, color.red(), color.green(), color.blue(), color.alpha()); @@ -266,16 +273,21 @@ namespace ae::draw void CairoPdf::text_font(double x, double y, std::string_view utf8, double font_size, Color color, bool bold, bool italic) { - // Baseline-left anchor (matches kateri drawString at (origin.dx, origin.dy)): (x, y) is - // the left edge / baseline of the first glyph. + // Baseline-origin anchor (matches kateri drawString at (origin.dx, origin.dy)): (x, y) is + // the pen origin — baseline at y, first glyph's pen position at x (ink starts at + // x + left-side-bearing, exactly as the golden's drawString does). const std::string str{utf8}; // kateri renders Latin1 text in Helvetica (Type1); use the same face so the native - // render's title/legend/labels match the golden's glyph shapes/metrics. + // render's title/legend/labels match the golden's glyph shapes/metrics. On this + // toolchain cairo's toy "Helvetica" resolves (via fontconfig) to the same face poppler + // substitutes for the golden PDF's non-embedded base-14 Helvetica, so an embedded + // subset rasterises pixel-identically — the remaining tail is glyph *placement*. cairo_select_font_face(context_, "Helvetica", italic ? CAIRO_FONT_SLANT_ITALIC : CAIRO_FONT_SLANT_NORMAL, bold ? CAIRO_FONT_WEIGHT_BOLD : CAIRO_FONT_WEIGHT_NORMAL); - cairo_set_font_size(context_, font_size); - cairo_text_extents_t ext; - cairo_text_extents(context_, str.c_str(), &ext); - cairo_move_to(context_, x - ext.x_bearing, y); // left edge at x, baseline at y + cairo_set_font_size(context_, font_size * kFontScaleToMatchCanvas); + // kateri's PdfGraphics.drawString anchors the glyph *pen origin* (baseline-left) at the + // supplied point — it does NOT shift by the first glyph's left side-bearing. Match that + // (a prior x_bearing subtraction pushed every string ~1px left of the golden). + cairo_move_to(context_, x, y); set_source(context_, color); cairo_show_text(context_, str.c_str()); } @@ -283,7 +295,20 @@ namespace ae::draw std::pair CairoPdf::text_size(std::string_view utf8, double font_size, bool helvetica) { const std::string str{utf8}; - cairo_select_font_face(context_, helvetica ? "Helvetica" : "sans-serif", CAIRO_FONT_SLANT_NORMAL, CAIRO_FONT_WEIGHT_NORMAL); + if (helvetica) { + // Match kateri's PdfGraphics.textSize so callers (title box, legend rows) lay text out + // exactly where the golden has it: width = the font's *advance* width (kateri uses the + // base-14 AFM metrics.width, i.e. the pen advance, not the ink extent), height = the + // em size itself (kateri hard-codes textSize height to 1.0*fontSize, not the 1.156 line + // height nor the ink height). Both scaled by fontScaleToMatchCanvas, as kateri does. + cairo_select_font_face(context_, "Helvetica", CAIRO_FONT_SLANT_NORMAL, CAIRO_FONT_WEIGHT_NORMAL); + const double scaled = font_size * kFontScaleToMatchCanvas; + cairo_set_font_size(context_, scaled); + cairo_text_extents_t ext; + cairo_text_extents(context_, str.c_str(), &ext); + return {ext.x_advance, scaled}; + } + cairo_select_font_face(context_, "sans-serif", CAIRO_FONT_SLANT_NORMAL, CAIRO_FONT_WEIGHT_NORMAL); cairo_set_font_size(context_, font_size); cairo_text_extents_t ext; cairo_text_extents(context_, str.c_str(), &ext); From fd28a1a36a2d6e57f1464d0843c1e4a4771a4a15 Mon Sep 17 00:00:00 2001 From: Sarah James Date: Wed, 8 Jul 2026 17:21:22 +0100 Subject: [PATCH 06/13] =?UTF-8?q?tools:=20P2=20milestone=20K=20=E2=80=94?= =?UTF-8?q?=20reusable=20renderer=20fidelity=20harness?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add tools/p2-fidelity/ — the pixel-diff scoreboard every P2 map-renderer milestone uses to prove itself. Given a manifest of (chart, style, [golden], [ad-ref]) entries it: - renders each map natively (ae_backend.map_draw.export_styled_map -> PDF), - rasterises native + reference PDFs to equal-size PNGs (pdftoppm -scale-to), - pixel-compares native vs kateri golden (and AD single-map ref where an isolable one exists) with `magick compare -metric AE` at strict AND -fuzz 30%, - emits a per-map table to stdout + scoreboard.{csv,md} + optional native|kateri[|AD] montages. Every external command is wrapped in `timeout` so a hung tool can't stall a run. Charts + reference PDFs are referenced by PATH only (real report data lives outside the repo, never committed); rendered output goes to an out-dir outside the repo. Only the synthetic manifest.example.json is shipped. Files: fidelity.py (driver), manifest.example.json (annotated synthetic schema), README.md (interface + WHO-data policy + how to read the scoreboard). Co-Authored-By: Claude Opus 4.8 (1M context) --- tools/p2-fidelity/README.md | 73 +++++ tools/p2-fidelity/fidelity.py | 376 ++++++++++++++++++++++++ tools/p2-fidelity/manifest.example.json | 50 ++++ 3 files changed, 499 insertions(+) create mode 100644 tools/p2-fidelity/README.md create mode 100644 tools/p2-fidelity/fidelity.py create mode 100644 tools/p2-fidelity/manifest.example.json diff --git a/tools/p2-fidelity/README.md b/tools/p2-fidelity/README.md new file mode 100644 index 0000000..cf76af5 --- /dev/null +++ b/tools/p2-fidelity/README.md @@ -0,0 +1,73 @@ +# P2 fidelity harness — the renderer scoreboard + +`fidelity.py` is the reusable pixel-diff harness that every P2 map-renderer milestone uses to +prove itself. Given a manifest of `(chart, style, [golden], [ad-ref])` entries it renders each +map with the **native** renderer (`ae_backend.map_draw.export_styled_map`) and pixel-compares +it against (a) the **kateri** golden PDF and (b) an **AD** single-map reference where one is +isolable — emitting a per-map AE (differing-pixel) table at both `-fuzz 30%` and strict. + +## What it does per map + +1. **render** native → PDF: `export_styled_map(chart, out.pdf, style, width, projection)` +2. **rasterise** native + each reference PDF to equal-size PNGs: `pdftoppm -png -scale-to SIZE` +3. **compare** native vs each reference: `magick compare -metric AE [-fuzz 30%]` → AE pixel count +4. **report**: per-map table to stdout + `scoreboard.csv` + `scoreboard.md` in the out-dir, + plus optional `native|kateri[|AD]` montages (`--montage`). + +`AE` is the count of differing pixels; `%` is `AE / (SIZE·SIZE)`. `-fuzz 30%` tolerates small +colour/AA deltas so it isolates *structural/content* differences from anti-aliasing noise; +strict is the raw count. + +## Invocation + +```bash +# ae_backend lives in the frozen render-spike build; point --build-dir at it (no rebuild). +python3.14 tools/p2-fidelity/fidelity.py MANIFEST.json \ + --build-dir /path/to/ae-p2-render-spike/build-py314 \ + --out-dir /tmp/p2-fidelity-run \ + --size 800 --montage +``` + +Flags: `--build-dir DIR` (dir with `ae_backend*.so`; else `$AE_BUILD`, else an already-importable +`ae_backend` on `PYTHONPATH`, else `/build-py314`), `--out-dir DIR` (default a scratch dir), +`--size PX` (raster scale-to, default 800), `--width` / `--projection` (native render overrides), +`--montage` (write side-by-side PNGs), `--only SUBSTR` (filter by label), `--timeout SEC` +(per-command bound, default 120). + +Requires: `python3.14` with `ae_backend`, `pdftoppm` (poppler), ImageMagick 7 (`magick`), +and `timeout` — every external command is wrapped in `timeout` so a hung tool cannot stall a run. + +## Manifest format + +JSON. Either a top-level object with config + a `maps` list, or a bare list of entries. See +[`manifest.example.json`](manifest.example.json) for the full annotated schema. Essentials: + +| key | scope | meaning | +|-----|-------|---------| +| `chart` | entry (req) | path to a styled `.ace` (outside the repo) | +| `style` | entry (req) | a `c["R"]` front-style name: `clades-v1`, `serology`, `ts-2025-08`, `info-clades-v1`, … | +| `golden` | entry | kateri reference PDF path; `"auto"` derives `/out.1.