Extend decoded paths to valid (junction) nodes per OpenLR Rule 4

.gitignore

@@ -24,3 +24,5 @@ site/
 # OS
 .DS_Store
 tags
+qa/screenshots/
+qa/results/

CLAUDE.md

@@ -136,6 +136,24 @@ The app shows:
 - Background network edges (hover to see edge IDs)
 - Success/failure status with error messages
 
+### Screenshots of the real UI
+
+The web app exposes `GET|POST /api/screenshot.png`: it opens its own page in headless
+Chromium (Playwright) with `?code=…&config=<json>`, waits for the decode to render, and
+returns a PNG — exactly what a user sees. `focus_lrp` + `span_m` zoom on an endpoint,
+`map_only=true` crops out the sidebar, `scale=2` gives retina output. The same URL
+parameters work in a normal browser for shareable links. `qa/screenshot.py` wraps it
+and stitches before/after configs side by side:
+
+```bash
+# server running with the network loaded (see above), then:
+~/openlr-web/.venv/bin/python qa/screenshot.py CODE --out qa/screenshots/<topic> \
+  --before '{"snap_to_valid_nodes": false}' --after '{"snap_to_valid_nodes": true}' \
+  --zoom-lrps 0 1 --span-m 50
+```
+
+Use this to attach visual evidence to PRs and to inspect benchmark regressions.
+
 ### Common Failure Modes
 
 1. **"No candidates found for LRP N"**
@@ -192,9 +210,29 @@ config = DecoderConfig(
     max_candidates=10,        # More candidates = more path options
     length_tolerance=0.35,    # 35% relative tolerance
     frc_tolerance=2,          # Allow ±2 FRC class difference
+    snap_to_valid_nodes=True, # Extend path ends to junctions (OpenLR Rule 4)
+    max_snap_extension_m=25.0,
 )
 ```
 
+### Terminal-node snap (OpenLR Rule 4)
+
+Encoders place LRPs on *valid* nodes (junctions); a node with one way in and one
+way out is invalid because a route search can step over it. After path selection
+the decoder checks whether the path starts/ends on an invalid node and, if so,
+follows the forced continuation to the first junction (`RoadNetwork::is_valid_node`,
+`forced_continuation`, `forced_predecessor` in `graph.rs`; `finalize_path` in
+`decoder.rs`). The added length is folded into `positive_offset`/`negative_offset`,
+so the offset-trimmed geometry is unchanged — only the edge set grows. Extension is
+capped at `max_snap_extension_m` (25 m; deliberately not scaled by DNP — HERE links are
+often only 20–40 m long and still stop a ~7–10 m crossing stub short of the junction),
+must keep the segment within the length tolerance, and is only applied when the LRP coordinate is
+closer to the new terminal node than to the current one — this guards against HERE
+having a junction our OSM car graph lacks (measured on the KC corpus: the guard removed
+all ~100 such overshoots while keeping every extension that landed on HERE's endpoint). Because the offset then spans more
+than one edge, `*_offset_fraction` (relative to the first/last edge) can exceed 1.0;
+the meter offsets are authoritative.
+
 ## Building & Testing
 
 Use `uv` for Python environment management.
@@ -223,11 +261,14 @@ The `qa/` directory contains tooling for measuring decode quality across a 48K-c
 
 ```bash
 # 1. Build the version you want to test and install into the openlr-web venv
-maturin build --release -i python3.12
-cd ~/openlr-web && uv pip install --reinstall ~/openlr-decoder/target/wheels/openlr_decoder-*-cp312-*.whl
+# (match the venv's interpreter: `~/openlr-web/.venv/bin/python --version`)
+uvx maturin build --release -i python3.11
+cd ~/openlr-web && uv pip install --reinstall ~/openlr-decoder/target/wheels/openlr_decoder-*-cp311-*.whl
 
 # 2. Run benchmark (saves results keyed by commit hash)
+#    --config accepts DecoderConfig overrides (ints, floats, true/false)
 ~/openlr-web/.venv/bin/python qa/benchmark.py \
+  --network ~/openlr-web/openlr_test_edges.parquet \
   --save qa/results/$(git rev-parse --short HEAD).parquet
 
 # 3. Compare two runs
@@ -244,7 +285,13 @@ cd ~/openlr-web && uv pip install --reinstall ~/openlr-decoder/target/wheels/ope
 
 ### Test corpus
 
-`qa/test_codes.csv` contains ~48K OpenLR codes with HERE reference geometries (from `bqutils.geo.openlr_to_geography`), randomly sampled from `model-159019.michelin.arity_link_matches_usa_v2` within the KC bounding box (38.53–39.11°N, 95.15–94.47°W). Network: `openlr_test_edges.parquet` (458K edges).
+`qa/test_codes.csv` contains ~48K OpenLR codes with HERE reference geometries (from `bqutils.geo.openlr_to_geography`), randomly sampled from `model-159019.michelin.arity_link_matches_usa_v2` within the KC bounding box (38.53–39.11°N, 95.15–94.47°W). Network: `~/openlr-web/openlr_test_edges.parquet` (487K edges, `startOsmNode`/`endOsmNode`
+ids). Regenerate it from BigQuery with `qa/regenerate_network.py` (run with the
+openlr-web venv python); it pulls the KC bbox from
+`model-159019.tomtom.tomtom_segments_updated_street_export_2025_Q2_1_5_0`.
+
+The benchmark reconstructs geometry from the meter offsets (`positive_offset`,
+`negative_offset`) expressed as fractions of the whole edge path.
 
 ## Parquet Schema (Network Input)
 

qa/benchmark.py

@@ -161,17 +161,14 @@ def run_benchmark(
     df = batch.to_pandas()
     cols = set(df.columns)
 
-    # Handle schema differences across versions
-    pos_off_col = (
-        "positive_offset_fraction"
-        if "positive_offset_fraction" in cols
-        else "positive_offset"
-    )
-    neg_off_col = (
-        "negative_offset_fraction"
-        if "negative_offset_fraction" in cols
-        else "negative_offset"
-    )
+    # Offsets: prefer meter values and express them as fractions of the *whole*
+    # edge path (length + both offsets), which is what reconstruct_path_geometry
+    # expects. The per-edge `*_offset_fraction` columns are relative to the first/
+    # last edge only and can exceed 1.0 when a terminal-node snap spans edges,
+    # so they cannot be applied to the full path directly.
+    has_meter_offsets = {"positive_offset", "negative_offset"} <= cols
+    pos_off_col = "positive_offset" if has_meter_offsets else "positive_offset_fraction"
+    neg_off_col = "negative_offset" if has_meter_offsets else "negative_offset_fraction"
     has_primary = "primary_edge_id" in cols
 
     def safe_float(val):
@@ -190,6 +187,13 @@ def safe_float(val):
 
         pos_frac = safe_float(row[pos_off_col]) or 0.0
         neg_frac = safe_float(row[neg_off_col]) or 0.0
+        if has_meter_offsets:
+            lrp_len = safe_float(row["length"]) or 0.0
+            total_m = lrp_len + pos_frac + neg_frac
+            if total_m > 0:
+                pos_frac, neg_frac = pos_frac / total_m, neg_frac / total_m
+            else:
+                pos_frac, neg_frac = 0.0, 0.0
 
         result = {
             "openlr_code": tc["openlr_code"],
@@ -426,7 +430,11 @@ def main():
         config_kwargs = {}
         for kv in args.config:
             k, v = kv.split("=", 1)
-            # Auto-cast numeric values
+            # Auto-cast boolean / numeric values
+            if v.lower() in ("true", "false"):
+                v = v.lower() == "true"
+                config_kwargs[k] = v
+                continue
             try:
                 v = int(v)
             except ValueError:

qa/regenerate_network.py

@@ -0,0 +1,38 @@
+"""Regenerate ~/openlr-web/openlr_test_edges.parquet (KC metro test network).
+
+Pulls ALLOWS_CAR edges in the KC bounding box from the Replica street export in
+BigQuery via the Storage Read API, with startOsmNode/endOsmNode as the node ids
+(required since #25). Reuses the web app's loader with the node columns swapped.
+
+Usage:
+    cd ~/openlr-web && .venv/bin/python ~/openlr-decoder/qa/regenerate_network.py [TABLE]
+"""
+import inspect
+import sys
+from pathlib import Path
+
+import pyarrow.parquet as pq
+
+WEB = Path.home() / "openlr-web"
+sys.path.insert(0, str(WEB))
+import app  # noqa: E402
+
+TABLE = sys.argv[1] if len(sys.argv) > 1 else app.DEFAULT_BQ_TABLE
+# KC bbox used for qa/test_codes.csv
+BBOX = dict(min_lat=38.53, max_lat=39.11, min_lon=-95.15, max_lon=-94.47)
+OUT = WEB / "openlr_test_edges.parquet"
+
+code = inspect.getsource(app.load_from_bigquery).replace(
+    '"startVertex",\n        "endVertex",', '"startOsmNode",\n        "endOsmNode",'
+)
+assert '"startOsmNode"' in code, "app.load_from_bigquery column list changed; update this script"
+ns = dict(vars(app))
+exec(code, ns)
+
+table = ns["load_from_bigquery"](TABLE, **BBOX)
+if OUT.exists():
+    backup = OUT.with_suffix(".bak.parquet")
+    OUT.rename(backup)
+    print(f"Previous network moved to {backup}")
+pq.write_table(table, OUT)
+print(f"Wrote {table.num_rows} edges to {OUT}")

qa/screenshot.py

@@ -0,0 +1,97 @@
+"""Capture before/after screenshots of the real openlr-web UI for OpenLR codes.
+
+Uses the app's /api/screenshot.png endpoint (headless Chromium via Playwright), so
+the images are exactly what the web app shows. Requires a running server with the
+network loaded, e.g.:
+
+    cd ~/openlr-web && .venv/bin/uvicorn app:app --port 8000
+    curl -X POST localhost:8000/api/network/load -H 'content-type: application/json' \
+         -d '{"path":"openlr_test_edges.parquet"}'
+
+Usage:
+    python qa/screenshot.py CODE [CODE ...] --out DIR \
+        --before '{"snap_to_valid_nodes": false}' --after '{"snap_to_valid_nodes": true}' \
+        [--zoom-lrps 0 1] [--span-m 60] [--map-only] [--server http://localhost:8000]
+
+For each code this writes `<slug>_overview.jpg` plus `<slug>_lrp<N>.jpg` for each
+requested zoom LRP (zoom shots are cropped to the map). With --after, each image is the before and after shots stitched
+side by side (before on the left). Pass only --before for single shots.
+"""
+import argparse
+import io
+import json
+import re
+import urllib.request
+from pathlib import Path
+
+
+def fetch(server: str, body: dict) -> bytes:
+    req = urllib.request.Request(
+        f"{server}/api/screenshot.png",
+        data=json.dumps(body).encode(),
+        headers={"content-type": "application/json"},
+    )
+    with urllib.request.urlopen(req, timeout=120) as r:
+        return r.read()
+
+
+def combine(pngs: list[bytes], jpeg_quality: int | None, gap: int = 16) -> bytes:
+    """Stitch shots side by side (before on the left) and encode as JPEG or PNG."""
+    from PIL import Image
+
+    imgs = [Image.open(io.BytesIO(p)).convert("RGB") for p in pngs]
+    h = max(i.height for i in imgs)
+    w = sum(i.width for i in imgs) + gap * (len(imgs) - 1)
+    out = Image.new("RGB", (w, h), "white")
+    x = 0
+    for im in imgs:
+        out.paste(im, (x, 0))
+        x += im.width + gap
+    buf = io.BytesIO()
+    if jpeg_quality:
+        out.save(buf, format="JPEG", quality=jpeg_quality, optimize=True)
+    else:
+        out.save(buf, format="PNG", optimize=True)
+    return buf.getvalue()
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("codes", nargs="+")
+    ap.add_argument("--out", type=Path, required=True)
+    ap.add_argument("--before", type=json.loads, default=None, help="DecoderConfig JSON for the left/only shot")
+    ap.add_argument("--after", type=json.loads, default=None, help="DecoderConfig JSON for the right shot")
+    ap.add_argument("--zoom-lrps", type=int, nargs="*", default=[], help="LRP indices to render zoomed views for")
+    ap.add_argument("--span-m", type=float, default=60.0, help="Width of zoomed views in meters")
+    ap.add_argument("--width", type=int, default=1100)
+    ap.add_argument("--height", type=int, default=750)
+    ap.add_argument("--map-only", action="store_true", help="Crop overview shots to the map too (zoom shots always are)")
+    ap.add_argument("--scale", type=int, default=1, help="Device pixel ratio (2 = retina)")
+    ap.add_argument("--server", default="http://localhost:8000")
+    ap.add_argument("--png", action="store_true", help="Write lossless PNG instead of JPEG (much larger with satellite imagery)")
+    ap.add_argument("--jpeg-quality", type=int, default=85)
+    args = ap.parse_args()
+
+    configs = [args.before] + ([args.after] if args.after is not None else [])
+    args.out.mkdir(parents=True, exist_ok=True)
+
+    for code in args.codes:
+        slug = re.sub(r"[^A-Za-z0-9]", "", code)[:16]
+        views = [("overview", {"map_only": args.map_only})] + [
+            (f"lrp{i}", {"focus_lrp": i, "span_m": args.span_m, "map_only": True}) for i in args.zoom_lrps
+        ]
+        for name, view in views:
+            shots = [
+                fetch(
+                    args.server,
+                    {"code": code, "config": cfg, "width": args.width, "height": args.height, "scale": args.scale, **view},
+                )
+                for cfg in configs
+            ]
+            out = args.out / f"{slug}_{name}.{'png' if args.png else 'jpg'}"
+            out.write_bytes(combine(shots, None if args.png else args.jpeg_quality))
+            print(out)
+
+
+if __name__ == "__main__":
+    main()