From 4f115fc15c9c5bedf89ecf6bc7358b213319ce20 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Kasper=20H=C3=A4gele?= Date: Mon, 7 Sep 2026 20:10:31 +0200 Subject: [PATCH] docs: bring README, AGENTS and the tools README up to date README: one component here, the pipeline is private; the basemap and relief tiles named as loaded directly; a tests-and-accuracy section pointing at the harness; the deep-link example without a stale cluster=6. AGENTS: MapLibre instead of Leaflet in the layout and the stack; changelog.json and tools/ in the layout; a "Measuring an estimator change" section with the paired baseline/compare commands in both modes and --prefix 2, and the note that --hop1-only is the common flow; the verify rule extended to harness numbers and a phone-sized viewport; mc-spamdetector.nl and the 429 budget in the upstream rule; two new rules, hide with the hidden attribute and every user-visible change gets a changelog entry. tools/README: flatness numbers as they are after #42 (4.1 nats, 28% tied; hard stratum 0.5 nats, 100% tied), surfaceStats and TIED_NATS as extracted from the page, current prefix-mode counts and tables, the paired line in --compare, and the sweeps of today listed by issue. Co-Authored-By: Claude Fable 5.1 --- AGENTS.md | 47 ++++++++++++++++++++++---- README.md | 16 +++++++-- web-standalone/tools/README.md | 62 +++++++++++++++++++++------------- 3 files changed, 92 insertions(+), 33 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index c082583..62c311b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -26,10 +26,12 @@ README.md Start here — which component to host, quick start for b .github/workflows/ CI: docker-build.yml builds/publishes web-standalone image to GHCR web-standalone/ Self-contained map tool (no DB) - index.html Map UI + triangulation logic (Leaflet, vanilla JS, :root CSS tokens) + index.html Map UI + triangulation logic (MapLibre GL, vanilla JS, :root CSS tokens) + changelog.json "What's new" entries, newest first; every user-visible change adds one server.py stdlib-only HTTP server + proxy for upstream feeds (mc-radar, meshcore.io, PDOK, mc-spamdetector.nl) spamdetector.py parses a spam-detector attack page into Step 1 clues (pure logic, tested) tests/ pytest for the pure logic; fixtures are trimmed copies of real upstream pages + tools/ accuracy.mjs (estimator error against known targets), its fixture, build-fixture.py Dockerfile Alpine, non-root, stdlib-only docker-compose.yml App + cloudflared sidecar, hardened (read-only rootfs, cap_drop ALL) .env.example TUNNEL_TOKEN, optional image override @@ -41,7 +43,7 @@ web-standalone/ Self-contained map tool (no DB) | Layer | Technology | |---|---| -| Web UI | Vanilla JS, Leaflet 1.9.4, plain CSS with `:root` custom-property tokens — no build step | +| Web UI | Vanilla JS, MapLibre GL JS 4.7.1 (OpenFreeMap Liberty basemap, AWS Terrain Tiles for relief), plain CSS with `:root` custom-property tokens — no build step | | Web server / proxy | Python 3.7+ stdlib only (`http.server`) — nothing to install | | Container | Docker (Alpine, non-root), multi-arch GHA build → GHCR | | Deploy | Cloudflare Tunnel | @@ -62,9 +64,28 @@ Or via Docker — see the root README's Docker quick-start section. ### Tests ```bash -python3 -m pytest web-standalone/tests # pure-logic tests (spamdetector.py) +python3 -m pytest web-standalone/tests # pure-logic tests (spamdetector.py); CI runs this before the image build ``` +### Measuring an estimator change + +Anything that touches scoring, ranges, clustering, ranking or the reported uncertainty is +judged with the accuracy harness, paired per case against main, before it is proposed: + +```bash +git show main:web-standalone/index.html > /tmp/main.html +node web-standalone/tools/accuracy.mjs --source /tmp/main.html --baseline /tmp/main.json +node web-standalone/tools/accuracy.mjs --compare /tmp/main.json # with 2nd-hop observers +node web-standalone/tools/accuracy.mjs --hop1-only --compare /tmp/main.json # without: the common flow +node web-standalone/tools/accuracy.mjs --prefix 2 --compare /tmp/main-prefix.json # region choice under prefix ambiguity +``` + +Read the paired line (better / worse / unchanged, paired mean) and the named worst regressions, +not only the medians. Operators rarely enter 2nd-hop prefixes, so `--hop1-only` is the flow +that matters most when the two modes disagree. Put the before/after numbers in the PR. The +harness extracts the shipped functions from `index.html`, so a variant is a copy of the file +with one line changed and `--source` pointing at it; see `web-standalone/tools/README.md`. + Going forward: - **Do** add unit tests (pytest) for new *pure logic* — clustering, weighted-estimate math, @@ -99,7 +120,9 @@ sketch the approach before writing the real implementation. ### 5.2 Verify before claiming done Run the affected script/page and observe the actual behaviour before saying a task is complete. -Intent is not verification. +Intent is not verification. For an estimator change that means the harness numbers above; for a +UI change it means the page in a browser, on a phone-sized viewport as well when layout is +involved. --- @@ -156,8 +179,20 @@ new hardcoded hex values. ### Don't hammer upstream feeds per-node -The proxied upstream APIs (mc-radar, map.meshcore.io, PDOK) are third-party and rate-limitable. -Batch/cache lookups (as the existing code does) rather than firing one request per node/prefix. +The proxied upstream APIs (mc-radar, map.meshcore.io, PDOK, mc-spamdetector.nl) are third-party +and rate-limitable; mc-radar answers 429 after a few hundred proven-link fetches in a day. +Batch/cache lookups (as the existing code does, including the in-memory cache in `server.py` +for the spam detector) rather than firing one request per node/prefix. + +### Hide with the `hidden` attribute + +A global `[hidden] { display: none !important }` rule exists. Toggle visibility with +`element.hidden`, not with a display style, and do not add a display rule that has to beat it. + +### Every user-visible change gets a changelog entry + +Prepend an entry to `web-standalone/changelog.json` in the words of the person using the tool, +not the words of the code. It feeds the "What's new" panel. --- diff --git a/README.md b/README.md index 3a54778..13bc38e 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ Localises repeaters/nodes/traffic sources on MeshCore from the mesh topology **Live demo:** [triangulator.dutchmeshcore.nl](https://triangulator.dutchmeshcore.nl/) -Two components, pick based on what you're hosting: +This repo holds the standalone map tool. The data-collection pipeline it grew out of is a separate private repo (see the end of this page). ## `web-standalone/` — start here @@ -58,7 +58,17 @@ control in front if you do; the proxy has no auth of its own. | map.meshcore.io | Global node feed (fallback). | | PDOK AHN | Terrain/surface elevation (NL). | | mc-spamdetector.nl | Incident list, and an incident's entry hops as Step 1 clues. | -| OpenFreeMap | 3D building tiles (no key). | + +Loaded by the page directly, not through the proxy: OpenFreeMap vector tiles (the Liberty basemap, no key) and AWS Terrain Tiles (relief in the 3D view, for looking at; line of sight is measured on AHN). + +### Tests and accuracy + +```bash +python3 -m pytest web-standalone/tests # pure-logic tests (spam-detector parser) +node web-standalone/tools/accuracy.mjs # estimator error against 92 known targets +``` + +The accuracy harness runs offline against a committed fixture and is how every estimator change in this repo is judged; see [web-standalone/tools/README.md](web-standalone/tools/README.md). ### Deep-linking (for other sites/tools) @@ -89,7 +99,7 @@ automatically clicks "Find Best Matching Region" on load — no further interaction needed to land on candidate clusters. ``` -https://triangulator.dutchmeshcore.nl/?prefixes=db:30,23,db11:12&prefixes2=a0:8,fc&cluster=6 +https://triangulator.dutchmeshcore.nl/?prefixes=db:30,23,db11:12&prefixes2=a0:8,fc ``` There's no `hop1` param — the "1st-hop km" input was removed; 1st-hop diff --git a/web-standalone/tools/README.md b/web-standalone/tools/README.md index 03168a9..0b99f04 100644 --- a/web-standalone/tools/README.md +++ b/web-standalone/tools/README.md @@ -35,10 +35,14 @@ indistinguishable, and a zero delta read as a green light. It also reports how FLAT the likelihood surface is: the log-likelihood range across the whole searched grid, the share of that grid tied with the winning cell, and how far the tied region reaches. Error alone cannot separate a model -that is right from one that had nothing to say and guessed the middle. On the -committed fixture the median case spans 28 nats with 6% of the grid tied, so -the argmax means something; the hard stratum (nearest observer >= 5 km) spans -3.8 nats with 42% tied and err med 18.8 km, so there it does not. +that is right from one that had nothing to say and guessed the middle. These +are the page's own `surfaceStats()` and `TIED_NATS`, extracted like the +scoring functions, so what the status line says ("likely within X km", #47) +and what this reports are one number. On the committed fixture the median +case spans 4.1 nats with 28% of the grid tied (it was 28 nats and 6% before +#42 made the ranges honest); the hard stratum (nearest observer >= 5 km, n=13) +spans 0.5 nats with 100% tied and err med 17.6 km, so there the argmax means +nothing and the page says so. Before the 2nd-hop half of the fixture existed the whole grid spanned 0.34 nats and was 100% tied, and #69's first kernel sweep read that as "kernel shape does @@ -58,29 +62,30 @@ node web-standalone/tools/accuracy.mjs --prefix 2 node web-standalone/tools/accuracy.mjs --prefix 2 --pick-cluster oracle ``` -At 2 hex that is 4407 decoys pulled in and 241 nodes dropped by dedupe across -50 of 92 cases, 179 of them decided by the circular nearest-the-centroid step, -93 of them nodes that really did hear the target. +At 2 hex that is 4407 decoys pulled in and 493 nodes dropped by dedupe across +56 of 92 cases, 343 of them decided by the circular nearest-the-centroid step, +166 of them nodes that really did hear the target (main after #106). `--pick-cluster oracle` locks the component nearest the known target instead of rank 1, standing in for the operator choosing correctly in step 2. Keep the two apart: they measure different failures, and the gap between them is the finding. -| n=82, oracle cluster | med | p75 | p90 | max | +| oracle cluster | med | p75 | p90 | max | |---|---|---|---|---| -| full id | 2.0 | 3.9 | 6.2 | 19.5 | -| 2-hex prefix | 2.1 | 4.8 | 6.9 | 19.5 | +| full id, n=91 | 1.9 | 4.7 | 10.2 | 96.8 | +| 2-hex prefix, n=86 | 1.9 | 4.6 | 9.6 | 45.4 | -| n=92, rank-1 cluster | med | p75 | p90 | max | +| rank-1 cluster | med | p75 | p90 | max | |---|---|---|---|---| -| full id | 2.4 | 5.8 | 12.2 | 93.8 | -| 2-hex prefix | 2.8 | 6.4 | 18.2 | 191.9 | +| full id, n=91 | 1.9 | 4.7 | 10.2 | 96.8 | +| 2-hex prefix, n=92 | 2.3 | 6.1 | 13.3 | 194.4 | -So the dedupe pick costs ~0.1 km on the median once the right region is locked, -while picking the region under prefix ambiguity costs over 100 km on single -cases. Dedupe runs identically in both oracle columns, and the worst rank-1 -blow-ups collapse when only the cluster choice changes: 8938F8CF 155.9 -> 2.5 -km, 29334479 191.9 -> 1.1 km. Cluster choice, not the tie-break, which is #85. +So the dedupe pick costs nothing measurable on the median once the right region +is locked, while picking the region under prefix ambiguity costs over 100 km on +single cases. Dedupe runs identically in both oracle columns, so the gap between +the two tables is cluster choice. That is where #110 (a chain wider than 30 km +ranks lower) and #33a (rival regions on the map, operator picks) act; #78 and +#85 are closed on these numbers. ### What "rank 1" means here @@ -98,8 +103,15 @@ node web-standalone/tools/accuracy.mjs --baseline /tmp/before.json node web-standalone/tools/accuracy.mjs --compare /tmp/before.json ``` -The comparison names the worst regressions individually, because a summary -statistic hides exactly the cases worth looking at. +The comparison prints the paired change first (better / worse / unchanged over +the cases both runs scored, paired median and mean), then the set-median line +with both case counts, then the worst regressions by name, because a summary +statistic hides exactly the cases worth looking at. Read the paired line: a +change that alters which clusters come out with a single node changes the case +set, and the set medians then compare different cases. + +Run it in both modes. Operators rarely enter 2nd-hop prefixes, so `--hop1-only` +is the common flow; several changes measured differently in the two (#103). To measure a variant without touching the working tree, point the extraction at another copy: @@ -110,10 +122,12 @@ sed 's/COVERAGE_EDGE_SHARPNESS = 6/COVERAGE_EDGE_SHARPNESS = 3/' \ node web-standalone/tools/accuracy.mjs --source /tmp/variant.html ``` -That is how the parameter sweeps in #69 were run. Note what those found: kernel -shape, range scale and grid extent all leave the error where it is, because the -surface is flat before any of them apply. Re-run the flatness numbers before -spending time on a change that only reshapes the kernel. +That is how the parameter sweeps in #69, #42, #103 and #47 were run. What they +found, in the issues: kernel sharpness mattered once 2nd-hop observers were in +the fixture (#81); the range is the longest proven link, capped at 30 km (#42); +Cluster km 12 (#106); a flat position sigma, a silent-observer term and a +wider search box all measure as no gain (#55, #59, #44). Check the flatness +numbers before spending time on a change that only reshapes the kernel. The scoring functions are extracted from `index.html` at runtime rather than reimplemented, so the harness cannot drift from what ships. If a rename breaks