diff --git a/.agents/notes/2026-09-10-prototype-explorer.md b/.agents/notes/2026-09-10-prototype-explorer.md new file mode 100644 index 00000000..10977cf5 --- /dev/null +++ b/.agents/notes/2026-09-10-prototype-explorer.md @@ -0,0 +1,63 @@ +# Prototype explorer handoff + +Status: completed +Updated: 2026-09-10 +Scope: initial prototype-space exploration, log-domain diagnostics, GBIF image labels +Related: `42cef5c`, `d4663e7`; [reproduction guide](../../dev/prototype_space/README.md) + +## Context and decision + +The feature is isolated on `feature/prototype-space` in +`/home/asger/mini_trainer/.worktrees/prototype-space`. The main `quant` checkout +received only the worktree workflow/rules; its pre-existing roadmap edits were +verified unchanged. No subagents were started. Shared `.venv` was used read-only. + +The real epoch-4 checkpoint supplies the baseline geometry. Existing effective +weight handling, cosine-to-z transform and Ward distances are retained. Added +float32 log-tail computation and adjustable colour clipping distinguish numerical +saturation from actual prototype identity. Public GBIF examples visually label +classes at their learned target directions; measured image embedding proximity +would be a separate selection criterion. Taxon names never replace checkpoint IDs. + +## Evidence and limits + +- Final source revision: `d4663e7` (includes numerical commit `42cef5c`). +- Full `bash dev/check.sh all`: 655 passed, 162 skipped, 1 expected failure; + exit 0 in 598.56 seconds. Ruff, formatting and both import contracts passed. + Used absolute worktree `PYTHONPATH`, `MPLCONFIGDIR=/tmp/mini-trainer-prototype-mpl`, + `OMP_NUM_THREADS=4 MKL_NUM_THREADS=4 OPENBLAS_NUM_THREADS=4`, and + `PYTEST_ADDOPTS='-o faulthandler_timeout=120'`. CUDA was hidden. This does not + establish GPU execution or performance. No package installation was performed. +- A preceding sandboxed full run was interrupted after waiting on loader workers: + 340 passed, 13 skipped. The successful rerun allowed local multiprocessing + sockets. Do not confuse that interrupted result with full-suite success. +- Browser helper: 25 interaction checks, zero JavaScript errors, including + colour controls, baseline comparisons, tree/search/neighbour navigation, + synthetic cases, photo cycling and photo navigation. Photo controls used + deterministic fixtures; a separate live check rendered actual GBIF images for + class 1837646 and its neighbours with credits. +- Real payload: 12,632 x 1,280 prototypes; 249,324 unordered baseline zero-distance + pairs, all positive under the independent float64 log-CDF reference; all + 79,777,396 off-diagonal float32 log-tail pairs finite. Three labelled synthetic + probes accompany the checkpoint; their smaller class counts are not matched + packing controls. +- Checkpoint SHA256: + `42d50f56cb0c6e3ee17ae64fbbcc4eb2ae24335ffe7080197c479c130fff5a81`. +- Ignored, machine-local artifacts: `tmp/prototype-report/`, + `tmp/prototype-browser-check/`, `tmp/prototype-live-browser-check/`. + The generated HTML was checked against current templates and stored numerical + JSON. The JSON records the numerical-generation script hash; later UI-only + refreshes did not recompute or relabel numerical provenance. +- Preview was healthy at `http://localhost:8765/explorer.html` on completion, + served with `python -m dev.prototype_space.serve`; it is not a persistent service. + GBIF metadata/thumbnails are cached outside the served directory. Images require + optional network access; the numerical report is self-contained. +- Evaluation logging changes apply to this feature branch, not to an already + running training process. Existing Ward clustering remains the baseline; + alternate stable clustering and measured sample occupancy are future work. + +## Next actions + +None required for this initial exploration. Gather feedback on the runnable views +before choosing further directions listed in the reproduction guide. Keep source +integration into `quant` separate from this completed feature-branch handoff. diff --git a/AGENTS.md b/AGENTS.md index 40f50e18..c34819dc 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -52,6 +52,18 @@ Complete a bounded, validated increment before moving to the next priority. - Slow backbone tests require `RUN_SLOW_TESTS=1` and may need model downloads. The compatibility runner can update the backbone blacklist; use it only when that mutation is part of the task. +- Be economical with validation: choose the smallest set of checks that covers the + changed behavior and credible regressions, while satisfying the requirements above. + Before an expensive suite or benchmark, identify the unresolved question it answers. +- Reuse passing evidence for unchanged code and environments. Batch related edits + before expensive checks; do not rerun the same suite for documentation changes or + automatically repeat focused checks already covered by a passing broader run. + Repeat or broaden checks when new changes, failures, integration conflicts or a + concrete unresolved risk justify it, not merely for additional reassurance. +- Add tests for meaningful behavior and failure modes, not assertions that mirror + implementation details or low-impact presentation changes. Prefer existing + coverage and a focused browser/manual check where appropriate. Keep required CI + gates intact; report the limits of focused validation rather than implying full coverage. - Report checks run, failures, skips, and limits honestly. Do not weaken checks or alter expected results merely to make a refactor pass. diff --git a/README.md b/README.md index f9f1d8b8..a9ca7659 100644 --- a/README.md +++ b/README.md @@ -16,6 +16,13 @@ All code in `mini_trainer` should follow the following core principles: * All hyperparameters and system configuration should have smart defaults that are as general as possible * All functionality should be extendable to custom model architectures, loss functions, training regimes, data formats etc. +## Explore model prototypes + +With the optional `explorer` dependencies installed, run `mt_explore weights.pt` +to open the interactive prototype explorer, or `mt_explore` to choose weights in +the browser. See the [prototype explorer guide](docs/prototype-explorer.md) for +installation, portable exports, supported checkpoints, and interpretation. + # Installation We recommend using `uv` for package and environment management. @@ -86,6 +93,9 @@ preloading; use uncached loading when you need that explicit bound. `mini_trainer` supports logging your training runs, including metrics, confusion matrices, and the probabilistic dendrogram, directly to [Weights & Biases](https://wandb.ai). +Class-matrix diagnostics use [log-domain probabilities](docs/prototype-diagnostics.md) +to retain small tails during evaluation logging. + To use this feature you must install `mini_trainer` with the `recommended` extras. See [Installation](#installation) for more information. 1. **Login to your wandb account**: diff --git a/dev/prototype_space/README.md b/dev/prototype_space/README.md new file mode 100644 index 00000000..f1246806 --- /dev/null +++ b/dev/prototype_space/README.md @@ -0,0 +1,340 @@ +# Prototype-space exploration + +The reusable tool is installed as `mt_explore`; see the +[launcher and export guide](../../docs/prototype-explorer.md). Runtime code and +viewer assets live in `mini_trainer.visualization.prototype_space`. These developer +commands remain as compatibility entry points and regression tools. The accompanying +[log-domain diagnostic API](../../docs/prototype-diagnostics.md) also supports +evaluation logging during training. The existing weight parametrization, +similarity transform and class distance remain the empirical reference. +Alternative evaluations and displays are labelled explicitly. Begin with real +checkpoint evidence; synthetic cases probe calculations and rendering, not how +the learned space ought to organize. + +Run from the feature worktree with the existing environment: + +```bash +export PYTHONPATH="$PWD" +CUDA_VISIBLE_DEVICES='' .venv/bin/python -m dev.prototype_space.explore \ + --weights /absolute/path/to/best_global-lepi-production-w32-1_epoch4.pt \ + --output tmp/prototype-report --angular-tsne +``` + +Open `tmp/prototype-report/explorer.html` in a browser. The numerical views are self-contained. For optional GBIF image labels and a +clickable local HTTP URL: + +```bash +.venv/bin/python -m dev.prototype_space.serve \ + --directory tmp/prototype-report --port 8765 +``` + +Then open . This serves only the report +directory on loopback. Keep each concurrent task's output path and port distinct. + +Generation uses CPU float32 repository APIs for the full class-by-class matrices +and SciPy for the unchanged Ward linkage. Several GB of RAM are needed for the +12,632-class example; this is an exploratory offline tool, not a bounded-memory +replacement for training-time logging. No dependency installation is performed. +The environment needs the existing visualization dependencies (including SciPy). + +## Try the views + +1. Search a checkpoint class ID or choose the strongest/weakest nearest pair. + Follow neighbours directly from the table. The selected class updates the + profile, highlighted tree path and global matrix position. +2. Click a collapsed dendrogram tip to focus on that subtree. Use **Parent**, + **Whole tree** and **Locate selected class** to retain context. This is the + current Ward tree in a different, progressively expanded layout. +3. Compare matrix block maxima with block minima using the same leaf order and + shared colour scale. The latter retains isolated close pairs that maxima can + conceal. Both exclude self-pairs and padding. Choose float32 log-domain tails + or the legacy rounded-CDF view. Choose the familiar `1e-8` to `1` display + window, an observed 0.1–99.9 percentile window, full range or manual log10 + bounds. Clipping affects colours only; each panel reports clipped block counts + and hover retains raw values. These are not pixel-identical copies of the + original heatmap. +4. In the local matrix, switch between **Baseline class distance**, **Pre-CDF + z-score** and **−log₁₀ tail (float32)**. Hover a cell for its stored values. Local + colour ranges adapt and are disclosed; use the table for exact comparisons. +5. Switch cases to independent unit vectors, four planted groups or algebraic + edge cases. All use the real embedding width, but have much smaller class + counts. Their nearest-neighbour distributions are not matched packing controls. + +## Spatial projection + +See the [hyperspherical visualization review](../../docs/hyperspherical-visualization.md) +for the geometry, measured compression losses, projection alternatives, Python +and browser implementations, and proposed anchor/slice experiments. + +The map projects all prototype directions into a common 2D coordinate system. +Choose **Angular t-SNE**, **PCA 1–2** or **PCA 3–4**, scroll to zoom, drag to pan, and click a point +to update the existing inspector and image browser. **Fit selected neighbourhood** +zooms to the selected class and its original-space neighbours without refitting +the projection. Selection preserves the viewing transform; **Fit all** resets it. + +The PCA calculation normalizes effective rows using the same directional geometry as +the cosine diagnostic, centers them, and computes PCA through float32 covariance +eigendecomposition. No whitening or per-feature standardization is applied. +Axes use equal spatial scale. Eigenvector signs are fixed for reproducibility; +degenerate eigenspaces can still rotate between numerical implementations. See +the [PCA reference](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html). + +Each plane reports its retained variance and the overlap between its Euclidean +top-k neighbours and the original z-ranked top-k neighbours, averaged across all +classes and for the selected class. Projected ties use checkpoint row order. Teal +points and links identify original-space neighbours; orange rings identify extra +2D neighbours. A low overlap is evidence that this view loses local geometry. +Neither projected proximity nor a visible gap replaces the original scores. +Each synthetic case is projected independently; axes are not aligned across cases. + +For the epoch-4 checkpoint, PC 1–2 retains 0.6784% of directional variance and +0.6551% of original top-12 neighbours on average; PC 3–4 retains 0.6234% and +0.7026%, respectively. These measured values show that global two-axis PCA is +very lossy for these weights. It provides a linear reference for future +neighbourhood-preserving projections. + +### Angular map + +`--angular-tsne` also fits a nonlinear map using angular distances in radians, +recovered from the unchanged repository transform as `pi/2 - z/sqrt(D-2)`. +This retains its cosine clamping and avoids the saturated CDF. t-SNE uses those +precomputed angular distances, random initialization with seed 42, perplexity +`min(30, (classes-1)/3)`, automatic learning rate, 1,000 iterations and Barnes-Hut +angle 0.5. It does not use PCA preprocessing. The angular map is selected first +when present; omit the flag for the faster PCA-only report. + +On the real checkpoint it retains 48.62% of original top-12 neighbours, compared +with 0.66% for PC1–2. This is an observed result for these settings and weights, +not evidence that map areas, gaps between groups or relative group sizes measure +spherical geometry. Variance explained is not defined for this map. Its +parameters, KL divergence and neighbour overlap are stored with the coordinates. +See the [t-SNE reference](https://scikit-learn.org/stable/modules/generated/sklearn.manifold.TSNE.html). + +### Thumbnails on the map + +Enable **GBIF thumbnails on map** independently of the image-card panel. Configure +image size (32–160 CSS pixels), density (5–200 images per megapixel of map viewport) +and allowable pairwise rectangle overlap (0–75%). The density gives a maximum +budget, capped at 200 images; viewport and overlap culling can produce fewer. +The overlap fraction uses the full thumbnail rectangle including its class label +when labels are enabled, normalized by the smaller rectangle for unequal sizes. +Disable **Labels beneath images** for square image-only +thumbnails with no caption, border or padding; images fill the square by cropping +as needed. Enable **Original aspect ratio** to preserve image proportions without +cropping; the size slider then sets the longest image edge. This works with or +without labels. Collision forces use the measured rectangular footprint. Cached +image proportions also inform candidate culling; unknown images initially reserve +a conservative square until decoding supplies their dimensions. With pushing +disabled, unknown proportions reserve nonoverlapping squares so a later narrow +image cannot become fully covered under a misleading overlap fraction. Names, IDs and +credits remain available on hover or keyboard focus. + +Images initially stay centered on actual projected points. Enable **Push thumbnails +apart** to admit candidates using the overlap setting, then separate their +rectangles with animated contact repulsion, spring attraction toward their anchors, +and damped velocity. Contacts use the rendered CSS border boxes (including labels +when enabled), with a two-pixel clearance and softly blended corner normals. +Separated boxes outside that clearance do not repel. Sizes, clearance, offsets +and velocities are measured in screen pixels, independent of projection zoom. +Images may move up to twice their configured size in screen pixels. +**Show thumbnail anchors** toggles the anchor markers and leader lines without +reloading images or restarting the simulation. They are hidden by default. +Settling never removes or fades an admitted thumbnail. The solver minimizes a +fixed sum of anchor displacement energy and squared, smoothed rectangle-contact +penetration. Contact curvature scales each coordinate's step; damped momentum +and backtracking keep accepted steps from increasing that energy. Anchor +attraction does not decay on a timer. + +Rest requires eight consecutive steps with movement below 0.04 CSS pixels, +energy change below 0.002 per thumbnail, and maximum overlap-fraction change +below 0.0001. At most two solver steps run per animation frame. A 480-step work +limit bounds expensive cases; the status distinguishes **resting** from **work +limit** and reports overlapping pairs that remain visible. Neither resting nor +the work limit guarantees zero overlap or a global energy minimum. Viewport and +movement bounds can make overlap unavoidable. Reduced-motion preferences run +the same bounded solver without animation. + +Numerical targets are separate from displayed thumbnail centers. Two cascaded +filters ease visible motion in and out, using elapsed seconds and CSS pixels: +a guide approaches the target at at most 140 px/s, and the displayed image +follows continuously with an 8/s response rate. This also bounds acceleration +(to 2240 px/s², including reversals). Substeps keep motion consistent across +refresh rates; background pauses advance at most 50 ms instead of catching up. +The animation continues after numerical convergence until displayed speed is +below 0.02 px/s and target error below 0.005 px, with no final snap. Image arrivals +join the existing frame loop and completion promise, retaining its clock, solver +destinations, and filter state. Each image waits independently for its appearance +delay before joining rectangle contacts; existing contacts keep advancing. +Targets follow class IDs even when admission order changes; pan/zoom translates it with the anchor. Hit rectangles +and optional tethers follow displayed positions. Smoothing can leave temporary +contacts during motion; energy descent describes numerical targets, not every +displayed frame. `node dev/prototype_space/check_layout.mjs` checks full-motion +speed and acceleration, reversals, and 30/60/120 Hz agreement. + +The status reports the remaining visible slots. Increase allowable overlap to try +a denser candidate set; +this does not guarantee that every admitted image can be separated within the +movement bounds. Initial viewport/density/overlap admission remains separate +from settling; an admitted image is never removed merely because motion stops. Thumbnails culled +by viewport navigation fade out over 100 ms, with hit targets removed immediately. +Reduced-motion preferences remove them immediately. + +Culling prioritizes the selected +class and its original-space neighbours, then uses a stable mixed class order. +Only fully visible, admitted rectangles trigger image requests. Up to four images +load concurrently. Each decoded image fades in at its anchor before it joins the +force simulation; unloaded images neither appear as blank cards nor exert forces. +Pan/zoom cancels stale requests and pauses the simulation. Loaded thumbnails +follow their transformed anchors immediately, retaining their screen-pixel +offsets and DOM image nodes. After a short pause, the visible candidate set is +updated and collision adjustment resumes; surviving images are neither reloaded +nor faded in again. No new images are fetched while dragging continuously. +Changing image size, aspect ratio mode, label mode, push mode, case or projection plane rebuilds the +layout because its footprints or coordinate meaning have changed. Cached metadata is +shared with the class cards, and changing a class example also refreshes map +thumbnails. Decoded image references are bounded to 128 entries; server caching +remains shared. Missing photos leave the underlying point visible. Retry clears +failed lookups. Synthetic cases make no photo requests. + +Click a thumbnail to inspect the class. Hover or keyboard focus exposes the +scientific name, exact ID, creator, image license and source links below the map. +Culling and image repulsion never change scores, prototype coordinates or +neighbour ranks. The controls affect image labels and their displayed positions. + +## Class image labels + +Enable **Interpret numeric class IDs as GBIF taxa and load photos** to see the +selected class and its twelve nearest prototype directions as image cards. Click +an image to navigate to that class; **Another example** cycles available images. +The prototype is the learned target direction, and the photo provides a class +label. Selecting images by their measured embedding proximity is a separate +possible extension. Exact checkpoint IDs remain unchanged, including when GBIF +reports a different accepted taxon ID. + +Only the visible neighbourhood is requested. Metadata and thumbnails are cached +in a sibling `prototype-report-gbif-cache` directory; upstream requests are +serialized, while cached images and already-resolved class metadata bypass that +queue. The browser can therefore load cached images while an upstream request +is still pending. Synthetic cases never trigger lookups. Missing images or network +access leave the numerical views usable. Media credits and licenses are shown +when supplied; occurrence-data licenses are never substituted for image licenses. +The service uses the [GBIF image API](https://techdocs.gbif.org/en/openapi/images). + +## Numerical contract + +- Supported inputs are float32 `Classifier` and `HierarchicalClassifier` + checkpoints with a single linear prototype matrix. Other head families and + weight encodings are rejected. This does not replace the general model loader. +- Reconstruct the parametrized linear layer with `Classifier._normalize_layer` + and strict state loading. Keep effective magnitudes, biases and exact class + order. Do not treat weight-norm direction parameters as effective weights. +- The diagnostic adapter exposes those effective weights to the unchanged + `class_similarity(cdf=False)`, `class_similarity(cdf=True)` and `class_distance` + APIs. It does not execute the backbone, hidden layers or hierarchical inference. + Like the existing diagnostics, these describe directions, not sample occupancy. +- The global tree uses Ward linkage on the current distance. The global heatmap + quantity is the current `1 − CDF(z)`. No alternative metric replaces either. +- Neighbours are explicitly ranked by pre-CDF z, with stable class-index ordering + for exact z ties. This refines saturated distance ties rather than arbitrarily + treating zero-distance pairs as identical prototypes. +- The primary tail view uses `class_log_similarity(model, complement=True)` and + displays `−log_tail / log(10)` in float32. It neither materializes a near-one CDF + nor promotes the z-score or log-probability matrix to float64. +- The separate float64 reference computes `−torch.special.log_ndtr(z.double())` + for comparison (hover baseline distances in the neighbour table). It retains + the existing z and does not apply the baseline probability floor. Reference + neighbour distances are stored as float64; baseline and log-tail values remain + float32. Numerical values are never recovered from colours. +- Linear probabilities can still underflow at extreme z even in float64; log-tail + remains useful there. The algebraic fixture includes this case. A log-domain + view improves numerical range, not the statistical assumptions of the transform. + +The prototype checkpoint has 12,632 unit-length rows in 1,280 dimensions and zero +biases. The initial full baseline evaluation found 249,324 unordered off-diagonal +pairs with distance zero, involving every class. That is numerical evidence to +investigate, not evidence that the vectors are identical or that training failed. +All 249,324 become positive under direct float64 log-CDF evaluation; that reference +confirms the saturation independently of the primary float32 log-tail display. +Current run statistics and checkpoint SHA256 are in `summary.json`; the complete +view payload is in `report-data.json`. Generated data and weights stay out of Git. + +## Validation and next directions + +```bash +bash dev/check.sh static +node dev/prototype_space/check_layout.mjs +bash dev/check.sh test tests/utils/test_prototype_exploration.py \ + tests/utils/test_dendrogram.py tests/utils/test_plot.py +``` + +Tests cover effective weights with non-unit magnitudes, class ordering, unchanged +checkpoint bytes, unsupported heads, self-pair/padding exclusion, saturation ties, +exact local baseline submatrices and log-domain values against an independent +`math.erfc` reference. Browser interaction checks additionally exercise case +switching, class search, neighbour navigation, tree navigation and matrix modes. + +The [maintained roadmap](../../docs/prototype-explorer-roadmap.md) owns priority +and status. Supporting research questions include: + +- Evaluate log-domain clustering with explicit representation and compatibility + decisions. Compare any changed tree against this preserved baseline before + promoting it into training-time logging. +- Add mutual-neighbour graph exploration to expose links across tree branches. +- Add taxonomy composition at multiple neighbourhood sizes, retaining unknown + labels separately and avoiding any assumption of taxonomic agreement. +- Compare neighbour and crowding trajectories across checkpoints using class IDs; + do not interpret unconstrained embedding rotations as motion between runs. +- With held-out embeddings, add image inspection, sample alignment, margins and + observed confusion. With an explicitly chosen direction distribution, examine + decision-region occupancy. Neither is established by the prototype matrix alone. + +The code is on `feature/prototype-space`. See the [worktree guide](../worktrees.md) +and [agent coordination rules](../../.agents/rules/worktrees.md) for parallel work. + +Optional browser interaction verification with an existing Chromium installation: + +```bash +CHROMIUM_BIN=/absolute/path/to/chrome node dev/prototype_space/check_browser.mjs \ + http://localhost:8765/explorer.html tmp/prototype-browser-check +``` + +No browser dependency is installed by this helper. Screenshots and the interaction +check report stay in the ignored output directory. + +## Packaged launcher checks + +`tests/utils/test_prototype_launcher.py` checks portable generation, class order, +HTTP upload, invalid-file recovery, and separation of weights from served files. +`dev/check-wheel.sh` checks the installed entry point, bundled assets, and the +missing-extra message in a minimal environment. + +For the browser file-picker path, start `mt_explore --no-browser` (or its module +equivalent) and run: + +```bash +CHROMIUM_BIN=/path/to/chrome node dev/prototype_space/check_launcher.mjs \ + http://localhost:PORT/ tmp/launcher-check /absolute/path/to/small-weights.pt +``` + +This selects the file with Chromium's native input, generates the default angular +map, opens the report, and returns to the picker. Use a small supported checkpoint +for this bounded regression check. The original `check_browser.mjs` exercises the +full real-checkpoint report with its three synthetic comparison cases. + +The [portable viewer qualification](portable-qualification.md) records the shared +browser GBIF client, optional metadata snapshots, prediction previews and focused +checks. See the [browser guide](../../docs/prototype-browser.md) for snapshot +preparation and static distribution. + +## Mobile image and touch regression + +Against a plain static export, this bounded check exercises all eight EXIF +orientations, unchanged opaque-image sampling, PNG transparency, WebP, decoder +fallback, camera input wiring, large-image sampling and CDP touch pinch/pan/cancel. +Physical camera capture and native HEIC support still require device review. + +```bash +.venv/bin/python -c "from PIL import Image; Image.new('RGB', (7000, 6000), (55, 115, 165)).save('/tmp/large-camera.jpg')" +CHROME_PATH=/path/to/chrome node dev/prototype_space/check_mobile.mjs \ + http://127.0.0.1:8771/explorer.html /tmp/large-camera.jpg /tmp/mobile-check.json +``` diff --git a/dev/prototype_space/check_browser.mjs b/dev/prototype_space/check_browser.mjs new file mode 100644 index 00000000..887059b3 --- /dev/null +++ b/dev/prototype_space/check_browser.mjs @@ -0,0 +1,323 @@ +// Optional browser smoke check. Set CHROMIUM_BIN; pass a report URL/path and output directory. +import {spawn} from 'node:child_process'; +import {writeFileSync,mkdtempSync,rmSync,mkdirSync,existsSync} from 'node:fs'; +import {tmpdir} from 'node:os'; +import {pathToFileURL} from 'node:url'; +import {resolve,join} from 'node:path'; +const executable=process.env.CHROMIUM_BIN; +if(!executable||!existsSync(executable)||!process.argv[2])throw Error('Set CHROMIUM_BIN and pass report URL/path [output directory]'); +const output=resolve(process.argv[3]||'tmp/prototype-browser-check');mkdirSync(output,{recursive:true}); +const profile=mkdtempSync(tmpdir()+'/prototype-browser-'); +const chrome=spawn(executable,['--headless','--no-sandbox','--disable-gpu','--disable-dev-shm-usage','--no-first-run','--no-default-browser-check','--disable-background-networking','--disable-component-update','--disable-extensions','--remote-debugging-pipe','--user-data-dir='+profile],{stdio:['ignore','ignore','ignore','pipe','pipe']}); +let sequence=0,buffer=Buffer.alloc(0);const pending=new Map();const errors=[]; +chrome.stdio[4].on('data',chunk=>{buffer=Buffer.concat([buffer,chunk]);let end;while((end=buffer.indexOf(0))!==-1){const msg=JSON.parse(buffer.subarray(0,end));buffer=buffer.subarray(end+1);if(msg.id){const cb=pending.get(msg.id);if(cb){pending.delete(msg.id);msg.error?cb.reject(Error(JSON.stringify(msg.error))):cb.resolve(msg.result);}}else if(msg.method==='Runtime.exceptionThrown')errors.push(msg.params.exceptionDetails);}}); +function send(method,params={},sessionId){const id=++sequence;return new Promise((resolve,reject)=>{pending.set(id,{resolve,reject});chrome.stdio[3].write(JSON.stringify({id,method,params,...(sessionId?{sessionId}:{})})+'\0');});} +const deadline=setTimeout(()=>{console.error('Browser smoke timed out');chrome.kill('SIGTERM');process.exitCode=1;},60000); +try{ + const {targetId}=await send('Target.createTarget',{url:'about:blank'});const {sessionId}=await send('Target.attachToTarget',{targetId,flatten:true}); + const call=(m,p)=>send(m,p,sessionId);await call('Page.enable');await call('Runtime.enable');await call('Emulation.setDeviceMetricsOverride',{width:1440,height:1100,deviceScaleFactor:1,mobile:false}); + await call('Page.navigate',{url:process.argv[2].startsWith('http')?process.argv[2]:pathToFileURL(process.argv[2]).href}); + let ready=false;for(let i=0;i<100;i++){const r=await call('Runtime.evaluate',{expression:'typeof captureViewerState === "function" ? document.querySelectorAll("#neighbours tr").length : 0',returnByValue:true});if(r.result.value>0){ready=true;break;}await new Promise(r=>setTimeout(r,100));} + if(!ready)throw Error('Inspector did not render: '+JSON.stringify(errors)); + await call('Runtime.evaluate',{expression:"new Promise(resolve=>{setWorkspaceView('all');requestAnimationFrame(()=>requestAnimationFrame(resolve));})",awaitPromise:true}); + const result=await call('Runtime.evaluate',{expression:`(()=>{const checks=[];function check(v,s){if(!v)throw Error(s);checks.push(s)} + check(document.querySelectorAll('#stats .stat').length===4,'production summary'); + check(projectionHits.length===data.names.length,'projection contains every class'); + check(document.getElementById('projection-quality').textContent.includes('neighbours retained'),'projection reports distortion'); + const projectionBefore=document.getElementById('projection-map').toDataURL(); + document.getElementById('projection-plane').selectedIndex=1;document.getElementById('projection-plane').dispatchEvent(new Event('change')); + check(document.getElementById('projection-map').toDataURL()!==projectionBefore,'alternate PCA plane'); + document.getElementById('projection-plane').selectedIndex=0;document.getElementById('projection-plane').dispatchEvent(new Event('change')); + const allScale=projectionView.scale;document.getElementById('projection-focus').click();check(projectionView.scale>=allScale,'fit selected projected neighbourhood'); + document.getElementById('projection-reset').click(); + const canvas=document.getElementById('projection-map'),bounds=canvas.getBoundingClientRect(); + const oldScale=projectionView.scale;canvas.dispatchEvent(new WheelEvent('wheel',{clientX:bounds.left+bounds.width/2,clientY:bounds.top+bounds.height/2,deltaY:-150,cancelable:true}));check(projectionView.scale>oldScale,'projection zoom'); + const oldX=projectionView.x;projectionDrag={point:projectionMouse({clientX:bounds.left+100,clientY:bounds.top+100}),view:{...projectionView},moved:false};canvas.onpointermove({clientX:bounds.left+150,clientY:bounds.top+100});canvas.onpointerup({pointerId:1});check(projectionView.x!==oldX,'projection pan'); + document.getElementById('projection-reset').click(); + const target=projectionHits.findIndex(([x,y],i)=>i!==selected&&x>20&&x<1180&&y>20&&y=0,'projected class is selectable'); + const [px,py]=projectionHits[target];const event={button:0,pointerId:1,clientX:bounds.left+px*bounds.width/1200,clientY:bounds.top+py*bounds.height/projectionHeight};projectionDrag={point:projectionMouse(event),view:{...projectionView},moved:false};canvas.onpointerup(event); + check(selected===target&&document.getElementById('selected-name').textContent===data.names[target],'projection selection updates inspector'); + + check(document.querySelectorAll('#neighbours tr').length===data.stats.k,'production neighbours'); + check(Number.isFinite(Number(document.querySelector('#neighbours tr').children[3].textContent)),'finite float32 log tail in neighbour table'); + document.getElementById('range-preset').value='full';document.getElementById('range-preset').dispatchEvent(new Event('change'));const logImage=document.getElementById('matrix-min').toDataURL();document.getElementById('global-metric').value='baseline';document.getElementById('global-metric').dispatchEvent(new Event('change'));check(document.getElementById('matrix-min').toDataURL()!==logImage,'legacy versus log-domain global matrix');document.getElementById('global-metric').value='logtail';document.getElementById('global-metric').dispatchEvent(new Event('change')); + document.getElementById('metric').value='logtail';document.getElementById('metric').dispatchEvent(new Event('change'));check(document.getElementById('local-scale').textContent.includes('−log₁₀ tail'),'stable log-tail matrix'); + const rawFirst=globalValues.logmin.find(Number.isFinite);document.getElementById('range-preset').value='focus';document.getElementById('range-preset').dispatchEvent(new Event('change'));check(document.getElementById('matrix-min').toDataURL()!==logImage,'focused versus full colour range');check(globalValues.logmin.find(Number.isFinite)===rawFirst,'display clipping preserves raw values');document.getElementById('range-preset').value='robust';document.getElementById('range-preset').dispatchEvent(new Event('change'));check(Number(document.getElementById('range-low').value)>data.matrix_log10_floor,'observed percentile window');document.getElementById('range-low').value='1';document.getElementById('range-high').value='0';document.getElementById('apply-range').click();check(document.getElementById('range-status').textContent.includes('lower < upper'),'invalid display range feedback');document.getElementById('range-preset').value='focus';document.getElementById('range-preset').dispatchEvent(new Event('change')); + + document.getElementById('crowded').click();check(document.getElementById('selected-name').textContent===data.names[data.profiles.reduce((best,p,i)=>p[0]>data.profiles[best][0]?i:best,0)],'strongest neighbour selection'); + document.getElementById('metric').value='z';document.getElementById('metric').dispatchEvent(new Event('change'));check(document.getElementById('local-scale').textContent.includes('z '),'z metric toggle'); + document.getElementById('tree-locate').click();check(document.querySelectorAll('#tree .tree-label').length<=32,'tree locate and collapse'); + document.getElementById('tree-root').click();const collapsed=[...document.querySelectorAll('#tree .tree-label')].find(n=>n.textContent.includes('expand'));if(collapsed){collapsed.dispatchEvent(new MouseEvent('click'));check(size(treeFocus)0,name);} + document.getElementById('case').value='Production checkpoint';document.getElementById('case').dispatchEvent(new Event('change')); + return checks;})()`,returnByValue:true}); + if(result.exceptionDetails)throw Error(JSON.stringify(result.exceptionDetails));if(errors.length)throw Error(JSON.stringify(errors)); + await call('Runtime.evaluate',{expression:'document.getElementById("projection-panel").scrollIntoView()'}); + const projectionShot=await call('Page.captureScreenshot',{format:'png'});writeFileSync(join(output,'projection.png'),Buffer.from(projectionShot.data,'base64')); + await call('Runtime.evaluate',{expression:'window.scrollTo(0,0)'}); + const shot=await call('Page.captureScreenshot',{format:'png'});writeFileSync(join(output,'overview.png'),Buffer.from(shot.data,'base64')); + await call('Runtime.evaluate',{expression:'document.getElementById("inspector").scrollIntoView()'}); + const local=await call('Page.captureScreenshot',{format:'png'});writeFileSync(join(output,'neighbourhood.png'),Buffer.from(local.data,'base64')); + await call('Runtime.evaluate',{expression:'document.getElementById("global-metric").closest(".panel").scrollIntoView()'});const globalShot=await call('Page.captureScreenshot',{format:'png'});writeFileSync(join(output,'matrix.png'),Buffer.from(globalShot.data,'base64'));const report={checks:result.result.value,javascript_errors:errors.length};writeFileSync(join(output,'browser-check.json'),JSON.stringify(report,null,2)+'\n');console.log(JSON.stringify(report)); + const photos=await call('Runtime.evaluate',{expression:`(async()=>{ + const checks=[];const check=(value,label)=>{if(!value)throw Error(label);checks.push(label)}; + const originalFetch=window.fetch,originalAlbum=gbifClient.album; + const pixel='data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7'; + gbifClient.album=async id=>({class_id:id,accepted_id:id,display_name:'Fixture class',photos:[0,1].map(i=>({image_path:pixel,creator:'Fixture photographer',license:'Fixture media license',source:'https://www.gbif.org',occurrence_id:'1'}))});$('class-id-namespace').value='gbif';$('class-id-namespace').onchange(); + document.getElementById('photo-enabled').checked=true;await renderPhotos(); + check(document.querySelectorAll('.photo-card').length===data.stats.k+1,'photo neighbourhood cards'); + check(document.querySelector('.photo-card img').naturalWidth===1,'photo image loaded'); + check(document.querySelector('.photo-credit').textContent.includes('Fixture photographer'),'photo attribution'); + const next=document.querySelector('.photo-card button:not(.photo-open)');await next.onclick(); + check(document.querySelector('.photo-card .caption + .caption').textContent.includes('2 / 2'),'cycle class example'); + const expected=data.neighbours[selected][0];document.querySelectorAll('.photo-open')[1].click(); + check(selected===expected,'photo navigates to neighbour'); + document.getElementById('case').value='Algebraic edge cases';document.getElementById('case').dispatchEvent(new Event('change'));await renderPhotos(); + check(document.getElementById('photo-status').textContent.includes('No online lookup'),'synthetic case skips photo lookup'); + document.getElementById('photo-enabled').checked=false;await renderPhotos(); + document.getElementById('case').value='Production checkpoint';document.getElementById('case').dispatchEvent(new Event('change')); + const fixturePoints=[[60,60],[100,60],[260,100],[450,50]], config={width:400,height:200,size:64,density:200,overlap:0}; + const forcePair=(size,gap,dy=0)=>[{id:0,x:200,y:200,anchorX:200,anchorY:200,width:size,height:size},{id:1,x:200+gap,y:200+dy,anchorX:200+gap,anchorY:200+dy,width:size,height:size}]; + const distant=forcePair(64,70);stepThumbnailForce(distant,1000,800,128,0); + check(distant.every(b=>b.vx===0&&b.vy===0),'separate thumbnail boxes exert no repulsion beyond two-pixel clearance'); + const small=forcePair(32,50),large=forcePair(64,50);stepThumbnailForce(small,1000,800,128,0);stepThumbnailForce(large,1000,800,128,0); + check(small.every(b=>b.vx===0)&&large.some(b=>Math.abs(b.vx)>0),'contact uses thumbnail image size'); + const corners=forcePair(64,67,67);stepThumbnailForce(corners,1000,800,128,0); + check(corners.every(b=>b.vx===0&&b.vy===0),'diagonally separated rectangles do not repel as oversized ellipses'); + const translated=forcePair(64,50),base=forcePair(64,50);for(const b of translated){b.x+=100;b.anchorX+=100;b.y+=50;b.anchorY+=50;}stepThumbnailForce(base,1000,800,128,0);stepThumbnailForce(translated,1000,800,128,0); + check(base.every((b,i)=>Math.abs(b.vx-translated[i].vx)<1e-10&&Math.abs(b.vy-translated[i].vy)<1e-10),'screen-space forces are invariant to viewport translation'); + + check(thumbnailLayout(fixturePoints,[0,1,2,3],config).boxes.map(b=>b.id).join(',')==='0,2','thumbnail viewport and overlap culling'); + check(thumbnailLayout(fixturePoints,[0,1,2,3],{...config,overlap:.5}).boxes.length===3,'thumbnail allowable overlap'); + check(thumbnailLayout(fixturePoints,[0,1,2,3],{...config,density:5}).boxes.length===1,'thumbnail density budget'); + check(thumbnailLayout(fixturePoints,[0,1,2,3],{...config,size:140}).boxes.length===1,'thumbnail size changes culling'); + const compact=thumbnailLayout([[32,32],[96,32]],[0,1],{...config,width:128,height:64,labels:false,density:2000}); + check(compact.boxes.length===2&&compact.boxes.every(b=>b.width===64&&b.height===64),'image-only culling has no label or padding footprint'); + const wideFootprint=thumbnailFootprint(80,false,2),tallFootprint=thumbnailFootprint(80,true,.5); + check(wideFootprint.width===80&&wideFootprint.height===40&&tallFootprint.width===48&&tallFootprint.height===104,'original aspect footprints include image proportions and optional labels'); + const aspectSlots=thumbnailLayout([[40,20],[40,60]],[0,1],{width:80,height:80,size:80,density:1000,overlap:0,labels:false,aspects:[2,2]}); + check(aspectSlots.boxes.length===2,'aspect-aware culling uses rectangular footprints'); + check(thumbnailLayout([[100,100],[130,100]],[0,1],{width:200,height:200,size:64,density:200,overlap:.75,unknownOverlap:0,labels:false}).boxes.length===1,'unknown aspect ratios reserve nonoverlapping boxes when forces are disabled'); + check(thumbnailLayout([[100,100],[100,100]],[0,1],{width:200,height:200,size:64,density:200,overlap:.5,labels:false,aspects:[4,1]}).boxes.length===1,'overlap limit protects the smaller rectangle from being covered'); + + const originalPoints=JSON.stringify(fixturePoints),pushed=thumbnailLayout(fixturePoints,[0,1,2,3],{...config,overlap:.5,push:true}); + check(pushed.boxes.length===3&&pushed.boxes.some(b=>b.x!==b.anchorX||b.y!==b.anchorY),'push apart retains and separates overlapping candidates'); + const separate=boxes=>boxes.every((b,i)=>boxes.slice(i+1).every(other=>thumbnailSharedArea(b,other)<1e-8)); + check(separate(pushed.boxes),'pushed rectangles do not overlap'); + check(pushed.boxes.every(b=>b.left>=0&&b.top>=0&&b.left+b.width<=config.width&&b.top+b.height<=config.height&&Math.hypot(b.x-b.anchorX,b.y-b.anchorY)<=2*config.size+1e-8),'push apart respects viewport and displacement bounds'); + check(JSON.stringify(fixturePoints)===originalPoints&&JSON.stringify(thumbnailLayout(fixturePoints,[0,1,2,3],{...config,overlap:.5,push:true}))===JSON.stringify(pushed),'push apart is deterministic and preserves source points'); + const dense=Array.from({length:60},(_,i)=>[40+(i%10)*18,40+Math.floor(i/10)*18]); + const tight=thumbnailLayout(dense,dense.map((_,i)=>i),{...config,width:260,height:180,labels:false,overlap:.75,push:true,density:2000}); + check(tight.boxes.length>0&&tight.culled===0&&tight.boxes.every(b=>b.left>=0&&b.top>=0&&b.left+b.width<=260&&b.top+b.height<=180),'crowded boundary fixture retains all admitted boxes within bounds'); + document.getElementById('map-photos').checked=true;scheduleMapThumbnails();clearTimeout(mapThumbTimer);await renderMapThumbnails(); + check(document.querySelectorAll('.map-thumb').length>0,'map thumbnails load'); + check([...document.querySelectorAll('.map-thumb img')].every(img=>img.naturalWidth>0),'map thumbnails decode'); + const first=mapThumbLayout[0];check(mapThumbnailHit([first.x*1200/projectionCanvas.getBoundingClientRect().width,first.y*600/projectionCanvas.getBoundingClientRect().height])===first.id,'thumbnail hit testing'); + showMapPhotoCredit(first.id);check(document.getElementById('map-photo-credit').textContent.includes('Fixture photographer'),'map thumbnail attribution'); + document.querySelector('.map-thumb').click();check(selected===first.id,'thumbnail selection updates class'); + const coordinatesBefore=JSON.stringify(projectionHits); + document.getElementById('map-photo-labels').checked=false;document.getElementById('map-photo-push').checked=true;document.getElementById('map-photo-overlap').value='75';document.getElementById('map-photo-density').value='200'; + scheduleMapThumbnails();clearTimeout(mapThumbTimer);await renderMapThumbnails(); + check(document.querySelectorAll('.map-thumb').length>0&&[...document.querySelectorAll('.map-thumb')].every(b=>!b.querySelector('span')&&getComputedStyle(b).padding==='0px'&&getComputedStyle(b).borderWidth==='0px'),'image-only toggle removes labels and whitespace'); + check(mapThumbRest&&mapThumbRest.pairs===thumbnailEnergy(mapThumbLayout).pairs&&document.getElementById('map-photo-status').textContent.includes('overlapping pairs retained'),'settled status reports residual overlap without hiding it'); + check(JSON.stringify(projectionHits)===coordinatesBefore,'thumbnail relaxation preserves projected coordinates'); + const displaced=mapThumbLayout.find(b=>Math.hypot(b.x-b.anchorX,b.y-b.anchorY)>=.5); + check(displaced&&document.querySelectorAll('.map-thumb-tether').length>0,'displaced photos retain visible anchor markers'); + const anchorNodes=[...document.querySelectorAll('.map-thumb-tether')]; + check(anchorNodes.every(n=>getComputedStyle(n).display==='none'),'thumbnail anchors default to hidden'); + const layoutBeforeToggle=mapThumbLayout; + document.getElementById('map-photo-anchors').checked=true;document.getElementById('map-photo-anchors').dispatchEvent(new Event('input')); + check(anchorNodes.every(n=>getComputedStyle(n).display!=='none')&&mapThumbLayout===layoutBeforeToggle,'anchor toggle does not restart or reload thumbnails'); + document.getElementById('map-photo-anchors').checked=false;setThumbnailAnchors(); + const beforeView={...projectionView},beforeNodes=new Map(mapThumbLayout.map(b=>[b.id,{button:b.button,x:b.x,anchorX:b.anchorX,offset:b.x-b.anchorX,width:b.width}])); + projectionView.x+=2/projectionView.scale;projectionView.scale*=1.01;drawProjection();clearTimeout(mapThumbTimer); + check(mapThumbLayout.length>0&&mapThumbLayout.every(b=>b.button===beforeNodes.get(b.id).button&&Math.abs(b.x-b.anchorX-beforeNodes.get(b.id).offset)<1e-8&&b.width===beforeNodes.get(b.id).width),'pan and zoom preserve loaded DOM nodes, pixel offsets and image dimensions'); + const fetchBefore=window.fetch;let refreshRequests=0;window.fetch=async(...args)=>{refreshRequests++;return fetchBefore(...args);};await renderMapThumbnails();window.fetch=fetchBefore; + check(mapThumbLayout.some(b=>beforeNodes.get(b.id)?.button===b.button),'settled viewport refresh reuses surviving thumbnail nodes'); + projectionView=beforeView;drawProjection();clearTimeout(mapThumbTimer);await renderMapThumbnails(); + + const motionConfig={width:projectionCanvas.getBoundingClientRect().width,height:projectionCanvas.getBoundingClientRect().height,size:64}; + for(const b of mapThumbLayout){b.x=b.anchorX;b.y=b.anchorY;b.left=b.x-b.width/2;b.top=b.y-b.height/2;b.vx=0;b.vy=0;} + const startPositions=mapThumbLayout.map(b=>[b.x,b.y]);const moving=animateThumbnails(motionConfig); + await new Promise(resolve=>setTimeout(resolve,120)); + check(mapThumbMotion!==null&&mapThumbLayout.some((b,i)=>Math.hypot(b.x-startPositions[i][0],b.y-startPositions[i][1])>.1),'force layout advances visibly over animation frames'); + check(mapThumbLayout.every(b=>Math.abs(parseFloat(b.button.style.left)-b.x)<.01&&Math.abs(parseFloat(b.button.style.top)-b.y)<.01),'animated images and hit rectangles move together'); + await moving;check(mapThumbMotion===null&&mapThumbLayout.length===startPositions.length,'solver stops without removing thumbnails'); + const originalMatchMedia=window.matchMedia;window.matchMedia=()=>({matches:true});await animateThumbnails(motionConfig);window.matchMedia=originalMatchMedia; + check(mapThumbMotion===null&&mapThumbLayout.length===startPositions.length,'reduced motion preserves the same thumbnail population'); + const savedLayout=mapThumbLayout; + const makeCrowded=()=>[0,1].map(i=>{ + const b={id:i,x:32,y:32,anchorX:32,anchorY:32,left:0,top:0,width:64,height:64,vx:0,vy:0}; + b.button=savedLayout[0].button.cloneNode(true);b.tether=thumbnailTether(b,true);document.getElementById('map-thumbnail-layer').append(b.button,b.tether);return b; + }); + mapThumbLayout=makeCrowded();const crowded=mapThumbLayout;await animateThumbnails({width:64,height:64,size:64}); + check(mapThumbLayout===crowded&&crowded.every(b=>b.button.isConnected&&b.button.style.opacity===''),'impossible packing keeps both thumbnails visible at rest'); + check(mapThumbRest.reason==='resting'&&mapThumbRest.pairs===1&&mapThumbRest.steps<480,'constrained equilibrium stops on measured stability and reports overlap'); + await new Promise(resolve=>setTimeout(resolve,300)); + check(crowded.every(b=>b.button.isConnected),'no deferred settling-time removal'); + for(const b of crowded){b.button.remove();b.tether.remove();}mapThumbLayout=savedLayout;paintThumbnailMotion(); + const fading={...savedLayout[0],id:-100,button:savedLayout[0].button.cloneNode(true),tether:null}; + document.getElementById('map-thumbnail-layer').append(fading.button); + // Exercise viewport culling with a valid, fully offscreen projected position. + mapThumbLayout=[{...fading,id:0}];reanchorThumbnails([[-1000,-1000]],motionConfig.width,motionConfig.height); + check(mapThumbLayout.length===0&&fading.button.isConnected&&fading.button.disabled&&fading.button.style.pointerEvents==='none','viewport culling removes hit targets immediately while keeping the exit visible'); + check(fading.button.getAnimations().some(a=>a.effect.getTiming().duration===100),'culled thumbnail gets a fast exit animation'); + await new Promise(resolve=>setTimeout(resolve,180)); + check(!fading.button.isConnected,'culled thumbnail is removed after fading'); + mapThumbLayout=savedLayout;paintThumbnailMotion(); + const cancelled=animateThumbnails(motionConfig);stopThumbnailMotion();await cancelled; + check(mapThumbMotion===null,'force animation cancellation resolves pending work'); + document.getElementById('map-photos').checked=false;scheduleMapThumbnails();document.getElementById('map-photos').checked=true;document.getElementById('map-photo-density').value='5';mapThumbImages.clear(); + const originalLoad=loadReferenceImage;let activeImages=0,maxImages=0; + loadReferenceImage=async(...args)=>{activeImages++;maxImages=Math.max(maxImages,activeImages);await new Promise(resolve=>setTimeout(resolve,100));const result=await originalLoad(...args);activeImages--;return result;}; + scheduleMapThumbnails();clearTimeout(mapThumbTimer);const delayed=renderMapThumbnails();await new Promise(resolve=>setTimeout(resolve,40)); + check(activeImages>1&&activeImages<=4&&mapThumbLayout.length===0&&mapThumbMotion===null,'bounded concurrent image loads do not move unloaded thumbnails'); + await delayed;loadReferenceImage=originalLoad; + check(maxImages<=4&&mapThumbLayout.length>0&&mapThumbLayout.every(b=>b.button.querySelector('img').naturalWidth>0),'only visible decoded images enter force layout'); + const wideImage='data:image/svg+xml,'+encodeURIComponent(''); + const portraitImage='data:image/svg+xml,'+encodeURIComponent(''); + let albumIndex=0;for(const album of photoAlbums.values()){for(const photo of album.photos)photo.image_path=albumIndex%2?wideImage:portraitImage;albumIndex++;} + document.getElementById('map-photo-aspect').checked=true;document.getElementById('map-photo-aspect').dispatchEvent(new Event('input'));clearTimeout(mapThumbTimer);await renderMapThumbnails(); + check(mapThumbLayout.length>0&&mapThumbLayout.every(b=>{const image=b.button.querySelector('img'),rect=image.getBoundingClientRect();return Math.abs(rect.width/rect.height-image.naturalWidth/image.naturalHeight)<.002&&Math.abs(b.width-rect.width)<.02&&Math.abs(b.height-rect.height)<.02;}),'original aspect option displays uncropped images with matching collision boxes'); + document.getElementById('map-photo-aspect').checked=false;document.getElementById('map-photo-aspect').dispatchEvent(new Event('input'));clearTimeout(mapThumbTimer);await renderMapThumbnails(); + check(mapThumbLayout.every(b=>Math.abs(b.width-b.height)<.01&&getComputedStyle(b.button.querySelector('img')).objectFit==='cover'),'square crop remains selectable'); + + + + const currentBox=mapThumbLayout[0];check(mapThumbnailHit([currentBox.x*1200/projectionCanvas.getBoundingClientRect().width,currentBox.y*projectionHeight/projectionCanvas.getBoundingClientRect().height])===currentBox.id,'displaced thumbnail hit testing'); + document.getElementById('map-photo-labels').checked=true;document.getElementById('map-photo-push').checked=false;document.getElementById('map-photo-overlap').value='0';document.getElementById('map-photo-density').value='40'; + scheduleMapThumbnails();clearTimeout(mapThumbTimer);await renderMapThumbnails(); + check([...document.querySelectorAll('.map-thumb')].every(b=>b.querySelector('span'))&&document.querySelectorAll('.map-thumb-tether').length===0,'label and fixed-position modes restore'); + document.getElementById('map-photos').checked=false;scheduleMapThumbnails();check(document.querySelectorAll('.map-thumb').length===0,'map thumbnail toggle clears images'); + + const fixtureAlbum=gbifClient.album;photoAlbums.clear(); + gbifClient.album=async id=>{await new Promise(r=>setTimeout(r,40));return fixtureAlbum(id);}; + document.getElementById('map-photos').checked=true;scheduleMapThumbnails();clearTimeout(mapThumbTimer);const pendingMap=renderMapThumbnails(); + document.getElementById('map-photos').checked=false;scheduleMapThumbnails();await pendingMap; + check(document.querySelectorAll('.map-thumb').length===0,'cancelled thumbnail load stays hidden'); + let syntheticRequests=0;gbifClient.album=async()=>{syntheticRequests++;throw Error('Unexpected synthetic lookup')}; + document.getElementById('case').value='Algebraic edge cases';document.getElementById('case').dispatchEvent(new Event('change'));document.getElementById('map-photos').checked=true;scheduleMapThumbnails(); + check(syntheticRequests===0&&document.getElementById('map-photo-status').textContent.includes('Synthetic'),'synthetic map skips photo requests'); + document.getElementById('map-photos').checked=false;scheduleMapThumbnails();gbifClient.album=fixtureAlbum; + window.fetch=originalFetch;gbifClient.album=originalAlbum; + document.getElementById('case').value='Production checkpoint';document.getElementById('case').dispatchEvent(new Event('change')); + return checks; + })()`,awaitPromise:true,returnByValue:true}); + if(photos.exceptionDetails)throw Error(JSON.stringify(photos.exceptionDetails)); + report.checks.push(...photos.result.value);report.javascript_errors=errors.length; + writeFileSync(join(output,'browser-check.json'),JSON.stringify(report,null,2)+'\n');console.log(JSON.stringify({photo_checks:photos.result.value,javascript_errors:errors.length})); + const focusChecks=[]; + for(const [width,height] of [[1800,900],[900,1500],[390,844]]){ + await call('Emulation.setDeviceMetricsOverride',{width,height,deviceScaleFactor:1,mobile:false}); + const focused=await call('Runtime.evaluate',{expression:`(async()=>{ + const checks=[],check=(v,label)=>{if(!v)throw Error(label);checks.push(label)}; + const before={selected,treeFocus,plane:$('projection-plane').value,x:projectionView.x,y:projectionView.y}; + setWorkspaceView('projection');await new Promise(r=>setTimeout(r,250)); + const rect=projectionCanvas.getBoundingClientRect(); + check(! $('inspector').hidden&&!$('photo-panel').hidden&&$('tree-panel').hidden,'projection groups related panels'); + check(rect.width>innerWidth*.8&&rect.height>=300,'projection uses viewport width and height'); + const nav=$('activity-nav').getBoundingClientRect();check(Math.abs(rect.top-nav.bottom)<1&&rect.left===0&&Math.abs(rect.bottom-innerHeight)<1&&Math.abs(rect.width-document.documentElement.clientWidth)<1,'projection fills available viewport below navigation'); + $('workspace-controls').open=true;$('projection-controls').open=true;await new Promise(r=>setTimeout(r,60)); + const expanded=projectionCanvas.getBoundingClientRect(); + check(expanded.width===rect.width&&expanded.height===rect.height&&expanded.top===rect.top,'floating controls do not resize map'); + check($('workspace-view').getBoundingClientRect().width>0&&$('projection-plane').getBoundingClientRect().width>0,'view and map settings remain accessible'); + $('workspace-controls').open=false;$('projection-controls').open=false; + check(Math.abs(rect.width/1200-rect.height/projectionHeight)<1e-8,'projection preserves equal screen scale'); + check(selected===before.selected&&treeFocus===before.treeFocus&&$('projection-plane').value===before.plane,'focus preserves selection tree and plane'); + check(projectionView.x===before.x&&projectionView.y===before.y,'focus preserves projection center'); + const origin=projectionPoint([projectionView.x,projectionView.y]); + const hit=projectionMouse({clientX:rect.left+origin[0]*rect.width/1200,clientY:rect.top+origin[1]*rect.height/projectionHeight}); + check(Math.hypot(hit[0]-origin[0],hit[1]-origin[1])<1e-8,'pointer conversion follows focused aspect ratio'); + setWorkspaceView('tree');await new Promise(r=>setTimeout(r,100)); + check(!$('tree-panel').hidden&&!$('inspector').hidden&&$('projection-panel').hidden,'tree groups selected-class context'); + check($('tree').viewBox.baseVal.width>=760,'tree preserves readable labels'); + check(document.documentElement.scrollWidth<=innerWidth+1,'focused layout avoids page-wide horizontal scrolling'); + setWorkspaceView('matrix');check(!$('matrix-panel').hidden&&!$('inspector').hidden&&$('tree-panel').hidden,'matrix groups local inspection'); + setWorkspaceView('all');await new Promise(r=>setTimeout(r,100)); + check([...document.querySelectorAll('main .panel')].every(p=>!p.hidden),'all views restored'); + return checks; + })()`,returnByValue:true,awaitPromise:true}); + if(focused.exceptionDetails)throw Error(JSON.stringify(focused.exceptionDetails)); + focusChecks.push({width,height,checks:focused.result.value}); + } + if(errors.length)throw Error(JSON.stringify(errors)); + report.focus_checks=focusChecks;report.javascript_errors=errors.length; + writeFileSync(join(output,'browser-check.json'),JSON.stringify(report,null,2)+'\n'); + console.log(JSON.stringify({focusChecks,javascript_errors:errors.length})); + await call('Emulation.setDeviceMetricsOverride',{width:1440,height:1100,deviceScaleFactor:2,mobile:false}); + const enhanced=await call('Runtime.evaluate',{expression:`(async()=>{ + const checks=[],check=(v,label)=>{if(!v)throw Error(label);checks.push(label)}; + setWorkspaceView('projection');await new Promise(r=>setTimeout(r,200)); + const before={selected,plane:$('projection-plane').value,x:projectionView.x,y:projectionView.y,names:JSON.stringify(data.names)}; + $('class-names').checked=true; + const local=new Map([[selected,'Selected species'],[data.neighbours[selected][0],'Neighbour species']]);importedNames.set(data,local);refreshClassLabels(); + check($('selected-name').textContent.includes('Selected species'),'selected class shows optional name and ID'); + check($('neighbours').textContent.includes('Neighbour species'),'neighbour table resolves names'); + check($('local').textContent.includes('Neighbour species'),'matrix axes and hover resolve names'); + check($('profile').textContent.includes('Neighbour species'),'profile retains rank with neighbour name'); + $('search').value='Neighbour species';$('find').click();check(selected===data.neighbours[before.selected][0],'manual selector searches resolved names');selectClass(before.selected); + treeFocus=selected;drawTree();check($('tree').textContent.includes('Selected species'),'dendrogram leaves resolve names'); + check(JSON.stringify(data.names)===before.names,'names never replace checkpoint IDs'); + $('score-display').value='tail';$('score-display').onchange(); + check($('score-column').textContent.includes('Φ')&&$('metric').value==='tail','probability toggle updates table and matrix'); + check(!$('score-note').hidden&&$('local-scale').textContent.includes('Upper-tail'),'probability reference and logarithmic colours disclosed'); + check($('profile').textContent.includes('Upper-tail'),'profile probability axis'); + $('score-display').value='z';$('score-display').onchange();check($('score-column').textContent==='z','z display restores'); + const original=gbifClient.name,ids=[selected,...data.neighbours[selected]],pending=[];let inFlight=0,peak=0; + importedNames.delete(data);resolvedNames.clear();nameFailures.clear(); + gbifClient.name=async id=>{inFlight++;peak=Math.max(peak,inFlight);pending.push(id);await new Promise(r=>setTimeout(r,30));inFlight--;return {class_id:id,display_name:'Async taxon '+id};}; + $('class-id-namespace').value='gbif';requestClassNames(); + for(let i=0;i<60&&!resolvedNames.size;i++)await new Promise(r=>setTimeout(r,30)); + await new Promise(r=>setTimeout(r,200)); + check(resolvedNames.size>0&&pending.every(id=>String(Number(id))===id),'asynchronous names need no photo lookup'); + check(peak===1&&pending.length<30,'name lookup is bounded and prioritizes a small visible set'); + $('class-names').checked=false;$('class-id-namespace').value='generic';await new Promise(r=>setTimeout(r,100));gbifClient.name=original;refreshClassLabels(); + check($('selected-name').textContent===data.names[selected],'IDs restore without losing selected class'); + const rect=projectionCanvas.getBoundingClientRect();check(projectionCanvas.width===Math.round(rect.width*devicePixelRatio)&&projectionCanvas.height===Math.round(rect.height*devicePixelRatio),'canvas backing matches physical screen pixels'); + check(rect.height>=300,'focused map retains vertical workspace'); + const geometry=()=>{const r=projectionCanvas.getBoundingClientRect();return JSON.stringify([r.x,r.y,r.width,r.height,projectionView.x,projectionView.y,projectionView.scale]);}; + const stableGeometry=geometry(),savedCredit=$('map-photo-credit').innerHTML; + for(const expanded of [false,true]){ + $('projection-details').open=expanded; + for(const text of ['', 'Long taxon name and attribution '.repeat(100),'Short credit']){ + $('map-photo-credit').textContent=text;$('map-photo-status').textContent=text; + await new Promise(r=>setTimeout(r,60)); + check(geometry()===stableGeometry,'hover and status preserve map geometry with details '+expanded); + } + } + $('thumbnail-options').open=true;await new Promise(r=>setTimeout(r,60)); + check(geometry()===stableGeometry,'thumbnail settings overlay preserves map geometry'); + $('thumbnail-options').open=false;$('projection-details').open=false;$('map-photo-credit').innerHTML=savedCredit; + check(rect.height>innerHeight*.75,'focused map uses over three quarters of landscape height'); + const saved=captureViewerState(); + selectClass((selected+1)%data.names.length);applyViewerState(saved); + await new Promise(r=>setTimeout(r,120)); + check(data.names[selected]===saved.selected&&projectionView.scale===saved.view.scale,'saved state restores selection and projection'); + let rejected=false;try{applyViewerState({...saved,identity:'other-model'});}catch{rejected=true;} + check(rejected,'state from another model is rejected'); + rejected=false;try{applyViewerState({...saved,view:{x:0,y:0,scale:-1}});}catch{rejected=true;} + check(rejected,'malformed saved geometry is rejected'); + saveViewerState();check(JSON.parse(localStorage.getItem('prototype-view:'+viewerIdentity())).version===2,'state persists in browser storage'); + const storageWrite=Storage.prototype.setItem;Storage.prototype.setItem=()=>{throw Error('quota');};saveViewerState();Storage.prototype.setItem=storageWrite; + check($('viewer-state-status').textContent.includes('unavailable'),'storage failure leaves viewer usable'); + const metadataFetch=gbifClient.album;$('class-id-namespace').value='gbif';let requests=0,active=0,peakMetadata=0; + gbifClient.album=async id=>{requests++;active++;peakMetadata=Math.max(peakMetadata,active);await new Promise(r=>setTimeout(r,40));active--;return {class_id:id,photos:[]};}; + const obsolete=new AbortController(),current=new AbortController(); + const first=referenceAlbum('fixture-shared',obsolete.signal);obsolete.abort(); + const second=referenceAlbum('fixture-shared',current.signal);await Promise.all([first,second]); + check(requests===1,'navigation reuses in-flight class metadata'); + await Promise.all(Array.from({length:8},(_,i)=>referenceAlbum('fixture-bound-'+i,current.signal))); + check(peakMetadata<=4,'metadata requests are globally bounded'); + const cachedRequests=requests;await referenceAlbum('fixture-shared',current.signal); + check(requests===cachedRequests,'resolved metadata is reused without fetching');gbifClient.album=metadataFetch; + + + return checks; + })()`,returnByValue:true,awaitPromise:true}); + if(enhanced.exceptionDetails)throw Error(JSON.stringify(enhanced.exceptionDetails)); + console.log(JSON.stringify({enhancedChecks:enhanced.result.value,javascript_errors:errors.length})); + await send('Browser.close'); +}catch(e){console.error(e);process.exitCode=1;chrome.kill('SIGTERM');}finally{clearTimeout(deadline);setTimeout(()=>rmSync(profile,{recursive:true,force:true}),1000);} diff --git a/dev/prototype_space/check_gbif.cjs b/dev/prototype_space/check_gbif.cjs new file mode 100644 index 00000000..7097d48e --- /dev/null +++ b/dev/prototype_space/check_gbif.cjs @@ -0,0 +1,37 @@ +/* Deterministic transport checks; no public API calls. */ +const assert=require('node:assert/strict'); +const {BrowserGBIF}=require('../../mini_trainer/visualization/prototype_space/gbif.js'); +(async()=>{ + let calls=0,active=0,peak=0; + const client=new BrowserGBIF({concurrency:2,fetcher:async url=>{ + calls++;active++;peak=Math.max(peak,active); + await new Promise(resolve=>setTimeout(resolve,5));active--; + return {ok:true,json:async()=>({key:Number(url.split('/').pop()),canonicalName:'Example'})}; + }}); + assert.equal(client.seed({schema:'mini-trainer-gbif-v1',taxa:{'42':{key:42,canonicalName:'Cached species'},'43':{key:99}}}),1); + assert.equal((await client.name('42')).display_name,'Cached species');assert.equal(calls,0); + const [a,b]=await Promise.all([client.name('123'),client.name('123')]); + assert.deepEqual(a,b);assert.equal(calls,1);await client.name('123');assert.equal(calls,1); + await Promise.all(['1','2','3','4'].map(id=>client.name(id)));assert.equal(peak,2); + await assert.rejects(client.name('not-a-taxon')); + const album=BrowserGBIF.albumFromRecords('1',{acceptedKey:2},[ + {key:10,speciesKey:9,media:[{type:'StillImage',identifier:'https://example.org/wrong.jpg'}]}, + {key:11,speciesKey:2,media:[{type:'StillImage',identifier:'javascript:alert(1)'},{type:'StillImage',identifier:'https://example.org/photo.jpg',creator:'Author',license:'CC BY'}]} + ]); + assert.equal(album.class_id,'1');assert.equal(album.accepted_id,'2');assert.equal(album.photos.length,1); + assert.equal(album.photos[0].creator,'Author');assert.equal(album.photos[0].license,'CC BY'); + let release; + const slow=new BrowserGBIF({fetcher:()=>new Promise(resolve=>release=resolve)}); + const pending=slow.name('1');slow.reset();release({ok:true,json:async()=>({key:1})}); + await assert.rejects(pending,{name:'AbortError'});assert.equal(slow.cache.size,0); + const limited=new BrowserGBIF({fetcher:async()=>({ok:false,status:429})}); + await assert.rejects(limited.name('1'),/request limit/);assert.equal(limited.pending.size,0); + const offline=new BrowserGBIF({fetcher:async()=>{throw Error('Offline');}});await assert.rejects(offline.name('1'),/Offline/); + const malformed=new BrowserGBIF({fetcher:async()=>({ok:true,json:async()=>null})});await assert.rejects(malformed.name('1'),/Invalid GBIF response/); + const wrong=new BrowserGBIF({fetcher:async()=>({ok:true,json:async()=>({key:2})})});await assert.rejects(wrong.name('1'),/different taxon/); + assert.deepEqual(BrowserGBIF.albumFromRecords('1',{key:1},[]).photos,[]); + assert.throws(()=>client.seed({schema:'unsupported',taxa:{}}),/snapshot/); + const timed=new BrowserGBIF({timeout:5,fetcher:async(url,{signal})=>new Promise((resolve,reject)=>signal.addEventListener('abort',()=>reject(signal.reason)))}); + const keepAlive=setTimeout(()=>{},100);await assert.rejects(timed.name('1'),{name:'TimeoutError'});clearTimeout(keepAlive); + console.log('GBIF checks passed: deduplication, cache, concurrency, IDs, taxonomy, URLs, attribution, cancellation, rate limit.'); +})().catch(error=>{console.error(error);process.exitCode=1;}); diff --git a/dev/prototype_space/check_inference.mjs b/dev/prototype_space/check_inference.mjs new file mode 100644 index 00000000..d3a77d13 --- /dev/null +++ b/dev/prototype_space/check_inference.mjs @@ -0,0 +1,34 @@ +import {spawn} from 'node:child_process'; +import {mkdtempSync,writeFileSync,mkdirSync} from 'node:fs'; +if(!process.env.CHROME_PATH||!process.argv[2]||!process.argv[3])throw Error('Usage: CHROME_PATH=/path/to/chrome node check_inference.mjs URL IMAGE_PATH'); +const chrome=spawn(process.env.CHROME_PATH,['--headless','--no-sandbox','--disable-gpu','--disable-dev-shm-usage','--no-first-run','--disable-background-networking','--remote-debugging-pipe','--user-data-dir='+mkdtempSync('/tmp/browser-inference-check-')],{stdio:['ignore','ignore','ignore','pipe','pipe']}); +let serial=0,buffer=Buffer.alloc(0);const pending=new Map(),errors=[]; +chrome.stdio[4].on('data',chunk=>{buffer=Buffer.concat([buffer,chunk]);let end;while((end=buffer.indexOf(0))!==-1){const msg=JSON.parse(buffer.subarray(0,end));buffer=buffer.subarray(end+1);if(msg.id){const p=pending.get(msg.id);pending.delete(msg.id);msg.error?p.reject(Error(JSON.stringify(msg.error))):p.resolve(msg.result);}else if(msg.method==='Runtime.exceptionThrown')errors.push(msg.params.exceptionDetails);}}); +function send(method,params={},sessionId){const id=++serial;return new Promise((resolve,reject)=>{pending.set(id,{resolve,reject});chrome.stdio[3].write(JSON.stringify({id,method,params,...(sessionId?{sessionId}:{})})+'\0');});} +const timeout=setTimeout(()=>{chrome.kill();process.exitCode=1;console.error('Timeout');},240000); +try{ + const {targetId}=await send('Target.createTarget',{url:'about:blank'});const {sessionId}=await send('Target.attachToTarget',{targetId,flatten:true});const call=(m,p)=>send(m,p,sessionId); + await call('Runtime.enable');await call('Page.enable');await call('Page.navigate',{url:process.argv[2]}); + const evaluate=async expression=>{const r=await call('Runtime.evaluate',{expression,returnByValue:true,awaitPromise:true});if(r.exceptionDetails)throw Error(JSON.stringify(r.exceptionDetails));return r.result.value;}; + for(let i=0;i<150;i++){if(await evaluate('typeof captureViewerState==="function" && !!document.getElementById("inference-load")'))break;await new Promise(r=>setTimeout(r,200));} + if(errors.length)throw Error(JSON.stringify(errors)); + await evaluate('setWorkspaceView("predict");document.getElementById("inference-load").click()'); + for(let i=0;i<600;i++){const status=await evaluate('document.getElementById("inference-status").textContent');if(i%25===0)console.log(status);if(status.startsWith('Model ready'))break;if(/failed|Unsupported|Error|HTTP|no available/.test(status))throw Error(status);await new Promise(r=>setTimeout(r,200));} + const parity=[]; + const {root}=await call('DOM.getDocument');const {nodeId}=await call('DOM.querySelector',{nodeId:root.nodeId,selector:'#inference-image'});await call('DOM.setFileInputFiles',{nodeId,files:[process.argv[3]]}); + for(let i=0;i<600;i++){const status=await evaluate('document.getElementById("inference-status").textContent');if(i%50===0)console.log(status);if(status.startsWith('Complete.'))break;await new Promise(r=>setTimeout(r,200));} + const result=await evaluate(`({status:document.getElementById('inference-status').textContent,rows:document.querySelectorAll('#inference-results li').length,embedding:inferenceQuery?.embedding.length,insertion:inferenceQuery?.insertion,plane:document.getElementById('projection-plane').value})`); + console.log(JSON.stringify({result,errors}));if(process.argv[4])writeFileSync(process.argv[4],JSON.stringify({result,errors},null,2)); + if(!result.status.startsWith('Complete.')||result.rows!==15||!result.embedding||errors.length)throw Error('Browser acceptance failed'); + if(process.argv[5]){ + mkdirSync(process.argv[5],{recursive:true}); + await evaluate('document.getElementById("class-id-namespace").value="gbif";document.getElementById("class-id-namespace").onchange()'); + await new Promise(resolve=>setTimeout(resolve,12000)); + for(const [name,width,height] of [['desktop',1440,1000],['mobile',390,844]]){ + await call('Emulation.setDeviceMetricsOverride',{width,height,deviceScaleFactor:1,mobile:false}); + await evaluate('window.scrollTo(0,0)');await new Promise(resolve=>setTimeout(resolve,400)); + const shot=await call('Page.captureScreenshot',{format:'png'});writeFileSync(process.argv[5]+'/'+name+'.png',Buffer.from(shot.data,'base64')); + } + } + await send('Browser.close'); +}catch(error){console.error(error);chrome.kill();process.exitCode=1;}finally{clearTimeout(timeout);} diff --git a/dev/prototype_space/check_launcher.mjs b/dev/prototype_space/check_launcher.mjs new file mode 100644 index 00000000..1637df6f --- /dev/null +++ b/dev/prototype_space/check_launcher.mjs @@ -0,0 +1,47 @@ +// Optional browser smoke check. Set CHROMIUM_BIN; pass a report URL/path and output directory. +import {spawn} from 'node:child_process'; +import {writeFileSync,mkdtempSync,rmSync,mkdirSync,existsSync} from 'node:fs'; +import {tmpdir} from 'node:os'; +import {pathToFileURL} from 'node:url'; +import {resolve,join} from 'node:path'; +const executable=process.env.CHROMIUM_BIN; +if(!executable||!existsSync(executable)||!process.argv[2])throw Error('Set CHROMIUM_BIN and pass report URL/path [output directory]'); +const output=resolve(process.argv[3]||'tmp/prototype-browser-check');mkdirSync(output,{recursive:true}); +const profile=mkdtempSync(tmpdir()+'/prototype-browser-'); +const chrome=spawn(executable,['--headless','--no-sandbox','--disable-gpu','--disable-dev-shm-usage','--no-first-run','--no-default-browser-check','--disable-background-networking','--disable-component-update','--disable-extensions','--remote-debugging-pipe','--user-data-dir='+profile],{stdio:['ignore','ignore','ignore','pipe','pipe']}); +let sequence=0,buffer=Buffer.alloc(0);const pending=new Map();const errors=[]; +chrome.stdio[4].on('data',chunk=>{buffer=Buffer.concat([buffer,chunk]);let end;while((end=buffer.indexOf(0))!==-1){const msg=JSON.parse(buffer.subarray(0,end));buffer=buffer.subarray(end+1);if(msg.id){const cb=pending.get(msg.id);if(cb){pending.delete(msg.id);msg.error?cb.reject(Error(JSON.stringify(msg.error))):cb.resolve(msg.result);}}else if(msg.method==='Runtime.exceptionThrown')errors.push(msg.params.exceptionDetails);}}); +function send(method,params={},sessionId){const id=++sequence;return new Promise((resolve,reject)=>{pending.set(id,{resolve,reject});chrome.stdio[3].write(JSON.stringify({id,method,params,...(sessionId?{sessionId}:{})})+'\0');});} +const deadline=setTimeout(()=>{console.error('Browser smoke timed out');chrome.kill('SIGTERM');process.exitCode=1;},60000); +try{ + const {targetId}=await send('Target.createTarget',{url:'about:blank'});const {sessionId}=await send('Target.attachToTarget',{targetId,flatten:true}); + const call=(m,p)=>send(m,p,sessionId);await call('Page.enable');await call('Runtime.enable'); + await call('Emulation.setDeviceMetricsOverride',{width:1440,height:1000,deviceScaleFactor:1,mobile:false}); + await call('Page.navigate',{url:process.argv[2]}); + const evaluate=async expression=>{const result=await call('Runtime.evaluate',{expression,returnByValue:true,awaitPromise:true});if(result.exceptionDetails)throw Error(JSON.stringify(result.exceptionDetails));return result.result.value;}; + for(let i=0;i<100;i++){if(await evaluate('!!document.getElementById("weights")'))break;await new Promise(r=>setTimeout(r,100));} + const {root}=await call('DOM.getDocument');const {nodeId}=await call('DOM.querySelector',{nodeId:root.nodeId,selector:'#weights'}); + await call('DOM.setFileInputFiles',{nodeId,files:[resolve(process.argv[4])]}); + await evaluate('document.getElementById("open").requestSubmit()'); + let ready=false; + for(let i=0;i<400;i++){ + if(await evaluate('!document.getElementById("view").hidden')){ready=true;break;} + const text=await evaluate('document.getElementById("status").textContent');if(text.startsWith('Could not'))throw Error(text); + await new Promise(r=>setTimeout(r,100)); + } + if(!ready)throw Error('File picker did not finish generation'); + await evaluate('document.getElementById("view").click()'); + for(let i=0;i<100;i++){if(await evaluate('!!document.getElementById("projection-map")'))break;await new Promise(r=>setTimeout(r,100));} + const result=await evaluate(`(async()=>{ + for(let i=0;i<50&&document.getElementById('open-model').hidden;i++)await new Promise(r=>setTimeout(r,50)); + return {classes:data.names.length,names:data.names,checkpoint:data.metadata.checkpoint,projection:document.getElementById('projection-plane').value,canOpenAnother:!document.getElementById('open-model').hidden,saveLink:!!document.querySelector('a[download]')}; + })()`); + if(!result.canOpenAnother||!result.saveLink||result.classes<2||result.projection!=='Angular t-SNE')throw Error(JSON.stringify(result)); + const shot=await call('Page.captureScreenshot',{format:'png'});writeFileSync(join(output,'uploaded-model.png'),Buffer.from(shot.data,'base64')); + await evaluate('document.getElementById("open-model").click()'); + for(let i=0;i<50;i++){if(await evaluate('!!document.getElementById("weights")'))break;await new Promise(r=>setTimeout(r,100));} + if(!await evaluate('!!document.getElementById("weights")'))throw Error('Cannot return to file picker'); + if(errors.length)throw Error(JSON.stringify(errors)); + console.log(JSON.stringify({browserUpload:result,returnedToPicker:true,javascriptErrors:errors.length})); + await send('Browser.close'); +}catch(e){console.error(e);process.exitCode=1;chrome.kill('SIGTERM');}finally{clearTimeout(deadline);setTimeout(()=>rmSync(profile,{recursive:true,force:true}),1000);} diff --git a/dev/prototype_space/check_layout.mjs b/dev/prototype_space/check_layout.mjs new file mode 100644 index 00000000..63d14b03 --- /dev/null +++ b/dev/prototype_space/check_layout.mjs @@ -0,0 +1,121 @@ +// Dependency-free numerical checks for the browser's pure layout functions. +import assert from 'node:assert/strict'; +import {readFileSync} from 'node:fs'; +const source=readFileSync(new URL('./thumbnails.js',import.meta.url),'utf8'); +const {thumbnailEnergy,stepThumbnailForce,stepThumbnailPresentation}=Function(source.slice(source.indexOf('function thumbnailSharedArea'),source.indexOf('let mapThumbMotion'))+'return {thumbnailEnergy,stepThumbnailForce,stepThumbnailPresentation};')(); +const box=(id,x,y,width=64,height=64)=>({id,x,y,anchorX:x,anchorY:y,width,height}); +const gradientCase=[box(0,100,100,72,60),box(1,149,127,64,80),box(2,210,150,50,64)]; +gradientCase[0].anchorX-=12; +const gradient=thumbnailEnergy(gradientCase,true).forces; +for(let i=0;ibox(i,200+i%6*50,180+Math.floor(i/6)*50)),width:900,height:700}, + {name:'impossible two-box viewport',boxes:[box(0,32,32),box(1,32,32)],width:64,height:64} +]){ + const state={},ids=fixture.boxes.map(b=>b.id),anchors=fixture.boxes.map(b=>[b.anchorX,b.anchorY]); + let previous=thumbnailEnergy(fixture.boxes).energy; + while(!state.done){ + stepThumbnailForce(fixture.boxes,fixture.width,fixture.height,128,state); + assert.ok(state.metrics.energy<=previous+1e-8,'accepted steps must not increase energy');previous=state.metrics.energy; + assert.deepEqual(fixture.boxes.map(b=>b.id),ids,'solver must preserve every admitted thumbnail'); + for(const b of fixture.boxes){assert.ok(b.left>=-1e-8&&b.top>=-1e-8&&b.left+b.width<=fixture.width+1e-8&&b.top+b.height<=fixture.height+1e-8);assert.ok(Math.hypot(b.x-b.anchorX,b.y-b.anchorY)<=128+1e-8);} + } + assert.deepEqual(fixture.boxes.map(b=>[b.anchorX,b.anchorY]),anchors); + assert.equal(state.reason,'resting');assert.ok(state.quiet>=8); + if(fixture.name.startsWith('feasible'))assert.equal(state.metrics.pairs,0); + if(fixture.name.startsWith('impossible'))assert.equal(state.metrics.pairs,1); + results.push({fixture:fixture.name,steps:state.steps,overlappingPairs:state.metrics.pairs}); +} +console.log(JSON.stringify({gradientChecks:6,fixtures:results})); + +// Measure the complete visible trajectory, including startup, a target reversal, +// and the deceleration tail, using real seconds at different display rates. +const trajectories=[]; +for(const hz of [30,60,120]){ + const boxes=[box(0,100,100)],target=[box(0,228,100)]; + let previousX=100,previousVelocity=0,peakSpeed=0,peakAcceleration=0,firstStep=0,result; + const samples=[]; + for(let frame=1;frame<=hz*5;frame++){ + if(frame===hz+1)target[0].x=68; + result=stepThumbnailPresentation(boxes,target,1/hz); + const velocity=(boxes[0].x-previousX)*hz; + peakSpeed=Math.max(peakSpeed,Math.abs(velocity)); + peakAcceleration=Math.max(peakAcceleration,Math.abs(velocity-previousVelocity)*hz); + if(frame===1)firstStep=boxes[0].x-100; + assert.ok(boxes[0].x>=68&&boxes[0].x<=228,'display remains inside feasible convex region'); + previousX=boxes[0].x;previousVelocity=velocity; + if(frame%(hz/10)===0)samples.push(boxes[0].x); + } + assert.ok(peakSpeed<=140.01,'visible speed bounded in CSS pixels per second'); + assert.ok(peakAcceleration<2240,'acceleration bounded even at target reversal'); + assert.ok(firstStep<.6,'startup must ease in instead of jumping a solver step'); + assert.ok(result.done&&result.speed<.02,'motion stops only after the slow tail'); + trajectories.push({hz,peakSpeed,peakAcceleration,firstStep,samples}); +} +for(let i=1;iMath.abs(x-trajectories[0].samples[j])<1e-8),'same elapsed time yields same presentation across refresh rates'); +console.log(JSON.stringify({trajectories:trajectories.map(({samples,...metrics})=>metrics)})); + +// Drive the real animation coordinator with deterministic browser frames. An +// arriving image must not cancel a frame, rewind destinations, or pause others. +let queued=null,clock=0,cancellations=0; +const coordinator=Function('requestAnimationFrame','cancelAnimationFrame','matchMedia', + source.slice(0,source.indexOf('function thumbnailFootprint'))+` + paintThumbnailMotion=()=>{}; + return {set:boxes=>{mapThumbLayout=boxes;},start:animateThumbnails, + motion:()=>mapThumbMotion,stop:stopThumbnailMotion}; +`)(callback=>{queued=callback;return 1;},()=>{queued=null;cancellations++;},()=>({matches:false})); +const actors=[box(0,200,200),box(1,220,200)];coordinator.set(actors); +const config={width:600,height:400,size:64}; +const running=coordinator.start(config); +const frame=()=>{clock+=1000/60;const callback=queued;queued=null;callback(clock);}; +for(let i=0;i<6;i++)frame(); +const original=coordinator.motion(),destination=original.byId.get(0),lastTime=original.last; +const newcomer=box(2,500,300);newcomer.readyAt=clock+100;actors.unshift(newcomer); +assert.equal(coordinator.start(config),running,'arrivals share the running completion promise'); +assert.equal(coordinator.motion(),original,'arrival preserves the animation loop'); +assert.equal(original.last,lastTime,'arrival does not reset frame time'); +const beforeArrival=actors[1].x,steps=original.state.steps; +for(let i=0;i<4;i++)frame(); +assert.equal(cancellations,0,'image appearance must not cancel frames'); +assert.equal(original.byId.get(0),destination,'existing numerical targets survive arrival and reordering'); +assert.ok(original.state.steps>steps,'existing contacts advance while a newcomer waits'); +assert.ok(Math.abs(actors[1].x-beforeArrival)>.5,'existing image keeps moving during appearance delay'); +assert.equal(newcomer.x,500,'new image stays still until its own appearance delay'); +assert.ok(!original.byId.has(2),'waiting image exerts no collision force'); +for(let i=0;i<4;i++)frame(); +assert.ok(original.byId.has(2),'new image joins the same solver after appearing'); +assert.equal(original.byId.get(0),destination,'joining contacts preserve existing destinations'); +for(let i=0;i<600&&coordinator.motion();i++)frame(); +assert.equal(coordinator.motion(),null,'joined animation eventually settles'); +await running; +console.log(JSON.stringify({arrivalContinuityChecks:12})); + +const template=readFileSync(new URL('./report.html',import.meta.url),'utf8'); +const {referenceLogTail,probabilityText}=Function(template.slice(template.indexOf('function referenceLogTail'),template.indexOf('function scoreText'))+'return {referenceLogTail,probabilityText};')(); +const probabilityReference=[[-56.0, -0.0], [-12.0, -1.776482112077648e-33], [-8.0, -6.220960574271758e-16], [-1.0, -0.17275377902344985], [0.0, -0.6931471805599453], [1.0, -1.841021645009264], [8.0, -35.01343715991456], [12.0, -75.41067300156881], [31.0, -484.8539636271794], [56.0, -1572.9446088476825]]; +for(const [z,logTail] of probabilityReference)assert.ok(Math.abs(referenceLogTail(z)-logTail)<3e-7,'direct browser log tail agrees with independent torch float64 reference'); +for(let z=-56;z<56;z+=.1)assert.ok(referenceLogTail(z+.1)<=referenceLogTail(z),'chance-alignment tail decreases with alignment'); +assert.notEqual(probabilityText(referenceLogTail(56)/Math.LN10),'0','extreme probability remains readable without underflow'); +assert.ok(probabilityText(Math.log10(.999999)).startsWith('1 −'),'near-one values show their complement'); +console.log(JSON.stringify({logTailReferenceChecks:probabilityReference.length})); + +const photoSource=readFileSync(new URL('./photos.js',import.meta.url),'utf8').split('function photoText')[0]; +const completed=[];let metadataCalls=0; +const albumLoader=Function('fetch',photoSource+'return {referenceAlbum};')(async url=>{ + metadataCalls++;await new Promise(resolve=>completed.push(resolve)); + return {ok:true,json:async()=>({class_id:url.split('/').pop(),photos:[]})}; +}); +const activeSignal=new AbortController(),obsoleteSignal=new AbortController(); +const inFlight=Array.from({length:4},(_,i)=>albumLoader.referenceAlbum(String(i),activeSignal.signal)); +const queuedMetadata=albumLoader.referenceAlbum('obsolete',obsoleteSignal.signal).catch(error=>error.name); +obsoleteSignal.abort();completed.splice(0).forEach(resolve=>resolve());await Promise.all(inFlight); +assert.equal(await queuedMetadata,'AbortError');assert.equal(metadataCalls,4,'obsolete queued metadata is never requested'); +console.log(JSON.stringify({queuedMetadataCancellationChecks:2})); diff --git a/dev/prototype_space/check_mobile.mjs b/dev/prototype_space/check_mobile.mjs new file mode 100644 index 00000000..00a23973 --- /dev/null +++ b/dev/prototype_space/check_mobile.mjs @@ -0,0 +1,64 @@ +// Focused image ingestion and multi-touch regression; camera hardware still needs manual review. +import {spawn} from 'node:child_process'; +import {mkdtempSync,writeFileSync,readFileSync,rmSync} from 'node:fs'; +if(!process.env.CHROME_PATH||!process.argv[2]||!process.argv[3])throw Error('Usage: CHROME_PATH=/path/to/chrome node check_mobile.mjs URL LARGE_JPEG [OUTPUT_JSON]'); +const profile=mkdtempSync('/tmp/browser-mobile-check-'); +const chrome=spawn(process.env.CHROME_PATH,['--headless','--no-sandbox','--disable-gpu','--disable-dev-shm-usage','--no-first-run','--disable-background-networking','--remote-debugging-pipe','--user-data-dir='+profile],{stdio:['ignore','ignore','ignore','pipe','pipe']}); +let serial=0,buffer=Buffer.alloc(0);const pending=new Map(),errors=[]; +chrome.stdio[4].on('data',chunk=>{buffer=Buffer.concat([buffer,chunk]);let end;while((end=buffer.indexOf(0))!==-1){const msg=JSON.parse(buffer.subarray(0,end));buffer=buffer.subarray(end+1);if(msg.id){const p=pending.get(msg.id);pending.delete(msg.id);msg.error?p.reject(Error(JSON.stringify(msg.error))):p.resolve(msg.result);}else if(msg.method==='Runtime.exceptionThrown')errors.push(msg.params.exceptionDetails);}}); +function send(method,params={},sessionId){const id=++serial;return new Promise((resolve,reject)=>{pending.set(id,{resolve,reject});chrome.stdio[3].write(JSON.stringify({id,method,params,...(sessionId?{sessionId}:{})})+'\0');});} +const timeout=setTimeout(()=>{chrome.kill();process.exitCode=1;console.error('Timeout');},60000); +try{ + const {targetId}=await send('Target.createTarget',{url:'about:blank'});const {sessionId}=await send('Target.attachToTarget',{targetId,flatten:true});const call=(m,p)=>send(m,p,sessionId); + await call('Runtime.enable');await call('Page.enable');await call('Emulation.setDeviceMetricsOverride',{width:390,height:844,deviceScaleFactor:1,mobile:true}); + await call('Emulation.setTouchEmulationEnabled',{enabled:true,maxTouchPoints:5});await call('Page.navigate',{url:process.argv[2]}); + const evaluate=async expression=>{const r=await call('Runtime.evaluate',{expression,returnByValue:true,awaitPromise:true});if(r.exceptionDetails)throw Error(JSON.stringify(r.exceptionDetails));return r.result.value;}; + for(let i=0;i<150;i++){if(await evaluate('typeof captureViewerState==="function"'))break;await new Promise(r=>setTimeout(r,100));} + if(!await evaluate('typeof prepareInferenceImage==="function"'))throw Error('Viewer did not load'); + const checks=await evaluate(`(async()=>{ + const checks=[],check=(v,name)=>{if(!v)throw Error(name);checks.push(name)}; + const canvas=document.createElement('canvas');canvas.width=80;canvas.height=40;const ctx=canvas.getContext('2d'); + for(let y=0;y<40;y++)for(let x=0;x<80;x++){ctx.fillStyle='rgb('+x*3+','+y*6+','+(x+y)%256+')';ctx.fillRect(x,y,1,1);} + const blob=await new Promise(r=>canvas.toBlob(r,'image/jpeg',1));const raw=new Uint8Array(await blob.arrayBuffer()); + const original=await createImageBitmap(blob);ctx.drawImage(original,0,0);original.close();const pixels=ctx.getImageData(0,0,80,40); + const m={preprocessing:'nearest-square-uint8-bilinear-center-imagenet-v1',size:16,resize:18}; + const sampled=await prepareInferenceImage(blob,16),before=browserPreprocess(pixels,m),after=browserPreprocess(sampled,m); + check(before.every((v,i)=>v===after[i]),'opaque JPEG preserves model input exactly'); + const mappings=[null,(x,y)=>[x,y],(x,y)=>[79-x,y],(x,y)=>[79-x,39-y],(x,y)=>[x,39-y],(x,y)=>[y,x],(x,y)=>[y,39-x],(x,y)=>[79-y,39-x],(x,y)=>[79-y,x]]; + for(let orientation=1;orientation<=8;orientation++){ + const exif=new Uint8Array([255,225,0,34,69,120,105,102,0,0,73,73,42,0,8,0,0,0,1,0,18,1,3,0,1,0,0,0,orientation,0,0,0,0,0,0,0]); + const jpeg=new Blob([raw.slice(0,2),exif,raw.slice(2)],{type:'image/jpeg'}),image=await prepareInferenceImage(jpeg,8); + let same=true;for(let y=0;y<8;y++)for(let x=0;x<8;x++){ + const [sx,sy]=mappings[orientation](x*(orientation>=5?40:80)/8,y*(orientation>=5?80:40)/8); + for(let c=0;c<3;c++)if(image.data[4*(y*8+x)+c]!==pixels.data[4*(sy*80+sx)+c])same=false; + } + check(same,'EXIF orientation '+orientation); + } + canvas.width=canvas.height=8;ctx.clearRect(0,0,8,8);const png=await new Promise(r=>canvas.toBlob(r));const white=await prepareInferenceImage(png,8); + check(white.data.every(v=>v===255),'transparent PNG composited on white'); + const webp=await new Promise(r=>canvas.toBlob(r,'image/webp'));check((await prepareInferenceImage(webp,8)).width===8,'WebP accepted'); + const bitmap=window.createImageBitmap;try{window.createImageBitmap=async()=>{throw Error('simulate decoder gap')};check((await prepareInferenceImage(blob,8)).width===8,'native image decoder fallback');}finally{window.createImageBitmap=bitmap;} + let error='';try{await prepareInferenceImage(new File(['invalid'],'example.heic',{type:'image/heic'}),8);}catch(e){error=e.message;} + check(error.includes('HEIC/HEIF')&&error.includes('JPEG'),'unsupported format guidance'); + check($('inference-camera').getAttribute('capture')==='environment'&&$('inference-camera').onchange===$('inference-image').onchange,'camera and gallery share inference path'); + setInferenceInputDisabled(false);let opened=false;const click=$('inference-camera').click;$('inference-camera').click=()=>{opened=true};$('inference-camera-button').click();$('inference-camera').click=click; + check(opened,'camera button activates capture input');setInferenceInputDisabled(true); + return checks; + })()`); + const large=readFileSync(process.argv[3]).toString('base64'); + const largeResult=await evaluate(`(async()=>{const blob=await (await fetch('data:image/jpeg;base64,${large}')).blob();const source=await createImageBitmap(blob);const megapixels=source.width*source.height/1e6;source.close();if(megapixels<=40)throw Error('Supply a JPEG larger than 40 MP');const start=performance.now();const image=await prepareInferenceImage(blob,384);return {megapixels,width:image.width,height:image.height,ms:performance.now()-start};})()`); + if(largeResult.width!==384||largeResult.height!==384)throw Error('Large image sampling failed');checks.push('42 MP JPEG sampled to model dimensions'); + await evaluate('setWorkspaceView("projection");window.scrollTo(0,0)');await new Promise(r=>setTimeout(r,300)); + const start=await evaluate('({view:{...projectionView},selected,rect:projectionCanvas.getBoundingClientRect().toJSON(),coordinates:JSON.stringify(currentProjection().coordinates)})'); + const cx=start.rect.x+start.rect.width/2,cy=start.rect.y+Math.min(start.rect.height/2,300); + const touch=async(type,points)=>call('Input.dispatchTouchEvent',{type,touchPoints:points.map(([id,x,y])=>({id,x,y,radiusX:4,radiusY:4,force:1}))}); + await touch('touchStart',[[1,cx-35,cy],[2,cx+35,cy]]);await touch('touchMove',[[1,cx-70,cy],[2,cx+70,cy]]);await touch('touchEnd',[]); + const zoomed=await evaluate('({view:{...projectionView},selected,coordinates:JSON.stringify(currentProjection().coordinates),pointers:projectionPointers.size})'); + if(Math.abs(zoomed.view.scale/start.view.scale-2)>0.03||zoomed.selected!==start.selected||zoomed.coordinates!==start.coordinates||zoomed.pointers!==0)throw Error('Pinch invariants failed: '+JSON.stringify(zoomed.view)); + checks.push('real touch pinch doubles scale without selection or geometry changes'); + await touch('touchStart',[[1,cx,cy]]);await touch('touchMove',[[1,cx+20,cy+15]]);await touch('touchEnd',[]); + const panned=await evaluate('({...projectionView})');if(panned.x===zoomed.view.x||panned.scale!==zoomed.view.scale)throw Error('Pan after pinch failed');checks.push('single finger pan after pinch'); + await touch('touchStart',[[1,cx,cy],[2,cx+40,cy]]);await touch('touchCancel',[]); + if(await evaluate('projectionPointers.size!==0||projectionDrag!==null'))throw Error('Cancelled gesture retained pointers');checks.push('touch cancellation clears gesture'); + if(errors.length)throw Error(JSON.stringify(errors));const result={checks,largeResult,errors};if(process.argv[4])writeFileSync(process.argv[4],JSON.stringify(result,null,2));console.log(JSON.stringify(result));await send('Browser.close'); +}catch(error){console.error(error);chrome.kill();process.exitCode=1;}finally{clearTimeout(timeout);chrome.once('exit',()=>rmSync(profile,{recursive:true,force:true}));} diff --git a/dev/prototype_space/check_portable.mjs b/dev/prototype_space/check_portable.mjs new file mode 100644 index 00000000..8504803f --- /dev/null +++ b/dev/prototype_space/check_portable.mjs @@ -0,0 +1,78 @@ +// Optional browser smoke check. Set CHROMIUM_BIN; pass a report URL/path and output directory. +import {spawn} from 'node:child_process'; +import {writeFileSync,mkdtempSync,rmSync,mkdirSync,existsSync} from 'node:fs'; +import {tmpdir} from 'node:os'; +import {pathToFileURL} from 'node:url'; +import {resolve,join} from 'node:path'; +const executable=process.env.CHROMIUM_BIN; +if(!executable||!existsSync(executable)||!process.argv[2])throw Error('Set CHROMIUM_BIN and pass report URL/path [output directory]'); +const output=resolve(process.argv[3]||'tmp/prototype-browser-check');mkdirSync(output,{recursive:true}); +const profile=mkdtempSync(tmpdir()+'/prototype-browser-'); +const chrome=spawn(executable,['--headless','--no-sandbox','--disable-gpu','--disable-dev-shm-usage','--no-first-run','--no-default-browser-check','--disable-background-networking','--disable-component-update','--disable-extensions','--remote-debugging-pipe','--user-data-dir='+profile],{stdio:['ignore','ignore','ignore','pipe','pipe']}); +let sequence=0,buffer=Buffer.alloc(0);const pending=new Map();const errors=[]; +chrome.stdio[4].on('data',chunk=>{buffer=Buffer.concat([buffer,chunk]);let end;while((end=buffer.indexOf(0))!==-1){const msg=JSON.parse(buffer.subarray(0,end));buffer=buffer.subarray(end+1);if(msg.id){const cb=pending.get(msg.id);if(cb){pending.delete(msg.id);msg.error?cb.reject(Error(JSON.stringify(msg.error))):cb.resolve(msg.result);}}else if(msg.method==='Runtime.exceptionThrown')errors.push(msg.params.exceptionDetails);}}); +function send(method,params={},sessionId){const id=++sequence;return new Promise((resolve,reject)=>{pending.set(id,{resolve,reject});chrome.stdio[3].write(JSON.stringify({id,method,params,...(sessionId?{sessionId}:{})})+'\0');});} +const deadline=setTimeout(()=>{console.error('Browser smoke timed out');chrome.kill('SIGTERM');process.exitCode=1;},60000); +try{ + const {targetId}=await send('Target.createTarget',{url:'about:blank'});const {sessionId}=await send('Target.attachToTarget',{targetId,flatten:true}); + const call=(m,p)=>send(m,p,sessionId);await call('Page.enable');await call('Runtime.enable');await call('Emulation.setDeviceMetricsOverride',{width:1440,height:1100,deviceScaleFactor:1,mobile:false}); + await call('Page.navigate',{url:process.argv[2].startsWith('http')?process.argv[2]:pathToFileURL(process.argv[2]).href}); + let ready=false;for(let i=0;i<100;i++){const r=await call('Runtime.evaluate',{expression:'typeof captureViewerState === "function" ? document.querySelectorAll("#neighbours tr").length : 0',returnByValue:true});if(r.result.value>0){ready=true;break;}await new Promise(r=>setTimeout(r,100));} + if(!ready)throw Error('Inspector did not render: '+JSON.stringify(errors)); + const result=await call('Runtime.evaluate',{expression:`(async()=>{ + const checks=[];const check=(v,s)=>{if(!v)throw Error(s);checks.push(s)}; + check(typeof BrowserGBIF==='function','browser client available'); + check(!document.getElementById('gbif-names'),'one namespace setting'); + check(document.body.dataset.view==='projection','explore default'); + document.getElementById('activity-predict').click(); + check(!document.getElementById('inference-panel').hidden,'predict navigation'); + const state=captureViewerState();check(state.version===2,'versioned state'); + document.getElementById('activity-explore').click(); + check(document.body.dataset.view==='projection','explore navigation'); + let terminated=false;inferenceWorker={terminate(){terminated=true;}};resetInferenceModel();check(terminated&&inferenceManifest===null&&$('inference-image').disabled,'model reset cancels stale inference'); + const id=data.names[0];let requests=0; + gbifClient.fetcher=async url=>{requests++;const taxon=url.includes('occurrence/search')?new URL(url).searchParams.get('taxonKey'):url.split('/').pop();return {ok:true,json:async()=>url.includes('occurrence/search')?{results:[{key:1,speciesKey:Number(taxon),media:[{type:'StillImage',identifier:location.origin+'/fixture.jpg',creator:'Fixture author',license:'CC BY'}]}]}:{key:Number(taxon),canonicalName:'Example '+taxon}};}; + $('class-id-namespace').value='gbif';$('class-id-namespace').onchange(); + check((await gbifClient.name(id)).display_name==='Example '+id,'browser names'); + const album=await referenceAlbum(id,new AbortController().signal); + check(album.photos[0].creator==='Fixture author','browser photo attribution'); + const legacy=captureViewerState();legacy.version=1;delete legacy.controls['class-id-namespace'];legacy.controls['gbif-names']=true; + applyViewerState(legacy);check($('class-id-namespace').value==='gbif','legacy namespace migration'); + await new Promise(resolve=>requestAnimationFrame(()=>requestAnimationFrame(resolve))); + let rejected=false;try{applyViewerState({...captureViewerState(),version:1,controls:null});}catch{rejected=true;}check(rejected,'malformed legacy state rejected'); + const before={...projectionView};const classes=[data.names,...[1,2].map(level=>[...new Set(data.groups.map(group=>group[level]))])]; + lastPrediction={result:{outputs:classes.map(names=>names.map((_,i)=>i===0?10:0))},m:{classes}};renderInferenceResults(); + await new Promise(resolve=>setTimeout(resolve,300)); + check($('inference-results').querySelectorAll('figure').length===5,'species prediction thumbnails'); + check($('inference-results').textContent.includes('Fixture author'),'prediction credits'); + check(JSON.stringify(before)===JSON.stringify(projectionView),'prediction labels preserve map transform'); + $('class-id-namespace').value='generic';$('class-id-namespace').onchange(); + const count=requests;await new Promise(resolve=>setTimeout(resolve,300)); + check(requests===count,'generic mode prevents new GBIF requests'); + check(!$('inference-results').querySelector('figure'),'generic prediction fallback'); + $('class-id-namespace').value='gbif';$('class-id-namespace').onchange();setWorkspaceView('predict'); + await new Promise(resolve=>setTimeout(resolve,300)); + return checks; + })()`,returnByValue:true,awaitPromise:true}); + if(result.exceptionDetails)throw Error(JSON.stringify({exception:result.exceptionDetails,errors})); + for(const [name,width,height] of [['desktop',1440,1000],['mobile',390,844]]){ + await call('Emulation.setDeviceMetricsOverride',{width,height,deviceScaleFactor:1,mobile:false}); + await new Promise(resolve=>setTimeout(resolve,500)); + const shot=await call('Page.captureScreenshot',{format:'png'});writeFileSync(join(output,name+'.png'),Buffer.from(shot.data,'base64')); + } + if(process.argv.includes('--live')){ + const live=await call('Runtime.evaluate',{expression:`(async()=>{ + const client=new BrowserGBIF();const name=await client.name('1775152');const album=await client.album('1775152'); + if(!album.photos.length)throw Error('No live reference images'); + const image=document.createElement('img');document.body.append(image); + let loaded=false;for(const photo of album.photos){if(await loadReferenceImage(image,photo.image_path,new AbortController().signal)){loaded=true;break;}} + image.remove();if(!loaded)throw Error('Live reference images unavailable'); + return {name:name.display_name,photos:album.photos.length,imageLoaded:loaded}; + })()`,awaitPromise:true,returnByValue:true}); + if(live.exceptionDetails)throw Error(JSON.stringify(live.exceptionDetails)); + console.log(JSON.stringify({liveGBIF:live.result.value})); + } + if(errors.length)throw Error(JSON.stringify(errors)); + const report={checks:result.result.value,errors:errors.length};writeFileSync(join(output,'portable-check.json'),JSON.stringify(report,null,2));console.log(JSON.stringify(report)); + await send('Browser.close'); +}catch(e){console.error(e);process.exitCode=1;chrome.kill('SIGTERM');}finally{clearTimeout(deadline);setTimeout(()=>rmSync(profile,{recursive:true,force:true}),1000);} diff --git a/dev/prototype_space/check_session_browser.mjs b/dev/prototype_space/check_session_browser.mjs new file mode 100644 index 00000000..345ad805 --- /dev/null +++ b/dev/prototype_space/check_session_browser.mjs @@ -0,0 +1,46 @@ +// Optional browser smoke check. Set CHROMIUM_BIN; pass a report URL/path and output directory. +import {spawn} from 'node:child_process'; +import {writeFileSync,mkdtempSync,rmSync,mkdirSync,existsSync} from 'node:fs'; +import {tmpdir} from 'node:os'; +import {pathToFileURL} from 'node:url'; +import {resolve,join} from 'node:path'; +const executable=process.env.CHROMIUM_BIN; +if(!executable||!existsSync(executable)||!process.argv[2])throw Error('Set CHROMIUM_BIN and pass report URL/path [output directory]'); +const output=resolve(process.argv[3]||'tmp/prototype-browser-check');mkdirSync(output,{recursive:true}); +const profile=mkdtempSync(tmpdir()+'/prototype-browser-'); +const chrome=spawn(executable,['--headless','--no-sandbox','--disable-gpu','--disable-dev-shm-usage','--no-first-run','--no-default-browser-check','--disable-background-networking','--disable-component-update','--disable-extensions','--remote-debugging-pipe','--user-data-dir='+profile],{stdio:['ignore','ignore','ignore','pipe','pipe']}); +let sequence=0,buffer=Buffer.alloc(0);const pending=new Map();const errors=[]; +chrome.stdio[4].on('data',chunk=>{buffer=Buffer.concat([buffer,chunk]);let end;while((end=buffer.indexOf(0))!==-1){const msg=JSON.parse(buffer.subarray(0,end));buffer=buffer.subarray(end+1);if(msg.id){const cb=pending.get(msg.id);if(cb){pending.delete(msg.id);msg.error?cb.reject(Error(JSON.stringify(msg.error))):cb.resolve(msg.result);}}else if(msg.method==='Runtime.exceptionThrown')errors.push(msg.params.exceptionDetails);}}); +function send(method,params={},sessionId){const id=++sequence;return new Promise((resolve,reject)=>{pending.set(id,{resolve,reject});chrome.stdio[3].write(JSON.stringify({id,method,params,...(sessionId?{sessionId}:{})})+'\0');});} +const deadline=setTimeout(()=>{console.error('Browser smoke timed out');chrome.kill('SIGTERM');process.exitCode=1;},60000); +try{ + const {targetId}=await send('Target.createTarget',{url:'about:blank'});const {sessionId}=await send('Target.attachToTarget',{targetId,flatten:true}); + const call=(m,p)=>send(m,p,sessionId);await call('Page.enable');await call('Runtime.enable');await call('Emulation.setDeviceMetricsOverride',{width:1440,height:1100,deviceScaleFactor:1,mobile:false}); + await call('Page.navigate',{url:process.argv[2]}); + let value; + for(let i=0;i<100;i++){ + const result=await call('Runtime.evaluate',{expression:`typeof data!=='undefined'&&document.querySelectorAll('#neighbours tr').length?{classes:data.names.length,checkpoint:data.metadata.checkpoint,sha:data.metadata.checkpoint_sha256,points:projectionHits.length,projection:document.getElementById('projection-plane').value}:null`,returnByValue:true}); + value=result.result.value;if(value)break;await new Promise(r=>setTimeout(r,100)); + } + if(!value||value.classes!==12632||value.points!==12632||!value.checkpoint.includes('epoch26')||value.projection!=='Angular t-SNE'||errors.length)throw Error(JSON.stringify({value,errors})); + const persisted=await call('Runtime.evaluate',{expression:`(async()=>{ + for(let i=0;i<100&&!viewerServerToken;i++)await new Promise(r=>setTimeout(r,50)); + if(!viewerServerToken)throw Error('Server state API not available'); + setWorkspaceView('projection');selectClass(1);await new Promise(r=>setTimeout(r,100)); + projectionView.scale*=1.25;drawProjection();saveViewerState();await viewerStateWrites; + const remote=await (await fetch('/api/view-state?key='+encodeURIComponent(viewerDocumentKey))).json(); + if(remote.state.view.selected!==data.names[1])throw Error('State not persisted by local server'); + localStorage.clear();restoringViewerState=true; + })()`,awaitPromise:true}); + if(persisted.exceptionDetails)throw Error(JSON.stringify(persisted.exceptionDetails)); + await call('Page.reload'); + let restored=false; + for(let i=0;i<100;i++){ + const result=await call('Runtime.evaluate',{expression:`typeof viewerServerToken!=='undefined'&&viewerServerToken&&selected===1&&document.body.dataset.view==='projection'`,returnByValue:true}); + if(result.result.value){restored=true;break;}await new Promise(r=>setTimeout(r,100)); + } + if(!restored||errors.length)throw Error('Server-backed reload did not restore state: '+JSON.stringify(errors)); + console.log('Server-backed state restored after clearing browser storage'); + const shot=await call('Page.captureScreenshot',{format:'png'});writeFileSync(join(output,'epoch26.png'),Buffer.from(shot.data,'base64')); + console.log(JSON.stringify({value,javascriptErrors:errors.length}));await send('Browser.close'); +}catch(e){console.error(e);process.exitCode=1;chrome.kill('SIGTERM');}finally{clearTimeout(deadline);setTimeout(()=>rmSync(profile,{recursive:true,force:true}),1000);} diff --git a/dev/prototype_space/explore.py b/dev/prototype_space/explore.py new file mode 100644 index 00000000..3d44425b --- /dev/null +++ b/dev/prototype_space/explore.py @@ -0,0 +1,7 @@ +"""Compatibility entry point for the packaged prototype explorer.""" + +from mini_trainer.visualization.prototype_space.explore import * # noqa: F403 +from mini_trainer.visualization.prototype_space.explore import main + +if __name__ == "__main__": + main() diff --git a/dev/prototype_space/measure_interaction.mjs b/dev/prototype_space/measure_interaction.mjs new file mode 100644 index 00000000..a2aee0cc --- /dev/null +++ b/dev/prototype_space/measure_interaction.mjs @@ -0,0 +1,57 @@ +// Optional browser smoke check. Set CHROMIUM_BIN; pass a report URL/path and output directory. +import {spawn} from 'node:child_process'; +import {writeFileSync,mkdtempSync,rmSync,mkdirSync,existsSync} from 'node:fs'; +import {tmpdir} from 'node:os'; +import {pathToFileURL} from 'node:url'; +import {resolve,join} from 'node:path'; +const executable=process.env.CHROMIUM_BIN; +if(!executable||!existsSync(executable)||!process.argv[2])throw Error('Set CHROMIUM_BIN and pass report URL/path [output directory]'); +const output=resolve(process.argv[3]||'tmp/prototype-browser-check');mkdirSync(output,{recursive:true}); +const profile=mkdtempSync(tmpdir()+'/prototype-browser-'); +const chrome=spawn(executable,['--headless','--no-sandbox','--disable-gpu','--disable-dev-shm-usage','--no-first-run','--no-default-browser-check','--disable-background-networking','--disable-component-update','--disable-extensions','--remote-debugging-pipe','--user-data-dir='+profile],{stdio:['ignore','ignore','ignore','pipe','pipe']}); +let sequence=0,buffer=Buffer.alloc(0);const pending=new Map();const errors=[]; +chrome.stdio[4].on('data',chunk=>{buffer=Buffer.concat([buffer,chunk]);let end;while((end=buffer.indexOf(0))!==-1){const msg=JSON.parse(buffer.subarray(0,end));buffer=buffer.subarray(end+1);if(msg.id){const cb=pending.get(msg.id);if(cb){pending.delete(msg.id);msg.error?cb.reject(Error(JSON.stringify(msg.error))):cb.resolve(msg.result);}}else if(msg.method==='Runtime.exceptionThrown')errors.push(msg.params.exceptionDetails);}}); +function send(method,params={},sessionId){const id=++sequence;return new Promise((resolve,reject)=>{pending.set(id,{resolve,reject});chrome.stdio[3].write(JSON.stringify({id,method,params,...(sessionId?{sessionId}:{})})+'\0');});} +const deadline=setTimeout(()=>{console.error('Browser smoke timed out');chrome.kill('SIGTERM');process.exitCode=1;},60000); +try{ + const {targetId}=await send('Target.createTarget',{url:'about:blank'});const {sessionId}=await send('Target.attachToTarget',{targetId,flatten:true}); + const call=(m,p)=>send(m,p,sessionId);await call('Page.enable');await call('Runtime.enable');await call('Emulation.setDeviceMetricsOverride',{width:1440,height:1100,deviceScaleFactor:1,mobile:false}); + await call('Page.navigate',{url:process.argv[2]}); + let value; + for(let i=0;i<100;i++){ + const result=await call('Runtime.evaluate',{expression:`typeof data!=='undefined'&&document.querySelectorAll('#neighbours tr').length?{classes:data.names.length,checkpoint:data.metadata.checkpoint,sha:data.metadata.checkpoint_sha256,points:projectionHits.length,projection:document.getElementById('projection-plane').value}:null`,returnByValue:true}); + value=result.result.value;if(value)break;await new Promise(r=>setTimeout(r,100)); + } + if(!value||value.classes!==12632||value.points!==12632||!value.checkpoint.includes('epoch26')||value.projection!=='Angular t-SNE'||errors.length)throw Error(JSON.stringify({value,errors})); + await call('Runtime.evaluate',{expression:`setWorkspaceView('projection')`});await new Promise(r=>setTimeout(r,300)); + const metrics=await call('Runtime.evaluate',{expression:`(async()=>{ + const samples=[],frameTimes=[];for(let frame=0;frame<20;frame++){ + await new Promise(requestAnimationFrame);projectionDrag={point:[600,300],view:{...projectionView},moved:true}; + const r=projectionCanvas.getBoundingClientRect(),start=performance.now(); + for(let i=0;i<30;i++)projectionCanvas.onpointermove({clientX:r.left+r.width/2+i,clientY:r.top+r.height/2}); + samples.push(performance.now()-start);projectionDrag=null;await new Promise(requestAnimationFrame);frameTimes.push(performance.now()-start); + } + projectionDrag=null;fitProjection(data.names.map((_,i)=>i));drawProjection(); + const originalFetch=window.fetch,pixel='data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7'; + let calls=0,aborts=0,first=null; + window.fetch=async(url,options={})=>{ + if(url==='/api/health')return {ok:true,json:async()=>({photo_api:true})}; + calls++;await new Promise((resolve,reject)=>{const timer=setTimeout(resolve,500);options.signal?.addEventListener('abort',()=>{clearTimeout(timer);aborts++;reject(new DOMException('Aborted','AbortError'));},{once:true});}); + return {ok:true,json:async()=>({class_id:String(url).split('/').pop(),display_name:'Controlled latency fixture',photos:[{image_path:pixel,creator:'Fixture',license:'Fixture',source:'https://example.org',occurrence_id:'1'}]})}; + }; + photoAlbums.clear();mapThumbImages.clear();mapThumbFailures.clear();$('map-photos').checked=true;$('map-photo-density').value='20';$('map-photo-push').checked=false; + const loadStart=performance.now();drawProjection(); + const observe=setInterval(()=>{if(first===null&&mapThumbLayout.length)first=performance.now()-loadStart;},10); + for(let i=0;i<3;i++){await new Promise(r=>setTimeout(r,250));projectionView.x+=.1;drawProjection();} + let finished=false; + for(let i=0;i<160;i++){await new Promise(r=>setTimeout(r,50));const counts=$('map-photo-status').textContent.match(/([0-9]+) images loaded · ([0-9]+) visible slots/);if(counts&&Number(counts[1])===Number(counts[2])&&Number(counts[1])>0){finished=true;break;}} + clearInterval(observe);const loading={firstVisibleMs:first,elapsedMs:performance.now()-loadStart,visible:mapThumbLayout.length,calls,aborts,finished,fixture:'500 ms metadata; tiny local image; 3 navigation updates'}; + $('map-photos').checked=false;drawProjection();window.fetch=originalFetch; + samples.sort((a,b)=>a-b);return {loading,frameTimes,samples,median:samples[10],p95:samples[Math.ceil(samples.length*.95)-1],classes:data.names.length,browser:navigator.userAgent}; +})()`,returnByValue:true,awaitPromise:true}); + if(metrics.exceptionDetails)throw Error(JSON.stringify(metrics.exceptionDetails)); + writeFileSync(join(output,'performance.json'),JSON.stringify(metrics.result.value,null,2)); + console.log(JSON.stringify(metrics.result.value)); + const shot=await call('Page.captureScreenshot',{format:'png'});writeFileSync(join(output,'epoch26.png'),Buffer.from(shot.data,'base64')); + console.log(JSON.stringify({value,javascriptErrors:errors.length}));await send('Browser.close'); +}catch(e){console.error(e);process.exitCode=1;chrome.kill('SIGTERM');}finally{clearTimeout(deadline);setTimeout(()=>rmSync(profile,{recursive:true,force:true}),1000);} diff --git a/dev/prototype_space/photos.js b/dev/prototype_space/photos.js new file mode 120000 index 00000000..9cdaf372 --- /dev/null +++ b/dev/prototype_space/photos.js @@ -0,0 +1 @@ +../../mini_trainer/visualization/prototype_space/photos.js \ No newline at end of file diff --git a/dev/prototype_space/portable-qualification.md b/dev/prototype_space/portable-qualification.md new file mode 100644 index 00000000..ecf3e19d --- /dev/null +++ b/dev/prototype_space/portable-qualification.md @@ -0,0 +1,75 @@ +# Portable viewer qualification + +Updated 2026-09-11. This increment extends `feature/prototype-browser-inference`. +The numerical report, model bundle, preprocessing and fixed-map insertion algorithm +are retained. UI and GBIF transport changes do not retrain or re-export the model. + +## Evidence + +- Shared browser client checks cover deduplication, cache hits, concurrency, + cancellation, timeout, 429/offline/malformed responses, taxon identity, safe media + URLs, taxonomy membership and attribution. Snapshot names need no network call. +- Static Chromium checks pass for one global namespace, legacy state migration, + malformed legacy state rejection, prediction thumbnails and credits, generic-mode + fallback and unchanged map transforms during label refresh. +- The retained browser harness passed 158 assertions across numerical views, + photo interactions, landscape/portrait/mobile layouts and saved views. Its + fixtures now mock the shared client instead of the removed Python transport. +- Plain static hosting successfully resolved `1775152` as *Dichonia aprilina*, + returned six reference photos and loaded an image directly in the browser. + The API returned `Access-Control-Allow-Origin: *`. No Python taxonomy/photo + service was running. Local-launcher capability/state probes are still optional. +- Real-image browser inference returned 15 prediction rows and a 1,280-dimensional + embedding. Fixed-map placement remained `[51.14414261925462, -33.22137451588429]`, + with the same five of twelve angular neighbours retained. Insertion took about + 44–47 ms in these checks. This is functionality evidence, not broad accuracy or + browser qualification. +- Sixteen affected Python tests passed in total. The launcher socket test required + an unsandboxed rerun; the initial failure was a local socket permission error. + Static lint/format/import contracts passed. Installed-wheel smoke passed; an + additional archive inspection confirmed inclusion of `gbif.js`. + +A small same-machine Chromium sample measured first usable view at 2.91 seconds +for the preceding viewer and 2.96 seconds for this candidate; ten synchronous class +selection updates averaged 6.99 and 6.86 ms respectively. Neither initial load +issued GBIF requests. These single samples are a responsiveness check, not evidence +of a performance improvement or a cross-machine baseline. + +## Candidate and remaining review + +The implementation is recorded in `1eb3efe` and `c3c4134`; merge `b0bd05f` +includes current local master `268d294` (including the testing-economy guidance). + +The candidate includes a partial 100-taxon snapshot with retrieval dates. The +preparation command can resume to fill the remaining vocabulary. Images remain +external resources; missing/blocked images retain useful labels and scores. + +Human-reviewable desktop/mobile previews use an actual query image and live GBIF +reference images. Generic-mode and error checks use deterministic fixtures. +The existing public production release is not overwritten. Human layout review +and PR integration are separate gates; passing these checks does not imply either +has happened. Firefox, Safari, WebGPU and complete offline image distribution are +outside this increment. + +## Reproduce + +Run from the feature checkout with the existing environment and a plain static +server serving an exported report and matching `browser-model/` directory: + +```bash +node dev/prototype_space/check_gbif.cjs +CHROMIUM_BIN=/path/to/chrome node dev/prototype_space/check_portable.mjs \ + http://127.0.0.1:8767/explorer.html /tmp/portable-check --live +CHROMIUM_BIN=/path/to/chrome node dev/prototype_space/check_browser.mjs \ + http://127.0.0.1:8767/regression/explorer.html /tmp/explorer-regression +CHROME_PATH=/path/to/chrome node dev/prototype_space/check_inference.mjs \ + http://127.0.0.1:8767/explorer.html /path/to/query.jpg \ + /tmp/inference-check.json /tmp/prediction-preview +``` + +The general browser regression requires the production and synthetic cases made +by the existing exploration harness. The portable check needs `fixture.jpg` in its +static directory for mocked photo rendering; `--live` additionally tests GBIF. +The inference harness's last argument optionally captures desktop/mobile previews. +These are local development checks, not a requirement to repeat every suite after +subsequent documentation-only edits. diff --git a/dev/prototype_space/prepare_gbif.py b/dev/prototype_space/prepare_gbif.py new file mode 100644 index 00000000..e1d9a979 --- /dev/null +++ b/dev/prototype_space/prepare_gbif.py @@ -0,0 +1,68 @@ +"""Prepare a resumable, optional species-name snapshot for a static report.""" + +import argparse +import json +import time +from datetime import UTC, datetime +from pathlib import Path +from urllib.request import Request, urlopen + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--report", type=Path, required=True, help="Exported report-data.json") + parser.add_argument("--output", type=Path, required=True, help="Destination gbif-snapshot.json; resumes existing snapshots") + parser.add_argument("--limit", type=int, default=0, help="Maximum new lookups; zero means all missing IDs") + args = parser.parse_args() + payload = json.loads(args.report.read_text()) + ids = set() + for case in payload.values(): + if case["metadata"].get("synthetic"): + continue + ids.update(str(name) for name in case["names"]) + for group in case.get("groups", []): + ids.update(str(name) for name in group) + ids = sorted(name for name in ids if name.isdigit()) + snapshot = ( + json.loads(args.output.read_text()) + if args.output.exists() + else { + "schema": "mini-trainer-gbif-v1", + "source": "https://api.gbif.org/v1/species/", + "taxa": {}, + "retrieved_at": {}, + } + ) + if snapshot.get("schema") != "mini-trainer-gbif-v1": + raise ValueError("Unsupported snapshot schema") + args.output.parent.mkdir(parents=True, exist_ok=True) + missing = [name for name in ids if name not in snapshot["taxa"]] + if args.limit: + missing = missing[: args.limit] + print(f"{len(ids)} IDs; {len(missing)} new lookups; existing entries retained", flush=True) + for index, name in enumerate(missing): + try: + request = Request("https://api.gbif.org/v1/species/" + name, headers={"User-Agent": "mini-trainer-prototype-snapshot/1"}) + with urlopen(request, timeout=15) as response: + taxon = json.load(response) + if str(taxon.get("key")) != name: + raise ValueError("Returned taxon ID differs") + snapshot["taxa"][name] = { + key: taxon[key] + for key in ("key", "canonicalName", "scientificName", "acceptedKey", "rank", "taxonomicStatus") + if key in taxon + } + snapshot["retrieved_at"][name] = datetime.now(UTC).isoformat() + except Exception as error: + print(f"{name}: {error}; left missing for browser lookup or later retry", flush=True) + temporary = args.output.with_suffix(".partial") + temporary.write_text(json.dumps(snapshot, ensure_ascii=False) + "\n") + temporary.replace(args.output) + if (index + 1) % 50 == 0: + print(f"{index + 1}/{len(missing)} attempted; {len(snapshot['taxa'])} cached", flush=True) + time.sleep(0.1) + print(f"Snapshot: {args.output}; {len(snapshot['taxa'])} taxa", flush=True) + + +if __name__ == "__main__": + main() diff --git a/dev/prototype_space/projection.js b/dev/prototype_space/projection.js new file mode 120000 index 00000000..684b3164 --- /dev/null +++ b/dev/prototype_space/projection.js @@ -0,0 +1 @@ +../../mini_trainer/visualization/prototype_space/projection.js \ No newline at end of file diff --git a/dev/prototype_space/projection.py b/dev/prototype_space/projection.py new file mode 100644 index 00000000..aa9f8e4d --- /dev/null +++ b/dev/prototype_space/projection.py @@ -0,0 +1,3 @@ +"""Compatibility entry point for the packaged prototype explorer.""" + +from mini_trainer.visualization.prototype_space.projection import * # noqa: F403 diff --git a/dev/prototype_space/report.html b/dev/prototype_space/report.html new file mode 120000 index 00000000..2bb448af --- /dev/null +++ b/dev/prototype_space/report.html @@ -0,0 +1 @@ +../../mini_trainer/visualization/prototype_space/report.html \ No newline at end of file diff --git a/dev/prototype_space/reuse-performance.md b/dev/prototype_space/reuse-performance.md new file mode 100644 index 00000000..7fd1b30d --- /dev/null +++ b/dev/prototype_space/reuse-performance.md @@ -0,0 +1,87 @@ +# Reuse and responsiveness qualification + +2026-09-11, epoch-26 checkpoint recorded in the explorer roadmap. Baseline viewer +assets: `3907f30`. Host: Intel i7-12800H, WSL2 Linux 5.15.167.4, Python 3.13.7, +torch 2.12.0+cu130, NumPy 2.4.6, headless Chromium 151. CPU analysis used four +threads; no GPU-correctness claim is made. These are local measurements, not +universal performance guarantees. + +## Preparation and numerical preservation + +| Run | Wall time | Peak RSS | +| --- | ---: | ---: | +| Uncached baseline | 82.02 s | 5,424,156 KiB | +| First cache-populating analysis | 88.59 s | 5,428,248 KiB | +| Cache reopen | 3.98 s | 1,080,716 KiB | + +The independent cold runs have identical stored distance/log-tail arrays, class +ordering, tree, neighbourhoods, PCA and t-SNE coordinates. Differences are source +provenance and the t-SNE KL diagnostic scalar (1.9826414585 versus 1.9826412201). +Cached/reopened numerical JSON is byte-identical, SHA-256 +`a474f491471b16ccb3197831a23a1dff7c17726eb8a0cc20644197c29d860b7d`. +The cache deliberately retains the original analysis provenance. + +Reproduce with `/usr/bin/time -v`, using distinct output directories for the cold +and warm runs, the same cache directory and these environment settings: + +```bash +export PYTHONPATH="$PWD" +export MPLCONFIGDIR=/tmp/mini-trainer-prototype-mpl +export OMP_NUM_THREADS=4 OPENBLAS_NUM_THREADS=4 +/usr/bin/time -v .venv/bin/python -m mini_trainer.visualization.prototype_space \ + /absolute/path/to/best_global-lepi-production-w32-1_epoch26.pt \ + --export --cache-dir tmp/qualification-cache --output tmp/qualification-cold +# Repeat with --output tmp/qualification-warm; use --no-cache for an uncached run. +``` + +## Interaction and thumbnail arrivals + +`measure_interaction.mjs` runs the same real-coordinate workload against a frozen +baseline report and the updated report. It emits raw samples and browser identity +in `performance.json`. The pan workload sends 30 pointer moves per frame over 20 +frames at 1440 × 1100. Frame times include the subsequent animation-frame callback; +handler times alone are insufficient to establish responsiveness. + +The loading workload uses a fixed 500 ms metadata delay, a tiny local image, and +three navigation updates 250 ms apart. It isolates wasted metadata requests; it +does not measure GBIF network throughput or large-image decode time. + +| Measure | Before | After | +| --- | ---: | ---: | +| Frame median | 16.9 ms | 17.3 ms | +| Frame p95 | 83.7 ms | 18.1 ms | +| First visible thumbnail | 1.484 s | 0.714 s | +| All 31 visible thumbnails | 5.064 s | 4.253 s | +| Metadata requests / navigation aborts | 43 / 12 | 31 / 0 | + +The improvement is in frame-time outliers and avoiding repeated lookups during +navigation; the median frame interval did not improve. Started metadata requests +are shared across generations/cards, with at most four active. Queued obsolete +requests are discarded. Image loading still follows the visible candidate set, +and existing appearance, motion, attribution and culling checks remain applicable. +Local matrix recolouring now transforms only its selected submatrix rather than +all classes' local matrices on every refresh. + +```bash +CHROMIUM_BIN=/path/to/chrome node dev/prototype_space/measure_interaction.mjs \ + file:///absolute/path/to/explorer.html tmp/interaction-measurement +CHROMIUM_BIN=/path/to/chrome node dev/prototype_space/check_session_browser.mjs \ + http://localhost:PORT/explorer.html tmp/session-check +``` + +Serve the epoch-26 model with the launcher for the session check. That check saves +a view through the server, clears browser storage, suppresses page-hide saving, +reloads and verifies restoration. The Python session test separately verifies +reuse across new workspace instances with different tokens. + +## Validation scope + +Targeted launcher, extraction and photo tests passed (15 tests, including a targeted real-process cancellation check), plus 158 explorer +browser assertions, the real-checkpoint session check, motion/log-tail checks, +static checks and wheel validation. The broad suite was stopped at the user's +request; it is not reported as passed. Later lifecycle-only changes are checked +with launcher tests, not unrelated training suites. + +Timing artifacts live in ignored `tmp/reuse-{baseline,cold,warm}-epoch26` and +`tmp/interaction-{before,after}-complete`. Re-run on the same host with unchanged +workload before making further speed claims. Live GBIF latency remains variable. diff --git a/dev/prototype_space/serve.py b/dev/prototype_space/serve.py new file mode 100644 index 00000000..e8f86382 --- /dev/null +++ b/dev/prototype_space/serve.py @@ -0,0 +1,7 @@ +"""Compatibility entry point for the packaged prototype explorer.""" + +from mini_trainer.visualization.prototype_space.serve import * # noqa: F403 +from mini_trainer.visualization.prototype_space.serve import main + +if __name__ == "__main__": + main() diff --git a/dev/prototype_space/thumbnails.js b/dev/prototype_space/thumbnails.js new file mode 120000 index 00000000..3ebc6bf5 --- /dev/null +++ b/dev/prototype_space/thumbnails.js @@ -0,0 +1 @@ +../../mini_trainer/visualization/prototype_space/thumbnails.js \ No newline at end of file diff --git a/dev/ucloud/README.md b/dev/ucloud/README.md index 3d9cd5f4..bf336df8 100644 --- a/dev/ucloud/README.md +++ b/dev/ucloud/README.md @@ -1,5 +1,10 @@ # UCloud global_lepi training comparison +For findings from the completed production campaign and proposed next-run +improvements, see the [workflow post-mortem](../../docs/training-workflow-postmortem.md). +The commands below retain historical comparison profiles and pins; they are not +a newly qualified production recipe for every node. + Run inside **one allocated node**, with 1, 2, 4 or 8 visible GPUs. More than one GPU uses one `torchrun` process per GPU and the package's existing DDP trainer (including SyncBatchNorm). One GPU uses ordinary training. Do not invoke the diff --git a/dev/wheel_smoke.py b/dev/wheel_smoke.py index 43a319df..7e098b52 100644 --- a/dev/wheel_smoke.py +++ b/dev/wheel_smoke.py @@ -52,6 +52,25 @@ def main(): if entry.group == "console_scripts": subprocess.run([str(Path(sys.executable).parent / entry.name), "--help"], check=True, timeout=60, capture_output=True) + assets = importlib.resources.files("mini_trainer.visualization.prototype_space") + for name in ( + "report.html", + "launcher.html", + "photos.js", + "gbif.js", + "projection.js", + "thumbnails.js", + "state.js", + "inference.js", + "inference_worker.js", + "insertion.js", + ): + assert assets.joinpath(name).read_text(), f"Missing packaged explorer asset: {name}" + missing_explorer = subprocess.run( + [str(Path(sys.executable).parent / "mt_explore"), "--no-browser"], capture_output=True, text=True, timeout=30 + ) + assert missing_explorer.returncode != 0 and "mini_trainer[explorer]" in missing_explorer.stderr + root = Path.cwd() data = root / "images" for label, color in (("dark", (20, 30, 40)), ("light", (200, 210, 220))): diff --git a/docs/hyperspherical-visualization.md b/docs/hyperspherical-visualization.md new file mode 100644 index 00000000..d7c4f89d --- /dev/null +++ b/docs/hyperspherical-visualization.md @@ -0,0 +1,492 @@ +# Visualizing prototype packing on a unit hypersphere + +Research review, 2026-09-10. Scope: mathematical methods, Python implementations, +and browser rendering for the prototype explorer. This is a research deliverable; +methods described as candidates have not been added to training or the explorer. + +The unit sphere is the geometry in which we interpret the observed prototypes. +It does not prescribe their distribution, require clusters, imply uniformity, or +establish a lower-dimensional latent surface. Evaluate a display against the +actual weights and the existing mini_trainer diagnostics. Synthetic arrangements +are controlled tests of what a method reveals or distorts. + +## 1. Separate geometry, dimensional reduction, and rendering + +There are three separate decisions: + +| Layer | Question | Examples | +| --- | --- | --- | +| Source geometry | What relationships do the weights define? | Angular separation, directional neighbours, spherical caps | +| Reduction or slice | Which relationships will the display retain? | Anchor distances, angular t-SNE, spherical stress, a great-sphere slice | +| Rendering | How are the resulting coordinates shown? | Plane, rotatable globe, cartographic map, culled photo labels | + +An angular input metric does not imply spherical output. Conversely, placing a +layout on a globe does not establish that it preserves the original angles. +Both planar and spherical layouts can be useful when their losses are measured. + +For effective weight rows, let + +\[ +u_i=w_i/\|w_i\|,\quad c_{ij}=u_i^\top u_j,\quad +\theta_{ij}=\arccos(c_{ij}). +\] + +Cosine dissimilarity, chord distance, and angular distance are related by + +\[ +1-c_{ij}=2\sin^2(\theta_{ij}/2),\qquad +\|u_i-u_j\|=2\sin(\theta_{ij}/2). +\] + +These give identical exact neighbour orderings, including ties. They do **not** +give identical distance weights, kernels, stress objectives, or fitted layouts. +Cosine dissimilarity is also not generally a metric satisfying the triangle +inequality. Changing it to angular distance is more than changing axis labels. + +### Repository baseline + +The explorer loads effective weights through the existing classifier +parametrization and calls the repository distance functions. See +[the numerical contract](../dev/prototype_space/README.md#numerical-contract), +[distance.py](../mini_trainer/modeling/distance.py), and +[the transform](../mini_trainer/utils/_core/math.py). + +The current pre-CDF transform is, algebraically, + +\[ +z_{ij}=\sqrt{D-2}\,(\pi/2-\theta_{ij}^{\rm clamped}). +\] + +Thus `pi/2 - z/sqrt(D-2)` recovers the baseline's clamped angles without +inverting a saturated CDF. It does not recover distinctions already lost to the +cosine clamp. Self-distances are explicitly zeroed. Baseline class distance, +pre-CDF z, and log-tail displays remain separately identified. + +For probability diagnostics, retain direct log-domain evaluation and separate +colour clipping from numerical values. A projection cannot repair CDF saturation, +and a better colour window cannot repair projection distortion. + +### What the real checkpoint establishes + +Checkpoint: `best_global-lepi-production-w32-1_epoch4.pt`. +SHA256: `42d50f56cb0c6e3ee17ae64fbbcc4eb2ae24335ffe7080197c479c130fff5a81`. +It contains 12,632 prototype directions in 1,280 coordinates, hence on +\(S^{1279}\). The observed effective norms are within about \(2.4\times10^{-7}\) +of one; biases are zero. + +| Measurement | Observed value | Interpretation | +| --- | --- | --- | +| PC1–2 directional variance | 0.6784% | Two linear coordinates retain little total variation | +| PC1–2 mean top-12 neighbour recall | 0.6551% | This particular plane loses most original neighbours | +| Angular t-SNE mean top-12 recall | 48.6212% | A useful measured improvement for the tested settings | +| Linear components for 50%, 80%, 90%, 95%, 99% variance | 354, 794, 1,003, 1,128, 1,246 | Quantifies linear compression loss | +| Centred spectral participation ratio | 901.76 | Descriptive spectrum summary, not intrinsic dimension | +| Mean resultant length | 0.01822 | Describes first-moment cancellation, not uniformity | + +Across classes, original-space angular separations have these quantiles: + +| Neighbour rank | 10th percentile | Median | 90th percentile | +| --- | --- | --- | --- | +| 1 | 58.32° | 66.29° | 73.21° | +| 5 | 70.88° | 74.83° | 77.65° | +| 12 | 75.64° | 77.93° | 79.85° | +| 32 | 79.53° | 80.89° | 82.21° | + +Nearest neighbours are therefore not automatically a small-angle patch. This +matters when assessing tangent approximations; it does not invalidate their +neighbourhood relationships or imply that a different packing was intended. +The spectrum does not rule out every nonlinear representation. + +The t-SNE result uses angular inputs, seed 42, perplexity 30, 1,000 iterations, +random initialization, automatic learning rate and Barnes–Hut angle 0.5. It is +one fitted comparison, not a method ranking across seeds or configurations. +Existing payloads are in ignored `tmp/prototype-report/`; additional descriptive +measurements are in `tmp/hypersphere-research/geometry.json`. + +## 2. Views that preserve specific geometric facts + +### Anchor-centred angular atlas + +Select a class direction \(a\). Place every displayed prototype at planar radius +\(r_i=\theta(a,u_i)\), with rings labelled in degrees. Radius then has an exact +meaning under the chosen angular convention. Bearings are a separate display +choice: a projected tangent direction or an optimized arrangement. + +For \(0<\theta<\pi\), the spherical logarithm is + +\[ +\log_a(u)=\frac{\theta}{\sin\theta}(u-\cos\theta\,a), +\qquad \|\log_a(u)\|=\theta. +\] + +With a two-column orthonormal tangent basis \(B\), let +\(q_i=B^\top\log_a(u_i)\). Ordinary tangent projection uses \(q_i\) and generally +shrinks radii. An exact-radius display instead uses +\(y_i=\theta_i q_i/\|q_i\|\). This preserves distances **to the anchor only**; +bearings and distances between other points remain lossy. If \(q_i=0\), its +bearing is unresolved and must be handled explicitly. The anchor maps to zero; +an antipode has no unique logarithm/bearing. + +This is especially suitable for photo browsing: selected class at the centre, +angular rings, true neighbours highlighted, thumbnails culled without moving +their coordinates. Switching anchors produces an atlas of partial views. It +does not require selecting a global mean. The formulas can use NumPy/PyTorch; +[Geomstats supplies hypersphere log/exp operations](https://geomstats.github.io/_modules/geomstats/geometry/hypersphere.html). + +### Two-anchor distance coordinates + +Plot \((\theta(a,u_i),\theta(b,u_i))\). Both coordinates are directly interpretable, +although many different directions can coincide. Triangle inequalities restrict +the feasible region, providing a useful diagnostic check. This offers a concrete +way to inspect a close pair, a cross-tree connection, or two competing directions. +It is a distance plot, not a complete spatial embedding. + +### Great-sphere slices of directional decision regions + +Choose orthonormal columns \(Q\in\mathbb R^{D\times3}\). Directions +\(x=Qs\), \(s\in S^2\), form a genuine great two-sphere in the original space. +On that slice, + +\[ +u_i^\top x=(Q^\top u_i)^\top s. +\] + +Precompute three coefficients per class, then colour each sampled slice direction +by the largest dot product. This evaluates the nearest-prototype directional +partition exactly on the slice, up to numerical and sampling resolution. +**Do not normalize \(Q^\top u_i\)**: doing so changes the competition. + +An anchor and two chosen tangent directions define a useful navigable slice. +Two-dimensional subspaces instead give great-circle sweeps. Adjacent cells, +winning-score margins and transitions can be inspected with GBIF image labels. +Some classes have no winning region in a chosen slice; that is a property of +the slice and must not be presented as global absence. + +Slice areas are not high-dimensional cell volumes. These are geometric +nearest-direction regions; reproducing the complete hierarchical classifier's +decision rule would require its actual scoring and inference path. Evaluate the +cost of `number_of_classes × number_of_pixels` before interactive full-resolution +rendering; tiles, coarse previews and batched matrix products are natural options. + +### Packing profiles alongside any map + +Compute original-space cap counts +\(N_i(r)=\#\{j\ne i:\theta_{ij}\le r\}\), neighbour radii \(r_{k,i}\), and +mutual-neighbour links. These expose local packing without treating 2D point +density as spherical density. Display distributions, selected-class curves and +linked outliers. A random/uniform reference can be an explicitly selected +comparison; it is not needed to define these measurements. + +## 3. Dimensional-reduction families + +| Method | What it fits or preserves | Useful role here | Main limitation | +| --- | --- | --- | --- | +| Centred PCA | Ambient squared reconstruction error | Measured linear reference and dominant directional variation | Optimizes neither angular error nor neighbour recall | +| Great-subspace projection | Directions projected into a low-rank linear subspace, optionally renormalized | Simple spherical reference with residuals | Renormalizing a tiny projection can amplify poorly retained directions | +| Tangent PCA / principal geodesic analysis | Variation expressed around a base point / fitted geodesic subspaces | Anchor or regional analysis | Linearized and exact geodesic objectives differ; base point and angular extent matter | +| Principal nested spheres | Successive fitted lower-dimensional great or small subspheres | Probe curved modes that great subspheres miss | Model flexibility, sequential fitting cost and residual loss need assessment | +| Angular t-SNE | Neighbour-affinity agreement in a plane | Already useful class navigation | Map areas and long-range gaps are not angular measurements | +| Angular-input UMAP | Fuzzy neighbour graph fitted to a chosen output geometry | Compare graph structure at several neighbourhood scales | Local rescaling changes what visible density means | +| Spherical stress / MDS | Explicit pairwise-distance objective on \(S^2\) | A globe with a declared angular fidelity target | Global/local weighting tradeoffs and optimization cost | +| Spherical-output UMAP | Neighbour graph embedded with a spherical output distance | A practical first optimized-globe experiment | Spherical output alone does not preserve source packing | +| DOSNES | Neighbour embedding after doubly stochastic affinity normalization | An informative alternative normalization experiment | Equalizes affinity mass, potentially hiding packing differences of interest | + +### PCA, tangent PCA and principal geodesic analysis + +PCA is mathematically well defined for unit vectors in ambient Euclidean space. +Its objective simply answers a different question from angular fidelity. Retain +it as a baseline with variance and neighbour measurements. + +For a great-subspace projection, inspect both the retained projection norm and +the angular residual to the subspace. Renormalization onto a display sphere +should not conceal the lost component. Centring first changes the geometry: +centred PCA coordinates are not automatically a great-sphere projection. + +Tangent PCA takes logarithms at a base point and performs linear PCA there. +Principal geodesic analysis more generally seeks geodesic subspaces; its exact +projection objective is not identical to this linearization. A fitted intrinsic +mean can be useful, but its uniqueness and stability cannot be assumed from +unit norms. An extrinsic mean direction is also different from an intrinsic +Fréchet mean. The observed small resultant alone does not settle either issue. +[Fletcher et al., 2004](https://www.cs.jhu.edu/~misha/ReadingSeminar/Papers/Fletcher04.pdf). + +### Principal nested spheres (PNS) + +PNS works backwards through successively lower-dimensional subspheres. Allowing +small spheres captures latitude-like curved variation that great subspheres can +miss. This is a fitted representation of the observations, not evidence that +the training objective should produce such a structure. Compare great-only and +small-sphere variants using held-out reconstruction and angular/neighbour errors. +The original work explicitly discusses small-sphere overfitting and selection +procedures; their statistical assumptions should be reviewed separately before +adoption. [Jung, Dryden and Marron, 2012](https://pmc.ncbi.nlm.nih.gov/articles/PMC3635703/). + +The direct Python candidate is +[scikit-pns](https://github.com/JSS95/scikit-pns), with `skpns.IntrinsicPNS` and +`ExtrinsicPNS`, fitting, transformation and inverse transformation. PyPI reported +stable version 1.3.0 during this review. Check the selected release against the +documentation before implementation; do not assume `latest` documentation and +stable package are identical. Sequential fitting through 1,280 input coordinates +needs a real runtime/memory pilot. + +[torch-pns](https://github.com/JSS95/torch-pns/blob/master/src/torchpns/intrinsic.py) +currently exposes `torchpns.InverseIntrinsicPNS`: a PyTorch inverse transform +constructed from a fitted scikit-pns estimator. It is not a GPU PNS fitting +implementation. Its source also records an intentional formula difference from +the referenced inverse implementation, making cross-library round-trip checks +necessary before relying on it. + +### Optimizing an actual spherical layout + +For display directions \(y_i\in S^2\), an explicit angular-stress candidate is + +\[ +L(Y)=\sum_{(i,j)\in E}w_{ij} +\left[\arccos(y_i^\top y_j)-\theta_{ij}\right]^2. +\] + +The pair set and weights define the question: all pairs, neighbours, or a disclosed +mixture of neighbours and sampled distant pairs. If sampling estimates an all-pair +objective, use the appropriate sampling weights; otherwise call it a deliberately +reweighted objective. A fitted global distance scale is another explicit option, +not an unnoticed rescaling of angular units. + +A smoother alternative fits dot products, +\(\sum w_{ij}(y_i^\top y_j-\cos\theta_{ij})^2\). It avoids `acos` gradients but +weights angular discrepancies differently, especially near zero and pi. Neither +objective guarantees a unique layout. Compare seeds and report residuals. +Spherical MDS has concrete research implementations in graph drawing, although +graph shortest-path distance and our angular input distance are different inputs. +[Miller, Huroyan and Kobourov, 2022](https://arxiv.org/abs/2209.00191). + +Exact preservation of every angle on a globe is impossible for this matrix: +it would require `U @ U.T == Y @ Y.T`, but the latter has rank at most three and +the observed source has many more independent directions. This is an algebraic +limitation of the display, independent of any model for how prototypes are packed. + +For custom optimization, +[Pymanopt's `Oblique(3, N)`](https://pymanopt.org/docs/stable/manifolds.html#oblique-manifold) +represents N unit columns, hence N display points. `Sphere(N, 3)` instead imposes +one Frobenius-norm constraint on the whole matrix and is the wrong constraint. +[Geoopt's sphere implementation](https://geoopt.readthedocs.io/en/latest/_modules/geoopt/manifolds/sphere.html) +offers a PyTorch route with the last axis representing a sphere point. Inspect +its dtype-dependent distance clamps before declaring parity with repository +angles. These are optimization building blocks, not turnkey validated layouts. + +### UMAP, t-SNE and density-oriented variants + +[UMAP input metrics](https://umap-learn.readthedocs.io/en/latest/parameters.html) +include cosine and precomputed distances. Using normalized vectors with cosine +is a sensible comparator; using the baseline-derived angular matrix is the +cleanest initial metric-controlled experiment. Monotone input transformations +retain neighbour order but change the fuzzy edge weights. + +[Spherical output is explicitly supported](https://umap-learn.readthedocs.io/en/latest/embedding_space.html) +through `output_metric="haversine"`. The output has two angular coordinates; +`n_components=3` alone would normally mean a Euclidean 3D layout, not a sphere. +The documented conversion to XYZ is +`(sin(t)*cos(p), sin(t)*sin(p), cos(t))`. + +There is a significant implementation convention: +[UMAP's source](https://github.com/lmcinnes/umap/blob/master/umap/distances.py) +uses latitude/longitude in `haversine`, but a pi/2 shift in +`haversine_grad`, which is the registered output metric with gradient. Its +first output coordinate therefore follows the colatitude convention used in +the tutorial. Test known poles, quarter-circle pairs, seams and the XYZ distance +identity against the pinned version. Do not reuse input latitude conversion +uncritically for output coordinates. + +The existing [scikit-learn t-SNE](https://scikit-learn.org/stable/modules/generated/sklearn.manifold.TSNE.html) +path already gives an angular-input planar reference. +[openTSNE](https://opentsne.readthedocs.io/en/stable/api/affinity.html) adds useful +affinity controls, including multiscale affinities and `PrecomputedAffinities`. +The latter expects affinities, not a raw angular-distance matrix. Neither is a +drop-in spherical-output t-SNE implementation. + +[densMAP](https://umap-learn.readthedocs.io/en/latest/densmap_demo.html) is worth a +controlled comparison when preserving local density is a display objective. +Its density regularization does not turn screen areas into high-dimensional +spherical volumes. Compare it with actual cap counts and neighbour radii. +UMAP's local normalization is part of its construction, not an empirical finding +about these weights. [UMAP paper](https://arxiv.org/abs/1802.03426). + +[DOSNES](https://yaolubrain.github.io/dosnes/) explicitly embeds on spheres after +making affinities doubly stochastic. For this task, the normalization is a major +scientific choice: equal affinity mass can remove variation we want to inspect. +The author's [repository](https://github.com/yaolubrain/DOSNES) contains MATLAB +fitting and a JS viewer and links a [separate Python port](https://github.com/Coni63/DOSNES). +That port documents limited checking and discrepancies from the MATLAB results. +Treat it as research code requiring verification, not the first production path. + +Graph-distance methods such as Isomap and diffusion-based displays are additional +questions one can ask of a neighbour graph. Their graph path or diffusion +distances must be labelled separately from direct great-circle separation. +They are not needed to respect the unit-sphere input constraint. + +## 4. Cartography applies after reduction to S² + +Stereographic projection of \(S^{1279}\) gives 1,279 Euclidean coordinates, not +two. A globe or map still needs dimensional reduction, or an actual 2D slice. +Conformality of a later cartographic step does not undo losses in that reduction. + +For an actual unit two-sphere, these azimuthal projections use angular distance +\(\theta\) from the map centre and retain bearing: + +| Projection | Planar radius | Preserves | Display consequence | +| --- | --- | --- | --- | +| Orthographic | \(\sin\theta\) | Appearance from an infinitely distant camera | Visible hemisphere only; strong limb compression | +| Stereographic | \(2\tan(\theta/2)\) | Local angles | Antipode at infinity; large area distortion | +| Gnomonic | \(\tan\theta\) | Great circles as straight lines | Only an open hemisphere; diverges at 90° | +| Azimuthal equidistant | \(\theta\) | Distances from map centre | Other distances and areas distorted | +| Lambert azimuthal equal-area | \(2\sin(\theta/2)\) | Two-sphere area | Shapes and most distances distorted | + +These are unit-sphere formulas, derived with unit scale at the centre. +[Snyder's USGS manual](https://pubs.usgs.gov/publication/pp1395) is the primary +reference for their geometry and domains. + +For a radial map of the full \(S^{D-1}\) into a \((D-1)\)-dimensional tangent +space, the volume scaling away from singularities is +`r'(theta) * (r(theta)/sin(theta))**(D-2)`. This follows by dividing the radial +volume elements. A two-sphere equal-area projection therefore does not promise +volume preservation for the original 1,280-coordinate prototypes. + +[D3-geo](https://d3js.org/d3-geo/azimuthal) supplies these projections, with +rotation and clipping. Its [coordinate interface](https://d3js.org/d3-geo/projection) +uses `[longitude, latitude]` in degrees. Keep this distinct from radians, +colatitude and XYZ used in numerical libraries. + +## 5. Concrete implementation and rendering choices + +### Python and computation + +| Tool | Verified capability | Appropriate experiment | +| --- | --- | --- | +| Existing NumPy/PyTorch + sklearn | Current effective-weight extraction, baseline angles, PCA and angular t-SNE | Reuse the numerical foundation; implement anchor/slice formulas directly | +| [Geomstats](https://geomstats.github.io/getting_started/first-steps.html) | Hypersphere geometry, `TangentPCA`; NumPy and optional PyTorch backends | Validate log/exp and investigate regional tangent coordinates | +| [scikit-pns](https://scikit-pns.readthedocs.io/en/latest/?badge=latest) | Intrinsic/extrinsic nested-sphere estimators | Small pilot followed by measured scaling to the full matrix | +| Pymanopt / Geoopt | Sphere-constrained optimization primitives | Explicit angular or dot-product stress on a display sphere | +| umap-learn | Custom/precomputed input distances and non-Euclidean output metrics | Compare planar and spherical output with controlled inputs | +| openTSNE | Flexible affinity construction | Multiscale planar comparison without silently changing source geometry | +| [RAPIDS cuML UMAP](https://docs.rapids.ai/api/cuml/legacy/api/generated/cuml.manifold.umap/) | GPU UMAP, cosine metric and neighbour-graph inputs | Larger planar experiments when GPU cost is justified | + +The reviewed cuML interface does not expose Python UMAP's `output_metric`; do +not assume spherical-output parity. Its API documentation also needs careful +version-specific reading for pairwise versus precomputed-neighbour inputs. +There is no performance claim here for these packages on this checkpoint. + +### Browser and image labels + +| Framework | Strength | Integration limit | +| --- | --- | --- | +| Existing Canvas + thumbnail overlay | Already supports this checkpoint, linked inspection and bounded photo requests | Extend first for anchor views; benchmark before replacing | +| [Three.js](https://threejs.org/docs/pages/SpriteMaterial.html) | Rotatable 3D globe, sprite image labels; `sizeAttenuation` control | Need hemisphere/depth handling, screen-space culling, picking and image budgets | +| [deck.gl IconLayer](https://deck.gl/docs/api-reference/layers/icon-layer) | GPU image icons, atlases and pixel-sized billboards | Its collision extension is not equivalent to our rectangle-overlap rule | +| [PAIR ScatterGL](https://github.com/PAIR-code/scatter-gl) | 2D/3D scatter, sprite-sheet images, selection and camera callbacks | Verify pinned dependency integration and atlas/culling behaviour | +| [regl-scatterplot](https://github.com/flekschas/regl-scatterplot) | WebGL planar point rendering and interaction | Add a separate bounded photo-label layer | +| D3-geo | Cartographic projection, rotation, clipping | Handles a two-sphere/map, not 1,280-dimensional reduction | + +[deck.gl CollisionFilterExtension](https://deck.gl/docs/api-reference/extensions/collision-filter-extension) +checks feature anchors against rendered coverage and supports collision +priorities. That differs from testing the allowable fraction of overlap between +thumbnail rectangles. Preserve the current size/density/overlap semantics unless +a deliberate UI change is made. Transparent icon anchors also need special care +under its documented alpha handling. + +[UMAP-JS](https://github.com/PAIR-code/umap-js) has custom input `distanceFn`, +asynchronous fitting and incremental steps. The reviewed API does not provide +Python UMAP's spherical output option; its README also notes differences in +initialization and specialized metric handling. Async fitting is not by itself +a worker-thread guarantee. Initially, fit in Python and serve coordinates; use +a worker if browser-side fitting later becomes useful. + +For every renderer, a thumbnail is a class image label associated with a numerical +point. Culling, visual size and overlap may change label visibility but must not +move prototypes or change their distances. Optional label displacement should +retain an explicit connection to the fixed prototype point; the explorer's +**Push thumbnails apart** mode uses anchor markers and leader lines. A globe adds back-side occlusion to +the existing viewport and overlap tests. Preserve photo provenance and cache +budgets, and expose point counts even where image labels are culled. + +## 6. Validation before choosing a default + +1. **Geometry contract:** use the checkpoint's effective rows and exact class + order. Compare against baseline z-derived angular distances, with the clamp + and self-pair conventions recorded. Test identical, orthogonal, antipodal and + nearly coincident directions. Verify every coordinate conversion separately. +2. **Neighbour fidelity:** recall at k = 1, 5, 12, 32 and 64; show per-class + distributions and poor cases as well as a mean. Use original full-population + neighbours even when selecting a subset of anchors for evaluation. +3. **Distance fidelity:** angular errors and Shepard plots on both near pairs + and uniformly sampled unordered pairs. Report strata separately so the many + distant pairs do not conceal failures among neighbours. Specify any scale + fitting and distinguish it from absolute angular preservation. +4. **Two sources of display loss:** for spherical layouts, evaluate distances on + the output sphere first, then the additional camera/cartographic distortion. + Screen-space neighbour recall alone cannot evaluate a globe's geometry. +5. **Packing fidelity:** compare local radii and cap counts to apparent density. + Do not label a dense t-SNE patch or a large slice cell as a measured + high-dimensional volume. Inspect mutual neighbours and cross-tree links. +6. **Stability and cost:** multiple seeds, neighbourhood scales, optimization + traces, peak memory, fit time and interactive latency. A layout change between + checkpoints needs rotation/alignment treatment; compare original distances + and neighbour changes regardless of display alignment. +7. **Synthetic probes:** algebraic edge cases, narrow caps, small-circle modes, + uneven local densities, separated groups and independent directions. Match + N and D when investigating packing scales. The existing six-point fixture + with k=5 trivially retains every neighbour; 100% there is not evidence of a + faithful projection. + +No full N-by-N objective is automatically cheap: this checkpoint has 79,777,396 +unordered off-diagonal pairs and about 638 MB per dense float32 square matrix +(609 MiB), before copies and working arrays. Offline projection, sparse retained +graphs, sampled diagnostics and periodic training-time summaries have different +cost boundaries. Keep them explicit. + +## 7. Research candidates + +The [explorer roadmap](prototype-explorer-roadmap.md) owns current implementation +order and status. The candidates below retain the research rationale; their +original ordering is not a separate development queue. + +**An exact-radius anchor atlas.** It extends the current photo browser, +answers a precise angular question and requires no new package. Add selectable +bearings, angular rings, unresolved-bearing handling and original-neighbour +overlays. Show that radii are exact while bearings are reduced. + +**A metric-controlled globe comparison.** Fit angular-input UMAP to a +plane and to spherical output, alongside the existing t-SNE. Add a small custom +spherical-stress pilot if fidelity warrants it. Inspect both original-angle +errors and neighbour recall before deciding whether a globe helps browsing. +Use Three.js for a genuinely rotatable sphere, with the current thumbnail policy. + +**Interactive great-circle/great-sphere slices.** These address the +partition question directly, complementing the neighbour atlas and compressed +global map. Retain all prototype coefficients when evaluating winning regions; +photo labels identify the classes appearing in the slice. + +PNS is a worthwhile research comparator after a bounded numerical/scaling pilot. +Its fitted subspheres answer a different question from browsing arbitrary +observed packing. No choice above changes the training constraint, baseline +distance transform, dendrogram, or colour semantics. + +## 8. Evidence and reproducibility boundary + +This review inspected primary papers, official documentation and selected +upstream source. Geomstats, Pymanopt, Geoopt, scikit-pns, torch-pns, openTSNE, +UMAP and cuML were not installed or runtime-tested in the shared environment. +Source inspection establishes the cited interface facts, not package parity, +scaling, numerical correctness, GPU support of every path or browser performance. +Pin versions and isolate dependencies when starting those experiments. + +The PCA/t-SNE measurements come from the existing runnable explorer and its +stored real-checkpoint payload. The additional spectrum uses float32 normalized +effective rows, centring without whitening, and eigenvalues of `X.T @ X`. +Participation ratio is `(eigenvalues.sum()**2 / eigenvalues.square().sum())`; +component counts use cumulative nonnegative eigenvalues. Mean resultant is the +norm of the average normalized row. Angular quantiles invert the existing +stored z profiles. Medians use `torch.median` (lower middle for even counts). + +Source URLs using `latest`, `stable`, `master` or `main` are moving references, +reviewed on the date above. Before implementation, record the exact package +versions/commits used and validate coordinate and numerical contracts against +them. Research references do not override the empirical baseline. diff --git a/docs/production-release-integration.md b/docs/production-release-integration.md new file mode 100644 index 00000000..8191baea --- /dev/null +++ b/docs/production-release-integration.md @@ -0,0 +1,79 @@ +# Production release storage and deployment integration + +## Existing convention + +The `MAMBO_v2` tag's `mini_trainer/deploy.py` downloaded named checkpoint files +from the public `/Models` share, below `MAMBO/`. Aliases such as `europe`, +`north_europe`, and `full` selected different files. A local weights path bypassed +the download. Retain that useful separation between a model identifier, an +immutable download, and an explicit user override. + +Do not replace the historical MAMBO files: this release uses a different backbone +and has different class coverage. Its final checkpoint identity is +`174b9214bfea2df69e4f5c5d16afd841fec961db4274f3e6bf474cef9cab5e8a`. + +## This publication + +Publish the extracted `global-lepi-production-release-20260911T150236Z` directory +under `/Models`, through SFTP. It contains 1,224 original files and 25,455,849,324 +bytes. The uploaded ZIP's checksum and all original internal file checksums passed. +Remote inventory sizes matched for all files; read-back SHA-256 checks also matched +for the final PyTorch weights, FP32 ONNX external weights and PTQ ONNX graph. +The remote size inventory is not a claim of full remote checksum readback. + +Original `SHA256SUMS` covers the original release. The added `viewer/` distribution +has a separate `SHA256SUMS` and browser-bundle artifact hashes. Keep these scopes +explicit instead of editing the original research-release manifest after the fact. +Local image parity fixtures are excluded from publication. + +The viewer uses a new FP32 export with prediction and embedding outputs. The main +release's original FP32 and PTQ exports remain untouched. A successful PTQ export +or runtime check does not establish PTQ accuracy or a deployment speedup. + +## Browser hosting + +Serve the viewer and its model/runtime bundle from the same public origin. The +existing share serves HTML and `.js` correctly, but did not provide MIME types for +`.mjs` or `.wasm` in the checked responses. The runtime module is copied unchanged +with a `.js` extension and selected through ONNX Runtime's explicit `wasmPaths` +override. Actual Chromium inference through the public share was verified. Avoid +assuming a separate website can fetch these files: the checked responses did not +advertise cross-origin permission. + +A standalone site on a different origin needs an explicit CORS and worker-loading +strategy, or a same-origin static mirror of the bundle. No inference backend or +credential-bearing proxy is required for the current distribution. Model loading +is explicit; images are processed client-side. Keep optional GBIF photo/name +services separate from this guarantee. + +## Rollout increments + +1. **Freeze and catalogue.** Record final URLs, hashes, class order, preprocessing, + output semantics, checkpoint identity, runtime requirements and evidence. Expose + models and documentation as individual downloadable files. Keep release directories + immutable once published; corrections become a new version. +2. **Add a small model registry.** Map human-facing model names to versioned manifests, + not mutable raw checkpoint filenames. Preserve explicit file/URL overrides. Validate + downloaded hashes before promoting a temporary download into a local cache. Keep + model source/version visible in predictions and saved outputs. +3. **Qualify the intended deployment.** Measure real-image parity and throughput on the + target runtime, confirm preprocessing and hierarchy semantics, and evaluate any PTQ + or geographical class filtering separately. Use the held-out in-domain and expert + evaluation protocols; choose thresholds on separate calibration data. Do not infer + production acceptance from the browser smoke test. +4. **Offer a reviewed default.** Introduce a release alias only after acceptance; + preserve rollback to the previous immutable manifest. The qualitative explorer can + link to that same model identity without being the authoritative evaluation tool. +5. **Extend selectively.** Add additional browser/providers or dataset inputs based on + measured benefit. Folder/ZIP inference, offline install, optional cached taxon names, + and cross-origin bundle loading are separate bounded increments. Recomputing global + t-SNE in the browser is unnecessary for inserting an image into this fixed map. + +## Evidence limits + +The first browser implementation was exercised with the final production checkpoint +in Chromium, using CPU WASM, one real-image fixture and synthetic preprocessing +patterns. It is not a Safari/Firefox/WebGPU qualification or a broad accuracy study. +Fixed-map t-SNE insertion is approximate; its measured neighbour retention accompanies +the image. Preserve both the numerical runtime comparison and the end-to-end image +comparison, since matching preprocessed tensors alone does not qualify image decoding. diff --git a/docs/prototype-browser-implementation.md b/docs/prototype-browser-implementation.md new file mode 100644 index 00000000..6cae911d --- /dev/null +++ b/docs/prototype-browser-implementation.md @@ -0,0 +1,190 @@ +# Standalone prototype explorer: implementation plan + +Status: the first browser-inference increment is implemented; see +[browser inference](prototype-browser.md). It uses the final production checkpoint, +not the historical epoch-26 reference below. Full browser-side global analysis and +the remaining broader milestones are still pending. This plan defines +execution and acceptance for the standalone direction in the +[compact roadmap](prototype-explorer-roadmap.md). It does not create an active goal. + +The [unified execution plan](prototype-explorer-implementation.md) schedules +milestones 1–3 as a bounded feasibility goal before the inference UI. The goal +below remains the full seven-milestone destination, not the next single increment. + +## Goal text + +> Build and validate a standalone prototype explorer that loads a mini_trainer +> export bundle and user-selected images, runs model inference in the browser, +> and explains predictions through embeddings in the model's prototype space. +> The distributed tool must require no Python, PyTorch, CUDA, or inference backend. +> Support local preprocessing, predictions, embedding-to-prototype diagnostics, +> query placement, and generation of the existing global exploration views. +> One-time model export may use mini_trainer. Precomputed diagnostics may accelerate +> opening a bundle, but must not be the only way to obtain the global views. +> Use the epoch-26 production checkpoint and existing numerical functions as the +> reference, preserve current explorer interactions, and provide reproducible +> browser parity, performance, offline-operation and installation evidence. + +## Reference and boundaries + +- Checkpoint: `/home/asger/mini_trainer/tmp/best_global-lepi-production-w32-1_epoch26.pt`. + SHA-256: `abb9f66d95fe867bd31847cb33782a353b1d3faf4c79c4b10d93037f33fe61f4`. +- Verified prototype shape: 12,632 × 1,280; normalized float32 effective rows, + zero biases. Epoch 4 remains a class-aligned comparison, not a replacement model. +- Initial supported heads: the explorer's current `Classifier` and + `HierarchicalClassifier` linear heads. Reject unsupported contracts explicitly. +- Start in `/home/asger/mini_trainer/.worktrees/prototype-space`, on + `feature/prototype-space`; revalidate HEAD and status before work. Further + concurrent implementation requires individually assigned worktrees and scopes. +- Preserve current Python entry points and training/export defaults. Export is an + authoring step; the browser consumes the resulting bundle, not arbitrary `.pt`. +- No architecture substitution, calibration claim, assumed packing distribution, + or unlabelled approximation. Browser inference does not establish GPU-training + parity or pixel-attribution explainability. + +## Architecture and deliverables + +Extend the existing exporter and explorer; avoid a second classifier implementation. +Keep authoring in Python, and inference/analysis in browser workers with a static +HTML/JS/WASM distribution. Prefer WASM CPU qualification first; add WebGPU only +with provider-specific parity and capability checks. Pin runtime assets locally. + +Define a versioned bundle with: + +| Artifact | Required content | +| --- | --- | +| Model | ONNX graph and every external tensor-data file; actual evaluation outputs plus a named preclassification embedding output. | +| Manifest | File hashes, checkpoint identity, output-tree semantics, effective class/hierarchy order, shapes/dtypes, embedding stage, preprocessing contract, supported runtime versions. | +| Prototypes | Binary effective weights, biases and necessary score metadata; retain magnitudes separately from directional geometry. | +| Projection state | PCA centering vector and basis; optional global coordinates/diagnostics with method, seed and numerical-configuration provenance. | +| Verification fixture | Legally usable real-image references or reproducible local fixture instructions, hashes, preprocessed tensors, embeddings and actual output tensors. | + +Allow selection of the complete bundle through the UI, including external ONNX +data. Choose and test a container/loading format in milestone 1; do not assume a +single ONNX file is a complete deployment. Keep large tensors binary rather than +embedding extra copies in JSON. No browser-to-server model/image upload. + +## Ordered implementation milestones + +### 1. Contract and reference fixture + +Inspect the epoch-26 architecture and exact evaluation preprocessing using the +existing checkpoint loader. Identify the embedding after head preclassification, +including any hidden transform and normalization; do not substitute raw backbone +features. Locate suitable real validation images locally and record their hashes. +Missing validation data must be reported, never replaced by synthetic-only evidence. + +Implement bundle schema validation and small flat/hierarchical fixtures. Define +explicit unsupported-input and version errors. Record the baseline environment, +model outputs and extraction semantics before changing export code. + +**Gate:** fixture outputs reproduce the current model, and every bundle field has +an identified producer and browser consumer. + +### 2. Export predictions plus embeddings + +Extend `mini_trainer/modeling/onnx.py` with an opt-in, export-compatible path for +both output kinds. Reuse the actual forward and do not depend on the training-only +`EmbeddingContext`: export currently rejects active training/supervision contexts. +Avoid a duplicate backbone pass. Preserve hierarchical output structure and masks. + +Add authoring support through the existing export/explorer commands, with weights +as the only required model argument. Extract executable preprocessing from supported +transforms; fail with a specific unsupported-transform error instead of guessing. + +**Gate:** real images and batches 1/2/4 match the PyTorch reference for predictions +and embeddings. Start with existing export tolerances (`rtol=1e-4`, `atol=1e-5`); +report error distributions and ranking differences. Any tolerance revision requires +an explained numerical investigation. Source checkpoint and model state stay intact. + +### 3. Browser inference qualification + +Load the real exported model with ONNX Runtime Web in a worker. First compare using +identical preprocessed tensors, isolating runtime errors from image decoding errors. +Then implement and verify image decoding, orientation, channel order, resizing, +cropping, scaling and normalization end to end. Include portrait/landscape images +and relevant alpha/orientation cases. General browser canvas resizing is not assumed +to reproduce the repository transform. + +**Gate:** WASM inference passes real-image parity; WebGPU is enabled only after its +own checks. Record runtime/browser versions, cold load, warm latency and peak memory. +Unsupported providers produce an actionable result and a verified fallback. + +### 4. Inference and geometric explanation UI + +Add model/image selection, progress, cancellation, bounded image batches, prediction +inspection and query thumbnails. Show actual head scores with their semantics. +Compute query-to-prototype geometry using the existing clamp/self-pair conventions; +port log-domain operations with independent tail/extreme-value tests. Query images +are not automatically class members or labelled correctly. + +Show angular neighbours, competing predictions, score margins and neighbourhood +profiles; distinguish geometric proximity from the full classification rule. + +**Gate:** browser calculations agree with repository references; latest selection +wins under rapid changes; cancelled work cannot publish stale results; loading does +not block navigation or restart thumbnail motion. + +### 5. Place inference embeddings in the views + +Implement fixed-transform PCA placement from saved means/bases and direct angular +anchor inspection. Add a separately validated method for inserting queries into a +fixed t-SNE layout; keep prototype coordinates fixed and disclose placement error. +Do not place images at predicted-class coordinates or label neighbour interpolation +as exact t-SNE projection. Evaluate poor as well as favourable query cases. + +**Gate:** PCA/anchor geometry matches direct computation; nonlinear placement has +measured neighbourhood/distance distortion; query selection links maps, neighbour +inspection and prototype/class photos without changing the underlying prototypes. + +### 6. Generate global diagnostics client-side + +Port the computations necessary for the current matrix summaries, Ward dendrogram, +PCA and angular t-SNE to workers/WASM/WebGPU as appropriate. Use blocked data access, +bounded allocations, persistent caches, cancellation and progress. Estimate real +peak allocations before running the full case. A single dense float32 matrix for +this model is about 638 MB; several simultaneous copies are not an acceptable default. + +**Gate:** the epoch-26 bundle can construct the global views without precomputed +results or a Python service on a documented reference machine. Distances and log tails +match baseline tolerances; tie/order differences in linkage are explained and tested. +Projection equivalence is assessed by its objective, fidelity and stability, not +pixel equality. Alternative/approximate algorithms remain explicitly selectable; +they do not silently replace the baseline. If this gate is infeasible, report it +as unfinished rather than declaring an inference-only viewer complete. + +### 7. Standalone distribution and release audit + +Package as a static application with an installable/offline route. Static hosting +may serve files, but must perform no inference or numerical analysis. Do not promise +that double-clicking `file://` supports every browser capability. Bundle runtime +assets; record required secure-context/isolation settings and tested browsers. + +Use local model/image selection and verify offline reopening after installation. +Treat GBIF images as optional network content with attribution and explicit offline +behaviour. Test model replacement, memory/resource disposal and failure recovery. + +**Gate:** operate from a clean environment without Python/PyTorch/CUDA, with model +and images retained locally. Network inspection proves no inference-service calls. +All existing explorer interactions still work, including labels/aspect ratios, +arrival continuity, culling fades and fixed prototype positions. + +## Validation and completion evidence + +Extend `tests/export/test_onnx.py`, the affected `tests/utils/test_prototype_*` +coverage, and the existing browser checks. Keep new fixtures small for regression +runs; maintain a separate reproducible real-checkpoint qualification command. + +Run `bash dev/check.sh static`, affected tests, and `bash dev/check.sh all` for the +export/loading changes as required by repository policy. Run +`bash dev/check-wheel.sh` for packaging/extras, plus installed-bundle browser tests. +Use isolated dependency environments with an explicit PyTorch backend; never sync +the shared CUDA environment implicitly. Preserve real fixtures and outputs outside +Git; commit schemas, tests, commands and compact evidence summaries. + +Completion requires all seven gates, a runnable distribution, updated usage docs, +and an evidence table recording artifact hashes, platforms, numerical errors, +latency/memory, skips and unresolved limitations. A miniature model, CPU ONNX check, +precomputed-only viewer or screenshot alone is insufficient. Commit bounded stable +increments and update the compact roadmap; do not mark the overall goal complete +when only an earlier milestone has passed. diff --git a/docs/prototype-browser.md b/docs/prototype-browser.md new file mode 100644 index 00000000..ba38f754 --- /dev/null +++ b/docs/prototype-browser.md @@ -0,0 +1,153 @@ +# Browser inference in the prototype explorer + +The explorer can run an embedding-enabled ONNX bundle locally using ONNX Runtime +Web 1.24.3 (WASM CPU). User images stay in the browser. The static server serves +model and application files; it performs no inference. + +Load the model, then choose an image or use **Take photo** to request the phone’s +rear camera. Camera capture depends on the browser/device; the ordinary gallery +picker remains available. JPEG, PNG, WebP and other browser-decodable images are +accepted. HEIC/HEIF depends on native browser support; unsupported files get a +conversion hint. No decoder service or image upload is required. + +EXIF rotation and mirroring are applied before sampling. Transparent pixels are +composited onto white. The preview shows the oriented, square image supplied to +the model. The former 40-megapixel restriction is removed: preprocessing samples +straight into a model-sized canvas instead of allocating a full-resolution canvas +and pixel array. The browser still needs enough memory to decode the source image. +Opaque, unrotated images retain the existing nearest-neighbour sampling recipe. + +Inspect the five highest-scoring classes at each hierarchy level. Species buttons +select the corresponding prototype; higher-rank results do not select an arbitrary +species. Scores are softmax probabilities, not calibrated reliability. On the map, +use one finger to pan and two to pinch-zoom; mouse drag and wheel zoom remain. +Folder and ZIP ingestion remain future work. + +## Image placement + +When the report contains angular t-SNE, the browser computes angular distances +from the image embedding to all prototype directions. It forms Gaussian affinities +at perplexity 30 over the nearest 90 prototypes, then minimizes conditional KL +against Student-t affinities in the fixed map. Four deterministic starting points +and a bounded, decreasing-loss line search reduce sensitivity to initialization. +Only the image moves. This follows the fixed-reference insertion approach used by +[openTSNE](https://opentsne.readthedocs.io/en/stable/api/index.html#opentsne.TSNEEmbedding.transform), +but is a separate implementation, not a port or an identical objective to a joint +refit. No t-SNE coordinates are assigned merely by selecting the predicted class. + +The purple diamond shows the result. The status reports top-12 angular-neighbour +retention in the displayed neighbourhood and computation time. Low retention is a +limitation of the placement/map, not evidence that the model prediction is wrong. +A saved PCA mean and basis also support exact fixed-transform PCA placement of the +normalized embedding. Neither method changes the prototype coordinates. + +## Authoring a distribution + +Use the matching checkpoint and an ONNX export created with +`mt_export --include-embeddings`. This option preserves default export behaviour +when omitted; the backbone runs once and the small eval preclassification transform +is replayed to expose the embedding. Flat `Classifier` and `HierarchicalClassifier` +heads are supported. Other head families fail explicitly. + +The browser currently supports one explicit preprocessing contract: repository +nearest-neighbour square uint8 resize, torchvision-style bilinear resize and centre +crop, then ImageNet RGB scaling and normalization. Supply this recipe to export +only after verifying it against the checkpoint's evaluation transform: + +```json +{"contract":"nearest-square-uint8-bilinear-center-imagenet-v1","size":384,"resize":438} +``` + +The browser bilinear implementation can differ by one uint8 value from torchvision; +qualify end-to-end image results as well as identical-input runtime parity. This +contract does not cover arbitrary user transforms or dataset preprocessing overrides. + +1. Export the checkpoint with the recipe, embeddings, and normal parity checks. +2. Generate the existing explorer from the same checkpoint (`mt_explore WEIGHTS + --export --output REPORT`). Keep angular t-SNE enabled. +3. Unpack `onnxruntime-web@1.24.3` from npm. Add `LICENSE.txt` from the matching + [ONNX Runtime release](https://github.com/microsoft/onnxruntime/blob/v1.24.3/LICENSE) + to the unpacked package root; npm does not include this file. +4. Package the browser model beside the report: + +```bash +uv run --no-sync python -m mini_trainer.visualization.prototype_space.browser \ + --export ONNX_BUNDLE --weights BEST_PT \ + --runtime UNPACKED_RUNTIME_PACKAGE --output REPORT/browser-model +``` + +5. Serve `REPORT` as static HTTP(S), and open `explorer.html`. The default bundle + URL is `browser-model/manifest.json`. The bundle carries its checkpoint identity, + class order, graph, external tensors, prototype directions, runtime and hashes. + +The package is created atomically and existing destinations are not overwritten. +The runtime package and supplied ONNX export are trusted authoring inputs. Use only +trusted browser-bundle URLs: bundles also contain executable runtime assets. + +## Hosting and limitations + +Serve the explorer and browser bundle from the same origin, with correct HTML, +JavaScript, MJS and WASM MIME types. The initial implementation uses a worker from +the model-bundle origin, so a cross-origin bundle requires more than permitting +model-file CORS alone. No special COOP/COEP headers are needed for its single-thread +WASM configuration. A downloaded HTML retains numerical exploration, but browser +inference requires HTTP(S) serving and the model/runtime files; double-clicking the +HTML is not the complete inference distribution. + +GBIF names and reference images are fetched directly by the browser, controlled +by **Settings → Class IDs**. Generic IDs make no GBIF requests. Species prediction +rows share attributed reference examples with the map and inspector; those photos +are not the user's query image. Missing names/images leave IDs and scores usable. +Optional packaged names work offline; remote photo URLs still require networking. +The precomputed numerical views do not depend on GBIF. The broader standalone-analysis roadmap is still open: this version does +not recompute global Ward/PCA/t-SNE diagnostics in the browser. + +## Browser smoke check + +With a Chromium executable available, the repository check opens the real hosted +URL, loads its model, selects the supplied local image, and requires predictions, +an embedding, and no uncaught page errors: + +```bash +CHROME_PATH=/path/to/chrome node dev/prototype_space/check_inference.mjs \ + https://host/path/explorer.html /path/to/image.jpg /tmp/browser-check.json +``` + +Identical-input ONNX parity and original-image preprocessing comparisons remain +separate numerical qualification checks. The smoke check alone does not establish +accuracy or cross-browser compatibility. The pinned runtime module is distributed +with a `.js` extension because some static servers do not assign `.mjs` a JavaScript +MIME type; its contents are unchanged. + +## Optional packaged GBIF names + +For a GBIF-labelled report, prepare a resumable metadata snapshot before rendering +the release candidate. No credentials are needed. Each entry retains the original +ID and retrieval date; name resolution never changes the model vocabulary. + +```bash +.venv/bin/python dev/prototype_space/prepare_gbif.py \ + --report /path/to/report/report-data.json \ + --output /path/to/report/gbif-snapshot.json +``` + +Use `--limit 100` for a bounded first pass; rerunning fills missing entries. Failed +lookups remain missing and can be retried. `render_report` embeds a neighbouring +`gbif-snapshot.json` in the HTML when present, so it survives downloading the report. +Re-render existing numerical data without recomputing geometry: + +```bash +.venv/bin/python - /path/to/report <<'PY' +import sys +from pathlib import Path +from mini_trainer.visualization.prototype_space.explore import render_report +root = Path(sys.argv[1]) +render_report(root, (root / "report-data.json").read_text()) +PY +``` + +A snapshot explicitly supplies a GBIF default; saved user preferences take precedence. +The current helper packages taxon names, not image bytes or occurrence albums. +Static publication should include the snapshot as well as the regenerated HTML, +and regenerate its asset hashes. Keep a new release candidate separate from an +already published production release. diff --git a/docs/prototype-diagnostics.md b/docs/prototype-diagnostics.md new file mode 100644 index 00000000..9eb65021 --- /dev/null +++ b/docs/prototype-diagnostics.md @@ -0,0 +1,86 @@ +# Prototype diagnostics in log space + +Prototype diagnostics use the classifier's effective last-layer weights and the +existing dimension-dependent cosine-to-z transform. The log-domain path changes +how the resulting Gaussian quantities are evaluated and displayed; it does not +fit a new distribution or impose a different organization on the learned space. + +```python +from mini_trainer.modeling import class_log_similarity +from mini_trainer.visualization import plot_class_distance_matrix + +log_cdf = class_log_similarity(model) # log Phi(z), one matrix per level +log_tail = class_log_similarity(model, complement=True) # log Phi(-z) +figures = plot_class_distance_matrix(model, log_domain=True) +``` + +Numerical range and colour range are independent. For example, add +`log_range=(-8 * math.log(10), 0.0)` (with `import math`) to focus colours on +probabilities from `1e-8` to `1`. These are natural-log display bounds. Values +outside them receive endpoint colours; the computed matrix is not clipped. +Omit `log_range` for the full computed colour range. Small outliers can compress +the visually interesting region in that full-range view. + +`class_log_similarity` returns float32 natural logs on the weights' device. It +uses the existing z-score computation, disables any surrounding autocast for the +diagnostic, and evaluates `torch.special.log_ndtr` directly. The self-similarity +diagonal is exactly 0 in log-CDF and `-inf` in log-tail. No gradients or probability +floor are used. Quantized-training weights follow the existing dequantization +handling before the diagnostic, not a separate prototype interpretation. + +The complementary representation is useful for strongly aligned prototypes: +`log Phi(-z)` remains representable when `Phi(z)` rounds to 1. In contrast, even +directly evaluating `-log Phi(z)` can yield a distance too small for float32. Keep +log-tail values for analysis/rendering rather than exponentiating them back into +probabilities. Higher precision is useful as an independent reference, not required +by this path. See the [PyTorch definition of log_ndtr](https://docs.pytorch.org/docs/stable/special.html#torch.special.log_ndtr). + +## Rendering and training-time evaluation + +`plot_heatmap(..., log_input=True)` accepts natural-log values. It performs maximum +aggregation and colour normalization directly on logs, pads with `-inf`, and +formats colourbar ticks in exponent notation without creating tiny linear tick +values. `min_val_display`, if supplied, remains a nonnegative linear threshold. +Zeros (`-inf`), NaNs and infinities are masked. Ordinary `plot_heatmap` defaults +and its legacy pixel mapping remain unchanged. + +Evaluation logging during training now calls the log-domain renderer with +`log_range=(-8 * math.log(10), 0.0)` under the existing +`Class distance matrix/lvl...` figure tags. The represented quantity is still the +complementary probability, with percentage labels; near-neighbour distinctions +are no longer lost by subtracting a rounded CDF from 1. This uses float32, including +the log-probability matrix, and no matrix-wide float64 promotion. +The initial `1e-8` to `1` colour window follows the familiar legacy display range, +so extremely tiny tails do not consume most of the palette. Endpoints show +clipping with ≤/≥ labels. Distinctions below the display floor remain in the +log scores and can be explored with a different range; this is not a statistical +cutoff or a universal choice of interesting scale. + +The existing `class_similarity`, `class_distance`, and default +`plot_class_distance_matrix(model)` remain available as the known baseline. +The dendrogram still uses its existing Ward linkage and distance. Do not feed +log-tail or log-distance values directly into Ward and call that the same tree: +nonlinear distance transformations can change its merges. A future stable +clustering implementation needs its own numerical and behavioural validation. + +These diagnostics run during evaluation and do not alter training logits, loss, +optimizer updates or saved checkpoint formats. They apply to runs using this +feature branch; an already-running training process does not pick them up. + +## Evidence and exploration + +The epoch-4 example has 12,632 unit-length prototypes in 1,280 dimensions, with +zero biases. Full evaluation found 249,324 unordered off-diagonal pairs whose +legacy distance is zero, involving every class. Direct float64 log-CDF evaluation +gives a positive distance for all of those pairs, confirming numerical saturation +rather than exact equality of their vectors. Float32 log-tail is the primary +representation for the improved views; the higher-precision evaluation is only +a comparison reference. + +Tests compare log-tail values against independent `math.erfc` evaluations, reject +CDF calls in the log path, check autocast/device/dtype and multiple prototype levels, +preserve legacy heatmap pixels, and exercise padding and tails beyond linear +probability range. CPU evidence does not establish CUDA execution or GPU cost. + +For a runnable report using the real checkpoint and labelled synthetic probes, +see [the prototype explorer](../dev/prototype_space/README.md). diff --git a/docs/prototype-explorer-implementation.md b/docs/prototype-explorer-implementation.md new file mode 100644 index 00000000..30654d45 --- /dev/null +++ b/docs/prototype-explorer-implementation.md @@ -0,0 +1,173 @@ +# Prototype explorer: next implementation increment + +Status: implemented and qualified; human layout review and PR integration remain pending. Updated 2026-09-11. The +[roadmap](prototype-explorer-roadmap.md) owns priorities; this document is the +executable plan for the next bounded increment. + +## Objective and baseline + +Make the existing static prototype explorer coherent to navigate and portable for +GBIF-labelled models: one global class-ID interpretation, browser-side names and +reference images, and predicted-species thumbnails integrated with the existing +map and inspector. + +Start from `feature/prototype-browser-inference` at `1eb2a32`, which includes +master `1d2dae0251989b067908a7c06c0701a890b11580`. Before implementation, check the +current master head and integrate any newer changes through the normal reviewed +branch workflow. Preserve unrelated work and the existing published release. +Use the assigned `prototype-browser-inference` worktree or a newly assigned +worktree following repository rules; do not use the older prototype-space worktree. + +The first browser inference and fixed-map t-SNE insertion are already implemented. +Preserve their model/preprocessing contracts, class order, scores, embeddings, +projection coordinates, pan/zoom, selection and saved-view functionality. No new +projection method, model re-export or inference engine is required for this increment. +Prefer the existing architecture where practical. Evaluate a framework if it offers +clear maintainability or usability benefits; explain the migration cost and +compatibility implications before committing to that direction. + +## Problems at the starting point + +- `report.html` resolves names through `/api/taxon/`. +- `photos.js` fetches albums through `/api/gbif/`; it and `thumbnails.js` probe + `/api/health`, and images use the Python `/gbif-image/` proxy. +- `gbif-names`, `photo-enabled` and `map-photos` independently imply GBIF + interpretation; `state.js` persists those independent controls. +- `inference.js` renders raw class IDs and scores without reference thumbnails. +- Model loading, prediction, exploration and numerous settings compete for space. + Earlier focused-layout work does not resolve the current combined interface. + +## Reviewable implementation slices + +### 1. Global class identity and shared browser GBIF access + +Add one model-scoped `classIdNamespace` setting with `generic` and `gbif` values. +Expose it once in shared settings. Numeric strings alone must not enable GBIF. +Allow explicit report/bundle metadata to supply a default, with the user's saved +choice taking precedence. Generic is the fallback when neither exists; synthetic +cases make no GBIF requests. Treat names and photos as presentation preferences, +not alternative namespace settings. + +Extend `state.js` rather than creating a parallel state store. Version the saved +state and migrate older views: an explicitly enabled legacy GBIF name/photo option +maps to GBIF mode; otherwise retain generic mode. Record this migration in tests +and documentation. Keep imported aliases usable in either mode. Use one shared +label resolver, with imported aliases, packaged names, resolved online names and +raw IDs in that order. Namespace/model changes cancel obsolete work and invalidate +render generations so late results cannot overwrite current labels or images. + +Put GBIF transport and metadata normalization in one logical `gbif.js` module. +Share it across name labels, selected-class cards, map thumbnails and prediction +cards. Reuse existing request deduplication, visible-item prioritization and photo +rendering where possible. Use session caching and bounded requests, seeded by an optional versioned GBIF +metadata snapshot packaged with the artifact. Precompute scientific names for the +model vocabulary so labels can render offline without thousands of initial calls. +Include snapshot provenance, retrieval dates and original taxon IDs; allow partial +snapshots and fetch missing entries on demand. Photo metadata (URLs and attribution) +may also be included, but does not imply that remote image bytes work offline. +Keep snapshot preparation optional and resumable; do not make live API availability +a prerequisite for export or build a general taxonomy service. + +Use public GBIF species and occurrence APIs directly from the browser. Verify +actual CORS access from a static HTTP origin before expanding implementation. +Select a browser-accessible thumbnail source, preserving creator, licence, +occurrence link and source URL. Respect the chosen endpoint's documented request +limits. Bound timeouts/retries and handle 429, unavailable images and offline +operation without blocking numerical views. Do not replace the Python dependency +with a mandatory third-party proxy. If image delivery needs a fallback, prove it +on the static host before claiming portability. + +Accepted-name/synonym resolution is display metadata: retain original class IDs +and model row ordering. Validate occurrence taxonomy before associating an image +with a predicted species; do not silently use unrelated or merely same-genus +images. Render external text safely and restrict media/link URL schemes. + +Keep existing Python endpoints compatible for local users during this increment, +but remove their use from browser name/photo features. The local weight picker +and optional server-side saved state remain separate Python capabilities. + +### 2. Prediction cards with species reference images + +Extend `inference.js` to use the shared label resolver and photo components. +Species results show name, original ID, unchanged score and a small reference +thumbnail with accessible attribution. Label these as reference examples, distinct +from the user's inference image. Load only visible results; share albums with map +and inspector consumers, and discard stale requests after new inference/model changes. + +Genus/family results use shared names but need not acquire species thumbnails. +Do not imply that an arbitrary descendant is the predicted species. Preserve +existing map linkage where meaningful and make any parent-rank selection behavior +explicit. Missing images, unknown taxa and offline operation retain useful ID/score +rows. Generic classes render normally with no GBIF calls. + +### 3. Cohesive layout using existing components + +Organize the interface around Explore and Predict, with shared Settings. Keep the +full-pane map available and the current query overlay linked to prediction results. +Place model loading and image selection together, with one clear progress/error +area. Group prediction results by rank; put advanced projection, thumbnail and +numerical controls behind labelled disclosure panels. Consolidate duplicated +controls rather than adding another toolbar. + +Preserve keyboard operation, visible focus, readable status, attribution access, +mobile usability and pan/zoom across navigation. Prepare desktop and narrow-screen +previews for human review before polishing. Do not redesign the underlying +analytical views or remove expert controls to simplify the first screen. + +## Acceptance and validation + +| Area | Required evidence | +| --- | --- | +| Static portability | Serve the exported viewer using a plain static server with no Python API endpoints. Names, map photos, inspector photos and prediction thumbnails work. Network assertions find no `/api/taxon`, `/api/gbif`, `/gbif-image` or GBIF health-probe dependency. Test the intended HTTPS hosting origin as well. | +| One setting | A single global namespace choice governs every consumer. Switching to generic prevents new GBIF requests and stale results; numeric generic IDs and synthetic cases remain generic. | +| Compatibility | Legacy saved views migrate deterministically; aliases, class order, hierarchy, scores and query positions stay unchanged. Browser storage failure does not prevent use. | +| Shared resources | Multiple consumers requesting one taxon share in-flight work and cached metadata. Navigation/inference replacement does not create unbounded queues or stale cards. | +| Failure handling | Mock timeout, 429, malformed responses, missing ranks/images, synonym records and offline mode. Useful labels and predictions remain; retries are bounded. | +| Predictions | Known species has a credited thumbnail; missing-image species has a stable placeholder; genus/family rows do not masquerade as species. Local query pixels are never sent to GBIF. | +| UX | Desktop and narrow-screen screenshots plus browser checks cover navigation, keyboard focus, settings, prediction cards, map selection, zoom retention and readable loading/error states. Obtain human review of the preview. | +| Performance | Record same-machine initial display, interaction responsiveness and visible-thumbnail request counts before/after. Do not resolve all 12,632 names or fetch all species albums at startup. | + +Browser acceptance for this increment targets the existing Chromium/WASM setup. +Narrow-screen checks use that browser; Firefox, Safari and other runtime backends +are deferred to the roadmap and are not completion gates for this goal. + +Extend the existing browser harnesses in `dev/prototype_space/` with mocked GBIF +responses for deterministic checks and a small separate live static-host smoke test. +Run `bash dev/check.sh static` and affected prototype Python tests; use the existing +inference harness to establish unchanged predictions and embedding placement. +If packaged assets change, run the installed-wheel smoke check. Follow repository +full-suite requirements if model loading or the validation harness scope expands. +Report live-network failures separately from deterministic regressions. + +## Completion and distribution + +Deliver focused normal commits for the state/client contract, prediction integration +and layout changes, with tests alongside each. Keep agent-only notes separate. +Update usage and browser guides, this plan and the roadmap. Integrate through a +reviewed PR against current master and validate the combined state. Prepare a +versioned static preview with provenance and asset hashes; do not overwrite the +published production release as a side effect of development. + +## Executable goal + +> Implement the portable viewer UX increment in +> `docs/prototype-explorer-implementation.md`, keeping compatibility with current +> master. Consolidate GBIF interpretation into one model-scoped global setting; +> replace Python-backed name/photo lookups with one shared browser GBIF client; +> show attributed predicted-species thumbnails; and reorganize existing controls +> into coherent Explore, Predict and Settings flows. Preserve numerical behavior, +> class IDs/order, saved-view compatibility and fixed-map t-SNE query placement. +> Complete the acceptance matrix, static-host browser proof, focused checks and +> human-reviewable desktop/mobile preview. Deliver focused commits and a reviewed +> integration PR, updated documentation and a versioned release candidate. Prefer +> the existing architecture where practical; a framework may be justified by clear +> maintainability or usability benefits, with migration cost and compatibility +> implications explained before committing to that direction. Do not +> change training/inference semantics or implement new geometry, +> require a proxy service, or overwrite the existing public release. Report any +> externally blocked check explicitly rather than claiming it passed. + +This goal is active. The [qualification record](../dev/prototype_space/portable-qualification.md) +contains passing check evidence. Remaining completion gates are human review, +the integration PR and candidate delivery. No token or time +budget is implied. diff --git a/docs/prototype-explorer-roadmap.md b/docs/prototype-explorer-roadmap.md new file mode 100644 index 00000000..04b79b38 --- /dev/null +++ b/docs/prototype-explorer-roadmap.md @@ -0,0 +1,99 @@ +# Prototype explorer roadmap + +Updated 2026-09-11. This is the maintained development order for the explorer; +research detail belongs in the [geometry review](hyperspherical-visualization.md), +and operation in the [usage guide](prototype-explorer.md). Later stages are +proposals, not active implementation commitments. + +## Delivered + +Implemented on `feature/prototype-space` (not yet integrated into the main +development branch; reusable-package baseline `4763041`): + +- Reusable package, `mt_explore`, browser weight selection, and portable export. +- Baseline distance matrices and Ward tree; direct log-tail diagnostics and + configurable display clipping; linked class and neighbourhood inspection. +- PCA and angular t-SNE, with measured neighbourhood retention. +- Focused feature layouts with related panels, responsive projection aspect ratio, + and retained class/plane/subtree state; verified at landscape, portrait and mobile sizes. +- GBIF photo labels, configurable footprints/culling, continuous arrival motion, + pan/zoom retention, and quick navigation exit fades. +- Optional asynchronous taxon names with visible-label priority and offline aliases; + z/chance-alignment probability views with explicit reference semantics. +- Display-density projection rendering and viewport-height focused panels; map + details and settings overlays prevent hover/status text from resizing the canvas. + Projection focus fills the viewport with collapsible floating navigation and controls. +- Saved views (browser storage, local-server persistence, JSON import/export), + completed-analysis caching, stage/elapsed progress and cancellable replacement jobs. +- Coalesced pan/zoom drawing and shared bounded thumbnail metadata lookups; + see the [reuse and responsiveness qualification](../dev/prototype_space/reuse-performance.md). + +Evidence: real checkpoint diagnostics match the preserved baseline; 84 browser +checks, 12 affected Python tests, motion checks, static checks, and installed-wheel +CLI/file-picker checks passed. Current extraction supports float32 linear +`Classifier` and `HierarchicalClassifier` heads; dense analysis needs several GB +for the 12,632-class case. These checks do not establish full-suite/GPU coverage. + +The latest viewer increment passed 150 browser assertions; the preceding +names/probability increment also passed +11 affected Python tests and log-tail reference and motion checks. Static checks +passed for the latest changes. These are scoped results, not a new full-suite audit. + +## Current browser delivery + +The subsequent `feature/prototype-browser-inference` increment (`1eb2a32`) +includes master `1d2dae0` and implements browser WASM inference, exported +preclassification embeddings and browser-side insertion into the fixed t-SNE map. +See the [browser guide](prototype-browser.md) and +[distribution plan](production-release-integration.md). + +The final production checkpoint replaces epoch 26 as the browser reference: +SHA-256 `174b9214bfea2df69e4f5c5d16afd841fec961db4274f3e6bf474cef9cab5e8a`. +It retains 12,632 species and 1,280-dimensional normalized prototypes. Earlier +checkpoint evidence above remains historical. Browser inference was checked on +Chromium/WASM with identical tensors and one real image; this is not broad browser +or accuracy qualification. Global diagnostic construction remains offline +preparation. The current portable-UX increment replaces the name/photo Python +transport with a shared browser client and optional packaged names; see its +[qualification evidence](../dev/prototype_space/portable-qualification.md). +Human layout review and integration remain pending. + +## Ordered next work + +| Order / status | Increment | Done when | +| --- | --- | --- | +| Delivered | Reliable repeated exploration and first browser inference/query placement. | Existing evidence is retained; limitations remain explicit in the browser guide. | +| **Implemented · Review pending** | **Coherent UI and portable GBIF integration.** One global class-ID setting, shared browser-side names/photos and predicted-species thumbnails. | Static hosting needs no Python name/photo service; all consumers obey one namespace setting; attributed prediction thumbnails and desktop/mobile navigation pass the implementation plan's acceptance checks. | +| Planned | Local packing: angular cap counts, neighbour-radius curves, mutual neighbours and taxonomy composition. | Radius/rank selection links the same original-space class set across views; calculations match exact references. | +| Planned | Single-anchor view, then bearings and two-anchor comparison. | Angular radii match direct calculation; degeneracies and non-anchor distortion are disclosed. | +| Later | Client-side global diagnostics and offline distribution. | Real bundles generate global views locally within measured resource bounds; packaged data and optional GBIF networking have explicit offline behavior. | +| Later · Compatibility | Alternate-browser qualification, starting with Firefox and then Safari. Separate from the current UI/GBIF goal. | Verify model loading, preprocessing/output parity, one-image inference, fixed-map t-SNE insertion, GBIF names/photos, saved state and core interactions on named browser versions; document capability limits. WebGPU qualification remains a separate decision. | +| Later · Research | Great-sphere slices, alternative layouts, checkpoint trajectories and empirical calibration. | Each proposal earns implementation through a bounded experiment and original-space validation. | + +The [next-increment implementation plan](prototype-explorer-implementation.md) +contains the executable goal, code boundaries, state migration and acceptance +matrix. The [standalone plan](prototype-browser-implementation.md) retains broader +inference/distribution contracts, not a competing priority order. + +UI cleanup is now an explicit priority, extending earlier focused-layout work in +response to the combined viewer's usability problems. Keep expert controls +accessible and preserve the full-pane map and human review of previews. +Take one bounded increment at a time. Integration into master requires review and +combined validation; do not overwrite the published production artifact during +implementation. + +## Rules for research and prioritization + +- Observed effective weights and existing distance functions are the baseline. + Synthetic cases test hypotheses and edge cases; they do not impose a packing prior. +- Preserve direct log-domain computation. Treat clipping as display configuration. + Alternative clustering inputs require explicit comparison with the retained tree. +- Do not infer spherical density or volume from projected density or slice area. +- Profile before replacing rendering or introducing approximations. Validate blocked, + sparse or sampled computations against exact results, disclosing approximation error. +- Broader head/dtype support needs an extraction contract and checkpoint parity tests. + It is a compatibility backlog item, not a reason to guess unsupported weights. + +Update this page with each completed increment or priority change: move delivered +work out of the queue, attach compact evidence, and revise the next step. Keep +experiment logs and detailed method comparisons in their existing linked documents. diff --git a/docs/prototype-explorer.md b/docs/prototype-explorer.md new file mode 100644 index 00000000..f3fa85b7 --- /dev/null +++ b/docs/prototype-explorer.md @@ -0,0 +1,181 @@ +# Prototype explorer + +Explore a mini_trainer classifier's learned prototype geometry through linked +class-distance matrices, dendrograms, spatial projections, and optional GBIF +class photos. The viewer uses the repository's effective-weight handling and +class-distance functions. Projections and movable image labels do not change +prototype coordinates or neighbour rankings. + +See the [compact development and research roadmap](prototype-explorer-roadmap.md) +for delivered capabilities and ordered next steps. + +Install the optional dependencies in your intended environment: + +```bash +uv pip install 'mini_trainer[explorer]' --torch-backend=auto +``` + +For an existing training environment, preserve its chosen PyTorch backend as +explained in the [installation guide](../README.md#installation). + +## Open a model + +```bash +mt_explore path/to/weights.pt +``` + +This opens a local browser page while the report is prepared. No output path, +class mapping, model architecture, or embedding dimension argument is required. +The information comes from the saved classifier metadata. The tool uses CPU +analysis and defaults to both angular t-SNE and PCA projections. + +To select a file using the browser instead: + +```bash +mt_explore +``` + +Choose **Model weights**, then **Open weights**. After preparation, click +**Open explorer**. The viewer's **Open another model** link returns to the picker. +The file is sent only to the local process, stored outside the served report +folder, and read with PyTorch's `weights_only=True` loader. The server binds to +loopback on an available port. Stop it with Ctrl+C when finished. Session files +are temporary; use **Save HTML report** to retain the portable numerical viewer. +Names and reference photos use the browser directly; no Python photo server is required. Live lookups and uncached images require network access. + +Preparation reports the current stage and elapsed time. **Cancel preparation** +stops the owned analysis process; selecting replacement weights starts a new job. +Cancelled or superseded jobs cannot publish over the latest result, and the last +completed explorer remains available after cancellation or failure. + +Completed analysis is cached by checkpoint content, analysis version and numerical +options. The default cache is `$XDG_CACHE_HOME/mini-trainer/prototype-analysis` +(or `~/.cache/mini-trainer/prototype-analysis`). It retains at most 2 GiB of completed +entries. A cache hit renders current viewer assets over the saved numerical data; +the launcher reports whether analysis was reused. Corrupt/incomplete entries rebuild. + +```bash +mt_explore weights.pt --cache-dir /path/to/analysis-cache +mt_explore weights.pt --no-cache +mt_explore --clear-cache --cache-dir /path/to/analysis-cache +``` + +**Saved view** provides JSON export/import and reset. Selection, case, projection, +pan/zoom, subtree, focused view, colour settings and thumbnail/name preferences +are saved automatically. Browser storage supports offline HTML; the local launcher +also saves state in the cache's `.view-state` directory so it survives a change of +localhost port. Clearing analysis entries does not clear saved views. Imported +state must match the model/case and supported version. Storage failures leave the +viewer usable with explicit state export. Transient thumbnail forces and image +files are not part of saved state. + +For SSH or an environment without a browser, use `--no-browser` and open the +printed URL through your usual local port forwarding. `--port` optionally fixes +the listening port. File selection reads from the browser's computer. + +The equivalent module command works without an installed console entry point: + +```bash +python -m mini_trainer.visualization.prototype_space path/to/weights.pt +``` + +## Focus a view + +Use **Focus view** on a feature panel, or the global **View** selector, to open +spatial projection, dendrogram, distance matrices, class images, or local +neighbourhood inspection. Each focus keeps related panels together: for example, +projection includes the neighbourhood profile, local inspector, and class photos. +Choose **All views** to return to the overview. + +Focused projection and dendrogram panels use the available viewport height as +well as width. Thumbnail settings and **Map details & credits** open as overlays, leaving the +map size fixed while hover credits and loading status change. The focused projection +fills the viewport edge to edge. **Explorer · view & class** and **Map controls** +are collapsible floating menus for navigation, selection, projection and thumbnail +settings. Related inspection panels remain below the map. Leaving projection +focus restores the regular toolbar and panel headings. +The spatial map adapts its aspect ratio while preserving equal geometric scale +on both axes; its raster follows display pixel density with readable screen-size labels. +Dendrograms use the wider layout with readable labels and internal scrolling on +narrow displays. Class selection, projection centre/plane, and the current tree +subtree survive view changes. Related panels wrap below the main feature. + +## Class names and chance alignment + +Open **Settings → Names and class identity** to enable aliases in selection, neighbourhood profiles, +dendrogram leaves, local matrix axes, and neighbour tables. Set the global **Class IDs** setting to **GBIF taxon IDs** when appropriate. +Name and photo visibility are separate display preferences. Names resolve asynchronously, +prioritizing visible labels and the selected neighbourhood; unresolved classes +retain their IDs. Online resolution requires GBIF access; packaged names work offline. +Search accepts IDs, already resolved names, or `row N` for a checkpoint row. + +For offline aliases, import JSON such as `{"1837646": "Taxon name"}`, or +`{"by_id": {"1837646": "Taxon name"}, "by_index": {"0": "Another name"}}`. +Aliases affect display only and never change checkpoint class ordering or IDs. + +The score selector switches between z-scores and **Chance-alignment p-value +(approx.)**, `q = Φ(−z)`. This is the approximate one-sided tail under independent +uniform directions on the unit hypersphere: small q means random directions +would rarely be this closely aligned. It is not a posterior probability of +similarity or a calibrated test of whether two learned classes differ. +The reference does not assume that the learned prototypes themselves are uniform. + +Probability formatting uses a direct log-tail approximation; the local matrix +retains the repository's stored log-tail values. Logarithmic colour limits remain +configurable, and tooltips retain unclipped values. Histogram bins retain their +original z positions, while the neighbourhood probability profile uses a log +axis. Ward linkage heights keep their original units. + +## Export without serving + +```bash +mt_explore path/to/weights.pt --export +``` + +The default output directory is `weights-explorer` in the current directory. +Use `--output /path/to/report` to choose another directory. Export writes +`explorer.html`, `report-data.json`, and `summary.json`; the HTML contains its +numerical data and scripts and can be opened offline. + +For programmatic generation: + +```python +from mini_trainer.visualization.prototype_space.launch import generate + +generate('weights.pt', 'report') +``` + +Optional controls: + +- `--pca-only`: skip angular t-SNE for a faster initial inspection. +- `--synthetic`: include independent vectors, planted groups, and algebraic + edge cases as explicitly labelled comparison cases. +- `--threads 4`: bound CPU analysis threads (default: 4). + +No dependencies are installed automatically and the source checkpoint is not +modified. Full pairwise analysis uses quadratic memory; a 12,632-class report +needs several GB of RAM and can take minutes. Synthetic cases probe the +calculations and rendering; they are not a prior for the learned geometry. + +## Supported weights and interpretation + +The loader currently supports floating-point **float32 linear heads** saved by +mini_trainer's `Classifier` and `HierarchicalClassifier`, including normalized +weight parametrizations and the checkpoint's class ordering. It reads the head +without constructing or downloading a backbone. Bias and norm ranges and the +checkpoint SHA-256 remain in report provenance. + +Other head families, quantized encodings, and files without class metadata are +rejected explicitly. They require a defined prototype-extraction contract; +the explorer does not guess tensor keys or substitute raw parametrizations. +The supplied file should be the model weights file, not optimizer/training state. + +See the [view guide](../dev/prototype_space/README.md#try-the-views), +[diagnostic API](prototype-diagnostics.md), and +[hyperspherical visualization review](hyperspherical-visualization.md) for +metric semantics, display clipping, projection distortion and thumbnail controls. + +## Browser inference + +See [browser inference](prototype-browser.md) for the optional ONNX bundle, local +image predictions, and fixed-map t-SNE/PCA query placement. diff --git a/docs/roadmap.md b/docs/roadmap.md index 6ca554bb..06b46edf 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -1,8 +1,20 @@ # Repository strengthening roadmap -The active quantization feature branch has a separate -[execution roadmap](quantization-roadmap.md) covering measured bottlenecks, -target-machine dependencies, integration gates and deferred work. +The quantization branch has been merged into master. Its +[execution roadmap](quantization-roadmap.md) remains the specialist backlog; +fully quantized training performance is not established by that merge. + +The completed four-GPU production campaign is assessed in the +[training workflow post-mortem and next-run plan](training-workflow-postmortem.md). +For the next training run, first deliver its bounded P0 workflow slice: durable +stage state and recovery, preallocated evaluation/export preparation, and separate +compute/storage qualification. Then measure concurrent staging before investing in +prepared shards. Preserve normal CLIs, operator overrides, figures and W&B. +These are proposed development priorities, not implemented capabilities. They fit +the safeguards, export, loading and evaluation boundaries below; defer another +large optimization matrix until it can change a specific production decision. + +The prototype explorer has a focused [development and research roadmap](prototype-explorer-roadmap.md). This is the implementation order. Each increment should leave the existing default training and prediction interfaces working and include its own validation evidence. diff --git a/docs/training-workflow-postmortem.md b/docs/training-workflow-postmortem.md new file mode 100644 index 00000000..67ec9fc2 --- /dev/null +++ b/docs/training-workflow-postmortem.md @@ -0,0 +1,392 @@ +# Production training workflow: post-mortem and next-run plan + +Status: proposed development plan; no implementation is authorized by this document. +Campaign: 10–11 September 2026. The quant branch has since been merged into master. + +## Executive assessment + +The campaign produced a useful model, completed four-GPU training with figures and +W&B, evaluated in-domain and independent expert data, exported floating and PTQ +ONNX artifacts, and packaged the evidence with verified checksums. The original +objective of faster, memory-efficient fully quantized training was not established. +These are separate outcomes: merging useful improvements does not qualify every +experimental backend or prove absence of regressions against a full master run. + +The largest opportunity for the next run is operational reliability and data access, +not another optimization matrix. Small warmed trials selected a viable compute +configuration but substantially underestimated first-pass storage costs. Repeated +manual handoffs, opaque status and late preparation of evaluation/export consumed +scarce allocation time. Preserve the flexible CLI/API design and human decisions; +make the routine steps repeatable, inspectable and recoverable. + +This assessment uses the operator-provided logs, metric tables and command results +from the campaign, checked against the current repository interfaces. It is not an +independent audit of the downloaded archive or a replay of the experiments. The +archive checksum was reported as passing. Its final public URL and immutable model +hashes must be attached when available; do not invent them from version names. +Older notes that say production was incomplete, or retain four loader workers as +the production setting, are superseded by this campaign evidence. + +## Outcomes and strength of evidence + +| Area | Observed result | What it establishes / does not establish | +| --- | --- | --- | +| Production | EfficientNetV2-S, normalized hierarchical head, size 384; 4 full B200 GPUs; batch 256/rank; FP16 AMP; model compilation; 32 loader workers/rank in the successful run | A functional recipe on this allocation, not an optimum across precisions, machines or models | +| Optional features | Optimizer compilation, explicit CUDA prefetch, INT8 training and EMA disabled | Deliberate scope reduction; EMA and native INT8 DDP remain unsupported | +| Training | 30 epochs, 17:34:23 total; training 16:27:22, evaluation timer 43:33; best model reported at epoch 30 | Finished within allocation; phase timers do not cover every logging/teardown cost | +| Final metrics | Train species micro accuracy 98.1645%; validation species/genus/family 94.2353% / 97.6693% / 99.5047%; finite recorded losses | Strong completed-run result; not a controlled comparison with the old master model | +| In-domain test | 632,913 images; baseline micro accuracy 94.21% / 97.64% / 99.50%; macro accuracy 92.94% / 97.00% / 98.48% | Test agrees closely with validation; no dataset-leakage audit was completed here | +| Expert test | 58,640 images, 522 species; baseline species micro accuracy 57.59% all labels, 66.74% known only | Substantial domain/vocabulary shift remains despite strong in-domain results | +| Figures/W&B | Full-head diagnostics and distributed logging completed throughout production | Required product functionality worked; cost and summary semantics still need improvement | +| Resume probe | All 4 ranks reported identical checkpoint hash, start epoch 3 and restoration of model/optimizer/scheduler/scaler | Controlled restoration worked; arbitrary interrupted stochastic continuation is not proven bit-exact | +| FP32 ONNX | Dynamic batches 1, 2, 4 passed at rtol=1e-4, atol=1e-4; largest observed absolute error 3.84e-5 | Synthetic-input numerical parity, not real-image end-to-end parity or target-GPU performance | +| ONNX PTQ | 128 training images; Percentile 99.9; Q/DQ graph validation and finite runtime smoke passed | Quantized ONNX artifact exists; retained quality and native integer execution remain unqualified; no quantized .pt was created | +| Retention | 23.71 GiB before ZIP; ZIP integrity check and subsequent archive checksum passed | Evidence was preserved; size, discoverability and clean-environment reuse need refinement | + +### Allocation timeline and where time was exposed + +The four-GPU allocation began around 17:24. The successful production training +started around 19:01 (inferred from the final duration), about 97 minutes later. +Training finished around 12:35 the following day; test inference was reported +complete at 16:11, evaluation output around 16:44, and archive verification at +17:13, shortly before the 17:24 expiry. These are operational timestamps, not a +profile attributing every minute to a specific component. Qualification had value; +not all of this interval was avoidable waste. Nevertheless, hours of post-training +storage/evaluation and last-minute packaging were clearly on the critical path. + +The reported source split contained 5,063,857 training images, 633,224 validation +images and 632,913 test images. Thirty training epochs divided by the reported +training timer imply about 2,564 images/s aggregate, or 641/rank, consistent with +the observed roughly 650 training images/s/GPU. Use the training split rather than +all six-million-plus source rows for runtime planning, and budget validation and +diagnostics separately. This cross-check also illustrates why units belong in the +report rather than being reconstructed from ETA afterward. + +### Compute selection: useful, but deliberately bounded + +Reported aggregate warmed training throughput from four-GPU qualification: + +| Batch per GPU | Images/s | Peak allocated bytes per reported maximum | Peak reserved fraction | +| --- | ---: | ---: | ---: | +| 32 | 1,059.648 | 8,332,350,464 | 0.059 | +| 64 | 1,741.601 | 15,640,267,776 | 0.098 | +| 128 | 2,272.305 | 30,251,580,416 | 0.178 | +| 256 | 2,517.928 | 59,508,456,960 | 0.324 | + +Increasing batch 128 to 256 gained about 10.8% throughput with nearly twice the +allocated memory. Larger batches were not exhaustively tested. Spare memory is an +opportunity, not evidence that a further increase pays off. Use equal-memory-budget +and time-to-quality comparisons where appropriate, not just equal batch size. +Changing global batch also changes optimizer updates per epoch and schedule behavior. +The qualitative impression of improved prototypes/generalization at larger batches +is useful feedback, not a controlled convergence result. + +Earlier single-GPU trials repeatedly found model compilation reduced later-epoch +training time by roughly one third and allocated memory by roughly one third. +Optimizer compilation alone or combined with model compilation did not improve +those steady-state timings and added substantial cold startup. Explicit prefetch +showed no convincing additional gain. INT8 combined runs timed out and were not +qualified. Keep these negative results so that the next run does not repeat the +same matrix without a new hypothesis, implementation change or target requirement. +FP16 is the tested baseline; BF16 and other compiler modes were not eliminated by +a comprehensive comparison. + +## Failure modes and lessons + +### 1. Storage behavior invalidated small-subset extrapolation + +Full-data training and test inference initially stalled while data workers had low +CPU use and waited in `D` state / `folio_wait_bit_common`. A 32-step storage window +spent about 146–150 of 211 seconds waiting for the loader across ranks. Test +inference reached only 69/2,473 batches after 43 minutes. GPU utilization snapshots +sometimes looked high despite poor progress; they did not identify the bottleneck. + +The mounted filesystem was WEKA. Repeated reads of the same expert subset went +from hundreds of seconds to below one second. The operator observed improved cold +staging with 512 readers, and a later disjoint-sample read sweep favored 512 readers +(~117 images/s confirmation median) over lower concurrency. Another workload with +larger files provisionally favored 64. These are workload/cache-state results, not +universal worker defaults. Strong evidence supports cache-state-dependent storage +latency; the exact client/server cache or network mechanism was not instrumented. + +The successful production run used 32 loader workers/rank and eventually stabilized +near 650 training images/s/GPU and 2,000–2,500 evaluation images/s/GPU. The long +warmup within the run supports a cache-related explanation; it does not quantify a +separate causal percentage for each storage/concurrency effect. Practical mitigation +need not wait for a filesystem-internal diagnosis. + +Lessons: + +- Separate process workers for decode/augmentation from concurrent outstanding + encoded-byte reads. More processes are not the only way to hide blocking I/O. +- Measure first-pass and repeated-pass behavior. Disjoint paths avoid direct sample + reuse but cannot guarantee cold shared caches. Never drop global caches on a + shared allocation to manufacture a benchmark. +- Start with bounded but genuinely high concurrency candidates when latency is + evidenced; do not spend the allocation creeping from 4 to 8 to 16 threads. +- Separate reader startup, first result, steady work and drain/cancellation time. + Time-limited trials can be informative without completing their target file count. +- Report images/s and bytes/s, file-size distribution, errors, actual concurrency + and sample coverage. A sampled optimum is a provisional operating point. +- Prefer optional staging/encoded-byte preparation. Decoded caching of millions of + images is not a safe default even on a high-RAM node. + +### 2. Orchestration was too easy to interrupt and too hard to resume + +Examples included relative commands launched from the wrong directory, missing +inference YAML files, torchrun interpreting `--run` as its own abbreviated option, +preparation tied to historical commits, and cancelled stages refusing an existing +directory. A trial had final weights and finite metrics but was marked timed out +when the shared wall budget expired. This is neither evidence of failed training +nor permission to relabel the whole process successful: export, logging, teardown +or restore checks may still be incomplete. + +The package pin and checkout revision sometimes differed intentionally, while +`PYTHONPATH` overrides added a third possible source identity. Hand-built commands +made that hard to see. The late archive command also lost its terminal connection; +file descriptors could not recover previously lost stdout. Durable files and a +completion checksum, rather than terminal appearance, ultimately established success. + +The necessary response is a small extension to existing harness state and commands, +not a general workflow engine. Record phase completion separately from process exit, +retain partial results, and make retry/resume/archive-incomplete actions explicit. +A checkpoint's existence is not a replacement for a loss audit or clean shutdown. + +### 3. Qualification tested the right features, but not early enough in combination + +Figures were initially disabled to avoid expensive rendering, although they were a +required training diagnostic. At larger taxonomy size the dendrogram caused long +validation pauses and recursion warnings. Confusion/dendrogram improvements allowed +production diagnostics to remain on: at 12,632 species, confusion generation was +about 8–13 seconds and warmed dendrogram rendering about 12–13 seconds. First label +resolution added roughly a minute in one run; subsequent resolution was much faster. + +Whole-matrix confusion patterns and prototype organization matter to the operator. +Do not replace them with a few selected classes or disable them as the standard +performance fix. Preserve visual interpretation while bounding file/display costs. +Which hierarchy levels render must be an explicit option recorded in the resolved +configuration; the unexpected level-0-only output showed that defaults were opaque. + +The production smoke should exercise the actual CLI, full head dimensions, figures, +W&B, validation and save/reload on the intended topology. A small subset tests those +contracts; a separate bounded storage probe tests uncached access. Neither replaces +the other. Four GPUs passing does not establish eight-GPU behavior. + +### 4. Numerical and compiler warnings need a triage budget + +Early validation loss NaNs appeared in both branch baselines and optional-feature +runs; later epochs could be finite and the final production loss audit passed. +This weakens attribution to prefetch or quantization but does not prove the NaNs +were harmless. Keep a cheap first-occurrence record: phase/batch/sample identities, +input/output/loss finiteness, dtype and enabled features. Capture a bounded replay +only when triggered; avoid a permanent expensive debug path. + +Observed compiler issues were stochastic-depth specialization/recompile limits, +hierarchy initialization using `Tensor.item()`, and DDP gradient/bucket stride +mismatches. They are performance targets, not demonstrated model-quality failures. +Prioritize a change only after a representative trace shows repeated fallback, +recompilation or copy cost. Do not globally raise limits, suppress warnings or change +numerics merely to make the log clean. Keep EMA repair outside the next-run critical +path unless the operator explicitly needs EMA. + +### 5. Inference ingestion and evaluation preparation lagged behind training + +Folder inference unnecessarily depended on training-oriented index construction +and model-vocabulary membership. Valid expert species could be absent from the +model. A taxonomy rank-count/index mismatch was initially plausibly attributed to +network resolution. The bounded discovery fix was useful; broader ingestion should +keep discovery, taxonomy resolution, split policy and model indexing separate. + +A model-vocabulary filter restricts candidate predictions, not the validity of +ground-truth labels. Future source formats must reuse this separation for both +training and inference. External taxonomy access needs cache/provenance and clear +transport-versus-local-mapping errors; a hardcoded socket timeout is not a batch +or job deadline. + +Inference should normally require input, output and weights. Derive shape, +preprocessing and head/collector behavior from saved metadata, with explicit +operator overrides and actionable errors for ambiguous legacy metadata. Generate +resolved config for inspection; do not fill YAML with guessed defaults. + +Staging eventually made expert inference finish in 2:49 and full test inference in +30:02, after slow source reads had dominated. Prepare the evaluation plan before +training and overlap independent preparation where resource budgets allow. Do not +launch competing cold scans blindly. Pin mini_metrics in its own compatible Python +environment and preserve its pretty tables plus machine-readable outputs. + +### 6. Evaluation success and deployment readiness are different gates + +The expert set's 16 unseen species accounted for 8,042 images (13.71%) and errors; +five of those species contributed 84.3% of unseen-species errors. Error concentration +also existed among known species. This supports the operator's interpretation of +uneven domain/vocabulary effects. Geographic candidate lists are promising opt-in +priors, but their provenance/scope and excluded true labels must remain visible. +No regional-filter accuracy gain was established in the supplied results. + +Retain all-label/known-only and unfiltered/threshold-optimized reports. Clarify +micro versus macro, rank, aggregation period and abstention denominator. In-domain +optimized species accuracy was 95.73% at 97.46% coverage; expert known-only optimized +species accuracy was 92.25% at 53.66% coverage. These are not equivalent operating +points. Threshold search/split semantics and the distinction between an applied +threshold and a subsequently reported optimum must be documented before rollout. +Choose deployment thresholds using suitable calibration data and evaluate them +frozen; current optimized test reports remain exploratory. Empty-abstention summary +NaNs are distinct from nonfinite model outputs or loss. + +FP16 AMP training did not imply an FP16 ONNX export. FP32 export required an explicit +absolute-tolerance adjustment after small near-zero discrepancies; tolerances and +errors were retained. Do not silently weaken defaults. Add real-image parity and +rank/top-k changes alongside absolute errors, with predeclared acceptable limits. +PTQ happened directly on ONNX, so no quantized PyTorch artifact exists. Q/DQ node +counts and runtime loadability are not proof of integer kernel placement, target +speed or retained accuracy. These must be separate candidate acceptance checks. + +### 7. Packaging worked, but happened too late and bundled too much together + +The verified archive preserved a large set of results, logs and figures. Packaging +was improvised near expiry, and the chosen public bundle also contained a resume +checkpoint and extensive history. Use one relocatable directory but distinguish a +small deployment subset, reproducibility evidence and optional training archive. +Never omit ONNX external tensors. Retain omissions in the manifest; calibration +manifests referring to omitted tensors must not imply self-contained replay. + +A publication command should inventory intended files, sizes and public/private +scope before copying; omit credentials, caches and raw training images by default. +Record package revision, harness revision, weights hash, splits, preprocessing, +class mapping, evaluation versions, export tolerances and target qualification. +Write logs to disk from process start and publish completion atomically. Checksums +establish integrity, not accuracy, data provenance or deployment correctness. + +## Prioritized development plan + +Effort below is a planning estimate for implementation plus focused validation, +not a promise: S = roughly 1–2 developer days; M = several days; L = one or more +weeks with integration work. Re-estimate after inspecting the existing boundary. +Prefer the smallest vertical slice that removes an observed failure. + +| Priority | Increment | Value / effort | Done when | +| --- | --- | --- | --- | +| P0 | Durable stage state and resolved execution plan | High reliability, S–M | A fresh job can inspect, run, stop, retry and resume each existing stage; logs survive disconnect; completed stages are reused only after input/hash validation | +| P0 | Prepare evaluation/export/publication before allocation | High saved allocation time, S–M | One tiny installed-CLI fixture runs train → predict → metrics → export → package; commands, required paths and dependencies are checked before expensive work | +| P0 | Separate compute qualification from storage qualification | High decision value, S | A compact report distinguishes startup, warm training, first-pass I/O, validation/figures and teardown; it states scope and units and supplies a bounded next action | +| P1 | Reusable concurrent read/staging calibration | High on affected storage, M | Startup and work budgets are separate; incomplete trials remain informative; bounded cancellation/recovery, disjoint sampling, file sizes and actual concurrency are reported; operator override survives | +| P1 | Optional prepared encoded-byte dataset round trip | Potentially high recurring gain, L | Existing input compatibility matrix and sampler/DDP contracts pass; end-to-end savings exceed preparation/maintenance cost on representative data | +| P1 | Real-image export and candidate evaluation | High rollout confidence, M | Same preprocessed held-out inputs compare PyTorch, floating ONNX and optional PTQ; score/rank changes, quality, coverage, placement and target resource evidence are retained | +| P2 | Figure/metadata caching and explicit figure policy | Medium operational gain, S–M | Whole-matrix diagnostics and requested hierarchy levels remain inspectable; first/warm rendering and file sizes are recorded without repeated taxonomy resolution | +| P2 | Targeted compiler/numerical fixes | Conditional gain, M per demonstrated issue | A bounded reproducer demonstrates the cost/failure and a paired check verifies the fix without changing checkpoint or prediction contracts | +| Deferred | Native INT8 DDP, EMA, expanded precision/compiler matrices, automated allocation, universal inference CLI | Uncertain value, M–L | A concrete requirement or measured bottleneck justifies a separately reviewed experiment; none blocks the baseline run | + +### A. Next-run minimum: complete the P0 vertical slice first + +Build on `dev/ucloud/{setup.sh,compare.py,scaling.py,production.py}`, the evaluation +scripts and existing ONNX tools. Do not replace the public training/inference APIs. +A small shared run manifest should reference existing stage outputs rather than +copy their schemas into another database. Record stage input identities, resolved +command/config, start/end, output/log paths, exit cause and completion checks. +Differentiate preparation, compilation, training, validation/figures, checkpoint, +logger synchronization and teardown. Start with phases that are already observable. + +Use absolute paths in generated commands, explicit interpreter selection, package +and harness identities, and inspected CLI-over-YAML precedence. Keep W&B's existing +interactive authentication in the foreground. No secrets in manifests. Pin an +accepted master revision; do not embed a permanent dependency on the old quant +branch name or four-GPU topology in otherwise reusable planning helpers. + +Define an allocation deadline separately from stage limits, compilation allowance +and cleanup reserve. Human pauses consume allocation time but must not masquerade +as model regressions. Estimate feasible epochs from measured full-data throughput, +actual training split count, validation cost and reserve; do not silently alter the +learning-rate schedule to fit. At a supported checkpoint boundary, give the operator +an explicit continuation/stop decision. Do not promise exact arbitrary-batch resume +without the necessary RNG/sampler contract. + +Acceptance should include interruption after preparation, failed child exit, timeout +after checkpoint save, logger teardown failure and re-running a completed stage. +Use tiny fixtures and existing integration tests, not a second production matrix. +A single topology smoke on a changed target remains necessary. + +### B. Storage work: validate the cheap mitigation before building shards + +First improve the existing calibration/staging helper and, if needed, add bounded +encoded-byte read-ahead inside the current reader boundary. Bound requests and +bytes, preserve sample order, propagate failures with identities, and avoid repeating +large Python metadata per reader. Do not conflate read concurrency, DataLoader +processes and CUDA prefetch. Keep direct filesystem loading available. + +Only then implement the optional prepared-data backend already specified in the +[main roadmap](roadmap.md#optional-dataset-preparation-for-scalable-loading). +Start with byte-preserving indexed shards and explicit manifests. Defer distributed +cache eviction services and alternative shuffle policies until measured need. +Prepared formats must support all existing cases or explicitly remain an opt-in +partial prototype; they must never silently redefine splits, labels or sample order. + +Decision rule: report total preparation plus expected training/evaluation time and +break-even reuse count. Prefer a modest throughput gain delivered simply over a +complex backend that saves less than its preparation cost for the intended run. +Higher-concurrency staging may be enough for an immediate run; repeated campaigns +on the same corpus strengthen the case for persistent preparation. + +### C. Keep human decisions first-class + +The proposed workflow exposes `plan`, per-stage execution, compact `status`, and +explicit retry/resume actions; exact command spelling is an implementation choice. +Each stage can still be run through the normal CLI. The human can inspect figures, +change a proposed batch/worker count, select a candidate, pause or reject promotion. +Changes produce a new resolved configuration and identity; completed unrelated +stages are retained rather than overwritten or automatically rerun. + +Operator decisions are required at scientific boundaries: dataset/split choice, +global batch and schedule, geographic priors, thresholds, acceptable PTQ quality +loss and production promotion. Routine bounded retries and report generation do +not require repeated permission prompts. Do not turn every warning into a blocker. +Expose reason, scope and recovery command when intervention really is required. + +The next-run sequence is: prepare the plan and tiny end-to-end fixture before +allocation; inspect the actual node/storage; perform one production-feature smoke +and bounded storage calibration; choose a batch using existing evidence plus at +most a few useful candidates; verify save/reload; launch production in the same +allocation; evaluate and export at completion; finalize durable artifacts. Preserve +manual allocation and permit an explicit decision to skip optional experiments. + +## Experiment and validation budget + +- Every experiment states the decision it changes, baseline, measurement scope, + maximum elapsed/allocated GPU time, and stop criterion before launch. +- Reuse the qualified floating recipe unless hardware, code or workload changes + invalidate it. Run a full old-master training comparison only as a separately + funded scientific question, not a prerequisite for ordinary deployment. +- For the first batch/worker search, set a small candidate budget. Stop when gains + are within observed noise or too small to affect run completion. Expand only with + evidence and an operator decision; do not exhaust a Cartesian product. +- A single null measurement is not proof of equivalence. Repeat only close decisions + or unexpected failures that matter; multiple seeds are for quality claims, not + every operational smoke. Keep sample-cache caveats in the report. +- Preserve figures and W&B in qualification. Report end-to-end time as well as + compute-only timing so optimizations cannot hide costs in validation or shutdown. +- Default PR checks should exercise contracts with tiny fixtures. Report test + durations, consolidate duplicated behavior checks, and reserve real-backbone, + GPU, full-scale storage and performance experiments for the relevant changes. + Do not weaken numerical assertions simply to get green CI. +- Continuous benchmarks should use small representative jobs, not large nodes. + Allocation credentials remain separate from candidate code and publishing; + automatic cloud provisioning is not part of this next-run plan. + +## Success criteria and deliberately open questions + +The next campaign should require no hand-edited browser YAML, no reconstruction of +missing commands from chat, and no ambiguity about which stage finished. It should +produce a usable selected-model bundle even if optional PTQ or an upload fails. +Measure operator interventions, allocation time before first productive training, +first-pass/warm throughput, evaluation turnaround, failed/repeated stages and final +artifact size. Set concrete budgets from the next dataset/node plan; the current +record does not support a universal target percentage or preparation-time guarantee. + +Still open: unbiased convergence comparisons across batch sizes/precisions; exact +resume semantics under stochastic training; real-image exported-model parity; +PTQ quality and target performance; deployment threshold policy; regional prior +provenance; whether storage preparation amortizes for the next corpus; and the +published immutable artifact URL. None should be reported as solved by this plan. + +The existing [quantization roadmap](quantization-roadmap.md) remains the specialist +backlog for quantized performance and deployment work. This document supplies the +operational order for the next production campaign, not a competing feature matrix. diff --git a/mini_trainer/export.py b/mini_trainer/export.py index 535f9ed4..28984f60 100644 --- a/mini_trainer/export.py +++ b/mini_trainer/export.py @@ -20,6 +20,7 @@ def main( preprocessing: dict | None = None, model_args: dict | None = None, dynamic_batch: bool = True, + include_embeddings: bool = False, batch_size: int = 2, reference_device: str = "cpu", materialize_int8_training: bool = False, @@ -56,6 +57,7 @@ def main( output, preprocessing=preprocessing, dynamic_batch=dynamic_batch, + include_embeddings=include_embeddings, checkpoint_sha256=checkpoint_hash, reference_device=reference_device, ) @@ -74,6 +76,7 @@ def run(): parser.add_argument("--input-shape", type=int, nargs="+", help="Preprocessed input dimensions, excluding batch (usually C H W).") parser.add_argument("--preprocessing", type=Path, help="JSON deployment recipe for preprocessing outside the graph.") parser.add_argument("--model-args", type=json.loads, help="JSON constructor arguments for the existing architecture loader.") + parser.add_argument("--include-embeddings", action="store_true", help="Also export the linear head preclassification embedding.") parser.add_argument("--static-batch", action="store_false", dest="dynamic_batch", help="Export a fixed batch size.") parser.add_argument("--batch-size", type=int, default=2, help="Example/static batch size (default: 2).") parser.add_argument("--reference-device", default="cpu", help="PyTorch parity device; native INT8 training requires cuda.") diff --git a/mini_trainer/logging/core.py b/mini_trainer/logging/core.py index 50d55ae9..52f869ff 100644 --- a/mini_trainer/logging/core.py +++ b/mini_trainer/logging/core.py @@ -1141,7 +1141,7 @@ def figures(self, model: nn.Module | None): with main_process_first(): if get_rank() == 0 and model is not None: - cdm_figs = plot_class_distance_matrix(model) + cdm_figs = plot_class_distance_matrix(model, log_domain=True, log_range=(-8 * math.log(10), 0.0)) for lvl, fig in enumerate(cdm_figs): self.add_figure(f"Class distance matrix/lvl{lvl}", fig) try: diff --git a/mini_trainer/modeling/__init__.py b/mini_trainer/modeling/__init__.py index 388ff728..b678e7c4 100644 --- a/mini_trainer/modeling/__init__.py +++ b/mini_trainer/modeling/__init__.py @@ -11,7 +11,7 @@ predict, ) from .context import EmbeddingContext, SupervisionContext -from .distance import class_distance, class_similarity +from .distance import class_distance, class_log_similarity, class_similarity from .ema import EMATeacher, ema_lambda_per_update from .mask import mask_classifier, set_classification_mask from .prior import get_prior_method, prior_from_labels @@ -40,6 +40,7 @@ "SupervisionContext", "class_distance", "class_similarity", + "class_log_similarity", "mask_classifier", "set_classification_mask", ] diff --git a/mini_trainer/modeling/distance.py b/mini_trainer/modeling/distance.py index 45d13631..02ec7b35 100644 --- a/mini_trainer/modeling/distance.py +++ b/mini_trainer/modeling/distance.py @@ -26,6 +26,34 @@ def class_similarity(model: nn.Module, cdf: bool = False): return [_class_similarity(w, cdf=cdf) for w in W] +@torch.no_grad() +def class_log_similarity(model: nn.Module, complement: bool = False): + """Return float32 log probabilities without materializing the Normal CDF. + + Uses the existing prototype z-score. By default returns log Phi(z), with + diagonal 0. With complement=True returns log Phi(-z) = log(1 - Phi(z)), + with diagonal -inf (the exact zero self-dissimilarity). The complementary + log representation retains strongly aligned pairs even when Phi(z) rounds + to 1. No probability floor is applied. This is a representation for analysis + and plotting, not a replacement distance to pass directly into linkage. + + Like the existing diagnostics, returns one matrix per prototype level and + does not propagate gradients. Disable enclosing autocast for the diagnostic + computation; no float64 promotion is needed. + """ + weights = classification_module(model).last_layer_weights + if isinstance(weights, torch.Tensor): + weights = [weights] + results = [] + for weight in weights: + with torch.autocast(device_type=weight.device.type, enabled=False): + z = _class_similarity(weight, cdf=False) + log_probability = torch.special.log_ndtr(-z if complement else z) + log_probability.fill_diagonal_(-torch.inf if complement else 0.0) + results.append(log_probability) + return results + + @torch.no_grad() def class_distance(model: nn.Module, eps: float | None = None): return [ diff --git a/mini_trainer/modeling/onnx.py b/mini_trainer/modeling/onnx.py index 29e947cf..c329c2ef 100644 --- a/mini_trainer/modeling/onnx.py +++ b/mini_trainer/modeling/onnx.py @@ -69,6 +69,32 @@ def forward(self, images): return tuple(_flatten(self.model(images))) +class _PredictionEmbeddings(nn.Module): + """Keep the actual prediction forward and expose its head input geometry. + + The backbone runs once. The small eval-only preclassification transform is + replayed from its captured input, without activating training contexts. + """ + + def __init__(self, model): + super().__init__() + self.model = model + heads = [(name, module) for name, module in model.named_modules() if isinstance(module, Classifier)] + if len(heads) != 1 or type(heads[0][1]).__name__ not in {"Classifier", "HierarchicalClassifier"}: + raise ValueError("Embedding export requires one Classifier or HierarchicalClassifier linear head.") + self.head_path = heads[0][0] + self.register_buffer("head_input", None, persistent=False) + heads[0][1].register_forward_pre_hook(self._capture) + + def _capture(self, module, inputs): + self.head_input = inputs[0] + + def forward(self, images): + predictions = self.model(images) + embeddings = self.model.get_submodule(self.head_path).preclassification(self.head_input) + return {"predictions": predictions, "embedding": embeddings} + + def _dependencies(): try: import onnx @@ -111,6 +137,7 @@ def export_onnx( *, preprocessing: dict[str, Any] | None = None, dynamic_batch: bool = True, + include_embeddings: bool = False, verification_inputs: Sequence[torch.Tensor] = (), output_names: Sequence[str] | None = None, opset_version: int = 18, @@ -158,6 +185,8 @@ def export_onnx( translations[torch.ops.mini_trainer.scaled_int8_mm.default] = scaled_int8_mm sample = example_input.detach().cpu().clone() reference = _copy_for_export(model, sample.dtype, reference_device) + if include_embeddings: + reference = _PredictionEmbeddings(reference) if native_int8: # Tensor-subclass decomposition must not assign requires_grad to codes. reference.requires_grad_(False) @@ -180,6 +209,8 @@ def export_onnx( or any(not isinstance(name, str) or not name or name == "images" for name in names) ): raise ValueError("output_names must contain one unique nonempty name per output tensor.") + if include_embeddings and output_names is None: + names[-1] = "embedding" structure = _structure(observed, iter(names)) classifiers = [ {"module": name, "type": class_path(module), "metadata": _json_value(module.metadata)} @@ -262,7 +293,7 @@ def export_onnx( for name, value in zip(names, tensors) ], "output_structure": structure, - "output_semantics": "model_eval_forward", + "output_semantics": "predictions_and_preclassification_embedding" if include_embeddings else "model_eval_forward", "quantized_training_forward": native_int8, "classifiers": classifiers, "preprocessing": {"in_graph": False, "recipe": preprocessing, "requires_configuration": preprocessing is None}, diff --git a/mini_trainer/visualization/plot.py b/mini_trainer/visualization/plot.py index 12f0f9fc..9ed05721 100644 --- a/mini_trainer/visualization/plot.py +++ b/mini_trainer/visualization/plot.py @@ -5,11 +5,11 @@ import torch from matplotlib import pyplot as plt from matplotlib.backends import backend_agg -from matplotlib.colors import LogNorm +from matplotlib.colors import LogNorm, Normalize from torch import nn from torchvision.transforms.functional import resize -from mini_trainer.modeling import class_similarity +from mini_trainer.modeling import class_log_similarity, class_similarity # --- Constants --- MIN_DISPLAY_DIM_HEATMAP = 500 @@ -19,7 +19,7 @@ # --- Helper: Matrix Aggregation --- -def _aggregate_matrix_max(matrix: np.ndarray, block_shape: tuple[int, int]) -> np.ndarray: +def _aggregate_matrix_max(matrix: np.ndarray, block_shape: tuple[int, int], pad_value: float = 0) -> np.ndarray: """Aggregates matrix by taking the maximum in blocks. Handles non-divisible shapes by padding. @@ -32,7 +32,7 @@ def _aggregate_matrix_max(matrix: np.ndarray, block_shape: tuple[int, int]) -> n padded_matrix = matrix if pad_rows > 0 or pad_cols > 0: - padded_matrix = np.pad(matrix, ((0, pad_rows), (0, pad_cols)), mode="constant", constant_values=0) + padded_matrix = np.pad(matrix, ((0, pad_rows), (0, pad_cols)), mode="constant", constant_values=pad_value) new_rows, new_cols = padded_matrix.shape target_rows, target_cols = new_rows // block_rows, new_cols // block_cols @@ -42,7 +42,7 @@ def _aggregate_matrix_max(matrix: np.ndarray, block_shape: tuple[int, int]) -> n # --- Helper: Matrix Scaling --- -def _get_scaled_matrix_for_display(mat: np.ndarray) -> np.ndarray: +def _get_scaled_matrix_for_display(mat: np.ndarray, pad_value: float = 0) -> np.ndarray: """Resizes matrix: downscales then upscales to fit display dimension constraints.""" processed_mat = mat orig_rows, orig_cols = mat.shape @@ -52,7 +52,7 @@ def _get_scaled_matrix_for_display(mat: np.ndarray) -> np.ndarray: block_c = math.ceil(orig_cols / MAX_DISPLAY_DIM_HEATMAP) if orig_cols > MAX_DISPLAY_DIM_HEATMAP else 1 if block_r > 1 or block_c > 1: - processed_mat = _aggregate_matrix_max(processed_mat, (block_r, block_c)) + processed_mat = _aggregate_matrix_max(processed_mat, (block_r, block_c), pad_value=pad_value) curr_rows, curr_cols = processed_mat.shape @@ -108,6 +108,54 @@ def chunks(): return rgb, norm, norm_vmin, norm_vmax +def _generate_log_heatmap_rgb_array(display_mat, min_val_display, cmap_name, percent, log_range=None): + """Map natural-log inputs directly to colours; never exponentiate a matrix.""" + if min_val_display is not None and min_val_display < 0: + raise ValueError("min_val_display must be nonnegative for log inputs") + if log_range is not None and (len(log_range) != 2 or not all(math.isfinite(x) for x in log_range) or log_range[0] >= log_range[1]): + raise ValueError("log_range must contain two finite, increasing natural-log bounds") + threshold = math.log(min_val_display) if min_val_display is not None and min_val_display > 0 else -np.inf + + def chunks(): + rows = max(1, 262144 // max(1, display_mat.shape[1])) + for start in range(0, len(display_mat), rows): + values = np.ma.masked_invalid(display_mat[start : start + rows]) + yield start, np.ma.masked_less_equal(values, threshold) + + minimum, maximum = float("inf"), -float("inf") + for _, values in chunks(): + if values.count(): + minimum = min(minimum, float(values.min())) + maximum = max(maximum, float(values.max())) + rgb = np.zeros((*display_mat.shape, 3), dtype=np.uint8) + if not np.isfinite(minimum): + return rgb, None, 0.0, 1.0 + vmin = math.floor(min(-1, minimum / math.log(10))) * math.log(10) + vmax = 0.0 if percent else maximum + if vmin >= vmax: + vmax = vmin + math.log(10) + if log_range is not None: + vmin, vmax = log_range + norm = Normalize(vmin, vmax, clip=log_range is not None) + cmap = mpl.colormaps[cmap_name].copy() + cmap.set_bad((0, 0, 0) if cmap_name == "magma" else (1, 1, 1), alpha=1) + for start, values in chunks(): + rgb[start : start + len(values)] = cmap(norm(values), bytes=True)[:, :, :3] + return rgb, norm, vmin, vmax + + +def _log_colorbar_ticks(vmin, vmax, max_ticks, percent): + """Format log probabilities without underflowing a linear tick value.""" + ticks = np.linspace(vmin, vmax, max(0, max_ticks)).tolist() + labels = [] + for tick in ticks: + exponent = tick / math.log(10) + (2 if percent else 0) + if abs(exponent - round(exponent)) < 1e-6: + exponent = round(exponent) + labels.append(f"10^{exponent:g}" + ("%" if percent else "")) + return ticks, labels + + # --- Helper: Colorbar Ticks --- def _get_colorbar_ticks_and_labels(norm_vmin: float, norm_vmax: float, max_ticks: int, percent: bool) -> tuple[list[float], list[str]]: """Generates tick values and labels for the colorbar.""" @@ -172,7 +220,7 @@ def _get_colorbar_ticks_and_labels(norm_vmin: float, norm_vmax: float, max_ticks # --- Helper: Colorbar Array Generation --- def _generate_colorbar_rgb_array( - norm_obj: mpl.colors.LogNorm, + norm_obj: mpl.colors.Normalize, cmap_name_str: str, tick_list: list[float], tick_label_list: list[str], @@ -225,10 +273,17 @@ def plot_heatmap( percent: bool = True, min_val_display: float | None = None, colorbar: bool = True, + *, + log_input: bool = False, + log_range: tuple[float, float] | None = None, ): """Plots a high-resolution confusion matrix using NumPy and Matplotlib. - Returns a combined RGB NumPy array (heatmap + colorbar), or None for empty input. + Returns a combined RGB NumPy array (heatmap + colorbar). + With log_input=True, inputs are natural logs and stay logarithmic through + aggregation, normalization and colourbar formatting. min_val_display remains + a linear threshold. -inf represents zero and is masked; padding uses -inf. + log_range optionally clips colours to natural-log bounds, not input values. """ if isinstance(mat, torch.Tensor): mat = mat.cpu().detach().float().numpy() @@ -238,10 +293,17 @@ def plot_heatmap( return img # 1. Scale matrix for display - display_mat = _get_scaled_matrix_for_display(mat) + display_mat = _get_scaled_matrix_for_display(mat, pad_value=-np.inf if log_input else 0) # 2. Generate heatmap RGB array - heatmap_rgb_array, norm_obj, vmin, vmax = _generate_heatmap_rgb_array(display_mat, min_val_display, cmap_name, percent) + if log_range is not None and not log_input: + raise ValueError("log_range requires log_input=True") + if log_input: + heatmap_rgb_array, norm_obj, vmin, vmax = _generate_log_heatmap_rgb_array( + display_mat, min_val_display, cmap_name, percent, log_range + ) + else: + heatmap_rgb_array, norm_obj, vmin, vmax = _generate_heatmap_rgb_array(display_mat, min_val_display, cmap_name, percent) if not colorbar: return heatmap_rgb_array @@ -253,7 +315,12 @@ def plot_heatmap( return np.hstack((heatmap_rgb_array, empty_cbar_space)) # 3. Get colorbar ticks and labels - tick_values, tick_labels = _get_colorbar_ticks_and_labels(vmin, vmax, max_colorbar_ticks, percent) + tick_generator = _log_colorbar_ticks if log_input else _get_colorbar_ticks_and_labels + tick_values, tick_labels = tick_generator(vmin, vmax, max_colorbar_ticks, percent) + if log_range is not None and tick_labels: + tick_labels[0] = "≤" + tick_labels[0] + if len(tick_labels) > 1: + tick_labels[-1] = "≥" + tick_labels[-1] # 4. Generate colorbar RGB array colorbar_rgb_array = _generate_colorbar_rgb_array( @@ -266,11 +333,16 @@ def plot_heatmap( return final_rgb_image -def plot_class_distance_matrix(model: nn.Module, **kwargs): +def plot_class_distance_matrix(model: nn.Module, *, log_domain: bool = False, **kwargs): """Plot the pairwise class distance matrix. CDF of pairwise inner products between rows in the last-layer weight matrix, assuming that these have unit norm. + log_domain=True computes the same complementary probability directly in + float32 log space, avoiding subtraction from a rounded CDF. The default + preserves the legacy rendering for comparisons. """ + if log_domain: + return [plot_heatmap(log_tail.cpu(), log_input=True, **kwargs) for log_tail in class_log_similarity(model, complement=True)] return [plot_heatmap(1 - csi.cpu(), **kwargs) for csi in class_similarity(model, cdf=True)] diff --git a/mini_trainer/visualization/prototype_space/__init__.py b/mini_trainer/visualization/prototype_space/__init__.py new file mode 100644 index 00000000..32b8604e --- /dev/null +++ b/mini_trainer/visualization/prototype_space/__init__.py @@ -0,0 +1 @@ +"""Interactive exploration of classifier prototype geometry.""" diff --git a/mini_trainer/visualization/prototype_space/__main__.py b/mini_trainer/visualization/prototype_space/__main__.py new file mode 100644 index 00000000..a58a8ef0 --- /dev/null +++ b/mini_trainer/visualization/prototype_space/__main__.py @@ -0,0 +1,5 @@ +"""Run with python -m mini_trainer.visualization.prototype_space.""" + +from .launch import run + +run() diff --git a/mini_trainer/visualization/prototype_space/browser.py b/mini_trainer/visualization/prototype_space/browser.py new file mode 100644 index 00000000..16fc31f2 --- /dev/null +++ b/mini_trainer/visualization/prototype_space/browser.py @@ -0,0 +1,101 @@ +"""Package a verified embedding ONNX export for the static browser explorer.""" + +import argparse +import hashlib +import json +import shutil +from pathlib import Path +from tempfile import TemporaryDirectory + + +def package_bundle(export, output, runtime, weights): + """Package the supported RGB nearest-square/ImageNet evaluation contract. + + Runtime is the unpacked, pinned onnxruntime-web npm package. This command + copies assets; it never downloads code or changes a Python environment. + """ + export, output, runtime = Path(export), Path(output), Path(runtime) + manifest = json.loads((export / "manifest.json").read_text()) + if manifest["output_semantics"] != "predictions_and_preclassification_embedding": + raise ValueError("Export with include_embeddings=True first.") + if len(manifest["classifiers"]) != 1: + raise ValueError("Browser bundle requires one classifier.") + metadata = manifest["classifiers"][0]["metadata"] + mapping = metadata["cls2idx"] + levels = [mapping[str(i)] for i in range(len(mapping))] if isinstance(next(iter(mapping.values())), dict) else [mapping] + classes = [[str(name) for name, _ in sorted(level.items(), key=lambda item: item[1])] for level in levels] + shape = manifest["input"]["shape"] + recipe = manifest["preprocessing"]["recipe"] + if ( + manifest["input"]["dtype"] != "float32" + or shape[1] != 3 + or shape[2] != shape[3] + or not recipe + or recipe.get("contract") != "nearest-square-uint8-bilinear-center-imagenet-v1" + or recipe["size"] != shape[2] + or recipe["resize"] < shape[2] + ): + raise ValueError("Bundle requires an explicitly verified nearest-square RGB and ImageNet preprocessing recipe.") + from .explore import load_prototypes + + prototypes, names, _, provenance = load_prototypes(Path(weights)) + if provenance["checkpoint_sha256"] != manifest["source"]["checkpoint_sha256"] or names != classes[0]: + raise ValueError("Prototype checkpoint differs from the ONNX export.") + version = json.loads((runtime / "package.json").read_text())["version"] + if version != "1.24.3": + raise ValueError("This browser bundle supports onnxruntime-web 1.24.3.") + if output.exists(): + raise FileExistsError(output) + # Check every supplied export artifact before creating a distribution. + for name, digest in manifest["artifacts"].items(): + if Path(name).name != name or hashlib.sha256((export / name).read_bytes()).hexdigest() != digest: + raise ValueError(f"Invalid export artifact: {name}") + destination = output + destination.parent.mkdir(parents=True, exist_ok=True) + with TemporaryDirectory(prefix=".browser-bundle-", dir=destination.parent) as temporary: + output = Path(temporary) / "bundle" + output.mkdir() + for name in manifest["artifacts"]: + shutil.copy2(export / name, output / name) + (output / "prototypes.f32").write_bytes(prototypes.numpy().astype(" max_bytes: + shutil.rmtree(entry) + + +def clear_analysis_cache(root): + """Remove only complete content-addressed entries owned by this cache.""" + root = Path(root) + removed = 0 + if root.exists(): + for entry in root.iterdir(): + if entry.is_dir() and len(entry.name) == 64 and all(c in "0123456789abcdef" for c in entry.name): + shutil.rmtree(entry) + removed += 1 + return removed diff --git a/mini_trainer/visualization/prototype_space/explore.py b/mini_trainer/visualization/prototype_space/explore.py new file mode 100644 index 00000000..a1d67fd5 --- /dev/null +++ b/mini_trainer/visualization/prototype_space/explore.py @@ -0,0 +1,304 @@ +"""Offline prototype exploration using mini_trainer's existing diagnostic APIs.""" + +import argparse +import base64 +import hashlib +import json +import math +from pathlib import Path + +import numpy as np +import torch +from matplotlib import colormaps +from scipy.cluster.hierarchy import linkage +from scipy.spatial.distance import squareform +from torch import nn + +from mini_trainer.modeling import Classifier, class_distance, class_log_similarity, class_similarity +from mini_trainer.visualization._dendrogram_layout import linkage_layout + +from .projection import angular_tsne, prototype_projections + + +def weight_model(weight: torch.Tensor) -> nn.Module: + """A linear-only diagnostic head: values are already effective weights.""" + with torch.device("meta"): + head = Classifier(weight.shape[1], weight.shape[0], hidden=False, normalized=False) + head.linear = nn.Linear(weight.shape[1], weight.shape[0], bias=False, device="meta") + head.linear.load_state_dict({"weight": weight}, assign=True) + return nn.Sequential(head) + + +def load_prototypes(path: Path): + """Read supported floating-point linear heads without building a backbone. + + This is deliberately not a general checkpoint loader. Reject other head + families instead of treating a sublayer as the complete prototype space. + """ + state = torch.load(path, map_location="cpu", weights_only=True) + metas = [(key, value) for key, value in state.items() if key.endswith("._extra_state") and isinstance(value, dict)] + if len(metas) != 1: + raise ValueError("Expected exactly one classifier metadata record") + key, meta = metas[0] + if meta.get("classifier_class") not in { + "mini_trainer.modeling.classifier:Classifier", + "mini_trainer.hierarchical.model:HierarchicalClassifier", + }: + raise ValueError("Explorer supports only Classifier and HierarchicalClassifier linear prototypes") + prefix = key.removesuffix("_extra_state") + "linear." + linear = {key[len(prefix) :]: value for key, value in state.items() if key.startswith(prefix)} + normalized = meta.get("normalized") + weight_key = "parametrizations.weight.original1" if normalized else "weight" + raw = linear.get(weight_key) + if not isinstance(raw, torch.Tensor) or raw.ndim != 2 or raw.dtype != torch.float32: + raise ValueError("Expected float32 linear weights; quantized/other checkpoint encodings are not supported") + layer = nn.Linear(raw.shape[1], raw.shape[0], bias="bias" in linear) + if normalized: + layer = Classifier._normalize_layer(layer, orthogonal_init=False) + layer.load_state_dict(linear, strict=True) + weight = layer.weight.detach().clone() + if not torch.isfinite(weight).all() or not (weight.norm(dim=1) > 0).all(): + raise ValueError("Prototype rows must be finite and nonzero") + mapping = meta["cls2idx"] + if all(isinstance(value, dict) for value in mapping.values()): + mapping = mapping.get("0", mapping.get(0)) + if not mapping or sorted(mapping.values()) != list(range(len(weight))): + raise ValueError("Class mapping must cover every prototype exactly once") + names = [""] * len(weight) + for name, index in mapping.items(): + names[index] = str(name) + hierarchy = meta.get("labels", {}) + groups = [list(map(str, hierarchy.get(name, [name]))) for name in names] + with path.open("rb") as stream: + digest = hashlib.file_digest(stream, "sha256").hexdigest() + return ( + weight, + names, + groups, + { + "checkpoint_sha256": digest, + "checkpoint": path.name, + "classifier": meta["classifier_class"], + "normalized": normalized, + "norm_range": [weight.norm(dim=1).min().item(), weight.norm(dim=1).max().item()], + "bias_range": [layer.bias.min().item(), layer.bias.max().item()] if layer.bias is not None else None, + "analysis_script_sha256": hashlib.sha256(Path(__file__).read_bytes()).hexdigest(), + "torch_version": str(torch.__version__), + "numpy_version": np.__version__, + }, + ) + + +def array_payload(array, dtype="=2 classes, >2 dimensions and >=1 neighbour") + model = weight_model(weight) + print(f"Computing repository z-scores and distances for {n:,} × {dim:,}", flush=True) + z = class_similarity(model, cdf=False)[0].numpy() + distance = class_distance(model)[0].numpy() + tail = 1 - class_similarity(model, cdf=True)[0].numpy() + log_tail = class_log_similarity(model, complement=True)[0].numpy() + assert np.isfinite(distance).all() and np.isfinite(z).all() + k = min(neighbours, n - 1) + # z is a separately labelled refinement of saturated distance ties. + # Stable sorting uses checkpoint class order for exactly equal z values. + ranks = [1, min(5, n - 1), min(12, n - 1), min(32, n - 1)] + profile_neighbours = np.empty((n, len(ranks)), dtype=np.int32) + near = np.empty((n, k), dtype=np.int32) + for i in range(n): + row = z[i].copy() + row[i] = -np.inf + ordered = np.argsort(-row, kind="stable") + near[i] = ordered[:k] + profile_neighbours[i] = ordered[np.asarray(ranks) - 1] + ids = np.column_stack((np.arange(n), near)) + local_d = distance[ids[:, :, None], ids[:, None, :]] + local_z = z[ids[:, :, None], ids[:, None, :]] + stable_near, _ = stable_scores(z[np.arange(n)[:, None], near]) + local_logtail = log_tail[ids[:, :, None], ids[:, None, :]] / math.log(10) + zero_count = (distance == 0).sum(axis=1) - 1 + recovered = 0 + for row in range(n): + zero_ids = np.flatnonzero(distance[row] == 0) + zero_ids = zero_ids[zero_ids > row] + stable_zero, _ = stable_scores(z[row, zero_ids]) + recovered += int((stable_zero > 0).sum()) + upper = np.triu_indices(n, 1) if n < 1024 else None + rng = np.random.default_rng(seed) + if upper is None: + i, j = rng.integers(n, size=(2, 200_000)) + i, j = i[i != j], j[i != j] + else: + i, j = upper + sampled = z[i, j] + robust_range = np.quantile(log_tail[i, j] / math.log(10), [0.001, 0.999]).tolist() + bins = np.linspace(-6, max(8, float(sampled.max()) + 0.1), 100) + hist, edges = np.histogram(sampled, bins) + stats = { + "classes": n, + "dimensions": dim, + "off_diagonal_zero_distance_pairs": int(zero_count.sum() // 2), + "classes_with_zero_distance_neighbours": int((zero_count > 0).sum()), + "zero_pairs_resolved_by_float64_logcdf": recovered, + "finite_off_diagonal_log_tail_pairs": int(np.isfinite(log_tail).sum() // 2), + "pair_sample_count": len(sampled), + "seed": seed, + "pair_sampling": "all unordered pairs" if upper is not None else "uniform ordered pairs with replacement, diagonal excluded", + "pair_z_mean": float(sampled.mean()), + "pair_z_std": float(sampled.std()), + "pair_z_quantiles": dict(zip(["1%", "50%", "99%", "max_sampled"], np.quantile(sampled, [0.01, 0.5, 0.99, 1]).tolist())), + "nearest_z_median": float(np.median(z[np.arange(n), near[:, 0]])), + "kth_z_median": float(np.median(z[np.arange(n), near[:, -1]])), + "k": k, + "histogram_excluded": int(len(sampled) - hist.sum()), + } + print(json.dumps(stats), flush=True) + print("Building unchanged Ward linkage on class_distance", flush=True) + tree = linkage(squareform(distance, checks=False), method="ward") + order, _, _, _ = linkage_layout(tree, np.ones(n, dtype=int)) + order = np.asarray(order) + low, high, block = block_extrema(tail, order) + log_low, log_high, _ = block_extrema(log_tail, order) + log_floor = math.floor(float(np.nanmin(log_low)) / math.log(10)) * math.log(10) + del distance, tail + ranks = [1, min(5, n - 1), min(12, n - 1), min(32, n - 1)] + profiles = np.empty((n, len(ranks)), dtype=np.float32) + for i in range(n): + row = z[i].copy() + row[i] = -np.inf + profiles[i] = -np.partition(-row, np.asarray(ranks) - 1)[np.asarray(ranks) - 1] + print("Computing PCA projections and neighbourhood retention", flush=True) + projections = prototype_projections(weight, near.tolist()) + if include_tsne: + print("Fitting angular t-SNE", flush=True) + projections["Angular t-SNE"] = angular_tsne(z, dim, near.tolist(), seed) + return { + "projections": projections, + "metadata": metadata or {}, + "stats": stats, + "names": names, + "groups": groups, + "neighbours": near.tolist(), + "local_distance": array_payload(local_d), + "local_z": array_payload(local_z), + "local_log10_tail": array_payload(local_logtail), + "stable_neighbour_distance": array_payload(stable_near.numpy(), dtype="Object.assign(job,{resolve,reject})); + this.pending.set(key,job);this.queue.push(job);this.pump();return job.promise; + } + pump(){ + while(this.active{ + try{ + const signal=AbortSignal.any([job.signal,AbortSignal.timeout(this.timeout)]); + const response=await this.fetcher('https://api.gbif.org/v1/'+job.key,{signal,credentials:'omit'}); + if(!response.ok)throw Error(response.status===429?'GBIF request limit reached; retry later.':`GBIF HTTP ${response.status}`); + const value=await response.json(); + if(job.signal.aborted)throw new DOMException('Lookup cancelled','AbortError'); + if(!value||typeof value!=='object'||Array.isArray(value))throw Error('Invalid GBIF response'); + this.cache.set(job.key,value);job.resolve(value); + }catch(error){job.reject(error);} + finally{this.active--;if(this.pending.get(job.key)===job)this.pending.delete(job.key);this.pump();} + })(); + } + } + async taxon(id){ + id=String(id);if(!/^\d+$/.test(id))throw Error('Not a GBIF taxon ID'); + const value=await this.request('species/'+id); + if(String(value.key)!==id)throw Error('GBIF returned a different taxon'); + return value; + } + async name(id){ + const taxon=await this.taxon(id); + return {class_id:String(id),display_name:taxon.canonicalName||taxon.scientificName||String(id)}; + } + async album(id){ + id=String(id);const generation=this.controller.signal; + const taxon=await this.taxon(id); + if(generation.aborted)throw new DOMException('Lookup cancelled','AbortError'); + const records=await this.request('occurrence/search?taxonKey='+id+'&mediaType=StillImage&limit=20'); + if(generation.aborted)throw new DOMException('Lookup cancelled','AbortError'); + if(!Array.isArray(records.results))throw Error('Invalid GBIF occurrence response'); + return BrowserGBIF.albumFromRecords(id,taxon,records.results); + } + static webURL(value){ + if(typeof value!=='string')return false; + try{return ['https:','http:'].includes(new URL(value).protocol);}catch{return false;} + } + static albumFromRecords(id,taxon,records){ + const accepted=String(taxon.acceptedKey||id),photos=[],seen=new Set(); + for(const record of records){ + if(!record||!/^\d+$/.test(String(record.key)))continue; + const members=['taxonKey','acceptedTaxonKey','speciesKey','genusKey','familyKey','orderKey','classKey','phylumKey','kingdomKey'].map(key=>String(record[key])); + if(!members.includes(id)&&!members.includes(accepted))continue; + for(const media of Array.isArray(record.media)?record.media:[]){ + if(!media||media.type!=='StillImage'||!this.webURL(media.identifier)||seen.has(media.identifier))continue; + seen.add(media.identifier); + photos.push({occurrence_id:String(record.key),image_path:media.identifier,original:media.identifier, + source:this.webURL(media.references)?media.references:'https://www.gbif.org/occurrence/'+record.key, + creator:media.creator||media.rightsHolder||'Creator not supplied',license:media.license||'Image license not supplied',scientific_name:record.scientificName||''}); + if(photos.length===6)break; + } + if(photos.length===6)break; + } + return {class_id:id,accepted_id:accepted,display_name:taxon.canonicalName||taxon.scientificName||id,taxonomic_status:taxon.taxonomicStatus,photos,selection:'GBIF class examples'}; + } +} +if(typeof module!=='undefined')module.exports={BrowserGBIF}; diff --git a/mini_trainer/visualization/prototype_space/inference.js b/mini_trainer/visualization/prototype_space/inference.js new file mode 100644 index 00000000..c4bfb811 --- /dev/null +++ b/mini_trainer/visualization/prototype_space/inference.js @@ -0,0 +1,149 @@ +/* Predictions use the exported forward; PCA queries use the fitted prototype transform. */ +let inferenceWorker=null,inferenceManifest=null,inferenceQuery=null,inferenceSequence=0,predictionPhotos=null,lastPrediction=null,inferencePreviewURL=null; +function setInferenceInputDisabled(disabled){for(const id of ['inference-image','inference-camera','inference-camera-button'])$(id).disabled=disabled;} +const inferenceStatus=message=>$('inference-status').textContent=message; +function resetInferenceModel(){++inferenceSequence;inferenceWorker?.terminate();inferenceWorker=null;inferenceManifest=null;inferenceQuery=null;lastPrediction=null;predictionPhotos?.abort();$('inference-results').replaceChildren();setInferenceInputDisabled(true);$('inference-preview').hidden=true;if(inferencePreviewURL){URL.revokeObjectURL(inferencePreviewURL);inferencePreviewURL=null;}inferenceStatus('Load the matching browser bundle to begin.');} + +$('inference-load').onclick=async()=>{ + inferenceWorker?.terminate();inferenceWorker=null;inferenceQuery=null;inferenceManifest=null;lastPrediction=null;predictionPhotos?.abort();$('inference-results').replaceChildren(); + setInferenceInputDisabled(true);++inferenceSequence; + try{ + const url=new URL($('inference-url').value,location.href); + if(!['http:','https:'].includes(url.protocol))throw Error('Use an HTTP or HTTPS browser-bundle manifest URL.'); + inferenceStatus('Loading model metadata…'); + const response=await fetch(url);if(!response.ok)throw Error(`Manifest HTTP ${response.status}`); + const m=await response.json(); + if(m.schema!=='mini-trainer-browser-v1'||m.checkpoint_sha256!==data.metadata.checkpoint_sha256)throw Error('This bundle does not match the displayed checkpoint.'); + if(m.classes[0].length!==data.names.length||m.classes[0].some((name,i)=>name!==data.names[i]))throw Error('Model class order differs from the explorer.'); + const worker=new Worker(new URL('inference_worker.js',url));inferenceWorker=worker; + worker.onerror=event=>inferenceStatus(`Browser worker failed: ${event.message}. Check same-origin hosting and JavaScript MIME types.`); + worker.onmessage=({data:result})=>{ + if(worker!==inferenceWorker)return; + if(result.kind==='ready'){inferenceManifest=m;setInferenceInputDisabled(false);inferenceStatus('Model ready. Choose an image; inference runs on this device.');} + if(result.kind==='error'){inferenceStatus(result.message);setInferenceInputDisabled(!inferenceManifest);} + if(result.kind==='result'&&result.id===inferenceSequence){showInference(result,m);setInferenceInputDisabled(false);} + }; + worker.postMessage({kind:'load',base:url.href,manifest:m,coordinates:data.projections?.['Angular t-SNE']?.coordinates});inferenceStatus('Loading the ONNX model into this browser…'); + }catch(error){inferenceStatus(`${error.message} Cross-origin hosts must allow CORS; this model bundle can be served beside the explorer.`);} +}; +$('inference-cancel').onclick=()=>{++inferenceSequence;inferenceWorker?.terminate();inferenceWorker=null;inferenceManifest=null;setInferenceInputDisabled(true);inferenceStatus('Stopped. Load the model to continue.');}; +$('inference-camera-button').onclick=()=>$('inference-camera').click(); +$('inference-image').onchange=$('inference-camera').onchange=async event=>{ + const file=event.target.files[0];event.target.value='';if(!file||!inferenceManifest)return; + const id=++inferenceSequence,m=inferenceManifest,worker=inferenceWorker; + setInferenceInputDisabled(true);inferenceStatus('Preparing image…'); + try{ + const pixels=await prepareInferenceImage(file,m.size); + if(id!==inferenceSequence)return; + if(inferencePreviewURL){URL.revokeObjectURL(inferencePreviewURL);inferencePreviewURL=null;} + const preview=document.createElement('canvas');preview.width=preview.height=m.size; + preview.getContext('2d').putImageData(pixels,0,0); + $('inference-preview').src=preview.toDataURL();$('inference-preview').hidden=false; + const tensor=browserPreprocess(pixels,m); + inferenceStatus('Running inference locally…');worker.postMessage({kind:'infer',id,tensor},[tensor.buffer]); + }catch(error){if(id===inferenceSequence){inferenceStatus(error.message);setInferenceInputDisabled(!inferenceManifest);}} +}; +// Decode orientation once, then sample only the pixels required by the model. +// No full-resolution canvas or RGBA readback: camera images can exceed 40 MP. +async function prepareInferenceImage(file,size){ + let image,url; + try{ + try{image=await createImageBitmap(file,{imageOrientation:'from-image',premultiplyAlpha:'none'});} + catch{ + // Some browser-native formats are available through but not ImageBitmap. + image=new Image();url=URL.createObjectURL(file);image.src=url;await image.decode(); + } + const width=image.naturalWidth||image.width,height=image.naturalHeight||image.height; + if(!width||!height)throw Error('Empty image'); + const canvas=document.createElement('canvas');canvas.width=canvas.height=size; + const ctx=canvas.getContext('2d',{willReadFrequently:true}); + ctx.fillStyle='white';ctx.fillRect(0,0,size,size);ctx.imageSmoothingEnabled=false; + for(let y=0;y{ + const max=Math.max(...scores),exp=scores.map(x=>Math.exp(x-max)),sum=exp.reduce((a,b)=>a+b,0); + const heading=document.createElement('h3');heading.textContent=`Level ${level}`;container.append(heading); + const list=document.createElement('ol');list.setAttribute('aria-label',`Level ${level} predictions`);list.setAttribute('data-level',level); + scores.map((score,i)=>({score,i})).sort((a,b)=>b.score-a.score||a.i-b.i).slice(0,5).forEach(({i})=>{ + const row=document.createElement('li'),button=document.createElement('button'),id=m.classes[level][i];const label=level===0?classLabel(i):id;button.textContent=`${label}${label===id?'':' · '+id} · ${(100*exp[i]/sum).toFixed(2)}%`; + button.onclick=()=>{if(level===0)selectClass(i);};if(level!==0){button.disabled=true;button.title='Higher-rank prediction; no individual species selected.';}row.append(button);list.append(row); + if(gbifEnabled()){ + const id=m.classes[level][i]; + gbifClient.name(id).then(value=>{if(!signal.aborted&&gbifEnabled()&&!(level===0&&(importedNames.get(data)?.has(i)||data.display_names?.[i])))button.textContent=`${value.display_name} · ${id} · ${(100*exp[i]/sum).toFixed(2)}%`;}).catch(()=>{}); + if(level===0)showPredictionPhoto(row,id,signal); + } + });container.append(list); + }); +} +function showInference(result,m){ + lastPrediction={result,m};renderInferenceResults(); + inferenceQuery={embedding:result.embedding,checkpoint:m.checkpoint_sha256,insertion:result.insertion}; + const plane=result.insertion?'Angular t-SNE':Object.keys(data.projections).find(name=>data.projections[name].transform); + if(plane){$('projection-plane').value=plane;drawProjection();} + inferenceStatus(`Complete. Scores are not calibrated reliability. Purple diamond: ${result.insertion?`fixed-map t-SNE insertion; ${(100*result.insertion.retained_fraction).toFixed(0)}% of top-12 angular neighbours retained, ${result.insertion.milliseconds.toFixed(0)} ms`:'PCA projection'}. Prototype positions remain fixed.`); +} +function drawInferenceQuery(ctx,p,pixels){ + if(!inferenceQuery||inferenceQuery.checkpoint!==data.metadata.checkpoint_sha256)return; + if(!p.transform){ + if(p!==data.projections['Angular t-SNE']||!inferenceQuery.insertion)return; + paintInferencePoint(ctx,inferenceQuery.insertion.coordinates,pixels);return; + } + const e=inferenceQuery.embedding,t=p.transform;if(e.length!==t.mean.length)return; + const norm=t.normalize?Math.hypot(...e):1;const point=[0,0]; + for(let i=0;i { + try { + if(message.kind==='load') { + manifest=message.manifest;coordinates=message.coordinates; + importScripts(new URL('insertion.js',message.base).href); + const response=await fetch(new URL(manifest.prototypes,message.base));if(!response.ok)throw Error('Could not load prototypes.'); + prototypes=new Float32Array(await response.arrayBuffer()); + importScripts(new URL('runtime/ort.wasm.min.js',message.base).href); + ort.env.wasm.numThreads=1; + ort.env.wasm.wasmPaths={mjs:new URL('runtime/ort-wasm-simd-threaded.js',message.base).href,wasm:new URL('runtime/ort-wasm-simd-threaded.wasm',message.base).href}; + session=await ort.InferenceSession.create(new URL(manifest.model,message.base).href,{ + executionProviders:['wasm'],externalData:(manifest.external_data||[]).map(path=>({path,data:new URL(path,message.base).href})) + }); + self.postMessage({kind:'ready'}); + } else if(message.kind==='infer') { + if(!session)throw Error('Load a model first.'); + const result=await session.run({images:new ort.Tensor('float32',message.tensor,[1,3,manifest.size,manifest.size])}); + const outputs=manifest.outputs.map(name=>Array.from(result[name].data)); + const embedding=Array.from(result[manifest.embedding].data); + const insertion=coordinates?insertIntoFixedTSNE(embedding,prototypes,coordinates):null; + self.postMessage({kind:'result',id:message.id,outputs,embedding,insertion}); + } + } catch(error) {self.postMessage({kind:'error',id:message.id,message:String(error.message||error)});} +}; diff --git a/mini_trainer/visualization/prototype_space/insertion.js b/mini_trainer/visualization/prototype_space/insertion.js new file mode 100644 index 00000000..2a3874af --- /dev/null +++ b/mini_trainer/visualization/prototype_space/insertion.js @@ -0,0 +1,49 @@ +/* Fixed-map t-SNE insertion. Only the query moves; all reference coordinates stay fixed. + * Minimize KL(P(query -> prototypes) || normalized Student-t(query -> map)). + * This is an out-of-sample approximation, not a refit of the joint t-SNE map. + */ +function insertionAffinities(distances,perplexity=30){ + const ids=Array.from(distances,(_,i)=>i).sort((a,b)=>distances[a]-distances[b]||a-b).slice(0,Math.min(distances.length,Math.ceil(3*perplexity))); + const target=Math.log(Math.min(perplexity,ids.length));let beta=1,lo=0,hi=Infinity,probabilities=[]; + const minimum=distances[ids[0]]**2; + for(let iteration=0;iteration<80;iteration++){ + probabilities=ids.map(i=>Math.exp(-beta*(distances[i]**2-minimum))); + const total=probabilities.reduce((a,b)=>a+b,0);probabilities=probabilities.map(p=>p/total); + const entropy=-probabilities.reduce((s,p)=>s+(p>0?p*Math.log(p):0),0); + if(Math.abs(entropy-target)<1e-7)break; + if(entropy>target){lo=beta;beta=Number.isFinite(hi)?(lo+hi)/2:beta*2;}else{hi=beta;beta=(lo+hi)/2;} + } + return {ids,probabilities,perplexity:Math.exp(-probabilities.reduce((s,p)=>s+(p?p*Math.log(p):0),0))}; +} +function insertionObjective(point,coordinates,affinities){ + let sum=0,rx=0,ry=0,loss=0,gx=0,gy=0; + for(const q of coordinates){const dx=point[0]-q[0],dy=point[1]-q[1],w=1/(1+dx*dx+dy*dy);sum+=w;rx+=w*w*dx;ry+=w*w*dy;} + affinities.ids.forEach((id,i)=>{const p=affinities.probabilities[i];if(!p)return;const q=coordinates[id],dx=point[0]-q[0],dy=point[1]-q[1],w=1/(1+dx*dx+dy*dy);loss+=p*Math.log(p/w);gx+=p*w*dx;gy+=p*w*dy;}); + return {loss:loss+Math.log(sum),gradient:[2*(gx-rx/sum),2*(gy-ry/sum)]}; +} +function insertIntoFixedTSNE(embedding,prototypes,coordinates,{perplexity=30,iterations=200,k=12}={}){ + const began=performance.now(),n=coordinates.length,d=embedding.length; + if(prototypes.length!==n*d||n<2)throw Error('Prototype dimensions do not match the t-SNE map.'); + const norm=Math.hypot(...embedding);if(!Number.isFinite(norm)||norm===0)throw Error('Invalid image embedding.'); + const distances=new Float64Array(n); + for(let i=0;i{for(let j=0;j<2;j++)centroid[j]+=affinities.probabilities[i]*coordinates[id][j];}); + const starts=[centroid,...affinities.ids.slice(0,3).map(id=>coordinates[id].slice())];let best=null; + for(const start of starts){ + let point=start.slice(),state=insertionObjective(point,coordinates,affinities),rate=10;const initialLoss=state.loss; + for(let iteration=0;iterationv-step*state.gradient[j]),candidate=insertionObjective(next,coordinates,affinities); + if(candidate.loss<=state.loss-1e-4*step*norm*norm){point=next;state=candidate;rate=Math.min(100,step*1.5);accepted=true;break;}step*=.5; + }if(!accepted)break; + } + if(!best||state.loss({i,d:(p[0]-best.coordinates[0])**2+(p[1]-best.coordinates[1])**2})).sort((a,b)=>a.d-b.d||a.i-b.i).slice(0,count).map(p=>p.i); + return {...best,neighbours:original,projected_neighbours:nearby,retained_fraction:original.filter(i=>nearby.includes(i)).length/count,perplexity:affinities.perplexity,milliseconds:performance.now()-began,method:'fixed-map conditional KL insertion'}; +} +if(typeof module!=='undefined')module.exports={insertionAffinities,insertionObjective,insertIntoFixedTSNE}; diff --git a/mini_trainer/visualization/prototype_space/launch.py b/mini_trainer/visualization/prototype_space/launch.py new file mode 100644 index 00000000..bba47dd2 --- /dev/null +++ b/mini_trainer/visualization/prototype_space/launch.py @@ -0,0 +1,368 @@ +"""Launch the prototype explorer with a checkpoint path or browser file picker.""" + +import argparse +import hashlib +import importlib.util +import json +import os +import secrets +import shutil +import subprocess +import sys +import tempfile +import threading +import time +import webbrowser +from functools import partial +from http.server import ThreadingHTTPServer +from pathlib import Path +from urllib.parse import parse_qs, unquote, urlsplit + + +def require_dependencies(angular=True): + missing = [name for name in (["scipy", "sklearn"] if angular else ["scipy"]) if importlib.util.find_spec(name) is None] + if missing: + raise RuntimeError("Prototype exploration requires: " + ", ".join(missing) + ". Install mini_trainer[explorer].") + + +def generate(weights, output, *, angular=True, synthetic=False, threads=4, cache_dir=None): + require_dependencies(angular) + from .cache import analysis_key, read_analysis, store_analysis + + output = Path(output) + key = contract = cached = None + if cache_dir is not None: + print("Checking analysis cache", flush=True) + key, contract = analysis_key(weights, angular=angular, synthetic=synthetic, threads=threads) + cached = read_analysis(cache_dir, key, contract) + if cached is not None: + from .explore import render_report + + output.mkdir(parents=True, exist_ok=True) + for name, raw in cached.items(): + (output / name).write_bytes(raw) + render_report(output, cached["report-data.json"].decode()) + print("Analysis cache hit; rendered current viewer", flush=True) + return output / "explorer.html" + import torch + + from .explore import create_report + + torch.set_num_threads(threads) + result = create_report(weights, output, include_tsne=angular, synthetic=synthetic) + if cache_dir is not None: + store_analysis(cache_dir, key, contract, output) + print("Analysis cache miss; stored completed result", flush=True) + return result + + +class Workspace: + """Owned preparation jobs; only the current job can publish a report.""" + + def __init__(self, root, angular=True, synthetic=False, threads=4, cache_dir=None): + self.root = root + empty = root / "empty" + empty.mkdir() + self.public = root / "report" + self.public.symlink_to(empty, target_is_directory=True) + self.angular, self.synthetic, self.threads = angular, synthetic, threads + self.cache_dir = cache_dir + self.token = secrets.token_urlsafe(32) + self.lock = threading.Lock() # One upload at a time, independent of analysis. + self.guard = threading.RLock() + self.process = None + self.job = 0 + self.workers = [] + self.state = {"state": "idle"} + self.service = None + self.ready = False + + def view_state_path(self, key): + if not key or len(key) > 512: + raise ValueError("Invalid view-state identity") + return (self.cache_dir or self.root) / ".view-state" / (hashlib.sha256(key.encode()).hexdigest() + ".json") + + def read_view_state(self, key): + path = self.view_state_path(key) + try: + return json.loads(path.read_text()) + except (OSError, ValueError): + return None + + def write_view_state(self, key, value): + path = self.view_state_path(key) + path.parent.mkdir(parents=True, exist_ok=True) + temporary = path.with_suffix("." + secrets.token_hex(8)) + try: + temporary.write_text(json.dumps(value)) + os.replace(temporary, path) + finally: + temporary.unlink(missing_ok=True) + + def cancel(self): + with self.guard: + self.job += 1 + process = self.process + if process is not None and process.poll() is None: + process.terminate() + try: + process.wait(timeout=5) + except subprocess.TimeoutExpired: + process.kill() + process.wait() + self.process = None + self.state = {"state": "cancelled", "ready": self.ready} + + def close(self): + self.cancel() + for worker in self.workers: + worker.join(timeout=10) + + def status(self): + with self.guard: + state = dict(self.state, ready=self.ready) + if state["state"] == "running": + state["elapsed"] = round(time.monotonic() - state["started"], 1) + log = self.root / f"job-{self.job}" / "generation.log" + if log.exists(): + lines = log.read_text(errors="replace").splitlines() + stages = [line for line in lines if line and not line.startswith("{")] + state["stage"] = stages[-1] if stages else "Starting analysis" + return state + + def start(self, weights): + self.cancel() + with self.guard: + job = self.job + self.state = {"state": "running", "file": weights.name, "started": time.monotonic(), "job": job} + worker = threading.Thread(target=self.build, args=(weights, job), daemon=True) + self.workers = [item for item in self.workers if item.is_alive()] + self.workers.append(worker) + worker.start() + + def build(self, weights, job): + directory = self.root / f"job-{job}" + directory.mkdir() + output = directory / "report" + process = None + try: + command = [ + sys.executable, + "-m", + "mini_trainer.visualization.prototype_space", + str(weights), + "--export", + "--output", + str(output), + "--threads", + str(self.threads), + ] + command += ["--cache-dir", str(self.cache_dir)] if self.cache_dir is not None else ["--no-cache"] + if not self.angular: + command.append("--pca-only") + if self.synthetic: + command.append("--synthetic") + with (directory / "generation.log").open("w") as log: + with self.guard: + if job != self.job: + return + process = subprocess.Popen(command, stdout=log, stderr=subprocess.STDOUT) + self.process = process + result = process.wait() + with self.guard: + if job != self.job: + return + if result: + raise RuntimeError((directory / "generation.log").read_text()[-4000:]) + data = json.loads((output / "report-data.json").read_text()) + ids = {name for case in data.values() if not case["metadata"].get("synthetic") for name in case["names"] if name.isdigit()} + link = directory / "publish" + link.symlink_to(output, target_is_directory=True) + previous = self.public.resolve().parent + os.replace(link, self.public) + if previous.parent == self.root and previous.name.startswith("job-"): + shutil.rmtree(previous, ignore_errors=True) + self.service.allowed_ids = ids + self.ready = True + log_text = (directory / "generation.log").read_text() + cache_status = "hit" if "Analysis cache hit" in log_text else "miss" if self.cache_dir is not None else "disabled" + self.state = {"state": "ready", "file": weights.name, "url": "/explorer.html", "cache": cache_status} + except Exception as error: + with self.guard: + if job == self.job: + self.state = {"state": "error", "error": str(error)} + finally: + with self.guard: + if self.public.resolve() != output.resolve(): + shutil.rmtree(directory, ignore_errors=True) + if weights.parent.parent == self.root / "inputs": + shutil.rmtree(weights.parent, ignore_errors=True) + pid = getattr(process, "pid", None) + if pid is not None and self.cache_dir is not None and self.cache_dir.exists(): + for pending in self.cache_dir.glob(f".pending-{pid}-*"): + shutil.rmtree(pending, ignore_errors=True) + + +def session_handler(workspace): + from .serve import Handler + + class SessionHandler(Handler): + def do_GET(self): + path = urlsplit(self.path).path + if path == "/": + page = Path(__file__).with_name("launcher.html").read_text().replace("__SESSION_TOKEN__", workspace.token).encode() + self.send_response(200) + self.send_header("Content-Type", "text/html; charset=utf-8") + self.send_header("Content-Length", str(len(page))) + self.send_header("Cache-Control", "no-store") + self.end_headers() + self.wfile.write(page) + return + if path == "/api/health": + self.send_json({"photo_api": True, "model_picker": True}) + return + if path == "/api/view-state": + key = parse_qs(urlsplit(self.path).query).get("key", [""])[0] + if not key or len(key) > 512: + self.send_json({"error": "Invalid view-state identity"}, 400) + else: + self.send_json({"state": workspace.read_view_state(key), "token": workspace.token}) + return + if path == "/api/session": + self.send_json(workspace.status()) + return + return super().do_GET() + + def do_POST(self): + if ( + self.path not in {"/api/weights", "/api/cancel", "/api/view-state"} + or self.headers.get("X-Explorer-Token") != workspace.token + ): + self.send_json({"error": "Invalid local session request"}, 403) + return + if self.path == "/api/view-state": + try: + length = int(self.headers.get("Content-Length", "0")) + if not 0 < length <= 65536: + raise ValueError("View state must be at most 64 KiB") + value = json.loads(self.rfile.read(length)) + workspace.write_view_state(value["key"], value["state"]) + self.send_json({"saved": True}) + except (OSError, ValueError, KeyError, TypeError) as error: + self.send_json({"error": str(error)}, 400) + return + if self.path == "/api/cancel": + workspace.cancel() + self.send_json(workspace.status()) + return + if not workspace.lock.acquire(blocking=False): + self.send_json({"error": "A checkpoint is already being processed"}, 409) + return + try: + size = int(self.headers.get("Content-Length", "0")) + if size <= 0: + raise ValueError("Choose a nonempty checkpoint file") + workspace.state = {"state": "uploading"} + name = unquote(self.headers.get("X-Weights-Name", "selected-weights.pt")).replace("\\", "/").rsplit("/", 1)[-1] + if not name or name in {".", ".."}: + raise ValueError("Invalid checkpoint filename") + inputs = workspace.root / "inputs" + inputs.mkdir(exist_ok=True) + upload = inputs / secrets.token_hex(8) + upload.mkdir() + weights = upload / name + self.connection.settimeout(120) + with weights.open("wb") as stream: + remaining = size + while remaining: + chunk = self.rfile.read(min(1024 * 1024, remaining)) + if not chunk: + raise ValueError("Checkpoint upload was interrupted") + stream.write(chunk) + remaining -= len(chunk) + workspace.start(weights) + self.send_json({"state": "running"}, 202) + except (OSError, ValueError) as error: + workspace.state = {"state": "error", "error": str(error)} + self.send_json(workspace.state, 400) + finally: + workspace.lock.release() + + return partial(SessionHandler, directory=str(workspace.public), service=workspace.service) + + +def run(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("weights", nargs="?", type=Path, help="mini_trainer weights; omit to choose a file in the browser") + parser.add_argument( + "--cache-dir", + type=Path, + default=Path(os.environ.get("XDG_CACHE_HOME", Path.home() / ".cache")) / "mini-trainer" / "prototype-analysis", + help="Persistent completed-analysis cache", + ) + parser.add_argument("--clear-cache", action="store_true", help="Clear completed analysis cache entries and exit") + parser.add_argument("--no-cache", action="store_true", help="Recompute analysis without reading or writing the cache") + parser.add_argument("--export", action="store_true", help="Write a portable HTML report and exit") + parser.add_argument("--output", type=Path, help="Export directory (default: -explorer)") + parser.add_argument("--pca-only", action="store_true", help="Skip the slower angular t-SNE projection") + parser.add_argument("--synthetic", action="store_true", help="Include synthetic comparison cases") + parser.add_argument("--threads", type=int, default=4, help="CPU analysis threads (default: 4)") + parser.add_argument("--port", type=int, default=0, help="Local server port (default: choose an available port)") + parser.add_argument("--no-browser", action="store_true", help="Print the local URL without opening a browser") + args = parser.parse_args() + if args.clear_cache: + from .cache import clear_analysis_cache + + print(f"Removed {clear_analysis_cache(args.cache_dir)} analysis cache entries") + return + if args.threads < 1: + parser.error("--threads must be positive") + if args.weights is not None and not args.weights.is_file(): + parser.error(f"Checkpoint does not exist: {args.weights}") + if args.export and args.weights is None: + parser.error("--export requires a weights path") + if args.output and not args.export: + parser.error("--output is used with --export") + try: + require_dependencies(not args.pca_only) + except RuntimeError as error: + parser.error(str(error)) + if args.export: + output = args.output or Path(args.weights.stem + "-explorer") + generate( + args.weights, + output, + angular=not args.pca_only, + synthetic=args.synthetic, + threads=args.threads, + cache_dir=None if args.no_cache else args.cache_dir, + ) + return + + from diskcache import Cache + + from mini_trainer.integrations import gbif + + from .serve import PhotoService + + with tempfile.TemporaryDirectory(prefix="mini-trainer-explorer-") as temporary: + workspace = Workspace(Path(temporary), not args.pca_only, args.synthetic, args.threads, None if args.no_cache else args.cache_dir) + with Cache(str(workspace.root / "photos"), size_limit=256 * 1024 * 1024) as cache: + previous_cache = gbif._CACHE + gbif._CACHE = cache + workspace.service = PhotoService([], cache) + server = ThreadingHTTPServer(("127.0.0.1", args.port), session_handler(workspace)) + if args.weights: + workspace.start(args.weights.resolve()) + url = f"http://localhost:{server.server_port}/" + print(f"Prototype explorer: {url}", flush=True) + if not args.no_browser: + webbrowser.open(url) + try: + server.serve_forever() + except KeyboardInterrupt: + pass + finally: + server.server_close() + workspace.close() + gbif._CACHE = previous_cache diff --git a/mini_trainer/visualization/prototype_space/launcher.html b/mini_trainer/visualization/prototype_space/launcher.html new file mode 100644 index 00000000..11e7dcad --- /dev/null +++ b/mini_trainer/visualization/prototype_space/launcher.html @@ -0,0 +1,15 @@ + + +Prototype explorer — open weights + +

Explore model prototypes

Choose a mini_trainer weight file. The explorer reads the classifier weights and class labels, then prepares the distance matrix, dendrogram and spatial projections.

+

Analysis runs locally and can take several minutes for large models. Optional GBIF images are requested only when you enable them in the viewer.

+
+

Choose a checkpoint to begin.

+ diff --git a/mini_trainer/visualization/prototype_space/photos.js b/mini_trainer/visualization/prototype_space/photos.js new file mode 100644 index 00000000..1668e0ef --- /dev/null +++ b/mini_trainer/visualization/prototype_space/photos.js @@ -0,0 +1,149 @@ +/* Optional reference photos using the shared browser GBIF client. */ +let photoController = null; +let photoGeneration = 0; +const photoAlbums = new Map(); +const photoChoices = new Map(); +const photoNumber = id => /^\d+$/.test(id); + +// Share metadata between map generations and the neighbourhood cards. Navigation +// discards queued work; up to four started lookups finish into the session cache. +const albumRequests=new Map(),albumQueue=[]; +let activeAlbumRequests=0; +function referenceAlbum(id,signal){ + if(photoAlbums.has(id))return Promise.resolve(photoAlbums.get(id)); + const existing=albumRequests.get(id);if(existing&&existing.generation===gbifClient.controller.signal){existing.signals.push(signal);return existing.promise;} + const job={id,signals:[signal],generation:gbifClient.controller.signal};job.promise=new Promise((resolve,reject)=>{job.resolve=resolve;job.reject=reject;}); + albumRequests.set(id,job);albumQueue.push(job);pumpAlbumRequests();return job.promise; +} +function pumpAlbumRequests(){ + while(activeAlbumRequests<4&&albumQueue.length){ + const job=albumQueue.shift(); + if(job.generation.aborted||job.signals.every(signal=>signal.aborted)){if(albumRequests.get(job.id)===job)albumRequests.delete(job.id);job.reject(new DOMException('Navigation cancelled','AbortError'));continue;} + activeAlbumRequests++; + (async()=>{try{ + if(!gbifEnabled())throw new DOMException('GBIF disabled','AbortError'); + const album=await gbifClient.album(job.id); + photoAlbums.set(job.id,album);job.resolve(album); + }catch(error){job.reject(error);}finally{if(albumRequests.get(job.id)===job)albumRequests.delete(job.id);activeAlbumRequests--;pumpAlbumRequests();}})(); + } +} + +function photoText(tag, text, className = '') { + const node = document.createElement(tag); + node.textContent = text; + node.className = className; + return node; +} + +function photoLink(text, href) { + const node = photoText('a', text); + node.href = href; + node.target = '_blank'; + node.rel = 'noopener noreferrer'; + return node; +} + +function loadReferenceImage(image, src, signal) { + return new Promise(resolve => { + let timer; + const done = ok => { + clearTimeout(timer); + image.onload = image.onerror = null; + signal.removeEventListener('abort', abort); + resolve(ok); + }; + const abort = () => { image.removeAttribute('src'); done(false); }; + if (signal.aborted) { done(false); return; } + signal.addEventListener('abort', abort, {once: true}); + image.onload = () => done(true); + image.onerror = () => done(false); + timer = setTimeout(abort, 18000); + // Keep the image visible during loading so progressive decoding can appear. + image.src = src; + }); +} + +async function showReference(card, album, classIndex, generation, signal) { + const photos = album.photos; + card.title.textContent = album.display_name; + if (!photos.length) { card.status.textContent = 'No matching class examples in the first 20 GBIF records.'; return; } + const choice = (photoChoices.get(album.class_id) || 0) % photos.length; + const photo = photos[choice]; + card.credit.replaceChildren( + photoText('span', photo.creator + ' · ' + photo.license + ' '), + photoLink('Source', photo.source), + photoText('span', ' · '), + photoLink('GBIF record', 'https://www.gbif.org/occurrence/' + photo.occurrence_id) + ); + card.image.alt = `GBIF example of ${album.display_name}; class ${album.class_id}`; + card.status.textContent = `Loading example ${choice + 1} / ${photos.length}…`; + card.next.hidden = photos.length < 2; + card.next.disabled = true; + card.next.onclick = async () => { + if (generation !== photoGeneration) return; + photoChoices.set(album.class_id, choice + 1); + card.next.disabled = true; + await showReference(card, album, classIndex, generation, signal); + if(typeof scheduleMapThumbnails==='function')scheduleMapThumbnails(); + card.next.disabled = false; + }; + const loaded = await loadReferenceImage(card.image, photo.image_path, signal); + if (generation !== photoGeneration) return; + card.next.disabled = false; + card.status.textContent = loaded ? `Example ${choice + 1} / ${photos.length}${album.accepted_id !== album.class_id ? ' · GBIF accepted ID ' + album.accepted_id : ''}` : 'Image unavailable or timed out. Try another example.'; +} + +async function renderPhotos() { + const generation = ++photoGeneration; + photoController?.abort(); + photoController = new AbortController(); + const signal = photoController.signal; + const grid = $('photo-grid'); + grid.replaceChildren(); + const ids = [selected, ...data.neighbours[selected]]; + if (data.metadata.synthetic || !ids.some(i => photoNumber(data.names[i]))) { + $('photo-status').textContent = 'This case has no GBIF-ID candidates. No online lookup was made.'; + return; + } + if (!gbifEnabled() || !$('photo-enabled').checked) { + $('photo-status').textContent = 'Select GBIF taxon IDs in Settings and enable reference photos to browse examples.'; + return; + } + $('photo-status').textContent = 'Loading class examples sequentially; class IDs and numerical neighbourhoods stay unchanged.'; + const cards = ids.map((classIndex, rank) => { + const card = photoText('article', '', 'photo-card' + (rank === 0 ? ' photo-selected' : '')); + const open = photoText('button', '', 'photo-open'); + open.type = 'button'; + open.setAttribute('aria-label', `Explore class ${data.names[classIndex]}`); + const image = document.createElement('img'); + image.decoding = 'async'; + open.append(image); + open.onclick = () => selectClass(classIndex); + const title = photoText('strong', data.names[classIndex], 'photo-title'); + const identity = photoText('span', `${rank === 0 ? 'Selected' : 'Neighbour ' + rank} · class ${data.names[classIndex]}`, 'caption'); + const status = photoText('p', 'Waiting…', 'caption'); + const next = photoText('button', 'Another example'); + next.type = 'button'; next.hidden = true; + const credit = photoText('p', '', 'photo-credit'); + card.append(open, title, identity, status, next, credit); + grid.append(card); + return {image, title, status, next, credit}; + }); + // Only fetch the visible neighbourhood. Cache metadata in-session and images + // on the local server; do not download one image for every checkpoint class. + for (let rank = 0; rank < ids.length; rank++) { + if (signal.aborted || generation !== photoGeneration) return; + const classIndex = ids[rank], id = data.names[classIndex], card = cards[rank]; + if (!photoNumber(id)) {card.status.textContent = 'Not a GBIF ID.'; continue;} + try { + const album = await referenceAlbum(id, signal); + if (signal.aborted || generation !== photoGeneration) return; + await showReference(card, album, classIndex, generation, signal); + } catch (error) { + if (!signal.aborted) card.status.textContent = 'Example lookup unavailable. Toggle photos to retry.'; + } + } + if (generation === photoGeneration) $('photo-status').textContent = 'Click a photo to explore that class’s neighbours. Images label classes at their learned prototype directions.'; +} +$('photo-enabled').onchange = renderPhotos; +renderPhotos(); diff --git a/mini_trainer/visualization/prototype_space/projection.js b/mini_trainer/visualization/prototype_space/projection.js new file mode 100644 index 00000000..1e8cc206 --- /dev/null +++ b/mini_trainer/visualization/prototype_space/projection.js @@ -0,0 +1,134 @@ +/* Coordinates are fitted offline. This canvas changes only the viewing transform. */ +let projectionCase = null; +let projectionView = {x:0, y:0, scale:1}; +let projectionDrag = null; +const projectionPointers=new Map(); +let projectionHits = []; +const projectionCanvas = $('projection-map'); +let projectionHeight=600; +function currentProjection() { return data.projections?.[$('projection-plane').value]; } +function projectionPoint(point) {return [600+(point[0]-projectionView.x)*projectionView.scale,projectionHeight/2-(point[1]-projectionView.y)*projectionView.scale];} +function fitProjection(ids) { + const p=currentProjection();if(!p)return; + const points=ids.map(i=>p.coordinates[i]); + let x0=Infinity,x1=-Infinity,y0=Infinity,y1=-Infinity; + for(const [x,y] of points){x0=Math.min(x0,x);x1=Math.max(x1,x);y0=Math.min(y0,y);y1=Math.max(y1,y);} + projectionView={x:(x0+x1)/2,y:(y0+y1)/2,scale:Math.min(1050/Math.max(.0001,x1-x0),Math.max(100,projectionHeight-150)/Math.max(.0001,y1-y0))}; +} +function drawProjection(refreshThumbnails=true) { + resizeProjectionRaster(); + const pixels=1200/(projectionCanvas.getBoundingClientRect().width||1200); + const ctx=projectionCanvas.getContext('2d');ctx.clearRect(0,0,1200,projectionHeight); + if(projectionCase!==data){ + projectionCase=data;$('projection-plane').replaceChildren(); + for(const name of Object.keys(data.projections||{})){const option=document.createElement('option');option.value=option.textContent=name;$('projection-plane').append(option);} + if(data.projections?.['Angular t-SNE'])$('projection-plane').value='Angular t-SNE'; + fitProjection(data.names.map((_,i)=>i)); + } + const p=currentProjection();if(!p){$('projection-quality').textContent='Regenerate this report to include spatial projections.';return;} + projectionHits=p.coordinates.map(projectionPoint); + const original=new Set(data.neighbours[selected]),projected=new Set(p.neighbours[selected]); + ctx.lineWidth=pixels;ctx.strokeStyle='#dce3e9';ctx.fillStyle='#52687b';ctx.font=`${12*pixels}px system-ui`; + // Both axes use the same units per pixel, including after pan/zoom. + for(let i=0;i<=4;i++){ + const x=60+i*270,y=30+i*(projectionHeight-60)/4; + ctx.beginPath();ctx.moveTo(x,20);ctx.lineTo(x,projectionHeight-30);ctx.stroke(); + ctx.fillText((projectionView.x+(x-600)/projectionView.scale).toPrecision(3),x+3,projectionHeight-10); + ctx.beginPath();ctx.moveTo(35,y);ctx.lineTo(1180,y);ctx.stroke(); + ctx.fillText((projectionView.y-(y-projectionHeight/2)/projectionView.scale).toPrecision(3),2,y+13); + } + ctx.fillText(p.axes[0]+' →',1120,15);ctx.fillText(p.axes[1],5,15); + for(let i=0;i1200||y<0||y>projectionHeight)continue; + ctx.fillStyle='#7c90a34d';ctx.beginPath();ctx.arc(x,y,2*pixels,0,2*Math.PI);ctx.fill(); + } + const [sx,sy]=projectionHits[selected]; + if($('projection-edges').checked){ctx.strokeStyle='#087e8b88';for(const i of original){const [x,y]=projectionHits[i];ctx.beginPath();ctx.moveTo(sx,sy);ctx.lineTo(x,y);ctx.stroke();}} + for(const i of projected){if(original.has(i))continue;const [x,y]=projectionHits[i];ctx.strokeStyle='#c87719';ctx.lineWidth=1.8*pixels;ctx.beginPath();ctx.arc(x,y,5*pixels,0,2*Math.PI);ctx.stroke();} + ctx.font=`${12*pixels}px system-ui`; + for(const i of original){const [x,y]=projectionHits[i];ctx.fillStyle='#087e8b';ctx.beginPath();ctx.arc(x,y,4.5*pixels,0,2*Math.PI);ctx.fill();if(!$('map-photos').checked)ctx.fillText(classLabel(i),x+7,y-5);} + ctx.fillStyle='#c2344b';ctx.beginPath();ctx.arc(sx,sy,6*pixels,0,2*Math.PI);ctx.fill();ctx.font=`bold ${13*pixels}px system-ui`;if(!$('map-photos').checked)ctx.fillText(classLabel(selected),sx+9,sy+17); + if(typeof drawInferenceQuery==='function')drawInferenceQuery(ctx,p,pixels); + const fractions=p.variance_fraction?.map(v=>(v*100).toFixed(2)); + const summary=fractions?`Variance retained: ${(100*p.variance_fraction.reduce((a,b)=>a+b,0)).toFixed(2)}% (${p.axes[0]} ${fractions[0]}%, ${p.axes[1]} ${fractions[1]}%). `:`Angular t-SNE · perplexity ${p.parameters.perplexity.toFixed(1)} · seed ${p.parameters.seed}. Map gaps and areas do not measure spherical distances or areas. `; + $('projection-quality').textContent=`${summary}Original top-${data.stats.k} neighbours retained among plane top-${data.stats.k}: ${(100*p.mean_retained_fraction).toFixed(1)}% averaged over classes; ${(100*p.retained_fraction[selected]).toFixed(1)}% for selected class. Exact plane-distance ties use class order.`; + if(refreshThumbnails!==false&&typeof scheduleMapThumbnails==='function')scheduleMapThumbnails(); +} +function projectionMouse(event){const r=projectionCanvas.getBoundingClientRect();return [(event.clientX-r.left)*1200/r.width,(event.clientY-r.top)*projectionHeight/r.height];} +function projectionHit(point){if(typeof mapThumbnailHit==='function'){const hit=mapThumbnailHit(point);if(hit>=0)return hit;}let best=-1,distance=(10*1200/(projectionCanvas.getBoundingClientRect().width||1200))**2;projectionHits.forEach(([x,y],i)=>{const d=(x-point[0])**2+(y-point[1])**2;if(d{projectionPaintFrame=null;drawProjection();}); +} +function flushProjectionDraw(){if(projectionPaintFrame!==null){cancelAnimationFrame(projectionPaintFrame);projectionPaintFrame=null;drawProjection();}} +function beginProjectionGesture(moved=false){ + const points=[...projectionPointers.values()]; + if(!points.length){projectionDrag=null;return;} + const a=points[0],b=points[1]; + projectionDrag={point:b?[(a[0]+b[0])/2,(a[1]+b[1])/2]:a, + distance:b?Math.max(1,Math.hypot(a[0]-b[0],a[1]-b[1])):null, + view:{...projectionView},moved:moved||!!b}; +} +projectionCanvas.onpointerdown=e=>{ + if(e.button!==0)return; + projectionPointers.set(e.pointerId,projectionMouse(e)); + beginProjectionGesture(projectionDrag?.moved);projectionCanvas.setPointerCapture(e.pointerId); +}; +projectionCanvas.onpointermove=e=>{ + const point=projectionMouse(e); + if(projectionPointers.has(e.pointerId)&&projectionDrag){ + projectionPointers.set(e.pointerId,point); + const points=[...projectionPointers.values()],a=points[0],b=points[1],g=projectionDrag; + const center=b?[(a[0]+b[0])/2,(a[1]+b[1])/2]:a; + const dx=center[0]-g.point[0],dy=center[1]-g.point[1]; + if(Math.hypot(dx,dy)>3)g.moved=true; + const scale=b&&g.distance?Math.max(1,Math.min(1e7,g.view.scale*Math.hypot(a[0]-b[0],a[1]-b[1])/g.distance)):g.view.scale; + projectionView={scale,x:g.view.x+(g.point[0]-600)/g.view.scale-(center[0]-600)/scale, + y:g.view.y-(g.point[1]-projectionHeight/2)/g.view.scale+(center[1]-projectionHeight/2)/scale}; + queueProjectionDraw();return; + } + if(projectionDrag)return; + const id=projectionHit(point); + if(typeof showMapPhotoCredit==='function')showMapPhotoCredit(id); + $('projection-hover').textContent=id<0?'Hover a point for its class ID.':`Class ${data.names[id]} · checkpoint row ${id}${id===selected?' · selected':data.neighbours[selected].includes(id)?' · original-space neighbour':currentProjection().neighbours[selected].includes(id)?' · nearby in this plane only':''}`; +}; +function endProjectionGesture(e){ + if(!projectionPointers.has(e.pointerId))return; + const click=e.type==='pointerup'&&projectionPointers.size===1&&!projectionDrag?.moved; + projectionPointers.delete(e.pointerId);beginProjectionGesture(true); + if(projectionCanvas.hasPointerCapture(e.pointerId))projectionCanvas.releasePointerCapture(e.pointerId); + flushProjectionDraw(); + if(click){const id=projectionHit(projectionMouse(e));if(id>=0)selectClass(id);} +} +projectionCanvas.onpointerup=projectionCanvas.onpointercancel=projectionCanvas.onlostpointercapture=endProjectionGesture; +projectionCanvas.addEventListener('wheel',e=>{ + e.preventDefault();const [x,y]=projectionMouse(e),old=projectionView.scale; + const scale=Math.max(1,Math.min(1e7,old*Math.exp(-Math.max(-300,Math.min(300,e.deltaY))*.002))); + projectionView.x+=(x-600)*(1/old-1/scale);projectionView.y-=(y-projectionHeight/2)*(1/old-1/scale);projectionView.scale=scale;queueProjectionDraw(); +},{passive:false}); +$('projection-plane').onchange=()=>{fitProjection(data.names.map((_,i)=>i));drawProjection();}; +$('projection-reset').onclick=()=>{fitProjection(data.names.map((_,i)=>i));drawProjection();}; +$('projection-focus').onclick=()=>{fitProjection([selected,...data.neighbours[selected]]);drawProjection();}; +$('projection-edges').onchange=drawProjection; +drawProjection(); + +let projectionScreenWidth=0; +new ResizeObserver(()=>{ + const rect=projectionCanvas.getBoundingClientRect();if(!rect.width||!rect.height)return; + const nextHeight=1200*rect.height/rect.width; + if(Math.abs(nextHeight-projectionHeight)<.01&&Math.abs(rect.width-projectionScreenWidth)<.1)return; + if(projectionScreenWidth)projectionView.scale*=projectionScreenWidth/rect.width; + projectionScreenWidth=rect.width;projectionHeight=nextHeight; + resizeProjectionRaster(); + drawProjection(); +}).observe(projectionCanvas); + +function resizeProjectionRaster(){ + const rect=projectionCanvas.getBoundingClientRect();if(!rect.width||!rect.height)return; + const dpr=window.devicePixelRatio||1,w=Math.round(rect.width*dpr),h=Math.round(rect.height*dpr); + if(projectionCanvas.width!==w||projectionCanvas.height!==h){projectionCanvas.width=w;projectionCanvas.height=h;} + projectionCanvas.getContext('2d').setTransform(w/1200,0,0,h/projectionHeight,0,0); +} +window.addEventListener('resize',()=>{if(projectionCanvas.getBoundingClientRect().width)drawProjection();}); diff --git a/mini_trainer/visualization/prototype_space/projection.py b/mini_trainer/visualization/prototype_space/projection.py new file mode 100644 index 00000000..e5c85200 --- /dev/null +++ b/mini_trainer/visualization/prototype_space/projection.py @@ -0,0 +1,113 @@ +"""Linear spatial views with measured neighbourhood preservation.""" + +import numpy as np +import torch +from scipy.spatial import cKDTree + + +def projected_neighbours(coordinates, k): + """Euclidean neighbours in the displayed plane; exact ties use class order.""" + coordinates = np.asarray(coordinates, dtype=np.float64) + tree = cKDTree(coordinates) + distances, _ = tree.query(coordinates, k=k + 1) + result = [] + for i, radius in enumerate(distances[:, -1]): + candidates = np.asarray(tree.query_ball_point(coordinates[i], np.nextafter(radius, np.inf)), dtype=int) + candidates = candidates[candidates != i] + squared = ((coordinates[candidates] - coordinates[i]) ** 2).sum(axis=1) + result.append(candidates[np.lexsort((candidates, squared))[:k]].tolist()) + return result + + +@torch.no_grad() +def prototype_projections(weight, neighbours): + """Project the same row directions used in the cosine diagnostic. + + PCA centers unit rows without whitening or feature standardization. It does + not change prototype weights, repository distances or their neighbour ranks. + Both displayed axes share one spatial scale. Fit each case independently. + """ + weight = weight.detach().cpu().float() + norms = weight.norm(dim=1, keepdim=True) + if not torch.isfinite(weight).all() or (norms == 0).any(): + raise ValueError("Projection requires finite, nonzero prototype rows") + directions = weight / norms + centered = directions - directions.mean(dim=0) + # Symmetric covariance eigendecomposition avoids an N x N pairwise matrix. + gram = centered.T @ centered + eigenvalues, eigenvectors = torch.linalg.eigh(gram) + total = centered.square().sum().item() + count = min(4, weight.shape[1]) + axes = eigenvectors[:, -count:].flip(1) + # Fix the otherwise arbitrary sign; degenerate eigenspaces may still rotate. + pivots = axes.abs().argmax(dim=0) + axes *= torch.where(axes[pivots, torch.arange(count)] < 0, -1.0, 1.0) + coordinates = (centered @ axes).numpy() + variances = eigenvalues[-count:].flip(0).clamp_min(0).numpy() + result = {} + original = [set(row) for row in neighbours] + k = len(neighbours[0]) + for start in range(0, count - 1, 2): + points = coordinates[:, start : start + 2] + near = projected_neighbours(points, k) + retained = [len(original[i].intersection(row)) / k for i, row in enumerate(near)] + key = f"PCA {start + 1}–{start + 2}" + result[key] = { + "coordinates": points.tolist(), + "transform": {"mean": directions.mean(dim=0).tolist(), "basis": axes[:, start : start + 2].tolist(), "normalize": True}, + "axes": [f"PC {start + 1}", f"PC {start + 2}"], + "variance_fraction": (variances[start : start + 2] / total).tolist() if total > 0 else [0.0, 0.0], + "neighbours": near, + "retained_fraction": retained, + "mean_retained_fraction": float(np.mean(retained)), + "method": "Centered PCA of unit prototype directions; float32 covariance eigendecomposition; no whitening", + } + return result + + +def angular_distances(z, dimensions): + """Invert the repository's clamped cosine-to-z transform to radians.""" + angles = np.maximum(0, np.pi / 2 - np.asarray(z, dtype=np.float32) / np.sqrt(np.float32(dimensions - 2))) + np.fill_diagonal(angles, 0) + return angles + + +def angular_tsne(z, dimensions, neighbours, seed=42, *, iterations=1000): + """An explicit nonlinear display of angular neighbourhoods, not a new metric.""" + from sklearn.manifold import TSNE + + perplexity = min(30.0, (len(z) - 1) / 3) + estimator = TSNE( + metric="precomputed", + init="random", + perplexity=perplexity, + random_state=seed, + learning_rate="auto", + max_iter=iterations, + method="barnes_hut", + angle=0.5, + n_jobs=4, + ) + points = estimator.fit_transform(angular_distances(z, dimensions)) + near = projected_neighbours(points, len(neighbours[0])) + retained = [len(set(a).intersection(b)) / len(a) for a, b in zip(neighbours, near)] + return { + "coordinates": points.tolist(), + "axes": ["t-SNE 1", "t-SNE 2"], + "variance_fraction": None, + "neighbours": near, + "retained_fraction": retained, + "mean_retained_fraction": float(np.mean(retained)), + "method": ( + "t-SNE of angular distances recovered from repository z; map areas and inter-cluster gaps are not spherical areas or distances" + ), + "parameters": { + "perplexity": perplexity, + "seed": seed, + "iterations": iterations, + "init": "random", + "metric": "precomputed angular radians", + "angle": 0.5, + }, + "kl_divergence": float(estimator.kl_divergence_), + } diff --git a/mini_trainer/visualization/prototype_space/report.html b/mini_trainer/visualization/prototype_space/report.html new file mode 100644 index 00000000..9be37c39 --- /dev/null +++ b/mini_trainer/visualization/prototype_space/report.html @@ -0,0 +1,370 @@ + + + + +Prototype observatory + + +
mini_trainer · empirical exploration

Prototype observatory

Move between the global tree, pairwise structure and a class’s local neighbourhood. The existing mini_trainer distance is the reference; the pre-CDF z-score exposes detail that can be hidden by probability rounding.

+
Explorer · view & class
+ +Save HTML report + + + + +
Names and class identity
+ +
+
+
+

Try your own image

+

Run this model locally in your browser. Images are not uploaded. Model download and inference may take time on this device.

+
Model source
+
+
+

Load the matching browser bundle to begin.

+ +

+
+
+

Distribution of pairwise z-scores

The actual transform used by mini_trainer, before the Normal CDF. Self-pairs excluded. No fitted distribution or assumed organization.

+

Neighbourhood signature

Similarity to the 1st, 5th, 12th and 32nd neighbour in the full space. High values across ranks indicate several strongly aligned directions.

Teal: selected class. Grey: per-rank median across classes. Rank points are not an embedding projection.

+
+
Map controls

Prototype space · spatial projection

+

Compare linear PCA with angular t-SNE when available. Scroll or pinch to zoom, drag to pan, tap a point or thumbnail to inspect its class. Each case is fitted separately.

+
Thumbnail settings
+
+
Map details & credits

+

Hover a point for its class ID.

Red: selected class · teal: its neighbours in the original space · orange rings: nearby in this plane only · grey: other classes. Links follow the original-space ranking. Projected gaps and apparent overlaps can differ from the full embedding geometry.

+

Browse by class image

Photo labels for the selected class and its nearest prototype directions. The prototype marks the learned target direction for the class; a GBIF class example provides its visual label. Credits and exact class IDs stay attached.

+
+

Explore the existing dendrogram

+

Same Ward linkage on class_distance, shown as a rectangular tree with at most 32 visible tips. Click a collapsed tip to expand its subtree; click a class to inspect it. Heights retain the existing linkage units.

+

+
+
+

What survives matrix aggregation?

Both panels represent the current heatmap quantity, 1 − CDF(z), in the same dendrogram leaf order. The log-domain option evaluates log Φ(−z) directly in float32. Legacy mode exposes CDF saturation; the two panels compare block summaries. Self-pairs and padding are excluded from both.

+

Clipping affects colours only, not numerical values or neighbour rankings.

+

Maximum dissimilarity per block

Uses the current renderer’s maximum-summary idea. A distant pair can dominate a block.

Closest pair per block

An explicit alternative. Reveals isolated close pairs; does not imply the entire block is close.

+

+
+
+

Local observatory ·

+

+

Hover a cell for its exact stored values; click to select its column class.

Nearest directions

Ranked by repository z-score, descending. This explicitly refines ties in baseline distance; exactly tied z-scores retain checkpoint class order.

RankClass IDBaseline dlog₁₀ tailzSame parent
+
+

Questions this opens up

With prototype weights alone

Compare local crowding across taxonomy; follow cross-branch nearest neighbours; measure how many baseline distances tie; examine neighbour stability across checkpoints. Keep alternate metrics alongside the reference rather than silently replacing it.

With held-out image embeddings

Add sample-to-prototype alignment, decision margins, observed occupancy and actual confusion. Those quantities are not inferred from this weight-only report. A sampled decision-territory experiment would be a separate geometric reference.

Provenance and limits
Class and parent IDs come from checkpoint metadata; no online taxonomic remapping. A single checkpoint does not establish a training trajectory. Synthetic cases are deliberately constructed display/calculation probes, not targets. Their smaller class counts do not provide a matched packing comparison. Zero distance may result from CDF rounding and does not prove identical vectors. Grey matrix cells contain no off-diagonal pairs; display flooring never alters stored numerical values.

+
Self-contained numerical report · optional GBIF photo lookup when enabled · computations use mini_trainer’s class_similarity and class_distance APIs.

+

+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/mini_trainer/visualization/prototype_space/serve.py b/mini_trainer/visualization/prototype_space/serve.py
new file mode 100644
index 00000000..9578d857
--- /dev/null
+++ b/mini_trainer/visualization/prototype_space/serve.py
@@ -0,0 +1,209 @@
+"""Serve the local prototype report with optional, cached GBIF reference photos."""
+
+import argparse
+import hashlib
+import json
+import re
+import threading
+from functools import partial
+from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
+from pathlib import Path
+from urllib.error import HTTPError
+from urllib.parse import urlsplit
+from urllib.request import urlopen
+
+from diskcache import Cache
+
+from mini_trainer.integrations import gbif
+
+
+def web_url(value):
+    return isinstance(value, str) and urlsplit(value).scheme in {"http", "https"}
+
+
+def reference_photos(class_id, taxon, occurrences, limit=6):
+    """Keep requested IDs, verify taxonomic membership, and retain media credits.
+
+    These are GBIF examples, not images selected by model similarity. In
+    particular, an occurrence's data license is not assumed to license its photo.
+    """
+    accepted = str(taxon.get("acceptedKey") or class_id)
+    photos, seen = [], set()
+    for occurrence in occurrences:
+        membership = {
+            str(occurrence.get(key))
+            for key in (
+                "taxonKey",
+                "acceptedTaxonKey",
+                "speciesKey",
+                "genusKey",
+                "familyKey",
+                "orderKey",
+                "classKey",
+                "phylumKey",
+                "kingdomKey",
+            )
+        }
+        occurrence_id = str(occurrence.get("key", ""))
+        if not {str(class_id), accepted}.intersection(membership) or not occurrence_id.isdigit():
+            continue
+        for media in occurrence.get("media", []):
+            identifier = media.get("identifier")
+            if media.get("type") != "StillImage" or not web_url(identifier) or identifier in seen:
+                continue
+            seen.add(identifier)
+            digest = hashlib.md5(identifier.encode(), usedforsecurity=False).hexdigest()
+            photos.append(
+                {
+                    "occurrence_id": occurrence_id,
+                    "image_path": f"/gbif-image/{occurrence_id}/{digest}",
+                    "original": identifier,
+                    "source": media.get("references")
+                    if web_url(media.get("references"))
+                    else f"https://www.gbif.org/occurrence/{occurrence_id}",
+                    "creator": media.get("creator") or media.get("rightsHolder") or "Creator not supplied",
+                    "license": media.get("license") or "Image license not supplied",
+                    "scientific_name": occurrence.get("scientificName", ""),
+                }
+            )
+            if len(photos) >= limit:
+                break
+        if len(photos) >= limit:
+            break
+    return {
+        "class_id": str(class_id),
+        "display_name": taxon.get("canonicalName") or taxon.get("scientificName") or str(class_id),
+        "accepted_id": accepted,
+        "taxonomic_status": taxon.get("taxonomicStatus"),
+        "photos": photos,
+        "selection": "GBIF class examples",
+    }
+
+
+class PhotoService:
+    def __init__(self, allowed_ids, cache):
+        self.allowed_ids = set(allowed_ids)
+        self.cache = cache
+        self.image_paths = set()
+        self.metadata_cache = {}
+        # GBIF asks scripted image-cache users to use a single HTTP connection.
+        # Serialize all upstream requests, including image proxy requests.
+        self.lock = threading.Lock()
+
+    def taxon_name(self, class_id):
+        """Resolve a name without occurrence or image lookups; preserve the ID."""
+        if class_id not in self.allowed_ids or not class_id.isdigit():
+            raise ValueError("Class is not a GBIF-ID candidate in this report")
+        with self.lock:
+            taxon = gbif.retrive_request(f"{gbif.GBIF_SPECIES_API_ENDPOINT}{class_id}")
+        return {"class_id": class_id, "display_name": taxon.get("canonicalName") or taxon.get("scientificName") or class_id}
+
+    def metadata(self, class_id):
+        if class_id not in self.allowed_ids or not class_id.isdigit():
+            raise ValueError("Class is not a GBIF-ID candidate in this report")
+        if class_id in self.metadata_cache:
+            return self.metadata_cache[class_id]
+        with self.lock:
+            if class_id in self.metadata_cache:
+                return self.metadata_cache[class_id]
+            taxon = gbif.retrive_request(f"{gbif.GBIF_SPECIES_API_ENDPOINT}{class_id}")
+            result = gbif.retrive_request(f"https://api.gbif.org/v1/occurrence/search?taxonKey={class_id}&mediaType=StillImage&limit=20")
+            data = reference_photos(class_id, taxon, result.get("results", []))
+            self.image_paths.update(photo["image_path"] for photo in data["photos"])
+            self.metadata_cache[class_id] = data
+            return data
+
+    def image(self, path):
+        if path not in self.image_paths:
+            raise ValueError("Image has not been resolved for a class in this report")
+        cache_key = "prototype-image:" + path
+        cached = self.cache.get(cache_key)
+        if cached is not None:
+            return cached
+        with self.lock:
+            cached = self.cache.get(cache_key)
+            if cached is not None:
+                return cached
+            occurrence, digest = path.split("/")[-2:]
+            url = f"https://api.gbif.org/v1/image/cache/400x/occurrence/{occurrence}/media/{digest}"
+            with urlopen(url, timeout=15) as response:
+                content_type = response.headers.get_content_type()
+                if content_type not in {"image/jpeg", "image/png", "image/webp", "image/gif"}:
+                    raise ValueError("Upstream response was not a supported image")
+                content = response.read(8 * 1024 * 1024 + 1)
+            if len(content) > 8 * 1024 * 1024:
+                raise ValueError("Thumbnail exceeded the 8 MiB limit")
+            result = content, content_type
+            self.cache.set(cache_key, result, expire=gbif.CACHE_TIME)
+            return result
+
+
+class Handler(SimpleHTTPRequestHandler):
+    def __init__(self, *args, service, **kwargs):
+        self.service = service
+        super().__init__(*args, **kwargs)
+
+    def send_json(self, payload, status=200):
+        content = json.dumps(payload).encode()
+        self.send_response(status)
+        self.send_header("Content-Type", "application/json")
+        self.send_header("Content-Length", str(len(content)))
+        self.end_headers()
+        self.wfile.write(content)
+
+    def do_GET(self):
+        path = urlsplit(self.path).path
+        try:
+            if path == "/api/health":
+                return self.send_json({"photo_api": True})
+            if path.startswith("/api/taxon/"):
+                return self.send_json(self.service.taxon_name(path.removeprefix("/api/taxon/")))
+            if path.startswith("/api/gbif/"):
+                return self.send_json(self.service.metadata(path.removeprefix("/api/gbif/")))
+            if re.fullmatch(r"/gbif-image/\d+/[a-f0-9]{32}", path):
+                content, content_type = self.service.image(path)
+                self.send_response(200)
+                self.send_header("Content-Type", content_type)
+                self.send_header("Content-Length", str(len(content)))
+                self.send_header("Cache-Control", "private, max-age=86400")
+                self.end_headers()
+                self.wfile.write(content)
+                return None
+            return super().do_GET()
+        except (BrokenPipeError, ConnectionResetError):
+            return None  # Navigation can cancel a queued photo request.
+        except (ValueError, HTTPError, OSError, RuntimeError) as error:
+            try:
+                return self.send_json({"error": str(error)}, status=502)
+            except (BrokenPipeError, ConnectionResetError):
+                return None
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--directory", type=Path, default=Path("tmp/prototype-report"))
+    parser.add_argument("--port", type=int, default=8765)
+    args = parser.parse_args()
+    directory = args.directory.resolve()
+    data = json.loads((directory / "report-data.json").read_text())
+    ids = {name for case in data.values() if not case["metadata"].get("synthetic") for name in case["names"] if name.isdigit()}
+    del data
+    # Reuse the repository request cache without writing into the user's shared
+    # home cache. Keep it outside the directory exposed by the report server.
+    cache = Cache(str(directory.with_name(directory.name + "-gbif-cache")), size_limit=256 * 1024 * 1024)
+    gbif._CACHE = cache
+    service = PhotoService(ids, cache)
+    handler = partial(Handler, directory=str(directory), service=service)
+    server = ThreadingHTTPServer(("127.0.0.1", args.port), handler)
+    print(f"Prototype + optional GBIF photos: http://localhost:{args.port}/explorer.html", flush=True)
+    try:
+        server.serve_forever()
+    except KeyboardInterrupt:
+        pass
+    finally:
+        server.server_close()
+        cache.close()
+
+
+if __name__ == "__main__":
+    main()
diff --git a/mini_trainer/visualization/prototype_space/state.js b/mini_trainer/visualization/prototype_space/state.js
new file mode 100644
index 00000000..3a8bbac8
--- /dev/null
+++ b/mini_trainer/visualization/prototype_space/state.js
@@ -0,0 +1,63 @@
+/* Versioned presentation state. Numerical data and transient image motion are excluded. */
+const viewerStateControls=['workspace-view','projection-plane','metric','global-metric','range-preset','range-low','range-high','score-display','class-names','class-id-namespace','photo-enabled','map-photos','map-photo-size','map-photo-density','map-photo-overlap','map-photo-labels','map-photo-push','map-photo-aspect','map-photo-anchors','projection-edges'];
+let restoringViewerState=false,viewerStateTimer,viewerServerToken=null,viewerStateChanged=false,viewerStateWrites=Promise.resolve();
+const viewerDocumentKey='prototype-case:'+(Object.values(cases).find(c=>!c.metadata.synthetic)?.metadata.checkpoint_sha256||Object.keys(cases).join(':'));
+function viewerIdentity(){
+ let hash=2166136261;for(const name of data.names)for(const c of name+'\0')hash=Math.imul(hash^c.charCodeAt(0),16777619);
+ return [data.metadata.checkpoint_sha256||'synthetic',data.metadata.seed||0,$('case').value,data.names.length,hash>>>0].join(':');
+}
+function captureViewerState(){
+ return {version:2,identity:viewerIdentity(),selected:data.names[selected],tree:treeFocus,view:{...projectionView},controls:Object.fromEntries(viewerStateControls.map(id=>[id,$(id).type==='checkbox'?$(id).checked:$(id).value]))};
+}
+function stateMessage(message){$('viewer-state-status').textContent=message;}
+function saveViewerState(){
+ if(restoringViewerState)return;
+ try{localStorage.setItem('prototype-view:'+viewerIdentity(),JSON.stringify(captureViewerState()));localStorage.setItem(viewerDocumentKey,$('case').value);stateMessage('View saved locally.');}
+ catch{stateMessage('Browser storage unavailable. Export state to keep this view.');}
+ if(viewerServerToken){const body=JSON.stringify({key:viewerDocumentKey,state:{case:$('case').value,view:captureViewerState()}});viewerStateWrites=viewerStateWrites.then(()=>fetch('/api/view-state',{method:'POST',headers:{'Content-Type':'application/json','X-Explorer-Token':viewerServerToken},body,keepalive:true})).catch(()=>stateMessage('Local server state unavailable; browser save/export remains available.'));}
+}
+function applyViewerState(state){
+ if(![1,2].includes(state?.version)||state.identity!==viewerIdentity())throw Error('State belongs to a different model/case or unsupported version.');
+ if(!state.controls||typeof state.controls!=='object'||Array.isArray(state.controls))throw Error('Missing saved controls');
+ if(state.version===1){const controls={...state.controls};controls['class-id-namespace']=['gbif-names','photo-enabled','map-photos'].some(id=>controls[id]===true)?'gbif':'generic';delete controls['gbif-names'];state={...state,version:2,controls};}
+ const index=data.names.indexOf(state.selected),v=state.view;
+ if(index<0||!v||![v.x,v.y,v.scale].every(Number.isFinite)||v.scale<=0||!Number.isInteger(state.tree)||state.tree<0||state.tree>=data.names.length*2-1)throw Error('Invalid saved selection or geometry.');
+ for(const [id,value] of Object.entries(state.controls)){
+  if(!viewerStateControls.includes(id))continue;const node=$(id);
+  if(node.type==='checkbox'){if(typeof value!=='boolean')throw Error('Invalid checkbox state.');}
+  else if(node.tagName==='SELECT'){if(![...node.options].some(o=>o.value===value))throw Error('Unavailable saved view or projection.');}
+  else if(!Number.isFinite(Number(value))||(node.min!==''&&Number(value)Number(node.max)))throw Error('Invalid saved control range.');
+ }
+ restoringViewerState=true;
+ try{
+  for(const [id,value] of Object.entries(state.controls)){if(!viewerStateControls.includes(id))continue;const node=$(id);if(node.type==='checkbox')node.checked=value;else node.value=value;}
+  $('class-id-namespace').onchange();
+  setWorkspaceView($('workspace-view').value);treeFocus=state.tree;selectClass(index);
+  $('score-note').hidden=$('score-display').value!=='tail';drawHistogram();drawGlobal();setThumbnailAnchors();
+  // Wait for the new focused layout's ResizeObserver before restoring its transform.
+  requestAnimationFrame(()=>requestAnimationFrame(()=>{projectionView={...v};drawProjection();restoringViewerState=false;stateMessage('Saved view restored.');}));
+ }catch(error){restoringViewerState=false;throw error;}
+}
+function restoreViewerState(){
+ try{const raw=localStorage.getItem('prototype-view:'+viewerIdentity());if(raw)applyViewerState(JSON.parse(raw));else stateMessage('No saved view for this model/case.');}
+ catch(error){stateMessage('Saved view not restored: '+error.message);}
+}
+function queueViewerState(){if(restoringViewerState)return;viewerStateChanged=true;clearTimeout(viewerStateTimer);viewerStateTimer=setTimeout(saveViewerState,400);}
+const stateMenu=document.createElement('details');stateMenu.className='label-settings';stateMenu.innerHTML='Saved view
';
+$('shared-settings-content').append(stateMenu);
+$('viewer-state-export').onclick=()=>{const url=URL.createObjectURL(new Blob([JSON.stringify(captureViewerState(),null,2)],{type:'application/json'}));const a=document.createElement('a');a.href=url;a.download='prototype-view.json';a.click();setTimeout(()=>URL.revokeObjectURL(url),1000);};
+$('viewer-state-import').onchange=async()=>{try{const file=$('viewer-state-import').files[0];if(file)applyViewerState(JSON.parse(await file.text()));}catch(error){stateMessage(error.message);}};
+$('viewer-state-reset').onclick=async()=>{restoringViewerState=true;clearTimeout(viewerStateTimer);try{localStorage.removeItem('prototype-view:'+viewerIdentity());await viewerStateWrites;if(viewerServerToken)await fetch('/api/view-state',{method:'POST',headers:{'Content-Type':'application/json','X-Explorer-Token':viewerServerToken},body:JSON.stringify({key:viewerDocumentKey,state:null})});}catch{}location.reload();};
+document.addEventListener('change',event=>{if(event.target.id==='case')restoreViewerState();else queueViewerState();});
+document.addEventListener('click',queueViewerState);document.addEventListener('pointerup',queueViewerState);document.addEventListener('wheel',queueViewerState,{passive:true});
+window.addEventListener('pagehide',saveViewerState);
+try{const savedCase=localStorage.getItem(viewerDocumentKey);if(savedCase&&cases[savedCase]&&$('case').value!==savedCase){$('case').value=savedCase;setCase();}}catch{}
+restoreViewerState();
+
+// The local launcher can change port between sessions, unlike browser storage.
+if(location.protocol.startsWith('http'))fetch('/api/view-state?key='+encodeURIComponent(viewerDocumentKey)).then(r=>r.ok?r.json():null).then(result=>{
+ if(!result)return;viewerServerToken=result.token;
+ if(!viewerStateChanged&&result.state?.view&&cases[result.state.case]){
+  $('case').value=result.state.case;setCase();applyViewerState(result.state.view);
+ }
+}).catch(()=>{});
diff --git a/mini_trainer/visualization/prototype_space/thumbnails.js b/mini_trainer/visualization/prototype_space/thumbnails.js
new file mode 100644
index 00000000..d426c6c2
--- /dev/null
+++ b/mini_trainer/visualization/prototype_space/thumbnails.js
@@ -0,0 +1,322 @@
+/* Optional label relaxation changes thumbnail positions only, never prototype coordinates. */
+let mapThumbTimer = null, mapThumbController = null, mapThumbGeneration = 0;
+let mapThumbLayout = [];
+const mapThumbImages = new Map();
+const mapThumbFailures = new Set();
+function thumbnailSharedArea(a,b){
+ return Math.max(0,Math.min(a.left+a.width,b.left+b.width)-Math.max(a.left,b.left))*Math.max(0,Math.min(a.top+a.height,b.top+b.height)-Math.max(a.top,b.top));
+}
+function thumbnailEnergy(boxes,gradient=false){
+ let energy=0,overlap=0,pairs=0;
+ // Diagonal contact-curvature estimate preconditions crowded and free axes
+ // differently; a floor bounds steps along weakly constrained directions.
+ const curvature=gradient?boxes.map(()=>[.05,.05]):null;
+ const forces=gradient?boxes.map(b=>[(b.anchorX-b.x)*.003,(b.anchorY-b.y)*.003]):null;
+ for(const b of boxes)energy+=.0015*((b.x-b.anchorX)**2+(b.y-b.anchorY)**2);
+ for(let i=0;i2&&py>2){pairs++;overlap=Math.max(overlap,(px-2)*(py-2)/Math.min(a.width*a.height,b.width*b.height));}
+  // Smooth minimum penetration of the actual border boxes. The forces below
+  // are the negative gradient of this same, time-independent contact energy.
+  const depth=Math.min(px,py),wx=Math.exp(-(px-depth)/2),wy=Math.exp(-(py-depth)/2);
+  const contact=Math.max(0,depth-2*Math.log(wx+wy));energy+=.5*contact*contact;
+  if(!gradient||contact===0)continue;
+  const angle=((a.id*137.508+b.id*71.3)%360)*Math.PI/180;
+  const sx=dx===0?(Math.cos(angle)<0?-1:1):Math.sign(dx),sy=dy===0?(Math.sin(angle)<0?-1:1):Math.sign(dy);
+  const fx=sx*contact*wx/(wx+wy),fy=sy*contact*wy/(wx+wy);
+  const hx=(wx/(wx+wy))**2,hy=(wy/(wx+wy))**2;curvature[i][0]+=hx;curvature[j][0]+=hx;curvature[i][1]+=hy;curvature[j][1]+=hy;
+  forces[i][0]-=fx;forces[i][1]-=fy;forces[j][0]+=fx;forces[j][1]+=fy;
+ }
+ return {energy,overlap,pairs,forces,curvature};
+}
+function stepThumbnailForce(boxes,width,height,maxShift,state={}){
+ if(typeof state!=='object')state={};
+ const before=thumbnailEnergy(boxes,true),positions=boxes.map(b=>[b.x,b.y]);
+ let rate=state.rate||1,after=before,speed=0,accepted=false;
+ for(let trial=0;trial<14;trial++){
+  speed=0;
+  boxes.forEach((b,i)=>{
+   const momentum=trial===0?.55:0;
+   let dx=before.forces[i][0]*rate/before.curvature[i][0]+(b.vx||0)*momentum,dy=before.forces[i][1]*rate/before.curvature[i][1]+(b.vy||0)*momentum;
+   const distance=Math.hypot(dx,dy);if(distance>8){dx*=8/distance;dy*=8/distance;}
+   let x=Math.max(b.width/2,Math.min(width-b.width/2,positions[i][0]+dx)),y=Math.max(b.height/2,Math.min(height-b.height/2,positions[i][1]+dy));
+   const offset=Math.hypot(x-b.anchorX,y-b.anchorY);
+   if(offset>maxShift){x=b.anchorX+(x-b.anchorX)*maxShift/offset;y=b.anchorY+(y-b.anchorY)*maxShift/offset;}
+   b.x=x;b.y=y;speed=Math.max(speed,Math.hypot(x-positions[i][0],y-positions[i][1]));
+  });
+  after=thumbnailEnergy(boxes);
+  if(after.energy<=before.energy+1e-9){accepted=true;break;}
+  // Retry without inertia before shortening the gradient step.
+  if(trial>0)rate*=.5;
+ }
+ if(!accepted){boxes.forEach((b,i)=>{b.x=positions[i][0];b.y=positions[i][1];});after=before;speed=0;}
+ boxes.forEach((b,i)=>{b.vx=b.x-positions[i][0];b.vy=b.y-positions[i][1];b.left=b.x-b.width/2;b.top=b.y-b.height/2;});
+ state.rate=Math.min(4,rate*1.1);state.steps=(state.steps||0)+1;
+ const small=speed<.04&&Math.abs(after.energy-before.energy)/Math.max(1,boxes.length)<.002&&Math.abs(after.overlap-before.overlap)<1e-4;
+ state.quiet=small?(state.quiet||0)+1:0;
+ state.metrics={energy:after.energy,overlap:after.overlap,pairs:after.pairs,speed};
+ state.done=state.quiet>=8||state.steps>=480;
+ state.reason=state.quiet>=8?'resting':'work limit';
+ return speed;
+}
+function relaxThumbnails(boxes,width,height,maxShift){
+ const state={};while(!state.done)stepThumbnailForce(boxes,width,height,maxShift,state);
+ return boxes;
+}
+// Cascaded time-domain filters separate solver convergence from visible motion.
+// A speed-limited guide and its continuous follower use CSS pixels and seconds.
+// Convex combinations preserve viewport and anchor-radius bounds; retaining the
+// guide on each box preserves velocity when loading or navigation reheats layout.
+function stepThumbnailPresentation(boxes,targets,seconds){
+ const omega=8,maxSpeed=140;
+ let remaining=Math.min(.05,Math.max(0,seconds));
+ while(remaining>1e-9){
+  const dt=Math.min(1/240,remaining);remaining-=dt;
+  boxes.forEach((b,i)=>{
+   const target=targets[i];b.guideX??=b.x;b.guideY??=b.y;
+   const dx=target.x-b.guideX,dy=target.y-b.guideY,distance=Math.hypot(dx,dy);
+   const fraction=distance?Math.min(1-Math.exp(-omega*dt),maxSpeed*dt/distance):0;
+   const decay=Math.exp(-omega*dt);
+   for(const [key,guide] of [['x','guideX'],['y','guideY']]){
+    const start=b[guide],velocity=(target[key]-start)*fraction/dt;
+    // Exact follower response to a linearly moving guide in this substep.
+    b[key]=start+velocity*dt-velocity/omega+(b[key]-start+velocity/omega)*decay;
+    b[guide]=start+velocity*dt;
+   }
+  });
+ }
+ let speed=0,error=0;
+ boxes.forEach((b,i)=>{
+  b.left=b.x-b.width/2;b.top=b.y-b.height/2;
+  speed=Math.max(speed,omega*Math.hypot((b.guideX??b.x)-b.x,(b.guideY??b.y)-b.y));
+  error=Math.max(error,Math.hypot(targets[i].x-b.x,targets[i].y-b.y),Math.hypot(targets[i].x-(b.guideX??b.x),targets[i].y-(b.guideY??b.y)));
+ });
+ return {speed,error,done:speed<.02&&error<.005};
+}
+let mapThumbMotion=null;
+function stopThumbnailMotion(){
+ if(!mapThumbMotion)return;
+ cancelAnimationFrame(mapThumbMotion.frame);
+ mapThumbMotion.resolve();mapThumbMotion=null;
+}
+function paintThumbnailMotion(){
+ for(const box of mapThumbLayout){
+  box.button.style.left=box.x+'px';box.button.style.top=box.y+'px';
+  if(!box.tether)continue;
+  const line=box.tether.querySelector('line');
+  line.setAttribute('x1',box.anchorX);line.setAttribute('y1',box.anchorY);
+  const dot=box.tether.querySelector('circle');dot.setAttribute('cx',box.anchorX);dot.setAttribute('cy',box.anchorY);
+  line.setAttribute('x2',Math.max(box.left,Math.min(box.left+box.width,box.anchorX)));
+  line.setAttribute('y2',Math.max(box.top,Math.min(box.top+box.height,box.anchorY)));
+  box.tether.style.visibility=Math.hypot(box.x-box.anchorX,box.y-box.anchorY)<.5?'hidden':'visible';
+ }
+}
+let mapThumbRest=null;
+function animateThumbnails(config){
+ // Image arrivals join the existing frame loop; they never reset its clock,
+ // numerical destinations, or presentation velocity.
+ if(mapThumbMotion)return mapThumbMotion.promise;
+ mapThumbRest=null;
+ const motion={frame:0,state:{},last:0,targets:[],byId:new Map()};
+ mapThumbMotion=motion;
+ return motion.promise=new Promise(resolve=>{
+  motion.resolve=resolve;
+  const reduced=matchMedia('(prefers-reduced-motion: reduce)').matches;
+  const finish=()=>{
+   const measured=thumbnailEnergy(mapThumbLayout);
+   mapThumbRest={...motion.state.metrics,energy:measured.energy,overlap:measured.overlap,pairs:measured.pairs,reason:motion.state.reason,steps:motion.state.steps,presentationSpeed:motion.presentation?.speed||0};
+   paintThumbnailMotion();mapThumbMotion=null;resolve();
+  };
+  const syncTargets=time=>{
+   const active=mapThumbLayout.filter(b=>reduced||!b.readyAt||time>=b.readyAt);
+   const changed=active.length!==motion.byId.size||active.some(b=>!motion.byId.has(b.id));
+   motion.targets=active.map(b=>motion.byId.get(b.id)||({...b,vx:0,vy:0}));
+   motion.byId=new Map(motion.targets.map(b=>[b.id,b]));
+   if(changed)motion.state={};
+   return active.length===mapThumbLayout.length;
+  };
+  const advance=()=>stepThumbnailForce(motion.targets,config.width,config.height,2*config.size,motion.state);
+  if(reduced){
+   syncTargets(Infinity);
+   while(!motion.state.done)advance();
+   mapThumbLayout.forEach((b,i)=>{b.x=motion.targets[i].x;b.y=motion.targets[i].y;b.guideX=b.x;b.guideY=b.y;b.left=b.x-b.width/2;b.top=b.y-b.height/2;});
+   finish();return;
+  }
+  const tick=time=>{
+   if(mapThumbMotion!==motion)return;
+   const dt=motion.last?(time-motion.last)/1000:0;motion.last=time;
+   const allReady=syncTargets(time);
+   for(let i=0;i<2&&!motion.state.done;i++)advance();
+   // Each new image waits independently; existing contacts and destinations
+   // continue evolving while it fades in. Match by ID, not admission order.
+   const targets=mapThumbLayout.map(b=>motion.byId.get(b.id)||b);
+   motion.presentation=stepThumbnailPresentation(mapThumbLayout,targets,dt);
+   paintThumbnailMotion();
+   if(allReady&&motion.state.done&&motion.presentation.done){finish();return;}
+   motion.frame=requestAnimationFrame(tick);
+  };
+  motion.frame=requestAnimationFrame(tick);
+ });
+}
+function thumbnailFootprint(size,labels,aspect=1){
+ const ratio=Number.isFinite(aspect)&&aspect>0?aspect:1;
+ const imageWidth=size*Math.min(1,ratio),imageHeight=size*Math.min(1,1/ratio);
+ return {imageWidth,imageHeight,width:imageWidth+(labels?8:0),height:imageHeight+(labels?24:0)};
+}
+function thumbnailLayout(points, priority, {width,height,size,density,overlap,labels=true,push=false,aspects=[],unknownOverlap=overlap}) {
+ const budget=Math.min(200,Math.max(1,Math.floor(width*height*density/1e6)));
+ const chosen=[],used=new Set();
+ for(const id of priority){
+  if(used.has(id))continue;used.add(id);const [x,y]=points[id];
+  const {width:boxWidth,height:boxHeight}=thumbnailFootprint(size,labels,aspects[id]);
+  const box={id,x,y,anchorX:x,anchorY:y,left:x-boxWidth/2,top:y-boxHeight/2,width:boxWidth,height:boxHeight};
+  if(box.left<0||box.top<0||box.left+boxWidth>width||box.top+boxHeight>height)continue;
+  const area=boxWidth*boxHeight;
+  const blocked=chosen.some(other=>{const limit=aspects[id]>0&&aspects[other.id]>0?overlap:unknownOverlap;return thumbnailSharedArea(box,other)/Math.min(area,other.width*other.height)>limit+1e-12;});
+  if(!blocked)chosen.push(box);if(chosen.length>=budget)break;
+ }
+ const boxes=push?relaxThumbnails(chosen,width,height,2*size):chosen;
+ return {boxes,budget,culled:chosen.length-boxes.length};
+}
+function thumbnailTether(box,always=false){
+ if(!always&&Math.hypot(box.x-box.anchorX,box.y-box.anchorY)<.5)return null;
+ const ns='http://www.w3.org/2000/svg',svg=document.createElementNS(ns,'svg');
+ svg.classList.add('map-thumb-tether');svg.setAttribute('aria-hidden','true');svg.style.zIndex='201';
+ const line=document.createElementNS(ns,'line');
+ const edgeX=Math.max(box.left,Math.min(box.left+box.width,box.anchorX)),edgeY=Math.max(box.top,Math.min(box.top+box.height,box.anchorY));
+ for(const [key,value] of Object.entries({x1:box.anchorX,y1:box.anchorY,x2:edgeX,y2:edgeY,stroke:'#435b70','stroke-width':1}))line.setAttribute(key,String(value));
+ const dot=document.createElementNS(ns,'circle');
+ for(const [key,value] of Object.entries({cx:box.anchorX,cy:box.anchorY,r:3,fill:'#ffffff',stroke:'#243b50','stroke-width':1.5}))dot.setAttribute(key,String(value));
+ svg.append(line,dot);return svg;
+}
+function mapThumbnailHit(point){
+ const rect=projectionCanvas.getBoundingClientRect(),x=point[0]*rect.width/1200,y=point[1]*rect.height/projectionHeight;
+ const box=mapThumbLayout.find(b=>x>=b.left&&x<=b.left+b.width&&y>=b.top&&y<=b.top+b.height);
+ return box?.id??-1;
+}
+function showMapPhotoCredit(index){
+ const node=$('map-photo-credit');node.replaceChildren();if(index<0)return;
+ const id=data.names[index],album=photoAlbums.get(id);if(!album?.photos.length)return;
+ const photo=album.photos[(photoChoices.get(id)||0)%album.photos.length];
+ node.append(photoText('span',`${album.display_name} · class ${id} · ${photo.creator} · ${photo.license} · `),photoLink('Image source',photo.source),photoText('span',' · '),photoLink('GBIF record','https://www.gbif.org/occurrence/'+photo.occurrence_id));
+}
+let mapThumbCase=null,mapThumbKey='',mapThumbOffsets=new Map();
+function retireThumbnail(box){
+ // Remove interaction immediately; the short visual exit is independent of
+ // solver membership and cannot delay newly visible thumbnails.
+ box.button.disabled=true;box.button.style.pointerEvents='none';
+ box.button.setAttribute('aria-hidden','true');
+ for(const node of [box.button,box.tether]){
+  if(!node)continue;
+  if(matchMedia('(prefers-reduced-motion: reduce)').matches){node.remove();continue;}
+  const opacity=getComputedStyle(node).opacity;
+  const fade=node.animate([{opacity},{opacity:0}],{duration:100,fill:'forwards'});
+  fade.onfinish=()=>node.remove();
+ }
+}
+function reanchorThumbnails(points,width,height){
+ const kept=[];
+ for(const box of mapThumbLayout){
+  const point=points[box.id];if(!point){retireThumbnail(box);continue;}
+  const dx=box.x-box.anchorX,dy=box.y-box.anchorY;
+  if(box.guideX!==undefined){box.guideX+=point[0]-box.anchorX;box.guideY+=point[1]-box.anchorY;}
+  box.anchorX=point[0];box.anchorY=point[1];box.x=point[0]+dx;box.y=point[1]+dy;
+  box.left=box.x-box.width/2;box.top=box.y-box.height/2;
+  if(box.left<0||box.top<0||box.left+box.width>width||box.top+box.height>height){retireThumbnail(box);continue;}
+  kept.push(box);
+ }
+ mapThumbLayout=kept;paintThumbnailMotion();
+}
+function setThumbnailAnchors(){
+ $('map-thumbnail-layer').classList.toggle('hide-thumb-anchors',!$('map-photo-anchors').checked);
+}
+function scheduleMapThumbnails(){
+ clearTimeout(mapThumbTimer);stopThumbnailMotion();mapThumbController?.abort();++mapThumbGeneration;
+ if(!projectionCanvas.getBoundingClientRect().width)return;
+ const size=Number($('map-photo-size').value),density=Number($('map-photo-density').value),overlap=Number($('map-photo-overlap').value);
+ const key=[size,$('map-photo-labels').checked,$('map-photo-push').checked,$('map-photo-aspect').checked,$('projection-plane').value].join(':');
+ if(gbifEnabled()&&$('map-photos').checked&&mapThumbCase===data&&mapThumbKey===key){
+  const rect=projectionCanvas.getBoundingClientRect();
+  reanchorThumbnails(projectionHits.map(([x,y])=>[x*rect.width/1200,y*rect.height/projectionHeight]),rect.width,rect.height);
+  mapThumbOffsets=new Map(mapThumbLayout.map(b=>[b.id,[b.x-b.anchorX,b.y-b.anchorY]]));
+ }else{mapThumbOffsets.clear();mapThumbLayout=[];$('map-thumbnail-layer').replaceChildren();}
+ mapThumbCase=data;mapThumbKey=key;
+ $('map-size-value').value=size+' px';$('map-density-value').value=density+' / MP';$('map-overlap-value').value=overlap+'%';
+ if(data.metadata.synthetic){$('map-photo-status').textContent='Synthetic case: no GBIF photo requests.';return;}
+ if(!gbifEnabled()||!$('map-photos').checked){$('map-photo-status').textContent='Select GBIF class IDs in settings and enable map thumbnails. Push apart connects displaced images to their fixed prototype points.';return;}
+ $('map-photo-status').textContent='Updating visible thumbnails…';
+ mapThumbTimer=setTimeout(renderMapThumbnails,180);
+}
+async function renderMapThumbnails(){
+ const generation=mapThumbGeneration,caseData=data;
+ mapThumbController=new AbortController();const signal=mapThumbController.signal;
+ const rect=projectionCanvas.getBoundingClientRect(),size=Number($('map-photo-size').value);
+ const config={width:rect.width,height:rect.height,size,density:Number($('map-photo-density').value),overlap:Number($('map-photo-overlap').value)/100,labels:$('map-photo-labels').checked,push:$('map-photo-push').checked,aspect:$('map-photo-aspect').checked};
+ if(config.aspect)config.aspects=data.names.map(id=>{const album=photoAlbums.get(id),photo=album?.photos[(photoChoices.get(id)||0)%(album?.photos.length||1)],image=photo&&mapThumbImages.get(photo.image_path);return image?.naturalWidth/image?.naturalHeight;});
+ const points=projectionHits.map(([x,y])=>[x*rect.width/1200,y*rect.height/projectionHeight]);
+ // Selected class first, then its true neighbours; other classes have stable
+ // mixed row order so dense regions do not win simply by checkpoint ordering.
+ const remaining=data.names.map((_,i)=>i).sort((a,b)=>(Math.imul(a,2654435761)>>>0)-(Math.imul(b,2654435761)>>>0));
+ const priority=[selected,...data.neighbours[selected],...remaining].filter(i=>photoNumber(data.names[i])&&!mapThumbFailures.has(data.names[i]));
+ const {boxes,budget}=thumbnailLayout(points,priority,{...config,push:false,unknownOverlap:config.aspect&&!config.push?0:config.overlap});
+ const ranks=new Map(boxes.map((b,i)=>[b.id,i]));
+ const retained=new Map();
+ for(const box of mapThumbLayout){
+  if(ranks.has(box.id)&&box.photoChoice===(photoChoices.get(data.names[box.id])||0))retained.set(box.id,box);
+  else{retireThumbnail(box);}
+ }
+ mapThumbLayout=[...retained.values()].sort((a,b)=>ranks.get(a.id)-ranks.get(b.id));
+
+ let motionFinished=false;
+ let loaded=0,unavailable=0;
+ const status=()=>{$('map-photo-status').textContent=`${loaded} images loaded · ${motionFinished?mapThumbLayout.length:boxes.length} visible slots · budget ${budget} for this viewport. ${config.push?`Push apart: ${motionFinished?`${mapThumbRest?.reason||'resting'} · ${mapThumbRest?.pairs||0} overlapping pairs retained`:'adjusting rectangle contacts'}. Images move at most ${2*size} px.`:`Maximum pairwise rectangle overlap ${Math.round(config.overlap*100)}%.`} ${unavailable} unavailable. Hover for names and credits; click to inspect. Prototype coordinates and neighbour ranks stay fixed.`;};
+ if(!boxes.length){status();return;}
+ const loadBox=async box=>{
+  if(signal.aborted||generation!==mapThumbGeneration||data!==caseData)return;
+  const id=data.names[box.id];
+  if(retained.has(box.id)){loaded++;const old=retained.get(box.id);old.button.style.zIndex=String(boxes.length-ranks.get(box.id));old.button.style.borderColor=box.id===selected?'#c2344b':data.neighbours[selected].includes(box.id)?'#087e8b':'#7c90a3';return;}
+  try{
+   const album=await referenceAlbum(id,signal);
+   if(signal.aborted||generation!==mapThumbGeneration)return;
+   if(!album.photos.length){mapThumbFailures.add(id);unavailable++;status();return;}
+   const choice=(photoChoices.get(id)||0)%album.photos.length,photo=album.photos[choice];
+   let image=mapThumbImages.get(photo.image_path)?.cloneNode();
+   const button=photoText('button','','map-thumb');button.type='button';button.style.left=box.x+'px';button.style.top=box.y+'px';button.style.width=box.width+'px';button.style.height=box.height+'px';button.style.zIndex=String(boxes.length-boxes.indexOf(box));button.style.borderColor=box.id===selected?'#c2344b':data.neighbours[selected].includes(box.id)?'#087e8b':'#7c90a3';button.setAttribute('aria-label',`Inspect ${album.display_name}, class ${id}`);button.onclick=()=>selectClass(box.id);button.onfocus=()=>showMapPhotoCredit(box.id);
+   button.classList.toggle('image-only',!config.labels);button.classList.toggle('original-aspect',config.aspect);button.title=`${album.display_name} · class ${id}`;
+   if(!image){image=new Image();image.decoding='async';}
+   image.draggable=false;image.alt=album.display_name;image.style.height=size+'px';
+   button.append(image);if(config.labels)button.append(photoText('span',id));
+   const tether=thumbnailTether(box,config.push);
+   const okay=(image.complete&&image.naturalWidth>0)||await loadReferenceImage(image,photo.image_path,signal);
+   if(signal.aborted||generation!==mapThumbGeneration)return;
+   if(okay){
+    const footprint=thumbnailFootprint(size,config.labels,config.aspect?image.naturalWidth/image.naturalHeight:1);
+    box.width=footprint.width;box.height=footprint.height;box.left=box.x-box.width/2;box.top=box.y-box.height/2;
+    button.style.width=box.width+'px';button.style.height=box.height+'px';image.style.height=footprint.imageHeight+'px';
+    box.button=button;box.tether=tether;box.photoChoice=photoChoices.get(id)||0;box.readyAt=performance.now()+80;
+    const offset=config.push?mapThumbOffsets.get(box.id):null;
+    if(offset){box.x=Math.max(box.width/2,Math.min(config.width-box.width/2,box.anchorX+offset[0]));box.y=Math.max(box.height/2,Math.min(config.height-box.height/2,box.anchorY+offset[1]));box.left=box.x-box.width/2;box.top=box.y-box.height/2;button.style.left=box.x+'px';button.style.top=box.y+'px';}
+    if(tether)$('map-thumbnail-layer').append(tether);$('map-thumbnail-layer').append(button);
+    const bounds=button.getBoundingClientRect();box.width=bounds.width;box.height=bounds.height;box.left=box.x-box.width/2;box.top=box.y-box.height/2;
+    mapThumbLayout.push(box);mapThumbLayout.sort((a,b)=>ranks.get(a.id)-ranks.get(b.id));
+    if(config.push)animateThumbnails(config);
+    loaded++;mapThumbImages.delete(photo.image_path);mapThumbImages.set(photo.image_path,image);while(mapThumbImages.size>128)mapThumbImages.delete(mapThumbImages.keys().next().value);}
+   else{button.remove();tether?.remove();mapThumbLayout=mapThumbLayout.filter(b=>b!==box);mapThumbFailures.add(id);unavailable++;}
+   status();if(box.id===selected)showMapPhotoCredit(selected);
+  }catch(error){if(signal.aborted)return;mapThumbFailures.add(id);unavailable++;status();}
+ };
+ // Bound in-flight work; a slow image no longer stalls every later class.
+ let next=0;
+ await Promise.all(Array.from({length:Math.min(4,boxes.length)},async()=>{
+  while(next{mapThumbFailures.clear();scheduleMapThumbnails();};
+$('map-photo-anchors').oninput=setThumbnailAnchors;setThumbnailAnchors();
+new ResizeObserver(scheduleMapThumbnails).observe(projectionCanvas);
+scheduleMapThumbnails();
diff --git a/pyproject.toml b/pyproject.toml
index 38b21a7f..664bd4f6 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -43,6 +43,7 @@ mt_predict = "mini_trainer.predict:run"
 mt_export = "mini_trainer.export:run"
 mt_htrain = "mini_trainer.hierarchical.train:run"
 mt_hpredict = "mini_trainer.hierarchical.predict:run"
+mt_explore = "mini_trainer.visualization.prototype_space.launch:run"
 mt_plot_dendrogram = "mini_trainer.visualization.dendrogram:run"
 mt_create_combinations = "mini_trainer.utils.create_combinations:run"
 mt_create_data_index = "mini_trainer.utils.create_data_index:run"
@@ -77,6 +78,7 @@ dev = [
 ]
 
 [project.optional-dependencies]
+explorer = ["scipy", "scikit-learn>=1.5"]
 recommended = [
   "tensorboard",
   "setuptools<=81.0.0", # Temporary fix to the error "ModuleNotFoundError: No module named 'pkg_resources'" from tensorboard
@@ -93,6 +95,7 @@ all = [
   "mini_trainer[bioclip]",
   "mini_trainer[transformers]",
   "mini_trainer[export]",
+  "mini_trainer[explorer]",
 ]
 quantization = ["torchao>=0.17,<0.18"]
 export = ["onnx>=1.17", "onnxscript>=0.3", "onnxruntime>=1.20"]
diff --git a/tests/export/test_onnx.py b/tests/export/test_onnx.py
index 03554500..bb0c1cce 100644
--- a/tests/export/test_onnx.py
+++ b/tests/export/test_onnx.py
@@ -225,3 +225,31 @@ def test_efficientnet_v2_s_symmetric_normalized_heads(tmp_path, head):
     assert {case["batch_size"] for case in manifest["verification"]["cases"]} == {1, 2, 3, 4}
     assert len(manifest["outputs"]) == (2 if head is HierarchicalClassifier else 1)
     assert manifest["classifiers"][0]["metadata"]["in_features"] == 1280
+
+
+@pytest.mark.parametrize("head", [Classifier, HierarchicalClassifier])
+def test_embedding_export_matches_actual_head(tmp_path, head):
+    import onnxruntime as ort
+
+    from mini_trainer.modeling.context import EmbeddingContext
+
+    model, _ = build_classifier(head, normalized=True)
+    model.eval()
+    inputs = torch.randn(2, 3, 5, 5)
+    captured = []
+    hook = model.fc.register_forward_pre_hook(lambda module, args: captured.append(args[0]))
+    with torch.no_grad():
+        predictions = model(inputs)
+        embedding = model.fc.preclassification(captured[-1])
+    hook.remove()
+    path = export_onnx(model, inputs, tmp_path / "embedding", include_embeddings=True)
+    options = ort.SessionOptions()
+    options.intra_op_num_threads = 1
+    session = ort.InferenceSession(str(path / "model.onnx"), sess_options=options)
+    actual = session.run(None, {"images": inputs.numpy()})
+    expected = predictions if isinstance(predictions, list) else [predictions]
+    for result, value in zip(actual, [*expected, embedding], strict=True):
+        np.testing.assert_allclose(result, value.numpy(), rtol=1e-4, atol=1e-5)
+    assert session.get_outputs()[-1].name == "embedding"
+    assert not EmbeddingContext.active()
+    assert not model.fc._forward_pre_hooks
diff --git a/tests/logging/test_logging_simple.py b/tests/logging/test_logging_simple.py
index c76cd037..fd875a05 100644
--- a/tests/logging/test_logging_simple.py
+++ b/tests/logging/test_logging_simple.py
@@ -23,7 +23,13 @@ def test_dendrogram_export_retains_text_and_closes_all_figures_on_failure(monkey
     monkeypatch.setattr(logger, "confusion_matrix", lambda: {})
     monkeypatch.setattr(core, "main_process_first", nullcontext)
     monkeypatch.setattr(core, "get_rank", lambda: 0)
-    monkeypatch.setattr(core, "plot_class_distance_matrix", lambda model: [])
+
+    def class_matrix(model, *, log_domain, log_range):
+        assert log_domain is True
+        assert log_range[0] < -18 and log_range[1] == 0
+        return []
+
+    monkeypatch.setattr(core, "plot_class_distance_matrix", class_matrix)
     figures = [plt.figure(), plt.figure()]
     figures[0].text(0.5, 0.5, "Species test")
     monkeypatch.setattr(core, "plot_probabilistic_dendrogram", lambda model: [(fig, {}) for fig in figures])
diff --git a/tests/utils/test_browser_bundle.py b/tests/utils/test_browser_bundle.py
new file mode 100644
index 00000000..885838fc
--- /dev/null
+++ b/tests/utils/test_browser_bundle.py
@@ -0,0 +1,50 @@
+"""Publishing a browser bundle must preserve model identity and be atomic."""
+
+import hashlib
+import json
+
+import pytest
+
+from mini_trainer.visualization.prototype_space.browser import package_bundle
+from tests.utils.test_prototype_launcher import checkpoint
+
+
+def test_browser_bundle_checks_identity_and_cleans_failed_publication(tmp_path):
+    weights = tmp_path / "weights.pt"
+    checkpoint(weights)
+    export = tmp_path / "onnx"
+    export.mkdir()
+    (export / "model.onnx").write_bytes(b"fixture graph")
+    manifest = {
+        "output_semantics": "predictions_and_preclassification_embedding",
+        "classifiers": [{"metadata": {"cls2idx": {"alpha": 2, "beta": 0, "gamma": 3, "delta": 1}}}],
+        "source": {"checkpoint_sha256": hashlib.sha256(weights.read_bytes()).hexdigest()},
+        "input": {"dtype": "float32", "shape": ["batch", 3, 384, 384]},
+        "outputs": [{"name": "output_0"}, {"name": "embedding"}],
+        "preprocessing": {"recipe": {"contract": "nearest-square-uint8-bilinear-center-imagenet-v1", "size": 384, "resize": 438}},
+        "artifacts": {"model.onnx": hashlib.sha256(b"fixture graph").hexdigest()},
+    }
+    (export / "manifest.json").write_text(json.dumps(manifest))
+    runtime = tmp_path / "runtime"
+    (runtime / "dist").mkdir(parents=True)
+    (runtime / "package.json").write_text(json.dumps({"version": "1.24.3"}))
+    for name in ["ort.wasm.min.js", "ort-wasm-simd-threaded.mjs", "ort-wasm-simd-threaded.wasm"]:
+        (runtime / "dist" / name).write_bytes(b"fixture runtime")
+    output = tmp_path / "published"
+    with pytest.raises(FileNotFoundError):
+        package_bundle(export, output, runtime, weights)
+    assert not output.exists()
+    assert not list(tmp_path.glob(".browser-bundle-*"))
+    (runtime / "LICENSE.txt").write_text("fixture license")
+    assert package_bundle(export, output, runtime, weights) == output
+    bundle = json.loads((output / "manifest.json").read_text())
+    assert bundle["classes"] == [["beta", "delta", "alpha", "gamma"]]
+    assert (output / "runtime/ort-wasm-simd-threaded.js").exists()
+    assert (output / "prototypes.f32").stat().st_size == 4 * 5 * 4
+    for name, digest in bundle["artifacts"].items():
+        assert hashlib.sha256((output / name).read_bytes()).hexdigest() == digest
+    with pytest.raises(FileExistsError):
+        package_bundle(export, output, runtime, weights)
+    (export / "model.onnx").write_bytes(b"corrupted graph")
+    with pytest.raises(ValueError, match="Invalid export artifact"):
+        package_bundle(export, tmp_path / "corrupt", runtime, weights)
diff --git a/tests/utils/test_browser_insertion.py b/tests/utils/test_browser_insertion.py
new file mode 100644
index 00000000..1a2c18f1
--- /dev/null
+++ b/tests/utils/test_browser_insertion.py
@@ -0,0 +1,33 @@
+"""Numerical checks for client-side fixed-map insertion, without a browser runtime."""
+
+import json
+import shutil
+import subprocess
+from pathlib import Path
+
+import numpy as np
+import pytest
+
+SCRIPT = Path(__file__).parents[2] / "mini_trainer/visualization/prototype_space/insertion.js"
+
+
+@pytest.mark.skipif(shutil.which("node") is None, reason="Node is required for the browser numerical implementation")
+def test_fixed_map_insertion_and_gradient():
+    runner = r"""
+const {insertionAffinities,insertionObjective,insertIntoFixedTSNE}=require(process.argv[1]);
+const points=Array.from({length:100},(_,i)=>[10*Math.cos(i*2*Math.PI/100),10*Math.sin(i*2*Math.PI/100)]);
+const prototypes=Float32Array.from(points.flatMap(([x,y])=>[x/10,y/10,0]));
+const before=JSON.stringify(points),aff=insertionAffinities(points.map(([x,y])=>Math.hypot(x-8,y-2)),10);
+const q=[8.2,2.3],state=insertionObjective(q,points,aff),epsilon=1e-5;
+const finite=q.map((_,j)=>{const a=q.slice(),b=q.slice();a[j]+=epsilon;b[j]-=epsilon;
+return (insertionObjective(a,points,aff).loss-insertionObjective(b,points,aff).loss)/(2*epsilon);});
+const result=insertIntoFixedTSNE([1,.01,0],prototypes,points,{perplexity:5});
+console.log(JSON.stringify({gradient:state.gradient,finite,result,unchanged:before===JSON.stringify(points)}));
+"""
+    output = subprocess.run(["node", "-e", runner, str(SCRIPT)], capture_output=True, text=True, check=True)
+    report = json.loads(output.stdout)
+    np.testing.assert_allclose(report["gradient"], report["finite"], rtol=1e-6, atol=1e-8)
+    assert report["unchanged"]
+    assert report["result"]["kl"] <= report["result"]["initial_kl"]
+    assert report["result"]["retained_fraction"] >= 0.9
+    assert np.isfinite(report["result"]["coordinates"]).all()
diff --git a/tests/utils/test_log_class_similarity.py b/tests/utils/test_log_class_similarity.py
new file mode 100644
index 00000000..826ebbc9
--- /dev/null
+++ b/tests/utils/test_log_class_similarity.py
@@ -0,0 +1,99 @@
+"""Log-domain prototype diagnostics retain representable scores, not rounded CDFs."""
+
+import math
+from types import SimpleNamespace
+
+import numpy as np
+import pytest
+import torch
+
+from mini_trainer.modeling import distance
+from mini_trainer.visualization.plot import (
+    _aggregate_matrix_max,
+    _generate_log_heatmap_rgb_array,
+    _log_colorbar_ticks,
+    plot_heatmap,
+)
+
+
+@pytest.mark.parametrize("device", ["cpu", "cuda"])
+def test_log_tail_matches_erfc_without_cdf_or_autocast_rounding(monkeypatch, device):
+    if device == "cuda" and not torch.cuda.is_available():
+        pytest.skip("CUDA not enabled")
+    dim = 1280
+    targets = torch.tensor([0.0, 5.0, 8.0, 12.0, 20.0, 31.0], device=device)
+    cosine = (targets / math.sqrt(dim - 2)).sin()
+    weights = torch.zeros(len(targets) + 1, dim, device=device)
+    weights[0, 0] = 1
+    weights[1:, 0] = cosine
+    weights[1:, 1] = (1 - cosine.square()).sqrt()
+    weights.requires_grad_()
+    original = weights.detach().clone()
+    monkeypatch.setattr(distance, "classification_module", lambda model: SimpleNamespace(last_layer_weights=weights))
+    z = distance.class_similarity(None, cdf=False)[0]
+
+    def reject_cdf(*args, **kwargs):
+        raise AssertionError("Log-domain diagnostic must not materialize a CDF")
+
+    monkeypatch.setattr(torch.distributions.Normal, "cdf", reject_cdf)
+    with torch.autocast(device_type=device, dtype=torch.bfloat16):
+        actual = distance.class_log_similarity(None, complement=True)[0]
+        log_cdf = distance.class_log_similarity(None)[0]
+    expected = [math.log(0.5 * math.erfc(value / math.sqrt(2))) for value in z[0, 1:].tolist()]
+    np.testing.assert_allclose(actual[0, 1:].cpu(), expected, rtol=2e-6, atol=1e-6)
+    assert actual.dtype == log_cdf.dtype == torch.float32
+    assert actual.device == weights.device
+    assert not actual.requires_grad and weights.grad is None
+    assert torch.isneginf(actual.diag()).all() and (log_cdf.diag() == 0).all()
+    assert torch.isfinite(actual[0, 1:]).all()
+    assert actual[0, 1:].unique().numel() == len(targets)
+    torch.testing.assert_close(actual, actual.T, rtol=0, atol=0)
+    torch.testing.assert_close(weights.detach(), original, rtol=0, atol=0)
+
+
+def test_log_similarity_preserves_multiple_prototype_levels(monkeypatch):
+    weights = [torch.eye(3), torch.eye(4)]
+    monkeypatch.setattr(distance, "classification_module", lambda model: SimpleNamespace(last_layer_weights=weights))
+    results = distance.class_log_similarity(None, complement=True)
+    assert [tuple(result.shape) for result in results] == [(3, 3), (4, 4)]
+    for result in results:
+        off_diagonal = ~torch.eye(len(result), dtype=torch.bool)
+        torch.testing.assert_close(result[off_diagonal], torch.full_like(result[off_diagonal], -math.log(2)))
+
+
+def test_log_heatmap_retains_extreme_tails_and_masks_zero_without_exp(monkeypatch):
+    values = np.array([[-np.inf, -50, -100], [-50, -500, -1000]], dtype=np.float32)
+    before = values.copy()
+    rgb, norm, _, _ = _generate_log_heatmap_rgb_array(values, None, "magma", True)
+    assert rgb.shape == (2, 3, 3)
+    assert len({tuple(rgb[r, c]) for r, c in [(0, 1), (0, 2), (1, 1), (1, 2)]}) == 4
+    assert (rgb[0, 0] == 0).all() and norm is not None
+    np.testing.assert_array_equal(values, before)
+
+    def reject_exp(*args, **kwargs):
+        raise AssertionError("Log rendering must not exponentiate matrix values")
+
+    monkeypatch.setattr(np, "exp", reject_exp)
+    image = plot_heatmap(values, log_input=True)
+    assert image.dtype == np.uint8 and image.shape[2] == 3
+
+
+def test_log_padding_and_ticks_do_not_materialize_tiny_probabilities():
+    values = -np.arange(1, 10, dtype=float).reshape(3, 3)
+    actual = _aggregate_matrix_max(values, (2, 2), pad_value=-np.inf)
+    np.testing.assert_array_equal(actual, [[-1, -3], [-7, -9]])
+    _, labels = _log_colorbar_ticks(-1000 * math.log(10), 0, 3, True)
+    assert labels == ["10^-998%", "10^-498%", "10^2%"]
+
+
+def test_colour_clipping_preserves_scores_and_ignores_extreme_outlier():
+    values = np.array([[-5.0, -10.0, -15.0, -500.0]], dtype=np.float32)
+    before = values.copy()
+    image, norm, low, high = _generate_log_heatmap_rgb_array(values, None, "magma", True, log_range=(-20.0, 0.0))
+    reference, _, _, _ = _generate_log_heatmap_rgb_array(values[:, :3], None, "magma", True, log_range=(-20.0, 0.0))
+    np.testing.assert_array_equal(image[:, :3], reference)
+    np.testing.assert_array_equal(values, before)
+    assert (low, high) == (-20, 0) and norm.clip
+    assert len({tuple(colour) for colour in image[0, :3]}) == 3
+    with pytest.raises(ValueError, match="increasing"):
+        plot_heatmap(values, log_input=True, log_range=(0.0, -10.0))
diff --git a/tests/utils/test_prototype_exploration.py b/tests/utils/test_prototype_exploration.py
new file mode 100644
index 00000000..43a9fee6
--- /dev/null
+++ b/tests/utils/test_prototype_exploration.py
@@ -0,0 +1,108 @@
+"""Offline explorer invariants: checkpoint identity, self-pairs and tied scores."""
+
+import base64
+import math
+
+import numpy as np
+import pytest
+import torch
+from torch import nn
+
+pytest.importorskip("scipy")
+
+from dev.prototype_space.explore import analyze, block_extrema, load_prototypes, stable_scores, synthetic_cases, weight_model
+from mini_trainer.modeling import Classifier, class_distance, class_similarity
+
+
+def test_extraction_preserves_magnitudes_bias_and_checkpoint_class_order(tmp_path):
+    layer = Classifier._normalize_layer(nn.Linear(4, 3), orthogonal_init=False)
+    with torch.no_grad():
+        layer.parametrizations.weight.original0.copy_(torch.tensor([[2.0], [0.5], [3.0]]))
+        layer.bias.copy_(torch.tensor([0.1, -0.2, 0.3]))
+    state = {f"classifier.linear.{key}": value for key, value in layer.state_dict().items()}
+    state["classifier._extra_state"] = {
+        "normalized": True,
+        "classifier_class": "mini_trainer.hierarchical.model:HierarchicalClassifier",
+        "cls2idx": {"0": {"last": 2, "first": 0, "middle": 1}},
+        "labels": {"first": ["first", "parent"]},
+    }
+    path = tmp_path / "checkpoint.pt"
+    torch.save(state, path)
+    before = path.read_bytes()
+    weight, names, groups, meta = load_prototypes(path)
+    torch.testing.assert_close(weight, layer.weight, rtol=0, atol=0)
+    assert names == ["first", "middle", "last"]
+    assert groups[0] == ["first", "parent"]
+    assert meta["bias_range"] == [layer.bias.min().item(), layer.bias.max().item()]
+    assert path.read_bytes() == before
+    state["classifier._extra_state"]["classifier_class"] = "unsupported:MultiPrototypeHead"
+    torch.save(state, path)
+    with pytest.raises(ValueError, match="supports only"):
+        load_prototypes(path)
+
+
+def test_block_summaries_exclude_diagonal_and_partial_padding():
+    matrix = np.array([[0.0, 0.2, 0.8], [0.2, 0.0, 0.6], [0.8, 0.6, 0.0]])
+    low, high, block = block_extrema(matrix, np.array([2, 0, 1]), size=2)
+    assert block == 2
+    np.testing.assert_allclose(low, [[0.8, 0.2], [0.2, np.nan]])
+    np.testing.assert_allclose(high, [[0.8, 0.6], [0.6, np.nan]])
+    assert np.diag(matrix).tolist() == [0.0, 0.0, 0.0]
+
+
+def test_algebraic_cases_keep_baseline_ties_and_original_submatrix_values():
+    weight = synthetic_cases(1280)["Algebraic edge cases"]
+    model = weight_model(weight)
+    distance = class_distance(model)[0].numpy()
+    z = class_similarity(model, cdf=False)[0].numpy()
+    # An actual duplicate and a near duplicate both saturate the CDF, but their
+    # pre-CDF scores remain distinct. They must not be presented as equal vectors.
+    assert distance[0, 1] == distance[0, 5] == 0
+    assert z[0, 1] > z[0, 5] > z[0, 3] > z[0, 2]
+    names = [f"class-{i}" for i in range(len(weight))]
+    result = analyze(weight, names, [[name] for name in names], neighbours=3)
+    assert result["neighbours"][0][:2] == [1, 5]
+    for i, ranks in enumerate(result["profile_neighbours"]):
+        row = z[i].copy()
+        row[i] = -np.inf
+        expected = np.argsort(-row, kind="stable")[np.asarray(result["profile_ranks"]) - 1]
+        assert ranks == expected.tolist()
+    assert all(i not in row for i, row in enumerate(result["neighbours"]))
+    assert result["zero_count"][0] == 2
+    assert result["stats"]["pair_sample_count"] == 15
+    for field, original in [("local_distance", distance), ("local_z", z)]:
+        actual = np.frombuffer(base64.b64decode(result[field]), dtype=" 0).all()
+    # Beyond the float64 linear-probability range, log-tail still has meaning.
+    extreme_d, extreme_logtail = stable_scores(torch.tensor([56.0]))
+    assert extreme_d.item() == 0 and torch.isfinite(extreme_logtail).all()
+
+
+def test_optional_gbif_snapshot_is_embedded_as_data(tmp_path):
+    import json
+
+    from mini_trainer.visualization.prototype_space.explore import render_report
+
+    render_report(tmp_path, "{}")
+    assert '' in (tmp_path / "explorer.html").read_text()
+    snapshot = {"schema": "mini-trainer-gbif-v1", "taxa": {"1": {"key": 1, "canonicalName": ""}}}
+    (tmp_path / "gbif-snapshot.json").write_text(json.dumps(snapshot))
+    render_report(tmp_path, "{}")
+    html = (tmp_path / "explorer.html").read_text()
+    assert "