Playable brain: deterministic model, real spiking network, per-sentence math, calibration (which caught an inverted trust score) - #92
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces 'Mind-Hack Autopsy' (experiment 013), a new client-side interactive lab allowing users to simulate brain reactions to various content types locally. It updates the arcade layout, landing page, sharing dialogs, and E2E tests to support thirteen experiments, and adds a new evidence export feature for the attractor playground. The review feedback suggests several robustness and compatibility improvements, including defensive checks for clipboard API availability, safer destructuring of parameters in the evidence builder, and appending the temporary download link to the document body to ensure cross-browser compatibility.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
…ce export
Content Reaction Lab (arcade experiment 013):
- New src/features/gaugegap/ContentReactionLab.jsx: paste an ad / tweet /
landing page / email, run a fully client-side scan (analysisEngine +
layerRouter, no /api round-trip), watch the live 3D brain react, and read
four headline scores — Attention, Trust, Emotional Charge, Manipulation
Risk — with delta badges against the previous run.
- One-click rewrites (build trust / add real urgency / reduce manipulation)
scored in-browser with before→after deltas; new additive "urgency" goal in
improve/rewrite.js. Share score card reuses ShareDialog.
- Registered as lab 013 in ExperimentArcade with XP/daily-mission integration;
the landing's Mind-Hack Autopsy card now opens the lab in place instead of
bouncing to /app ("Open full analysis" remains the escape hatch).
- New src/lib/headlineScores.js maps getBusinessMetrics to the four headline
scores; score card, share text and share preview now show the same four
rows so the playground and shared cards never disagree (this also changes
/app's share card rows from Hook/Viral Pull to the headline set).
Truth layer:
- New src/features/gaugegap/evidence.js: "Export evidence" button on the
Butterfly Effect (Lorenz) lab downloads a content-hashed JSON proofpack
(schema gaugegap.browser_attractor_proofpack.v1) with live parameters,
solver settings, scores, share URL and an explicit claim boundary.
- Deep Foundry link from the landing's research section to the published
gaugegap-foundry Foundry Experience playground.
Tests: new tinyVitest suites for headlineScores and evidence; urgency-goal
coverage in rewrite.test.js; scoreCard test updated to the new rows. Stale
e2e landing specs (pre-arcade selectors) rewritten against the arcade:
content lab local-scan flow asserts no /api/analyze request; 3D-mount and
Reconstruct-nav specs target the current DOM. Full suite: 79 unit tests,
13 e2e passed locally.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
…load link Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
3d48582 to
138dc92
Compare
A shared link (?lab=content&state=<text>) now prefills the lab and runs the scan automatically, matching the challenge-link loop of the other arcade labs (same `state` param, so lab-switch URL cleanup applies unchanged). "Share challenge" / "Copy link" actions reuse the DeepLabChrome helpers. Shared text is capped and validated before auto-running. Tests: unit coverage for the URL round-trip, foreign-lab guarding, size cap and stale-run-param cleanup; new e2e spec proves a shared link auto-runs to scored tiles. The auto-run effect keys on the parsed share payload only, so StrictMode's dev double-invoke re-arms the cleared scan timer instead of deadlocking, and keystrokes never re-apply the shared text. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
The 3D brain ran on Math.random(), so the same content produced a different brain every time — scores could not be reproduced, shared or verified. Its ~20 model constants were hardcoded literals inside the step function, and it had no test coverage at all. - New src/features/brain3d/brainModel.js: the 7-region rate dynamics lifted out of the React hook into a pure, seeded, parameterized model. Adds interventions (lesion a region, cut a pathway, inject current) and exposes the excitatory/inhibitory split and per-pathway STDP delta that the old loop computed and discarded. DEFAULT_PARAMS reproduces the original behaviour exactly, so the live visual is unchanged. - runBrainTrial() runs the model headlessly: same seed and inputs always produce an identical trace, which is what makes brain readouts scoreable and verifiable. - New brainMetrics.js: firing rates in Hz, PFC/AMY control ratio, hijack index, gain around the THL->CTX->AMY->BG->|THL control loop, gate integrity, per-region E/I balance, mean pairwise spike correlation, settling time, plasticity, weight drift and net STDP flux — each with a descriptor stating its direction, plus an explicit claim boundary. - New src/lib/rng.js: shared FNV-1a + mulberry32 seeding, matching the implementation solitonLayer.js already used privately. - useBrainSimulation.js becomes a thin wrapper over the model and regains pause/step/reset/burst (dropped during the r3f port) plus stimulate, lesion, cut and speed control. - 23 new tests covering determinism, bounds, interventions and metrics. One pins a real quirk found while testing: activity is capped at 1.0, so spike probability maxes at 0.836 and the spikeMax 0.85 ceiling is unreachable. - tinyVitest gains the toBeLessThan matcher it was missing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
"Manipulation Risk 72" said nothing about why, and the single `confidence` number was asserted rather than estimated. This adds attribution and a real uncertainty estimate by re-running the scorer on modified inputs — genuinely new information, not a restatement of the same features. - New src/lib/ablation.js: leave-one-sentence-out scoring gives each sentence its exact contribution to each headline score. The same replicates are then read as a jackknife to produce a standard error and range per score, which is a standard estimator rather than an invented confidence. Seeded sentence shuffles measure whether a verdict depends on what was said or merely on the running order. - topDrivers() ranks the sentences behind one score with each one's share. - The lab now renders a "Where this score comes from" panel: ranked sentence bars, a tab per headline score, and ± on each tile. On mixed content the attribution correctly assigns the pressure sentence 100% of Manipulation Risk while splitting Trust across both sentences. - Runs inline with the scan: measured at 21-27 ms including the 18-sentence ceiling, so it needs no button gate. - 12 unit tests plus an e2e spec covering attribution and tab switching. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
The ten gallery classics already carried hand-labelled expectations that nothing ever checked. This turns them into a real evaluation, and the first run found a significant defect. - New calibrationCorpus.js: 18 labelled archetypes (the 10 classics plus 8 new originals) scored on four ordinal dimensions. Kept separate from CLASSIC_PRESETS so the evaluation set can grow without changing the UI gallery, and so re-ordering the gallery cannot alter a published figure. - New calibration.js: labels are ordinal, so agreement is reported as Spearman rank correlation plus the count of labelled pairs the engine orders backwards — a falsifiable claim, rather than asserted precision. Measured baseline: 66% of 442 labelled comparisons ranked correctly, mean Spearman 0.214. Urgency is decent (0.64); manipulation risk (0.39) and viral pull (0.33) are weak. FINDING — trust is ANTI-correlated with its labels (Spearman -0.505, 65% of pairs inverted). The engine ranks outrage bait as more trustworthy than a sincere apology. Cause: `trust` rewards trust *vocabulary*, not evidence. TRUST_TERMS includes the bare connective "because", so "share this because once it's gone" earns +13, while "on Tuesday our update broke checkout for six hours, here is what we refunded" earns nothing — none of its concrete, checkable words are in the bank. Thresholds in the test pin MEASURED behaviour, not aspiration, so scoring changes cannot quietly regress. The trust defect is asserted explicitly rather than hidden, and the assertion flips once the signal is fixed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
Trust was anti-correlated with its labels (Spearman -0.505): the engine
ranked outrage bait as more trustworthy than a sincere apology, because
`trust` counted trust *vocabulary* rather than evidence.
Each change was measured against the calibration harness rather than
guessed, and one hypothesis was rejected by the data:
- Removing the connective "because" from TRUST_TERMS made trust WORSE
(-0.505 -> -0.611). It turns out "closes on 30 November because the cohort
starts in December" is a genuine evidence marker. Reverted.
- The real gap was that concrete detail earned nothing. Added a specificity
signal (numerals, dates, durations) and a stated-limitation signal
("not yet", "uncertain", "approximately"), since admitting limits is a
credibility cue. Trust: -0.505 -> +0.560.
- Weighted specifics above evidence-vocabulary, on the principle that
checkable detail is stronger than claiming to be transparent: -> +0.628.
- Dropped trust from viralScore: careful writing is not more viral, and
including it was dragging viral pull down. Viral: 0.333 -> 0.366.
Measured across 442 labelled comparisons:
before after
trust -0.505 +0.628
risk 0.387 0.629
urgency 0.641 0.641
viral pull 0.333 0.366
accuracy 65.8% 80.1%
mean rho 0.214 0.566
Every dimension now ranks in the right direction. Test thresholds move to
the new measured values, and a remaining limitation is pinned rather than
hidden: "verify within 24 hours" still reads as concrete detail, so phishing
scores higher on trust than it should. Discounting numerals adjacent to
urgency terms is the next step.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
Despite the name, BrainSNN was a rate model: 7 scalar units, no membrane
potential, no threshold, no refractory period. This adds the real thing.
- src/lib/snn/lifNetwork.js: sparse random network of leaky integrate-and-fire
neurons in the Brunel (2000) formulation — Dale's law (80/20 excitatory/
inhibitory), fixed-amplitude delta synapses, axonal delays, Poisson external
drive, exact exponential leak. Typed arrays plus CSR adjacency; ~150 ms for
800 neurons over 250 ms of biological time, and fully seeded.
- src/lib/snn/snnMetrics.js: the measurements that characterise a network
state — firing rate, CV of ISI, Fano factor, population synchrony, and a DFT
of the population rate. The last one matters: gamma-band power becomes a
MEASURED property rather than a closed-form assertion.
- brunelValidation.test.js checks published consequences of the model, so the
simulation is falsifiable rather than merely plausible:
* silent below the threshold drive nu_thr, firing above it
* rate monotonically increasing in external drive
* rate monotonically decreasing in inhibition strength g
* irregularity monotonically increasing in g — the central Brunel result
* near clock-regular firing (CV < 0.15) when excitation dominates
* rate never exceeding the refractory ceiling
Found and fixed a delay bug while validating: the ring buffer wrote into the
same slot it read that step, so a spike reached later-indexed neurons with
zero delay instead of 1.5 ms. With correct delays the balanced network settles
at 42 Hz instead of 69 Hz, and the sweeps become cleanly monotonic.
Known limitation, asserted rather than overclaimed: the asynchronous-irregular
regime needs N ~ 10^4-10^5 at epsilon ~ 0.1 for presynaptic independence. At
browser-affordable sizes the network reaches high irregularity but keeps more
population synchrony than true AI, so the tests pin the monotonic trends
rather than claiming all four published regimes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
All twelve existing labs print a mission string that nothing verifies. This one has a real win condition, a real fail state and a finite budget, all evaluated against the deterministic model from brainModel.js. - brainGame.js: mission rules, a pressure profile that ramps threat drive up while suppressing judgment, five interventions (lesion the amygdala, cut the salience or action route, stimulate PFC or cerebellum), and a three-axis score in the arcade's house style where control, stability and budget efficiency pull against each other. - BrainGameLab.jsx: arcade experiment 014. A 2D circuit view of the seven regions and ten pathways, live activity and spikes, cut pathways drawn dashed, lesioned regions crossed out. Sandbox / Mission / Challenge modes, pause, single-step, reset, and a shareable seed. Thresholds were measured, not guessed. Sweeping the hijack limit against the model showed doing nothing breaches for ~179 consecutive ticks, silencing threat or boosting judgment holds the line, and cutting the salience route helps but is not sufficient on its own — so the limit is set at 60 to keep that gradient. A plausible-looking intervention that still loses is what makes the mission worth replaying. Two bugs found while testing: - The loss condition read the breach streak on the final tick, so a run that failed mid-way but recovered was scored as a win. It now latches on the worst streak seen. - Control ratio diverged when a lesioned amygdala drove the denominator to zero (21.73 on screen). Capped at 10, past which the ratio says nothing extra. 17 unit tests plus an e2e that confirms the mission is genuinely losable and that intervening keeps a run alive past the point it previously failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
Two pieces of the evidence layer. Krippendorff's alpha (src/lib/agreement.js) — a labelled corpus without an agreement statistic says nothing about whether the labels are reproducible by another person. Full implementation with the coincidence-matrix algorithm, supporting any number of annotators, missing ratings, and nominal / ordinal / interval distance. Ordinal matters here: "low vs extreme" should count as a bigger disagreement than "low vs moderate". Reported with the conventional reliability reading (>=0.8 reliable, >=0.667 tentative). Verifiable run proofs (src/features/gaugegap/runProof.js) — because the model is deterministic, a Defend the Brain run is fully described by its seed, mode and the tick at which each intervention was applied. So a score is checkable by RECOMPUTATION: replay the log and see whether the claimed result falls out. Inflating a score means finding a log that actually replays to it, not editing a number. The lab now records intervention ticks and exports a hashed proof. Privacy: a proof carries a seed and a list of tick/id pairs. No text, no personal data — so the lab's "nothing leaves your browser" promise survives even if proofs are later submitted somewhere. Note on scope: durable cross-user leaderboards need storage this deployment does not have. Supabase is wired for magic-link auth only — no tables, no persistence, no rate limiting, and the server is stateless. The verification logic that makes a leaderboard trustworthy is what ships here; the storage behind it is a separate, infrastructural piece of work. Fixed two bugs found by the tests: evaluateRun derived elapsed ticks from the rolling frame window (so a completed replay read as still running), and the proof hash was taken over an object without the hash key while verification spread it back as undefined, which canonical JSON renders as null. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
Completes the evidence path from labels to a defensible number. - src/lib/evalMetrics.js: AUC (ranking), Brier (probabilistic accuracy) and Expected Calibration Error with a reliability table, plus precision/recall/F1 at the best operating point. ECE is the one that matters for honesty: users read these scores as percentages, and a scorer can rank items perfectly while its numbers mean nothing as probabilities. One test pins exactly that case. - scripts/eval-corpus.mjs: evaluate the engine against any externally labelled JSONL corpus. Public datasets are NOT vendored — they carry their own licences — so datasets/ is gitignored and the script points at a local copy. - docs/ANNOTATION_RUBRIC.md: dimensions, ordinal levels, worked anchors, and the rule that a corpus release without a Krippendorff alpha does not get to claim reliability. Maps the published persuasion-technique taxonomies (SemEval-2023 Task 3, SemEval-2020 Task 11 PTC, Webis clickbait) onto our dimensions, and records that only one of the four tactics the firewall detects today maps cleanly to a published class — so those corpora should drive building a real detector, not just validate the current one. Smoke run on a six-item set: AUC 0.778, ECE 0.290 — ranks reasonably, but the scores do not yet behave like percentages. Worth knowing and worth stating. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
Summary
Turns the arcade's brain from decoration into an instrument, and turns its numbers from asserted into calculated. Along the way the new rigor caught a significant scoring defect and two simulation bugs.
201 unit tests · 19 Playwright e2e · lint and build green.
The headline finding
Calibrating against a labelled corpus revealed trust was ranked backwards (Spearman −0.505): the engine scored outrage bait as more trustworthy than a sincere apology.
trustcounted trust vocabulary, not evidence — "share this because once it's gone" earned +13 for the bare connective, while "on Tuesday our update broke checkout for six hours, here is what we refunded" earned nothing.Fixing it was evidence-driven, and one hypothesis was rejected by the data: removing "because" made things worse (−0.611), because it's sometimes a genuine evidence marker. Reverted; added specificity and stated-limitation signals instead.
What shipped
Deterministic brain — the sim ran on
Math.random(), so the same text produced a different brain every time and no score was reproducible. Now a pure, seeded, parameterized model with interventions (lesion / cut / stimulate) and headless trials. New readouts: firing rates in Hz, PFC/AMY control ratio, hijack index, control-loop gain, E/I balance, spike correlation, settling time, STDP flux. It previously had no tests at all.A real spiking network — despite the name, BrainSNN was a rate model. Added a proper LIF network in the Brunel (2000) formulation (Dale's law, delta synapses, axonal delays, Poisson drive) with standard measurements — CV of ISI, Fano factor, population spectrum. Validated against published regime behaviour: silent below the threshold drive, rate monotonic in drive, rate falling and irregularity rising with inhibition, near clock-regular firing when excitation dominates. Gamma-band power becomes measured rather than asserted.
Per-sentence math — leave-one-sentence-out attribution, with the same replicates read as a jackknife for genuine error bars replacing the asserted
confidence. Runs inline (~27 ms).Defend the Brain — the arcade's first machine-checked mission. All twelve existing labs print a mission nothing verifies; this one has a real win condition, fail state and budget. Thresholds were measured: doing nothing breaches for ~179 ticks, silencing threat or boosting judgment holds, and cutting the salience route helps but isn't enough — a real difficulty gradient.
Evidence layer — Krippendorff's alpha for inter-annotator agreement; AUC/Brier/ECE metrics; an external-corpus eval script; an annotation rubric; and verifiable run proofs where a score is checked by recomputation — inflating it requires finding a log that replays to it, not editing a number.
Bugs found by the new rigor
Honesty notes
Not done, and why
Durable cross-user leaderboard storage: Supabase is wired for magic-link auth only — no tables, no persistence, no rate limiting. The verification logic that would make such a leaderboard trustworthy ships here; the storage is separate infrastructure. Public corpora are not vendored (licensing) — loader, rubric and metrics ship, data is fetched locally.
🤖 Generated with Claude Code
https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD