Skip to content

Playable brain: deterministic model, real spiking network, per-sentence math, calibration (which caught an inverted trust score) - #92

Draft
slavazeph-coder wants to merge 11 commits into
mainfrom
claude/simulator-playground-strategy-6dl25j
Draft

Playable brain: deterministic model, real spiking network, per-sentence math, calibration (which caught an inverted trust score)#92
slavazeph-coder wants to merge 11 commits into
mainfrom
claude/simulator-playground-strategy-6dl25j

Conversation

@slavazeph-coder

@slavazeph-coder slavazeph-coder commented Jul 12, 2026

Copy link
Copy Markdown
Owner

Summary

Turns the arcade's brain from decoration into an instrument, and turns its numbers from asserted into calculated. Along the way the new rigor caught a significant scoring defect and two simulation bugs.

201 unit tests · 19 Playwright e2e · lint and build green.

The headline finding

Calibrating against a labelled corpus revealed trust was ranked backwards (Spearman −0.505): the engine scored outrage bait as more trustworthy than a sincere apology. trust counted trust vocabulary, not evidence — "share this because once it's gone" earned +13 for the bare connective, while "on Tuesday our update broke checkout for six hours, here is what we refunded" earned nothing.

Fixing it was evidence-driven, and one hypothesis was rejected by the data: removing "because" made things worse (−0.611), because it's sometimes a genuine evidence marker. Reverted; added specificity and stated-limitation signals instead.

dimension before after
trust −0.505 +0.628
manipulation risk 0.387 0.629
viral pull 0.333 0.366
pair accuracy 65.8% 80.1%

What shipped

Deterministic brain — the sim ran on Math.random(), so the same text produced a different brain every time and no score was reproducible. Now a pure, seeded, parameterized model with interventions (lesion / cut / stimulate) and headless trials. New readouts: firing rates in Hz, PFC/AMY control ratio, hijack index, control-loop gain, E/I balance, spike correlation, settling time, STDP flux. It previously had no tests at all.

A real spiking network — despite the name, BrainSNN was a rate model. Added a proper LIF network in the Brunel (2000) formulation (Dale's law, delta synapses, axonal delays, Poisson drive) with standard measurements — CV of ISI, Fano factor, population spectrum. Validated against published regime behaviour: silent below the threshold drive, rate monotonic in drive, rate falling and irregularity rising with inhibition, near clock-regular firing when excitation dominates. Gamma-band power becomes measured rather than asserted.

Per-sentence math — leave-one-sentence-out attribution, with the same replicates read as a jackknife for genuine error bars replacing the asserted confidence. Runs inline (~27 ms).

Defend the Brain — the arcade's first machine-checked mission. All twelve existing labs print a mission nothing verifies; this one has a real win condition, fail state and budget. Thresholds were measured: doing nothing breaches for ~179 ticks, silencing threat or boosting judgment holds, and cutting the salience route helps but isn't enough — a real difficulty gradient.

Evidence layer — Krippendorff's alpha for inter-annotator agreement; AUC/Brier/ECE metrics; an external-corpus eval script; an annotation rubric; and verifiable run proofs where a score is checked by recomputation — inflating it requires finding a log that replays to it, not editing a number.

Bugs found by the new rigor

  1. Zero-delay spike transmission — the LIF ring buffer wrote into the slot it was reading, so spikes reached later-indexed neurons instantly instead of after 1.5 ms. With correct delays the balanced network settles at 42 Hz instead of 69 Hz and the sweeps become cleanly monotonic.
  2. Mission loss didn't latch — a run that failed mid-way but recovered scored as a win.
  3. Divergent control ratio — a lesioned amygdala drove the denominator to ~0 (21.73 on screen). Capped.

Honesty notes

  • The brain is driven by the lexical scores, so its dynamics add structure but no new information about the text. Every readout carries a claim boundary: simulated dynamics of a 7-region model, not a measurement of any human brain.
  • Two limitations are pinned by tests rather than hidden: "verify within 24 hours" still reads as concrete detail (phishing over-scores on trust), and spike probability can't reach its own cap because activity is capped at 1.0.
  • Brunel's asynchronous-irregular regime needs N ~ 10⁴–10⁵ for presynaptic independence, so the tests assert monotonic trends rather than claiming all four published regimes.
  • A six-item corpus smoke run gives AUC 0.778 but ECE 0.290 — the scores rank reasonably but don't yet behave like percentages.

Not done, and why

Durable cross-user leaderboard storage: Supabase is wired for magic-link auth only — no tables, no persistence, no rate limiting. The verification logic that would make such a leaderboard trustworthy ships here; the storage is separate infrastructure. Public corpora are not vendored (licensing) — loader, rubric and metrics ship, data is fetched locally.

🤖 Generated with Claude Code

https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces 'Mind-Hack Autopsy' (experiment 013), a new client-side interactive lab allowing users to simulate brain reactions to various content types locally. It updates the arcade layout, landing page, sharing dialogs, and E2E tests to support thirteen experiments, and adds a new evidence export feature for the attractor playground. The review feedback suggests several robustness and compatibility improvements, including defensive checks for clipboard API availability, safer destructuring of parameters in the evidence builder, and appending the temporary download link to the document body to ensure cross-browser compatibility.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread brainsnn-r3f-app/src/features/gaugegap/ContentReactionLab.jsx
Comment thread brainsnn-r3f-app/src/features/gaugegap/evidence.js Outdated
Comment thread brainsnn-r3f-app/src/features/gaugegap/evidence.js
slavazeph-coder and others added 2 commits July 12, 2026 20:56
…ce export

Content Reaction Lab (arcade experiment 013):
- New src/features/gaugegap/ContentReactionLab.jsx: paste an ad / tweet /
  landing page / email, run a fully client-side scan (analysisEngine +
  layerRouter, no /api round-trip), watch the live 3D brain react, and read
  four headline scores — Attention, Trust, Emotional Charge, Manipulation
  Risk — with delta badges against the previous run.
- One-click rewrites (build trust / add real urgency / reduce manipulation)
  scored in-browser with before→after deltas; new additive "urgency" goal in
  improve/rewrite.js. Share score card reuses ShareDialog.
- Registered as lab 013 in ExperimentArcade with XP/daily-mission integration;
  the landing's Mind-Hack Autopsy card now opens the lab in place instead of
  bouncing to /app ("Open full analysis" remains the escape hatch).
- New src/lib/headlineScores.js maps getBusinessMetrics to the four headline
  scores; score card, share text and share preview now show the same four
  rows so the playground and shared cards never disagree (this also changes
  /app's share card rows from Hook/Viral Pull to the headline set).

Truth layer:
- New src/features/gaugegap/evidence.js: "Export evidence" button on the
  Butterfly Effect (Lorenz) lab downloads a content-hashed JSON proofpack
  (schema gaugegap.browser_attractor_proofpack.v1) with live parameters,
  solver settings, scores, share URL and an explicit claim boundary.
- Deep Foundry link from the landing's research section to the published
  gaugegap-foundry Foundry Experience playground.

Tests: new tinyVitest suites for headlineScores and evidence; urgency-goal
coverage in rewrite.test.js; scoreCard test updated to the new rows. Stale
e2e landing specs (pre-arcade selectors) rewritten against the arcade:
content lab local-scan flow asserts no /api/analyze request; 3D-mount and
Reconstruct-nav specs target the current DOM. Full suite: 79 unit tests,
13 e2e passed locally.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
@slavazeph-coder
slavazeph-coder force-pushed the claude/simulator-playground-strategy-6dl25j branch from 3d48582 to 138dc92 Compare July 12, 2026 20:59
slavazeph-coder and others added 5 commits July 13, 2026 02:43
A shared link (?lab=content&state=<text>) now prefills the lab and runs the
scan automatically, matching the challenge-link loop of the other arcade
labs (same `state` param, so lab-switch URL cleanup applies unchanged).
"Share challenge" / "Copy link" actions reuse the DeepLabChrome helpers.
Shared text is capped and validated before auto-running.

Tests: unit coverage for the URL round-trip, foreign-lab guarding, size cap
and stale-run-param cleanup; new e2e spec proves a shared link auto-runs to
scored tiles. The auto-run effect keys on the parsed share payload only, so
StrictMode's dev double-invoke re-arms the cleared scan timer instead of
deadlocking, and keystrokes never re-apply the shared text.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
The 3D brain ran on Math.random(), so the same content produced a different
brain every time — scores could not be reproduced, shared or verified. Its
~20 model constants were hardcoded literals inside the step function, and it
had no test coverage at all.

- New src/features/brain3d/brainModel.js: the 7-region rate dynamics lifted
  out of the React hook into a pure, seeded, parameterized model. Adds
  interventions (lesion a region, cut a pathway, inject current) and exposes
  the excitatory/inhibitory split and per-pathway STDP delta that the old
  loop computed and discarded. DEFAULT_PARAMS reproduces the original
  behaviour exactly, so the live visual is unchanged.
- runBrainTrial() runs the model headlessly: same seed and inputs always
  produce an identical trace, which is what makes brain readouts scoreable
  and verifiable.
- New brainMetrics.js: firing rates in Hz, PFC/AMY control ratio, hijack
  index, gain around the THL->CTX->AMY->BG->|THL control loop, gate
  integrity, per-region E/I balance, mean pairwise spike correlation,
  settling time, plasticity, weight drift and net STDP flux — each with a
  descriptor stating its direction, plus an explicit claim boundary.
- New src/lib/rng.js: shared FNV-1a + mulberry32 seeding, matching the
  implementation solitonLayer.js already used privately.
- useBrainSimulation.js becomes a thin wrapper over the model and regains
  pause/step/reset/burst (dropped during the r3f port) plus stimulate,
  lesion, cut and speed control.
- 23 new tests covering determinism, bounds, interventions and metrics. One
  pins a real quirk found while testing: activity is capped at 1.0, so spike
  probability maxes at 0.836 and the spikeMax 0.85 ceiling is unreachable.
- tinyVitest gains the toBeLessThan matcher it was missing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
"Manipulation Risk 72" said nothing about why, and the single `confidence`
number was asserted rather than estimated. This adds attribution and a real
uncertainty estimate by re-running the scorer on modified inputs — genuinely
new information, not a restatement of the same features.

- New src/lib/ablation.js: leave-one-sentence-out scoring gives each sentence
  its exact contribution to each headline score. The same replicates are then
  read as a jackknife to produce a standard error and range per score, which
  is a standard estimator rather than an invented confidence. Seeded sentence
  shuffles measure whether a verdict depends on what was said or merely on
  the running order.
- topDrivers() ranks the sentences behind one score with each one's share.
- The lab now renders a "Where this score comes from" panel: ranked sentence
  bars, a tab per headline score, and ± on each tile. On mixed content the
  attribution correctly assigns the pressure sentence 100% of Manipulation
  Risk while splitting Trust across both sentences.
- Runs inline with the scan: measured at 21-27 ms including the 18-sentence
  ceiling, so it needs no button gate.
- 12 unit tests plus an e2e spec covering attribution and tab switching.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
The ten gallery classics already carried hand-labelled expectations that
nothing ever checked. This turns them into a real evaluation, and the first
run found a significant defect.

- New calibrationCorpus.js: 18 labelled archetypes (the 10 classics plus 8
  new originals) scored on four ordinal dimensions. Kept separate from
  CLASSIC_PRESETS so the evaluation set can grow without changing the UI
  gallery, and so re-ordering the gallery cannot alter a published figure.
- New calibration.js: labels are ordinal, so agreement is reported as
  Spearman rank correlation plus the count of labelled pairs the engine
  orders backwards — a falsifiable claim, rather than asserted precision.

Measured baseline: 66% of 442 labelled comparisons ranked correctly,
mean Spearman 0.214. Urgency is decent (0.64); manipulation risk (0.39) and
viral pull (0.33) are weak.

FINDING — trust is ANTI-correlated with its labels (Spearman -0.505, 65% of
pairs inverted). The engine ranks outrage bait as more trustworthy than a
sincere apology. Cause: `trust` rewards trust *vocabulary*, not evidence.
TRUST_TERMS includes the bare connective "because", so "share this because
once it's gone" earns +13, while "on Tuesday our update broke checkout for
six hours, here is what we refunded" earns nothing — none of its concrete,
checkable words are in the bank.

Thresholds in the test pin MEASURED behaviour, not aspiration, so scoring
changes cannot quietly regress. The trust defect is asserted explicitly
rather than hidden, and the assertion flips once the signal is fixed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
Trust was anti-correlated with its labels (Spearman -0.505): the engine
ranked outrage bait as more trustworthy than a sincere apology, because
`trust` counted trust *vocabulary* rather than evidence.

Each change was measured against the calibration harness rather than
guessed, and one hypothesis was rejected by the data:

- Removing the connective "because" from TRUST_TERMS made trust WORSE
  (-0.505 -> -0.611). It turns out "closes on 30 November because the cohort
  starts in December" is a genuine evidence marker. Reverted.
- The real gap was that concrete detail earned nothing. Added a specificity
  signal (numerals, dates, durations) and a stated-limitation signal
  ("not yet", "uncertain", "approximately"), since admitting limits is a
  credibility cue. Trust: -0.505 -> +0.560.
- Weighted specifics above evidence-vocabulary, on the principle that
  checkable detail is stronger than claiming to be transparent: -> +0.628.
- Dropped trust from viralScore: careful writing is not more viral, and
  including it was dragging viral pull down. Viral: 0.333 -> 0.366.

Measured across 442 labelled comparisons:

              before   after
  trust       -0.505  +0.628
  risk         0.387   0.629
  urgency      0.641   0.641
  viral pull   0.333   0.366
  accuracy     65.8%   80.1%
  mean rho     0.214   0.566

Every dimension now ranks in the right direction. Test thresholds move to
the new measured values, and a remaining limitation is pinned rather than
hidden: "verify within 24 hours" still reads as concrete detail, so phishing
scores higher on trust than it should. Discounting numerals adjacent to
urgency terms is the next step.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
@slavazeph-coder slavazeph-coder changed the title Make Mind-Hack Autopsy a playable arcade lab + attractor evidence export Playable brain lab: deterministic simulation, per-sentence math, and calibration (which caught an inverted trust score) Jul 29, 2026
slavazeph-coder and others added 4 commits July 29, 2026 04:49
Despite the name, BrainSNN was a rate model: 7 scalar units, no membrane
potential, no threshold, no refractory period. This adds the real thing.

- src/lib/snn/lifNetwork.js: sparse random network of leaky integrate-and-fire
  neurons in the Brunel (2000) formulation — Dale's law (80/20 excitatory/
  inhibitory), fixed-amplitude delta synapses, axonal delays, Poisson external
  drive, exact exponential leak. Typed arrays plus CSR adjacency; ~150 ms for
  800 neurons over 250 ms of biological time, and fully seeded.
- src/lib/snn/snnMetrics.js: the measurements that characterise a network
  state — firing rate, CV of ISI, Fano factor, population synchrony, and a DFT
  of the population rate. The last one matters: gamma-band power becomes a
  MEASURED property rather than a closed-form assertion.
- brunelValidation.test.js checks published consequences of the model, so the
  simulation is falsifiable rather than merely plausible:
    * silent below the threshold drive nu_thr, firing above it
    * rate monotonically increasing in external drive
    * rate monotonically decreasing in inhibition strength g
    * irregularity monotonically increasing in g — the central Brunel result
    * near clock-regular firing (CV < 0.15) when excitation dominates
    * rate never exceeding the refractory ceiling

Found and fixed a delay bug while validating: the ring buffer wrote into the
same slot it read that step, so a spike reached later-indexed neurons with
zero delay instead of 1.5 ms. With correct delays the balanced network settles
at 42 Hz instead of 69 Hz, and the sweeps become cleanly monotonic.

Known limitation, asserted rather than overclaimed: the asynchronous-irregular
regime needs N ~ 10^4-10^5 at epsilon ~ 0.1 for presynaptic independence. At
browser-affordable sizes the network reaches high irregularity but keeps more
population synchrony than true AI, so the tests pin the monotonic trends
rather than claiming all four published regimes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
All twelve existing labs print a mission string that nothing verifies. This
one has a real win condition, a real fail state and a finite budget, all
evaluated against the deterministic model from brainModel.js.

- brainGame.js: mission rules, a pressure profile that ramps threat drive up
  while suppressing judgment, five interventions (lesion the amygdala, cut
  the salience or action route, stimulate PFC or cerebellum), and a
  three-axis score in the arcade's house style where control, stability and
  budget efficiency pull against each other.
- BrainGameLab.jsx: arcade experiment 014. A 2D circuit view of the seven
  regions and ten pathways, live activity and spikes, cut pathways drawn
  dashed, lesioned regions crossed out. Sandbox / Mission / Challenge modes,
  pause, single-step, reset, and a shareable seed.

Thresholds were measured, not guessed. Sweeping the hijack limit against the
model showed doing nothing breaches for ~179 consecutive ticks, silencing
threat or boosting judgment holds the line, and cutting the salience route
helps but is not sufficient on its own — so the limit is set at 60 to keep
that gradient. A plausible-looking intervention that still loses is what
makes the mission worth replaying.

Two bugs found while testing:
- The loss condition read the breach streak on the final tick, so a run that
  failed mid-way but recovered was scored as a win. It now latches on the
  worst streak seen.
- Control ratio diverged when a lesioned amygdala drove the denominator to
  zero (21.73 on screen). Capped at 10, past which the ratio says nothing
  extra.

17 unit tests plus an e2e that confirms the mission is genuinely losable and
that intervening keeps a run alive past the point it previously failed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
Two pieces of the evidence layer.

Krippendorff's alpha (src/lib/agreement.js) — a labelled corpus without an
agreement statistic says nothing about whether the labels are reproducible by
another person. Full implementation with the coincidence-matrix algorithm,
supporting any number of annotators, missing ratings, and nominal / ordinal /
interval distance. Ordinal matters here: "low vs extreme" should count as a
bigger disagreement than "low vs moderate". Reported with the conventional
reliability reading (>=0.8 reliable, >=0.667 tentative).

Verifiable run proofs (src/features/gaugegap/runProof.js) — because the model
is deterministic, a Defend the Brain run is fully described by its seed, mode
and the tick at which each intervention was applied. So a score is checkable
by RECOMPUTATION: replay the log and see whether the claimed result falls out.
Inflating a score means finding a log that actually replays to it, not editing
a number. The lab now records intervention ticks and exports a hashed proof.

Privacy: a proof carries a seed and a list of tick/id pairs. No text, no
personal data — so the lab's "nothing leaves your browser" promise survives
even if proofs are later submitted somewhere.

Note on scope: durable cross-user leaderboards need storage this deployment
does not have. Supabase is wired for magic-link auth only — no tables, no
persistence, no rate limiting, and the server is stateless. The verification
logic that makes a leaderboard trustworthy is what ships here; the storage
behind it is a separate, infrastructural piece of work.

Fixed two bugs found by the tests: evaluateRun derived elapsed ticks from the
rolling frame window (so a completed replay read as still running), and the
proof hash was taken over an object without the hash key while verification
spread it back as undefined, which canonical JSON renders as null.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
Completes the evidence path from labels to a defensible number.

- src/lib/evalMetrics.js: AUC (ranking), Brier (probabilistic accuracy) and
  Expected Calibration Error with a reliability table, plus precision/recall/F1
  at the best operating point. ECE is the one that matters for honesty: users
  read these scores as percentages, and a scorer can rank items perfectly while
  its numbers mean nothing as probabilities. One test pins exactly that case.
- scripts/eval-corpus.mjs: evaluate the engine against any externally labelled
  JSONL corpus. Public datasets are NOT vendored — they carry their own
  licences — so datasets/ is gitignored and the script points at a local copy.
- docs/ANNOTATION_RUBRIC.md: dimensions, ordinal levels, worked anchors, and
  the rule that a corpus release without a Krippendorff alpha does not get to
  claim reliability. Maps the published persuasion-technique taxonomies
  (SemEval-2023 Task 3, SemEval-2020 Task 11 PTC, Webis clickbait) onto our
  dimensions, and records that only one of the four tactics the firewall
  detects today maps cleanly to a published class — so those corpora should
  drive building a real detector, not just validate the current one.

Smoke run on a six-item set: AUC 0.778, ECE 0.290 — ranks reasonably, but the
scores do not yet behave like percentages. Worth knowing and worth stating.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01725FqR5KopKJJXN3RTfLpD
@slavazeph-coder slavazeph-coder changed the title Playable brain lab: deterministic simulation, per-sentence math, and calibration (which caught an inverted trust score) Playable brain: deterministic model, real spiking network, per-sentence math, calibration (which caught an inverted trust score) Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant