Section 2 identity-eviction hypothesis tested: prefixing the trained identity does NOT reopen the gate (0/40); any ~30-token tail text does, on both quant builds

[Blackwellboy]

A community-reported wire-level result proposed that the Laguna thinking collapse under long system prompts is a pure prefix prior: any caller system prompt evicts the template's trained default identity, and restoring that identity as the literal first line restores thinking (reported 6 to 7 of 8 under a 40K agent prompt that fired 0/8 bare). Since every condition in the C0-C9 grid that section 2 is built on replaced the default identity by construction, the two accounts (instruction dose vs identity eviction) were confounded in our published data. We unconfounded them.

Setup. Laguna S 2.1 3.25bpw hybrid (vLLM, poolside_v1 parser) as the gated model; Qwen3.6-35B-A3B NVFP4 as the ungated control. C6 (10-rule block, our published 3/40 crater) and C7 (full agent prompt, 24/40) byte-identical from the gate-study driver; four variants each: as-published, trained identity prepended as the literal first line, identity appended at the end, identity present but spliced mid-sentence. The identity string was taken verbatim from the serving checkpoint's chat_template.jinja default system message (byte-identical between the hybrid and NVFP4 checkpoints). 4 task types x 10 samples per cell = 320 turns/lane, single-turn throughout (so the #16 history-rendering mechanism cannot be a confound), thinking enabled, ceiling 4096, model-card sampling. 0 transport errors.

Result (Laguna, fired / 40):

variant	C6	C7
published	5	17
identity prefix	0	17
identity suffix	18	24
identity spliced	2	20

Qwen control: 320/320 fired regardless of variant, so there is no general prefix-prior effect of this text on an ungated model.

Reading.

1. On this build the prefix-prior account fails at its critical cell: identity-first ahead of C6 fired 0/40 (directionally worse than published, p about 0.055), and C7 was unmoved (17 to 17). The firing-axis numbers the rebuilt section 2 (merged in Laguna §2: the thinking gate is two axes, not one dose curve (closes #11) #12) is built on are unaffected. The community effect looks stack-specific (different CLI/proxy path, vLLM 0.25.1, NVFP4-TP1/FP8-TP2), and that report itself found FP8 and NVFP4 have different thinking policies, so build/quant scoping belongs next to any section 2 citation; ours is a third build.
2. New nuance we did not predict, now replicated interleaved on BOTH quant builds: a tail effect that is POSITION-GENERIC, not identity-specific. In the first (blocked) run the identity text at the END of C6 tripled firing (5/40 to 18/40). A dedicated interleaved control (4 variants x 4 tasks x 10 samples, per-quartet shuffled order, token-band-matched suffixes) then showed ANY matched tail text does the same: bare C6 0/40 vs identity-suffix 13/40, neutral filler 14/40, topical text 10/40 (every suffix vs bare p <= 0.001; identity vs neutral p = 1.0; pooled any-suffix vs bare p = 6.0e-06). The same interleaved design then replicated on the NVFP4 build, the build the section 2 curve was measured on, with byte-identical filler strings, re-token-matched on that tokenizer: bare 2/40 vs identity 17/40 / neutral 10/40 / topical 11/40 (every suffix vs bare p <= 0.025; identity vs either filler NS at n=40), and the bare cell replicates the published 3/40 crater (p = 1). Restoration magnitudes match across builds almost exactly (identity tail 17 vs 18 per 40, p = 1). Same instruction dose, ~30 extra tokens of anything. Dose remains the primary axis; system-prompt tail composition is a real secondary axis on this family, and the identity text carries no special weight in it on either build.
3. Template mechanics worth a section 2 footnote either way: any caller system message replaces the trained default identity, and an explicitly empty system message opts out of the default entirely. So "no system prompt" and "any system prompt" differ by more than instruction content on every stack.

A synthesis point for the two-axis model. The C7 identity arms are the second independent case of an intervention raising or holding firing while suppressing depth. Identity presence at C7 held firing flat or up (17/40 published, 17 to 24/40 across the identity arms) while collapsing fired-turn depth from a median of 1080 reasoning tokens to 120 to 200; the tools result had the same shape (C7 to C8: firing 60 to 72 percent, median tokens 745 to 282). Two unrelated interventions now move the two axes in opposite directions, which we read as independent confirmation that the two-axis decomposition section 2 was rebuilt on (whether it fires versus how deep it goes) is the right one; a single dose axis cannot produce either datapoint. The tail-composition controls add a softer third instance of the same dissociation: with firing statistically flat across the three matched suffixes, the identity suffix produced the shallowest fired turns relative to neutral filler on both builds (median 656 vs 809 reasoning tokens on the hybrid, 573 vs 782.5 on NVFP4), so identity text again trades depth away without buying firing. Caveats stated plainly: fired n per cell is 10 to 17, the comparison was not preregistered, and the topical suffix does not hold a consistent ordering across builds (1015 on hybrid, 391 on NVFP4), so we offer this as a consistent direction on the identity-versus-neutral contrast, not a third confirmed case at the strength of the first two.

Raw JSONL, drivers, and writeups: https://github.com/Blackwellboy/laguna-s21-lab/tree/main/identity-prefix (n=640 main grid, n=320 hybrid interleaved suffix control, n=160 NVFP4 interleaved replication; per-turn firing/reasoning-tokens/finish-path logged; three-way cross-build table in the study doc). Latency fields in the main-grid raw logs are flagged polluted (a concurrent study shared the lanes at low concurrency); both interleaved controls ran on idle/exclusive lanes; firing counts are unaffected throughout.

Credit for the originating observation: reported publicly on X; cited as community-reported here.

[Blackwellboy]

Correction to the synthesis paragraph in the issue text above: we ran the dedicated depth experiment, and the depth half of the two-axis model does not survive it. Retracting the depth axis as a causal finding.

The test: 200 turns, in-run interleaved (per-quintet shuffled order, single driver, exclusive lane), five arms at C7 on the NVFP4 build: bare, +identity suffix, +neutral suffix, +tool schemas (the C8 configuration), and +identity+tools. Depth is the dependent variable, so every arm was chosen to fire at usable rates.

Result: median reasoning tokens among fired turns are statistically flat across all five arms (858 / 806 / 721 / 933 / 474; every pairwise comparison p >= 0.13). The tools arm, which the cross-run comparison had collapsed to 282, was the HIGHEST median in this run. The identity arms sit within noise of bare. Both "second independent case" datapoints in the synthesis paragraph above were cross-run reads, and neither reproduces under interleaved control. The softer third instance (identity suffix shallower than neutral filler) reverses direction outright (806 vs 721).

What the collapsed cross-run medians actually were:

1. Task composition. Depth is strongly task-structured (math is a ~120-token floor in every arm, code runs 1064-2130, summarization fired 0/200 here, a third replication of that floor). Pooled medians move by hundreds of tokens depending on which tasks happen to fire in a given run, with no per-task depth change.
2. Tool-boundary truncation. Within the tool arms, turns that end in a tool call carry short pre-call reasoning (median 462 and 136 in the two arms) while turns that answer directly reason at 1293 and 847, and ceiling turns at ~3100. Half of tool-arm turns exit to calls. A fired set dominated by tool-call exits shows a collapsed pooled median for purely structural reasons: the reasoning episode is truncated at the tool boundary, not suppressed. This is the parsimonious mechanism for the published 282, and it also touches the Qwen C8 cell in our cross-model doc (the one Qwen condition that produces tool calls).

What stands and what does not: the measured medians (3536 / 745 / 282, and the 1080 vs 120-200 identity contrast) are real numbers from those runs; the causal reading "apparatus dose suppresses reasoning length" is what we retract. The firing axis is untouched: firing directions replicate here (tools 27/40 vs bare 20/40), and everything in the issue text above about firing, the prefix refutation, and the tail effect stands as posted. The identity+tools arm had the lowest median (474) but at p = 0.13 that is hypothesis-grade, not a replacement claim.

For the guide: this affects the section 2 material rebuilt on our grid, specifically axis 2 ("how long it thinks collapses monotonically with apparatus"), the "tools are a heavy suppressor of reasoning length" sentence, and the changelog line "cut median reasoning 62%". Axis 1 (whether it fires) and every firing number are unaffected. How section 2 should read is your call; the corrected framing on our side is: one firing axis with the known conjunction structure, plus depth variation explained by task mix and tool-call truncation rather than by a second suppression axis. We have added dated corrections in place across our repo rather than rewriting, and the same applies to our issue #11 wording ("tools are a major suppressor of reasoning length"), which this supersedes.

Study, driver, raw per-turn JSONL, and the tool-call/direct-answer split: https://github.com/Blackwellboy/laguna-s21-lab/tree/main/c7-depth-collapse

Methods note for anyone comparing depth numbers on this family: same-cell magnitudes drift between runs (documented in the prompt-topology study for firing; this extends it to depth), so cross-run depth comparisons are not valid evidence here. In-run interleaved arms or nothing.

[Blackwellboy]

Drafted with AI assistance (Claude); reviewed and submitted by @Blackwellboy.

Following up on my own comment above, because the numbers in section 2 are
mine and the causal reading I gave them was wrong. You have had no reason to
act on the note yet, and I would rather hand you the correction than leave you
carrying my error.

Short version: the axis-1 half of your rebuild stands. Only the depth axis
goes.

What I originally reported

Reasoning length collapses monotonically with apparatus: median 3,536 tokens
bare, 745 under an agent prompt, 282 under an agent prompt with tool schemas,
a 62% cut from tool schemas alone.

What a controlled replication found

Those medians are real measurements of those cells. The monotonic-dose reading
is not, because it was a cross-run comparison. I ran a dedicated grid, 200
turns, 5 arms x 4 tasks x 10 samples, interleaved in-run so arm and run are not
confounded.

Depth among turns that actually fired was statistically flat across bare,
+identity, +neutral, +tools and +identity+tools. All pairwise Mann-Whitney
comparisons came back p >= 0.13, and the arm carrying tool schemas had the
highest median, not the lowest. The 282 that reads as a collapse across
runs was 933 in-run.

The mechanism, which is the part worth having in the guide

Apparatus changes the mixture of exit paths, not the depth of reasoning. A turn
that exits via a tool call carries only the reasoning produced before the model
decided to call, because the reasoning episode ends at the tool boundary. Pool
those turns with turns that answered directly and the median falls for a purely
mechanical reason. Measured as score it reads as capability loss; measured as
depth it reads as reasoning suppression. It is neither.

So the supported claim is narrower than what section 2 says now:

• tools alter the exit-path mixture;
• pooled reasoning length can fall because tool-call turns end the reasoning
  episode early;
• controlled fired-turn depth did not fall.

Where it is still live, so you do not have to go looking

Checked against f846417 today:

• models/laguna-s-2.1.md:101 carries the monotonic-collapse sentence with the
  3,536 / 745 / 282 sequence.
• cards/laguna-s-2.1.html:48 carries the same 282 as bolded headline data, so
  the card and the guide would disagree if only one were fixed.
• Two follow-on lines rest on it: models/laguna-s-2.1.md:103 (the worked
  example, "cut median reasoning by 62%") and :110 ("tools are a heavy
  suppressor of reasoning length"). The changelog entry at :318 repeats the
  62%.

Your firing-rate findings are untouched, including the non-monotonic
persona-times-task result and the 3/40 for the dense ten-rule block. The
two-axis structure is untouched and is still the right frame. It is
specifically the causal claim on the length axis that goes.

The offer

I have a PR ready and have deliberately not opened it, because it is your
guide. It would:

• preserve your framing and the PR history, and not remove the medians;
• add a dated correction inline at section 2 rather than a silent rewrite, in
  the visible-correction style your changelog already uses;
• rescope the line at :110 to the exit-path statement above;
• update the card so it does not disagree with the guide;
• add a changelog entry dated to the correction, attributing the error to me.

Say the word and I will open it against whatever branch you prefer. If you would
rather write it yourself, everything above is yours to use, and I am happy to
review rather than author. If you would rather leave it, that is fine too and
the correction stays on my side.

Raw, driver and analysis for the replication:
https://github.com/Blackwellboy/laguna-s21-lab/tree/main/c7-depth-collapse

The harness-side statement of the same mechanism, with an independent n=492
measurement by @apollo-mg on a different stack:
https://github.com/Blackwellboy/model-serving-minefield/blob/main/traps/evaluation/42-single-turn-harness-scores-tool-calls-as-wrong.md

No rush on any of this. You shipped thirty rewritten files today and flagged a
rewrap of mine that nobody would have caught, so this is meant as help rather
than a chase.