Skip to content

patterns.md cap-hit entry: the Qwen 28/30 row is truncation, not degeneration — measured, raw published #17

Description

@Blackwellboy

The cap-hit entry currently reads "Usually it is the opposite, a degeneration loop that more budget would only feed" and "4,096 is not a stingy ceiling for that task," with our Qwen 28/30 as the headline row. We ran the direct test on the same lane with the byte-identical criteria task; raw: https://github.com/Blackwellboy/laguna-s21-lab/tree/main/qwen-ceiling

  • ceiling 4096 → 10/10 cap-hits, 8/10 empty (the 28/30 replicates)
  • ceiling 8192 → 10/10 non-empty, criteria-valid, zero cap-hits; flat through 12288 and 16384; reasoning demand plateaus ~5.2–5.7K tokens median
  • every 4096 cap-hit tail is non-degenerate: unique-line ratio 0.80–0.97, zlib ratio 0.31–0.38 — ordinary mid-task reasoning, no loops

So for Qwen on this task, 4,096 IS a stingy ceiling (natural cost ~7K completion tokens) and budget converts completely. This coexists with, rather than contradicts, the thread's other data: your Q4_K_M cap-retry showed stochastic loops where budget did NOT convert (/145 at 0.086 unique-line), and apollo-mg's /47 passed at the cap. The synthesis all three stacks reached in #10bucket on zero-extractable-code, check per model, use a loop signal (unique-line / compression ratio) rather than finish_reason — is exactly what the entry should say.

Concrete suggestion: keep the entry's two harness consequences (they stand), drop "more budget would only feed" as the general reading, and split the table row: Qwen 28/30 = demand-above-ceiling truncation (converts at 2x); degeneration loops = real but per-sample stochastic, detectable by tail metrics. Shape data if useful: @12288 constrained math still caps 3/10 while JSON-schema/table tasks (~1.5–2K reasoning) never cap — it is reasoning-demand-driven, not criteria-list-specific.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions