Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
152 changes: 152 additions & 0 deletions campaign/b1-cohort-ranking.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
# B1 cohort ranking — depth-blocking mechanism cohorts by size and leverage

Sources (every claim below cites one of these):

- **[C1]** `campaign/b1-depth-census.md` — fresh executed run, 2026-08-23, HEAD
`af9ded2b`, all 48 currently-PASSING languages exercised at depth against the
fixtures in `campaign/b1-depth-fixtures.md`; raw log
`campaign/fixtures/b1/depth-census-run.log` (`PASS=34 DIVERGE=0 FALLBACK=14
SKIP=0 ERROR=0 total=48`, log line 99).
- **[C2]** `docs/compact-route-coverage-census.md` — 2026-07-20 census over all
206 registered grammars (153 FALLBACK broken into eleven `[mechanism=…]`
classes; class-size table lines 118–121; burn-down discussion lines
160–205, 271–291, 359–362).
- **[C3]** `campaign/smoke-census-summary.md` — summary of [C2] including the
corrected interpretation that every `eof-byte-short-frontier` decline has
exactly one derivation and fails only because the accepted head's boundary
ends one byte short of the source.

## Scope note: two complementary rankings

The three named classes (`zero-width-shift`, `repetition-shift-class`,
`eof-byte-short-frontier`) come from the whole-campaign census [C2]. Note that
Ranking A below is census-PASS **at the authored fixtures used** [C1] — it does
**not** establish PASS-on-real-files-today; this carries the G-2 authored-fixture
caveat (`campaign/b1-harness-gaps.md`). The fresh
depth run [C1] re-exercised only the 48 currently-PASSING languages and found
**none** of those three tags on any decline — the declines there carry newer,
finer shapes (`scheduler-frontier-shape`, `recovery-entered`) plus one
unclassified case. Both views are ranked below; they rank different things:

1. **Depth-blocked languages among the current PASS set** ([C1]) — what blocks
us from converting PASS-at-smoke into PASS-on-real-files-today.
2. **Campaign-wide leverage** ([C2]/[C3]) — what converts the most of the 206
grammars overall.

---

## Ranking A — languages blocked at depth among the 48-PASS set [C1]

| Rank | Cohort | Languages blocked | Members |
|--:|---|--:|---|
| A1 | `scheduler-frontier-shape` | 8 | cylc, dtd, earthfile, elixir, julia, nushell, purescript, v |
| A2 | `recovery-entered` (production error tree) | 5 | bitbake, chatito, disassembly, mermaid, templ |
| A3 | unclassified decline | 1 | ledger ("accepted compact root is incomplete … span=0..932 expected=1..932", log line 77; no `[mechanism=…]` tag) |

(Note: `b1-depth-census.md` says "6" for this bucket but enumerates five rows;
the row list is authoritative — 34 + 8 + 5 + 1 = 48.)

### Campaign-leverage overlay within Ranking A

- The 8-language `scheduler-frontier-shape` bucket splits by decline boundary
(verified against `campaign/fixtures/b1/depth-census-run.log`):
**no_action (6)** — dtd, earthfile, elixir, julia, purescript, v ([C1] rows 8,
12, 13, 15, 38, 46) — sharing the "converged-path reduction split no-action
drop" shape family (julia's variant descends from an unproved historical
boundary resurrection); and **accept_without_materialization (2)** — cylc
("did not accept EOF: generic scheduler requires one certified accepted
derivation") and nushell ("EOF recovery admission requires scanner
quiescence"). One fix shape touches 6 of 8 immediately.
- All declines are fail-closed: DIVERGE=0 [C1], so none of these is a
correctness risk — pure upside.

## Ranking B — campaign-wide leverage over all 206 grammars [C2][C3]

| Rank | Cohort | Languages declined | Share of FALLBACK | Tractability per [C2] |
|--:|---|--:|--:|---|
| B1 | `zero-width-shift` | 33 | 21.6% | High — one uniform shape, confirmed to recur at real-corpus depth (line 119, 171) |
| B2 | `repetition-shift-class` | 27 | 17.6% | Medium — scheduler explicitly documents it as unimplemented (lines 120, 178, 291) |
| B3 | `eof-byte-short-frontier` | 90 | 58.8% | Certain but narrow — every decline is a single-byte trailing-newline frontier gap, exactly one derivation (lines 134, 160, 271; [C3]) |

Leverage rationale (per [C2] lines 199–205, 247, 359–362): although
`eof-byte-short-frontier` is the largest raw bucket, the depth check showed it
does **not** block real-file depth for flagship languages — the depth-persistent
mechanisms are `repetition-shift-class` and `zero-width-shift`. Hence B1/B2 outrank B3 despite B3's raw count.

---

## Recommended burn-down order

1. **A3 unclassified (ledger)** — cheapest, unblocks measurement itself: until
this decline routes through `admission_census.go`'s
`admissionCensusClassify`, every future census will have an unexplained cell.
*Named first fix:* route the "accepted compact root is incomplete"
exit path through `admissionCensusClassify` so it emits a `[mechanism=…]`
tag; then re-run the manifest-driven matrix (no routing change to the
decision itself). Evidence base: [C1] row 27; `campaign/b1-harness-gaps.md`
G-3.
2. **A1 `scheduler-frontier-shape`** — biggest depth-blocked bucket (8/14
declines), and 6 of its members share one verbatim detail. *Named first
fixes (one per boundary):* (a) **no_action boundary** — give the
converged-path reduction split's no-action drop alternative-set coverage by
blending the surviving derivation with the certified accept; this addresses
dtd, earthfile, elixir, julia, purescript, v. (b)
**accept_without_materialization boundary** — separately ensure the generic
scheduler can produce one certified accepted derivation at EOF so cylc and
nushell stop declining there; the no_action fix alone does not give
`accept_without_materialization` a certified derivation to see.
Smallest repro: earthfile (116 bytes), purescript (148 bytes), v
(235 bytes), julia (1089 bytes) per `campaign/fixtures/b1/depth-census-run.log`
(`campaign/b1-depth-fixtures.md`); dtd (198 KB) is the stress case.
Evidence: [C1] rows 8, 12, 13, 15, 38, 46.
3. **A2 `recovery-entered`** — 5 depth-blocked languages, all fail-closed
error trees from "scheduler has no table action for the elected token".
*Named first fix:* teach the compact route to admit the recovery-elected
token's derivation (certify the production error-tree root as an accepted
derivation rather than declining), starting from chatito (99 bytes, the
minimal repro). Evidence: [C1] rows 3, 4, 10, 32, 42.
4. **B1 `zero-width-shift` (33)** — highest campaign-wide leverage per [C2]
line 171. *Named first fix:* add the missing scheduler table entry for the
uniform "generic zero-width shift" shape so the head advances without
consuming input. Evidence: [C2] lines 119, 171, 284.
5. **B2 `repetition-shift-class` (27)** — explicitly documented as an
unimplemented scheduler feature. *Named first fix:* implement the
repetition-shift table action the scheduler already names in its decline.
Evidence: [C2] lines 120, 178, 291.
6. **B3 `eof-byte-short-frontier` (90)** — largest but shallowest: every
decline is a one-byte short accepted-head boundary on a single-derivation
parse. *Named first fix:* when the accepted head's end offset is exactly
source length − 1 and the final byte is a newline, extend the certified
span to include it. Certain win, fixture-only scope [C2] lines 134, 271;
deliberately last because it does not block real-file depth [C2] lines
199–205.

## Verification hook

Every fix above is checkable with the existing harness, unchanged:
regenerate the manifest with
`campaign/fixtures/b1/run-depth-census.sh`, then

```
GTS_ADMISSION_REAL_CORPUS=1 \
GTS_ADMISSION_REAL_CORPUS_MANIFEST=campaign/fixtures/b1/depth-manifest.json \
GTS_ADMISSION_CENSUS=1 \
go test -tags gts_parsercorephase0 -run TestAdmissionCandidateRealCorpusMatrix -v .
```

and confirm the targeted rows flip FALLBACK→PASS with digest logged ([C1]'s
exact command). No code change is made by this document; it is evidence only.

## Buried headline: depth work surfaced what the smoke census predicted

This depth run [C1] surfaced two mechanism classes that the 2026-07-20 smoke
census [C2] recorded as near-empty:

- **`scheduler-frontier-shape`**: 8 languages blocked at depth, versus 1
language (0.7%) in [C2] (its line 122).
- **`recovery-entered`**: 5 languages blocked at depth, versus zero in [C2].

This is exactly what [C2] predicted at its lines 130–132. It is the strongest
argument for funding depth work: the mechanisms that actually block real-file
depth are invisible to a smoke census until you run one.

111 changes: 111 additions & 0 deletions campaign/b1-depth-census.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
# B1 depth census — per-language result table (48 currently-PASSING languages)

PASS set source: `docs/compact-route-coverage-census.md`, "Full per-language
table → PASS (48)". Fixture inventory: `campaign/b1-depth-fixtures.md`.

**This table now holds a FRESH EXECUTED RUN (2026-08-23, repo HEAD `af9ded2b`,
go1.25.1 linux/amd64), not predictions.** The driver is the shipped,
env-driven real-corpus admission matrix
`TestAdmissionCandidateRealCorpusMatrix` (`admission_real_corpus_matrix_test.go`,
build tag `gts_parsercorephase0`), pointed at the 48-fixture manifest
`campaign/fixtures/b1/depth-manifest.json` generated by
`campaign/fixtures/b1/run-depth-census.sh`. Exact command:

```
GTS_ADMISSION_REAL_CORPUS=1 \
GTS_ADMISSION_REAL_CORPUS_MANIFEST=campaign/fixtures/b1/depth-manifest.json \
GTS_ADMISSION_CENSUS=1 \
go test -tags gts_parsercorephase0 -run TestAdmissionCandidateRealCorpusMatrix -v .
```

Raw output: `campaign/fixtures/b1/depth-census-run.log`.
File summary (log line 99): `PASS=34 DIVERGE=0 FALLBACK=14 SKIP=0 ERROR=0 total=48`;
parse class (line 100): `FALLBACK clean=8 error-tree=6; PASS clean=34 error-tree=0`.

Status legend: `PASS` = compact route served a byte-exact tree of the real
fixture at depth (digest logged); `FALLBACK <mechanism>` = the route declined;
the mechanism is the `[mechanism=…]` tag emitted under `GTS_ADMISSION_CENSUS=1`
(`admission_census.go`, `admissionCensusClassify`). No routing changes were
made; this is evidence only.

| # | Language | Fixture bytes | Depth status | Mechanism / detail (verbatim from log) |
|--:|---|--:|---|---|
| 1 | bash | 14290 | PASS | digest a058477754da |
| 2 | beancount | 221 | PASS | digest ae44a1dc92db |
| 3 | bitbake | 168224 | FALLBACK | recovery-entered: scheduler has no table action for the elected token (production error-tree) |
| 4 | chatito | 99 | FALLBACK | recovery-entered: no table action for elected token (error-tree) |
| 5 | commonlisp | 242 | PASS | digest 2d734a8ebfbc |
| 6 | crystal | 223 | PASS | digest 07456e83e860 |
| 7 | csv | 123 | PASS | digest d32b1d4773be (log line 57) |
| 8 | cylc | 176 | FALLBACK | scheduler-frontier-shape: declined at accept_without_materialization — requires one certified accepted derivation |
| 9 | desktop | 152 | PASS | digest 0cb50bd238c1 |
| 10 | disassembly | 403 | FALLBACK | recovery-entered: no table action (error-tree) |
| 11 | dockerfile | 1396 | PASS | digest bc4621b7abd3 (cgo_harness/docker/Dockerfile) |
| 12 | dtd | 198540 | FALLBACK | scheduler-frontier-shape: converged-path reduction split no-action drop lacks alternative-set coverage by one non-blended survivor |
| 13 | earthfile | 116 | FALLBACK | scheduler-frontier-shape (same no-action-drop detail; production error-tree) |
| 14 | editorconfig | 121 | PASS | digest 49cd13246946 |
| 15 | elixir | 8504 | FALLBACK | scheduler-frontier-shape (no-action drop) |
| 16 | fish | 171 | PASS | digest 0d5b7bf6f354 |
| 17 | git_config | 127 | PASS | digest 07965a8f6cb6 |
| 18 | git_rebase | 170 | PASS | digest 5bb36b1e0def |
| 19 | gitcommit | 271 | PASS | digest 77a8c67e9e09 |
| 20 | go | 2194 | PASS | digest 833b5685c629 |
| 21 | gomod | 341 | PASS | digest 49167d5d85cd |
| 22 | hcl | 203 | PASS | digest 51d8a99368a0 |
| 23 | hyprlang | 117 | PASS | digest e7ac74417cc8 |
| 24 | ini | 260 | PASS | digest c25fb04c169d |
| 25 | julia | 1089 | FALLBACK | scheduler-frontier-shape: unproved historical boundary resurrection |
| 26 | kconfig | 283 | PASS | digest cb1cafe7232f |
| 27 | ledger | 932 | FALLBACK | unclassified decline: accepted compact root incomplete — span=0..932 expected=1..932 error=false allowErrorRoot=false |
| 28 | liquid | 233 | PASS | digest 4fb0ebfb9d25 |
| 29 | make | 217 | PASS | digest 3007b07a2aa5 |
| 30 | markdown | 2941 | PASS | digest f6b552589036 |
| 31 | matlab | 242 | PASS | digest d98d7e7f7aa8 |
| 32 | mermaid | 211 | FALLBACK | recovery-entered: no table action (production error-tree) |
| 33 | ninja | 793 | PASS | digest a0783ff8f7a1 |
| 34 | nushell | 121 | FALLBACK | scheduler-frontier-shape: EOF recovery admission requires scanner quiescence |
| 35 | odin | 263 | PASS | digest 47020fc04ab4 |
| 36 | pascal | 157 | PASS | digest 77bca8ae9b49 |
| 37 | prolog | 132 | PASS | digest fbd1bd717e78 |
| 38 | purescript | 148 | FALLBACK | scheduler-frontier-shape (no-action drop) |
| 39 | requirements | 85 | PASS | digest 0514e003e85c |
| 40 | svelte | 256 | PASS | digest 7697435b1072 |
| 41 | tcl | 213 | PASS | digest 666666fc5bfb |
| 42 | templ | 2161 | FALLBACK | recovery-entered: no table action (production error-tree) |
| 43 | tmux | 140 | PASS | digest 4072c661f6b7 |
| 44 | twig | 205 | PASS | digest 2324248deb55 |
| 45 | uxntal | 134 | PASS | digest 23be1cc10e9c |
| 46 | v | 235 | FALLBACK | scheduler-frontier-shape (no-action drop) |
| 47 | vimdoc | 436 | PASS | digest b1de1d91afef |
| 48 | xml | 331 | PASS | digest a02018448256 |

All cells above are verbatim from `campaign/fixtures/b1/depth-census-run.log`
(log lines 51–100), which is the authoritative record. The log is gitignored;
its SHA-256 is `329869642604d528297bb6ebdac8f36a545daf1755e8405ed9980b497b1cd875`.

## Cohort ranking at depth

- **Held PASS at depth (34/48):** bash, beancount, commonlisp, crystal, csv,
desktop, dockerfile, editorconfig, fish, git_config, git_rebase, gitcommit,
go, gomod, hcl, hyprlang, ini, kconfig, liquid, make, markdown, matlab,
ninja, odin, pascal, prolog, requirements, svelte, tcl, tmux, twig, uxntal,
vimdoc, xml.
- **Declined, mechanism `scheduler-frontier-shape` (8, of which 7 clean):**
cylc, dtd, earthfile, elixir, julia, nushell, purescript, v — earthfile is
error-tree (row 13), the rest are on a clean production tree.
- **Declined, `recovery-entered` with production error tree (5):** bitbake,
chatito, disassembly, mermaid, templ — inputs needing recovery; the route
declines fail-closed. (The earlier count of 6 was the error-tree parse-class
count, which also contains earthfile.)
- **Unclassified (1):** ledger — accepted-root-incompleteness decline that did
not carry a `[mechanism=…]` tag.

## Relation to the 2026-07-20 census

The prior census's 10-file depth check (docs/compact-route-coverage-census.md,
"Depth check") found only go holding. The measured split for this full-48 run:
34 PASS, of which 26 ran on authored fixtures of 85–436 bytes. The four July
regressions that flipped back (hcl, make, xml, svelte) did so on fixtures
1.5x–45x smaller than the corresponding July corpus files, so this run neither
confirms nor contradicts the July depth extrapolation. Zero DIVERGE in either
run: every decline is fail-closed.
Loading
Loading