Repository navigation
Train 26c: whereis definitions from the parser, owner hop, map inventory, lean expand, CLI speed, #358 JS/TS + Go, #362, #339 - #384
Merged
Conversation
…mns)
A Rails schema is recognized BY CONTENT, never by path: a Ruby file whose tree holds
a `create_table "x", … do |t| … end` call (the call's do_block carries one bare block
parameter) is a rendered schema. Each `t.<type> "name"` / `t.<type> :name` in a table
block mints ONE SymKind::Section def at Lang::Ruby — the data-kind slot a doc heading
or YAML key already occupies, now with call edges: Rails-generated attribute uses
(`product.price`) bind to the column and reach the call graph + PageRank on every
Rails corpus (accepted, disclosed in CHANGELOG + tags.scm). model.h's Section wording
changes with it ("isolated in the graph" was true only while every Section was a
markdown heading/YAML key; the Ruby ones admit edges).
The id rule, the four real-world spellings (pinned by the gate):
- no id column/spelling -> implicit `id` def at the table-name
string (no `id` token exists; anchor is
per-table unique, so the multi-def floor
id on N tables -> defs="N" holds)
- `id: false` -> no id def
- `id: :uuid` (simple_symbol value) -> `id` def at the pair key (the uuid is a
TYPE of the id column, never a column of
its own — Rails docs; type not modelled)
- `primary_key: "x"` -> def named the string instead
`t.timestamps` mints created_at AND updated_at (anchored at the method token's edges).
The DSL CALLS keep their reference posture (string/datetime/create_table stay external).
MIGRATIONS are excluded by a class/module-nest gate: a rendered schema's tables live
at file level or under Schema[].define — never inside a class body — so a MIGRATION's
class-wrapped create_table (string- or symbol-named) mints nothing. That is what makes
the content gating honest: the capture is content-addressable everywhere, but the
rendered schema stays the ONE source of column names — indexing a migration beside its
schema would double every def and its PageRank weight. add_column/remove_column/
change_column are argument data and read nowhere, so a column added then dropped never
registers. All four interference points are pinned by the three migration fixtures.
Disclosed floors, each pinned by an arm: t.index/references/belongs_to/polymorphic name
no column; a symbol-named table is a migration spelling, not a schema surface; string
fragments (`where("price > ?")`) stay opaque; duplicate columns across tables stay
SEPARATE defs. The picker splits honestly: `rec.name`/rich receivers are
graph_ambiguous, `self.name` in the defining class pins its own def.
Gate: test/rubyschemacheck.sh, 32 arms, ALL PASS on build/ and asan/; registered in
test/regression.sh same-commit (per repo rule). Fixture test/rubyschemafix: schema text
from a real, running Rails 8.1 app's `bin/rails db:schema:dump` (verified
db:schema:load-able), original domain tables scrubbed to spike_* names; two spellings
no dump emits are restored by hand and marked in-file (id: :uuid, which that dump
actually failed on, and a literal t.timestamps); three migration fixtures pin the
interference floors. Before-evidence recorded: on the pre-change binary every column
read defs="0" external="1"; --uses=name was defs="9" and now reads defs="19"
external="0" with binding rows.
kParserVer is 123 on this branch (carried 122 pre-rebase; train-20's .astro work took
122, so this capture sits at 123 — renumber again at merge if main has moved on),
with the quality.h mirror bumped in the same diff. No record layout change
(kCacheVersion 25 / kQSnapCacheScheme 14 stay). Tags.scm, CHANGELOG and the fixture's
USECASES.md carry the disclosures; gatecount re-pinned (649 -> 650 on this base) and
qschemetrip re-pinned in the same commit.
Verification on this head (rebased onto train-20 b343b98 / 0.6.4): determinism
byte-identical + xmllint clean; ASan+LSan repo sweep clean (g1fresh + cachefuzz ALL
PASS, both binaries built from one tree); quality-delta vs origin/main regressions="0"
gating="0"; local battery ALL PASS (rubyschemacheck 32 ×2 binaries, rubyattrscheck 47,
rubysettercheck, manifest, loopconservation, gatecount 650, qschemetrip, versioncheck,
qackorigin, qackconcurrency, ackonly); full suite alone in leap-build: gates=665
pass=655 skip=4 fail=6 — four environment/upstream (crossdirinclude + legendcoverage:
documented -j6 timeouts on this laptop, each proven passing standalone ~253s/~187s;
htmlrendercheck: known HOME artifact; regexcheck: the train-19-oracle arm needs rg,
absent from the container) and TWO self-inflicted on the first rebase draft (the
regression.sh absorb list was merged with a `;;` spelling error: manifestcheck
"regression.sh does not parse" + loopconservation's zoomcheck-vanish) — fixed, both
pass standalone on this committed head. This message states the verified numbers.
AI-assistance note: developed with AI assistance (DeepSeek V4 Flash and Qwen 3.8 Flash
models) under the author's direct supervision; code, fixture provenance and every
claim in this commit were reviewed and verified by the author before the commit.
Four findings, addressed:
1. CHANGELOG: the rebase auto-merge had left a STALE DUPLICATE of the schema
entry inside [0.6.3] carrying the pre-rebase text ("D-BIND", "byte-faithful",
"kParserVer 121 → 122") — an unreleased feature was claimed in a released
section with the wrong parser history. Deleted; the [Unreleased] copy (the
truthful 122 → 123 wording) is the only one left.
2. Receiver binding: a column call now must be ON the block parameter the
create_table call bound (`t` in a rendered do |t|) — previously ANY bare
identifier receiver was accepted, so `helper.string "x"` minted a phantom
column. rubySchemaColumnCall takes the parameter's name from the table walk
and refuses a different receiver. New negative fixture + gate arm
(`helper.string "unbound_column"` stays undefinable).
3. Gate map-kind arms are now scoped INSIDE the schema.rb file block (the yaml
Section key can no longer satisfy them) and assert the overloads counts
(name overloads="9").
4. Locality fixture: consumer_locality.rb claimed to demonstrate the locality
pin from a DIFFERENT class than the one defining `name`. The genuine arm —
`self.name` inside PairDefColumn (its own file) — was already pinned at
pair_def_column.rb:15 by the gate; the standalone file added nothing (its
bare `name` call contributes no row) and is deleted. The bare-name-with-no-
evidence floor stays covered by the count arms.
Gate test/rubyschemacheck.sh: 34 arms, ALL PASS on build/ and asan/;
qschemetrip re-pinned; kParserVer stays 123 (unreleased; the receiver floor is
folded into its note — a 122/123 cache written before this round differs only
there). Verification: deterministic byte-identical runs + xmllint clean; the
full local battery green (rubyschemacheck 34 x2 binaries, rubyattrscheck,
rubysettercheck, manifest, loopconservation, gatecount, qschemetrip,
versioncheck, g1fresh, cachefuzz, qackorigin, qackconcurrency, ackonly);
ASan + LSan sweep clean; quality-delta vs origin/main regressions="0"
gating="0".
AI-assistance note: developed with AI assistance (DeepSeek V4 Flash and Qwen
3.8 Flash models) under the author's direct supervision; every change here was
reviewed and verified by the author.
…339) Response to joyful-ii-V-I's review. The design change: schema columns are now DEFINITIONS ONLY. The "admission consequence" (columns reaching the call graph + PageRank) is retracted after the maintainer measured that untyped receivers bind unrelated calls to columns (`response.code` against a `code` column) and that a 200-table app's map was taken over by columns (196/200 rows; declines 292->1324). Columns still answer --uses/--grep/--whereis and appear in the map as defs. Required, all addressed: 1. Definitions-only: buildGraph/1d's byName fill skips SymKind::Section && Lang::Ruby (the same table feeds call binding AND common-name damping, so both consequences disappear at one seam, mirroring contextratio.h's isMeasurableKind). Gate §3 is now defs-only arms: callers=id -> defs=14 count=0, callers=created_at -> defs=2 count=0, a false-binding arm (a column-ONLY name, ref, gains no caller), and callers=name -> defs=19 count=3 (the METHOD/attr defs' edges only). model.h's sec comment is back to "no call edges"; CHANGELOG/tags.scm reworded. The ambiguity gauge drops 4 -> 2 (the id/created_at column sites left the gauge), pinned. 2. Shared walker instead of acking the duplication: captureRubyAttrDefs and captureRubySchemaDefs are now ONE pre-order walk (captureRubyDefs) dispatching both lanes per call node, with the per-file signals preserved (quality-delta duplication row must be gone — verified on the range, below). 3. Constraint floors: check_constraint / exclusion_constraint / unique_constraint added to kRubySchemaNonColumns — `t.check_constraint "price > 0", name: ...` no longer mints n="price > 0" (real dumps were measured); pinned by a fixture + undefinable arm. 4. kParserVer lands at 126 (123 is reserved for #325, 124 in flight, 125 queued for #338; the merge commit sets the final number). Mirror 126 in the same diff. 5. Version-stale "122" comments made number-free: tags.scm, the ingest_names.h lane header, model.h, rubyschemacheck.sh header, both .ripwire_quality_acks reasons. Optional, all addressed: composite/symbol primary_key now mints NO implicit id (an array-valued key has no id; the rendered columns are the block's own t.<type> lines) pinned by a composite-key fixture + arms, with the id defs=14 arm as the regression guard; spike_quote_single now spells t.string 'name' so the single-quote path is pinned; the lane header no longer claims "the container does not matter" (Gate 0 refuses class/module-nested tables); CHANGELOG "before" corrected to defs=10 (9 attr/def defs + the yaml key, per the maintainer's measurement). The lpin locality row is moot under definitions-only — answered on the review thread. Nice-to-have, all addressed: rubyFirstNonCommentArg's unused src parameter dropped; CHANGELOG floor lines for references/belongs_to (no <x>_id), create_join_table, and the Rails 8 queue_/cache_/cable_schema.rb content-match. Gate test/rubyschemacheck.sh: 39 arms, ALL PASS on build/ and asan/. Verification: determinism byte-identical + xmllint; full battery green (rubyattrscheck 47, rubysettercheck, manifest, loopconservation, gatecount, qschemetrip re-pinned, versioncheck, g1fresh, cachefuzz, qackorigin, qackconcurrency, ackonly); ASan + LSan sweep clean; quality-delta on the maintainer's range b343b98..HEAD: the captureRubyAttrDefs|captureRubySchemaDefs duplication row is GONE (shared walker) and gating=0 (verified after this commit). AI-assistance note: developed with AI assistance (DeepSeek V4 Flash and Qwen 3.8 Flash models) under the author's direct supervision; every change here was reviewed and verified by the author.
…explore limit parity — PoC for issue #294 * test(truncvocabcheck): arm (H) — the budgeted-bundle continuation contract (issue #294), red vs the pre-change binary Gate-first for the paging PoC (upstream redhat-et issue #294): the arm is written BEFORE the code and is RED on the pre-feature binary, which is the intended state (runtracecheck.sh discipline). It greens when the byte-budgeted --for/--pack-task bundle accepts --offset and carries the pageview quintet. SCOPE CORRECTED BY MEASUREMENT (this is the discovery that changes issue #294): - --for=TASK --limit=N (+offset=M) ALREADY pages: the file-grain widening page, full pageview quintet, owned by forwidencheck — not re-gated here (§6.4). - The MCP 'for' verb serves that widening page today (limit=5 → <files shown= total= has_more= next_offset= next=…>); MCP 'explore' refuses limit entirely. - --for --token-budget=N --offset=M REFUSES: the byte-budgeted compact bundle is the actual open gap — cut disclosed (over_ce ceiling= reason=) with no continuation handle. Arm (H) guards exactly that, plus the offset=0 byte-identity regression floor (acceptance criterion 2). Arms: quintet presence on paged budgeted roots (for_budget_full red on the ignored-offset=0 answer; for_budget_page/pack_budget_page red on the refusal), capped 0|1 bit, shown<=total coherence, the byte-identity floor, and a mutation control (a bare paged root with no has_more=/next_offset= IS detected). Existing sweep assertions untouched: the (F) vocabulary arm and its mutation control still PASS on the same run. * test(listingpagingcheck): arm (G) — the budgeted-bundle continuation seam (issue #294), red vs the pre-change binary Gate-first, second arm of the paging PoC. The seam contract for the budget-mode continuation once it exists: --offset=N --limit=M must be the EXACT continuation of the previous page over the ranked candidate set — page[0:3]+page[3:6] == page[0:6] in candidates, rank order, no gap, no overlap — the same contract arm (A) holds for <a> rows, measured on the sig rows of the paged budgeted bundle. RED on the pre-feature binary, intended (today --for --token-budget refuses --offset/--limit; the arm records the refusal). Mutation control: an overlapping fabricated page pair IS detected — the arm can fail. Affected-reference note for the implementation phase: pagingsweepcheck owns the honoring/not-ignored contract — its arm (L) already lists --for (the file page, 2026-09-12 L-W) and pins the BARE --for root as quintet-free, which the budgeted continuation preserves (quintet only on PAGED budgeted answers). If the feature updates --help's HONORED-by paragraph for budget-mode paging, pagingsweepcheck's arm (L) universe check must be reconciled in the same commit. * test(paging PoC): arms C/D/E/F + explore-parity — the complete gate-first phase, red vs the pre-change binary Completes the gate-first phase for the budgeted-bundle continuation (upstream issue #294, corrected scope). Every arm is RED on the pre-feature binary with a green mutation control, and every pre-existing arm in its host gate stays green on the same run. Home corrections vs the v2 spec, per §6.4 (the gate that owns the subject takes the arm): - ARM C (within-page <bodies> rule-5 disclosure) folded into truncvocabcheck's arm (H) — that gate owns the <bodies> budgeted-section vocabulary (§B8.3). v2's handoffcheck home was a mis-assignment (it gates --handoff, a different frozen spec). - ARM F (cliff invariance across pages, CONTINGENT on the #294 ruling — it encodes the proposed compute-once answer and is rewritten if maintainers pick per-page re-cliff) folded into arm (H), which already parses the paged docs; adaptivecutshapecheck keeps gating the pure adaptiveCut shape. - ARM D -> budgetpolicycheck: per-page est_tokens<=budget; a past-the-end page is shown="0" capped="1" has_more="0" (pageview.h past-the-end contract). - ARM E -> packtaskcheck: continuation pages carry their OWN tier marker (the packtask.h:1690 two-dialects-one-count lesson); offset=0 byte-identical to the un-paged pack-task. - (6d) explore-parity -> mcpverbscheck: explore refuses limit today (unknown field) while for serves the bounded widening page; the arm records the refusal and demands the same mcpverbs.h:327-333 argument path. Also noted for the implementation phase: pack-task's --offset=0 is accepted-and-ignored today (byte-identity floor passes trivially); the nonzero offset refusal is where the continuation must land. Environment: xmllint installed (libxml2-utils) so packtaskcheck runs its full F5 arms locally. * feat(paging): the --for budgeted-bundle candidate page (issue #294) — renderer + guard carve, arms (G)/(D) green, (H) green for --for The first implementation slice of the paging PoC (upstream issue #294, corrected scope). --for under --token-budget with an explicit --limit/--offset now serves a resumable page over the SAME ranked candidate set the bundle cuts: - window [offset, offset+limit) of the positive-score candidates, (score desc, id asc) — the same total order packSignatures selects with (radixSortByScoreDescId); - packed by the SAME sigs emitter (redaction, notes, doc excerpting, H1 ladder all reused — a window, not a new surface); - the pageview quintet spliced onto the <sigs> opening (the rootOpenStr.insert idiom; the open tag is byte-pinned by forbudgetmonotoncheck, which is why the splice happens after render, never inside serialize.h); - tier="head"/"below-cliff" from forCut.cliffRank — computed ONCE, the same statistic --adaptive and the confidence disclosure read (arm F's invariance holds by construction); - est_tokens MEASURED from the page's own final bytes, two-pass to close the digit-count feedback loop. The UN-PAGED bundle is untouched and byte-identical (forbudgetmonotoncheck PASS on this change): --offset=0 ALONE stays the un-paged bundle — its resume point is its own <sigs shown=>, so an agent resumes with --offset=<shown>. The file page keeps every windowless-budget call exactly as forwidencheck pins it. cli.h: the kTokenBudgetGuard refusal now exempts --for/--pack-task beside an explicit window (the pages-AND-shapes-by-budget class --pr-context held alone), and the guard sentence says so. Red→green evidence, and what the process caught: - arms (G) seam: 5 real <d> rows, page[0:3]+page[3:6]==page[0:6], no overlap — ALL PASS (the arm's first cut matched <s>/<sig>, found 0 rows, and produced a VACUOUS green; fixed with the real tag + an anti-vacuous row guard) - arm (D): est_tokens<=budget per page; past-the-end is shown="0" capped="1" has_more="0" — ALL PASS - arm (H): the --for captures green (quintet + offset=0-alone byte-identity floor); pack_budget_page stays RED by design — the --pack-task slice is next - smoke-caught bugs: escapeXml scratch-buffer reuse mangled route= (per-attr buffers); the shown= digit parse was off by one ("shown=" is 7 chars); fixed char[160]/char[96] buffers truncated has_more=/root= mid-attribute on longer tasks (std::string assembly now — caller-controlled task length has no sane fixed size) - determinism: the page is byte-identical across runs (diff -q) Still red, by design, awaiting their slices: --pack-task (arm E + (H)'s pack_budget_page) and MCP explore limit parity (mcpverbscheck 6d). * feat(paging): the --pack-task candidate page + MCP explore limit parity (issue #294) — all six arms green Completes the paging PoC implementation. Three slices: 1. SHARED RENDERER: the --for candidate page (commit 0fb9397) is refactored into emitForCandidatePage( ing, lensRank, forCut, task, routeNote, rootArg, redactPtr, tokenBudget, pageLimit, pageOffset ) — one implementation, two callers (runForLens, runPackTask), so the two bundle verbs cannot serve a different page (the file's own one-implementation rule). 2. --pack-task: budget + explicit window pages its ranked candidates (same lensRank, lr.routeNote, one adaptiveCut computed ONCE — the same call --for's disclosure reads). --partition + window refuse together (a partitioned bundle has N+1 slices, no single candidate order). cli.h: --pack-task joins honorsPaging ONLY under budget+window — the bare bundle still refuses the pair. 3. MCP explore: limit/offset parity with the for twin — the same mcpPageArgs bounded pair, the same file-grain widening page (computeForFilePage/ renderForFilePageXml on the same lr), one kMcpVerbFields row driving the inputSchema, the unknown-field refusal and mcpw3fixcheck together. partition + limit/offset refuse together, mirroring the CLI. GATE-FIRST EVIDENCE (all arms were red vs the pre-change binary, commit 75b8c70): - truncvocabcheck (H): ALL PASS — quintet, within-page <bodies> rule-5, cliff invariance, offset=0-alone byte-identity floor, mutation control - listingpagingcheck (G): ALL PASS — real seam, 5 <d> rows, no gap/overlap - budgetpolicycheck (D + A): ALL PASS — est_tokens<=budget per page; past-the-end is shown=0 capped=1 has_more=0; the guard sentence still parses - packtaskcheck (E + F1..F5): ALL PASS — tier marker on continuation pages, offset=0 byte-identical - mcpverbscheck (6b/6c/6d + all): ALL PASS — explore now serves the quintet page - mcpw3fixcheck: ALL PASS — declared fields match the live inputSchemas - forwidencheck: ALL PASS — arm (6) REWRITTEN in the same commit as the behavior change: --token-budget beside a window no longer refuses (the combination now has a defined meaning, the candidate page); the arm now pins the STRONGER contract (budget+window answers <sigs>-rooted with the quintet, so the file page stays budgetless by construction, never by refusal). This is a deliberate contract change, not weakened coverage. - shapingflagcheck, forbudgetmonotoncheck: ALL PASS — the guard buckets and the byte-pinned un-paged sigs ladder untouched. Determinism: --for, --pack-task and explore pages are byte-identical across runs (diff -q). Well-formedness: xmllint --noout clean on all page shapes. * fix(paging): reword the escape-buffer comment — prose drifted the fixedbufsweep mentions pin CI caught what the local sweep could not see: fixedbufsweep arm (S6) pins the count of lines mentioning snprintf/formatTo in src/ — comments included — and this branch's comment naming formatTo in prose drifted it 333→334 on every platform and shard. The gate's own instruction names the rule: do not bump the number without re-reading the sites; the honest fix is prose that does not drift a pinned enumeration — the comment now says 'one emit call's argument list'. * fix(rebase): reconstruct verbs_for.h from pristine upstream + the four paging transforms The rebase onto the discarded main (c001c47) hit a structural conflict: upstream's new renderForHdrRowsXml landed at the exact anchor our refactor inserted the shared emitForCandidatePage inline, and the hunk-resolution glue nested our function inside upstream's mid-body cut (three function-definition errors + parser cascades). Reconstruction, deterministic: upstream's pristine verbs_for.h + four transforms applied programmatically — (1) the budget-aware file-page dispatch condition, (2) the shared inline after renderForHdrRowsXml's true close, (3) the runForLens call site after deriveForConfidence (which upstream evolved to three parameters — kept), (4) the runPackTask page branch before the partition anchor — plus the 29b2980 comment reword (a prose 'formatTo' had drifted fixedbufsweep's pinned mentions enumeration 333→334, the root cause of all 19 PR-3 CI failures). Verified on the reconstructed tree: 10/10 gates ALL PASS (fixedbufsweep, truncvocabcheck, listingpagingcheck, budgetpolicycheck, packtaskcheck, mcpverbscheck, mcpw3fixcheck, forwidencheck, shapingflagcheck, forbudgetmonotoncheck) — the byte-pinned un-paged sigs ladder included. * test(modifierguardcheck): budget+window (--for --limit --token-budget) answers with the candidate page — arm rewritten with the behavior change CI caught the sweep-miss: modifierguardcheck's guard-table row pinned the OLD kTokenBudgetGuard refusal for --token-budget beside --for --limit. The carve removed that refusal (the combination now has a defined meaning — the budgeted-bundle candidate page), so the row's $NOROOT trick (the old guard fired BEFORE root resolution) died on the bad root with usage output — the 'wrong message' it reported. The replacement pins the STRONGER contract on a REAL corpus: exit 0, a <sigs> root carrying the pageview quintet — the same seam forwidencheck arm (6) holds from its side. Sweep-miss correction: the guard-surface gates beyond the original ten are now swept locally and green — modifierguardcheck, pagingsweepcheck, argvdiffcheck, tokenbudgetcheck, fordisclosurecheck, helpbudgetcheck (16 gates total ALL PASS on this head). * fix(paging): sink-form DISCLOSE for the candidate page's degrade paths — the page document is the sink selfcheckcheck arm (R) caught what my 16-gate sweep could not count: two sink-less DISCLOSE( msg ) sites in emitForCandidatePage raised the pinned enumeration 49 -> 51. The contract (Diagnostics.h §4b): the sink is the object whose field the emitter reads — so the page IS the sink. ForPageRender models DisclosureSink (two scoped-enum reasons: ChargeStreamRefused / ChargeStreamTorn) and the emitter READS it: on a degraded render, the page ships with the full quintet, zero rows, and reason= naming why the render was lost — the release reader is told IN the answer, never by a stderr-only note that ships nothing. The reword discipline repeated on me: this fix's own comment named the counted emitter-API tokens in prose, drifting fixedbufsweep's pin AGAIN (344 -> 345 on this base) — reworded to name no counted token. Lesson recorded in the comment itself: prose that NAMES a counted token IS a counted line. Verified: selfcheckcheck PASS (pin 49 restored), fixedbufsweep PASS (344 held), truncvocabcheck PASS (the page contract arms), and the happy-path smoke unchanged. * fix(mcpmanifest): move the tools/list ceiling for explore's declared limit/offset — 46,600 -> 46,700, measured 46,677 B mcpmanifestcheck arms (1)/(1b): explore's limit/offset schema properties (+184 B, 46,493 -> 46,677) crossed the 46,600 B per-session ceiling and the CHANGELOG's pinned [Unreleased] manifest-growth claim. Rule 5's sanctioned case, applied exactly as the gate's own RE-ANCHORED precedents prescribe: a ceiling moves UP only for a DECLARED argument the contract obliges to carry a description, in the commit that lands it, with its bytes attributed in the gate header — never for prose. The attribution entry names the ask (issue #294 parity, PR #3), the two properties, the measured bytes, and the 23 B headroom. CHANGELOG's claim re-derived to the live measurement (46,677 B) so arm (1b) asserts the pair stays matched. Verified: mcpmanifestcheck ALL PASS (ceiling + claim arms). --------- Co-authored-by: usehoplite[bot] <288093033+usehoplite[bot]@users.noreply.github.com>
…dget+window; the page grows next= * test(paging twins): arms (6e)/(6f)/(G2) — MCP for/explore budget+window must serve the candidate page; the page must carry next= — red vs the pre-change binary Gate-first for the MCP-twins follow-up (issue #294). Measured reds, each the contract the implementation must green: - (6e) MCP for {budget_tokens, limit}: REFUSES today ('limit/offset select the file page, which has no token budget to shape against') — honest, but the CLI serves the candidate page; the twin must not diverge. - (6e) MCP explore {budget_tokens, limit}: serves the BUDGETLESS FILE PAGE with budget_tokens silently ignored — the accept-and-ignore class (W3FIX M4's exact hole); must serve the candidate page. - (G2) the candidate page root carries NO next= — the resumable argv handle the file page's next= contract serves (next_offset= alone is machine-only); the handle must name the declared continuation offset. All three red on this binary with green mutation controls; every pre-existing arm in both gates untouched. * feat(paging twins): MCP for/explore serve the candidate page under budget+window; the page grows next= — arms (6e)/(G2) green Completes the issue #294 follow-up: the MCP twins join the CLI on the budgeted-bundle candidate page, and the page carries the pasteable continuation handle. - forpage.h is now the page's ONE home: emitForCandidatePage moved from verbs_for.h and split — forCandidatePageDoc returns the document (the same bytes for every dialect), the CLI wrapper fwrite's it. The twins (mcpverbs.h forTaskText/packTaskText) build the SAME doc on their own lr, so CLI and MCP cannot serve a different page (the file's one-implementation rule, now true for both pages). - next=: the page root carries the pasteable argv handle (--for=TASK --token-budget=N [--limit=N] --offset=<next_offset>), the file page's own forPageInvocation shape via nextFlag/nextAttrXml, dropped past kNextAttrMaxBytes (a hint that pastes wrong is worse than no hint; the machine next_offset= always remains). est_tokens measures the final bytes including the handle (the two-pass loop already did). - MCP for: budget+window no longer refuses — it serves the candidate page (the refusal's carve note stays where it stood). - MCP explore: budget+window no longer serves the BUDGETLESS FILE PAGE with budget_tokens silently ignored — the accept-and-ignore class, arm (6e)'s recorded red, now the candidate page. Gate-first: arms (6e) x2 and (G2) landed red first (commit bd404d5), measured on the pre-change binary. Now green with the full sweep: 16/16 gates ALL PASS (mcpverbscheck, listingpagingcheck, truncvocabcheck, budgetpolicycheck, packtaskcheck, fixedbufsweep, selfcheckcheck, forwidencheck, forbudgetmonotoncheck, mcpmanifestcheck, mcpw3fixcheck, shapingflagcheck, pagingsweepcheck, modifierguardcheck, tokenbudgetcheck, fordisclosurecheck). The sink-form DISCLOSE sites moved with the function — selfcheckcheck's pin holds. No stanza text changed — the manifest ceiling is unmoved. Determinism: CLI and both MCP twins byte-identical across runs; xmllint clean on all three surfaces. * ci: re-run the matrix — two wall-clock arms flaked on the macos runners traceasanlinearcheck B1 (timing ratio 10x vs the 8x bound; passes locally at 2.7x with margin) and runtracecheck G2 ('the sleeps did not land') are wall-clock assertions, and neither surface is touched by this branch: parseAsan and --run-trace are untouched, main's recent CI runs on the same runners are green, and both gates pass locally. Empty commit to re-trigger the matrix — if the same arms fail again in a fresh run, this was not jitter and gets investigated for real. --------- Co-authored-by: usehoplite[bot] <288093033+usehoplite[bot]@users.noreply.github.com>
…=, the at= index stamp, and the end-to-end walk arm (#5) The maintainer ruling (redhat-et issue #294, comment 5868657455) blessed the design as-is and offered five optional suggestions. Two we already satisfied (#4 per-page trim disclosure — truncvocabcheck arm (H)'s within-page <bodies> rule-5 sweep; #3's canonical tie-break — radixSortByScoreDescId is the same (score desc, id asc) key the bundle selects with). Three land here: - above_cliff= (suggestion 1): the head-tier candidate count next to total=, so an agent decides from page 1 whether the below-cliff remainder is worth fetching. cliffRank is 1-based, so above_cliff = cliffRank-1 — the same boundary tier= uses, one definition. Also on the degrade page. - next_tier= (suggestion 2): the tier the CONTINUATION page will carry, beside next= — a consumer skips the below-cliff remainder deliberately, not by accident. Fixed literals; always present. - at= (suggestion 3): the git index stamp on every page (the same gitstamp::stampAt the un-paged bundle anchors with), so a stale continuation is detectable — the consumer compares at= across pages instead of silently mixing two rankings. Empty (non-git / multi-root) => honestly absent. - (G3) the end-to-end walk (suggestion 5): concatenating every page equals the full ranked candidate list — the pages' own shown= arithmetic tiles the candidate set (pseudo-symbols like <file-scope> are served but emit no row, so named rows <= shown), the (file, line, name) TRIPLE tiles the named-def rows in rank order (same-name definitions are distinct candidates even in one file — the packtask.h:1690 lesson, learned twice this arm), the tier sequence is head* then below-cliff* (one boundary, unmoved mid-walk), and the un-paged bundle's served rows are a PREFIX of the walk. Walked on src/: 5 pages, 946 rows tiling 947 candidates, head -> below-cliff x4. Arms extended in the same commits as the behavior: (H) asserts above_cliff= (< total) and at= on every paged root; (G2) asserts next_tier= is one of the two values and present beside next=. Verified: 11/11 gates ALL PASS (truncvocabcheck+H, listingpagingcheck+G2+G3, mcpverbscheck, fixedbufsweep, budgetpolicycheck, packtaskcheck, mcpmanifestcheck, selfcheckcheck, forwidencheck, forbudgetmonotoncheck); the page is byte-deterministic and xmllint-clean; the MCP twins carry the same document (above_cliff= live). Co-authored-by: usehoplite[bot] <288093033+usehoplite[bot]@users.noreply.github.com>
…edaction tally, budget enforcement + over_ceiling, pack-task flag guard, help text CodeRabbit's nine findings on PR #362, each verified against the live code first (one false alarm declined with the evidence); seven real, all fixed: - GLOBAL r= (forpage.h:193, Major): packSignatures gains a rankBase parameter (LAST, after cutOut; 0 = the un-windowed default, byte-identical) — the windowed serving passes win.begin, so a continuation page's rows carry their GLOBAL ranks (offset=198 now prints r=199/200/201, not r=1/2/3): the rank-adaptive doc/sig tiers key on global rank, the r=1 row's next= hint follows it, and merged pages have no duplicate r=. - next= names ITS verb (forpage.h:342, Major): forCandidatePageDoc takes verbName; --pack-task pages hand back --pack-task= continuations (pasting runs the same verb, the same ranking, the same cut). - Redaction tally (verbs_for.h:2228, Major): both page call paths report reportRedactions after serving, exactly like the --json early return — a redacted credential is never silent on the page. - --pack-task flag guard (verbs_for.h:3824, Major): the shapeFlags loop now covers the candidate page (both verbs, budget+window); --json/--compress/ --with-graph beside a --pack-task window refuse with the page named, instead of accepted-and-ignored XML at exit 0. - Budget enforcement + over_ceiling (forpage.h:371 + mcpverbs.h:1900): the assembly is rebuilt as buildHead/buildNextAttr/buildNextTierAttr/buildTail lambdas; a page that exceeds budget_tokens after the H1 ladder trims trailing whole rows (the handoff.h tail-first policy) and re-derives its own arithmetic (shown/next_offset/has_more/next=) from what shipped; a page that cannot fit even empty (a pathological task length) serves empty with over_ceiling=1 — the same marker the un-paged bundle uses — plus the stderr note. est_tokens two-pass convergence preserved per construction. - Help text (cli.h, Minor): the --limit/--offset paragraph names both pages (the file page; the budgeted-bundle candidate page = budget + window). - (6e) arm hardened (test/mcpverbscheck.sh): parses budget_tokens= and requires est_tokens <= budget_tokens or over_ceiling=1. BONUS FINDING the rank fix exposed (mine, fixed with it): when the H1 ladder trims within a page, next_offset= used to say win.end — silently SKIPPING the trimmed rows. next_offset is now always win.begin + shown (identical when nothing trimmed; gapless at any budget). Declined with evidence: the est-splice 'corrupts the next attribute' claim — the code replaces by SEARCH STRING (' est_tokens="0"'), not pos+count; a python replay and live output both show the splice correct. Verified: 13/13 gates ALL PASS (the 11-gate sweep + forrankordercheck + callsrankordercheck — the rank-order gates the r= fix could have moved); enforcement smoke: a 300-token budget trims to fit with shown/next_offset re-derived; every page shape xmllint-clean and byte-deterministic.
… tier boundary, the legend, and the true resume point The maintainer review (joyful-ii-V-I, 9 required + 7 optional). Part 1 — the renderer and its arms, each fix verified against the review's own repro commands first: 1. Malformed XML on a trimmed page + over_ceiling when the empty page fits: the trim now cuts at the START of the last row (a cut at its close left the row open); the one-row floor (kForPageSigsFloorBytes, named) keeps the top row whenever the window is non-empty, so a budget that cannot fit a row serves one row with over_ceiling=1 (repro budget 320: shown=1 est=966 over_ceiling=1; budget 300: shown=1 next=1 — forward-moving). 2. The duplicate seam: the resume point is now the last PRINTED row's candidate index + 1 (packSignatures' shownIdsOut; a pseudo-symbol uses a candidate slot without printing a row, so a served-count arithmetic lands one short and re-serves) — plus the pushShownSigId local-rank fix in serialize.h (under rankBase the global rank indexed order past the window into the masked tail: next_offset=2^32). (G4) the trimming walk (the review's own ask): 12 pages at budget 600, zero duplicates. 3. next= always moves forward; dropped with next_tier= when has_more=0 (a last page handing back a next= that returns the same empty page is a loop with no exit). 4. above_cliff= cliffRank (the head count itself — 'a cut at rank i keeps ranks [1,i]', lexical.h), and SINGLE-TIER windows: a head window ends AT the cliff, so tier= never labels below-cliff rows as head. 5. The legend: the page carries its own full legend comment (G4-safe, dashless verbs), and the compact dialect runs through the CENTRAL table (compactlegend.h: the candidate-page spec row + six vocabulary readings + the pack-task root-attr adjust, the expand/whole-file precedent; the CLI rule keyed on above_cliff above pack-task's; nativeCompactLegendVerb no longer skips the page; MCP for mapped) — so --legend=full|compact works through the same machinery as every other answer, and schema= rides the root (verified: schema=ripwire.candidate-page/v1, est repriced by the machinery). ref rides the twins' existing textResult path (legendRefPosture) — inherited, not re-implemented. 6. (partial) MCP explore's page branch now sits AFTER every ranking boost (the early position returned before applyDocMentionBoost and ranked a different candidate set). KNOWN RESIDUAL, flagged for the maintainers: the CLI pack-task page and the MCP explore page still disagree on the candidate denominator (907 vs 1161 on the same task) — the un-paged answers' kept heads MATCH (6=6), so this is a pre-existing divergence between computeLensRanking and packTaskText's inline pipeline that the page's positiveHits denominator exposes; an upstream ranking-parity question beyond the page, not introduced by it. 8. (partial) ForCandidatePageReq — the request struct (12 params -> 4); rw::emitRaw in the wrapper; kForPageSpliceReserve named; the degrade page built by the same lambdas so it cannot drift. Arms (each fixed WITH the behavior it pins, and hardened by the same review): (6e) parses the FULL text (over_ceiling sits past byte 400); (G) seam at a no-trim budget on src/ (the legend rides the budget, and the single-tier clamp means the repo-root corpus clamps every head window to the 1-row cliff — a correct page this identity cannot test); (G3) tiling under pseudo-symbols (shown-sum <= total; rows-extracted == shown-counted); (G4) new — the trimming walk; all extractions strip comments first (the central vocabulary DEFINES <d r=N> in prose — 12 false rows per walk). Verified: listingpagingcheck ALL PASS (G/G2/G3/G4), mcpverbscheck ALL PASS (6b/6c/6d/6e), truncvocabcheck, fixedbufsweep, selfcheckcheck PASS. The review's own repros re-run green (R1a well-formed, R1b one row + marker, R2 forward seam, R3a one-row floor, R3b no next= at the end, R4a above_cliff correct, R4b single-tier). Part 2 (next commit): the help lines (missing \n, the pack-task file-page wording, the signatures-only note) + COMMANDS.md regen + help_all re-pin; the runPackTaskPage helper (--quality-delta); the CHANGELOG ([0.6.2] restored, the [Unreleased] entry); the residual parity diagnosis.
…DS regen, the released-changelog restore, and the [Unreleased] entry - Help (item 7): the three added lines end with their \n again (one ran 336 columns); the wording corrected — --for takes --limit/--offset as its FILE page (bare --pack-task --limit REFUSES: it has no page without a budget), and --for/--pack-task beside --token-budget take the window as the CANDIDATE page, signatures only (bodies come via next= or --expand). - docs/COMMANDS.md regenerated (179 flags, 163 samples); help_all re-pinned with the both-emitters proof — the movement is identical on the std::print build and the std::format+fputs fallback build (printffmtparitycheck 42/42 ALL PASS on both). - CHANGELOG (item 9): the released [0.6.2] entry restored verbatim (the ceiling edit had landed in it); the [Unreleased] entry added for the candidate page — the full surface, the explore budget_tokens fix, the manifest figures (46,493 -> 46,677 B; the ceiling 46,600 -> 46,700 B for the one sanctioned case), referring to issue #294 / PR #362. - cli.h (optional): the emitTo call that passed 4 arguments for 3 placeholders — the 4th is gone. KNOWN RED, disclosed: showcasecapturecheck (I) — the capture's --legend-dict figure is stale (the live dictionary gained this round's 7 entries: 730 -> 737); the gate demands a regeneration and forbids hand-editing, but test/showcase_capture.py REFUSES to write — its --since=HEAD~N promise check for the --in=src case fails for every N in [3,14] ('the scoped <recent scope=> page itself (past the display cut: the global block is too wide)'), a refusal that reproduces independent of this round's changes and looks like an interaction with the #350 train's recent-scope display behavior; it needs its own investigation before the capture can be regenerated. Next round.
…, the in-doc compact dialect (ref posture + honest measure) The maintainer's round-2 re-review verified items 1-8 of the first list fixed by running them; four remained. Three land here: Item 1 (the parity ask, the honest minimum the maintainer named): the page now SAYS which ranking it paged — ranking="for-lens" (verbs_for.h computeLensRanking, the CLI --for/--pack-task pipeline) or ranking="mcp-lens" (the twins' shared inline pipeline, mcpverbs.h). Root cause, measured: the un-paged heads agree but the page denominators differ (CLI --pack-task 907 vs the twins 1161 on 'how does pagerank converge') — computeLensRanking and the twins' inline pipeline rank slightly different candidate sets. And the MCP pages no longer hand back a pasteable CLI next= (a --pack-task= argv would walk a different list): pasteHandle=false on the twins — the machine attributes (next_offset=/limit=/next_tier=) carry the programmatic continuation through the same verb; the CLI dialects keep the handle (same pipeline, pastes true). CHANGELOG's 'identical document' sentence corrected accordingly (the same page SHAPE; ranking= names the list). Item 3 (legend under ref on MCP): the page is compacted IN-DOC (compactLegend=true from every dialect that defaults compact) — the central layer (applyCompactDialect, candidate-page) writes schema= and reprices est_tokens=/settles over_ceiling= on the bytes that actually ship, so the outer layers see AlreadyCompact and the ref posture reduces the page like any other spec-keyed answer. VERIFIED LIVE: in a session that read ripwire://legend-dict, the for page takes the ref posture — the legend comment reduced, legend="ref" dict= dictv= in the trailer, every page attribute intact; the second identical call shrinks further (the session's served entries). The un-paged for answer is untouched. Item 4 (the stderr/page contradiction): the trim loop's stderr notice is GONE; the one notice left reads est_tokens off the FINAL emitted root and fires only when the final page carries over_ceiling="1" — the line can never contradict the document. The maintainer's own repro (src --for=rank symbols --token-budget=750 --limit=40 --offset=100) now: est_tokens=714, no over_ceiling, no stderr line. The in-doc compaction also fixes the second half (a compact page holding fewer rows than its budget allows): the trim loop measures the full dialect, the machinery reprices the compact dialect it emits. Verified: the full decisive sweep ALL PASS (truncvocabcheck, listingpaging [G/G2/G3/G4], mcpverbscheck [6b/6c/6d/6e], fixedbufsweep, budgetpolicycheck, selfcheckcheck, forbudgetmonotoncheck, mcpmanifestcheck). The one-row floor re-verified on the MCP twin (budget 600: shown=1 over_ceiling=1 — the honest no-fit answer, arm (6e) accepts it). REMAINING, disclosed: item 2 — the runPackTaskPage helper (--quality-delta still exits 2: runPackTask complexity 19->28, packTaskText params 6->8) is designed (ForCandidatePageReq landed; the helper extraction is the next commit) and the two optional asks (limit="0" on offset-only pages; the bare --pack-task --limit refusal naming the budgeted window) are noted.
…+ PackTaskForm, the quality gate's two findings closed --quality-delta=0e5fdcd2..HEAD exited 2 on exactly two gating findings; both are this round's shape, both closed by extraction/bundling (behavior byte-identical; the branch body moves, nothing changes): - runPackTask complexity 19 -> 28: the budgeted-window branch extracted into runPackTaskPage( d, lr, task, in ) — one helper, one job (refuse a partitioned window, serve the candidate page, report the redaction tally); runPackTask keeps the one-line gate. - packTaskText params 6 -> 8: the four optional form knobs (partition fan-out, route posture, and the page window) bundled into PackTaskForm — one aggregate struct, one parameter; the call site in mcp.h initializes it from the request's own fields.
…Not-run line, the compact-default pairs, the order-insensitive (H) read Three arms a fresh regeneration trips, none of them the paging change: - (D): --lsp and --sections were hand-added to the 2026-09-14 capture's 'Not run (and why)' line but never to the generator's own static list, so every regenerated capture silently dropped them and coverage went 174/176. Added to the generator, with the reasons the hand edit gave. - (F): compact became the DEFAULT legend posture on 2026-09-19 (bcb7969), after the 2026-09-14 capture was recorded. The three adjacent +--legend=compact variant demos (--callers, --quality-delta, --edit-check) now contrast nothing against their bare partners — the pair is byte-identical. Flipped to --legend=full, which IS the contrast now, captions rewritten to say so. - (H): the <at> root gained schema= before p= in the compact dialect; the arm's p-first grep read 'no sym= at all' from any regenerated capture. Extract the whole tag, then read sym= out of it.
Generated by test/showcase_capture.py on a clean tree, 255 commands, with this commit's generator fixes in place (--lsp/--sections on the Not-run line, the --legend=full contrast pairs, the order-insensitive (H) read in the gate). --legend-dict now records dictv=861226192676dedb entries=733, matching the live binary, so showcasecapturecheck (I) passes instead of tripping the stale 09-14 figure.
docscommandscheck (G) requires the doc to match a regeneration from the newest capture; it was still built from the 09-14 one. 163 samples now come from the new capture; no flag-surface change.
…t on the rebased tree Upstream main's 181 commits moved the seed line (graph.h grew), changed the legend dict (entries 733 -> 736) and grew the flag surface to 181 (--max-memory, --mcp-tools). The capture is regenerated against the rebased binary with the Not-run list extended for both new flags; the manifest is re-pinned (UPDATE_GOLDEN, help_all moved with the new help) and COMMANDS.md rebuilt. The generator patch is committed here so the capture is reproducible from this tree.
The rebase onto 40c83e4 (train 23) carries new --help lines; help_all, help and one more verb re-pinned via UPDATE_GOLDEN from the merged binary.
…ase) Re-capture the showcase against 2026-10-03's merged tree (new whereis/ edit-check help lines moved the legend dict), regenerate docs/COMMANDS.md (181 flags), the deck pptx/pdf, and restate the paging entry's tools/list figure honestly: 46,581 -> 46,765 B measured live on both trees, ceiling 46,700 -> 46,800. help_all re-pinned for the new --help lines.
…ase) Re-capture the showcase against 2026-10-07's merged tree, regenerate docs/COMMANDS.md, the deck pptx/pdf, and restate both [Unreleased] tools/list figures to the live merged-tree measurement: the paging schema takes it 46,732 -> 46,916 B, ceiling 46,750 -> 46,950 (the valueRefs entry re-pinned to the same live figure, per the gate's mutable-[Unreleased] contract). help_all and four more pins re-pinned for the train-25 help lines.
…arserVer 126 to 144
…e page paths; the ranking flags ride next= B1: runPackTask computed its ranking with the H2 MaxScore pruning on, so the CLI --pack-task page read past the top-K the pruning is exact for — total= depended on a performance optimisation (measured on src/: 931 vs 1232 candidates, and 301/522 candidates unreachable by walking). A window now forces fullDistribution, the rule --for's bundle path already followed; the un-paged bundle keeps the pruning. All five page paths page ONE list, so: ranking= is dropped (nothing left to distinguish), the MCP twins carry the same pasteable next=/next_tier= as the CLI (pasteHandle=true), and the CHANGELOG/forpage/mcpverbs text about the two pipelines is corrected. B2: every accepted flag that changes the ranking now rides next= — --no-route, --no-mention-boost, --no-doc-mention, --cochange-boost on the CLI (the bundle-shaping set is refused beside a window instead), no_route on the MCP twins. A pasted handle re-ranks identically; the measured break (--no-route page 1 total=563, its flagless next= pasted: total=1, has_more=0) cannot recur. Gate: pagingsweepcheck gains the B1 arm (all five surfaces walked to has_more=0 with pruning in its default state, identical row sequences, r=1..total) and the B2 arms (each flag's page-1 next= pasted verbatim must equal page 2 of the same ranking; MCP for no_route's handle pasted onto the same list).
…he derived artifacts dropped B3: docs/limits_build.py re-run — capped='s disclosure moves two lines of the generated table. limitstablecheck passes. B4: the showcase capture, the deck and the wholesale COMMANDS regen are dropped, restored to main's train-25 state. Our capture was recorded on a dirty local tree, was stale on the integration branch (its --at seed resolved to the wrong symbol there), and published local branch names. Maintainers regenerate captures and the deck at merge time; the feature itself needs only the 2 help-derived lines docs_commands_build.py adds (--limit under Shaped by, the HONORED-by list), which is all this keeps. The recorder's caption edits and the showcasecapturecheck (H) grep fix stay. Known consequence, by design of this ask: the (I) dictv arm is red on this head until the merge-time regen — the in-tree capture is main's, whose legend dict predates the paging specs.
…ated; (1b) reads the last [Unreleased] figure The +184 B is explore's limit/offset alone (774 -> 958 B): for already declared the pair (777 B, unchanged), pack_task is a dispatch alias, not a listed tool, and next_tier= is a response attribute, not a schema property — both sentences (CHANGELOG and the gate's ceiling note) now say exactly that. The REFVAL entry's Manifest line is back verbatim (46,591 -> 46,732 B); arm (1b) now reads the LAST claim in [Unreleased] — several entries restate the figure and the last one describes the live manifest, so ours (46,732 -> 46,916 B) is the checked number.
The doc post-pass ran `markitdown <path>` through /bin/sh for every pdf/docx/pptx/xlsx on every run. On a machine without markitdown the shell answers rc 127, the bridge reads that as "", and the empty result is (deliberately) never cached, so every warm call paid one shell start per binary doc for an answer known before the fork: ~25 ms of a warm call on this repository (two present/ decks). docparse::markitdownMayRun() now asks the shell's own question without a process: os::which walks PATH as sh does (an empty entry is the current directory; a directory or a non-executable file is not a program). It answers true whenever it cannot rule the shell out (PATH unset, an executable ./markitdown, an exported shell function), since a false "absent" would drop a Section while a false "present" only costs the popen it always cost. It is asked once per post-pass and never memoized across ingests, so a markitdown installed under a running MCP server is seen on its next ingest. The content-hash cache is still read first, so a blob extracted while markitdown was installed keeps answering after it is removed. Output is unchanged. test/docmdcachecheck.sh arms 6-9: absent then installed between runs, a directory ahead on PATH, a non-executable file, an empty PATH entry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… one scan per blob; refuse an unread blob stream --whereis on a checkout with 137 local branches took 12-14 s against 2.3 s on one ref. Most of it was reading: every `git ls-tree -r` (one per branch) came back through popenTrimmed one locked fgetc per byte, 55% of the main thread. Three changes, output byte-identical (all 337,743 rows of a --limit=400000 listing compared, base vs head): - popenTrimmed reads the stream in 64 KB blocks. The answer is every byte the command wrote, then the same trailing trim: identical to the line reader it replaces, and every git one-liner in the tool reads through it. - computeWhereis lists each DISTINCT tip once, across the crossref git pool (parallelIndexed), and assembles the sites in ref order as the serial loop did. A listing whose worker never ran forfeits complete= like any other suspect tree. - A blob is line-scanned once however many (ref, path) sites point at it; its rows are stamped per site in site order. Measured on the 139-head checkout (dev builds): 12.1 s -> 3.7 s. Honesty fix (cli-floor §4): the one `git cat-file --batch` takes its sha list from a file under the cache dir. With that dir unusable (TMPDIR naming a missing directory) the stream never started and the answer printed on-head="0" hits="0" with a near-miss note retrying the same name, on a repository whose HEAD defines it. The diagnosis attributed it to a fresh one-commit repository; the trigger is the unusable cache dir. WhereResult::blobsUnread now records a stream that never started or stopped early, and both hosts refuse: the CLI exits 1 naming the cause, the MCP tool answers -32603. test/crossrefdegradecheck.sh arm 6: a control, the refusal (red on the base binary: exit 0 with hits="0"), a genuine zero still answered (negative), and the MCP twin (red on base). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eByName for names with no function-local def The resolve loop went 22 -> 79 ms at 24bb6e5 (built in isolation with the profile flavour, snap, median of 3). That commit gave every module-scope call an owner, and the newly owned calls are the intended consequence, not a defect: an instrumented build shows the +7,563 unresolved sites are module-scope Bash (7,327) and Python (208) calls to names whose every in-repo definition is another language's (`head`, `git`, `mkdir`, `set`, `join` ...), counted at the language-filter site exactly like the same calls inside a function; every reference that had an owner before keeps its edges (the commit's own verification). The cost is where those calls land: the test scripts' top-level `ok "..."` / `no "..."` lines, ~28k calls to names with ~650 same-named shell definitions each. Per call the loop walked that whole list several times, each step reading one field of a Symbol at a random id. Timed per disposition, module-scope Bound calls cost 1.9 us each against 0.55 us for the rest. - symLang / symKind (beside the existing symFileId) are dense copies of the fields those walks read: the language filter, the namespace gate, the same-file and same-directory tiers, Rule 2d and Rule 3's filter. - nameHasFnLocalDef holds the byName entries with a function-local definition. A name with none cannot yield (localDefOutOfReach is false for every candidate, and every list handed to reachableByName is a subset of the name's list), so that walk is skipped with the answer it would have returned. Resolve loop on snap, profile flavour, median of 5: 128 -> 77 ms. The --pin-census of every call site is byte-identical base vs head, as are the map, --callers, --callees, --impact and --for on snap. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ous call's answer, checked before the parse Every call on a git checkout ran `git ls-files --others --ignored --exclude-standard --directory` before the crawl could start: ~60 ms of a ~400 ms warm call on this repository, most of it git walking the tree the crawl walks next. A cache keyed on the files that hold ignore rules would not be exact: the answer also changes when the untracked files change (a new x.log under an existing *.log rule, a file added to a directory git reported whole because everything in it was ignored). So nothing is keyed. The crawl now walks with the answer the PREVIOUS call got for this root (a sidecar under the cache dir, header naming the root) while git computes this call's answer on a worker (std::async), and ingest() compares the two raw -z streams byte for byte before the parse pool. Equal: the crawl that ran is the crawl git's fresh answer gives. Different: the crawl, the cache load and the scan arrays run again on the fresh answer (the crawl's memory stop re-armed and re-recorded with it), and the sidecar is replaced. Every answer is git's own answer for this call; only the wait moved. collectGitIgnored is split into runGitIgnoreProbe (the process, classified as before: non-zero exit outranks overflow) and parseGitIgnoreProbe (every DISCLOSE where it was, on the thread that applies the answer). Spawn conditions are unchanged: same point of the crawl, no child without a .git above the root, none under --no-ignore; under --no-cache there is no sidecar and the probe runs inline. test/gitignorecheck.sh arm 14 (every earlier arm runs --no-cache and so never speculated): 14a the sidecar exists (red on the base binary); 14b-14g a cached run after each change equals a --no-cache run and shows the change: an edited .gitignore, a new nested .gitignore, .git/info/exclude, the default global excludes file, core.excludesFile's file, and a new untracked file an old rule ignores; 14h repeat runs; 14i a lost speculation under a crawl memory stop. The census gates follow the renamed popen site (crashsweepcheck) and the sidecar name buffer (fixedbufsweep). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ged tree At 850 the tiny fixture is the 4-row floor with its next= unpaid, labelled at est_tokens=967 in the full legend (lego-charge's charge + #362's tail next=; red on two macOS shards of CI run 38041394072 and locally). Swept 900..1200: 1000 serves six rows at 940 in the full legend (60 tokens of headroom, the dialect this arm measures), 989 at the default. Assertions unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…n-memory blobs The tags path over blobs read from other git refs (for --whereis): the crawl's language rule with the .h -> ObjC sniff, the size/binary/nesting refusals, the member-macro re-parse, the #if 0 ranges, captureTagsFacts, the markdown sections, then the model build's dedupRawDefs / collapseObjCDeclDefs / partitionFieldDefs, so a blob answers what HEAD's index answers for the same bytes. Parallel over hardware_concurrency(), biggest blob first, one slot per blob; the tags queries a batch needs are installed single-threaded before any worker (callers in a long-lived server hold headSnapshotIngestMutex, like every other ingest). captureTagsFacts gains TagsCapture::DefinitionsOnly: the same definitions without the reference arm, import emission, complexityOf, the function-scope span, the IS-A / HAS-A / macro-body reference minting and Ruby's reference and bind passes (~60% of its samples on 104 C++ header blobs). The Full posture, the index's, is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d discloses what it leaves; refs read from tree objects Round 1 labelled a non-HEAD row with a stricter lexical reading of its line, which lost real definitions (#define NAME(, decltype(auto) NAME(, @test(...) public void NAME(), #[derive(Debug)] struct NAME) while the looser base reading kept calls (go f(a), @f(1), f( a, [&]( int x ) {). Now (labelBranchRowsByParse) each distinct (blob, path) behind such rows is labelled the way HEAD's rows are: HEAD's own blob at the same path takes HEAD's index labels; any other blob holding a parse-worthy line is parsed (definitionLinesInBlobs) and each definition of the name promotes the nearest row in relabelHeadHitsFromIndex's window. The lexical test only chooses blobs (parseWorthyLine, a looser superset of definitionShaped, which is restored to its 93aeab1 form for HEAD's lexical fallback). A runaway guard (256 blobs / 64 MB, most rows first; --detail=N lifts it) bounds the parse; a blob it leaves or the parser cannot finish confirms nothing: its parse-worthy rows read kind="text" (listed and folded on the defs page, never def) and <unparsed blobs= rows= next=> counts them. ref_labels="parsed" rides any answer with another ref's row; the legend, compact readings, --help, the MCP description (701 characters kept) and the change-check skill say so. The "shown + more + folded + refs = hits" sentence now says "on the first page". Speed: every ref's tree comes from content-addressed tree objects, one git cat-file --batch per tree level (listTreesOfRefs, ls-tree's own order and quoting; a root below the work-tree top keeps ls-tree), the blob stream is read through four cat-file processes (streamBlobsInSlices) with each blob scanned once on its worker. --whereis=escapeXml on this repo (162 refs): 7.8-9.2 s at load 24-29, base 93aeab1 ~40-60 s at load 23-30. Arms: crossrefcheck 5c (a well-formed file per language family, branch == marker == HEAD's index on every marked line; the C1 shapes; the branch-only #define / annotated Java method; ref_labels; CLI = MCP), 5d (the guard), 5e (tree-walk spelling against ls-tree); 5b and the B8.2 twin restated with their reasons (rule 4); selectorhonestycheck A7's doc residual restated; rv nits N1/N2/N3. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…chooser each split into named steps No output byte moves (row sets of four symbols on this repo equal 93aeab1's; crossrefcheck 111/111). computeWhereis keeps the order of operations and hands each one to a helper (whereisSitesOf, scanWhereisBlobs, stampWhereisBlob, noteBranchRows); listTreesOfRefs reads root trees, tree levels and one ref's listing in three functions, with a checked cat-file header parser; labelBranchRowsByParse's mirror pass and status switch, parseWorthyLine's two extra tests, definitionLinesInBlobs' query install and worker, and the blob's tags step each get their own function. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…; the --help=all pin for the whereis help sentences docs/LIMITS.md regenerated (python3 docs/limits_build.py: 241 caps, 7 parameters), README's cap sentence follows, and test/printf_parity.manifest pins the --help=all bytes the --whereis / --whereis-listing help changes produce (an intended text change, not a printf conversion). docs/COMMANDS.md and the showcase capture's --legend-dict figure are the train's one regeneration from the merged binary, as for every lane. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicts: CHANGELOG.md (union: the train's entries, then the lane's K75/K51/K50 entries); src/quality.h gitRawCommitStream (both guards kept: a walk K51 marks unread, and a walk the memory guard abandoned, are returned uncached through one branch). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…read blob batch in one sentence crossref::blobsUnreadCause( noun ) is the one renderer for both blob readers (the trees' blobs, the branches' blobs): the --whereis CLI refusal now asks about TMPDIR and its MCP twin names the cache directory, as the cross-branch refusal always did. crossrefdegradecheck (6e) compares the two sentences, CLI and MCP. The cli-perf-068 CHANGELOG entry no longer says the cross-branch verbs answer without saying the read failed (honesty-small-b's K75 fixed that). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…uild cached is not served A binary before K51 stored a git log walk that git could not finish (an empty stream or the newest prefix) under the same (repo, HEAD, window) key, and a K51 binary on scheme 2 served that blob warm: no walk, no churn=, no history_unread= until HEAD moved. The bump makes every such blob a clean miss (one cold walk per repository). qchurncheck "K51 scheme" arm: a blob written by RIPWIRE_BASE_BIN (a pre-K51 binary) through a git shim whose walk dies is not served; skipped by name without one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… listing rides, the MCP kind filter does not The next= named --stray-content= for an MCP kind filter, a different verb that takes precedence over --whereis, and dropped an explicit --whereis-listing=, so it replayed the default listing. It now carries the listing the question named and, like <refs next=>, no ref filter (the CLI has no whereis ref filter); the UNPARSED legend clause and the CHANGELOG say so. crossrefcheck 5d: the next= of a listing=all answer, run as written, is byte for byte the listing=all --detail=1 page; the default page's next= names no listing; an MCP kind= answer's next= runs whereis with the guard lifted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s wider misses and an ill-formed blob The loose lexical test that chooses which blobs to parse also misses a definition with no parameter list and no declaration keyword before the name (`name: function (`, Rust `pub const NAME`, a C# property); alone in its blob such a definition reads kind="ref". And a parsed blob is labelled as HEAD's index labels the same bytes, so a file its grammar cannot parse cleanly can lose a definition or show a false one exactly as HEAD's index would. Both are now named in the CHANGELOG floor paragraph, the full legend and parseWorthyLine's header. Wording only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ompared, not the first stderr line Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ormat), not a concatenation chain The concatenation shape read as a 24-26 token clone of four short string builders elsewhere (--quality-delta duplication, gating). Same bytes: crossrefdegradecheck 6e compares them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r-misses and a parse that cannot finish
The parsed-labels fixture gains eight well-formed files, each holding a true definition of probeName beside a call
shape: a Python decorator call, a Java annotation with arguments, `#define WRAP( x ) probeName( x )`, a call inside
an `if(` that opens a block, a C++ call whose lambda argument opens a block, a doc-comment `@see`, a base class,
JS `probeName(function () {` / `probeName('y', () => {` and `export default probeName(3)`, a TS decorator call
and a Ruby parameter. Each definition reads def and each call ref, on the branch and on HEAD's index alike; the
call lines carry no marker so the shape is bare. Ten of them read def on 93aeab1 and on the lexical-promotion
mutant. A new arm holds the "parser could not finish" claim: a branch .astro with an unclosed frontmatter fence
reads kind="text" with <unparsed blobs="1" rows="1"/> and no next=, while its closed twin reads def.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nder every listing, the default page included Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n enum), not a noun string 211125b's one-statement std::format body still read as a 21-token near-clone of os_win32.cpp's Unique::release (--quality-delta duplication + new-clone-of-reused-helper, gating 2 vs the merge base). The renderer now picks the noun itself from UnreadBlobBatch { Branches, Trees }, so its callers pass a typed choice and the body is its own shape (gating 0). Same bytes: crossrefdegradecheck 6e compares the CLI and MCP sentences of both verbs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicts and resolutions: - src/crossref.h computeWhereis: the lane's whereisSitesOf / scanWhereisBlobs / stampWhereisBlob structure, with cli-perf-068's two halves kept: its distinct-tip listing (listDistinctTips) is the fallback when the tree-object walk cannot answer (a root below the work-tree top), and its blobsUnread refusal is set after the blob scan (returning before the parse batch: the answer is refused). cli-perf's stampBlobRows is gone: scanWhereisBlobs scans each blob once. - src/ingest_sidecap.h: #339's captureRubyDefs, then the lane's DefinitionsOnly early return. - CHANGELOG: both blocks (train first); the lane's "Faster" sentence says what it replaces on the train (one ls-tree per distinct tip, kept as the fallback). - README cap count, docs/LIMITS.md, test/printf_parity.manifest help_all: the train's side here; regenerated from the merged tree/binary in the next commits. - test/mcpmanifestcheck.sh: both notes, the train's ceiling 47,150 (re-measured next). kParserVer stays 158 (the lane carries 156). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the merged tree python3 docs/limits_build.py (+1 cap: whereis's kWhereisParseMaxBlobs, the same row the lane generated); python3 bench/capsweep/capsweep.py emit leaves TUNING.md unchanged; README's cap sentence set to the printed count (244 -> 245). limitstablecheck rc=0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… B) on the merged tree
From the merged binary (built_from=65d62fe5): printffmtparitycheck UPDATE_GOLDEN moved exactly
{help_all} (whereis-defs-fix's --whereis help sentences: parser labels, kind="text", folded=).
mcpmanifestcheck: 47,109 B, ceiling 47,150 unmoved; per-tool JSON bytes before/after the merge
differ only in whereis (1,689 -> 1,684, the lane's -5). CHANGELOG: the whereis entry states the
merged figure, and #362's restated figure (the one (1b) reads) is 46,930 -> 47,109 with both
deltas attributed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…case capture, COMMANDS.md, dictv From build/ripwire at 65d62fe (every 26c lane merged; src unchanged since): - docs/captures/COMMANDS_showcase_2026-10-10.md captured (test/showcase_capture.py, 261 commands): it covers --no-inventory, re-derives the --at seed (rankGraphTeleport's line after #339 and cli-perf moved graph.h) and records dictv=74f30d013d02c2a2 entries=829. - docs/COMMANDS.md rebuilt from it (docs/docs_commands_build.py: 185 flags, 168 samples). - CHANGELOG versions paragraph: the session legend dictionary is dictv=74f30d013d02c2a2 entries=829 (`ripwire --legend-dict`). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… joined 5f (#339 x the parsed branch labels): a column a branch adds to db/schema.rb reads kind="def", as HEAD's index reads its own (the Ruby def walk runs before the DefinitionsOnly return in ingest_sidecap.h). Premise: HEAD's index labels its own schema column a def. RED on a mutant with the walk behind the return (the column reads kind="ref"). 5g (cli-perf-068 x the tree-object walk): a root below the work-tree top falls back to `git ls-tree`, one per DISTINCT tip — HEAD + 2 refs on 2 tips = 2 listings, all 8 rows. RED on the lane's per-ref fallback (3 listings, same rows). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…apturecheck (D)) map-inventory's --no-inventory had no capture case, so the regenerated capture still left it uncovered. The case pairs it with the --top-k=5 map above it: that cap leaves files out, so the default map carries <inv>, and --no-inventory drops the tier with the ranked rows unchanged (checked on the merged binary: the four <f> groups are identical). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…es) and the deck's count The same regeneration as b6f7ef3, once more after the capture gained `--top-k=5 --no-inventory` (262 commands): showcasecapturecheck rc=0 ((D) 0 uncovered, (H) the seed resolves, (I) the dictv the binary serves). docs/COMMANDS.md rebuilt from it: 185 flags, 169 samples; the deck's recorded-invocation count 168 -> 169 (deckclaimcheck, deckcheck rc=0). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… it checks, not an audit coordinate ripwirepubliccheck arm 3 (no audit-round coordinates in shipped docs) was red on the merged tree on the whereis-defs-fix entry's "(§A7)" — red on the lane head 574f758 too (lane-caused; the lane ran no CI). Green after. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… promise may call selfcheckcheck (C) was red on the merged tree: whereis-defs-fix's labelJobRows EXPECTS calls h.inCheckout() (`fromWorktree || ref == "HEAD"`, const noexcept), which is not on the accessor list — red on the lane head 574f758 too (lane-caused; the lane ran no CI). ALLOW gains it with its one-line reason, the gate's own mechanism for a pure callee (as `any` was added). Green after. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ead by every path crashsweepcheck (S2) was red on the merged tree: whereis-defs-fix's crossref.h catFileBatch (the tree-object walk's one `git cat-file --batch`) opens its sha list with fopen and the batch with popen, unregistered — red on the lane head 574f758 too (lane-caused; the lane ran no CI). Read: the list's only early return is the failed open, and it is fclosed after the write; the pipe's only early return is a failed popen, the read loop leaves by break or its own condition, then pclose and unlink — the same shape as streamBlobs' two registered rows. Registered with those reasons. Green after. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…h read on every path CI run 38075402771 @ c088780: hazardpatterncheck red on 6 shards (B1, C, E) — red on the lane head 574f758 too (lane-caused; the lane ran no CI). Each site read: - B1 definitionLinesWorker's catch: the sink form DISCLOSE( ( BlobDefsSlot{ slot } ), WorkerThrew ) marks the blob Failed in every build; the parenthesized sink hides it from the RECORDS pattern. - C listTreeRows .at x2 / listTreesOfRefs .at: readTreeLevels and rootTreesOf return a table only when catFileBatch served every id, and each served id is stored, so every lookup is a key. - E definitionLinesWorker's ts_query_cursor_new: deleted on both exits of the noexcept worker. Registered with those reasons (no FINDING: none drops an answer). rc=0 after. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
|
@coderabbitai review |
Collaborator
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lanes in this train
--for: the function a question names gets the first hop row, with all its callee names.--whereison a many-branch checkout about 70% less;--whereisrefuses instead of printing a zero it never measured.@import.pathcapture instead of per-language extractors. Answers are byte-identical.ripwire wraprecommends--forat its default budget (the old recipe's 2,000-token budget dropped needed rows).--expand/--outlineship no ride-along map when every definition they name is served.--for: the<sigs>budget is charged for the<lego>/<compose>stub it serves, not the full render.--help-taskprompt no longer trips the sanitizer build.--whereislabels another ref's rowkind="def"only when a parser says so (a lexical rule had labelled call sites as definitions); a line several refs share is one row; refs are read from tree objects. Floors and the parse guard are disclosed withnext=.--for/--pack-taskunder a budget take a resumable--limit/--offsetwindow; MCPexploregainslimit/offset(issue #294).db/schema.rbrenders its columns as Section definitions.Also: the five deferred CodeRabbit Minors from #383 plus its MCP
greptyped-argument sibling (each with a red-first arm); README orient text ("orient once, then lean lookups") and methodology notes.Pins:
kParserVer158,kCacheVersion29,kQSnapCacheScheme19 (test/qschemetrip.hashre-derived once). Generated docs (LIMITS — 245 caps, TUNING, COMMANDS — 169 samples, showcase capture, dictionary) are regenerated once from the merged binary;tools/list47,109 B (ceiling 47,150).Checked: every lane and both community PRs were reviewed and signed before merging into the train; a pre-review pass over the conflict-resolved and new hunks (one finding, fixed). Byte identity against the phase-A head 51/51 outside the late merges. The last
workflow_dispatchrun was green excepthazardpatterncheck, fixed in the head commit — this PR's own CI run is the first on that head.🤖 Generated with Claude Code