diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 5e5b54b7f..242a14c91 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -507,9 +507,12 @@ rules it enforces: bare-name ladder, so #349's Clojure `str/join` binds an in-tree `join`. A shared fix is planned (slice 6a). Until it lands, this rule does **not** mean a qualified call in a new language is free of wrong edges. Pin the wrong edge as a known-gap arm (see the definition of done below), and do not add a guard for your language alone. -- **Known gap.** Tags-pass predicates (`#eq?`/`#any-of?`) do not run yet, so keyword-headed definitions - (Elixir today; a Lisp-style `defn` if one is added) keep a small keyword table in C++. That table belongs in `tags.scm` once the - mechanism lands (#355, slice 5a); it is not a pattern to extend. +- **Keyword heads are data.** The tags pass evaluates the string predicates `#eq?`, `#not-eq?`, `#any-of?` and + `#not-any-of?` (`src/infra/tspredicates.h`, #355; never `#match?`), so a keyword-headed definition keeps its keyword list in its + `tags.scm`: Elixir's `def`/`defp`/… is the `#any-of?` list on `@_keyword`, and C++ that must recognise the same head outside a match + reads that list back from the compiled query (`predicateStrings`) instead of keeping a copy. Elixir's metadata attributes + (`@doc`, `@spec`, …) are the `#not-any-of?` list on `@_attribute` the same way. `test/tagspredicatecheck.sh` holds these rules. + Clojure's head tables move the same way (#389); `macroBodyKeyword` (`ingest_relations.h`) is still C++. #### Where language-specific code lives @@ -594,7 +597,7 @@ review will credit it, but it is never a condition for merging the language. | 2. Aliases | None. Renames reach the resolver as `RawBind` records (`ingest_cache.h`) from separate producers: Python's `@import.names` / `@import.alias` captures, read in `captureTagsFacts` (`ingest_sidecap.h`) and turned into binds by the Python-only `emitCapturedPythonImportBind` (`ingest_importcap.h`); `captureJsImportFacts`, `captureJsModuleAliases` and `captureGoImportFacts` (`ingest_jsimports.h`), C/C++ aliases in `fnPtrAliasName` (`ingest_binds.h`) and `captureTypeAlias` (`ingest_relations.h`). The readers are separate too: `addTypeAliasBases` and `buildUsingReexports` (`resolve.h`). | duplicated: one "local name → target" concept in about five shapes. | **2a** one shared binds channel (`@import.alias` / `@import.names` → one `RawBind` kind): M, *maintainers*, after the #358 import slices. **2b** after 2a, one language per PR moves its renames onto it: S each (Rust `use a as b`, **wanted**; Kotlin `import x as y`, PHP `use … as`, C# `using X = …`). | | 3. Types | `captureTagsFacts` (`ingest_sidecap.h`) reads `@definition.*` captures for every language. | shared, but language decisions are `Lang::` branches inside the loop: 36 non-comment `Lang::` mentions, 15 of them Elixir. | **3a** #374: one constexpr per-language hook table that `captureTagsFacts` calls once per capture, one language per commit, Elixir first. M. | | 4. Inheritance | `captureBases` (`ingest_relations.h`): base clauses for C++, TS/JS, Java, Python, Swift, C#, PHP, Kotlin and Ruby, chosen by node kind (`isBaseTypeNodeIn`). | partly. Rust `impl … for` is a separate pass; Ruby mixins read the same path through `@reference.mixin` captures since slice 4a (#400). | **4a** done (#400): Ruby `include`/`extend`/`prepend` read the shared path through `queries/ruby/tags.scm` captures, `captureRubyMixinBases` retired. **4b** the base-clause node kinds move from `isBaseTypeNodeIn` into `tags.scm` captures: M, all of the languages above. **4c** structural conformance (#386): a class or type whose methods cover a structural base's required methods (Python `typing.Protocol`, Go interfaces, possibly TS interfaces) is listed as an implementor marked `structural=1`, never as a nominal subclass, with a disclosed minimum-method threshold; step (c), one shared method-set check, per-language data for which bases are structural. M; Python and Go first (Go **wanted**). | -| 5. Callables | Definitions come from `tags.scm`, read by `captureTagsFacts`. | partly. Keyword-headed definitions keep keyword tables in C++: `elixirFunctionKeyword` and `elixirMetadataAttribute` (`ingest_elixir.h`) repeat `macroBodyKeyword` (`ingest_relations.h`), 0.82 and 0.96 similar by `--clones`. | **5a** #355: the tags pass evaluates `#eq?`/`#any-of?`/`#not-any-of?`, then Elixir's table moves into its `tags.scm`. M; Elixir, then Clojure. | +| 5. Callables | Definitions come from `tags.scm`, read by `captureTagsFacts`. | partly. The tags pass evaluates `#eq?`/`#not-eq?`/`#any-of?`/`#not-any-of?` and Elixir's definition keywords and metadata attribute names are `tags.scm` data (#355). Still C++: `macroBodyKeyword` (`ingest_relations.h`) and Clojure's head tables. | **5a** #355 landed the evaluator and Elixir's two tables; Clojure's tables next (#389). M. | | 6. Dispatch | `buildGraph` (`graph.h`): a SCIP pin, then the canonical tier (`appendCanonicalCandidates`, `resolve.h`), bindings and receiver rules (`receiverevidence.h`), then name tiers. Per-language qualified-call rules: `ElixirResolver` (`elixir_resolve.h`), `keepRustQualifiedCandidates` and `keepStdQualifiedCandidates` (`graph.h`). | partly. Facts about each language are scattered `l == Lang::X \|\| …` predicates (`implicitReceiverLang`, `memberNeverReachesFreeFunction` and `classNameReceiverLang` in `receiverevidence.h`; `langUsesHashComment` in `quality.h`), and `--clones` reports them as one group. | **6a** the shared refusal for an unplaced module qualifier (the known gap above): S, *maintainers*, not yet scheduled. **6b** the **language-neutral qualified-call resolver**: generalise `ElixirResolver`'s module table (alias/import/require facts → `Module::name`) so that Clojure `ns/f`, Elixir and other module-qualified languages use one resolver driven by per-language data. M; Elixir byte-identical, Clojure as the first new user. **6c** those predicates become columns of one constexpr per-`Lang` table: S, all languages. | | 7. Data-driven dispatch | One mechanism per shape: fn-pointer and callback bindings `buildFnPtrBindTables` (`graph.h`, C/C++); HTTP routes `routesVisitNode` (`ingest_sidecap.h`, a JS/TS and Python walk) into `RouteDef`/`RouteUse` (`model.h`); FFI aliases `BindingAlias` (`model.h`). | partly. Calls by a literal method name (`send(:m)`) resolve for Ruby only. | **7a** #375: one capture for a call whose method name is a literal, resolved through the ordinary receiver rules. M; Python `getattr`, JS/TS `obj["m"]`, Go `MethodByName` (**wanted**). **7b** route registration read from captures instead of the walk: M; JS/TS and Python. | | 8. Preprocessing | `retagMacroCallReferences` (`model.h`) for macro calls. Inert regions (quote, discard, comment forms): `inInertRegion` (`ingest_inert.h`) for Elixir's quote and live holes; Clojure's `clojureInert` (`ingest_clojure.h`) is still per language. | partly. The shared inert-region walk `inInertRegion` (`ingest_inert.h`) landed with #380 (#357); Elixir's quote and live holes use it, Clojure's filter does not yet. | **8a** Clojure's quote/discard filter (`clojureInert`, `ingest_clojure.h`) moves onto `inInertRegion`: S, Clojure. Ready now that #380 and #349 are both on `main`. | diff --git a/README.md b/README.md index a38725826..63ac4f1a0 100644 --- a/README.md +++ b/README.md @@ -2120,9 +2120,9 @@ wrong, and it has. These are the results that say so, all in-tree, all published ### In the tests
-701 gate scripts, five contracts no unit test can hold, and the house rule: write the gate before the code it measures +702 gate scripts, five contracts no unit test can hold, and the house rule: write the gate before the code it measures -`test/regression.sh` names **701 gate scripts** and is the authoritative list; +`test/regression.sh` names **702 gate scripts** and is the authoritative list; `python3 test/pargates.py . ./build/ripwire -j 6` runs the same set in parallel. On top of them sit the contracts that do not fit a unit test: two runs byte-identical, warm output identical to cold, output that pipes clean through `xmllint --noout`, a sanitizer build with `-fno-sanitize-recover=all`, and a diff --git a/docs/EVALS.md b/docs/EVALS.md index 709ecdbbe..53e2f04a0 100644 --- a/docs/EVALS.md +++ b/docs/EVALS.md @@ -21,7 +21,7 @@ section, and it is not an afterthought. | **Co-change / known-item evals** | `--eval`, `--eval-retrieval` (see `bench/ANSWERQUALITY.md`) | Whether the tool surfaces the other files a real historical commit touched; and known-item retrieval across four rankers. | | **Ensemble calibration harness** | `bench/ensemblecal/` | Whether `--ensemble`'s four evidence families are actually orthogonal, how often each fires, how stable each is across commits — and the preset ladder derived from that (§9). | | **Differential argv harness** | `test/argvdiffcheck.sh` | That a refactor changed *nothing observable*: two binaries, every argv vector, stdout + stderr + exit code byte-identical. | -| **The gate suite** | `test/regression.sh`, `test/pargates.py` | 701 gate scripts plus the determinism, cache-transparency and golden contracts. | +| **The gate suite** | `test/regression.sh`, `test/pargates.py` | 702 gate scripts plus the determinism, cache-transparency and golden contracts. | | **`--quality-delta`** | `src/quality.h` | Ten measured code-quality failure modes, reported only where a change made them worse. | ### The labeling protocol (why the held-out eval is allowed to disagree with the ranker) @@ -5888,7 +5888,7 @@ copy here would be exactly the dialect divergence that gate exists to catch. Com tags, wrap, stable-order defaults), seven individually invoked standalone gates (`g1freshcheck`, `skillscan`, `htmlexport`, `compresscheck`, `handoffcheck`, `releaseinstallcheck`, `taskroutecheck`), and a single loop -naming **701 gate scripts**, all of which exist on disk. +naming **702 gate scripts**, all of which exist on disk. `python3 test/pargates.py . ./build/ripwire -j 6` runs the same scripts in parallel so a full verification fits in one sitting. It does not modify `regression.sh`. @@ -7005,7 +7005,7 @@ Listed because the reason is more useful than the silence. shipped**. See `bench/locbench/anchorhop_calib.json`. The mention anchor's reproducible numbers are the ablations in §4. - **A single round gate-count.** Two in-tree numbers disagree (`test/pargates.py`'s docstring says - ~210; `test/argvdiffcheck.sh` says 200+), while the loop in `test/regression.sh` names 701. The + ~210; `test/argvdiffcheck.sh` says 200+), while the loop in `test/regression.sh` names 702. The loop is the authority; the stale docstrings are a known drift. Since 2026-09-10 the number is not written by hand anywhere: `docs/gatecount_build.py` derives it from the loop and rewrites every published site, `test/gatecountcheck.sh` fails if any of them drifts, and `test/manifestcheck.sh` diff --git a/present/deck5_ripwire_build.js b/present/deck5_ripwire_build.js index 670595996..6d9b06e22 100644 --- a/present/deck5_ripwire_build.js +++ b/present/deck5_ripwire_build.js @@ -1214,7 +1214,7 @@ function storyCards(s, { kick, head, stories, footText }){ kicker(s, "// how it stays true", AMBER); title(s, "Proven, not promised"); const cards = [ - ["701 gate scripts", "the suite runs on every push — plus determinism, cache-transparency and golden contracts; the gate count itself is gated against the runner's own loop"], // gatecount + ["702 gate scripts", "the suite runs on every push — plus determinism, cache-transparency and golden contracts; the gate count itself is gated against the runner's own loop"], // gatecount ["byte-identical, always", "two runs over the same tree produce the same bytes; warm equals cold. Enforced in CI, twice — Release AND a plain flavour, because NDEBUG once blinded a whole class of checks"], ["differential refactoring", "a refactor must prove it changed nothing observable: two binaries, hundreds of argv vectors, stdout + stderr + exit codes byte-identical"], ["held-out labels, authored blind", "eval labels were written by reading source before the ranker ever ran on them — so the eval is allowed to say the ranker is wrong. It has."], @@ -1238,7 +1238,7 @@ function storyCards(s, { kick, head, stories, footText }){ title(s, "Claims you can trust, because we publish what failed", { size: 32 }); card(s, MX, 1.72, 3.86, 1.72); - stat(s, "701", "gate scripts named by test/regression.sh — and the COUNT itself is gated against the runner's own loop, so it cannot go stale quietly", // gatecount + stat(s, "702", "gate scripts named by test/regression.sh — and the COUNT itself is gated against the runner's own loop, so it cannot go stale quietly", // gatecount MX+0.15, 1.86, 3.56, CYAN, { bsize: 42, bh: 0.66, lsize: 9.5 }); card(s, 4.68, 1.72, 3.86, 1.72, CARD2); stat(s, "8", "registered NEGATIVES — changes built, gated green, measured against a band written before the code, and reverted rather than tuned", @@ -1540,7 +1540,7 @@ function storyCards(s, { kick, head, stories, footText }){ ["192 long flags · 36 slides", "bash test/deckclaimcheck.sh"], ["every --flag named here exists", "bash test/deckcheck.sh"], ["74.7% fewer element bytes", "bash test/showcasecapturecheck.sh"], - ["701 gate scripts", "bash test/manifestcheck.sh"], // gatecount + ["702 gate scripts", "bash test/manifestcheck.sh"], // gatecount ["49 repos · 71 papers · 237 surveyed","bash test/readmedriftcheck.sh"], ["the ten moments, any row", "ripwire . --callers=SYM | wc -c"], ["the head-to-head table", "bench/headtohead/r4-2026-08-06/"], diff --git a/queries/cpp/tags.scm b/queries/cpp/tags.scm index 2fb82fe5f..b90d24606 100644 --- a/queries/cpp/tags.scm +++ b/queries/cpp/tags.scm @@ -153,7 +153,7 @@ ; field), so the module-scope patterns above never see it: pre-fix NO class-static constant was ; indexed, not even a SCREAMING one (probed 2026-08-12). This pattern is deliberately LOOSE — it ; also matches every plain default-member-initializer (`int retries = 3;`) — because tags-pass -; predicates never run (see the cast-keyword note below) and child ORDER between `static` and +; predicates did not run before #355 (see the cast-keyword note below) and child ORDER between `static` and ; `constexpr` is not fixed in source. The whole keep decision lives in ingest.cpp ; (fieldConstantCaptureKept): keep iff the field carries BOTH a `static` storage_class_specifier ; AND a const/constexpr/constinit type_qualifier — per-instance fields (default-initialized or @@ -172,7 +172,7 @@ ; ---- instance FIELDS (ripwire addition — the member-variable round, card A3, 2026-09-02) ---- ; Every per-object data member of a class/struct/union body: `int count = 0;`, `char* label;`, ; `int& ref;`, `int buf[8];`, a bitfield `unsigned x : 3;` — declared or default-initialized alike. -; Deliberately LOOSE like the class-static-constant pattern above (tags-pass predicates never run): +; Deliberately LOOSE like the class-static-constant pattern above (tags-pass predicates did not run before #355): ; it also matches `static int live;` and `static constexpr int kMax = 3;`. The keep decision is ; ingest_names.h fieldCaptureKept: a `static` storage_class_specifier DROPS the capture — a class-static ; constant keeps its @definition.constant row above (t="var"), and a mutable static member is not a @@ -203,7 +203,7 @@ ; ANONYMOUS token child of the declaration (its only named children are the type and the declarator), ; invisible to any named-node constraint. The whole qualifier test therefore lives in ingest.cpp ; (cudaMemorySpaceQualifierOf), the same capture-time home as isCppCastKeyword, because tags-pass -; predicates never run (see the cast-keyword note below). Policy enforced there: `__constant__` +; predicates did not run before #355 (see the cast-keyword note below). Policy enforced there: `__constant__` ; extracts case-blind (the keyword is the evidence — the Rust const_item rationale); ; `__device__`/`__managed__` are mutable device globals and stay behind the SCREAMING gate; no ; memory-space token drops. The BLAST RADIUS of these unconstrained patterns is all of Lang::Cpp @@ -360,8 +360,8 @@ ; ; WHY THE CAST KEYWORDS ARE NOT EXCLUDED HERE. tree-sitter-cpp parses `static_cast(x)` as this exact ; shape, so this pattern also matches all four C++ cast keywords (171 sites in src/ alone). The natural fix -; — a `(#not-eq? @name "static_cast")` predicate — does NOT work: ripwire's passesPredicates is wired into -; --match/--lint only, never into the tags pass (measured: the predicate left --uses=static_cast at 165). +; — a `(#not-eq? @name "static_cast")` predicate — did NOT work before #355: the tags pass ran no predicates +; (measured: the predicate left --uses=static_cast at 165). #355 runs the string ones, so this filter can move here. ; The exclusion therefore lives at capture time in ingest.cpp (isCppCastKeyword), where it is enforceable. (call_expression function: (template_function @@ -398,8 +398,8 @@ ; ; This pattern captures the SHAPE ONLY: any identifier-call statement whose immediate next named ; sibling is a compound_statement. It deliberately over-matches — a real `logCall( "x" );` followed by -; an unrelated block inside a function body is this exact shape — because tags-pass predicates never -; run (same story as the cast keywords above). The three real gates live at capture time in +; an unrelated block inside a function body is this exact shape — because tags-pass predicates did not +; run before #355 (same story as the cast keywords above). The three real gates live at capture time in ; ingest.cpp's testMacroBlockPartsOf: the callee must be a KNOWN test macro (kTestBlockMacroNames), ; the statement must carry the error-recovery MISSING ";" (a real semicolon disqualifies), and a title ; string literal must be present in the argument list (it becomes the symbol's name; the identifier diff --git a/queries/elixir/tags.scm b/queries/elixir/tags.scm index 88794a9be..ce914b3aa 100644 --- a/queries/elixir/tags.scm +++ b/queries/elixir/tags.scm @@ -1,11 +1,18 @@ -; Elixir definitions are ordinary calls in the grammar. captureTagsFacts applies -; the keyword/head filters in ingest_elixir.h and the shared inert-region -; walk in ingest_inert.h; tags predicates are not evaluated. Capturing every call as -; @inert/@live would still need C++ head checks and enlarge this hot capture stream. +; Elixir definitions are ordinary calls in the grammar. The tags pass evaluates the +; string predicates below (#eq?/#any-of?/#not-any-of?, #355; never #match?), so the +; definition keywords are data HERE: the #any-of? list on @_keyword is the only copy, +; and ingest_elixir.h's tree walks read it back from the compiled query. The remaining +; head checks (quoted syntax, test titles, typed attributes) are in ingest_elixir.h, with the +; shared inert-region walk in ingest_inert.h. Capturing every call as @inert/@live +; would still need C++ head checks and enlarge this hot capture stream. ; Anchors restrict definition names to the FIRST argument (never a call's body). (call target: (identifier) (arguments . [(alias) (dot) (atom)] @name)) @definition.module -(call target: (identifier) +; A keyword-headed definition: `def f(x)`, `defp g(a) when a > 0`, `def a + b`. @_keyword is +; predicate-only (the tags pass ignores its role); the two patterns carry the SAME lists, and +; test/tagspredicatecheck.sh arm A keeps them equal. `def unquote(name)(x)` and a bare `when` +; head name nothing static, so #not-any-of? drops them. +((call target: (identifier) @_keyword (arguments . [ (identifier) @name (call target: (identifier) @name) @@ -14,7 +21,9 @@ (identifier) @name ] operator: "when") ])) @definition.function -(call target: (identifier) + (#any-of? @_keyword "def" "defp" "defmacro" "defmacrop" "defguard" "defguardp" "defdelegate") + (#not-any-of? @name "unquote" "when")) +((call target: (identifier) @_keyword (arguments . [ (binary_operator operator: _ @name) (unary_operator operator: _ @name) @@ -23,6 +32,8 @@ (unary_operator operator: _ @name) ] operator: "when") ])) @definition.function + (#any-of? @_keyword "def" "defp" "defmacro" "defmacrop" "defguard" "defguardp" "defdelegate") + (#not-any-of? @name "unquote" "when")) (call target: [ (identifier) @name @@ -36,8 +47,14 @@ (identifier) @name @reference.bare ; Types and callbacks are declarations; ordinary attributes may evaluate expressions. -(unary_operator operator: "@" operand: - (call target: (identifier) @name)) @definition.attribute +; A metadata attribute (@doc, @spec, ...) defines nothing: the #not-any-of? list on @_attribute +; is the only copy of those names, and ingest_elixir.h reads it back so a reference INSIDE one +; is not a call site (#355). The typed attributes (@type, @callback, ...) stay a C++ kind decision. +((unary_operator operator: "@" operand: + (call target: (identifier) @name @_attribute)) @definition.attribute + (#not-any-of? @_attribute "doc" "moduledoc" "typedoc" "spec" "impl" "behaviour" "behavior" "derive" "enforce_keys" + "dialyzer" "on_load" "before_compile" "after_compile" "after_verify" "optional_callbacks" "compile" "external_resource" + "vsn" "file")) (unary_operator operator: "@" operand: (call target: (identifier) (arguments . (binary_operator left: [ diff --git a/queries/javascript/tags.scm b/queries/javascript/tags.scm index 85146e267..3f4e139c1 100644 --- a/queries/javascript/tags.scm +++ b/queries/javascript/tags.scm @@ -66,7 +66,7 @@ ; CJS export assignment — `module.exports.NAME = fn` / `exports.NAME = fn` mints a function ON the ; export object (47 webpack/lib sites, 0 extracted before this round). Structurally these patterns ; capture EVERY `a.b = fn` / `a.b.c = fn`; the object-text scoping (exports / module.exports) lives -; in ingest.cpp (isCjsExportTarget) because tags-pass predicates never run — see the SCREAMING_SNAKE +; in ingest.cpp (isCjsExportTarget) because tags-pass predicates did not run before #355 — see the SCREAMING_SNAKE ; precedent above. `!name`-scoped to ANONYMOUS values: a named function expression already defines ; its inner name, and capturing both would double the row. (assignment_expression @@ -126,7 +126,7 @@ ; lane lego-transitive (M2): a class FACTORY — `const Sub = Base.extend('Sub', …)` (nunjucks' node classes). The call ; returns a class extending its receiver, so the declarator is a class definition. Captured by shape here, kept only by -; ingest_names.h isJsClassFactory (tags-pass predicates never run): property `extend`, first argument a string literal +; ingest_names.h isJsClassFactory (tags-pass predicates did not run before #355): property `extend`, first argument a string literal ; spelling the declared name — `$.extend({}, a)` and an unnamed Backbone `Base.extend({…})` are object merges by shape and ; drop (a disclosed floor). The receiver becomes the base (ingest_sidecap.h, emitBaseRef). (lexical_declaration @@ -181,7 +181,7 @@ ; call_expression or a jsx_*_element). An intrinsic tag (`
`, `

`) parses as the SAME ; (identifier) shape as a component tag — the grammar carries no case distinction — so the ; lower-case-first-letter filter lives in C++ at capture time (isJsxIntrinsicTagIdentifier, -; src/ingest_names.h): tags-pass predicates never run (`(#match? @name "^[A-Z]")` here would be +; src/ingest_names.h): tags-pass predicates did not run before #355 (`(#match? @name "^[A-Z]")` here would be ; silently ignored — see isCppCastKeyword's note above, measured, not assumed). A namespaced tag ; (``) needs no filter at all: verified with --match that its name field is a DIFFERENT ; grammar node, jsx_namespace_name, which no pattern below names, so it is never captured in the @@ -213,7 +213,7 @@ ; require('x') / import('x') — a call whose FIRST named argument is a string ; ; The call pattern is deliberately broader than a module load: it matches any bare-identifier (or `import`) call -; with a leading string, `foo('x')` included, because a tags-pass predicate (`#eq? @f "require"`) is NOT evaluated +; with a leading string, `foo('x')` included, because a tags-pass predicate (`#eq? @f "require"`) was NOT evaluated before #355 ; in this tree (pinned by test/matchgrammarcheck.sh). So the C++ normaliser owns the three guards that keep an ; ordinary call out of the dependency graph: the callee TEXT is `require`/`import`, there is exactly ONE argument, ; and that argument is the string. The leading `.` anchors the capture to the first named child, so a 16 000-argument diff --git a/queries/kotlin/tags.scm b/queries/kotlin/tags.scm index 8f1ad9f03..e4a402828 100644 --- a/queries/kotlin/tags.scm +++ b/queries/kotlin/tags.scm @@ -26,7 +26,7 @@ ; Deliberately NOT captured (first cut, disclosed floor, not an oversight): ; - property_declaration (`val`/`var`, class or top-level): Kotlin's constant-vs-mutable ; distinction is the `const` MODIFIER keyword, not a case convention, and tags-pass predicates -; never run — the same reason Rust's const_item/static_item skip constCaptureNeedsScreamingGate +; did not run before #355 — the same reason Rust's const_item/static_item skip constCaptureNeedsScreamingGate ; entirely (the keyword IS the evidence there). Kotlin's `const val` needs that same ; keyword-checked gate in ingest.cpp, not yet written; capturing every property unconditionally ; here would flood the symbol table with every mutable field as a spurious "constant" the day diff --git a/queries/python/tags.scm b/queries/python/tags.scm index 3d0ca57c9..45e9c9351 100644 --- a/queries/python/tags.scm +++ b/queries/python/tags.scm @@ -25,7 +25,7 @@ ; Instance attribute bound in a method: `self.x = …` / `self.x: T = …` at ANY depth under the method — the ONE ; Python FIELD shape (IngestResult::fields, never a map symbol) ; (an `if` in __init__ still declares it). tree-sitter has no descendant operator and tags-pass -; predicates never run, so the pattern is LOOSE — it matches `obj.x = …` too — and the gate lives in +; predicates did not run before #355, so the pattern is LOOSE — it matches `obj.x = …` too — and the gate lives in ; ingest_names.h fieldCaptureKept: the receiver must be the bare identifier `self` and the assignment ; must sit inside a function inside a class. ONE symbol per (class, name): ingest_sidecap.h keeps the ; lowest-byte assignment per key (the definition), every later one is a role="write" use-site @@ -34,7 +34,7 @@ ; Plain class-body assignment, kept ONLY when the enclosing class's base NAME is an enum family ; (132 sites in django — stdlib enum plus the Choices family — and 108 in pydantic). The semantic -; half lives in dropGatedCapture/isPyEnumMemberTarget — tags-pass predicates never run, so the name +; half lives in dropGatedCapture/isPyEnumMemberTarget — tags-pass predicates did not run before #355, so the name ; test cannot live here. (class_definition body: (block (expression_statement (assignment left: (identifier) @name !type) @definition.enummember))) diff --git a/queries/tsx/tags.scm b/queries/tsx/tags.scm index ca3661c98..f55dcf26b 100644 --- a/queries/tsx/tags.scm +++ b/queries/tsx/tags.scm @@ -95,7 +95,7 @@ ; Shape verified with --match: the (program …) wrapper keeps function-local consts out; the export ; form nests (program (export_statement (lexical_declaration …))) so it needs its own pattern. ; "const"-keyword-anchored (a top-level `let` is a mutable counter, not config). Scoped to -; SCREAMING_SNAKE names in ingest.cpp (constCaptureNeedsScreamingGate — tags predicates never run), +; SCREAMING_SNAKE names in ingest.cpp (constCaptureNeedsScreamingGate — tags predicates did not run before #355), ; so `const retryBudget = 3` stays unindexed and an ALL-CAPS arrow const dedups to its Function def. (program @@ -119,7 +119,7 @@ ; lane lego-transitive (M2): a class FACTORY — `const Sub = Base.extend('Sub', …)` (nunjucks' node classes). The call ; returns a class extending its receiver, so the declarator is a class definition. Captured by shape here, kept only by -; ingest_names.h isJsClassFactory (tags-pass predicates never run): property `extend`, first argument a string literal +; ingest_names.h isJsClassFactory (tags-pass predicates did not run before #355): property `extend`, first argument a string literal ; spelling the declared name — `$.extend({}, a)` and an unnamed Backbone `Base.extend({…})` are object merges by shape and ; drop (a disclosed floor). The receiver becomes the base (ingest_sidecap.h, emitBaseRef). (lexical_declaration @@ -235,7 +235,7 @@ ; require('x') / import('x') — a call whose FIRST named argument is a string ; ; The call pattern is deliberately broader than a module load: it matches any bare-identifier (or `import`) call -; with a leading string, `foo('x')` included, because a tags-pass predicate (`#eq? @f "require"`) is NOT evaluated +; with a leading string, `foo('x')` included, because a tags-pass predicate (`#eq? @f "require"`) was NOT evaluated before #355 ; in this tree (pinned by test/matchgrammarcheck.sh). So the C++ normaliser owns the three guards that keep an ; ordinary call out of the dependency graph: the callee TEXT is `require`/`import`, there is exactly ONE argument, ; and that argument is the string. The leading `.` anchors the capture to the first named child, so a 16 000-argument @@ -264,7 +264,7 @@ ; call_expression or a jsx_*_element). An intrinsic tag (`
`, `

`) parses as the SAME ; (identifier) shape as a component tag — the grammar carries no case distinction — so the ; lower-case-first-letter filter lives in C++ at capture time (isJsxIntrinsicTagIdentifier, -; src/ingest_names.h): tags-pass predicates never run (`(#match? @name "^[A-Z]")` here would be +; src/ingest_names.h): tags-pass predicates did not run before #355 (`(#match? @name "^[A-Z]")` here would be ; silently ignored — see isCppCastKeyword's note above, measured, not assumed). A namespaced tag ; (``) needs no filter at all: verified with --match that its name field is a DIFFERENT ; grammar node, jsx_namespace_name, which no pattern below names, so it is never captured in the diff --git a/queries/typescript/tags.scm b/queries/typescript/tags.scm index e6f529091..17eaf926b 100644 --- a/queries/typescript/tags.scm +++ b/queries/typescript/tags.scm @@ -95,7 +95,7 @@ ; Shape verified with --match: the (program …) wrapper keeps function-local consts out; the export ; form nests (program (export_statement (lexical_declaration …))) so it needs its own pattern. ; "const"-keyword-anchored (a top-level `let` is a mutable counter, not config). Scoped to -; SCREAMING_SNAKE names in ingest.cpp (constCaptureNeedsScreamingGate — tags predicates never run), +; SCREAMING_SNAKE names in ingest.cpp (constCaptureNeedsScreamingGate — tags predicates did not run before #355), ; so `const retryBudget = 3` stays unindexed and an ALL-CAPS arrow const dedups to its Function def. (program @@ -119,7 +119,7 @@ ; lane lego-transitive (M2): a class FACTORY — `const Sub = Base.extend('Sub', …)` (nunjucks' node classes). The call ; returns a class extending its receiver, so the declarator is a class definition. Captured by shape here, kept only by -; ingest_names.h isJsClassFactory (tags-pass predicates never run): property `extend`, first argument a string literal +; ingest_names.h isJsClassFactory (tags-pass predicates did not run before #355): property `extend`, first argument a string literal ; spelling the declared name — `$.extend({}, a)` and an unnamed Backbone `Base.extend({…})` are object merges by shape and ; drop (a disclosed floor). The receiver becomes the base (ingest_sidecap.h, emitBaseRef). (lexical_declaration @@ -235,7 +235,7 @@ ; require('x') / import('x') — a call whose FIRST named argument is a string ; ; The call pattern is deliberately broader than a module load: it matches any bare-identifier (or `import`) call -; with a leading string, `foo('x')` included, because a tags-pass predicate (`#eq? @f "require"`) is NOT evaluated +; with a leading string, `foo('x')` included, because a tags-pass predicate (`#eq? @f "require"`) was NOT evaluated before #355 ; in this tree (pinned by test/matchgrammarcheck.sh). So the C++ normaliser owns the three guards that keep an ; ordinary call out of the dependency graph: the callee TEXT is `require`/`import`, there is exactly ONE argument, ; and that argument is the string. The leading `.` anchors the capture to the first named child, so a 16 000-argument diff --git a/queries/yaml/tags.scm b/queries/yaml/tags.scm index 94774fcf8..b49ecf925 100644 --- a/queries/yaml/tags.scm +++ b/queries/yaml/tags.scm @@ -22,7 +22,7 @@ ; arbitrarily (`- - key:`), so the ancestor chain between a pair and the document is unbounded. ; The patterns below therefore capture EVERY mapping pair, and the depth cut lives in ingest ; (dropGatedCapture's definition.yamlkey arm — the same in-C++ home as every other predicate a -; tags pattern cannot express; tags-pass #eq?/#match? predicates never run in ripwire). +; tags pattern cannot express; tags-pass predicates did not run in ripwire before #355, and #match? still does not). ; ; ── THE TABLE OF DECISIONS (each from measured frequency — PLAN 2026-08-10 / docs/EVALS.md) ── ; mapping key → a symbol IF its mapping-depth <= 2, sequences not counted (ingest diff --git a/src/infra/tspredicates.h b/src/infra/tspredicates.h new file mode 100644 index 000000000..2ade7a427 --- /dev/null +++ b/src/infra/tspredicates.h @@ -0,0 +1,209 @@ +#pragma once +// tspredicates.h — ONE walk over a tree-sitter pattern's predicate steps, and the STRING predicates decided on it. +// +// WHY THIS EXISTS. `ts_query` parses `(#eq? @a "x")` and friends into steps but never applies them; every caller +// must. Before #355 there were two spellings of the step-group walk (passesPredicates and its regex precompiler +// buildPredicateRegexTable, src/ingest_astquery.h — "the two must agree about which steps they consume") and the +// tags pass had NONE: its predicates were silently ignored, so a keyword-headed definition (Elixir `def`, a Lisp +// `defn`) had to keep its keyword list in C++. This header is the walk all three readers share, and the string +// predicates the tags pass now evaluates. +// +// WHAT THE TAGS PASS RUNS, AND WHAT IT DOES NOT. Only the string predicates below: #eq?, #not-eq?, #any-of?, +// #not-any-of?. Never #match? / #not-match? — a regex per tags match is the cost src/ingest_astquery.h's header +// measured (13.52% of busy CPU on --lint), and the tags pass runs over every file of every crawl. An operator this +// header does not decide is StringPredicate::NotString; the tags pass then filters nothing for it, exactly as before, +// and test/tagspredicatecheck.sh arm A refuses any tags.scm that uses one. +// +// NO ALLOCATION ON THE PER-MATCH PATH. The group is a view into the query's own step array, the operator and string +// arguments are views into the query's string table, and a capture's text is a view into the file's bytes. + +#include + +#include +#include +#include +#include + +namespace rw +{ + +// The query's string `id` as a view into the query's own table; empty when tree-sitter has no such string. +// TWO statements, deliberately: the length `len` and the call that WRITES it must not be two arguments of one call, +// whose evaluation order is unspecified (gcc on x86-64 reads `len` first, while it is still 0 — the bug +// passesPredicates' sequencing note in src/ingest_astquery.h records, measured 2026-08-02). Do not re-inline. +inline std::string_view queryString( const TSQuery* q, std::uint32_t id ) noexcept +{ + std::uint32_t len = 0; + const char* text = ts_query_string_value_for_id( q, id, &len ); + return text != nullptr ? std::string_view( text, len ) : std::string_view(); +} + +// The source text of capture `capIndex` (its first node) in match `m`, as a view into `src`; false when the match +// has no such capture or the node's span falls outside `src`. +inline bool captureView( const TSQueryMatch& m, std::uint32_t capIndex, std::string_view src, std::string_view& out ) noexcept +{ + for( std::uint16_t i = 0; i < m.capture_count; ++i ) + { + if( m.captures[i].index == capIndex ) + { + const TSNode n = m.captures[i].node; + const std::uint32_t a = ts_node_start_byte( n ), b = ts_node_end_byte( n ); + if( a <= b && b <= src.size() ) + { + out = src.substr( a, b - a ); + return true; + } + return false; + } + } + return false; +} + +// Call fn( group ) for each predicate of pattern `pattern`, in order. `group` is the predicate's steps without its +// trailing Done step: group[0] the operator name, then its arguments. fn returns false to stop the walk early. +template inline void forEachPredicate( const TSQuery* q, std::uint32_t pattern, Fn&& fn ) +{ + std::uint32_t count = 0; + const TSQueryPredicateStep* steps = ts_query_predicates_for_pattern( q, pattern, &count ); + std::uint32_t begin = 0; + for( std::uint32_t i = 0; i < count; ++i ) + { + if( steps[i].type != TSQueryPredicateStepTypeDone ) + { + continue; + } + if( !fn( std::span( steps + begin, i - begin ) ) ) + { + return; + } + begin = i + 1; + } +} + +// The operator of a predicate group whose first step is a string, or empty for a malformed or empty group. +inline std::string_view predicateOperator( const TSQuery* q, std::span group ) noexcept +{ + if( group.empty() || group[0].type != TSQueryPredicateStepTypeString ) + { + return {}; + } + return queryString( q, group[0].value_id ); +} + +enum class StringPredicate : std::uint8_t { Pass, Fail, NotString }; + +// Decide ONE predicate group if it is a string predicate: +// (#eq? A B) / (#not-eq? A B) — A and B each a capture or a string +// (#any-of? @c "s" …) / (#not-any-of? @c "s" …) — the capture's text against a list of strings +// NotString: any other operator (#match? included), or a group this cannot read — an absent capture, a short group, a +// non-string list entry. The caller filters nothing for NotString, which is what an unevaluated predicate always did. +inline StringPredicate evalStringPredicate( const TSQuery* q, std::span group, const TSQueryMatch& m, + std::string_view src ) noexcept +{ + const std::string_view op = predicateOperator( q, group ); + const bool eq = op == "eq?" || op == "not-eq?"; + const bool anyOf = op == "any-of?" || op == "not-any-of?"; + if( ( !eq && !anyOf ) || group.size() < 3 ) + { + return StringPredicate::NotString; + } + const auto argText = [ & ]( const TSQueryPredicateStep& s, std::string_view& out ) -> bool + { + if( s.type == TSQueryPredicateStepTypeCapture ) + { + return captureView( m, s.value_id, src, out ); + } + if( s.type == TSQueryPredicateStepTypeString ) + { + out = queryString( q, s.value_id ); + return true; + } + return false; + }; + std::string_view lhs; + if( !argText( group[1], lhs ) ) + { + return StringPredicate::NotString; + } + const bool negated = op.starts_with( "not-" ); + if( eq ) + { + std::string_view rhs; + if( !argText( group[2], rhs ) ) + { + return StringPredicate::NotString; + } + return ( ( lhs == rhs ) != negated ) ? StringPredicate::Pass : StringPredicate::Fail; + } + bool listed = false; + for( std::size_t i = 2; i < group.size(); ++i ) + { + if( group[i].type != TSQueryPredicateStepTypeString ) + { + return StringPredicate::NotString; + } + listed = listed || queryString( q, group[i].value_id ) == lhs; + } + return ( listed != negated ) ? StringPredicate::Pass : StringPredicate::Fail; +} + +// True when no string predicate of the match's pattern fails. The tags pass's whole predicate surface: operators this +// header does not decide (#match? among them) filter nothing here, as they never did. +inline bool passesStringPredicates( const TSQuery* q, const TSQueryMatch& m, std::string_view src ) noexcept +{ + bool ok = true; + forEachPredicate( q, m.pattern_index, [ & ]( std::span group ) + { + ok = evalStringPredicate( q, group, m, src ) != StringPredicate::Fail; + return ok; + } ); + return ok; +} + +// The strings of every (# @capture …) predicate in the query (op "any-of?" or "not-any-of?"), in pattern order, +// without repeats. A tags.scm that filters a definition head by a keyword list is the ONE place that list is written; +// a tree walk that must recognise the same head outside a match (an enclosing `def`, a reference inside `@spec`) reads +// it here instead of keeping a second copy in C++. The views point into the query's string table and live as long as +// the query. +inline std::vector predicateStrings( const TSQuery* q, std::string_view op, std::string_view capture ) +{ + std::vector out; + const std::uint32_t patterns = ts_query_pattern_count( q ); + for( std::uint32_t p = 0; p < patterns; ++p ) + { + forEachPredicate( q, p, [ & ]( std::span group ) + { + if( predicateOperator( q, group ) != op || group.size() < 3 || group[1].type != TSQueryPredicateStepTypeCapture ) + { + return true; + } + std::uint32_t len = 0; // two statements: see queryString + const char* name = ts_query_capture_name_for_id( q, group[1].value_id, &len ); + if( name == nullptr || std::string_view( name, len ) != capture ) + { + return true; + } + for( std::size_t i = 2; i < group.size(); ++i ) + { + if( group[i].type != TSQueryPredicateStepTypeString ) + { + continue; + } + const std::string_view s = queryString( q, group[i].value_id ); + bool seen = false; + for( const std::string_view& prior : out ) + { + seen = seen || prior == s; + } + if( !seen ) + { + out.push_back( s ); + } + } + return true; + } ); + } + return out; +} + +} // namespace rw diff --git a/src/ingest.cpp b/src/ingest.cpp index cddb26107..359252b22 100644 --- a/src/ingest.cpp +++ b/src/ingest.cpp @@ -19,6 +19,7 @@ #include "infra/namesplit.h" // H4: stripTemplateArgs for the C++ qualified-call re-split (shared with tracelocus.h) #include "infra/jsonesc.h" // rw::shSingleQuote - the git ignore probe quotes its root the same way every other git popen does #include "infra/fixedStr.h" // rw::findByte — the NEON/SSE2 byte scan buildNewlineOffsets rides +#include "infra/tspredicates.h" // rw::passesStringPredicates — the tags pass evaluates #eq?/#any-of?/#not-any-of? (#355); --match/--lint share the walk #include "infra/ownedfile.h" // rw::OwnedFile — readFile/readFilePrefix own their stream, so no return path skips the close #include "infra/statclock.h" // rw::saturatingNanoseconds — statSizeTimes' stat timestamps without signed overflow past 2262 #include "lexindex.h" // B0.1/B0.2: shared subtoken state machine + per-def lexical statistics builder diff --git a/src/ingest_astquery.h b/src/ingest_astquery.h index ec6031435..c5e2ce1ec 100644 --- a/src/ingest_astquery.h +++ b/src/ingest_astquery.h @@ -74,7 +74,7 @@ struct PredicateRegexTable }; // Walk every predicate of every pattern once and compile the constant #match?/#not-match? arguments. -// Deliberately a mirror of passesPredicates' own step-group loop below — the two must agree about which +// It walks with forEachPredicate, the same walk passesPredicates uses below — the two must agree about which // argument is the pattern, and a second spelling of that rule is how a filter quietly changes meaning. inline PredicateRegexTable buildPredicateRegexTable( const TSQuery* q ) { @@ -87,54 +87,31 @@ inline PredicateRegexTable buildPredicateRegexTable( const TSQuery* q ) const std::uint32_t patterns = ts_query_pattern_count( q ); for( std::uint32_t patternIndex = 0; patternIndex < patterns; ++patternIndex ) { - std::uint32_t pc = 0; - const TSQueryPredicateStep* steps = ts_query_predicates_for_pattern( q, patternIndex, &pc ); - for( std::uint32_t i = 0; i < pc; ) + forEachPredicate( q, patternIndex, [ & ]( std::span group ) { - const std::uint32_t begin = i; - for( ; i < pc && steps[i].type != TSQueryPredicateStepTypeDone; ++i ) + const std::string_view op = predicateOperator( q, group ); + if( ( op != "match?" && op != "not-match?" ) || group.size() < 3 ) { + return true; } - const std::uint32_t n = i - begin; - ++i; // skip the Done step - if( n < 3 || steps[begin].type != TSQueryPredicateStepTypeString ) - { - continue; - } - std::uint32_t nl = 0; // two statements — see the sequencing note in passesPredicates - const char* opText = ts_query_string_value_for_id( q, steps[begin].value_id, &nl ); - if( opText == nullptr ) - { - continue; - } - const std::string_view op( opText, nl ); - if( op != "match?" && op != "not-match?" ) - { - continue; - } - const TSQueryPredicateStep& arg = steps[ begin + 2 ]; + const TSQueryPredicateStep& arg = group[2]; if( arg.type != TSQueryPredicateStepTypeString || arg.value_id >= table.slotOfStringId.size() || table.slotOfStringId[ arg.value_id ] != -1 ) { - continue; // capture-typed, out of range, or already decided - } - std::uint32_t rl = 0; - const char* rv = ts_query_string_value_for_id( q, arg.value_id, &rl ); - if( rv == nullptr ) - { - continue; + return true; // capture-typed, out of range, or already decided } - const std::string pattern( rv, rl ); + const std::string pattern( queryString( q, arg.value_id ) ); RegexCompile compiled = compileGuardedRegex( pattern, kRegexEcmaScript ); // std::regex's default flags, as before if( compiled.refusal ) { table.slotOfStringId[ arg.value_id ] = -2; // refused — the built-in packs keep the old per-match catch table.refusals.push_back( "'" + pattern + "' refused: " + *compiled.refusal ); - continue; + return true; } table.res.push_back( std::move( compiled.regex ) ); table.slotOfStringId[ arg.value_id ] = static_cast( table.res.size() - 1 ); - } + return true; + } ); } return table; } @@ -226,85 +203,56 @@ inline void noteUndecidedPredicate( const PredicateSite& site, const TSQueryMatc site.fileId, byte, lineAtByte( site.nlOffsets, byte ), pattern, refused.refusal.value_or( std::string() ) ); } -// evaluate a pattern's query predicates against a match — #eq? / #not-eq? (string/capture equality) and -// #match? / #not-match? (ECMAScript regex). ts_query never applies these itself; without this, #eq? is a no-op. +// evaluate a pattern's query predicates against a match — the string predicates #eq? / #not-eq? / #any-of? / +// #not-any-of? (src/infra/tspredicates.h, shared with the tags pass, #355) and #match? / #not-match? (ECMAScript +// regex, --match/--lint only). ts_query never applies these itself; without this, #eq? is a no-op. +// The step-group walk is forEachPredicate — the same one buildPredicateRegexTable walks, so the two agree about which +// steps a predicate consumes by construction. The operator name is read in TWO statements (queryString, whose note +// records the gcc-on-x86-64 evaluation-order bug that once made every predicate here a silent no-op). inline bool passesPredicates( const TSQuery* q, const PredicateRegexTable& rx, const TSQueryMatch& m, std::string_view src, const PredicateSite& site ) { - std::uint32_t pc = 0; - const TSQueryPredicateStep* steps = ts_query_predicates_for_pattern( q, m.pattern_index, &pc ); - const auto argText = [ & ]( const TSQueryPredicateStep& s, std::string& out ) -> bool - { - if( s.type == TSQueryPredicateStepTypeCapture ) - { - return captureText( m, s.value_id, src, out ); - } - if( s.type == TSQueryPredicateStepTypeString ) - { - std::uint32_t l = 0; - const char* v = ts_query_string_value_for_id( q, s.value_id, &l ); - out.assign( v, l ); - return true; - } - return false; - }; - for( std::uint32_t i = 0; i < pc; ) + bool ok = true; + forEachPredicate( q, m.pattern_index, [ & ]( std::span pr ) { - std::vector pr; - for( ; i < pc && steps[i].type != TSQueryPredicateStepTypeDone; ++i ) + const StringPredicate decided = evalStringPredicate( q, pr, m, src ); + if( decided != StringPredicate::NotString ) { - pr.push_back( steps[i] ); + ok = ( decided == StringPredicate::Pass ); + return ok; } - ++i; // skip the Done step - if( pr.size() < 3 || pr[0].type != TSQueryPredicateStepTypeString ) + const std::string_view op = predicateOperator( q, pr ); + if( ( op != "match?" && op != "not-match?" ) || pr.size() < 3 ) { - continue; + return true; // unknown operator or short group ⇒ don't filter } - // TWO statements, deliberately. Written as ONE — `string_view( f( …, &nl ), nl )` — the length - // argument `nl` and the call that WRITES it are two arguments of the SAME call, and the order in - // which a call's arguments are evaluated is UNSPECIFIED. GCC on x86-64 evaluates them right-to-left, - // so it read `nl` while it was still 0: `op` came out EMPTY, matched none of the operator names - // below, and every #eq?/#not-eq?/#match?/#not-match? predicate was silently skipped (`ok` stays - // true ⇒ nothing filtered). GCC on aarch64 and Clang everywhere evaluate left-to-right and happened - // to get it right, which is why this only ever reddened on x86-64 Linux/gcc — measured 2026-08-02: - // g++-13 -O0 AND -O2 on x86-64 print len=0, the same g++-13 on aarch64 and clang-18 on x86-64 print - // len=6. Sequencing the write before the read IS the fix; do not re-inline these two lines. - std::uint32_t nl = 0; - const char* opText = ts_query_string_value_for_id( q, pr[0].value_id, &nl ); - if( opText == nullptr ) - { - continue; // no operator name ⇒ can't evaluate ⇒ don't filter - } - const std::string_view op( opText, nl ); std::string lhs, rhs; - if( !argText( pr[1], lhs ) || !argText( pr[2], rhs ) ) - { - continue; // can't evaluate → don't filter - } - bool ok = true; - if( op == "eq?" ) - { - ok = ( lhs == rhs ); - } - else if( op == "not-eq?" ) + const auto argText = [ & ]( const TSQueryPredicateStep& s, std::string& out ) -> bool { - ok = ( lhs != rhs ); - } - else if( op == "match?" || op == "not-match?" ) - { - const MatchPredicateOutcome outcome = evalMatchPredicate( op == "not-match?", lhs, rhs, rx, pr[2] ); - ok = ( outcome != MatchPredicateOutcome::Fail ); // undecided ⇒ filters nothing, as it always did - if( ok && outcome != MatchPredicateOutcome::Pass ) + if( s.type == TSQueryPredicateStepTypeCapture ) { - noteUndecidedPredicate( site, m, outcome, rhs ); + return captureText( m, s.value_id, src, out ); } + if( s.type == TSQueryPredicateStepTypeString ) + { + out = std::string( queryString( q, s.value_id ) ); + return true; + } + return false; + }; + if( !argText( pr[1], lhs ) || !argText( pr[2], rhs ) ) + { + return true; // can't evaluate → don't filter } - if( !ok ) + const MatchPredicateOutcome outcome = evalMatchPredicate( op == "not-match?", lhs, rhs, rx, pr[2] ); + ok = ( outcome != MatchPredicateOutcome::Fail ); // undecided ⇒ filters nothing, as it always did + if( ok && outcome != MatchPredicateOutcome::Pass ) { - return false; + noteUndecidedPredicate( site, m, outcome, rhs ); } - } - return true; + return ok; + } ); + return ok; } // ---- per-file newline-offset index → O(log n) 1-based line lookup (A4 perf) ---- diff --git a/src/ingest_cache.h b/src/ingest_cache.h index 8b44965c6..3cd53c2d8 100644 --- a/src/ingest_cache.h +++ b/src/ingest_cache.h @@ -345,7 +345,11 @@ constexpr std::uint32_t kCacheVersion = 36; // 36: train 30 — strict // (Py `pkg.mod`, TS `./x`, Rust `crate::a::b`/`mod:x`) — // a target FORMAT change → old caches must be rejected. // 4: Include gained a `bool isAngle` (quote/angle) field -constexpr std::uint32_t kParserVer = 206; // bump on any grammar/.scm/extraction change +constexpr std::uint32_t kParserVer = 216; // bump on any grammar/.scm/extraction change + // 216 = 2026-10-22 (#355): the tags pass evaluates tags.scm's string predicates, and Elixir's + // definition keywords and metadata attribute names move into queries/elixir/tags.scm. A + // placeholder strictly above every live number (main 206/207, PR #424 212/213, PR #420 + // 214/215); the number a maintainer reserves replaces it. Rich file tag 217. kCacheVersion stays 36. // 206 = 2026-10-18 (#409, PR #414, renumbered again at merge with main 939dbb69): strictly above every // live number — main 196 (rich 197), train 31 198 (199), lane rust-receivers 202 (203), lane ix1-slot // 204 (205). Rich file tag 207. kCacheVersion stays 36 (no record format change). diff --git a/src/ingest_elixir.h b/src/ingest_elixir.h index 4ab76702e..fb4888a9b 100644 --- a/src/ingest_elixir.h +++ b/src/ingest_elixir.h @@ -12,8 +12,8 @@ namespace { // Included by ingest.cpp after the shared AST helpers. -// Elixir keywords are identifiers, so tags.scm alone cannot distinguish def f(x) from -// ordinary(f(x)). Keep the text gates local to this grammar instead of changing every tags pass. +// Elixir keywords are identifiers: tags.scm tells def f(x) from ordinary(f(x)) only through its +// string predicates (#355). The text gates that predicates cannot express stay local to this grammar. /// Return an identifier call target as a view into src, or an empty view for null or remote targets. std::string_view elixirTarget( TSNode node, std::string_view src ) noexcept { @@ -24,12 +24,24 @@ std::string_view elixirTarget( TSNode node, std::string_view src ) noexcept return fieldIdentifierText( node, NodeField::Target, src ); } -/// Return whether target introduces a function, macro, guard, or delegate definition. -bool elixirFunctionKeyword( std::string_view target ) noexcept +/// The definition keywords (def, defp, defmacro, ...) are DATA: the #any-of? list on the @_keyword capture of +/// queries/elixir/tags.scm's keyword-headed @definition.function patterns, which the tags pass applies itself (#355). +/// A tags match is a keyword-headed definition when it carries @_keyword. The tree walks below that must recognise a +/// definition head OUTSIDE a match (an enclosing `def` while scoping a binding, a `def` call that is not a reference) +/// read the same list here, filled by ElixirContext::prepare from the file's compiled query, instead of a C++ copy. +/// The metadata attributes (@doc, @spec, ...) are the same kind of data: the #not-any-of? list on @_attribute of the +/// @definition.attribute pattern, read back for the references INSIDE such an attribute, which are not call sites. +struct ElixirKeywords { - constexpr std::string_view keywords[] = { "def", "defp", "defmacro", "defmacrop", "defguard", "defguardp", "defdelegate" }; - return std::find( std::begin( keywords ), std::end( keywords ), target ) != std::end( keywords ); -} + std::vector function; // views into the TSQuery's string table, which outlives every file + std::vector metadata; // the same, for the metadata attribute names + + /// Return whether target introduces a function, macro, guard, or delegate definition. + bool isFunction( std::string_view target ) const noexcept + { + return std::find( function.begin(), function.end(), target ) != function.end(); + } +}; /// Return whether target introduces a statically named module or protocol, excluding defimpl. bool elixirModuleKeyword( std::string_view target ) noexcept @@ -144,11 +156,11 @@ bool elixirTypedAttribute( std::string_view name ) noexcept return name == "type" || name == "typep" || name == "opaque" || name == "callback" || name == "macrocallback"; } -bool elixirMetadataAttribute( std::string_view name ) noexcept +/// Return whether @name is compile-time metadata, never a runtime value: a typed attribute, or one of tags.scm's +/// metadata names (ElixirKeywords::metadata, #355). +bool elixirMetadataAttribute( std::string_view name, const ElixirKeywords& keywords ) noexcept { - constexpr std::string_view metadata[] = { "doc", "moduledoc", "typedoc", "spec", "impl", "behaviour", "behavior", "derive", "enforce_keys", - "dialyzer", "on_load", "before_compile", "after_compile", "after_verify", "optional_callbacks", "compile", "external_resource", "vsn", "file" }; - return elixirTypedAttribute( name ) || std::find( std::begin( metadata ), std::end( metadata ), name ) != std::end( metadata ); + return elixirTypedAttribute( name ) || std::find( keywords.metadata.begin(), keywords.metadata.end(), name ) != keywords.metadata.end(); } bool elixirLiveHole( TSNode hole, TSNode site, std::string_view src ) noexcept; @@ -156,7 +168,10 @@ bool elixirLiveHole( TSNode hole, TSNode site, std::string_view src ) noexcept; /// Decide whether a candidate definition or call capture represents supported executable Elixir syntax. /// role and name are non-null query captures into src. Reject quoted and dynamic syntax and metadata /// references; retain executable defaults and attributes while excluding declaration heads and patterns. -bool elixirKeepCapture( TSNode role, TSNode name, bool isDef, SymKind kind, std::string_view src ) noexcept +/// keywordHead: the match carried @_keyword, so tags.scm's predicates already decided it is a keyword-headed definition +/// (#any-of? on the keyword, #not-any-of? "unquote" "when" on the name). +bool elixirKeepCapture( TSNode role, TSNode name, bool isDef, SymKind kind, std::string_view src, bool keywordHead, + const ElixirKeywords& keywords ) noexcept { // Module attributes (notably @spec/@type) are not runtime call sites. // The shared inert-region decision runs after this capture's other syntax filters, @@ -166,7 +181,7 @@ bool elixirKeepCapture( TSNode role, TSNode name, bool isDef, SymKind kind, std: if( elixirLiveHole( parent, role, src ) ) { break; } // outer quoted declarations do not constrain the evaluated argument if( std::strcmp( ts_node_type( parent ), "unary_operator" ) == 0 && nodeFieldText( parent, NodeField::Operator, src ) == "@" ) { - if( elixirMetadataAttribute( elixirAttribute( parent, src ) ) + if( elixirMetadataAttribute( elixirAttribute( parent, src ), keywords ) || ts_node_eq( fieldChild( parent, NodeField::Operand ), role ) ) { return false; } } } @@ -175,7 +190,9 @@ bool elixirKeepCapture( TSNode role, TSNode name, bool isDef, SymKind kind, std: { if( const auto attribute = elixirAttribute( role, src ); !attribute.empty() ) { - return kind == SymKind::Struct ? elixirTypedAttribute( attribute ) : !elixirMetadataAttribute( attribute ); + // A metadata attribute (@doc, @spec, ...) never reaches here: tags.scm's #not-any-of? on @_attribute + // rejected its @definition.attribute match (#355). A typed one is the @definition.type pattern's. + return kind == SymKind::Struct ? elixirTypedAttribute( attribute ) : !elixirTypedAttribute( attribute ); } if( target == "test" && std::strcmp( ts_node_type( name ), "string" ) == 0 ) { @@ -197,7 +214,7 @@ bool elixirKeepCapture( TSNode role, TSNode name, bool isDef, SymKind kind, std: } // `defimpl` defines a real module (`Protocol.For`); ElixirContext supplies its full name. if( kind == SymKind::Other ) { return elixirModuleKeyword( target ) || target == "defimpl"; } - return elixirFunctionKeyword( target ) && nodeTextOf( name, src ) != "unquote" && nodeTextOf( name, src ) != "when"; + return keywordHead; // the keyword list and the unquote/when exclusion live in queries/elixir/tags.scm (#355) } const TSNode firstArg = elixirFirstArgument( role ); if( target == "test" && !ts_node_is_null( firstArg ) && std::strcmp( ts_node_type( firstArg ), "string" ) == 0 && !ts_node_is_null( elixirBody( role, src ) ) ) @@ -217,7 +234,7 @@ bool elixirKeepCapture( TSNode role, TSNode name, bool isDef, SymKind kind, std: } constexpr std::string_view special[] = { "defimpl", "defstruct", "defexception", "defoverridable", "alias", "import", "require", "use", "unquote", "unquote_splicing", "case", "cond", "for", "if", "unless", "with", "receive", "try" }; - if( elixirFunctionKeyword( target ) || elixirModuleKeyword( target ) + if( keywords.isFunction( target ) || elixirModuleKeyword( target ) || ( target == "quote" && !ts_node_is_null( elixirBody( role, src ) ) ) || std::find( std::begin( special ), std::end( special ), target ) != std::end( special ) ) { @@ -233,7 +250,7 @@ bool elixirKeepCapture( TSNode role, TSNode name, bool isDef, SymKind kind, std: inDefault = inDefault || ( !ts_node_is_null( value ) && ts_node_start_byte( role ) >= ts_node_start_byte( value ) && ts_node_end_byte( role ) <= ts_node_end_byte( value ) ); } - if( !elixirFunctionKeyword( elixirTarget( parent, src ) ) ) + if( !keywords.isFunction( elixirTarget( parent, src ) ) ) { continue; } @@ -367,6 +384,7 @@ struct ElixirContext HashMap> variables; HashMap modules; ExtractShortfall* shortfall = nullptr; // the file's sink, set by captureTagsFacts before any lookup + ElixirKeywords keywords; // tags.scm's definition keywords, read by prepare (#355) std::string scopeOf( TSNode node, unsigned depth = 0 ) const { @@ -531,7 +549,7 @@ struct ElixirContext { if( elixirLiveHole( p, node, src ) ) { return {}; } // patterns outside the live hole are inert syntax if( elixirNodeIs( p, "unary_operator" ) && nodeFieldText( p, NodeField::Operator, src ) == "^" ) { return {}; } - if( elixirFunctionKeyword( elixirTarget( p, src ) ) ) + if( keywords.isFunction( elixirTarget( p, src ) ) ) { TSNode head = elixirFirstArgument( p ); if( elixirNodeIs( head, "binary_operator" ) && nodeFieldText( head, NodeField::Operator, src ) == "when" ) @@ -617,10 +635,16 @@ struct ElixirContext std::uint32_t fileId, std::vector& binds, std::vector& includes, std::vector& refs ) { src = bytes; + keywords.function = predicateStrings( query, "any-of?", "_keyword" ); + keywords.metadata = predicateStrings( query, "not-any-of?", "_attribute" ); ts_query_cursor_exec( cursor, query, root ); TSQueryMatch match; while( ts_query_cursor_next_match( cursor, &match ) ) { + if( !passesStringPredicates( query, match, src ) ) + { + continue; // the same query, the same predicates: one meaning for a pattern in both passes (#355) + } for( std::uint16_t i = 0; i < match.capture_count; ++i ) { const auto& capture = match.captures[ i ]; @@ -848,10 +872,10 @@ std::uint32_t elixirHeadDefaultCount( TSNode node, std::string_view src ) noexce return defaults; } -void elixirDefinitionFacts( const RawDef& def, TSNode node, std::string_view src, std::vector& binds ) +void elixirDefinitionFacts( const RawDef& def, TSNode node, std::string_view src, std::vector& binds, const ElixirKeywords& keywords ) { const auto keyword = elixirTarget( node, src ); - if( !elixirFunctionKeyword( keyword ) ) { return; } + if( !keywords.isFunction( keyword ) ) { return; } RawBind bind; bind.fileId = def.fileId; bind.startByte = def.startByte; bind.lang = Lang::Elixir; bind.kind = LocalBindKind::ElixirCallable; bind.var = def.name; bind.typeName = def.scope; bind.importedName = keyword; diff --git a/src/ingest_importcap.h b/src/ingest_importcap.h index 31a5dcdbb..bcc54b81c 100644 --- a/src/ingest_importcap.h +++ b/src/ingest_importcap.h @@ -16,7 +16,7 @@ // dependencyDialect already groups languages by exactly that. Dependency RESOLUTION stays in // src/resolve.h and is untouched by this round. // -// WHAT IS STILL IN C++, AND WHY IT CANNOT LEAVE. Tags-pass predicates never run here (stated in +// WHAT IS STILL IN C++, AND WHY IT CANNOT LEAVE. Tags-pass predicates did not run before #355 here (stated in // queries/python/tags.scm, pinned by test/matchgrammarcheck.sh, and the reason `using namespace ns;` is // gated in ingest.cpp rather than excluded by a query). So every gate that needs to READ TEXT stays in // C++: which preprocessor directive a captured `preproc_call` argument belongs to, whether a callee is @@ -185,7 +185,7 @@ ImportSpec normaliseCFamilyImport( TSNode directive, std::string_view raw, std:: // gives all of them the same `source:` string, which is the whole capture; every other export_statement // has no `source:` and is not matched by the query at all. // call_expression — the CommonJS `require('x')` and the dynamic `import('x')`. A query cannot say "the -// callee's TEXT is require" (tags-pass predicates never run), so the three guards that keep an ordinary +// callee's TEXT is require" (tags-pass predicates did not run before #355), so the three guards that keep an ordinary // call out of the dependency graph stay C++: jsModuleLoadTarget (src/ingest_relations.h), UNCHANGED by // this round and shared with captureJsImportFacts, reads the callee text, the argument count and the // string. The query only says where a candidate call is; the guards say whether it is a module load. diff --git a/src/ingest_names.h b/src/ingest_names.h index 2b3aa45d4..9829f7547 100644 --- a/src/ingest_names.h +++ b/src/ingest_names.h @@ -86,8 +86,8 @@ inline std::string immediateScope( std::string_view full ) // explicit-template-argument call — so the template_function reference pattern matches every cast in the // tree (171 sites in this repo's src/ alone). A cast is not a call and must not mint a reference: it is // VALID INPUT, not a corrupt invariant, so the capture loop simply skips it (never ASSUME, never -// DISCLOSE — nothing degraded). Query predicates cannot do this: passesPredicates is wired into -// --match/--lint only, not the tags pass (measured — a `#not-eq?` left --uses=static_cast at 165). +// DISCLOSE — nothing degraded). Query predicates could not do this before #355: the tags pass ran +// none (measured — a `#not-eq?` left --uses=static_cast at 165). It runs the string ones now, so this can move into the query. inline bool isCppCastKeyword( std::string_view name ) noexcept { return name == "static_cast" || name == "reinterpret_cast" || name == "const_cast" || name == "dynamic_cast"; @@ -99,8 +99,8 @@ inline bool isCppCastKeyword( std::string_view name ) noexcept // grammar carries no case distinction, so an intrinsic tag would otherwise mint a phantom call edge // to a symbol that is never defined. react.dev's own rule is the filter: a user component's tag name // MUST start with an uppercase letter for JSX to treat it as one rather than an HTML tag, so a -// lower-case-first-letter name at this shape is never a component. Query predicates cannot do this: -// passesPredicates is wired into --match/--lint only, not the tags pass (same measured fact +// lower-case-first-letter name at this shape is never a component. Query predicates could not do this before +// #355, when the tags pass ran none (the same measured fact // isCppCastKeyword's note above relies on) — so, exactly like that cast filter, this is VALID INPUT, // skipped at capture time, never ASSUME/DISCLOSE. // @@ -136,7 +136,7 @@ inline bool isJsxIntrinsicTagIdentifier( TSNode roleNode, TSNode nameNode, std:: // `using enum E;` (C++20; re-exports the ENUMERATORS, not the named type, so an import row for the type // would over-claim). Both are VALID INPUT, skipped at capture time exactly like the cast keywords above: // the grammar puts the keyword in an anonymous child with no field name, which a tags-query pattern -// cannot negate (passesPredicates is wired into --match/--lint only, never the tags pass). +// cannot negate (and before #355 the tags pass ran no predicate that could test its text). inline bool usingDeclarationIsDirective( TSNode n ) noexcept { return !ts_node_is_null( firstChildOfType( n, "namespace" ) ) || !ts_node_is_null( firstChildOfType( n, "enum" ) ); @@ -1889,7 +1889,7 @@ inline bool isScreamingSnakeName( std::string_view name ) noexcept // Which languages' @definition.constant captures are gated on SCREAMING_SNAKE. These grammars' new // constant patterns (queries/*/tags.scm, r3 q10) structurally capture EVERY module-level binding of the // right shape — the name gate is what scopes extraction to settings modules / feature-flag tables -// instead of every literal. Enforced HERE because tags-pass predicates never run (#match? is wired into +// instead of every literal. Enforced HERE because tags-pass predicates did not run before #355 (#match? is wired into // --match/--lint only — measured; see the note in queries/cpp/tags.scm). Deliberately NOT gated: // Python (vendored upstream pattern, case-blind since import — existing behavior pinned by constcheck), // Go (const/var patterns predate this and Go constants are conventionally CamelCase), Rust (const_item/ @@ -1973,7 +1973,7 @@ inline std::string_view childTokenAmong( TSNode node, std::string_view src, cons // CUDA memory-space qualifier of a module-scope declaration ("" = none). The uninitialized-declaration // patterns in queries/cpp/tags.scm are STRUCTURAL and unconstrained on purpose — that query also // compiles against tree-sitter-cpp (.cpp/.h/.metal), which has no `__constant__` token, so naming it -// there would make ts_query_new reject the whole query; tags-pass predicates never run (measured; see +// there would make ts_query_new reject the whole query; tags-pass predicates did not run before #355 (measured; see // the cast-keyword note in tags.scm); and a `(type_qualifier)` child constraint cannot see `__device__` // anyway — tree-sitter-cuda parses `__constant__`/`__managed__` as NAMED type_qualifier nodes but // `__device__` as an ANONYMOUS token child of the declaration. The qualifier test therefore lives here, @@ -2010,7 +2010,7 @@ inline bool declarationCarriesConstQualifier( TSNode declNode, std::string_view // The keep decision for the class-static-constant field_declaration captures (queries/cpp/tags.scm, // module-constant round). The pattern is deliberately loose — it matches EVERY default-member- -// initializer, because tags-pass predicates never run and static/constexpr child order is free — so +// initializer, because tags-pass predicates did not run before #355 and static/constexpr child order is free — so // this is where the real contract lives: keep iff the field carries BOTH a `static` // storage_class_specifier AND a const/constexpr/constinit type_qualifier. That keeps // `static constexpr int kMaxDepth = 3;` case-blind (one per-class constant, the census target) and @@ -2024,7 +2024,7 @@ inline bool fieldConstantCaptureKept( TSNode fieldDeclNode, std::string_view src } // The keep decision for the @definition.field captures (member-variable round, card A3; queries/cpp, -// queries/c, queries/python tags.scm). The patterns are LOOSE on purpose — tags-pass predicates never run +// queries/c, queries/python tags.scm). The patterns are LOOSE on purpose — tags-pass predicates did not run before #355 // — so the contract lives here, per language: // C/C++ : keep iff the field_declaration carries NO `static` storage_class_specifier. A class-static // CONSTANT keeps its own @definition.constant row (t="var", fieldConstantCaptureKept); a mutable @@ -2331,7 +2331,7 @@ inline bool dropConstantCapture( Lang lang, std::string_view name, TSNode nameNo // YAML's @definition.yamlkey gate — the yaml tier's one in-C++ predicate (see queries/yaml/tags.scm's // header for why the depth cut cannot live in the query: sequence nesting between a pair and its -// document is unbounded, so no finite pattern set expresses it, and tags-pass predicates never run). +// document is unbounded, so no finite pattern set expresses it, and tags-pass predicates did not run before #355). // A mapping key is a symbol iff its MAPPING depth is <= 2 — block and flow mappings counted alike // (flow is a presentation style of the same mapping node), sequences counted NOT AT ALL (sequence // transparency: 25.3% of real keys sit directly inside a sequence element — the steps:/containers:/ @@ -2370,7 +2370,7 @@ inline bool yamlKeyCaptureDropped( std::string_view name, TSNode roleNode ) noex // test/pyshapecheck.sh) drops when the enclosing class's base NAME is not an enum family; // @definition.cjsexport / @definition.protomethod (the JS shape round, test/jsshapecheck.sh) drop when // the LEFT side is not really exports/module.exports/.prototype. — the query captures every `a.b = fn` -// shape and cannot text-test, because tags-pass predicates never run (see constCaptureNeedsScreamingGate +// shape and cannot text-test, because tags-pass predicates did not run before #355 (see constCaptureNeedsScreamingGate // above). inline bool dropGatedCapture( std::string_view defCapSv, Lang lang, std::string_view name, TSNode nameNode, TSNode roleNode, std::string_view src ) // NOT noexcept: the class-factory gate's scope lookup allocates { diff --git a/src/ingest_sidecap.h b/src/ingest_sidecap.h index b9b5df0cf..285d0f8d9 100644 --- a/src/ingest_sidecap.h +++ b/src/ingest_sidecap.h @@ -1878,6 +1878,13 @@ void captureTagsFacts( TSQueryCursor* cursor, const LangEntry& le, std::uint32_t TSQueryMatch match; while( ts_query_cursor_next_match( cursor, &match ) ) { + // #355: a pattern's string predicates (#eq?/#any-of?/#not-any-of?) are part of the pattern. A match one of + // them rejects never existed, so it is skipped before any capture is read or counted. #match? is not run + // here (src/infra/tspredicates.h says why); a pattern without predicates pays one empty step-array lookup. + if( !passesStringPredicates( query, match, src ) ) + { + continue; + } // A tags pattern yields one @definition/@reference node + a child @name. // Walk this match's captures: remember the role node + the name text. SymKind kind = SymKind::Other; @@ -1894,6 +1901,7 @@ void captureTagsFacts( TSQueryCursor* cursor, const LangEntry& le, std::uint32_t TSNode nameNode {}; // the @name identifier node — for C++ scope/qualifier (E#4) TSNode importNamesNode {}; // Python: one clause's target, paired with its optional alias TSNode importAliasNode {}; + bool keywordHead = false; // #355: the pattern carried @_keyword, a head its tags.scm predicates accepted for( uint16_t ci = 0; ci < match.capture_count; ++ci ) { @@ -1902,6 +1910,10 @@ void captureTagsFacts( TSQueryCursor* cursor, const LangEntry& le, std::uint32_t uint32_t nameLen = 0; const char* capName = ts_query_capture_name_for_id( query, cap.index, &nameLen ); const std::string_view capSv( capName, nameLen ); + if( capSv == "_keyword" ) + { + keywordHead = true; // a predicate-only capture: roleOf ignores it, the definition keyword is its pattern's #any-of? + } SymKind k = SymKind::Other; switch( roleOf( capSv, k ) ) @@ -2031,7 +2043,7 @@ void captureTagsFacts( TSQueryCursor* cursor, const LangEntry& le, std::uint32_t continue; } - if( le.lang == Lang::Elixir && !elixirKeepCapture( roleNode, nameNode, isDef, kind, src ) ) + if( le.lang == Lang::Elixir && !elixirKeepCapture( roleNode, nameNode, isDef, kind, src, keywordHead, elixir.keywords ) ) { continue; } @@ -2367,7 +2379,7 @@ void captureTagsFacts( TSQueryCursor* cursor, const LangEntry& le, std::uint32_t else if( le.lang == Lang::Elixir ) { d.scope = kind == SymKind::Other ? std::string{} : elixir.scopeOf( roleNode ); - if( elixirFunctionKeyword( elixirTarget( roleNode, src ) ) ) + if( keywordHead ) // #355: the match's own @_keyword capture, decided by tags.scm, not a C++ keyword list { d.name = elixirFunctionName( nameTxt, d.params ); // B2.2 for Elixir: a clause's arity IS its parameter count (no variadic form), so `params` @@ -2376,7 +2388,7 @@ void captureTagsFacts( TSQueryCursor* cursor, const LangEntry& le, std::uint32_t // unflaggable by --edit-check (test/elixirnamearitycheck.sh arm C: run(x) -> run(x, y) // must flag run(x) callers; run(x, y \\ 1) must flag none). d.arityExact = elixirHeadDefaultCount( roleNode, src ) == 0 ? std::uint8_t( 1 ) : std::uint8_t( 0 ); - elixirDefinitionFacts( d, roleNode, src, binds ); + elixirDefinitionFacts( d, roleNode, src, binds, elixir.keywords ); if( elixirTarget( roleNode, src ) == "defdelegate" ) { RawRef delegated; diff --git a/src/quality.h b/src/quality.h index fcb61a615..67c818b45 100644 --- a/src/quality.h +++ b/src/quality.h @@ -2337,7 +2337,9 @@ inline std::string cacheRootKeyHex( const std::string& root ) // not include this header; it relies on ingest.cpp including quality.h (line 13) before ingest_cache.h, and a reorder // that broke that fails the build on the undeclared name rather than passing. constexpr std::uint32_t kIngestCacheVersionMirror = 36; // MUST equal ingest.cpp's kCacheVersion (gated); 36 = train 30 (above call-result-typing's and fcall-forward's 33, name-template-dispatch's 35, main's 34), 34 = train 29 (above lane arity-min's and call-result-typing's 33), 33 = train 29 lane arity-min: def record paramsMin (the train renumbers), 33 = lane call-result-typing: RawBind resultIndex + unwrapDepth, 33 = lane fcall-forward: ref `forwarded` byte, 35 = lane name-template-dispatch: RefRole Pattern 11, 32 = train 28: RefRole Subscript 9 + Inert 10, 31 = train 27: lego-transitive's inherit-ref baseBind (the lane's 30), 29 = FE-B ref memberPath/memberCtor + 4 bind kinds, 28 = FE-A ref memberCall/memberRoot, 27 = corrected fnScope values, 26 = function-local def scope span (25 = #157 + #150) -constexpr std::uint32_t kIngestParserVerMirror = 206; // MUST equal ingest.cpp's kParserVer (gated) +constexpr std::uint32_t kIngestParserVerMirror = 216; // MUST equal ingest.cpp's kParserVer (gated) + // 216 = 2026-10-22 (#355, a placeholder above main 206/207, PR #424 212/213, + // PR #420 214/215; see the kParserVer note) // 206 = 2026-10-18 (#409, PR #414, renumbered again at merge: above main 196/197, // train 31 198/199, rust-receivers 202/203, ix1-slot 204/205; see the kParserVer note) // 198 = 2026-10-18 (train 31: above main 196/197 and h1-annot diff --git a/test/qschemetrip.hash b/test/qschemetrip.hash index 2ccfe2fbb..a3089548f 100644 --- a/test/qschemetrip.hash +++ b/test/qschemetrip.hash @@ -1 +1 @@ -a667ffc55eff871ff6ea06c1068cc5c73c8fe06b5a582beb87dae733548f8a05 +f224102f2c22c97c6496f873d3455bf83aa796863cc736ad988a90fe57b484ad diff --git a/test/regression.sh b/test/regression.sh index 621533b82..a6a5f93c3 100755 --- a/test/regression.sh +++ b/test/regression.sh @@ -278,7 +278,7 @@ else fi # retired: forspinecheck — the --for caller spine (Change A) was dropped by its own offline kill rule: a spine of up to 2 caller hops from the top 3 seeds named a missed rubric item on 1 of 7 in-sample rows, so the gate that pinned it went with it # retired: cacheexclkeycheck — the per-configuration auto-cache key it pinned is a registered NEGATIVE (docs/EVALS.md, "The auto-cache key ignores --exclude", RUN 2026-09-03: a 158K-file root with >= 12 gate configurations thrashed the 2 GiB sweep); the retry design keeps ONE superset blob per root and will bring its own gate -for _g in a9disclosurecheck abicheck accessshapecheck ackonlycheck adaptivecheck adaptivecutshapecheck affectedcheck agentloopclaudecheck agentloopcodexcheck agentloopeditsuitecheck agentloopfollowupcheck agentloopgradercheck agentlooplockcheck agentloopopencodecheck agentsurfacecheck agenttablecheck aiderbytescheck anchorbodycheck anchorcheck archcheck archmetricscheck argvdiffcheck arisefollowupcheck ariseshimcheck aritycheck artifactcheck astqueryregexcheck astrocheck atcheck atomscheck attribmultidefcheck attrvocabcheck baselinecheck baselinedirtycheck baselineportcheck bashsourcecheck batchcheck binoverridecheck blindspotcheck bm25boundcheck bm25check bodiesshowncheck bodydialectcheck budgetpolicycheck buildtypestampcheck builtinbindcheck bundleidcheck cachefuzzcheck cachehashcheck cacheidentitycheck cacheisolationcheck cachelintcheck cacheoffsetcheck cachereservecheck cachesplitcheck callerscheck callformcheck callsrankordercheck candheadcheck candidatescheck canoncheck capdisclosurecheck capsweepcheck ccheck ccjsoncheck ceilingverdictcheck chacheck chaconecheck chainguardcheck chainidcheck childwalkscalecheck churndecaycheck churnjoincheck churnjsonstampcheck claudeconfigdircheck clicheck clojurecheck clonebandcheck clonecachecheck clonededupcheck cloneidiomcheck clonelexcheck clsrecvcheck cochangeboostcheck cochangecliocheck cochangesurprisecheck codexinstallhonestycheck codexplugincheck codexwrapcheck collectioncapcheck columnarattrcheck columnarcheck columnarcommacheck commentcoherencecheck commenttokencheck communitydrillcheck communitylabelcheck compactlegendcheck compactroutecheck completecheck composelangcheck connectcheck connectcorecheck connectjoincheck constcheck contextratiocheck coplintcheck countfloorcheck cppbenchcheck cppoperatorcheck cppqualcheck cpptmplscopecheck crashsweepcheck crawlescapecheck crossdirincludecheck crossrefcheck crossrefdegradecheck csharpcheck csharpcondcheck cudacheck cyclecutcheck danglingwarncheck dartcheck deadcheck deadfiltercheck deadprecisioncheck deckcheck deckclaimcheck declinecheck declinedlistcheck declinedsitecheck decltodefcheck deeptailcheck defaultceilingcheck defectshapecheck defoverdeclcheck degradedhintcheck dependencypincheck deplangscheck depsprecisecheck detailcheck diagnoticecheck didyoumeancheck dispatchordercheck dmmcheck docanchorcheck docdemotecheck docdriftcheck docdriftcommentcheck docmdcachecheck docmentioncheck docscommandscheck doctorcheck donelegendcheck droppedpositivecheck duprowcheck dynmapsimdcheck editcheckanswercheck editcheckcheck editcheckdeclcheck editchecknotecheck edithandlehintcheck editpayloadbinarycheck editplancheck editplanpayloadconfinecheck editplanrecheckcheck editplanrollbackmsgcheck editpreviewcheck editroundtripcheck edittargetfileabscheck eliximportcheck elixircheck elixirnamearitycheck elixirsemanticcheck emitescapecheck emittertruthcheck emptycorpuscheck emptyvaluerefusecheck ensembleavailcheck ensemblecheck enumtablecheck essentialcxcheck estchargecheck evalcheck evictioncheck exemplarcheck exemplarconfcheck exercisescheck expandbodyfirstcheck expandcallscheck expandmodecheck expandrangecheck expandsibscheck expandtokencheck expandtopk0check extentcheck externalvetocheck falseedgecheck fcallforwardcheck fficheck fieldaffinitycheck fieldidcheck fieldnarrowcheck fieldusescheck filerootcheck fileselectorrefusecheck fillordercheck fixedbufsweep flagscheck flagsnoisecheck flagsurfacecheck flagtablecheck flipcheck floormarkcheck fnliteralcheck fnptrcheck forautobodycheck forblowupcheck forbodyfoldcheck forbodyslotcheck forbudgetmonotoncheck forcalibfactscheck forcompresscheck fordisclosurecheck forhdrshapecheck forheadrrcheck forlenscheck formatgatecheck formaxtokenscheck fornotesbudgetcheck fornotesjsoncheck forrankordercheck forrootlegendcheck forsectioncollapsecheck forsigspancheck forsoftbudgetcheck forwidencheck freshclonecheck freshnesscheck g1configcheck gateabilitycheck gatecountcheck gateexitcheck gdscriptcheck genrecallcheck gitchildoptscheck gitenvhermeticcheck githardencheck gitignorecheck gitquotepathcheck gitstampcheck goinstcheck gointerfacecheck gokindcheck goscopecheck graphlegendbudgetcheck graphqueryrefusecheck grepanchorcheck grepandcheck grepbytescheck grepcheck grepcontextcheck grepcorpuscheck grepfastcheck grepfollowupcheck grepignorecheck grepscancheck grepseamcheck greptiercheck guardmsgcheck hasacheck hazardpatterncheck headbinstagecheck headsnapcachecheck helpbudgetcheck hermesinstallcheck historyoraclecheck hookcheck hostilecheck hotspotsincecheck htmlcolorcheck htmlhostcheck htmlrendercheck identitycheck impactdepthcheck impactimportcheck impactpartitioncheck impacttypecheck importcapcheck importnarrowcheck includeanglecheck includeprecisecheck indexoutcheck infraportcheck isolateprovenancecheck javamethodrefcheck javarubycheck jslangcheck jsmetricscheck jsnestedcheck jsoncheck jsonlangcheck jsonparitycheck jsonredactcheck jsonrefusallegendcheck jsonwalkcheck jsshapecheck jsverbscheck jsxcallcheck knownitemcheck kotlincheck landingcheck langcensuscheck langcheck langidentitycheck layerquerycheck layoutcheck lb3namecheck legendcostcheck legendcoveragecheck legenddriftcheck legendrefcheck legobundlecheck legocheck legotransitivecheck liftdisclosurecheck limitstablecheck lintbudgetcheck lintcatalogcheck lintcheck lintdedupcheck lintpayloadcapcheck lintprecisioncheck lintrulescheck lintscopecheck lintselectcheck listingpagingcheck localdeclcheck localitycheck localscountcheck loopconservationcheck lpincheck luacheck luarequirecheck macroedgecheck macroreparsecheck manifestcheck mapdatasectioncheck mapdiffcheck mapinventorycheck matchcapturecheck matchgrammarcheck maxfilesizecheck mcpattrparitycheck mcpaudit4hardencheck mcpclidiffcheck mcpcodexmetacheck mcpcontractcheck mcpdegradedhintcheck mcpeditcheck mcpeditkindcheck mcpeditmodecheck mcpeditpresencecheck mcpeditracecheck mcpflagshipcheck mcpforparitycheck mcpframehonestycheck mcpgrepdegradedcheck mcphandlecheck mcpincrementalcheck mcpmanifestcheck mcprangeedgecheck mcpreadloopcheck mcpredactcheck mcpreloadasancheck mcpreloadcheck mcpremotecheck mcprobustcheck mcpslicecheck mcpstalecheck mcpstdiolinecapcheck mcpstrictschemacheck mcptoolprunecheck mcptoolsubsetcheck mcptranchecheck mcptwinclaimscheck mcpverbscheck mcpw2fixcheck mcpw3fixcheck mcpwatchercheck mdembedcheck mdsectioncheck memguardcheck mentioncapcheck mentioncheck mentionsverbcheck mergechurncheck mergescoutcheck mergescoutlonglinecheck metalcheck meterdisclosurecheck metricscheck modifierguardcheck moduleconstcheck morecontractcheck mrowalkcheck multirootcheck multiswecheck namedfileinputcheck nameinfocheck namingcalibrationcheck namingconsistencycheck naminglenscheck naminglocalscheck narrowcheck narrowlangcheck neighbourcapcheck nestedimportcheck nestedqualcheck nestprofilecheck nextverbcheck noaliascheck nodekindcheck nongitqmetricscheck nonlocalstatecheck notecanoncheck notescheck notesdegradecheck nsfiltercheck nulbytecheck numericrefusecheck objcfieldcheck objcsniffcheck opencodewrapcheck optremarkscheck optremarkshotcheck ordercheck osswitchcheck oswin32logiccheck outlinecheck overbudgetcommentcheck ownerhopcheck ownerscheck packcallersharecheck packtaskcheck packtaskmonotoncheck packtaskquotacheck padscalecheck paginationcheck pagingsweepcheck panellegendcheck pargatescheck parsehealthcheck partitioncheck pathgapcheck pathownercheck patterncheck perfharnesscheck phpcheck pincensuscheck planlanescheck planlintcheck pmccheck portablebuildcheck portablecachecheck postingscheck ppaltcheck ppdeadrolescheck pranchorcheck prbudgetcheck prcheck prcontextcheck prconvergecheck precedencecheck preproccondcheck preprocdeadscalecheck prmaskanchorcheck prnestedcapcheck probecheck propcostcheck prrefsafecheck prrenamecheck pyimportprecisecheck pymodulealiascheck pyshapecheck qackconcurrencycheck qackorigincheck qbaselineproducercheck qchurncheck qchurnmemocheck qddegradecheck qddialscheck qdrefpaircheck qextractionkeycheck qoriginoraclecheck qrevtokencheck qrowlocatorcheck qschemetripcheck qsnapcachecheck qsnapprefetchcheck qsnapproducercheck qualifiedresolvecheck qualitycheck qualitycrosslangcheck qualityexcludecheck qualitykeycheck qualitykindscheck qualityorigincheck qualitypanelcheck qualityscopecheck qualitysignalcheck qualitystalecheck qualitysymcheck qualnewcheck querycheck queryfilescancheck racymtimecheck radixsimdcheck rangecomposecheck rankbycheck reachcheck readabilitycheck readmedriftcheck readmeexamplecheck recallanchorcheck recallboundarycheck recallbudgetcheck recallbufcheck recallevalcheck recallparitycheck recallpassagecheck recallrankdepthcheck recallrelcheck recallshapecheck recalltablecheck recalltotalcheck receiptpostcheck receiverevidencecheck recentscopecheck redactcheck redactfixcheck refusaltailcheck regexbombcheck regexcheck regexguardcheck regexrefusecheck registermacrocheck relevancefloorcheck relinkcheck reportcheck resolvecheck resolverhonestycheck retrievalqualitycheck reusefirstworkflowcheck ripwirepubliccheck rootrelcheck rootrelemitcheck rootspellingcheck routecheck routeedgecheck routehookcheck routeoncecheck routingreportcheck rubyargcheck rubyattrscheck rubybarecallcheck rubyclassrecvcheck rubyconstcheck rubydeclrefcheck rubydescribedclasscheck rubyinheritcheck rubymetricscheck rubyrakejbuildercheck rubyreachcheck rubyrecvcheck rubyrecvnarrowcheck rubyrequirecheck rubyrspectargetcheck rubyschemacheck rubyscopecheck rubysettercheck rubytypedrecvcheck runhintcheck runtracecheck rustanccheck rustimportprecisecheck rustqualcheck safedeletecheck sarifcheck savecachecheck scipcheck scipjoincheck scorecardcheck scoutheadconflictcheck scoutkeycheck scroundtripcheck seedboundscheck selectorchaincheck selectorhonestycheck selectorrefusecheck selectorscopecheck selfcheckcheck selfcontainedcheck shadowcheck shallowhistorycheck shapingflagcheck shellgateindexcheck showcasecapturecheck sibliftcheck sidecarsymlinkcheck sigredactcheck sincecheck sincecochangecheck sincewindowcheck singledefcheck situdiffcheck situshapecheck skilldescbudgetcheck skillevalcheck skillevalsplitcheck skillinstallcheck skillroutingjudgedcheck skillscanreadcheck skilltruthcheck skipclassifycheck skippedcheck skipreasoncheck slicecheck slicediffcheck sliceflowcheck sliceflowsenscheck spectimingcheck staleackcheck statgatecheck stdmembergatecheck stdqualcheck strkerncheck structlayoutcheck sublistcountcheck subscriptopcheck substrfiltercheck subtokencheck svectorcheck swiftcheck swiftmemberscheck swiftshapecheck taskechocheck tempfilesymlinkcheck termmargincheck testedreachcheck testgatecheck testgatelegendbudgetcheck testgatepagecheck testgaterefusecheck testmacrocheck testrowruncheck testscopecheck textdocscheck timsortcheck tokenbudgetcheck tomllangcheck toolcallroutecheck tornreadcheck traceasanlinearcheck tracecheck tracehandoffcapcheck tracehopcheck traceminecheck treecheck truncvocabcheck tsimportprecisecheck tsshapecheck type3check type3clonecheck typerefcheck unreachablecheck unresolvedcheck usescheck usesliteralcheck usesselectorcheck usingdeclcheck utf8scrubcheck vendoredassetcheck vendoredbundlecheck vendorpatchcheck verifycheck versioncheck w2verbscheck w3fixbudgetcheck w3fixlegendcheck weaksignalcheck withgraphcheck withprofilecheck worktreeleakcheck wrapverbscheck writetargetcheck xmlwellformed yamllangcheck zonecheck zoneconsistencycheck zoomcheck; do +for _g in a9disclosurecheck abicheck accessshapecheck ackonlycheck adaptivecheck adaptivecutshapecheck affectedcheck agentloopclaudecheck agentloopcodexcheck agentloopeditsuitecheck agentloopfollowupcheck agentloopgradercheck agentlooplockcheck agentloopopencodecheck agentsurfacecheck agenttablecheck aiderbytescheck anchorbodycheck anchorcheck archcheck archmetricscheck argvdiffcheck arisefollowupcheck ariseshimcheck aritycheck artifactcheck astqueryregexcheck astrocheck atcheck atomscheck attribmultidefcheck attrvocabcheck baselinecheck baselinedirtycheck baselineportcheck bashsourcecheck batchcheck binoverridecheck blindspotcheck bm25boundcheck bm25check bodiesshowncheck bodydialectcheck budgetpolicycheck buildtypestampcheck builtinbindcheck bundleidcheck cachefuzzcheck cachehashcheck cacheidentitycheck cacheisolationcheck cachelintcheck cacheoffsetcheck cachereservecheck cachesplitcheck callerscheck callformcheck callsrankordercheck candheadcheck candidatescheck canoncheck capdisclosurecheck capsweepcheck ccheck ccjsoncheck ceilingverdictcheck chacheck chaconecheck chainguardcheck chainidcheck childwalkscalecheck churndecaycheck churnjoincheck churnjsonstampcheck claudeconfigdircheck clicheck clojurecheck clonebandcheck clonecachecheck clonededupcheck cloneidiomcheck clonelexcheck clsrecvcheck cochangeboostcheck cochangecliocheck cochangesurprisecheck codexinstallhonestycheck codexplugincheck codexwrapcheck collectioncapcheck columnarattrcheck columnarcheck columnarcommacheck commentcoherencecheck commenttokencheck communitydrillcheck communitylabelcheck compactlegendcheck compactroutecheck completecheck composelangcheck connectcheck connectcorecheck connectjoincheck constcheck contextratiocheck coplintcheck countfloorcheck cppbenchcheck cppoperatorcheck cppqualcheck cpptmplscopecheck crashsweepcheck crawlescapecheck crossdirincludecheck crossrefcheck crossrefdegradecheck csharpcheck csharpcondcheck cudacheck cyclecutcheck danglingwarncheck dartcheck deadcheck deadfiltercheck deadprecisioncheck deckcheck deckclaimcheck declinecheck declinedlistcheck declinedsitecheck decltodefcheck deeptailcheck defaultceilingcheck defectshapecheck defoverdeclcheck degradedhintcheck dependencypincheck deplangscheck depsprecisecheck detailcheck diagnoticecheck didyoumeancheck dispatchordercheck dmmcheck docanchorcheck docdemotecheck docdriftcheck docdriftcommentcheck docmdcachecheck docmentioncheck docscommandscheck doctorcheck donelegendcheck droppedpositivecheck duprowcheck dynmapsimdcheck editcheckanswercheck editcheckcheck editcheckdeclcheck editchecknotecheck edithandlehintcheck editpayloadbinarycheck editplancheck editplanpayloadconfinecheck editplanrecheckcheck editplanrollbackmsgcheck editpreviewcheck editroundtripcheck edittargetfileabscheck eliximportcheck elixircheck elixirnamearitycheck elixirsemanticcheck emitescapecheck emittertruthcheck emptycorpuscheck emptyvaluerefusecheck ensembleavailcheck ensemblecheck enumtablecheck essentialcxcheck estchargecheck evalcheck evictioncheck exemplarcheck exemplarconfcheck exercisescheck expandbodyfirstcheck expandcallscheck expandmodecheck expandrangecheck expandsibscheck expandtokencheck expandtopk0check extentcheck externalvetocheck falseedgecheck fcallforwardcheck fficheck fieldaffinitycheck fieldidcheck fieldnarrowcheck fieldusescheck filerootcheck fileselectorrefusecheck fillordercheck fixedbufsweep flagscheck flagsnoisecheck flagsurfacecheck flagtablecheck flipcheck floormarkcheck fnliteralcheck fnptrcheck forautobodycheck forblowupcheck forbodyfoldcheck forbodyslotcheck forbudgetmonotoncheck forcalibfactscheck forcompresscheck fordisclosurecheck forhdrshapecheck forheadrrcheck forlenscheck formatgatecheck formaxtokenscheck fornotesbudgetcheck fornotesjsoncheck forrankordercheck forrootlegendcheck forsectioncollapsecheck forsigspancheck forsoftbudgetcheck forwidencheck freshclonecheck freshnesscheck g1configcheck gateabilitycheck gatecountcheck gateexitcheck gdscriptcheck genrecallcheck gitchildoptscheck gitenvhermeticcheck githardencheck gitignorecheck gitquotepathcheck gitstampcheck goinstcheck gointerfacecheck gokindcheck goscopecheck graphlegendbudgetcheck graphqueryrefusecheck grepanchorcheck grepandcheck grepbytescheck grepcheck grepcontextcheck grepcorpuscheck grepfastcheck grepfollowupcheck grepignorecheck grepscancheck grepseamcheck greptiercheck guardmsgcheck hasacheck hazardpatterncheck headbinstagecheck headsnapcachecheck helpbudgetcheck hermesinstallcheck historyoraclecheck hookcheck hostilecheck hotspotsincecheck htmlcolorcheck htmlhostcheck htmlrendercheck identitycheck impactdepthcheck impactimportcheck impactpartitioncheck impacttypecheck importcapcheck importnarrowcheck includeanglecheck includeprecisecheck indexoutcheck infraportcheck isolateprovenancecheck javamethodrefcheck javarubycheck jslangcheck jsmetricscheck jsnestedcheck jsoncheck jsonlangcheck jsonparitycheck jsonredactcheck jsonrefusallegendcheck jsonwalkcheck jsshapecheck jsverbscheck jsxcallcheck knownitemcheck kotlincheck landingcheck langcensuscheck langcheck langidentitycheck layerquerycheck layoutcheck lb3namecheck legendcostcheck legendcoveragecheck legenddriftcheck legendrefcheck legobundlecheck legocheck legotransitivecheck liftdisclosurecheck limitstablecheck lintbudgetcheck lintcatalogcheck lintcheck lintdedupcheck lintpayloadcapcheck lintprecisioncheck lintrulescheck lintscopecheck lintselectcheck listingpagingcheck localdeclcheck localitycheck localscountcheck loopconservationcheck lpincheck luacheck luarequirecheck macroedgecheck macroreparsecheck manifestcheck mapdatasectioncheck mapdiffcheck mapinventorycheck matchcapturecheck matchgrammarcheck maxfilesizecheck mcpattrparitycheck mcpaudit4hardencheck mcpclidiffcheck mcpcodexmetacheck mcpcontractcheck mcpdegradedhintcheck mcpeditcheck mcpeditkindcheck mcpeditmodecheck mcpeditpresencecheck mcpeditracecheck mcpflagshipcheck mcpforparitycheck mcpframehonestycheck mcpgrepdegradedcheck mcphandlecheck mcpincrementalcheck mcpmanifestcheck mcprangeedgecheck mcpreadloopcheck mcpredactcheck mcpreloadasancheck mcpreloadcheck mcpremotecheck mcprobustcheck mcpslicecheck mcpstalecheck mcpstdiolinecapcheck mcpstrictschemacheck mcptoolprunecheck mcptoolsubsetcheck mcptranchecheck mcptwinclaimscheck mcpverbscheck mcpw2fixcheck mcpw3fixcheck mcpwatchercheck mdembedcheck mdsectioncheck memguardcheck mentioncapcheck mentioncheck mentionsverbcheck mergechurncheck mergescoutcheck mergescoutlonglinecheck metalcheck meterdisclosurecheck metricscheck modifierguardcheck moduleconstcheck morecontractcheck mrowalkcheck multirootcheck multiswecheck namedfileinputcheck nameinfocheck namingcalibrationcheck namingconsistencycheck naminglenscheck naminglocalscheck narrowcheck narrowlangcheck neighbourcapcheck nestedimportcheck nestedqualcheck nestprofilecheck nextverbcheck noaliascheck nodekindcheck nongitqmetricscheck nonlocalstatecheck notecanoncheck notescheck notesdegradecheck nsfiltercheck nulbytecheck numericrefusecheck objcfieldcheck objcsniffcheck opencodewrapcheck optremarkscheck optremarkshotcheck ordercheck osswitchcheck oswin32logiccheck outlinecheck overbudgetcommentcheck ownerhopcheck ownerscheck packcallersharecheck packtaskcheck packtaskmonotoncheck packtaskquotacheck padscalecheck paginationcheck pagingsweepcheck panellegendcheck pargatescheck parsehealthcheck partitioncheck pathgapcheck pathownercheck patterncheck perfharnesscheck phpcheck pincensuscheck planlanescheck planlintcheck pmccheck portablebuildcheck portablecachecheck postingscheck ppaltcheck ppdeadrolescheck pranchorcheck prbudgetcheck prcheck prcontextcheck prconvergecheck precedencecheck preproccondcheck preprocdeadscalecheck prmaskanchorcheck prnestedcapcheck probecheck propcostcheck prrefsafecheck prrenamecheck pyimportprecisecheck pymodulealiascheck pyshapecheck qackconcurrencycheck qackorigincheck qbaselineproducercheck qchurncheck qchurnmemocheck qddegradecheck qddialscheck qdrefpaircheck qextractionkeycheck qoriginoraclecheck qrevtokencheck qrowlocatorcheck qschemetripcheck qsnapcachecheck qsnapprefetchcheck qsnapproducercheck qualifiedresolvecheck qualitycheck qualitycrosslangcheck qualityexcludecheck qualitykeycheck qualitykindscheck qualityorigincheck qualitypanelcheck qualityscopecheck qualitysignalcheck qualitystalecheck qualitysymcheck qualnewcheck querycheck queryfilescancheck racymtimecheck radixsimdcheck rangecomposecheck rankbycheck reachcheck readabilitycheck readmedriftcheck readmeexamplecheck recallanchorcheck recallboundarycheck recallbudgetcheck recallbufcheck recallevalcheck recallparitycheck recallpassagecheck recallrankdepthcheck recallrelcheck recallshapecheck recalltablecheck recalltotalcheck receiptpostcheck receiverevidencecheck recentscopecheck redactcheck redactfixcheck refusaltailcheck regexbombcheck regexcheck regexguardcheck regexrefusecheck registermacrocheck relevancefloorcheck relinkcheck reportcheck resolvecheck resolverhonestycheck retrievalqualitycheck reusefirstworkflowcheck ripwirepubliccheck rootrelcheck rootrelemitcheck rootspellingcheck routecheck routeedgecheck routehookcheck routeoncecheck routingreportcheck rubyargcheck rubyattrscheck rubybarecallcheck rubyclassrecvcheck rubyconstcheck rubydeclrefcheck rubydescribedclasscheck rubyinheritcheck rubymetricscheck rubyrakejbuildercheck rubyreachcheck rubyrecvcheck rubyrecvnarrowcheck rubyrequirecheck rubyrspectargetcheck rubyschemacheck rubyscopecheck rubysettercheck rubytypedrecvcheck runhintcheck runtracecheck rustanccheck rustimportprecisecheck rustqualcheck safedeletecheck sarifcheck savecachecheck scipcheck scipjoincheck scorecardcheck scoutheadconflictcheck scoutkeycheck scroundtripcheck seedboundscheck selectorchaincheck selectorhonestycheck selectorrefusecheck selectorscopecheck selfcheckcheck selfcontainedcheck shadowcheck shallowhistorycheck shapingflagcheck shellgateindexcheck showcasecapturecheck sibliftcheck sidecarsymlinkcheck sigredactcheck sincecheck sincecochangecheck sincewindowcheck singledefcheck situdiffcheck situshapecheck skilldescbudgetcheck skillevalcheck skillevalsplitcheck skillinstallcheck skillroutingjudgedcheck skillscanreadcheck skilltruthcheck skipclassifycheck skippedcheck skipreasoncheck slicecheck slicediffcheck sliceflowcheck sliceflowsenscheck spectimingcheck staleackcheck statgatecheck stdmembergatecheck stdqualcheck strkerncheck structlayoutcheck sublistcountcheck subscriptopcheck substrfiltercheck subtokencheck svectorcheck swiftcheck swiftmemberscheck swiftshapecheck tagspredicatecheck taskechocheck tempfilesymlinkcheck termmargincheck testedreachcheck testgatecheck testgatelegendbudgetcheck testgatepagecheck testgaterefusecheck testmacrocheck testrowruncheck testscopecheck textdocscheck timsortcheck tokenbudgetcheck tomllangcheck toolcallroutecheck tornreadcheck traceasanlinearcheck tracecheck tracehandoffcapcheck tracehopcheck traceminecheck treecheck truncvocabcheck tsimportprecisecheck tsshapecheck type3check type3clonecheck typerefcheck unreachablecheck unresolvedcheck usescheck usesliteralcheck usesselectorcheck usingdeclcheck utf8scrubcheck vendoredassetcheck vendoredbundlecheck vendorpatchcheck verifycheck versioncheck w2verbscheck w3fixbudgetcheck w3fixlegendcheck weaksignalcheck withgraphcheck withprofilecheck worktreeleakcheck wrapverbscheck writetargetcheck xmlwellformed yamllangcheck zonecheck zoneconsistencycheck zoomcheck; do [ -f "$ROOT/test/$_g.sh" ] || continue if RIPWIRE_BIN="$BIN" bash "$ROOT/test/$_g.sh" >/dev/null 2>&1; then ok "absorb gate ($_g.sh)" diff --git a/test/tagspredicatecheck.sh b/test/tagspredicatecheck.sh new file mode 100755 index 000000000..302398e16 --- /dev/null +++ b/test/tagspredicatecheck.sh @@ -0,0 +1,156 @@ +#!/usr/bin/env bash +# tagspredicatecheck.sh — the tags pass evaluates tree-sitter STRING predicates (#eq?, #not-eq?, #any-of?, #not-any-of?), +# so a keyword-headed definition's keyword list is tags.scm data, not a C++ table (#355, slice 5a of #354). Elixir's +# metadata attribute names (@doc, @spec, ...) moved the same way. +# +# WHY. Before #355 the tags pass never ran a pattern's predicates (queries/javascript/tags.scm and queries/cpp/tags.scm +# carried the notes), so Elixir kept `def`/`defp`/… in a C++ table (`elixirFunctionKeyword`, src/ingest_elixir.h) and every +# Lisp-style language after it would have had to do the same. The shared evaluator is src/infra/tspredicates.h: one walk +# over a pattern's predicate steps, used by the tags pass, by --match/--lint's passesPredicates and by its regex +# precompiler. #match? stays OUT of the tags pass (src/ingest_astquery.h's header measured its cost). +# +# RECORDED RED STATE (the pre-#355 binary, main @ 27eddee1): +# arm A2 — queries/elixir/tags.scm carries no #any-of?, and src/ingest_elixir.h holds the keyword list → FAIL +# arm A3 — queries/elixir/tags.scm carries no #not-any-of? on @_attribute; the metadata list is C++ → FAIL +# arm B4 — --match with (#any-of? @n "keep_one" "keep_two") returns all three functions (hits=3, not 2) → FAIL +# arm B5 — --match with (#not-any-of? @n …) returns all three (hits=3, not 1) → FAIL +# arm B7 — `def "not a name" do … end` is indexed as a function named "not a name"/0 → FAIL +# arms A1, B1, B2, B3, B6 pass there too: B1/B2/B6 because the C++ tables then made the same decision the predicates +# make now, B3 because --match always evaluated #eq?. They pin that nothing was lost when the tables moved. +# +# Arms: +# (A1) every predicate in queries/*/tags.scm is one the tags pass evaluates. A #match? there would filter NOTHING, +# silently — the exact bug this gate exists to prevent from coming back in another form. +# (A2) Elixir's definition keywords are data: queries/elixir/tags.scm has #any-of? lists on @_keyword, all identical +# and including "def", and no C++ statement spells that whole list (the old elixirFunctionKeyword table did). +# (A3) Elixir's metadata attribute names are data the same way: one #not-any-of? list on @_attribute, including +# "moduledoc", and no C++ statement spells that whole list (the old elixirMetadataAttribute table did). +# (B1) #any-of?, through the tags pass: every keyword head in tagspredicatefix/ex defines its function, and an +# ordinary call of the same shape (`ordinary(not_a_def(x))`) defines nothing. +# (B2) #not-any-of?, through the tags pass: `def unquote(name)…` heads define no symbol named unquote. +# (B6) #not-any-of? on an attribute, through the tags pass: tagspredicatefix/ex/lib/attrs.ex keeps the ordinary +# @limit and the typed @type t, and defines no @moduledoc, @doc, @spec or @enforce_keys. +# (B7) the one intended output change: tagspredicatefix/ex/lib/titles.ex keeps ExUnit's `test "a literal title"`, +# and `def "not a name" do` (invalid Elixir) defines nothing. The string-title pattern carries no @_keyword, +# so a definition keyword no longer reaches it; the C++ table used to accept it as "not a name"/0. +# (B3..B5) #eq?, #any-of?, #not-any-of? through --match on tagspredicatefix/c: exactly 1, 2 and 1 hits. +# +# BYTE-IDENTITY (the issue's acceptance) is not an arm here — it needs the pre-change binary. Run +# RIPWIRE_BASE= bash test/langidentitycheck.sh "$RIPWIRE_BASE" ./build/ripwire --langs=ex,clj,js +# and every row must read SAME. +set -u +ROOT="$( cd "$( dirname "$0" )/.." && pwd )" +BIN="${1:-${RIPWIRE_BIN:-$ROOT/build/ripwire}}" +[ "${BIN#/}" = "$BIN" ] && BIN="$ROOT/$BIN" +FIX="$ROOT/test/tagspredicatefix" +fail=0 +ok(){ printf ' PASS %s\n' "$*" || { fail=1; printf ' FAIL could not write the PASS line for: %s\n' "$*"; }; return 0; } +no(){ printf ' FAIL %s\n' "$*"; fail=1; } + +TMP="$( mktemp -d )" +trap 'rm -rf "$TMP"' EXIT +export XDG_CACHE_HOME="$TMP/cache" + +# ── A1. tags.scm uses only predicates the tags pass evaluates ─────────────────────────────────────────────── +bad="$( python3 - "$ROOT/queries" <<'PY' +import pathlib, re, sys +allowed = {"eq?", "not-eq?", "any-of?", "not-any-of?"} +for path in sorted(pathlib.Path(sys.argv[1]).glob("*/tags.scm")): + for no, line in enumerate(path.read_text().splitlines(), 1): + code = line.split(";", 1)[0] + for op in re.findall(r"\(#([A-Za-z-]+\?)", code): + if op not in allowed: + print(f"{path.parent.name}/tags.scm:{no} #{op}") +PY +)" +if [ -z "$bad" ]; then ok "A1 every tags.scm predicate is one the tags pass evaluates"; else no "A1 tags.scm uses predicates the tags pass does not run: $bad"; fi + +# ── A2. Elixir's definition keywords are tags.scm data, with no C++ copy ──────────────────────────────────── +a2="$( python3 - "$ROOT" <<'PY' +import pathlib, re, sys +root = pathlib.Path(sys.argv[1]) +scm = "\n".join(l.split(";", 1)[0] for l in (root / "queries/elixir/tags.scm").read_text().splitlines()) +lists = [tuple(re.findall(r'"([^"]*)"', m)) for m in re.findall(r'\(#any-of\?\s+@_keyword((?:\s+"[^"]*")+)\s*\)', scm)] +if not lists: + print("queries/elixir/tags.scm has no (#any-of? @_keyword ...) list") +elif len(set(lists)) != 1: + print(f"the @_keyword lists differ: {sorted(set(lists))}") +elif "def" not in lists[0]: + print(f"the @_keyword list has no \"def\": {lists[0]}") +# A COPY of the list is a C++ statement that spells all of it. Statements that test a few of the keywords for another +# reason (src/elixir_resolve.h decides privacy and macro-ness from the keyword a definition used) are a different +# concept and stay: CONTRIBUTING's "two keyword lists can match token for token and mean different things". +if lists: + for path in sorted((root / "src").rglob("*")): + if path.suffix in (".h", ".cpp") and path.is_file(): + for stmt in path.read_text(errors="replace").split(";"): + if all(f'"{kw}"' in stmt for kw in lists[0]): + print(f"{path.relative_to(root)} spells the whole keyword list in one statement (a C++ copy of tags.scm's list)") + break +PY +)" +if [ -z "$a2" ]; then ok "A2 Elixir's definition keywords live only in queries/elixir/tags.scm"; else no "A2 $a2"; fi + +# ── A3. Elixir's metadata attribute names are tags.scm data, with no C++ copy ─────────────────────────────── +a3="$( python3 - "$ROOT" <<'PY' +import pathlib, re, sys +root = pathlib.Path(sys.argv[1]) +scm = "\n".join(l.split(";", 1)[0] for l in (root / "queries/elixir/tags.scm").read_text().splitlines()) +lists = [tuple(re.findall(r'"([^"]*)"', m)) for m in re.findall(r'\(#not-any-of\?\s+@_attribute((?:\s+"[^"]*")+)\s*\)', scm)] +if len(lists) != 1: + print(f"queries/elixir/tags.scm has {len(lists)} (#not-any-of? @_attribute ...) lists, want exactly 1") +elif "moduledoc" not in lists[0]: + print(f"the @_attribute list has no \"moduledoc\": {lists[0]}") +else: + for path in sorted((root / "src").rglob("*")): + if path.suffix in (".h", ".cpp") and path.is_file(): + for stmt in path.read_text(errors="replace").split(";"): + if all(f'"{name}"' in stmt for name in lists[0]): + print(f"{path.relative_to(root)} spells the whole metadata list in one statement (a C++ copy of tags.scm's list)") + break +PY +)" +if [ -z "$a3" ]; then ok "A3 Elixir's metadata attribute names live only in queries/elixir/tags.scm"; else no "A3 $a3"; fi + +# ── B1/B2. the tags pass applies #any-of? and #not-any-of? ───────────────────────────────────────────────── +if ! "$BIN" "$FIX/ex" --no-cache > "$TMP/map.xml" 2> "$TMP/map.err"; then + no "B1/B2 ripwire exited non-zero on tagspredicatefix/ex: $( head -c 300 "$TMP/map.err" )" +else + b="$( python3 - "$TMP/map.xml" <<'PY' +import sys, xml.etree.ElementTree as ET +syms = {s.get("n").split("/")[0] for s in ET.parse(sys.argv[1]).getroot().iter("s")} +kept = {"kept_def", "kept_defp", "kept_macro", "kept_guard", "kept_delegate", "caller"} +missing = sorted(kept - syms) +if missing: + print(f"B1 keyword heads lost their definition: {missing}") +for name in ("not_a_def", "ordinary"): + if name in syms: + print(f"B1 an ordinary call became a definition: {name}") +if "unquote" in syms: + print("B2 a generated `def unquote(...)` head became a symbol named unquote") +for name in ("@limit", "@type t"): + if name not in syms: + print(f"B6 an attribute that defines a value or a type lost its definition: {name}") +for name in ("@moduledoc", "@doc", "@spec", "@enforce_keys"): + if name in syms: + print(f"B6 a metadata attribute became a definition: {name}") +if "test a literal title" not in syms: + print("B7 ExUnit's literal test title lost its definition") +quoted = sorted(s for s in syms if s.startswith('"')) +if quoted: + print(f"B7 a string where a def's name belongs became a definition: {quoted}") +PY +)" + if [ -z "$b" ]; then ok "B1 #any-of? keeps every keyword head and no ordinary call; B2 #not-any-of? drops unquote heads; B6 drops metadata attributes; B7 keeps test titles, drops def \"string\""; else no "$b"; fi +fi + +# ── B3..B5. --match evaluates the same string predicates ───────────────────────────────────────────────── +FN='(function_definition declarator: (function_declarator declarator: (identifier) @n)' +hits(){ "$BIN" "$FIX/c" --no-cache "--match=$FN $1)" --legend=compact 2>/dev/null | grep -o ']*hits="[0-9]*"' | grep -o 'hits="[0-9]*"' | grep -o '[0-9]*'; } +check(){ local label="$1" pred="$2" want="$3" got; got="$( hits "$pred" )"; if [ "$got" = "$want" ]; then ok "$label $pred: hits=$got"; else no "$label $pred: hits=${got:-none}, want $want"; fi; } +check B3 '(#eq? @n "keep_one")' 1 +check B4 '(#any-of? @n "keep_one" "keep_two")' 2 +check B5 '(#not-any-of? @n "keep_one" "keep_two")' 1 + +if [ "$fail" = 0 ]; then echo "ALL PASS"; else echo "FAILURES ABOVE"; fi +exit $fail diff --git a/test/tagspredicatefix/c/preds.c b/test/tagspredicatefix/c/preds.c new file mode 100644 index 000000000..727876017 --- /dev/null +++ b/test/tagspredicatefix/c/preds.c @@ -0,0 +1,3 @@ +int keep_one( int x ) { return x; } +int keep_two( int x ) { return x; } +int drop_three( int x ) { return x; } diff --git a/test/tagspredicatefix/ex/lib/attrs.ex b/test/tagspredicatefix/ex/lib/attrs.ex new file mode 100644 index 000000000..3242608da --- /dev/null +++ b/test/tagspredicatefix/ex/lib/attrs.ex @@ -0,0 +1,10 @@ +defmodule Attrs do + # #not-any-of? on @_attribute: a metadata attribute defines nothing; an ordinary one is a value. + @moduledoc "Module attributes." + @limit 10 + @enforce_keys [:id] + @doc "Uses the ordinary attribute." + @spec within(integer) :: boolean + def within(x), do: x < @limit + @type t :: integer +end diff --git a/test/tagspredicatefix/ex/lib/heads.ex b/test/tagspredicatefix/ex/lib/heads.ex new file mode 100644 index 000000000..5158f72a5 --- /dev/null +++ b/test/tagspredicatefix/ex/lib/heads.ex @@ -0,0 +1,20 @@ +defmodule Heads do + # #any-of? on @_keyword: each definition keyword names its function. + def kept_def(x), do: x + defp kept_defp(x), do: x + defmacro kept_macro(x), do: x + defguard kept_guard(x) when is_integer(x) + defdelegate kept_delegate(x), to: Other + + # An ordinary call whose first argument is a call has the same shape as a definition head. It defines nothing. + def caller(x) do + ordinary(not_a_def(x)) + kept_def(x) + end + + # #not-any-of? on @name: a generated head names nothing static. + for name <- [:alpha, :beta] do + def unquote(name)(), do: 1 + def unquote(name), do: 2 + end +end diff --git a/test/tagspredicatefix/ex/lib/titles.ex b/test/tagspredicatefix/ex/lib/titles.ex new file mode 100644 index 000000000..11947b192 --- /dev/null +++ b/test/tagspredicatefix/ex/lib/titles.ex @@ -0,0 +1,13 @@ +defmodule Titles do + use ExUnit.Case + + # A literal title names an ExUnit test; `def` with a string where its name belongs names nothing (invalid + # Elixir). Before #355 the C++ keyword table also accepted the second as a function named "not a name"/0. + test "a literal title" do + :ok + end + + def "not a name" do + 1 + end +end