Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,16 @@ not published here — see `docs/EVALS.md` for the instruments behind the headli

## [Unreleased]

### Changed — Ruby mixins read the shared base-relations path (#400, slice 4a)

Ruby's `include`/`extend`/`prepend` are now captured by `queries/ruby/tags.scm` as `@reference.mixin` and turned into
inherit references by the shared capture consumer — the same emitter every language's base clause goes through. The
Ruby-only `captureRubyMixinBases` and its `captureIncludes` call site are gone; the query states the call shape and
the C++ reader keeps the two guards a tags pattern cannot state (the directive's text, and its class-body position).
Output is byte-identical: the map, `--deps --limit=100000`, `--skipped`, `--callers`, `--callees`, `--impact` and
`--uses` all match the base build over a real Rails corpus and the Ruby fixtures, and `--lego` lists the same
implementors. `kParserVer` moves 171 -> 173 (the extraction path changed; 172 is 171's rich file tag).

### Added — Clojure (`.clj`/`.cljs`/`.cljc`/`.bb`)

Vendored Yogthos' ABI-15 `tree-sitter-clojure` parser and added syntax-level definitions and direct-call captures for the four Clojure-family extensions. The grammar comparison parsed 4,930 of 4,933 files (99.94%, 55.5 MB) from Metabase, clojure-lsp, clj-kondo and Babashka checkouts in 2.24 seconds; two failures were intentionally invalid clj-kondo fixtures and one was a Metabase test. This measures parser completion on that corpus, not semantic call-resolution accuracy. Namespace/import resolution, macro expansion and exact multi-arity matching remain outside this first pass; see `docs/ARCHITECTURE.md` for the extraction limits.
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -559,7 +559,7 @@ review will credit it, but it is never a condition for merging the language.
| 1. Imports | `ingest_importcap.h`: `@import.path` captures, `emitCapturedImport`, one `normaliseImportSpecifier` arm per `DepDialect`. Resolution in `resolve.h`. | partly. C-family, JS/TS, Go and Python read the capture (Python since #382). The rest still go through per-language arms of `directiveTargetOf` (`ingest_relations.h`): `csharpUsingTarget`, `phpUseTarget`, `luaRequireTarget`, `bashSourceTarget`, `rubyRequireTarget`, `elixirDirectiveTarget`, plus Rust `use`/`mod`, Kotlin `import_header` and Swift `import_declaration`. | **1a** one language per PR onto `@import.path` plus its normaliser arm, deleting its arm in `directiveTargetOf`: S each for C#, PHP, Lua, Bash, Kotlin and Swift; M for Rust (`use` trees, `mod`), Ruby (`autoload`) and Elixir (`alias`/`use`), whose directives carry more meaning. Claim on #358. |
| 2. Aliases | None. Renames reach the resolver as `RawBind` records (`ingest_cache.h`) from separate producers: Python's `@import.names` / `@import.alias` captures, read in `captureTagsFacts` (`ingest_sidecap.h`) and turned into binds by the Python-only `emitCapturedPythonImportBind` (`ingest_importcap.h`); `captureJsImportFacts`, `captureJsModuleAliases` and `captureGoImportFacts` (`ingest_jsimports.h`), C/C++ aliases in `fnPtrAliasName` (`ingest_binds.h`) and `captureTypeAlias` (`ingest_relations.h`). The readers are separate too: `addTypeAliasBases` and `buildUsingReexports` (`resolve.h`). | duplicated: one "local name → target" concept in about five shapes. | **2a** one shared binds channel (`@import.alias` / `@import.names` → one `RawBind` kind): M, *maintainers*, after the #358 import slices. **2b** after 2a, one language per PR moves its renames onto it: S each (Rust `use a as b`, Kotlin `import x as y`, PHP `use … as`, C# `using X = …`). |
| 3. Types | `captureTagsFacts` (`ingest_sidecap.h`) reads `@definition.*` captures for every language. | shared, but language decisions are `Lang::` branches inside the loop: 36 non-comment `Lang::` mentions, 15 of them Elixir. | **3a** #374: one constexpr per-language hook table that `captureTagsFacts` calls once per capture, one language per commit, Elixir first. M. |
| 4. Inheritance | `captureBases` (`ingest_relations.h`): base clauses for C++, TS/JS, Java, Python, Swift, C#, PHP, Kotlin and Ruby, chosen by node kind (`isBaseTypeNodeIn`). | partly. Rust `impl … for` is a separate pass; Ruby mixins have their own `captureRubyMixinBases`. | **4a** Ruby `include`/`extend`/`prepend` as captures read by the shared path, retiring `captureRubyMixinBases`: S–M, Ruby (offered on #339). **4b** the base-clause node kinds move from `isBaseTypeNodeIn` into `tags.scm` captures: M, all of the languages above. **4c** structural conformance (#386): a class or type whose methods cover a structural base's required methods (Python `typing.Protocol`, Go interfaces, possibly TS interfaces) is listed as an implementor marked `structural=1`, never as a nominal subclass, with a disclosed minimum-method threshold; step (c), one shared method-set check, per-language data for which bases are structural. M; Python and Go first. |
| 4. Inheritance | `captureBases` (`ingest_relations.h`): base clauses for C++, TS/JS, Java, Python, Swift, C#, PHP, Kotlin and Ruby, chosen by node kind (`isBaseTypeNodeIn`). | partly. Rust `impl … for` is a separate pass; Ruby mixins read the same path through `@reference.mixin` captures since slice 4a (#400). | **4a** done (#400): Ruby `include`/`extend`/`prepend` read the shared path through `queries/ruby/tags.scm` captures, `captureRubyMixinBases` retired. **4b** the base-clause node kinds move from `isBaseTypeNodeIn` into `tags.scm` captures: M, all of the languages above. **4c** structural conformance (#386): a class or type whose methods cover a structural base's required methods (Python `typing.Protocol`, Go interfaces, possibly TS interfaces) is listed as an implementor marked `structural=1`, never as a nominal subclass, with a disclosed minimum-method threshold; step (c), one shared method-set check, per-language data for which bases are structural. M; Python and Go first. |
| 5. Callables | Definitions come from `tags.scm`, read by `captureTagsFacts`. | partly. Keyword-headed definitions keep keyword tables in C++: `elixirFunctionKeyword` and `elixirMetadataAttribute` (`ingest_elixir.h`) repeat `macroBodyKeyword` (`ingest_relations.h`), 0.82 and 0.96 similar by `--clones`. | **5a** #355: the tags pass evaluates `#eq?`/`#any-of?`/`#not-any-of?`, then Elixir's table moves into its `tags.scm`. M; Elixir, then Clojure. |
| 6. Dispatch | `buildGraph` (`graph.h`): a SCIP pin, then the canonical tier (`appendCanonicalCandidates`, `resolve.h`), bindings and receiver rules (`receiverevidence.h`), then name tiers. Per-language qualified-call rules: `ElixirResolver` (`elixir_resolve.h`), `keepRustQualifiedCandidates` and `keepStdQualifiedCandidates` (`graph.h`). | partly. Facts about each language are scattered `l == Lang::X \|\| …` predicates (`implicitReceiverLang`, `memberNeverReachesFreeFunction` and `classNameReceiverLang` in `receiverevidence.h`; `langUsesHashComment` in `quality.h`), and `--clones` reports them as one group. | **6a** the shared refusal for an unplaced module qualifier (the known gap above): S, *maintainers*, not yet scheduled. **6b** the **language-neutral qualified-call resolver**: generalise `ElixirResolver`'s module table (alias/import/require facts → `Module::name`) so that Clojure `ns/f`, Elixir and other module-qualified languages use one resolver driven by per-language data. M; Elixir byte-identical, Clojure as the first new user. **6c** those predicates become columns of one constexpr per-`Lang` table: S, all languages. |
| 7. Data-driven dispatch | One mechanism per shape: fn-pointer and callback bindings `buildFnPtrBindTables` (`graph.h`, C/C++); HTTP routes `routesVisitNode` (`ingest_sidecap.h`, a JS/TS and Python walk) into `RouteDef`/`RouteUse` (`model.h`); FFI aliases `BindingAlias` (`model.h`). | partly. Calls by a literal method name (`send(:m)`) resolve for Ruby only. | **7a** #375: one capture for a call whose method name is a literal, resolved through the ordinary receiver rules. M; Python `getattr`, JS/TS `obj["m"]`, Go `MethodByName`. **7b** route registration read from captures instead of the walk: M; JS/TS and Python. |
Expand Down
19 changes: 19 additions & 0 deletions queries/ruby/tags.scm
Original file line number Diff line number Diff line change
Expand Up @@ -96,3 +96,22 @@
; never contributes: a class-wrapped create_table (string- or symbol-named) is refused by the
; class/module-nest gate — the schema is the ONE source of column names; add_column/remove_column/
; change_column are argument data and read nowhere (a column added then dropped never registers).

; ---- mixin directives (family 4, slice 4a) ----
; `include M` / `extend M` / `prepend M` make M an ANCESTOR of the enclosing class or module: the class's own
; method lookup reaches M's methods, which is the edge `class C < B` mints. Each constant argument is captured
; here as @name and read by the shared base-relations path (src/ingest_sidecap.h captureTagsFacts -> emitBaseRef,
; the same emitter every language's base clause goes through), so no Ruby-only C++ function walks these
; directives any more (slice 4a retired captureRubyMixinBases).
;
; UNANCHORED on purpose, exactly like the import patterns above: the tags pass evaluates NO predicates
; (#eq?/#match? are silently ignored — test/matchgrammarcheck.sh), so this pattern states the call SHAPE and the
; C++ reader owns the two guards a query cannot state — the directive's own TEXT (without it `Logger.new( Config )`
; and `raise Error` at class-body level match the same shape) and the POSITION (class/module body, `class << self`,
; or a concern's `included do … end`; a method body, another block and the file top level emit nothing). Both guards
; already existed for this lane and stayed where they were; the query is the language gate, so the reader spends no
; Lang:: test on them. The division is the one the shared import vocabulary uses for `require` (issue #358,
; src/ingest_importcap.h).
(call
method: (identifier)
arguments: (argument_list [(constant) (scope_resolution)] @name)) @reference.mixin
2 changes: 1 addition & 1 deletion src/graph.h
Original file line number Diff line number Diff line change
Expand Up @@ -8788,7 +8788,7 @@ inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = null
for( NodeId baseId : *baseIds )
{
// lane lego-transitive (M1): a Ruby MODULE is a base here too. `include M` / `extend M` / `prepend M` put it in
// the class's ancestors (captureRubyMixinBases, parser version 131), so the class implements its contract the way
// the class's ancestors (the @reference.mixin capture, parser version 131), so the class implements its contract the way
// a subclass does — and --lego=M listed nobody (rake: `include Rake::DSL`). The test is BuiltinMethodGate's
// class-like rule, the one that already counts a Ruby module (Ruby && SymKind::Other: @definition.module maps to
// Other for every language, and only Ruby's tags have no other Other). Other languages need nothing here: an
Expand Down
10 changes: 8 additions & 2 deletions src/ingest_cache.h
Original file line number Diff line number Diff line change
Expand Up @@ -320,7 +320,13 @@ constexpr std::uint32_t kCacheVersion = 31; // 31: train 27 — lane l
// (Py `pkg.mod`, TS `./x`, Rust `crate::a::b`/`mod:x`) —
// a target FORMAT change → old caches must be rejected.
// 4: Include gained a `bool isAngle` (quote/angle) field
constexpr std::uint32_t kParserVer = 171; // bump on any grammar/.scm/extraction change
constexpr std::uint32_t kParserVer = 173; // bump on any grammar/.scm/extraction change
// 173 = 2026-10-15 (slice 4a, #400): the Ruby mixin directives (`include` / `extend` / `prepend`)
// move off the captureIncludes walk onto the shared base-relations path —
// queries/ruby/tags.scm captures the directive as @reference.mixin and captureTagsFacts
// emits each constant's inherit reference through emitBaseRef (captureRubyMixinBases retired).
// Output byte-identical; the extraction path changed, so the number moves. 173, not 172: 172 is
// 171's rich file tag. Rich file tag 174. kCacheVersion stays 31.
// 171 = 2026-10-13 (PR #349 lands, @tag-und-nacht): Clojure grammar, tags.scm and extraction
// (ingest_clojure.h) and the qualified-call namespace in Reference::qualifier. The branch built
// 156, 158 and 169 (rich file tag 170); main is 167 (rich file tag 168). 171, not 168: a lean 168
Expand Down Expand Up @@ -494,7 +500,7 @@ constexpr std::uint32_t kParserVer = 171; // bump on any grammar/.sc
// 131 = 2026-10-02 (test/rubyreachcheck.sh): a Ruby MIXIN is an
// ancestor — `include`/`extend`/`prepend` at class-DSL
// position mints one inherit RawRef per constant, at the
// constant's byte (ingest_relations.h captureRubyMixinBases).
// constant's byte (ingest_relations.h emitCapturedMixinBase).
// New records, same layout: kCacheVersion unchanged (27). A
// Ruby cache written at 130 holds none of them and must re-parse.
// 130 = 2026-10-01 (test/rubybarecallcheck.sh): Ruby's BARE-WORD call — an
Expand Down
Loading
Loading