diff --git a/CHANGELOG.md b/CHANGELOG.md index 3f6752442..7fc6a6283 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,744 @@ not published here — see `docs/EVALS.md` for the instruments behind the headli ## [Unreleased] +### Changed — a Ruby call to self, or on an instance the code builds, answers from its own side: an instance never reaches `def self.m` + +The entry below left one floor open, (i). A call to self, or on a receiver the code builds, read a class's defs from both +sides alike. In the service-object pattern (`def self.call( x ) = new( x ).call` beside `def call`), `svc.call` split +between `def self.call` and the instance `call`. A bare call in an instance method did the same, and a bare `new( x )` in +a class method bound to some other class's `new` or was refused. + +**What self is** (`graph.h RubyClassObjects::selfCall`, `typedLookup`): +- In a singleton method (`def self.m`, a def in `class << self`), self is the class object. Its call answers from the + class object's lookup (the entry below), so a bare `new( x )` there reaches the class's `initialize`. +- In a class's other defs, self is an instance. So is a receiver the code builds (`C.new`, a finder). Such a call answers + from the instances' lookup: the class's instance methods, what it includes, then its superclasses', by fully-qualified + constant. + +Rule 1 and its base walk, which read both sides of a class alike, step aside. What that lookup misses goes to the name +ladder. The ladder now keeps only the namesakes that answer on the side self stands on (`answersOn`). If none is left the +call declines rather than being refused, since self may be another object inside a block. + +A bare `new` the class object's lookup misses is refused as external, as `C.new` is, when no class in reach below it +defines an `initialize`. When one does, the call declines, since self may be that class. + +**Where self has no one side** the call is left as before: +- an instance method of a module (self is an includer's instance, or a class object the module extends); +- a def a concern defines for its includers; +- a class body; +- a module, or a FactoryBot build, as a receiver: a factory keeps only its class's final segment, which the call site + cannot place; +- a name the lookup misses when a `method_missing` holds it open. + +A built receiver's lookup by constant is trusted only when it finds the method. `described_class` keeps only the final +segment too, so on a miss the old lookup by name runs, keeping its instance methods. + +**Also fixed:** no class is its own superclass. `class SchemaCreation < SchemaCreation` inside `module MySQL` now names +the outer `SchemaCreation` (`resolve.h rubyResolveBaseConstant`). Before, the constant index read the class itself, so +the walk never reached the outer class. + +Measured with `--no-cache`, against the entry below (162255eb), both runs on one snapshot of each tree: + +| Corpus | Edges | Call-graph isolated | `declined=` | `external=` | Call sites changed | +| --- | --- | --- | --- | --- | --- | +| activerecord 7.2.3.2 `lib/` | 11,478 → 11,297 | 2,421 → 2,418 | 4,704 → 4,717 | 1,181 → 1,167 | 125 | +| activesupport 7.2.3.2 `lib/` | 3,417 → 3,421 | 1,266 → 1,261 | 1,380 → 1,388 | 967 → 954 | 41 | +| Rails app A | 27,503 → 27,382 | 18,589 → 18,584 | 39,336 → 39,355 | 39,578 → 39,554 | 260 | +| Rails app B | 26,044 → 26,146 | 5,736 → 5,623 | 34,560 → 34,577 | 49,253 → 49,093 | 203 | +| this repo's `src/` (C++), a JavaScript app, a Python repo | default map byte-identical | | | | | + +Where edges fall, splits collapse to one target: 208 target edges on activerecord, 204 on A. Each adapter's bare +`quote`, `execute` and `internal_exec_query` now reaches the module its adapter includes, not every adapter's. Where +edges rise, a service object's `new( … ).call` now reaches its `initialize`: 148 sites on B had been refused. + +Checked against the source: +- **activerecord:** a sample of 20 of the 98 changed sites, and every other changed site. + - 20 of 20 sampled were right. + - The 14 newly bound calls are bare `new` in class methods reaching their class's `initialize`. + - 12 of the 13 that lost an edge had been bound to a namesake in an unrelated class, or to a method Rails defines at + run time. The 13th is a template method whose two subclass overrides had split, and now declines. +- **activesupport:** all 41, all right. `Duration.days`' `new` reaches `Duration#initialize`, not `TaggedLogging.new`. + `Time#change`'s bare `zone` no longer binds to the class method `Time.zone`. +- **The applications:** 20 changed and 20 gained per application, and every site that lost an edge. + - Changed: 16 of 20 and 20 of 20 right. A's other 4 were wrong before and after: a property Hashie generates, bound + by name. + - Gained: 20 of 20 on each. + - Lost: all 24 (19 on A, 5 on B) had been bound to an unrelated namesake or a generated method, except one + template-method pick that now declines. + +The newly bound and the declined sites are read by the same rules. + +Lifted: floor (i) of the entry below, and in part floor (c) of `test/rubyreachcheck.sh` (instance and class methods +shared one name space). Its arm is kept, as an instance's call to an extended module's method is still admitted when +the lookup misses. + +Stated floors, each pinned by `test/rubyclassrecvcheck.sh`: +- **(i)** A side is known only where self is one class. An instance method of a module, a def a concern defines for its + includers, and a module or a FactoryBot build as a receiver are left as before. + +Not pinned, and older than these changes: the ladder's locality tie-break can cut a split down to the definer nearest the +caller by file. That covers two definers at one level of a lookup (floor (k) pins the split itself), and a template +method's subclass overrides, which a call to self in the base class reaches by name. + +`kParserVer` stays 145: this changes the resolver, not what ingest records. + +### Changed — a Ruby class object's lookup tells `include` from `extend`, in Ruby's order, and a `method_missing` answers only what it misses + +The class-object lookup in the entry below walked a class's mixins without telling how each one came in. So an +included module's instance method answered a call on the class (that entry's floor (b)), and so did a module-body +`def self.x` of a module the class includes (floor (h)). Ruby raises NoMethodError for both. A `method_missing` anywhere +in the lookup also sent every call on the class to the name ladder, even a name the lookup defines, though Ruby calls +`method_missing` only on a miss. `ActiveRecord::Base` extends `DynamicMatchers`, so every class method of `Base` went to +the ladder. That is why `Base.configurations` split or declined against `DatabaseConfigurations`' reader (the bare-word +entry gives the counts). + +**At ingest** (`ingest_binds.h captureRubyClassMixins`, `rubyDefSide`): +- each mixin the class object's lookup reaches mints a `LocalBindKind::RubyClassMixin` binding at its constant: an + `extend`, an `include` inside `class << self`, and a module's `extend self`; +- a def that a concern defines for its includers carries a mark on its `RubySingletonDef` binding: `included` for a + `def self.x` in `included do`, `class_methods` for a def in `class_methods do`. + +**The lookup** (`graph.h RubyClassObjects::sideLookup`) walks each ancestor on the side it joins, in Ruby's order: +- first the class's own singleton methods, with what an included concern's `included do` defines there; +- then the modules it extends, level by level: an `extend`, a concern's `class_methods do` and nested `ClassMethods`, + and a module that an `included do` extends; +- only then its superclass's, one round per class up the chain. + +What no longer answers a call on the class object: +- an included module's instance methods; +- a module's own `def self.x`, or the modules that module extends; +- on a concern itself, the class methods it gives its includers. + +An instance's lookup (what `C.new` runs as `initialize`, or a forwarding base's name) holds no extended module, and +searches a class's included modules before its superclass. A `method_missing` in the lookup now answers only a name +nothing there defines, which is left to the ladder as before. A `class << self` delegation still leaves its name there. + +Measured with `--no-cache`, against the entry below (0b6118a6), both runs on one snapshot of each tree: + +| Corpus | Edges | Call-graph isolated | `declined=` | Call sites changed | +| --- | --- | --- | --- | --- | +| activerecord 7.2.3.2 `lib/` | 11,482 → 11,478 | 2,422 → 2,421 | 4,728 → 4,704 | 82 | +| activesupport 7.2.3.2 `lib/` | 3,414 → 3,417 | 1,267 → 1,266 | 1,386 → 1,380 | 23 | +| Rails app A | 27,498 → 27,503 | 18,591 → 18,589 | 39,342 → 39,336 | 8 | +| Rails app B | 26,044 → 26,044 | 5,737 → 5,736 | 34,560 → 34,560 | 6 | +| this repo's `src/` (C++), a JavaScript app, a Python repo | default map byte-identical | | | | + +Every changed call site was read against the source, all 119 of them rather than a sample: +- **activerecord**: 21 changed target, 27 gained an edge where they declined, and 34 keep their target but are now + decided by the lookup rather than by name. + - `ActiveRecord::Base.configurations`, `.connection_handler`, `.lease_connection`, `.establish_connection` and the + like now reach `Core`'s `included do` defs or the `ConnectionHandling` that `Base` extends. + - Of the 13 callers of `Core.configurations` on `main`, the 12 that were right are single edges again. The 13th keeps + the reader, and the 9 sites in the `.rake` file join them. The two bare calls in `QueryCache::ClassMethods` still + split, since their `self` is any includer. + - `inspection_filter` reaches `Core`'s `ClassMethods`, not the instance reader. `CommandRecorder.new` reaches its + `initialize`. + - All 82 were right. +- **activesupport**: 16 changed target and 6 gained an edge. `Duration.new`, `TimeWithZone.new`, `OptionMerger.new` and + the like are classes with a `method_missing`; they now reach their `initialize`, not `TaggedLogging.new` or + `TimeZone.new`. One call is newly refused: `ERB::Util.html_escape`, which a `singleton_class.prepend` supplies (floor + (e)). It bound before only because the misread `prepend` of the instance side reached the same module. +- **The applications**: two calls on a subclass now reach the class method that its included concern's `ClassMethods` + defines, which Ruby searches before the superclass's (they bound to the superclass's before). 12 calls to `new` on + classes with a `method_missing` now reach their `initialize`, 6 of them instead of a controller's `new` action. All + 14 were right. + +Lifted: floors (b) and (h) of the entry below, whose arms are kept, inverted. Floor (c) is lifted in part: `extend self` +is read. + +Stated floors, each pinned by `test/rubyclassrecvcheck.sh`: +- **(c)** A module's every instance method answers a call on the module: `module_function` is not read. +- **(e)** A module mixed in by a call rather than a class-body directive is in no lookup. That covers + `Tool.extend( Tool::Ext )` and `singleton_class.prepend M` in the body (activesupport's `ERB::Util`). +- **(i)** An instance's lookup still reads the class's singleton methods as its own. That covers a receiver the code + builds (`svc = Service.new( 9 ); svc.call`) and a bare call in an instance method. So `svc.call` splits between + `def self.call` and the instance `call`, where Ruby runs the instance's. Lifted by the entry above. +- **(j)** A `method_missing` that only an instance answers still leaves a call on the class object to the ladder when the + lookup misses: `Proxy.anything` binds by name, where Ruby raises NoMethodError. +- **(k)** Two modules extended at one level answer their union: `Dual.pick` splits between `First` and `Second`, where + Ruby takes the later `extend`. + +`kParserVer` 142 → 145, above `main`'s 143 and 144. New records, same layout: `kCacheVersion` stays 27, and Ruby caches +re-parse once. + +### Changed — a Ruby call on a class answers from the class object's lookup: `C.new` runs `initialize`, a lookup that leaves the tree is refused + +A call whose receiver is a constant (`Report.generate`, `User.where( … )`, `Service.new( x )`, `JSON.parse( s )`) is +sent to the class object. Once the class itself missed the name, such a call bound to whichever in-tree method had it. +On application B, 2,537 calls to `new` (`Hash.new`, `Date.new`, every service's) bound to one controller's `new` action, +and a model's `where` to a mailer preview's mock. On application A, `User.where` and its kind bound to a feed model's +`def self.where` (1,200 call sites), and `JSON.parse` to a URI service's `parse`. + +**Which class** (`graph.h RubyClassObjects::receiverFqn`). The constant is read as Ruby resolves it from the call site, by +fully-qualified constant over the constant index. First lexically: the class or module open around the site, the ones +around it, then the top level. Then through the ancestors of that open, or of a qualified constant's head (`Sub::Failure` +for a Failure that Sub's superclass nests). Last by Zeitwerk's directories, a namespace no file opens +(`Alerts::Senders::Pay` is the tree's `Senders::Pay`). A receiver written with a path keeps it at ingest +(`Reference::fieldName`, `ingest_binds.h rubyReceiverWrittenPath`). `described_class` and `described_class::Worker` keep +the described class's path. So two classes of one name no longer share a lookup: `BankIntegration.process_event` +reaches BankIntegration's `HookMethods`, not MailIntegration's, and `Result.new` inside `module Billing` reaches +Billing's Result. A constant that resolves to nothing the tree opens answers only from a reopened root. That covers +`JSON`, `RSpec`, `Stripe::Customer` beside an app's `Customer`, and a `Row = Struct.new( … )` constant beside an +unrelated in-tree `Row`. + +**What answers** (`RubyClassObjects::lookup`). The shallowest of the class, its mixins and its superclasses to define the +name as a class method, walked along what their superclass and mixin references resolve to: +- a singleton method: `def self.m`, or a def or an accessor in `class << self`, each recorded at ingest as a + `LocalBindKind::RubySingletonDef` binding; +- a module's method: an extended module's, or a concern's `ClassMethods`'. + +A class's instance method is its instances' alone. So `Service.call` reaches `def self.call`, and no longer splits with +the instance `call` beside it. The same holds for a method of a concern that nests a `ClassMethods`. After those come a +reopened Module, Class, Object, Kernel or BasicObject. + +`C.new` is Class#new unless a `def self.new` answers first, and Class#new runs the first `initialize` an instance's lookup +finds, never an instance method named `new`. Some classes sit below a base whose class object forwards a name it lacks to +a new instance: `ActionMailer::Base`, `Devise::Mailer`, `ActiveSupport::CurrentAttributes`. Those answer from their +instances' lookup (`UserMailer.welcome( u )`, `Current.user = u`). + +**When nothing answers**, Ruby's method is outside the tree (ActiveRecord::Base's `where`, StandardError's `initialize`, +Struct's `new`), and the call is refused as external. + +Left as before: +- a SCREAMING constant's value (`LIMITS = Limits.new`, a value of a class nothing types); +- a class object that may answer any name: a `method_missing` in its lookup, or the name a `class << self` delegates + (activerecord's `ExplainRegistry.collect?`). The entry above narrows the first: a `method_missing` answers only what + the lookup misses. + +Measured with `--no-cache`, the RSpec matcher-chain change below (parser version 136) against this change, both runs on +one snapshot of each tree. + +| Corpus | Edges | Call-graph isolated | Call sites that lost an edge | Gained an edge (to an `initialize`) | Changed target (to an `initialize`) | Split call sites | Newly refused (no edge before) | +| --- | --- | --- | --- | --- | --- | --- | --- | +| activerecord 7.2.3.2 `lib/` | 11,436 → 11,482 | 2,551 → 2,422 | 236 | 327 (320) | 219 (202) | 1,289 → 1,099 | 633 | +| activesupport 7.2.3.2 `lib/` | 3,535 → 3,414 | 1,283 → 1,267 | 190 | 28 (26) | 70 (54) | 356 → 338 | 387 | +| Rails app A | 29,524 → 27,491 | 18,872 → 18,596 | 2,201 | 1,152 (1,130) | 483 (285) | 1,541 → 1,059 | 6,927 | +| Rails app B | 28,374 → 26,044 | 5,698 → 5,737 | 3,650 | 247 (0) | 2,238 (697) | 2,253 → 716 | 7,180 | +| this repo's `src/` (C++), a JavaScript app, a Python repo | default map byte-identical | | | | | | | + +Edges fall where a wrong binding goes, and rise where an `initialize` gains its callers. Methods with an in-tree caller: + +| Corpus | Before → after | Gain a first caller | Lose every caller | +| --- | --- | --- | --- | +| Rails app A | 7,905 → 8,235 | 357 | 27 | +| Rails app B | 7,493 → 7,613 | 142 | 22 | +| activerecord | 3,615 → 3,782 | 174 | 7 | + +The newly refused calls had no edge before (several namesakes, and nothing to choose between them); they are now counted +as external. Some wrong bindings remain on calls whose receiver is a value, not a constant (`klass.new`, +`relation.where`): 67 calls to `new` still reach the controller's action, and 570 calls to `where` the feed model's. + +Sampled against the source, 20 call sites per kind and application: +- **Lost an edge:** 20 of 20 on A and 20 of 20 on B were wrong bindings: ActiveRecord finders and `create!` on models, + `JSON.parse`, `Time.now`, `Rails.logger`, a Sidekiq worker's `perform_async`, + `ActiveModel::Type::Boolean.new`, an ActiveJob's `new`. +- **Gained an edge:** 20 of 20 on A, each `C.new` reaching C's `initialize`. 20 of 20 on B, each an integration module's + call reaching its own extended module. +- **Changed target:** 20 of 20 on A and on B. Each was one of: + - a split collapsed onto `def self.m`; + - a controller's `new` replaced by the class's `initialize`; + - a same-named class or module in another namespace replaced by the one the constant names; + - a class method the superclass defines (a service's `resolve`). +- **Lost every caller:** of the methods sampled from that list, each was a wrong-before binding, except the stated floors + below: a module extended at run time on A (floor (e)), two classes a block builds on B (floor (f)), and + activesupport's `Benchmark.ms` (floor (g)). + +Stated floors, each pinned by `test/rubyclassrecvcheck.sh`: +- **(a)** A constant the tree assigns (`DEFAULT_LIMITS = Limits.new`) is a value of a class the tree does not type: a + call on it binds by name as before. +- **(b)** The tree does not tell `include` from `extend`: an included module's instance method answers a call on the + class, unless the module nests a `ClassMethods`. +- **(c)** A module's every method answers a call on the module: `module_function` and `extend self` are not read. +- **(d)** A tree with no Ruby superclass, mixin or constant reference keeps no constant index. There a constant is read by + its final segment, and two classes of one name share a lookup. +- **(e)** A module mixed in at run time (`Tool.extend( Tool::Ext )`, a call rather than a class-body directive) is in no + lookup. +- **(f)** A class that `Struct.new( … ) do … end` or `Data.define( … ) do … end` builds is opened by no file, so a def in + its block answers no call on its constant. +- **(g)** `class << Clock` opens another object's singleton. Its defs read as top-level defs, which no receiver reaches. +- **(h)** A mixin's own singleton methods (`def self.x`) answer a call on each class that includes it. That is what Ruby + does for one written inside a concern's `included do … end` block, which runs on the includer, but not for one in the + module's body, which stays the module's: there a call on the includer raises NoMethodError in Ruby and binds here. The + tree does not tell the two apart. + +Lifted: `test/rubyinheritcheck.sh` floor (c) and `test/rubyrecvnarrowcheck.sh` floor (b), the final-segment probe that +calls on a class shared. `UsesAlpha.beta_make` no longer pins Beta::Base's method, and `Left::Shared.go` reaches Left's +alone. Their arms are kept, inverted. So are the three arms that pinned a missed class lookup to the name ladder +(`Rec.make`, `Calc.report`, the base-clause mutation): they now expect the call refused. + +`kParserVer` 136 → 137. This adds an appended binding kind and a path in an existing field, with the same layout, so +`kCacheVersion` stays 27, and Ruby caches re-parse once. + +### Changed — a call on an RSpec matcher's chain is RSpec's: `receive( :m ).with`, `change { }.from( a ).to( b )` + +`expect( v ).to` stopped binding to an in-tree `def to` in the RSpec round below, but a matcher's own chain still bound +by name. `receive( :m ).with( 1 )`, `.and_return( 2 )` after it, and the `to` of `change { }.from( a ).to( b )` are +called on the matcher a receiver-less builder returns, and every link is RSpec's. + +**What is typed** (`ingest_binds.h rubyMatcherChainType`; `model.h kRspecTargets`). Inside an example group, the matcher +builders `receive`, `have_received`, `receive_messages`, `receive_message_chain`, `change`, `raise_error` / +`raise_exception`, `output`, `be_within` and `yield_control` build that RSpec class, as `expect` and `allow` do. A call +on a chain rooted at one is typed as the root, at most 8 links from it. A link's own class (Change's `from` returns a +ChangeFromValue) is RSpec's all the same. A FactoryBot build at a chain's root types nothing: its methods return anything. + +**Which defs shadow a builder** (`graph.h rubyDefinedBuilders`). Only a def an example group's self can reach: a top-level +def, a reopened Object's, a def inside a group's block, or a module's in test code (what a `config.include` mixes in). A +class's method and an application module's are not in a group's lookup. Before, any def of the name counted, so every +Rails migration's `def change` turned the `change` builder off for the whole tree. + +Measured with `--no-cache`, the Rake/Jbuilder change below (parser version 135) against this change, both runs on one +snapshot of each tree. + +| Corpus | Edges | Call sites that lost an edge | Newly refused (no edge before) | +| --- | --- | --- | --- | +| activerecord 7.2.3.2 `lib/`, activesupport 7.2.3.2 `lib/` | default map byte-identical | 0 | 0 | +| Rails app A | 29,574 → 29,524 | 135 | 2,353 | +| Rails app B (no in-tree namesake of a chain method) | 28,374 → 28,374 | 0 | 3,720 | + +On application A, every lost edge was the presenter's `to`, from a `change { }` chain, as read off the source of +all 135 sites. The newly refused calls are the chain methods themselves: `with`, `and_return`, `by`, `and_call_original`, +`from`, `once`, `of`. 47 calls still bind to the presenter's `to`, 25 of them in spec/support helper modules (floor (a)). + +Stated floors, each pinned by `test/rubyrspectargetcheck.sh`: +- **(b)** A builder's name is shadowed by a def an example group can reach, as above. An application module that a + `config.include` does mix in is not read as one. +- **(c)** A link more than 8 calls from its builder binds by name as before. + +`kParserVer` 135 → 136. Same layout, so `kCacheVersion` stays 27, and Ruby caches re-parse once. + +### Changed — Rake task files and Jbuilder views are crawled as Ruby, and their top-level `self` is known + +A Rails application keeps Ruby in two more file kinds: Rake tasks (`lib/tasks/*.rake`) and Jbuilder JSON views +(`app/views/**/*.json.jbuilder`). Neither was crawled, so a method that only a task or a JSON view called had no +caller. Both now index as Ruby, with every Ruby rule a `.rb` file gets (`ingest_crawl.h kLangTable`, and the lint, +include, comment and seed-extension tables beside it). + +**Self outside any class** (`graph.h RubyTopSelf`). Crawling alone was wrong in a measurable way: a template's bare +`json` bound to the one in-tree `def json` (422 call sites on application A), and a partial's local +bound to whichever class defines a method of that name. Outside any class Ruby's self is what runs the file: + +| File | Self | A receiver-less call reaches | +| --- | --- | --- | +| `.rake` | Rake's `main`: the file is loaded at top level, and a task's block is called as written | a top-level def, a reopened Object, Kernel or BasicObject | +| `.jbuilder` | the view: ActionView compiles a template into a method of it | the same, each method of a module in an app/helpers/ `*_helper.rb` file (Rails' `helper :all`), and each controller method a `helper_method` declaration names | + +Any other in-tree namesake is refused as external: a partial's local, a route helper, ActionView's own. A def in test code +is refused too (`filter.h isTestSymbol`): a def inside an example group's block has no class around it, and is the group's +method. `helper_method :a, :b` at class-body position is recorded as a `LocalBindKind::RubyHelperMethod` binding against +the declaring class or module (`ingest_binds.h captureRubyHelperMethods`). + +**A template's `json`** (`ingest_binds.h typeRubyJbuilderLocal`) is the local the template handler binds to the view's +JbuilderTemplate. A read of it is no call. A call on it (`json.summary …`, `json.partial! …`) is a key the class's +method_missing writes, and is refused as external unless the tree opens JbuilderTemplate. + +Measured with `--no-cache`, the Rails declared-call change below (parser version 134) against this change, both runs +on one snapshot of each tree. + +| Corpus | Files added (`.jbuilder` / `.rake`) | Edges | Call-graph isolated | Template sites: an edge / refused | Task-file sites: an edge / refused | +| --- | --- | --- | --- | --- | --- | +| activerecord 7.2.3.2 `lib/` | 0 / 1 | 11,381 → 11,436 | 2,552 → 2,551 | — | 119 / 34 | +| activesupport 7.2.3.2 `lib/` | none: default map byte-identical | | | | | +| Rails app A | 62 / 24 | 29,350 → 29,574 | 18,827 → 18,872 | 166 / 467 | 116 / 4 | +| Rails app B | 107 / 35 | 27,684 → 28,374 | 5,607 → 5,698 | 384 / 2,101 | 506 / 97 | +| this repo's `src/` (C++), a JavaScript app, a Python repo | none: default map byte-identical | | | | | + +No symbol that had a call edge lost it. Isolated rises because the added files bring symbols with no in-tree edge: a +template whose every call is a Jbuilder key or a framework helper, a task's private def. Of the refused template sites, +`json` keys are 420 on A and 1,818 on B, and bare words 45 and 278. Five `.rb` methods on A and three on B gain their +first caller. On application B, 23 call sites in `.rb` files also bind to methods of two classes a task file defines +(a model's reader, a demo seeder's), where a column of that name is what runs. That is floor (g) of the typed-receiver round. + +Sampled against the source: +- **The view's rules:** every decision sampled was right. All 37 bare calls on A that bind go into app/helpers + modules. B's 3 go into `helper_method`s. Of the bare words crawling alone had bound, 9 on A and 56 on B are now + refused, and each is a partial's local. +- **Calls on a value:** 15 of 20 template edges on A and 8 of 20 on B; 8 of 15 task-file edges on A and 4 of 15 on B. + Every miss is a call on an untyped receiver, or on an out-of-tree constant (`Hash.new`, `JSON.parse`), bound to the + one in-tree def of that name. That is the same floor as in every `.rb` file, not a rule of this change. In application + B's `.rb` files, `new` binds to one controller's `new` action at 2,509 call sites, and `where` to a mailer preview's mock + at 1,174. + +Stated floors, each pinned by `test/rubyrakejbuildercheck.sh`: +- **(a)** `json.extract! user, :email` names attributes by symbol, and they are not read as calls. +- **(b)** A `.rb` file's top level is left as before: a block there may run with another self (`describe`, + `routes.draw`, `FactoryBot.define`), so a receiver-less call outside any class binds by name. + +Not in this change: +- Sorbet `.rbi` stubs (291 files on application A): each declares a gem's methods, and would add a namesake for nearly + every gem call. +- ERB, Slim and HAML templates: no grammar for them is vendored. +- `Rakefile`, `Gemfile` and `*.gemspec`: the crawl is keyed by extension. +- A def in a block outside any class in application code (`Struct.new( … ) do … end`) reads as a top-level def. + +`kParserVer` 134 → 135. New records and an appended binding kind, same layout, so `kCacheVersion` stays 27, and Ruby +caches re-parse once. + +### Changed — a method a Rails declaration names by symbol has a caller: callbacks, conditions, rescue handlers, `send` + +`before_action :authenticate`, `after_save :reindex`, `validate :name_present`, an `if: :published?` condition and +`rescue_from …, with: :not_found` each name, by symbol, a method the framework calls later on the class's instances. +Nothing in the source calls it. Until now every callback method in a Rails app therefore had no caller, ranked as a +leaf, and `--callers` answered nothing for it. The same was true of a method reached through `send( :m )`. + +**What is read** (`ingest_binds.h RubyBareCallWalk::noteDeclaredCall`). Each such symbol is a receiver-less call from the +class body, resolved like any call to self there: the class's own method, a superclass's, or a concern's +(`test/rubyreachcheck.sh`). A method no class in reach defines (Devise's `authenticate_user!`) is refused, not bound to +a namesake. The macro must sit at class-body position: the innermost def-or-class wall is a class or module body. A +concern's `included do` block counts; a def does not. + +| Declaration | The symbols read as calls | +| --- | --- | +| A callback (`kRubyCallbackMacros`): `before_action`/`after_action`/`around_action` and their `prepend_`/`append_` forms, ActiveRecord/ActiveModel `before_*`/`after_*`/`around_*` and `validate`, ActiveJob `*_perform`/`*_enqueue`/`after_discard`, mailer delivery and ActionCable channel callbacks | each positional symbol, and the `if:`/`unless:` options (a symbol or an array of them) | +| `validates`, `validates_*_of`, `validates_with` | only `if:`/`unless:` (the positional symbols are attributes) | +| `rescue_from` | `with:` | +| `send`, `public_send`, `__send__`, `try`, `try!`, `method` with a literal symbol | that method, called on the same receiver (`sendTargets`) | + +The `send` family resolves exactly as the call written out would. The call's own reference is copied, renamed, and +placed at the symbol, typed receiver included, so `helper.try( :assist )` answers from `Helper`. + +Measured with `--no-cache`, the RSpec expectation-target change below (parser version 133) against this change, both runs +on one snapshot of each tree. + +| Corpus | Edges | Call-graph isolated | Sites gaining an edge (declarations / `send` family) | Newly refused | +| --- | --- | --- | --- | --- | +| activerecord 7.2.3.2 `lib/` | 11,275 → 11,381 | 2,604 → 2,552 | 30 (2 / 28) | 0 | +| activesupport 7.2.3.2 `lib/` | 3,500 → 3,535 | 1,292 → 1,283 | 34 | 1 | +| Rails app A | 28,734 → 29,350 | 19,071 → 18,827 | 651 (533 / 118) | 11 | +| Rails app B | 27,173 → 27,610 | 5,831 → 5,579 | 464 (405 / 59) | 1 | +| this repo's `src/` (C++), a JavaScript app, a Python repo | default map byte-identical | | | | + +Sampled against the source: +- **Declarations:** 20 of 20 correct across the two applications. Each bound to the class's own method, the superclass's + (an `ApplicationController` guard, from a subclass's `before_action`), or a concern's (an integration gate's + guard, a notifier concern's hook). +- **The `send` family:** 7 of 10 on application A and 8 of 10 on application B. On B the other two split across classes + that share a final name, including the right one (Round 3's floor (g)). On A the three misses bind an untyped + receiver's call to the one in-tree namesake (`x.try( :email )` → one model's `email`), as the + same call written out, `x.email`, already did. +- On activerecord, `became.send( :initialize )` on an untyped receiver splits across all 78 `initialize` definitions, as + an untyped `x.m` with 78 definers does. That one site carries 78 of activerecord's 106 new edges. + +Stated floors, each pinned by `test/rubydeclrefcheck.sh`: +- **(a)** Option values that are not calls make none: `only:`/`except:` (action names), `on:` (an event), `prepend:`. + Neither do `validates`' attribute names. +- **(b)** `skip_before_action :x` removes a callback and calls nothing. +- **(c)** `set_callback` and other ActiveSupport::Callbacks plumbing are not read. +- **(d)** A string or computed name (`send( "m" )`, `send( name )`) is not read. +- **(e)** A callback macro inside a method body is not read. + +Not in this change: methods that `delegate`, `alias_method` and `define_method` define. A def minted for them would +need to be visible only to calls on its own class. Round 2 tried minting delegate defs as ordinary defs and produced +388 wrong edges on application A. + +`kParserVer` 133 → 134. New records, same layout, so `kCacheVersion` stays 27, and Ruby caches re-parse once. + +### Changed — a call on RSpec's own expectation target is RSpec's: `expect( v ).to` no longer binds to an in-tree `def to` + +Inside an example group, `expect( v ).to eq( 1 )` calls `to` on the object `expect` returns, an +RSpec::Expectations::ValueExpectationTarget. Its `to`, `not_to` and `to_not` are RSpec's. That receiver was untyped, so +the call bound by name alone. On Rails application A, whose only `def to` is a presenter's, 13,480 expectations bound to +that presenter's `to`. That ranked it second in the default map, and a utility method its body calls, fourth. + +**What is typed** (`ingest_binds.h rubyValueType`, the walk Round 3 types receivers on; `model.h kRspecTargets`). Inside an +example group (a describe/context, a shared group, a `def` in one), a receiver-less builder returns an instance of one of +RSpec's target classes, typed like any built receiver: in place, or held by a local. + +| Builder | Class | +| --- | --- | +| `expect( v )`, a bare `is_expected` | `RSpec::Expectations::ValueExpectationTarget` | +| `expect { … }` | `RSpec::Expectations::BlockExpectationTarget` | +| `allow( v )` | `RSpec::Mocks::AllowanceTarget` | +| `expect_any_instance_of( K )` | `RSpec::Mocks::AnyInstanceExpectationTarget` | +| `allow_any_instance_of( K )` | `RSpec::Mocks::AnyInstanceAllowanceTarget` | + +**How the call resolves** (`graph.h RubyTypedReceivers::outOfTreeTarget`). When the tree never opens the class, only a +reopened `Object`, `Kernel` or `BasicObject` can answer. Otherwise the method Ruby runs is RSpec's, and the call is +refused as external. A tree that opens the class (RSpec's own source, or a patch) answers from it, as from any in-tree +class. A tree that defines a method named after the builder (`def expect`) types nothing from that name. + +Measured with `--no-cache`, the typed-receiver change below (parser version 132) against this change, both runs on +one snapshot of each tree. Call sites are keyed by (file, line, callee) with `--pin-census`. + +| Corpus | Edges | Call-graph isolated | Edges lost (refused) | Newly refused (no edge before) | +| --- | --- | --- | --- | --- | +| activerecord 7.2.3.2 `lib/`, activesupport 7.2.3.2 `lib/` | default map byte-identical | | 0 | 0 | +| Rails app A | 29,670 → 28,734 | 18,999 → 19,071 | 13,480 | 1,661 | +| Rails app B (no in-tree `def to`) | 27,173 → 27,173 | 5,831 → 5,831 | 0 | 25,365 | + +On application A, every lost edge was the presenter's `to`. Twelve sites sampled at random are all `expect( … ).to`. +The presenter drops from rank 2 of the default map's 191 rows to 126, and the utility method from 4 to 127. A single +line can carry two `to` calls (the matcher chain in floor (c)). On 78 lines one call is now refused and the other still +binds. On application B, only the header's `external=` count moves (6,965 → 32,331). + +Stated floors, each pinned by `test/rubyrspectargetcheck.sh`: +- **(a)** Outside an example group, the call binds by name as before: a helper module in `spec/support` that a + `config.include` mixes in, or an `RSpec.configure` hook. On application A that is 25 sites in `spec/support`. +- **(b)** A tree that defines a method named after the builder types nothing from that name. +- **(c)** Matcher chains are not typed. `receive( :m ).with( … )`, and the `to` of `change { }.from( a ).to( b )`, bind by + name as before (137 `to` sites on application A). +- **(d)** An `expect` with a receiver (`helper.expect( 1 )`) is not RSpec's builder. + +The class check also exposed a Round 3 gap. The opens a qualified type is checked against came from the base scope, which +is built only for a tree with a Ruby superclass, mixin or constant reference. In any other tree, a qualified constant was +never trusted. `rubyOpenedPaths` now builds the full constant index for such a tree (`resolve.h rubyConstantIndexOf`, the +ungated core of `buildRubyConstantIndex`), once a typed receiver is qualified. `test/rubytypedrecvcheck.sh` pins it with a +flat tree. + +`kParserVer` 132 → 133. The layout is unchanged (a type is a string the record already carries), so `kCacheVersion` +stays 27, and Ruby caches re-parse once. + +### Changed — a Ruby call on a receiver the code builds answers from that class's lookup: locals, lets, factories, constructions + +A Ruby receiver the code builds is an instance of one class, and Ruby's method lookup on that class decides which `def` +answers a call on it. Until now every such call bound by name alone — same file, same directory, the one definer in the +tree, else no edge: +- `c = Client.new` then `c.get` declined between `Client#get` and `Other#get`. +- `u = User.find( id )` then `u.email`, a column Rails generates, bound to an unrelated `Post#email`. +- In a spec, `let( :user ) { create( :user ) }` then `user.activate!` bound to `Post#activate!` whenever the file also + built a Post: the include narrow reads `Post.new` as an import, and `RSpec.describe User` names User only as an + argument. + +**What is typed** (`ingest_binds.h`, on the walk that already tracks Ruby's locals and RSpec's lets). A value is typed +when it is `Const.new( … )`, an Active Record finder on a constant (`find`, `find_by`, `create`, `first`, … ), +`described_class.new`, or a FactoryBot build (`create`/`build`/`build_stubbed( :user, … )` inside an example group, or +on `FactoryBot`). A call's receiver is typed when it is: +- one of those written in place: `Client.new.close`, `User.find_by( … ).activate!`, `create( :user ).name`; +- a local whose every binding in its scope builds the same one; +- the `let`/`subject` RSpec runs for that name there: the innermost, so a nested group's override wins, and a local of + the same name hides it. A group's implicit `subject` is its parent's explicit one, else `described_class.new`. + +A factory's class is FactoryBot's rule, read from the tree's `FactoryBot.define` blocks (`captureRubyFactories`): the +`class:` option as written, else the enclosing factory's, else the `parent:` factory's, else the name camelised. The +type rides the call reference's receiver text (`model.h rubyTypedRecvOf`) with the receiver kind unchanged, so every +rule that does not read it sees the call exactly as before. + +**How a typed call resolves** (`graph.h RubyTypedReceivers`). The class's lookup answers: its own methods, its mixins +and superclass chain over the Ruby overlay of the entry below (the shallowest level that defines the name decides), then a reopened +`Object`, `Kernel` or `BasicObject`. When none of those defines it, a class below may (Active Record's single-table +inheritance hands back a subclass), so the call keeps the candidates the class's instances reach (`RubySelfReach`). With +none left the call is refused as external (`external=`): a column, an association, `errors`, `reload`, a framework +method — the method Ruby runs is outside the tree. The type is used only when it means what it says: +- a factory with one class tree-wide; +- a class the tree defines, and for a qualified constant, one whose path the tree opens: `OpenSSL::Cipher.new` is not + activerecord's `Encryption::Cipher`, `Stripe::Customer` is not an app's `Customer` model; +- no class in its lookup defining the method that built it: a `def self.find` may return anything. + +Measured with `--no-cache`, the method-lookup change below (parser version 131) against these changes, both runs on +one snapshot of each tree. Edges and isolated symbols are `--report` totals. Call sites are keyed by (file, line, +callee) with `--pin-census`. + +| Corpus | Edges | Call-graph isolated | Sites gaining an edge | Retargeted | Edges lost (refused) | Newly refused (no edge before) | +| --- | --- | --- | --- | --- | --- | --- | +| activerecord 7.2.3.2 `lib/` | 11,278 → 11,275 | 2,611 → 2,604 | 33 | 31 | 0 | 2 | +| activesupport 7.2.3.2 `lib/` | 3,498 → 3,500 | 1,293 → 1,292 | 3 | 0 | 1 | 4 | +| Rails app A | 28,733 → 29,670 | 19,023 → 18,999 | 2,616 | 212 | 625 | 3,880 | +| Rails app B | 27,093 → 27,192 | 5,867 → 5,831 | 926 | 292 | 1,370 | 1,795 | +| this repo's `src/` (C++), a JavaScript app, a Python repo | default map byte-identical | | | | | | + +Most of the applications' change is in specs, where a `let` built by a factory is the commonest receiver there is. +Sampled against the source: +- **Edges lost were wrong:** 10 of 10 on each application. Every one bound a generated method — an association, a + column or its setter, `errors`, `reload`, CanCan's `can?` — to an unrelated class's namesake, a test stub's + `body` among them. +- **Added edges correct:** 10 of 10 on activerecord (Arel managers built in place: `um.take`, `sm.project`) and 10 of 10 + on each application. On application A, 2 of the 10 bind to a Sorbet `abstract` stub in a mixin (`def id; end` in + a reports module). Ruby's lookup does find the stub first: Rails includes its generated attribute and + association modules when the class is created, so a later `include` comes before them, and sorbet-runtime's stub + forwards to `super`. +- **Retargets better:** 9 of 10 on activerecord (1 neutral), 6 of 6 on application B (`service.call` had bound to + a middleware's `call`), 9 of 10 on application A (1 now split between two classes that share a final name). + +Stated floors, each pinned by `test/rubytypedrecvcheck.sh`: +- **(a)** A type the tree does not define (`Net::HTTP.new`, `SimpleDelegator.new`, a `Point = Struct.new( … )` constant) + changes nothing; neither does a qualified constant whose path the tree never opens. +- **(b)** A class that defines its own `self.find` or `self.new`, or has an ancestor that does, may return anything: a + local built that way is untyped. +- **(c)** Control flow is not read. A local bound any other way anywhere in its scope — another assignment, a parameter, + a block parameter — is untyped everywhere in that scope. +- **(d)** Instance variables, method return values, `x.class` and a `let` defined in a shared context in another file are + not typed. +- **(e)** A class whose lookup can leave it (a delegator, `method_missing`, the delegation DSL) refuses nothing, as for + a call to self (the entry below). +- **(f)** Two factories sharing a name with different classes type nothing. +- **(g)** A method defined only through `alias`/`alias_method` is no definition the tree indexes, so a typed call to it + is refused rather than bound to another class's namesake (activerecord's `sm.limit = 1`, which had no edge before + either). + +Classes are keyed by final segment, as everywhere in the graph: two classes sharing a name share a lookup (the +split above; `test/rubyreachcheck.sh` floor (e)). + +Two earlier floors are lifted, and their gate arms are kept inverted: `test/rubyrecvnarrowcheck.sh` (c), where +`Calc.new.scale` now pins to `Calc::scale`, and `test/rubydescribedclasscheck.sh` (a), where `described_class.new.m` +now pins to the described class. + +`kParserVer` 131 → 132. Same layout (a typed receiver is a string the record already carries; factories are bindings of +the appended `LocalBindKind::RubyFactory`): `kCacheVersion` stays 27, and Ruby caches re-parse once. + + +### Changed — a Ruby call to self reaches what Ruby's method lookup reaches: mixins are ancestors, a class body's self is the class, an unreachable namesake is no target + +A receiver-less Ruby call (`m`, `m( x )`) or `self.m` inside a class or module sends `m` to self. Ruby answers it from +self's ancestors: the class, its mixins, its superclass and theirs. The resolver read three things wrong: +- **Mixins were not ancestors.** `include Hub::SignalMixin` put nothing in the caller's base walk, so `signal` + split between the mixin's def and an unrelated same-file `Hub#signal`, and a concern's call to its includer's + method declined. Each constant of an `include`/`extend`/`prepend` at class-DSL position, or inside a concern's + `included do … end`, now mints an inherit reference (`ingest_relations.h captureRubyMixinBases`). It is scoped by + Ruby's constant lookup, starting inside the class (`resolve.h rubyScopeMixinReferences`). A concern's nested + `module ClassMethods` is an ancestor of the concern, since `ActiveSupport::Concern` extends it onto every includer. +- **A class-body call ran against the namespace around the class.** `define_section :first` inside `Ns::Catalog` bound to + `Ns.define_section`. Its self is now the class (`graph.h callerSelfScope`). +- **A candidate no lookup can reach still bound by name.** Examples: + - a controller's `render` bound to a component's `def render`; + - every bare `raise` in activesupport bound to `ProxyObject#raise`; + - `request` bound to the `request` of a class the controller only names, because Rule 3's include narrow read a + constant reference as evidence; + - a scope lambda's `where` bound to an unrelated `def where`. + + `graph.h RubySelfReach` now keeps only the candidates whose owner lies in self's reach: the ancestors of self, and of + every class below it. So a template method reaches a subclass's hook, and a concern its includer's method. A call left + with none is vetoed as external (`external=`): the method Ruby runs is outside the indexed tree. + +Mixins feed only these Ruby self-call reads, through an overlay (`graph.h buildRubyCha`). Keyed by name like every base, +a generic module name in the shared inheritance graph widened CHA-lite cones and the builtin-method gate's evidence for +receiver calls (activerecord's `sql.dup` bound to `Inheritance::ClassMethods#dup`). A `ClassMethods` key is qualified by +the module that holds it, since dozens of concerns each nest one. + +What the rule leaves alone: +- **The implicit roots.** A method of a reopened `Object`, `Kernel`, `BasicObject`, `Module` or `Class` (activesupport's + core extensions), since every self has them as ancestors. +- **Top-level defs**, but only while no class in reach has an out-of-tree superclass or mixin. Ruby asks + `ActionController::Base` before `Object`, so a controller's `render` no longer reaches a top-level `def render` in a + spec (453 call sites on one application). +- **A class that answers any name**: one defining `method_missing`, one writing the delegation DSL (`delegate`, + `delegate_missing_to`, `def_delegator(s)` define methods the tree does not index yet), or one below `SimpleDelegator`, + `Delegator` or a Draper decorator. +- **A call outside any class**: a script's top level, or an RSpec example group's blocks. + +Measured with `--no-cache`, the bare-word-call changes below (parser version 130) against these changes. Edges and +isolated symbols are `--report` totals. Call sites are keyed by (file, line, callee) with `--pin-census`. + +| Corpus | Edges | Call-graph isolated | Sites gaining an edge | Retargeted | Refused (`external=`) | +| --- | --- | --- | --- | --- | --- | +| activerecord 7.2.3.2 `lib/` | 11,169 → 11,278 | 2,564 → 2,611 | 147 | 27 | 205 | +| activesupport 7.2.3.2 `lib/` | 3,717 → 3,498 | 1,279 → 1,293 | 11 | 6 | 275 | +| Rails app A | 30,044 → 28,733 | 18,850 → 19,023 | 91 | 277 | 1,758 (+6 declined) | +| Rails app B | 27,695 → 26,762 | 5,752 → 5,853 | 161 | 10 | 1,521 (+24 declined) | +| this repo's `src/` (C++), a JavaScript app, a Python repo | default map byte-identical | | | | | + +Sampled against the source: +- **Refusals that removed a wrong edge:** 13 of 15 on activerecord, 15 of 15 on activesupport, 14 of 15 and 15 of 15 on + the applications. The losses are calls inside a `Class.new( ActiveRecord::Base ) { … }` block (floor (d)). +- **Added edges correct:** 15 of 15 on activerecord, 14 and 15 of 15 on the applications. They are mostly mixin + base-walk hits: an adapter's `execute` reaching `DatabaseStatements`, a concern's call reaching its host. +- **Retargets better:** 7 of 10 on activerecord (2 neutral, 1 now a split), 12 of 12 and 6 of 8 (2 neutral) on the + applications. + +Stated floors, each pinned by `test/rubyreachcheck.sh`: +- **(a)** A class whose lookup can leave the class refuses nothing (see above). +- **(b)** A class that writes the delegation DSL is exempt. Indexing delegated names as methods needs a call-to-self-only + visibility, so a receiver call of that name is not taken as evidence; that is a later round's job. The DSL call itself + is no exception: on activesupport a `delegate :x, to: :y` line has two namesakes in reach, core_ext's `Module#delegate` + and `Notifications::Fanout`'s `attr_reader :delegate`, and declines. +- **(c)** Instance and class methods share one name space: `extend M` and `include M` both put M in reach. +- **(d)** Inside `instance_eval`, `class_eval`, `instance_exec` or `Class.new` blocks the rule reads the lexical self, + as Rule 1 always has. +- **(e)** Reach is read by class name, as the inheritance graph is keyed, so two classes sharing a name share their + reach. This refuses less, never more. +- **(f)** A call outside any class is untouched. +- **(g)** `prepend` is read as `include`. +- **(h)** A top-level def is reachable only while no class in reach has an out-of-tree ancestor (above). +- **(i)** A mixin written inside a method body runs when the method does, and is not read. Activerecord's `primary_key=` + includes `CompositePrimaryKey` this way. +- **(j)** Rails' shared view object, into which every `app/helpers` module is mixed, is not modelled. +- **(k)** A core class's own ancestry beyond the implicit roots is not modelled. A reopened `class Array` does not reach + a reopened `Enumerable`. + +`kParserVer` 130 → 131. New records, same layout: `kCacheVersion` stays 27, and Ruby caches re-parse once. + +### Added — a Ruby bare-word call is a call: `full_name`, `render_profile`, the `items` of `items.sum` + +A Ruby call with no receiver, no arguments and no parentheses parses as a plain `(identifier)` in tree-sitter-ruby. +That is the same node a local-variable read is, so `queries/ruby/tags.scm` never captured it, and one Ruby call site +in five minted no reference at all. Measured with Prism's `CallNode#variable_call?`: 4,157 of 28,584 call sites in +activerecord 7.2.3.2 `lib/`, and 50,893 of 250,320 and 56,897 of 295,879 in two Rails applications. + +Ruby's parser decides call-versus-local **lexically**, and `ingest_binds.h::captureRubyBareCalls` applies the same +rule in one source-order walk: +- An identifier is a local exactly when an assignment, parameter or pattern binds its name **earlier** in its own scope + or in an enclosing block's, up to the nearest `def`, `class`, `module` or `class <<`. +- Every other bare identifier becomes a receiver-less call reference. That is the implicit-self shape resolve.h's + Rule 1 and its base walk already read. + +The binding kinds are the one table #338's `described_class` floor already used (`kRubyLocalKinds`). It gains the +binders Ruby has beyond assignment and parameters: `for`, pattern matching (`in x`, `=> x`, array and find patterns, +`in { x: }`), and a regex literal's named groups on the left of `=~`. So `described_class` reads those locals too. +A few more shapes: +- A hash or keyword-argument shorthand (`{ payload: }`) reads `payload`. +- Inside a block with no parameter list, `it` and `_1` … `_9` are its implicit parameters. +- RSpec: a group's `let` / `let!` / `subject` / `subject!` (and test-prof's `let_it_be` family), and the implicit + `subject`, bind like locals over the whole group. That covers lines before the `let`, nested groups and a `def` + inside the group, so a spec's `user` never reaches an application `def user`. They are not minted as definitions: + application code could then name them, and 1,818 and 1,036 application call sites to a uniquely-defined method + share a name with a `let` on the two applications. + +### Fixed — Ruby has no declarations: an `attr_reader` accessor or an empty `def` hook is the definition a call names + +`model.h::isDefinitionNotDeclaration` (`endByte > sigEndByte`, plus a Kotlin type) decides what graph.h's decl/def +collapse may evict and what `canonByName` holds. Ruby has no prototype, abstract member or interface: every class, +module, `def`, attr accessor and constant it indexes is a definition. A body-less one read as a declaration and was +evicted whenever a same-named bodied `def` existed anywhere. So inside `Tracker`, `aliases`, `self.aliases` and +`aliases()` all bound to an unrelated `Join#aliases`, and an inherited empty hook lost to another class's method. + +The predicate gains a Ruby clause beside the Kotlin one. The collapse also gives Ruby its own family, as Kotlin has +one: in the shared family a Ruby `def` evicted a same-named C prototype (a C extension's header), and the C call read +unresolved. + +Measured with `--no-cache`, `main` (953818d6, parser version 129) against these changes. Edges and isolated symbols +are `--report` totals; `declined=` is the map header's. + +| Corpus | Edges | Call-graph isolated | `declined=` | +| --- | --- | --- | --- | +| activerecord 7.2.3.2 `lib/` | 8,146 → 11,169 | 3,221 → 2,564 | 5,367 → 5,485 | +| activesupport 7.2.3.2 `lib/` | 3,094 → 3,717 | 1,468 → 1,279 | 1,591 → 1,695 | +| Rails app A | 27,027 → 30,044 | 20,059 → 18,850 | 41,265 → 54,418 | +| Rails app B | 24,394 → 27,695 | 6,686 → 5,752 | 34,281 → 39,021 | +| this repo's `src/` (C++), a JavaScript app, a Python repo | default map byte-identical | | | + +How the new edges were checked (`--pin-census`, keyed by call site): +- **Every decided bare call is a Prism `variable_call?`** at the same file, line and name: 3,144 on activerecord, 766 + on activesupport, 7,887 and 8,498 on the applications. +- **Lost edges.** 195, 23, 6,449 and 1,372 call sites lost their edge. Each had been pinned to one def only because the + name's other definers were evicted, and it now declines. Of 50 sampled (10 activerecord, 20 per application), none + was correct. Examples: every RSpec `context "…" do` in app A was bound to one service's `context`, and + `self.class.name` to one model's `name`. +- **Added edges.** 21 of 22 sampled non-split activerecord edges were correct, and 29 of 40 application edges. +- **Baseline.** A sample of the applications' pre-existing edges reads 14 of 24 correct. + +Stated floors, each pinned by `test/rubybarecallcheck.sh`: +- **(a)** `defined?( x )` calls nothing. +- **(b)** A `let` from a shared context is unknown in the including file, so there the name reads as a call. +- **(c)** Locals made at run time (`binding.local_variable_set`, `eval`) are invisible to a lexical rule. +- **(d)** Inside `instance_eval` / `class_eval` blocks, a bare call resolves against the lexical class, as a + parenthesised `m()` always has. +- **(e)** The RSpec DSL call itself (`subject( :x ) { … }`, `it "…" do`) is an ordinary call and binds by name. + +A known limit of the ladder, not of these rounds: a method Rails **generates** (a column, an association, +ActiveModel's `errors`, Devise's `current_user`) has no in-tree definition. A call to it can therefore land on the +one unrelated in-tree method of that name. For example, app A's 269 `errors` edges to one concern's +`def errors; end` stub grow to 455. + +The same fix puts every attr accessor on the name ladder, so the untyped-receiver limit also reaches calls that bound +nothing before. On activerecord `connection_class.name` now reaches `Column#name`, and on activesupport the cache's +`entry.value` splits between `Duration#value` and `Scalar#value`; in a sample taken in review, about 4 in 10 such newly +bound calls named `Class#name` or another class's reader. An accessor can also become a second definer: `ActiveRecord::Base.configurations` (a +`def self.configurations` in `Core`) now meets `DatabaseConfigurations`' `attr_reader :configurations` on the ladder, +since `ActiveRecord::Base` extends `DynamicMatchers`, whose `method_missing` leaves its class-object lookup open (the +class-object entry above). Of the 13 methods that called `Core.configurations`, 5 now split between the two and 7 +decline; the 13th, `configs.configurations` in `DatabaseConfigurations#build_configs`, now reaches the reader, as it should. +The include-and-extend entry at the top of this section consults that `method_missing` only after the lookup misses, +which puts the 12 back on `Core`'s def as single edges. + +`kParserVer` 129 → 130. New records, same layout: `kCacheVersion` stays 27, and Ruby caches re-parse once. ### Added — MCP `grep` rows carry the matched text and a `fetch_body` handle (CLI parity) @@ -434,10 +1172,12 @@ per page instead. Gate: `test/impactdepthcheck.sh`. ### Changed — the versions this release moves, stated once -`kParserVer` 124 → 133 (the function-literal fix takes 128; #338 and #325 take 129; the body-less C/C++ type-specifier -span fix and the TypeScript `await f(x)` / `!f(x)` calls each took a number of their own on their branches, and 133 -then sits above every number a branch build of unreleased work has used, so no cache such a build wrote is read as this -release's), `kCacheVersion` 25 → 27 +`kParserVer` 124 → 145 (the function-literal fix takes 128; #338 and #325 take 129; the body-less C/C++ type-specifier +span fix and the TypeScript `await f(x)` / `!f(x)` calls each took a number of their own on their branches, as did +the Ruby method-lookup changes, one per step from 130 to 137: bare-word calls, mixins, typed receivers, RSpec targets, +Rails declared calls, Rake and Jbuilder files, RSpec matcher chains, class objects, then 145 for the review's +include-and-extend change. 145 sits above every number a branch build of unreleased work has used, so no cache such a +build wrote is read as this release's), `kCacheVersion` 25 → 27 (the function-literal fix's record changes) and `kQSnapCacheScheme` 15 → 16 (the `--quality-delta` error-masking and placeholder changes). Every ingest cache written by an earlier build is refused and re-indexed once, and every cached quality snapshot is recomputed. The session legend dictionary is `dictv=66409821069cf5cb entries=775`. diff --git a/README.md b/README.md index 38a852754..9ea4d5938 100644 --- a/README.md +++ b/README.md @@ -2082,9 +2082,9 @@ wrong, and it has. These are the results that say so, all in-tree, all published ### In the tests
-662 gate scripts, five contracts no unit test can hold, and the house rule: write the gate before the code it measures +669 gate scripts, five contracts no unit test can hold, and the house rule: write the gate before the code it measures -`test/regression.sh` names **662 gate scripts** and is the authoritative list; +`test/regression.sh` names **669 gate scripts** and is the authoritative list; `python3 test/pargates.py . ./build/ripwire -j 6` runs the same set in parallel. On top of them sit the contracts that do not fit a unit test: two runs byte-identical, warm output identical to cold, output that pipes clean through `xmllint --noout`, a sanitizer build with `-fno-sanitize-recover=all`, and a diff --git a/docs/EVALS.md b/docs/EVALS.md index 014eebc5d..9da43e4ea 100644 --- a/docs/EVALS.md +++ b/docs/EVALS.md @@ -21,7 +21,7 @@ section, and it is not an afterthought. | **Co-change / known-item evals** | `--eval`, `--eval-retrieval` (see `bench/ANSWERQUALITY.md`) | Whether the tool surfaces the other files a real historical commit touched; and known-item retrieval across four rankers. | | **Ensemble calibration harness** | `bench/ensemblecal/` | Whether `--ensemble`'s four evidence families are actually orthogonal, how often each fires, how stable each is across commits — and the preset ladder derived from that (§9). | | **Differential argv harness** | `test/argvdiffcheck.sh` | That a refactor changed *nothing observable*: two binaries, every argv vector, stdout + stderr + exit code byte-identical. | -| **The gate suite** | `test/regression.sh`, `test/pargates.py` | 662 gate scripts plus the determinism, cache-transparency and golden contracts. | +| **The gate suite** | `test/regression.sh`, `test/pargates.py` | 669 gate scripts plus the determinism, cache-transparency and golden contracts. | | **`--quality-delta`** | `src/quality.h` | Ten measured code-quality failure modes, reported only where a change made them worse. | ### The labeling protocol (why the held-out eval is allowed to disagree with the ranker) @@ -5866,7 +5866,7 @@ copy here would be exactly the dialect divergence that gate exists to catch. Com tags, wrap, stable-order defaults), seven individually invoked standalone gates (`g1freshcheck`, `skillscan`, `htmlexport`, `compresscheck`, `handoffcheck`, `releaseinstallcheck`, `taskroutecheck`), and a single loop -naming **662 gate scripts**, all of which exist on disk. +naming **669 gate scripts**, all of which exist on disk. `python3 test/pargates.py . ./build/ripwire -j 6` runs the same scripts in parallel so a full verification fits in one sitting. It does not modify `regression.sh`. @@ -6969,7 +6969,7 @@ Listed because the reason is more useful than the silence. shipped**. See `bench/locbench/anchorhop_calib.json`. The mention anchor's reproducible numbers are the ablations in §4. - **A single round gate-count.** Two in-tree numbers disagree (`test/pargates.py`'s docstring says - ~210; `test/argvdiffcheck.sh` says 200+), while the loop in `test/regression.sh` names 662. The + ~210; `test/argvdiffcheck.sh` says 200+), while the loop in `test/regression.sh` names 669. The loop is the authority; the stale docstrings are a known drift. Since 2026-09-10 the number is not written by hand anywhere: `docs/gatecount_build.py` derives it from the loop and rewrites every published site, `test/gatecountcheck.sh` fails if any of them drifts, and `test/manifestcheck.sh` diff --git a/present/deck5_ripwire_build.js b/present/deck5_ripwire_build.js index 84a7816eb..3749312d4 100644 --- a/present/deck5_ripwire_build.js +++ b/present/deck5_ripwire_build.js @@ -1214,7 +1214,7 @@ function storyCards(s, { kick, head, stories, footText }){ kicker(s, "// how it stays true", AMBER); title(s, "Proven, not promised"); const cards = [ - ["662 gate scripts", "the suite runs on every push — plus determinism, cache-transparency and golden contracts; the gate count itself is gated against the runner's own loop"], // gatecount + ["669 gate scripts", "the suite runs on every push — plus determinism, cache-transparency and golden contracts; the gate count itself is gated against the runner's own loop"], // gatecount ["byte-identical, always", "two runs over the same tree produce the same bytes; warm equals cold. Enforced in CI, twice — Release AND a plain flavour, because NDEBUG once blinded a whole class of checks"], ["differential refactoring", "a refactor must prove it changed nothing observable: two binaries, hundreds of argv vectors, stdout + stderr + exit codes byte-identical"], ["held-out labels, authored blind", "eval labels were written by reading source before the ranker ever ran on them — so the eval is allowed to say the ranker is wrong. It has."], @@ -1238,7 +1238,7 @@ function storyCards(s, { kick, head, stories, footText }){ title(s, "Claims you can trust, because we publish what failed", { size: 32 }); card(s, MX, 1.72, 3.86, 1.72); - stat(s, "662", "gate scripts named by test/regression.sh — and the COUNT itself is gated against the runner's own loop, so it cannot go stale quietly", // gatecount + stat(s, "669", "gate scripts named by test/regression.sh — and the COUNT itself is gated against the runner's own loop, so it cannot go stale quietly", // gatecount MX+0.15, 1.86, 3.56, CYAN, { bsize: 42, bh: 0.66, lsize: 9.5 }); card(s, 4.68, 1.72, 3.86, 1.72, CARD2); stat(s, "8", "registered NEGATIVES — changes built, gated green, measured against a band written before the code, and reverted rather than tuned", @@ -1540,7 +1540,7 @@ function storyCards(s, { kick, head, stories, footText }){ ["185 long flags · 36 slides", "bash test/deckclaimcheck.sh"], ["every --flag named here exists", "bash test/deckcheck.sh"], ["74.7% fewer element bytes", "bash test/showcasecapturecheck.sh"], - ["662 gate scripts", "bash test/manifestcheck.sh"], // gatecount + ["669 gate scripts", "bash test/manifestcheck.sh"], // gatecount ["49 repos · 71 papers · 237 surveyed","bash test/readmedriftcheck.sh"], ["the ten moments, any row", "ripwire . --callers=SYM | wc -c"], ["the head-to-head table", "bench/headtohead/r4-2026-08-06/"], diff --git a/queries/ruby/tags.scm b/queries/ruby/tags.scm index 87e8027d9..d83d44ba3 100644 --- a/queries/ruby/tags.scm +++ b/queries/ruby/tags.scm @@ -5,9 +5,11 @@ ; Ruby structure the call graph cares about: ; - class / module definitions → def nodes (containers; module is a namespace/mixin) ; - `def name` / `def self.name` method definitions → the def nodes calls resolve TO -; - method calls → the call references (edges). A bare `foo` with no receiver and no -; args parses as (identifier); `foo(..)` / `obj.foo` parses as (call). We capture the -; (call) form's method name — the reliable, unambiguous call site. +; - method calls → the call references (edges). `foo(..)` / `obj.foo` parses as (call), and +; this query captures the (call) form's method name. A bare `foo` with no receiver, args or +; parentheses parses as (identifier) — the same node a local-variable read is — and no query +; can tell the two apart: ingest_binds.h captureRubyBareCalls mints those calls by Ruby's own +; lexical local rule instead (parser version 130, test/rubybarecallcheck.sh). ; - require / require_relative → reference edges (they ARE plain (call) nodes, captured ; by the call rule below — no separate rule needed). ; diff --git a/src/darkflags.h b/src/darkflags.h index 5a386052c..9fda99a42 100644 --- a/src/darkflags.h +++ b/src/darkflags.h @@ -449,7 +449,7 @@ inline constexpr std::int32_t kTemplateText = -1; inline constexpr std::string_view kShellExtTable[] = { ".sh", ".bash", ".zsh" }; inline constexpr std::string_view kHashCommentExtTable[] = { - ".py", ".pyi", ".sh", ".bash", ".zsh", ".rb", ".pl", ".cmake", ".yml", ".yaml", ".toml", ".r", ".jl" + ".py", ".pyi", ".sh", ".bash", ".zsh", ".rb", ".rake", ".jbuilder", ".pl", ".cmake", ".yml", ".yaml", ".toml", ".r", ".jl" }; inline LineSyntax lineSyntaxFor( std::string_view path ) diff --git a/src/graph.h b/src/graph.h index ec2d37814..4ba39de4a 100644 --- a/src/graph.h +++ b/src/graph.h @@ -885,6 +885,14 @@ inline bool keepStdQualifiedCandidates( const IngestResult& ing, const Reference return keepCount != 0; } +// The decl/def collapse FAMILY of a language (collapseDeclarationsOfName below): a definition evicts a declaration only +// inside its own family. Kotlin is 1, Ruby is 2, and every other language shares 0. +inline constexpr std::size_t kCollapseFamilies = 3u; +inline std::size_t collapseFamilyOf( Lang lang ) noexcept +{ + return lang == Lang::Kotlin ? 1u : lang == Lang::Ruby ? 2u : 0u; +} + // THE DECL/DEF COLLAPSE, one name at a time (buildGraph step 1e; adversarial-review #1). A C++ header declaration and its // .cpp definition are two same-named symbols. Left alone they make tier 3 see two candidates and DROP every cross-directory // call to the function, and let a bodyless prototype shadow its own body in the same-file and same-directory tiers. So once @@ -894,22 +902,28 @@ inline bool keepStdQualifiedCandidates( const IngestResult& ing, const Reference // A declaration is evicted only by a definition of its own COLLAPSE KEY: // * its ROOT, in a multi-root workspace — root A's body must not evict root B's decl-only best-available target, so each // root resolves exactly as it does alone; -// * its FAMILY — Kotlin rows with Kotlin rows, and every other language together, which is what the whole collapse always -// was (the C-family bridge's header/.c pairing lives inside that one family). Kotlin shares CANDIDATES with Java through +// * its FAMILY (collapseFamilyOf) — Kotlin rows with Kotlin rows, Ruby rows with Ruby rows, and every other language +// together, which is what the whole collapse always was (the C-family bridge's header/.c pairing lives inside that one +// family). Kotlin shares CANDIDATES with Java through // langCompatible's JVM bridge, but never a declaration: no Java interface method is a prototype of a Kotlin function, // or the reverse. Collapsed together, a Kotlin body evicted a Java interface-only declaration — so ADDING a .kt file // moved a Java call's edge onto Kotlin code — and a Java body evicted a Kotlin interface member. A tree without a .kt // file has one family and collapses byte-identically. Gate: test/kotlincheck.sh §13, and §14c's invariant. +// Ruby is a family of its own for the same reason: no Ruby def is the body of another language's prototype, and +// every Ruby symbol is a definition (model.h isDefinitionNotDeclaration), so in the shared family each one evicted +// a same-named C declaration — a C extension's header — and the C call read unresolved. In its own family a Ruby +// symbol evicts nothing and is never evicted. A tree without a .rb file collapses byte-identically. +// Gate: test/rubybarecallcheck.sh ("Ruby has no declarations"). // One pass marks which keys hold a definition and one keeps — O(K) per name, where the per-root version it replaces // rescanned the name's ids once per declaration. `ids` keeps its order, and is untouched when nothing is evicted. inline void collapseDeclarationsOfName( const IngestResult& ing, bool multiRoot, rw::SmallVec& ids ) { - std::array keyHasDefinition {}; + std::array keyHasDefinition {}; const auto keyOf = [ & ]( NodeId id ) noexcept -> std::size_t { const Symbol& s = ing.symbols[ id ]; const std::size_t root = multiRoot ? std::min( ing.fileRoot[ s.fileId ], kMaxWorkspaceRoots - 1u ) : 0u; - return 2u * root + ( s.lang == Lang::Kotlin ? 1u : 0u ); + return kCollapseFamilies * root + collapseFamilyOf( s.lang ); }; bool anyDefinition = false; for( NodeId id : ids ) @@ -2659,255 +2673,1955 @@ struct BuiltinMethodGate } }; -// The gate's evidence, built once per graph. Deterministic: class ids follow symbol-id order, and each file's list is -// sorted and deduplicated before any lookup. Files with no symbol in a gated language get no list (their calls are -// never gated), which keeps a mostly-C++ corpus from paying for a Python corner of it. -// -// collectFileClassEvidence: every class NAME each gated file mentions — the classes it defines, any class a reference -// there names as callee, receiver or qualifier, or a binding names as type, variable or imported name — and every class -// an ES import binding in that file resolves to (a default or renamed import names it under another spelling). -// One class NAME a gated file mentions. A DEFINITION (the class-like symbol, or the bare declared name its statement -// leaves) names it in fileClasses only; any other mention names it in fileRefClasses too — the list a Python call on a -// receiver other than self/cls reads (BuiltinMethodGate::judge). -inline void noteFileClass( BuiltinMethodGate& gate, const std::vector& fileGated, std::uint32_t fileId, const std::string& name, bool definition ) +// The class a call runs against, keyed as canonByName and the inheritance graph key a class (its name). Inside a def it +// is the def's scope. A Ruby call in a class or module BODY runs with that class as self — a DSL call there +// (`define_section :x`, `scope :y, -> { where( … ) }`) — while the class symbol's own scope names the namespace around it. +inline const std::string& callerSelfScope( const IngestResult& ing, const Reference& r ) noexcept { - if( fileId >= fileGated.size() || fileGated[ fileId ] == 0 || name.empty() ) - { - return; - } - const auto it = gate.classId.find( name ); - if( it == gate.classId.end() ) + const Symbol& from = ing.symbols[ r.fromSymbol ]; + return ( r.lang == Lang::Ruby && BuiltinMethodGate::isClassLike( from ) ) ? from.name : from.scope; +} + +// Every name reached from `start` along `edges`, `start` first, breadth-first, each once. +template +inline void forEachNameReached( const HashMap>& edges, const std::string& start, Visit&& visit ) +{ + std::vector queue{ &start }; + HashMap seen; + seen.try_emplace( start, 1 ); + for( std::size_t at = 0; at < queue.size(); ++at ) { - return; + visit( *queue[ at ] ); + if( const auto next = edges.find( *queue[ at ] ); next != edges.end() ) + { + for( const std::string& n : next->second ) + { + if( seen.try_emplace( n, 1 ).second ) + { + queue.push_back( &n ); + } + } + } } - gate.fileClasses[ fileId ].push_back( it->second ); - if( !definition ) +} + +// `start` and every name reached from it along `edges` (forEachNameReached's order), memoised per start in `memo`. +inline const std::vector& memoNamesReached( const HashMap>& edges, + HashMap>& memo, const std::string& start ) +{ + if( const auto hit = memo.find( start ); hit != memo.end() ) { - gate.fileRefClasses[ fileId ].push_back( it->second ); + return hit->second; } + std::vector out; + forEachNameReached( edges, start, [ & ]( const std::string& a ) { out.push_back( a ); } ); + return memo.emplace( start, std::move( out ) ).first->second; } -// The class names one binding mentions: its type, its variable and its imported name. A bare declared NAME with no type -// and no import is how Python records the `class Pool:` statement's own module-level name (a VarDecl) — the definition -// again, not a use of the class — so that variable names it as a definition. -inline void noteBindingClasses( BuiltinMethodGate& gate, const std::vector& fileGated, const Binding& b ) +// The out-of-tree superclasses below which a class forwards what it lacks — Ruby's stdlib delegators and Draper's +// decorator, read by final segment as the inheritance graph names every base. +inline constexpr std::string_view kRubyDelegatorBaseNames[] = { "Decorator", "Delegator", "SimpleDelegator" }; + +// Every self's implicit ancestors: each object is an Object, so a Kernel and a BasicObject, and a class body's self is a +// Class, so a Module. A tree that REOPENS one (activesupport's core_ext: Object#blank?, Kernel#silence_warnings, +// Class#class_attribute) defines methods any self reaches. +inline constexpr std::string_view kRubyImplicitRoots[] = { "BasicObject", "Class", "Kernel", "Module", "Object" }; +// An instance's implicit ancestors — a typed receiver's, or `main`'s and a view's outside any class: not a Class or Module. +inline constexpr std::string_view kRubyInstanceRoots[] = { "BasicObject", "Kernel", "Object" }; + +// The delegation DSL (model.h kRubyDelegationCalls) is read here too: a class that writes one may answer a name no lookup over +// the tree finds. + +// Ruby's ancestors include its MIXINS (parser version 131), and they feed only what a Ruby call to self reads — Rule 1's +// base walk and RubySelfReach — through this overlay: the shared inheritance graph plus each in-tree mixin edge, built +// only when the tree holds one. Keyed by module NAME like every base, a generic module name (`ClassMethods`, which +// dozens of concerns define) merges every includer into one cone; in the shared graph that widened what its other +// readers decide — CHA-lite, the builtin-method gate's file evidence, class identity — and bound activerecord's +// `sql.dup` to Inheritance::ClassMethods#dup. Here it only ever ADDS reach, which refuses less. +struct RubyCha { - const bool bareDeclaration = b.kind == LocalBindKind::VarDecl && b.typeName.empty() && b.importedName.empty(); - noteFileClass( gate, fileGated, b.fileId, b.typeName, false ); - noteFileClass( gate, fileGated, b.fileId, b.var, bareDeclaration ); - noteFileClass( gate, fileGated, b.fileId, b.importedName, false ); + HashMap> up, down; // empty unless merged + bool merged = false; +}; + +// A module's nested `module ClassMethods` is extended onto every class that includes the module — ActiveSupport::Concern +// does it, and so does the `def self.included( base ) base.extend ClassMethods` idiom it replaced — so in the overlay it +// is one of the module's ancestors (`Callbacks` → `Callbacks::ClassMethods`: an includer's class-body `define_callbacks` +// reaches it). QUALIFIED by the module that holds it, unlike every other name here: dozens of concerns each nest one, +// and keyed by the bare name every includer of any concern would reach all of them (activerecord's AbstractAdapter +// bound `current_transaction` to Transactions::ClassMethods that way). RubySelfReach reads a candidate's qualified +// owner by span (rubyClassMethodsOwners). +inline constexpr std::string_view kRubyConcernClassMethods = "ClassMethods"; + +// A nested `module ClassMethods` — a Ruby module of that name with a module around it. +inline bool rubyIsConcernClassMethods( const Symbol& s ) noexcept +{ + return s.lang == Lang::Ruby && s.kind == SymKind::Other && s.name == kRubyConcernClassMethods && !s.scope.empty(); } -inline void collectFileClassEvidence( const IngestResult& ing, const std::vector& fileGated, const JsImportTables& jsImports, BuiltinMethodGate& gate ) +// The overlay key a mixin names: its final segment, as every base is keyed — except a `ClassMethods` mixin (`extend +// ClassMethods`), which is keyed by the module its resolved constant sits in (`A::Callbacks::ClassMethods` → Callbacks). +inline std::string rubyMixinKey( const std::string& resolved, const std::string& written ) { - gate.fileClasses.assign( ing.files.size(), {} ); - gate.fileRefClasses.assign( ing.files.size(), {} ); - const auto note = [ & ]( std::uint32_t fileId, const std::string& name, bool definition = false ) + const std::size_t cut = resolved.rfind( "::" ); + if( written != kRubyConcernClassMethods || cut == std::string::npos ) { - noteFileClass( gate, fileGated, fileId, name, definition ); + return written; + } + const std::size_t before = resolved.rfind( "::", cut - 1 ); + const std::size_t from = before == std::string::npos ? 0 : before + 2; + return resolved.substr( from, cut - from ) + "::" + written; +} + +inline RubyCha buildRubyCha( const IngestResult& ing, const RubyBaseScope& bases, const HashMap>& chaUp, + const HashMap>& chaDown ) +{ + RubyCha out; + const auto addEdge = [ & ]( const std::string& derived, const std::string& base ) + { + if( !out.merged ) + { + out.up = chaUp; + out.down = chaDown; + out.merged = true; + } + out.up[ derived ].push_back( base ); + out.down[ base ].push_back( derived ); }; + for( std::size_t i = 0; i < ing.references.size(); ++i ) + { + const Reference& r = ing.references[ i ]; + const std::string* scoped = ( i < bases.mixinRefs.size() && bases.mixinRefs[ i ] != 0 ) ? bases.resolvedBase( i ) : nullptr; + if( scoped != nullptr && !scoped->empty() && r.fromSymbol != kNoNode && BuiltinMethodGate::isClassLike( ing.symbols[ r.fromSymbol ] ) ) + { + addEdge( ing.symbols[ r.fromSymbol ].name, rubyMixinKey( *scoped, r.calleeName ) ); // an in-tree mixin of a class or module + } + } for( const Symbol& s : ing.symbols ) { - if( BuiltinMethodGate::isClassLike( s ) ) + if( rubyIsConcernClassMethods( s ) ) { - note( s.fileId, s.name, /*definition=*/true ); // defined here: every call in the file may be on an instance of it + addEdge( s.scope, s.scope + "::" + s.name ); } } - for( const Reference& r : ing.references ) + for( auto* lists : { &out.up, &out.down } ) { - if( !r.isDocLink ) // a backtick mention in prose is not code evidence + for( auto& [ k, v ] : *lists ) { - note( r.fileId, r.calleeName ); - note( r.fileId, r.recvVar ); - note( r.fileId, r.qualifier ); + std::sort( v.begin(), v.end() ); + v.erase( std::unique( v.begin(), v.end() ), v.end() ); } } - for( const Binding& b : ing.bindings ) + return out; +} + +// RUBY METHOD LOOKUP DECIDES WHAT A BARE CALL REACHES (test/rubyreachcheck.sh; parser version 131 makes mixins ancestors). +// +// A receiver-less Ruby call, or `self.m`, inside a class or module sends `m` to self, and Ruby answers it from self's +// ancestors: the class, its mixins, its superclass and theirs. Self may be any instance BELOW the class that holds the +// call — a subclass (the template-method idiom) or an includer (a concern calling its host's method) — so the REACH of a +// call whose self is S is the ancestors of S and of every class below S, over the inheritance name graph with mixins +// and a concern's ClassMethods (buildRubyCha). A candidate owned by a Ruby class or module outside that reach is a method no lookup finds: +// `render` in a controller is ActionController's, not a component's; `request` is not the request of a class the +// controller merely names; `outer_helper` from inside Outer::Inner is a NoMethodError (lexical nesting is constant +// lookup, not method lookup). Such candidates leave the call's set, and a call left with none is vetoed as external: +// the method Ruby runs is outside the indexed tree. +// Left alone: a method of a reopened Object, Kernel, BasicObject, Module or Class (kRubyImplicitRoots, every self's +// ancestors); a top-level def (a private method of Object) while no ancestor in reach is out of the tree; a candidate +// whose owner is no Ruby class or module the tree defines; a call outside any class; and every call of a self whose reach holds a class that +// answers ANY name — one defining method_missing, one writing the delegation DSL (kRubyDelegationCalls: it defines +// methods the tree does not index), or one below an out-of-tree delegator (kRubyDelegatorBaseNames). +// Reach is read by NAME, as the inheritance graph is keyed: two classes sharing a name share their reach, which refuses +// less, never more. Memoised per self; the ancestor sets it unions are memoised per class, so a wide hierarchy pays once. +struct RubySelfReach +{ + const HashMap>& chaUp; + const HashMap>& chaDown; + HashMap classNames; // every Ruby class and module name the tree defines + HashMap answersAny; // classes that answer any name sent to them + HashMap outOfTree; // classes with a superclass or mixin the tree never opens + HashMap qualifiedOwner; // a def inside a nested `module ClassMethods` → + // "::ClassMethods" (buildRubyCha's key) + + struct Reach + { + HashMap names; + bool open = false; // a class in reach answers any name: the rule refuses nothing + bool external = false; // a class in reach has an out-of-tree ancestor, which may answer a + // name before Object does: a top-level def is not reachable then + }; + mutable HashMap memo; + mutable HashMap> ancestorsMemo; + + // The key a call's SELF is reached by: a method of a nested `module ClassMethods` runs on the classes its holder is + // included into, so its self is "::ClassMethods" (buildRubyCha's key), not the bare name every concern shares. + const std::string& selfOf( const IngestResult& ing, const Reference& r ) const { - noteBindingClasses( gate, fileGated, b ); + const auto q = qualifiedOwner.find( r.fromSymbol ); + return q != qualifiedOwner.end() ? q->second : callerSelfScope( ing, r ); } - for( const auto& [ importKey, bound ] : jsImports.targets ) // key "#" (jsImportKey) + + // The calls the rule judges: a Ruby call to self — receiver-less or `self.` — whose self is a class or module. + static bool judges( const Reference& r, const std::string& self ) noexcept { - if( bound.outcome != JsImportOutcome::Pinned || bound.node >= ing.symbols.size() || !BuiltinMethodGate::isClassLike( ing.symbols[ bound.node ] ) ) + return r.lang == Lang::Ruby && r.role == RefRole::Call && r.qualifier.empty() && !self.empty() + && ( r.recv == RecvKind::None || r.recv == RecvKind::ThisObj ); + } + + // `start` and every name above it; memoised. + const std::vector& ancestorsOf( const std::string& start ) const + { + return memoNamesReached( chaUp, ancestorsMemo, start ); + } + + // The ancestors of self and of every class below it; memoised. + const Reach& reachOf( const std::string& self ) const + { + if( const auto hit = memo.find( self ); hit != memo.end() ) { - continue; + return hit->second; } - std::uint32_t fileId = 0; - const auto [ end, ec ] = std::from_chars( importKey.data(), importKey.data() + importKey.size(), fileId ); - if( ec == std::errc() && end != importKey.data() ) + Reach reach; + forEachNameReached( chaDown, self, [ & ]( const std::string& below ) { - note( fileId, ing.symbols[ bound.node ].name ); // order-free: every list is sorted below + for( const std::string& a : ancestorsOf( below ) ) + { + reach.names.try_emplace( a, 1 ); + reach.open = reach.open || answersAny.find( a ) != answersAny.end(); + reach.external = reach.external || outOfTree.find( a ) != outOfTree.end(); + } + } ); + for( const std::string_view root : kRubyImplicitRoots ) + { + reach.names.try_emplace( std::string( root ), 1 ); } + return memo.emplace( self, std::move( reach ) ).first->second; } - for( std::vector& named : gate.fileClasses ) + + // The candidates of `ids` self's lookup can reach, into `out` (cleared first); true when any was left out. A candidate + // owned by no Ruby class or module the tree defines is reachable; a top-level def (a private method of Object) only + // while every ancestor in reach is in the tree — an out-of-tree one (ActionController::Base) is found first by Ruby. + template + bool reachableOf( const IngestResult& ing, const std::string& self, const In& ids, Out& out ) const { - std::sort( named.begin(), named.end() ); - named.erase( std::unique( named.begin(), named.end() ), named.end() ); + const Reach& reach = reachOf( self ); + out.clear(); + for( const NodeId c : ids ) + { + const auto q = qualifiedOwner.find( c ); + const std::string& owner = q != qualifiedOwner.end() ? q->second : ing.symbols[ c ].scope; + const bool topLevel = owner.empty(); + const bool known = q != qualifiedOwner.end() || classNames.find( owner ) != classNames.end(); + if( reach.open || ( topLevel && !reach.external ) || ( !topLevel && ( !known || reach.names.find( owner ) != reach.names.end() ) ) ) + { + out.push_back( c ); + } + } + return out.size() < ids.size(); } - for( std::vector& named : gate.fileRefClasses ) +}; + +// The innermost of `holders` whose span contains `s`, or nullptr. +inline const Symbol* rubyInnermostHolder( const std::vector& holders, const Symbol& s ) noexcept +{ + const Symbol* inner = nullptr; + for( const Symbol* m : holders ) { - std::sort( named.begin(), named.end() ); - named.erase( std::unique( named.begin(), named.end() ), named.end() ); + const bool contains = m->id != s.id && m->sigStartByte <= s.sigStartByte && s.endByte <= m->endByte; + inner = ( contains && ( inner == nullptr || m->sigStartByte > inner->sigStartByte ) ) ? m : inner; } + return inner; } -inline BuiltinMethodGate buildBuiltinMethodGate( const IngestResult& ing, const ExternalVeto& veto, const ExternalVetoTables& vetoTables, - const std::vector>& directIncludes, const JsImportTables& jsImports ) +// Each def inside a nested `module ClassMethods`, keyed to "::ClassMethods" — the module whose span contains it +// in its file (innermost first), the holder being that module's own scope. +inline void rubyClassMethodsOwners( const IngestResult& ing, HashMap& out ) { - PROFILE_SCOPE_DESCRIBE( "buildGraph/2j: builtin-method name gate" ); - BuiltinMethodGate gate{ ing, veto, vetoTables, directIncludes, jsImports, {}, {}, {}, {}, {}, {}, {}, {}, {}, false }; - std::vector fileGated( ing.files.size(), 0 ); + HashMap> holders; // fileId → its ClassMethods modules for( const Symbol& s : ing.symbols ) { - if( s.fileId < fileGated.size() && !BuiltinMethodGate::tableFor( s.lang ).empty() ) + if( rubyIsConcernClassMethods( s ) ) { - fileGated[ s.fileId ] = 1; - gate.active = true; + holders[ s.fileId ].push_back( &s ); } } - if( !gate.active ) - { - return gate; - } for( const Symbol& s : ing.symbols ) { - if( BuiltinMethodGate::isClassLike( s ) && gate.classId.try_emplace( s.name, std::uint32_t( gate.className.size() ) ).second ) + const auto file = s.scope == kRubyConcernClassMethods ? holders.find( s.fileId ) : holders.end(); + const Symbol* inner = file == holders.end() ? nullptr : rubyInnermostHolder( file->second, s ); + if( inner != nullptr ) { - gate.className.push_back( s.name ); + out.try_emplace( s.id, inner->scope + "::" + inner->name ); } } - // Containers only in gated files: a gated call's targets are language-compatible with it, so they live there too. - gate.containersByFile = symbolsByFileInIdOrder( ing, [ & ]( const Symbol& s ) - { return fileGated[ s.fileId ] != 0 && ( BuiltinMethodGate::isClassLike( s ) || s.kind == SymKind::Function || s.kind == SymKind::Method ); } ); - collectFileClassEvidence( ing, fileGated, jsImports, gate ); - return gate; } -// Counts one call reference's disposition when its resolve-loop iteration ENDS — on every exit, each `continue` -// included — so no exit needs a counter of its own, only the assignment naming what happened. A reference that -// leaves naming nothing is counted Unaccounted, and the conservation line cannot balance silently. -struct DispositionTally +// What one reference inside class `owner` says of it: a delegation-DSL call or an out-of-tree delegator superclass — it +// answers any name; an inherit reference whose base the tree never opens (`scoped` empty) — it has an out-of-tree ancestor. +inline void rubyNoteClassEvidence( const Reference& r, const std::string* scoped, bool mixin, const std::string& owner, RubySelfReach& reach ) { - const CallDisposition& disposition; - CallDispositionCounts& counts; - ~DispositionTally() + const bool outOfTree = r.isInherit && scoped != nullptr && scoped->empty(); + const bool delegates = r.lang == Lang::Ruby && r.role == RefRole::Call && r.recv == RecvKind::None + && std::ranges::find( kRubyDelegationCalls, r.calleeName ) != std::end( kRubyDelegationCalls ); + const bool delegator = outOfTree && !mixin && std::ranges::find( kRubyDelegatorBaseNames, r.calleeName ) != std::end( kRubyDelegatorBaseNames ); + if( delegates || delegator ) { - ++counts[ std::size_t( disposition ) ]; + reach.answersAny.try_emplace( owner, 1 ); } -}; - -// `census` arms the eval-only S6-C silent-pin census (src/pincensus.h): every DECIDED call site records -// which narrowing stage committed it and to which canonical target. It adds rows to g.pinCensus and -// changes NOTHING else — no candidate is admitted, dropped or reordered by it, so the emitted map is -// byte-identical armed or not (test/pincensuscheck.sh arm (E) is the executable form of that sentence). + if( outOfTree ) + { + reach.outOfTree.try_emplace( owner, 1 ); + } +} -inline void internDeclinedList( Graph& g, HashMap>& listsByHash, std::span cand ); // defined beside declinedCallsNaming, its one reader -inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = nullptr, bool census = false ) +// RubySelfReach over one graph: every Ruby class and module name the tree defines; the classes that answer ANY name sent +// to them — one defining method_missing, one writing the delegation DSL, or one whose superclass is an out-of-tree +// delegator; and the classes with an out-of-tree superclass or mixin. +inline RubySelfReach buildRubySelfReach( const IngestResult& ing, const HashMap>& chaUp, + const HashMap>& chaDown, const RubyBaseScope& bases ) { - PROFILE_SCOPE_DESCRIBE( "buildGraph: resolve refs + build CSR" ); - const std::size_t N = ing.symbols.size(); - Graph g; - g.wOutDeg.assign( N, 0.0 ); - g.ambOut.assign( N, 0u ); // counted in the resolve loop: calls that stay split across >1 def after narrowing - g.locPinOut.assign( N, 0u ); // counted in the resolve loop: calls the S6-C locality tie-break alone pinned to one def - g.unresolvedOut.assign( N, 0u ); // counted in the resolve loop: calls whose in-repo defs were all lang-filtered - g.declinedOut.assign( N, 0u ); // counted in the resolve loop: calls tier 3 declined to guess at (no edge) - if( scip ) { g.scipDocsSeen = scip->documentsSeen; g.scipEdgesPinned = scip->edgesPinned; } - // #66: carry the crawl's own unindexed-extension roll-up onto the graph, so the verbs that answer off - // this CSR can disclose the same gap the map header already prints. Summed HERE, from the identical - // vector the header reads (ing.crawlSkips.unindexedExts), because a second traversal elsewhere is - // exactly the two-derivations-of-one-number defect M15 fixed for the gauge pair. - for( const UnindexedExt& ue : ing.crawlSkips.unindexedExts ) { g.unindexedFiles += ue.files; } - - // S6-C canonical ids: `path::scope::name` per symbol (bare name when no scope). Computed once here so the - // resolution locality tie-break (below) and serialize's `id=` attribute share one definition. Deterministic. - g.canonId.resize( N ); - g.localityKey.resize( N ); + RubySelfReach reach{ .chaUp = chaUp, .chaDown = chaDown }; + rubyClassMethodsOwners( ing, reach.qualifiedOwner ); + for( const Symbol& s : ing.symbols ) { - PROFILE_SCOPE_DESCRIBE( "buildGraph/1a: canonId + localityKey (per symbol)" ); - for( const Symbol& s : ing.symbols ) + if( s.lang == Lang::Ruby && BuiltinMethodGate::isClassLike( s ) ) { - g.canonId[ s.id ] = canonicalId( ing.files[ s.fileId ], s.scope, s.name ); - g.localityKey[ s.id ] = localityKeyOf( ing.files[ s.fileId ], s.scope, s.name ); // == canonId when scoped + reach.classNames.try_emplace( s.name, 1 ); + } + else if( s.lang == Lang::Ruby && s.name == "method_missing" && !s.scope.empty() ) + { + reach.answersAny.try_emplace( s.scope, 1 ); } } - - // A4-R5 JNI: decode every `Java_pkg_Cls_method` C/C++ def to its readable dotted Java name and stash it as - // the symbol's binding label. No ingest capture / cache change — it is a pure function of the def NAME. The - // vector stays EMPTY (no allocation) when the tree holds no JNI export, so a JNI-free corpus is unaffected. + for( std::size_t i = 0; i < ing.references.size(); ++i ) { - PROFILE_SCOPE_DESCRIBE( "buildGraph/1b: JNI name decode (per symbol)" ); - for( const Symbol& s : ing.symbols ) + const Reference& r = ing.references[ i ]; + if( r.fromSymbol != kNoNode && BuiltinMethodGate::isClassLike( ing.symbols[ r.fromSymbol ] ) ) { - if( ( s.lang != Lang::Cpp && s.lang != Lang::ObjC ) || s.name.size() <= 5 || s.name.compare( 0, 5, "Java_" ) != 0 ) - { - continue; - } - std::string readable = decodeJniName( s.name ); - if( readable.empty() ) - { - continue; - } - if( g.bindLabel.empty() ) - { - g.bindLabel.assign( N, std::string() ); - } - g.bindLabel[ s.id ] = std::move( readable ); + rubyNoteClassEvidence( r, bases.resolvedBase( i ), i < bases.mixinRefs.size() && bases.mixinRefs[ i ] != 0, ing.symbols[ r.fromSymbol ].name, reach ); } } + return reach; +} - // ── Multi-root workspace: name-based resolution NEVER crosses roots. Every - // name-tier candidate below (qualified canonical, Rule 1/2, the name-based-fallback byName fill, inheritance, doc - // mentions, HAS-A) is filtered to the REFERENCE's own root; cross-root edges enter ONLY via evidence - // (the path-resolved include set feeding Rule 3, and the FFI binding tables — both left unfiltered by - // design). fileRoot is EMPTY on a single-root run, so sameRoot is constant-true and the resolved graph - // is byte-identical to today (the G5 quarantine). - const bool multiRoot = !ing.fileRoot.empty(); - const auto sameRoot = [ & ]( NodeId candSym, std::uint32_t refFileId ) noexcept -> bool +// RUBY TOP-LEVEL SELF of a Rake task file and a Jbuilder template (test/rubyrakejbuildercheck.sh; parser version 135). +// Outside any class Ruby's self is what runs the file. Rake loads a `.rake` file at top level and calls a task's block as +// written, so self there is `main`: a receiver-less call reaches a top-level def (a private method of Object) or a +// reopened Object, Kernel or BasicObject. ActionView compiles a template into a method of the view, so self in a +// `.jbuilder` file is the view, which also mixes in each helper module the application defines (Rails' `helper :all`: +// the modules of the `*_helper.rb` files under app/helpers/) and answers each controller method a `helper_method` +// declaration names (LocalBindKind::RubyHelperMethod). A candidate owned by any other Ruby class or module the tree +// defines is a method no lookup finds — the call names a partial's local, a route helper, ActionView's own — and so is a +// def in test code (filter.h isTestSymbol: a def inside an example group's block has no class around it, and is the +// group's method). A call left with none is refused as external. A `.rb` file's top level is left to RubySelfReach, which leaves it alone: a block +// there may run with another self (`describe`, `routes.draw`, `FactoryBot.define`). +struct RubyTopSelf +{ + enum class Kind : std::uint8_t { None, Main, View }; + const IngestResult& ing; + const RubySelfReach& reach; + std::vector byFile; // per file: what self is outside any class + HashMap viewMethods; // the defs a view answers besides main's: helper modules', helper_methods' + + // the self a Ruby call outside any class runs against in r's file, as a key no class shares; nullptr when not known + const std::string* selfOf( const Reference& r ) const { - return !multiRoot || ing.fileRoot[ ing.symbols[ candSym ].fileId ] == ing.fileRoot[ refFileId ]; - }; + static const std::string kMain = "
", kView = ""; + const Kind kind = r.fileId < byFile.size() ? byFile[ r.fileId ] : Kind::None; + const bool call = r.lang == Lang::Ruby && r.role == RefRole::Call && r.qualifier.empty() && r.fromSymbol != kNoNode + && ( r.recv == RecvKind::None || r.recv == RecvKind::ThisObj ) && callerSelfScope( ing, r ).empty(); + return !call || kind == Kind::None ? nullptr : kind == Kind::Main ? &kMain : &kView; + } + static bool owns( const std::string& self ) noexcept { return self == "
" || self == ""; } - // file → directory id (path up to the last '/') - HashMap dirIds; - std::vector fileDir( ing.files.size(), 0 ); + // The candidates of `ids` self reaches, into `out` (cleared first); true when any was left out. A class's self is + // RubySelfReach's to read; for "
" and "", a candidate owned by no Ruby class or module the tree defines is + // reachable, as RubySelfReach reads it. + template + bool reachableOf( const std::string& self, const In& ids, Out& out ) const { - PROFILE_SCOPE_DESCRIBE( "buildGraph/1c: fileDir (dir interning)" ); - for( std::size_t f = 0; f < ing.files.size(); ++f ) + if( !owns( self ) ) { - std::string_view p = ing.files[f]; - const std::size_t sl = p.rfind( '/' ); - std::string dir = ( sl == std::string_view::npos ) ? std::string() : std::string( p.substr( 0, sl ) ); - fileDir[f] = dirIds.emplace( std::move( dir ), std::uint32_t( dirIds.size() ) ).first->second; + return reach.reachableOf( ing, self, ids, out ); + } + out.clear(); + for( const NodeId c : ids ) + { + const std::string& owner = ing.symbols[ c ].scope; + const bool root = owner.empty() || std::ranges::find( kRubyInstanceRoots, owner ) != std::end( kRubyInstanceRoots ); + const bool known = reach.qualifiedOwner.find( c ) != reach.qualifiedOwner.end() || reach.classNames.find( owner ) != reach.classNames.end(); + const bool view = self == "" && viewMethods.find( c ) != viewMethods.end(); + if( ( root || !known || view ) && !isTestSymbol( ing, c ) ) + { + out.push_back( c ); + } } + return out.size() < ids.size(); } +}; - // name → candidate definition ids. rw::svector<,2>: most names define 1-2 symbols, so the id-list is - // inline (no per-name malloc) and size() is branch-free — the measured-best value type for this - // write-once (here) / read-hot (resolve below) shape (see bench/bench_svector3.cpp). Iterates in - // insertion order exactly like std::vector, so the resolved graph — and the output — is unchanged. - HashMap> byName; - byName.reserve( N ); // ≤ one entry per symbol → skip the rehash cascade +// Is `path` a helper file Rails' `helper :all` loads — a `*_helper.rb` under an app/helpers/ directory? +inline bool rubyIsHelperFile( std::string_view path ) noexcept +{ + const std::size_t dir = path.rfind( "app/helpers/" ); + return dir != std::string_view::npos && ( dir == 0 || path[ dir - 1 ] == '/' ) && path.ends_with( "_helper.rb" ); +} + +// What self is outside any class in the file at `path`: Rake's main in a task file, the view in a Jbuilder template. +inline RubyTopSelf::Kind rubyTopSelfKindOf( std::string_view path ) noexcept +{ + return path.ends_with( ".rake" ) ? RubyTopSelf::Kind::Main : path.ends_with( ".jbuilder" ) ? RubyTopSelf::Kind::View : RubyTopSelf::Kind::None; +} + +// RubyTopSelf over one graph: each file's top-level self by extension, and the defs a view answers — every method of a +// helper file's modules, and each def a `helper_method` declaration names in its own class or module. +inline RubyTopSelf buildRubyTopSelf( const IngestResult& ing, const RubySelfReach& reach ) +{ + RubyTopSelf top{ .ing = ing, .reach = reach }; + top.byFile.reserve( ing.files.size() ); + for( const std::string& path : ing.files ) { - PROFILE_SCOPE_DESCRIBE( "buildGraph/1d: byName (name -> def ids)" ); - for( const Symbol& s : ing.symbols ) + top.byFile.push_back( rubyTopSelfKindOf( path ) ); + } + HashMap declared; // "#" + for( const Binding& b : ing.bindings ) + { + if( b.kind == LocalBindKind::RubyHelperMethod ) { - byName[ s.name ].push_back( s.id ); + declared.try_emplace( b.typeName + "#" + b.var, 1 ); } } - - // decl/def collapse (adversarial-review #1) — collapseDeclarationsOfName carries the rule, its per-root and per-family - // keys, and why a Kotlin body never evicts a Java declaration (or the reverse). + for( const Symbol& s : ing.symbols ) { - PROFILE_SCOPE_DESCRIBE( "buildGraph/1e: decl/def collapse" ); - for( auto& [ name, ids ] : byName ) + const bool method = s.lang == Lang::Ruby && ( s.kind == SymKind::Function || s.kind == SymKind::Method ) && !s.scope.empty(); + if( method && ( rubyIsHelperFile( ing.files[ s.fileId ] ) || ( !declared.empty() && declared.find( s.scope + "#" + s.name ) != declared.end() ) ) ) { - collapseDeclarationsOfName( ing, multiRoot, ids ); + top.viewMethods.try_emplace( s.id, 1 ); } } + return top; +} - // canonical scope::name → definition ids (E#4): lets a qualified call `A::b()` resolve to the `b` whose - // enclosing scope is `A`, BEFORE the bare-name spray — the deterministic [AST] cut to call-graph +// RUBY TYPED RECEIVERS (test/rubytypedrecvcheck.sh; parser version 132 types them — model.h rubyTypedRecvOf). +// +// A call whose receiver the code builds — `c = Client.new`, `User.find_by( … ).activate!`, a let whose block runs +// `create( :user )` — sends to an instance of one class, and Ruby's lookup on that class answers: the class, its mixins, +// its superclass chain (Narrower::methodOnTypeOrBases over the Ruby overlay, the shallowest level that defines the name +// deciding, a tie at one level kept as a split), then a reopened Object, Kernel or BasicObject. When none of those defines +// it a class BELOW may (Active Record's single-table inheritance hands back a subclass), so the call keeps the candidates +// RubySelfReach says the class's instances reach, and is refused as external when none is left: a column, an +// association, a framework method — the method Ruby runs is outside the tree. +// The type is used only when it means what it says: a factory resolves through the tree's factory definitions, one class +// per name; the class is one the tree defines (an out-of-tree `Net::HTTP` or a `Point = Struct.new( … )` constant says +// nothing about in-tree candidates) — and a constant written QUALIFIED names one only when the tree opens that path +// (`OpenSSL::Cipher` is not activerecord's `Encryption::Cipher`, `Stripe::Customer` not an app's Customer model, though +// the classes are keyed by final segment everywhere else); and no class in its lookup defines the method that built it +// (`def self.find` may return anything). Otherwise the call is left exactly as the name ladder decided it before. +// One family of out-of-tree classes is known exactly: RSpec's targets (model.h kRspecTargets — what `expect( v )`, +// `allow( v )`, `is_expected` return in an example group; test/rubyrspectargetcheck.sh). When the tree never opens the +// class, only a reopened instance root can answer a call on one, and otherwise RSpec's method runs: the call is refused as +// external. A tree that opens the class answers from it as above; a tree that defines a method of the builder's name +// (`def expect`) where an example group can reach it types nothing from that name (rubyDefinedBuilders). RSpec's matchers +// are such targets too: a call on a chain rooted at `receive( :m )` or `change { }` is RSpec's. A Jbuilder template's `json` is another (model.h kJbuilderTemplate): every +// call on it is a key the class's method_missing writes (test/rubyrakejbuildercheck.sh). +// +// RUBY CLASS OBJECTS (test/rubyclassrecvcheck.sh; parser version 137). A call whose receiver is a constant — +// `Report.generate`, `User.where( … )`, `JSON.parse( s )` — is sent to the class object, and Ruby's lookup on it answers +// (RubyClassObjects): +// * WHICH class: the constant as Ruby resolves it from the call site, by fully-qualified constant over resolve.h's index — +// lexically (the innermost class or module open around the site, its enclosing ones, the top level), then through the +// ancestors of that open or of a qualified constant's head, then by Zeitwerk's directories (a constant whose last two +// segments or more the tree opens). The ingest keeps a path-written receiver's path (Reference::fieldName), and the +// described class's for `described_class`. A constant that resolves to nothing the tree opens — `JSON`, `RSpec`, a +// `Point = Struct.new( … )` constant, a gem's class — answers only from a reopened root. A tree with no constant index +// reads the final segment, as every class is keyed elsewhere. +// * WHAT answers: the shallowest of the class, its mixins and its superclasses — read by fully-qualified constant along +// what the index scopes their superclass and mixin references to (fqnSides) — to define the name as a class method, +// each on the side of the lookup it joins (parser version 145, RubySide): the class's and its superclasses' singleton +// methods (`def self.m`, a def or an accessor in `class << self`: LocalBindKind::RubySingletonDef); what a module the +// class includes defines on its includer — a singleton def in a concern's `included do`, at the includer's own +// level, then a def in its `class_methods do`, its nested ClassMethods, a module its `included do` extends — and the +// instance methods of a module the class extends (LocalBindKind::RubyClassMixin: an `extend`, an `include` in +// `class << self`, a module's `extend self`). Ruby's order: the class's own singleton, then what it extends, level by +// level, and only then its superclass's (one round per class up the chain). A class's instance method is its +// instances' alone, and so is an included module's; a module's own singleton method, or one it extends, is the +// module's. A call on a module answers from its own methods too (floor (c): `module_function` is not read). Then a +// reopened Module, Class, Object, Kernel or BasicObject. `C.new` is Class#new +// unless a `def self.new` answers first, and Class#new runs the first `initialize` an instance's lookup finds — never +// an instance method named `new`. A class below a base that forwards a class object's call to a new instance +// (kRubyInstanceForwardingBases: a mailer's, ActiveSupport::CurrentAttributes) answers a name it lacks from its +// instances' lookup. +// * When nothing there defines the name, Ruby's answer is outside the tree (ActiveRecord::Base's `where`, +// StandardError's initialize), and the call is refused as external rather than bound to an unrelated namesake. +// Left as before: a SCREAMING constant the tree assigns (`LIMITS = Limits.new`, a value of a class nothing types), a name +// a `class << self` delegates, and a name nothing in the lookup defines when a method_missing there may answer it — Ruby +// calls a method_missing only when its lookup misses, so a def the lookup finds answers first (ActiveRecord::Base, which +// extends DynamicMatchers, still answers `configurations` from Core's `included do`). +// +// A CALL TO SELF AND A RECEIVER THE CODE BUILDS stand on one side too (parser version 145, test/rubyclassrecvcheck.sh floor +// (i); RubyClassObjects::selfCall, typedLookup): self in a singleton method is the class object, answered by `lookup` +// above; self in a class's other defs, and a receiver built by `C.new` or a finder, is an instance, answered by its +// instances' lookup (sideLookup on the instance side) — so `svc.call` reaches the instance `call`, not `def self.call`, +// and a bare `new` in a class method reaches its class's initialize. Rule 1 and its base walk, which read a class's defs +// of both sides alike, step aside. What that lookup misses goes to the ladder, which keeps only the namesakes that answer +// on the side (answersOn); none left — the call declines, as self may be another's in a block. No side, as before: an +// instance method of a module (its includers', or a class object it extends), a def a concern defines for its includers, +// a class body, a module or FactoryBot build as a receiver, and a lookup holding a method_missing for what it misses. + +// Which side of a Ruby lookup a walk stands on at a class or module (RubyClassObjects::sideLookup; parser version 145). +enum class RubySide : std::uint8_t +{ + ClassObject, // the class object, or a superclass's: its singleton methods answer, and a module's own methods + // (floor (c)) + Included, // a module the class includes: what its `included do` defines on the includer answers, at the + // includer's own level + ConcernClassMethods, // the same module's `class_methods do` block, which the concern extends onto the includer: its + // defs answer a level above + Extended, // a module the class object extends: its instance methods answer + Instance, // an instance's lookup (what Class#new's initialize is, a forwarding base's name): any instance + // method +}; +inline constexpr std::size_t kRubySideCount = static_cast( RubySide::Instance ) + 1; + +// How a Ruby ancestor joins the lookup of the class or module that writes it (RubyClassObjects::fqnSides). +enum class RubyAncestry : std::uint8_t +{ + Superclass, + Include, // `include` or `prepend` — in the body, or in a concern's `included do` (into the includer) + Extend, // `extend` in the body, an `include` in `class << self`, a module's `extend self`: the writer's own + // class object + IncluderExtend, // `extend` in a concern's `included do`: the includer's class object + ClassMethods, // a concern's nested ClassMethods, which ActiveSupport::Concern extends onto the includer +}; +inline constexpr std::size_t kRubyAncestryCount = static_cast( RubyAncestry::ClassMethods ) + 1; + +// One step of a lookup walk (RubyClassObjects::sideLookup): a constant, and the side the walk stands on there. +using RubySideStep = std::pair; + +struct RubyAncestorEdge +{ + std::string fqn; + RubyAncestry how = RubyAncestry::Superclass; +}; + +// The side a walk standing on `from` reaches an ancestor on across `how`, or nullopt when that ancestor is in no lookup from +// there: a module's own body-level `extend` is the module's, a concern's ClassMethods its includers', and an instance's +// lookup holds no extended module. (A concern's `class_methods do` block is no ancestor: sideSteps steps to it.) Rows +// follow RubySide's order, columns RubyAncestry's. +inline constexpr std::optional kRubySideAcross[ kRubySideCount ][ kRubyAncestryCount ] = { + // Superclass Include Extend IncluderExtend ClassMethods + /* ClassObject */ { RubySide::ClassObject, RubySide::Included, RubySide::Extended, std::nullopt, std::nullopt }, + /* Included */ { std::nullopt, RubySide::Included, std::nullopt, RubySide::Extended, RubySide::Extended }, + /* ConcernClassM. */ { std::nullopt, std::nullopt, std::nullopt, std::nullopt, std::nullopt }, + /* Extended */ { std::nullopt, RubySide::Extended, std::nullopt, std::nullopt, std::nullopt }, + /* Instance */ { RubySide::Instance, RubySide::Instance, std::nullopt, std::nullopt, std::nullopt }, +}; + +inline std::optional rubySideAcross( RubySide from, RubyAncestry how ) noexcept +{ + // The table's one reader proves both extents (in a body, not at namespace scope, where a call is a symbol). + static_assert( enumCountIsExact(), "kRubySideCount must name the LAST RubySide — move it with the append" ); + static_assert( enumCountIsExact(), "kRubyAncestryCount must name the LAST RubyAncestry — move it with the append" ); + return kRubySideAcross[ unsigned( from ) ][ unsigned( how ) ]; +} + +// What a Ruby def is to a class object, as bits (RubyClassObjects::defSides). +inline constexpr char kRubyDefSingleton = 1; // a singleton method of the class or module that holds it +inline constexpr char kRubyDefIncluder = 2; // a class method each class including its module defines (model.h kRubyIncluderMark) +inline constexpr char kRubyDefModule = 4; // a method of a module, which a call on the module answers (floor (c)) +inline constexpr char kRubyDefBlock = 8; // a def in a concern's `class_methods do` (model.h kRubyClassMethodsMark) + +// Does a def whose side bits are `bits` answer a walk standing on `side`? +inline bool rubyAnswersOn( RubySide side, char bits ) noexcept +{ + const bool singleton = ( bits & kRubyDefSingleton ) != 0; + const bool includers = ( bits & ( kRubyDefIncluder | kRubyDefBlock ) ) != 0; + if( side == RubySide::ClassObject ) + { + return !includers && ( singleton || ( bits & kRubyDefModule ) != 0 ); + } + if( side == RubySide::Included ) + { + return ( bits & kRubyDefIncluder ) != 0; + } + if( side == RubySide::ConcernClassMethods ) + { + return ( bits & kRubyDefBlock ) != 0; + } + return !singleton && !includers; // Extended and Instance: an instance method of what the walk stands on +} + +// What a Ruby call answers from on the side its receiver stands on (RubyClassObjects::selfCall, typedLookup). +struct RubySideAnswer +{ + std::optional side; // the side, when the lookup could be walked: the receiver's class resolves + // to a constant the tree opens + const rw::SmallVec* hit = nullptr; // what answers in the tree then; nullptr when nothing does + bool open = false; // a method_missing in the lookup (or, on a class object, a `class << self` + // delegation of the name) may answer what it misses +}; + +// How RubySelfReach's cut, read on a side (RubyClassObjects::reachableOnSide), left a call's candidates. +enum class RubyCut : std::uint8_t +{ + Unchanged, // every candidate stands + Cut, // some fell: `out` holds the rest (empty: none is in reach — the method Ruby runs is outside the tree) + Declined, // some are in reach, but none answers on the side self stands on: `out` holds them. Self may not be what the + // def says (a block run with another self, `instance_exec`), or the method be generated: the call declines +}; + +// What a call on a receiver whose every answer the tree knows reaches (rubyClosedLookup). +struct RubyClosedLookup +{ + bool closed = false; // the lookup is known: what `hit` lacks is outside the tree + const rw::SmallVec* hit = nullptr; // what answers in the tree; nullptr when nothing does +}; + +// The class object a Ruby constant receiver names (RubyClassObjects::of). +struct RubyClassObject +{ + const std::string* cls = nullptr; // the class or module the tree opens at the constant, as RubySelfReach keys it + std::string fqn; // its fully-qualified constant, when the tree keeps a constant index + bool external = false; // a constant the tree never opens: only a reopened root answers +}; + +inline constexpr std::string_view kRubyNew = "new"; // Class#new: it allocates, then runs initialize +inline constexpr std::string_view kRubyInitialize = "initialize"; + +// The out-of-tree superclasses whose class object forwards a name it lacks to a new instance, read as written: a mailer's +// `UserMailer.welcome( u )` runs the instance method `welcome` (ActionMailer::Base's method_missing, which Devise::Mailer +// inherits), and `Current.user = u` the instance's `user=` (ActiveSupport::CurrentAttributes). A class below one answers a +// call on the class object from its instances' lookup too. +inline constexpr std::string_view kRubyInstanceForwardingBases[] = { "ActionMailer::Base", "ActiveSupport::CurrentAttributes", "Devise::Mailer" }; + +struct RubyTypedReceivers; + +// Ruby's lookup on a class object (the RUBY CLASS OBJECTS note above): what a constant receiver names, and what answers a +// call on it. Read by rubyClosedLookup. +struct RubyClassObjects +{ + const RubyTypedReceivers& typed; // the reach, overlay, narrower and paths it shares + const IngestResult& ing; + const RubyBaseScope& bases; // the constant index a receiver's FQN is read off + HashMap constantValues; // every SCREAMING constant the tree assigns + HashMap missingOwners; // every class or module defining a method_missing + std::vector defSides; // per symbol: the kRubyDef* bits of a def a + // class object may answer from, else 0 + HashMap forwarders; // every class written below a forwarding base + HashMap classDelegated; // "#" a `class << self` delegates + HashMap> fqnUp; // a constant → what its superclass and mixins + // resolve to, and a concern's ClassMethods + HashMap> fqnSides; // the same, each with how it joins the lookup + Reference initializeCall; // a Ruby call of `initialize`: what C.new runs + mutable HashMap> fqnAncestorsMemo; + mutable rw::SmallVec hits, rootHits, sideHits; + mutable std::vector walkLevel, walkNext; + mutable HashMap walkSeen; + mutable std::vector sideLevel, sideNext, sideRound; + mutable HashMap sideSeen; // a constant → the sides the walk met it on, a bit each + + RubyClassObject of( const Reference& r ) const; // the class a constant names + RubyClassObject ofIndexed( const Reference& r ) const; + const rw::SmallVec* lookup( const Reference& r, const RubyClassObject& object ) const; // what answers a call on it + const rw::SmallVec* definerLookup( const RubyClassObject& object, const Reference& asked, bool classObject ) const; + const rw::SmallVec* rootLookup( const Reference& r ) const; // a reopened root's + bool delegates( const RubyClassObject& object, std::string_view callee ) const; // a class << self delegates it + RubySideAnswer selfCall( const Reference& r ) const; // a call to self in a def + RubySideAnswer typedLookup( const Reference& r, const std::string& cls ) const; // a receiver the code builds + bool isClass( const std::string& fqn ) const; // a class, not a module + bool answersOn( NodeId c, RubySide side ) const; // may c answer that side + const rw::SmallVec* onSide( const rw::SmallVec* ids, RubySide side ) const; // those of ids that may + template + RubyCut reachableOnSide( const RubyTopSelf& top, const std::string& self, std::optional side, const In& ids, Out& out ) const; + std::string fqnOfWritten( const Reference& r, std::string_view written ) const; + bool missingAnswers( const RubyClassObject& object ) const; // a method_missing may answer a miss + template + const rw::SmallVec* levelLookup( const HashMap>& edges, const std::string& start, Probe&& probe ) const; + void expandLevel( const HashMap>& edges, const std::string& name ) const; + void noteNamedDefs( const std::string& owner, std::string_view callee ) const; + const rw::SmallVec* sideLookup( const std::string& start, RubySide side, std::string_view callee ) const; + void sideSteps( const std::string& name, RubySide at ) const; + void sideVisit( const std::string& name, RubySide on, std::vector& into ) const; + void noteSideDefs( const std::string& fqn, std::string_view callee, RubySide side ) const; + bool ownedBy( NodeId c, const std::string& fqn ) const; + bool anyInLookup( const std::string& cls, const HashMap& owners ) const; + template + bool anyAncestor( const std::string& cls, Pred&& pred ) const; + bool pathOpened( std::string_view written ) const; + std::string lexicalConstant( const std::vector& opens, std::uint32_t site, std::string_view written ) const; + std::string inheritedConstant( const std::vector& opens, std::uint32_t site, std::string_view written ) const; + std::string ancestorConstant( std::string_view head, std::string_view rest ) const; + std::string zeitwerkConstant( std::string_view written ) const; + const std::vector& fqnAncestors( const std::string& fqn ) const; + bool opened( const std::string& fqn ) const; +}; + +struct RubyTypedReceivers +{ + const RubySelfReach& reach; + const HashMap>& up; + const Narrower& narrower; + HashMap factoryClass; // factory name → its class's final segment; "" when two disagree + HashMap openedPaths; // every trailing path of every constant the tree opens: + // `A::B::C` → "A::B::C", "B::C", "C" + std::array builderDefined {}; // the tree defines a method of that builder's name + mutable rw::SmallVec rootHits; + + const std::string* classOf( const Reference& r ) const; // r's receiver's class, when trusted + bool outOfTreeTarget( const Reference& r ) const; // RSpec's, of a class the tree never opens + const rw::SmallVec* lookup( const Reference& r, const std::string& cls ) const; // what lookup on `cls` reaches + const rw::SmallVec* rootLookup( const Reference& r ) const; // what a reopened instance root defines + + // does the tree define a method of the name of `t`'s builder (`def expect`)? Then nothing that name builds is RSpec's. + bool builderShadowed( const RspecTarget* t ) const noexcept + { + return t != nullptr && builderDefined[ std::size_t( t - std::begin( kRspecTargets ) ) ]; + } +}; + +// the class `typed` names, as written without a leading `::` (a factory's name for a factory) +inline std::string_view rubyTypedWritten( const RubyTypedRecv& typed ) noexcept +{ + return typed.type.substr( typed.type.starts_with( "::" ) ? 2 : 0 ); +} + +// the one of RSpec's targets `typed` is (model.h rspecTargetOf), or nullptr +inline const RspecTarget* rubyRspecTarget( const RubyTypedRecv& typed ) noexcept +{ + return typed.factory ? nullptr : rspecTargetOf( rubyTypedWritten( typed ), typed.via ); +} + +// the class a Ruby reference's receiver was built as, when the tree can trust it (the note above); nullptr otherwise +inline const std::string* RubyTypedReceivers::classOf( const Reference& r ) const +{ + const std::optional typed = rubyTypedRecvOf( r ); + if( !typed || builderShadowed( rubyRspecTarget( *typed ) ) ) + { + return nullptr; + } + const std::string_view written = rubyTypedWritten( *typed ); + const std::size_t cut = typed->factory ? std::string_view::npos : written.rfind( "::" ); + if( cut != std::string_view::npos && openedPaths.find( std::string( written ) ) == openedPaths.end() ) + { + return nullptr; // a qualified constant whose path the tree never opens: not the in-tree namesake + } + const auto factory = typed->factory ? factoryClass.find( std::string( written ) ) : factoryClass.end(); + const std::string name = !typed->factory ? std::string( written.substr( cut == std::string_view::npos ? 0 : cut + 2 ) ) + : factory != factoryClass.end() ? factory->second : std::string {}; + const auto cls = reach.classNames.find( name ); + if( name.empty() || cls == reach.classNames.end() ) + { + return nullptr; + } + for( const std::string& a : reach.ancestorsOf( cls->first ) ) + { + if( !typed->factory && narrower.definitionsIn( a, typed->via ) != nullptr ) + { + return nullptr; // the class says how it is built: `def self.find` may return anything + } + } + return &cls->first; +} + +// is r's receiver one of RSpec's targets, built by a builder the tree does not define, or a template's `json` — of a +// class the tree never opens? +inline bool RubyTypedReceivers::outOfTreeTarget( const Reference& r ) const +{ + const std::optional typed = rubyTypedRecvOf( r ); + const RspecTarget* const target = typed ? rubyRspecTarget( *typed ) : nullptr; + const bool jbuilder = typed && !typed->factory && typed->via == kJbuilderLocal && rubyTypedWritten( *typed ) == kJbuilderTemplate; + return ( jbuilder || ( target != nullptr && !builderShadowed( target ) ) ) + && openedPaths.find( std::string( rubyTypedWritten( *typed ) ) ) == openedPaths.end(); +} + +// what lookup on `cls` reaches for r's method: its own or an ancestor's definitions, else a reopened instance root's +inline const rw::SmallVec* RubyTypedReceivers::lookup( const Reference& r, const std::string& cls ) const +{ + if( const rw::SmallVec* hit = narrower.methodOnTypeOrBases( cls, r, up, /*skipSelf=*/false, /*unionOnMulti=*/true ) ) + { + return hit; + } + return rootLookup( r ); +} + +// what a reopened Object, Kernel or BasicObject defines for r's method — every instance's ancestors +inline const rw::SmallVec* RubyTypedReceivers::rootLookup( const Reference& r ) const +{ + rootHits.clear(); + for( const std::string_view root : kRubyInstanceRoots ) + { + if( const rw::SmallVec* defs = narrower.definitionsIn( root, r.calleeName ) ) + { + rootHits.insert( rootHits.end(), defs->begin(), defs->end() ); + } + } + return rootHits.empty() ? nullptr : &rootHits; +} + +// Is `name` a Ruby constant's — a class, a module, a SCREAMING value? Ruby reads a leading capital as one. +inline bool rubyIsConstantName( std::string_view name ) noexcept +{ + return !name.empty() && name.front() >= 'A' && name.front() <= 'Z'; +} + +// The constant a Ruby call's receiver names, as written and without a leading `::` (model.h Reference::fieldName keeps a +// qualified one's path); empty when the receiver is no constant — a variable, self, a receiver the code builds (whose +// recvVar is a typed token). +inline std::string_view rubyConstantReceiver( const Reference& r ) noexcept +{ + const bool constant = r.lang == Lang::Ruby && r.role == RefRole::Call && r.recv == RecvKind::NamedVar && r.qualifier.empty() + && rubyIsConstantName( r.recvVar ) && r.recvVar.find( kRubyTypedRecvSep ) == std::string::npos; + const std::string_view written = !constant ? std::string_view {} : r.fieldName.empty() ? std::string_view( r.recvVar ) : std::string_view( r.fieldName ); + return written.substr( written.starts_with( "::" ) ? 2 : 0 ); +} + +// The class object r's constant receiver names: the class or module the tree opens at it — by fully-qualified constant +// where the tree keeps a constant index (ofIndexed), else by final segment, a qualified constant only where the tree opens +// that path — and otherwise external, a constant the tree never opens. Neither for any other receiver, nor for a SCREAMING +// constant the tree assigns: a value whose class nothing types. +inline RubyClassObject RubyClassObjects::of( const Reference& r ) const +{ + const std::string_view written = rubyConstantReceiver( r ); + if( written.empty() ) + { + return {}; + } + const std::size_t cut = written.rfind( "::" ); + const std::string name( written.substr( cut == std::string_view::npos ? 0 : cut + 2 ) ); + if( constantValues.find( name ) != constantValues.end() ) + { + return {}; + } + if( !bases.ix.opensByFile.empty() ) + { + return ofIndexed( r ); + } + const auto cls = typed.reach.classNames.find( name ); + if( cls != typed.reach.classNames.end() && ( cut == std::string_view::npos || pathOpened( written ) ) ) + { + return { &cls->first, {}, false }; + } + return { nullptr, {}, true }; +} + +// The class object by fully-qualified constant (the receiver as written, or its path — Reference::fieldName — read from the +// call site by fqnOfWritten), keyed beside it by its final segment as RubySelfReach keys +// every class; external when the constant resolves to nothing the tree opens. +inline RubyClassObject RubyClassObjects::ofIndexed( const Reference& r ) const +{ + std::string fqn = fqnOfWritten( r, r.fieldName.empty() ? std::string_view( r.recvVar ) : std::string_view( r.fieldName ) ); + const std::size_t last = fqn.rfind( "::" ); + const auto cls = fqn.empty() ? typed.reach.classNames.end() : typed.reach.classNames.find( fqn.substr( last == std::string::npos ? 0 : last + 2 ) ); + return cls != typed.reach.classNames.end() ? RubyClassObject { &cls->first, std::move( fqn ), false } : RubyClassObject { nullptr, {}, true }; +} + +// What a reopened root defines for a call on a class object: C.new runs initialize, which an instance's roots answer +// (RubyTypedReceivers::rootLookup); any other name a reopened Object, Kernel or BasicObject answers, and a Class or a +// Module too, which a class object is besides. +inline const rw::SmallVec* RubyClassObjects::rootLookup( const Reference& r ) const +{ + const bool ctor = r.calleeName == kRubyNew; + const rw::SmallVec* instance = typed.rootLookup( ctor ? initializeCall : r ); + if( ctor ) + { + return instance; + } + rootHits.clear(); + if( instance != nullptr ) + { + rootHits.insert( rootHits.end(), instance->begin(), instance->end() ); + } + const auto alsoClassRoot = [ & ]( std::string_view root ) + { + if( const rw::SmallVec* defs = typed.narrower.definitionsIn( root, r.calleeName ) ) + { + rootHits.insert( rootHits.end(), defs->begin(), defs->end() ); + } + }; + alsoClassRoot( "Class" ); + alsoClassRoot( "Module" ); + return rootHits.empty() ? nullptr : &rootHits; +} + +// The names `name` reaches in one step along `edges` that the walk has not met, onto the next level. +inline void RubyClassObjects::expandLevel( const HashMap>& edges, const std::string& name ) const +{ + const auto next = edges.find( name ); + if( next == edges.end() ) + { + return; + } + for( const std::string& base : next->second ) // `edges` owns every name: the pointers outlive the walk + { + if( walkSeen.try_emplace( base, 1 ).second ) + { + walkNext.push_back( &base ); + } + } +} + +// The shallowest level of a lookup walked from `start` along `edges` — the class, then its mixins and superclasses, level by +// level — at which `probe`, handed one name at a time, notes a def into hits; nullptr when no level does. A level with two +// definers answers their union. +template +inline const rw::SmallVec* RubyClassObjects::levelLookup( const HashMap>& edges, const std::string& start, + Probe&& probe ) const +{ + hits.clear(); + walkSeen.clear(); + walkSeen.try_emplace( start, 1 ); + walkLevel.assign( 1, &start ); + while( !walkLevel.empty() && hits.empty() ) + { + walkNext.clear(); + for( const std::string* name : walkLevel ) + { + probe( *name ); + expandLevel( edges, *name ); + } + walkLevel.swap( walkNext ); + } + return hits.empty() ? nullptr : &hits; +} + +// The defs of `callee` that `owner`, read by name, holds where its class object answers from them (defSides), into +// hits. A nested `module ClassMethods` is keyed "::ClassMethods" in the overlay while its defs are indexed under the +// bare name every concern's shares, so its defs are read by the concern that holds them (RubySelfReach::qualifiedOwner). +inline void RubyClassObjects::noteNamedDefs( const std::string& owner, std::string_view callee ) const +{ + const std::size_t cut = owner.rfind( "::" ); + const bool nested = cut != std::string::npos && std::string_view( owner ).substr( cut + 2 ) == kRubyConcernClassMethods; + const rw::SmallVec* defs = typed.narrower.definitionsIn( nested ? kRubyConcernClassMethods : std::string_view( owner ), callee ); + if( defs == nullptr ) + { + return; + } + for( const NodeId c : *defs ) + { + const auto q = nested ? typed.reach.qualifiedOwner.find( c ) : typed.reach.qualifiedOwner.end(); + if( rubyAnswersOn( RubySide::ClassObject, defSides[ c ] ) && ( !nested || ( q != typed.reach.qualifiedOwner.end() && q->second == owner ) ) ) + { + hits.push_back( c ); + } + } +} + +// The defs of `callee` the class or module whose fully-qualified constant is `fqn` holds that answer a walk standing on +// `side` there, into hits. +inline void RubyClassObjects::noteSideDefs( const std::string& fqn, std::string_view callee, RubySide side ) const +{ + const std::size_t cut = fqn.rfind( "::" ); + const rw::SmallVec* defs = typed.narrower.definitionsIn( cut == std::string::npos ? std::string_view( fqn ) : std::string_view( fqn ).substr( cut + 2 ), callee ); + if( defs == nullptr ) + { + return; + } + for( const NodeId c : *defs ) + { + if( rubyAnswersOn( side, defSides[ c ] ) && ownedBy( c, fqn ) ) + { + hits.push_back( c ); + } + } +} + +// Onto `into` unless the walk has stood on `name` on side `on` already (sideSeen); fqnSides and the caller own every name. +inline void RubyClassObjects::sideVisit( const std::string& name, RubySide on, std::vector& into ) const +{ + const char bit = char( 1 << unsigned( on ) ); + const auto [ seen, fresh ] = sideSeen.try_emplace( name, bit ); + if( fresh || ( seen->second & bit ) == 0 ) + { + seen->second = char( seen->second | bit ); + into.emplace_back( &name, on ); + } +} + +// Where the walk goes from `name`, standing on `at`: an included module's `class_methods do` block, a level up; and each +// ancestor in fqnSides on the side it is reached on (rubySideAcross) — an included one onto this level (what its +// `included do` defines is the includer's own), a superclass onto the next round, any other onto the next level. +inline void RubyClassObjects::sideSteps( const std::string& name, RubySide at ) const +{ + if( at == RubySide::Included ) + { + sideVisit( name, RubySide::ConcernClassMethods, sideNext ); + } + const auto up = fqnSides.find( name ); + if( up == fqnSides.end() ) + { + return; + } + for( const RubyAncestorEdge& e : up->second ) + { + if( const std::optional to = rubySideAcross( at, e.how ) ) + { + sideVisit( e.fqn, *to, e.how == RubyAncestry::Superclass ? sideRound : *to == RubySide::Included ? sideLevel : sideNext ); + } + } +} + +// The first place along Ruby's lookup order, walked from `start` on `side` (sideSteps), at which a def of `callee` answers +// on the side the walk stands on, into hits; nullptr when none does. One round per class up the superclass chain; within +// it, level by level, the class, then what it extends or includes. A level with two definers answers their union. +inline const rw::SmallVec* RubyClassObjects::sideLookup( const std::string& start, RubySide side, std::string_view callee ) const +{ + hits.clear(); + sideSeen.clear(); + sideRound.clear(); + sideVisit( start, side, sideRound ); + while( !sideRound.empty() && hits.empty() ) // a round: one class of the superclass chain + { + sideLevel.swap( sideRound ); + sideRound.clear(); + while( !sideLevel.empty() && hits.empty() ) // a level: one step along its mixins + { + sideNext.clear(); + for( std::size_t i = 0; i < sideLevel.size(); ++i ) // by index: an included module joins this level as it is met + { + const auto [ name, at ] = sideLevel[ i ]; + noteSideDefs( *name, callee, at ); + sideSteps( *name, at ); + } + sideLevel.swap( sideNext ); + } + } + return hits.empty() ? nullptr : &hits; +} + +// What the lookup the class object stands at defines `asked` in: as a class method when `classObject`, else as its +// instances'. Over fully-qualified constants, side by side, when the tree keeps a constant index (sideLookup), else over +// names (the overlay), where two classes of one name share a lookup. +inline const rw::SmallVec* RubyClassObjects::definerLookup( const RubyClassObject& object, const Reference& asked, bool classObject ) const +{ + if( !object.fqn.empty() ) + { + return sideLookup( object.fqn, classObject ? RubySide::ClassObject : RubySide::Instance, asked.calleeName ); + } + if( classObject ) + { + return levelLookup( typed.up, *object.cls, [ & ]( const std::string& n ) { noteNamedDefs( n, asked.calleeName ); } ); + } + return typed.narrower.methodOnTypeOrBases( *object.cls, asked, typed.up, /*skipSelf=*/false, /*unionOnMulti=*/true ); +} + +// What Ruby's lookup on the class object reaches for r's method: the shallowest of the class, its mixins and its +// superclasses to define it as a class method, else a reopened root. C.new is Class#new unless a `def self.new` (or a +// concern's ClassMethods#new) answers first, and Class#new runs the first `initialize` an instance's lookup finds. A class +// below a forwarding base (kRubyInstanceForwardingBases) answers a name it lacks from its instances' lookup. +inline const rw::SmallVec* RubyClassObjects::lookup( const Reference& r, const RubyClassObject& object ) const +{ + if( const rw::SmallVec* hit = definerLookup( object, r, true ) ) + { + return hit; + } + const bool ctor = r.calleeName == kRubyNew; + if( ctor || anyInLookup( *object.cls, forwarders ) ) + { + if( const rw::SmallVec* hit = definerLookup( object, ctor ? initializeCall : r, false ) ) + { + return hit; + } + } + return rootLookup( r ); +} + +// The fully-qualified constant `written` names from the call site by Ruby's lexical rule — the innermost class or module +// open around it (`site`), its enclosing ones, then the top level (resolve.h rubyResolveMixinConstant) — or empty. +inline std::string RubyClassObjects::lexicalConstant( const std::vector& opens, std::uint32_t site, std::string_view written ) const +{ + if( site != kNoFile ) + { + return rubyResolveMixinConstant( bases.opened, opens, site, written ); + } + const std::string top( written.substr( rubyConstIsAbsolute( written ) ? 2 : 0 ) ); + return opened( top ) ? top : std::string {}; +} + +// Ruby's second step, through ancestors: a bare constant one the innermost open's superclass or mixins hold; a qualified +// one's tail held by its head's (`Sub::Failure` for a Failure Sub's superclass nests). Empty when neither does. +inline std::string RubyClassObjects::inheritedConstant( const std::vector& opens, std::uint32_t site, std::string_view written ) const +{ + const std::size_t cut = written.find( "::" ); + if( cut == std::string_view::npos ) + { + return site != kNoFile ? ancestorConstant( opens[ site ].fqn, written ) : std::string {}; + } + return ancestorConstant( lexicalConstant( opens, site, written.substr( 0, cut ) ), written.substr( cut + 2 ) ); +} + +// `rest` held by `head` (a fully-qualified constant) or by one of its ancestors, as the constant the tree opens; empty when +// none does. +inline std::string RubyClassObjects::ancestorConstant( std::string_view head, std::string_view rest ) const +{ + if( head.empty() || !opened( std::string( head ) ) ) + { + return {}; + } + for( const std::string& a : fqnAncestors( std::string( head ) ) ) + { + std::string cand = a; + cand.append( "::" ).append( rest ); + if( opened( cand ) ) + { + return cand; + } + } + return {}; +} + +// A namespace Zeitwerk makes of a directory is opened by no file: a constant whose last two segments or more are one the +// tree opens names it (`Alerts::Senders::Pay` is the tree's `Senders::Pay`). Empty when none is. +inline std::string RubyClassObjects::zeitwerkConstant( std::string_view written ) const +{ + for( std::string_view tail = written.substr( rubyConstIsAbsolute( written ) ? 2 : 0 ); tail.find( "::" ) != std::string_view::npos; + tail = tail.substr( tail.find( "::" ) + 2 ) ) + { + if( opened( std::string( tail ) ) ) + { + return std::string( tail ); + } + } + return {}; +} + +// The fully-qualified constant r's constant receiver names, as Ruby resolves it from the call site: lexically, then through +// ancestors, then by Zeitwerk's directories. Empty when none answers — a `Point = Struct.new( … )` constant, a gem's class. + +// The fully-qualified constant `written` names from r's call site: lexically (the innermost class or module open around it, +// its enclosing ones, the top level), then through the ancestors of that open, then by Zeitwerk's directories. +inline std::string RubyClassObjects::fqnOfWritten( const Reference& r, std::string_view written ) const +{ + if( r.fileId >= bases.ix.opensByFile.size() ) + { + return {}; + } + const std::vector& opens = bases.ix.opensByFile[ r.fileId ]; + const std::uint32_t site = rubyInnermostOpen( opens, r.startByte ); + std::string fqn = lexicalConstant( opens, site, written ); + if( fqn.empty() && !rubyConstIsAbsolute( written ) ) + { + fqn = inheritedConstant( opens, site, written ); + } + return fqn.empty() ? zeitwerkConstant( written ) : fqn; +} + +// `fqn` and every constant above it along fqnUp, breadth-first; memoised. +inline const std::vector& RubyClassObjects::fqnAncestors( const std::string& fqn ) const +{ + return memoNamesReached( fqnUp, fqnAncestorsMemo, fqn ); +} + +inline bool RubyClassObjects::opened( const std::string& fqn ) const +{ + return bases.opened.find( fqn ) != bases.opened.end(); +} + +// Is a class or module in cls's lookup, read by name, one of `owners`? +inline bool RubyClassObjects::anyInLookup( const std::string& cls, const HashMap& owners ) const +{ + return !owners.empty() && anyAncestor( cls, [ &owners ]( const std::string& a ) { return owners.find( a ) != owners.end(); } ); +} + +// Does `pred` hold of a class or module in cls's lookup, read by name? +template +inline bool RubyClassObjects::anyAncestor( const std::string& cls, Pred&& pred ) const +{ + return std::ranges::any_of( typed.reach.ancestorsOf( cls ), pred ); +} + +// Does a `class << self` in the class object's lookup delegate `callee` (model.h kRubyDelegationCalls)? It then defines a +// class method the tree indexes no def of, and the tree cannot say what answers the call. +inline bool RubyClassObjects::delegates( const RubyClassObject& object, std::string_view callee ) const +{ + std::string key; + const auto delegated = [ & ]( const std::string& a ) + { + key.assign( a ).append( 1, '#' ).append( callee ); + return classDelegated.find( key ) != classDelegated.end(); + }; + return !classDelegated.empty() && anyAncestor( *object.cls, delegated ); +} + +// Is a method_missing (or a `delegate_missing_to`) in the class object's lookup? It answers a name nothing there defines. +inline bool RubyClassObjects::missingAnswers( const RubyClassObject& object ) const +{ + return anyInLookup( *object.cls, missingOwners ); +} + +// Does the tree open the qualified constant `written` — is it, or its last two segments or more, a trailing path of a +// constant the tree opens (RubyTypedReceivers::openedPaths)? Read where the tree keeps no constant index. +inline bool RubyClassObjects::pathOpened( std::string_view written ) const +{ + for( std::string_view tail = written; tail.find( "::" ) != std::string_view::npos; tail = tail.substr( tail.find( "::" ) + 2 ) ) + { + if( typed.openedPaths.find( std::string( tail ) ) != typed.openedPaths.end() ) + { + return true; + } + } + return false; +} + +// Is the def `c` held by the class or module whose fully-qualified constant is `fqn` — is that the innermost open around +// it in its file? +inline bool RubyClassObjects::ownedBy( NodeId c, const std::string& fqn ) const +{ + const Symbol& s = ing.symbols[ c ]; + if( s.fileId >= bases.ix.opensByFile.size() ) + { + return false; + } + const std::vector& opens = bases.ix.opensByFile[ s.fileId ]; + const std::uint32_t o = rubyInnermostOpen( opens, s.sigStartByte ); + return o != kNoFile && opens[ o ].fqn == fqn; +} + +// Does the constant `fqn` name a class (not a module) — is any open of it a class symbol? +inline bool RubyClassObjects::isClass( const std::string& fqn ) const +{ + const auto it = bases.classesByFqn.find( fqn ); + return it != bases.classesByFqn.end() && std::ranges::any_of( it->second, [ this ]( NodeId c ) { return ing.symbols[ c ].kind == SymKind::Class; } ); +} + +// A Ruby call to self (a receiver-less or `self.` call, RubySelfReach::judges) inside a def, read on the side self stands +// on there (parser version 145): the class object in a singleton method (`def self.m`, a def in `class << self`) — the +// class's or the module's — answered by its lookup (`lookup`: its singleton methods, what it extends, Class#new's +// initialize, a reopened root); an instance in any other def of a CLASS, answered by its instances' lookup (the class's +// instance methods, what it includes, its superclasses', then a reopened instance root). No side where self is no one +// class: an instance method of a module (its includers', or a class object it extends), a def a concern defines for its +// includers, a class body, the top level — and none where the tree keeps no constant index. +inline RubySideAnswer RubyClassObjects::selfCall( const Reference& r ) const +{ + const Symbol* caller = r.fromSymbol != kNoNode ? &ing.symbols[ r.fromSymbol ] : nullptr; + if( caller == nullptr || !RubySelfReach::judges( r, caller->scope ) || caller->lang != Lang::Ruby || caller->fileId >= bases.ix.opensByFile.size() + || !( caller->kind == SymKind::Method || caller->kind == SymKind::Function ) || ( defSides[ r.fromSymbol ] & ( kRubyDefIncluder | kRubyDefBlock ) ) != 0 ) + { + return {}; + } + const std::vector& opens = bases.ix.opensByFile[ caller->fileId ]; + const std::uint32_t o = rubyInnermostOpen( opens, caller->sigStartByte ); + const std::string* fqn = o != kNoFile ? &opens[ o ].fqn : nullptr; + const std::size_t cut = fqn != nullptr ? fqn->rfind( "::" ) : std::string::npos; + const auto cls = fqn != nullptr ? typed.reach.classNames.find( fqn->substr( cut == std::string::npos ? 0 : cut + 2 ) ) : typed.reach.classNames.end(); + const bool singleton = ( defSides[ r.fromSymbol ] & kRubyDefSingleton ) != 0; + if( cls == typed.reach.classNames.end() || ( !singleton && !isClass( *fqn ) ) ) + { + return {}; + } + const RubyClassObject object { &cls->first, *fqn, false }; + if( singleton ) + { + return { RubySide::ClassObject, lookup( r, object ), missingAnswers( object ) || delegates( object, r.calleeName ) }; + } + const rw::SmallVec* hit = sideLookup( *fqn, RubySide::Instance, r.calleeName ); + return { RubySide::Instance, hit != nullptr ? hit : typed.rootLookup( r ), missingAnswers( object ) }; +} + +// What a call on a receiver the code builds (of class `cls`, RubyTypedReceivers::classOf) answers from: an instance's +// lookup (sideLookup on the instance side) from the class its type names at the call site, read by fully-qualified +// constant as a constant receiver is, then a reopened instance root. No side where the tree keeps no constant index, the +// constant names nothing it opens, or it names a module — which has no instances: RSpec's implicit `subject` for a +// described module is the module itself — nor for a FactoryBot build, whose factory keeps only its class's final +// segment (RubyTypedReceivers::factoryClass), which the call site cannot place. Open either way when a method_missing is +// in the class's lookup. +inline RubySideAnswer RubyClassObjects::typedLookup( const Reference& r, const std::string& cls ) const +{ + const std::optional built = rubyTypedRecvOf( r ); + const bool open = anyInLookup( cls, missingOwners ); + if( !built || built->factory ) + { + return { std::nullopt, nullptr, open }; + } + const std::string fqn = fqnOfWritten( r, rubyTypedWritten( *built ) ); + if( fqn.empty() || !opened( fqn ) || !isClass( fqn ) ) + { + return { std::nullopt, nullptr, open }; + } + const rw::SmallVec* hit = sideLookup( fqn, RubySide::Instance, r.calleeName ); + return { RubySide::Instance, hit != nullptr ? hit : typed.rootLookup( r ), open }; +} + +// May the Ruby def c answer a call standing on `side` (an instance, or the class object), among the namesakes a call to self +// reaches by name once self's own lookup missed? An instance answers no singleton method and no class method a concern +// gives its includers. A class object answers only a class-side def (a singleton method, a concern's class method for its +// includers), a top-level def or a reopened root's: a module's instance method answers it only through an `extend`, which +// its lookup already walked. Any def of another language: yes. +inline bool RubyClassObjects::answersOn( NodeId c, RubySide side ) const +{ + const Symbol& s = ing.symbols[ c ]; + const char bits = c < defSides.size() ? defSides[ c ] : char( 0 ); + if( s.lang != Lang::Ruby ) + { + return true; + } + if( side == RubySide::Instance ) + { + return ( bits & ( kRubyDefSingleton | kRubyDefIncluder | kRubyDefBlock ) ) == 0; + } + const auto root = [ &s ]( std::string_view r ) { return s.scope == r; }; + return ( bits & ( kRubyDefSingleton | kRubyDefIncluder | kRubyDefBlock ) ) != 0 || s.scope.empty() || std::ranges::any_of( kRubyInstanceRoots, root ) + || s.scope == "Module" || s.scope == "Class"; +} + + +// Those of `ids` that may answer a call standing on `side` (answersOn), or nullptr when none does. +inline const rw::SmallVec* RubyClassObjects::onSide( const rw::SmallVec* ids, RubySide side ) const +{ + sideHits.clear(); + for( const NodeId c : ids != nullptr ? std::span( ids->data(), ids->size() ) : std::span {} ) + { + if( answersOn( c, side ) ) + { + sideHits.push_back( c ); + } + } + return sideHits.empty() ? nullptr : &sideHits; +} + +// RubySelfReach's cut (RubyTopSelf::reachableOf) of `ids` into `out`, then — when the side self stands on is known — only +// what answers on that side (answersOn). +template +inline RubyCut RubyClassObjects::reachableOnSide( const RubyTopSelf& top, const std::string& self, std::optional side, const In& ids, Out& out ) const +{ + const bool cut = top.reachableOf( self, ids, out ); + const auto answers = [ & ]( NodeId c ) { return answersOn( c, *side ); }; + if( !side || out.empty() ) + { + return cut ? RubyCut::Cut : RubyCut::Unchanged; + } + if( std::none_of( out.begin(), out.end(), answers ) ) + { + return RubyCut::Declined; + } + const std::size_t before = out.size(); + out.resize( std::size_t( std::remove_if( out.begin(), out.end(), [ & ]( NodeId c ) { return !answers( c ); } ) - out.begin() ) ); + return cut || out.size() < before ? RubyCut::Cut : RubyCut::Unchanged; +} + +// What a call reaches on a receiver whose every answer the tree knows: one of RSpec's targets or a template's `json`, which +// only a reopened instance root answers (RubyTypedReceivers), and a class object, which its lookup answers +// (RubyClassObjects). Not closed for any other receiver, nor for a class object whose `class << self` delegates the name, +// nor for one whose lookup misses it where a method_missing may answer: Ruby calls that only on a miss. +inline RubyClosedLookup rubyClosedLookup( const Reference& r, const RubyTypedReceivers& typed, const RubyClassObjects& classes ) +{ + if( typed.outOfTreeTarget( r ) ) + { + return { true, typed.rootLookup( r ) }; + } + const RubyClassObject object = classes.of( r ); + if( object.external ) + { + return { true, classes.rootLookup( r ) }; + } + if( object.cls == nullptr || classes.delegates( object, r.calleeName ) ) + { + return {}; + } + const rw::SmallVec* hit = classes.lookup( r, object ); + if( hit == nullptr && classes.missingAnswers( object ) ) + { + return {}; + } + return { true, hit }; +} + +// Every trailing path of every constant the tree opens (`A::B::C` → "A::B::C", "B::C", "C"), read off the base scope's +// constant index — or, where that built none (no Ruby superclass, mixin or constant reference in the tree), the full +// index, built only when a typed or a constant receiver is qualified and so needs it. +// Is r's receiver a qualified constant — a typed one's class (not a factory), or a constant receiver — `A::B`? +inline bool rubyQualifiedReceiver( const Reference& r ) noexcept +{ + const std::optional typed = rubyTypedRecvOf( r ); + const std::string_view written = typed ? ( typed->factory ? std::string_view {} : typed->type ) : rubyConstantReceiver( r ); + return written.find( "::" ) != std::string_view::npos; +} + +inline void rubyOpenedPaths( const IngestResult& ing, const RubyBaseScope& bases, HashMap& out ) +{ + const bool own = bases.ix.opensByFile.empty() && std::ranges::any_of( ing.references, []( const Reference& r ) { return rubyQualifiedReceiver( r ); } ); + const RubyConstantIndex ownIx = own ? rubyConstantIndexOf( ing ) : RubyConstantIndex {}; + for( const std::vector& opens : ( own ? ownIx : bases.ix ).opensByFile ) + { + for( const RubyOpenRec& o : opens ) + { + for( std::size_t from = 0; from != std::string::npos; ) + { + out.try_emplace( o.fqn.substr( from ), 1 ); + const std::size_t next = o.fqn.find( "::", from ); + from = next == std::string::npos ? next : next + 2; + } + } + } +} + +// Which of RSpec's builders (model.h kRspecTargets) the tree defines a Ruby method of the name of — a def or an accessor — +// where an example group's self can reach it: a top-level def, a reopened Object's, a def inside a group's block (it has +// no class around it), a module's in test code (what a `config.include` mixes in). A class's method is not in a group's +// lookup, nor an application module's: a migration's `def change`, or its mixin's, shadows nothing +// (test/rubyrspectargetcheck.sh floor (b)). +// Every Ruby class or module name whose methods no example group's self reaches: opened, and never as a module in test +// code (filter.h isTestSymbol, or under a spec/ directory) — and not an instance root. +inline HashMap rubyGroupForeignOwners( const IngestResult& ing ) +{ + HashMap owners, testModules; + for( std::size_t i = 0; i < ing.symbols.size(); ++i ) + { + const Symbol& s = ing.symbols[ i ]; + if( s.lang == Lang::Ruby && ( s.kind == SymKind::Class || s.kind == SymKind::Other ) ) + { + owners.try_emplace( s.name, 1 ); + if( s.kind == SymKind::Other && ( isTestSymbol( ing, i ) || hasDirSegment( rootRelPath( ing, s.fileId ), "spec/" ) ) ) + { + testModules.try_emplace( s.name, 1 ); + } + } + } + HashMap foreign; + for( const auto& [ name, one ] : owners ) + { + if( testModules.find( name ) == testModules.end() && std::ranges::find( kRubyInstanceRoots, name ) == std::end( kRubyInstanceRoots ) ) + { + foreign.try_emplace( name, one ); + } + } + return foreign; +} + +inline void rubyDefinedBuilders( const IngestResult& ing, std::array& out ) +{ + const HashMap foreign = rubyGroupForeignOwners( ing ); + for( const Symbol& s : ing.symbols ) + { + const bool method = s.lang == Lang::Ruby && ( s.kind == SymKind::Function || s.kind == SymKind::Method || s.kind == SymKind::Var ); + const auto it = method ? std::ranges::find( kRspecTargets, std::string_view( s.name ), &RspecTarget::builder ) : std::end( kRspecTargets ); + if( it != std::end( kRspecTargets ) && foreign.find( s.scope ) == foreign.end() ) + { + out[ std::size_t( it - std::begin( kRspecTargets ) ) ] = true; + } + } +} + +// The kRubyDef* bit a RubySingletonDef binding's mark names (model.h kRubyIncluderMark, kRubyClassMethodsMark; none: a +// singleton method). +inline char rubyDefBitOfMark( std::string_view mark ) noexcept +{ + if( mark == kRubyIncluderMark ) + { + return kRubyDefIncluder; + } + return mark == kRubyClassMethodsMark ? kRubyDefBlock : kRubyDefSingleton; +} + +// The singleton bindings (LocalBindKind::RubySingletonDef): each def or accessor site — `def self.m`, a def or an accessor +// in `class << self` — as (fileId << 32) | its start byte, which a def's own start (Symbol::sigStartByte) joins, → its +// side: kRubyDefIncluder or kRubyDefBlock for one that is each includer's (parser version 145), else kRubyDefSingleton; +// and each name a +// `class << self` delegates, into out.classDelegated (`delegate_missing_to`, every name: out.missingOwners). +inline HashMap rubySingletonSites( const IngestResult& ing, RubyClassObjects& out ) +{ + HashMap sites; + for( const Binding& b : ing.bindings ) + { + if( b.kind != LocalBindKind::RubySingletonDef ) + { + continue; + } + if( b.typeName.empty() ) + { + sites.try_emplace( ( std::uint64_t( b.fileId ) << 32 ) | b.startByte, rubyDefBitOfMark( b.importedName ) ); + } + else if( b.var.empty() ) + { + out.missingOwners.try_emplace( b.typeName, 1 ); + } + else + { + out.classDelegated.try_emplace( b.typeName + "#" + b.var, 1 ); + } + } + return sites; +} + +// Every Ruby module name whose methods a class object may answer from when the module is extended onto it — every module the +// tree opens but a concern, one that nests a `module ClassMethods`: that module is included, and its class methods live +// in its ClassMethods. +inline HashMap rubyExtendableModules( const IngestResult& ing ) +{ + HashMap modules; + HashMap concerns; + for( const Symbol& s : ing.symbols ) + { + if( s.lang == Lang::Ruby && s.kind == SymKind::Other ) + { + modules.try_emplace( s.name, 1 ); + } + if( rubyIsConcernClassMethods( s ) ) + { + concerns.try_emplace( s.scope, 1 ); + } + } + for( const auto& [ name, one ] : concerns ) + { + modules.erase( name ); + } + return modules; +} + +// Every def a class object may answer from (RubyClassObjects::defSides, per symbol) → its kRubyDef* bits: a singleton +// method or an includer's class method (rubySingletonSites), and every method or accessor of an extendable module +// (rubyExtendableModules) or a concern's ClassMethods, which a call on the module answers. A class's instance method is +// its instances' alone: its bits are 0. +inline void rubyClassObjectDefs( const IngestResult& ing, RubyClassObjects& out ) +{ + const HashMap singletons = rubySingletonSites( ing, out ); + const HashMap modules = rubyExtendableModules( ing ); + const HashMap& nested = out.typed.reach.qualifiedOwner; + out.defSides.assign( ing.symbols.size(), 0 ); + for( const Symbol& s : ing.symbols ) + { + if( s.lang != Lang::Ruby || s.scope.empty() || !( s.kind == SymKind::Method || s.kind == SymKind::Function || s.kind == SymKind::Var ) ) + { + continue; + } + const auto site = singletons.find( ( std::uint64_t( s.fileId ) << 32 ) | s.sigStartByte ); + const bool module = nested.find( s.id ) != nested.end() || modules.find( s.scope ) != modules.end(); + out.defSides[ s.id ] = char( ( site != singletons.end() ? site->second : 0 ) | ( module ? kRubyDefModule : 0 ) ); + } +} + +// Every mixin the class object's lookup reaches (LocalBindKind::RubyClassMixin) as (fileId << 32) | its constant's start +// byte, which its inherit reference's own start joins → how it joins: Extend, or IncluderExtend for one a concern's +// `included do` writes. +inline HashMap rubyClassMixinSites( const IngestResult& ing ) +{ + HashMap sites; + for( const Binding& b : ing.bindings ) + { + if( b.kind == LocalBindKind::RubyClassMixin && b.var.empty() ) + { + sites.try_emplace( ( std::uint64_t( b.fileId ) << 32 ) | b.startByte, b.importedName == kRubyIncluderMark ? RubyAncestry::IncluderExtend : RubyAncestry::Extend ); + } + } + return sites; +} + +// How the Ruby inherit reference `i` joins its writer's lookup: a superclass; a mixin the class object's lookup reaches, +// an extend (rubyClassMixinSites); any other mixin, an include. +inline RubyAncestry rubyAncestryOf( const RubyBaseScope& bases, std::size_t i, const Reference& r, const HashMap& classSide ) +{ + if( i >= bases.mixinRefs.size() || bases.mixinRefs[ i ] == 0 ) + { + return RubyAncestry::Superclass; + } + const auto side = classSide.find( ( std::uint64_t( r.fileId ) << 32 ) | r.startByte ); + return side != classSide.end() ? side->second : RubyAncestry::Include; +} + +// Each Ruby inherit reference — superclass or mixin — the constant index scoped, from the constant of the class or module +// that writes it, into the ancestry by fully-qualified constant (RubyClassObjects::fqnSides), with how it joins the +// writer's lookup (rubyAncestryOf). +inline void rubyFqnInherits( const IngestResult& ing, const RubyBaseScope& bases, HashMap>& out ) +{ + const HashMap classSide = rubyClassMixinSites( ing ); + for( std::size_t i = 0; i < ing.references.size(); ++i ) + { + const Reference& r = ing.references[ i ]; + const std::string* base = r.isInherit && r.fromSymbol != kNoNode ? bases.resolvedBase( i ) : nullptr; + const std::string* derived = base != nullptr && !base->empty() ? bases.fqnOfSymbol( ing.symbols[ r.fromSymbol ] ) : nullptr; + if( derived != nullptr ) + { + out[ *derived ].push_back( { *base, rubyAncestryOf( bases, i, r, classSide ) } ); + } + } +} + +// Each module's `extend self` (a LocalBindKind::RubyClassMixin binding of var "self"), into the ancestry as an extend of +// itself, from the open around the directive. +inline void rubyFqnSelfExtends( const IngestResult& ing, const RubyBaseScope& bases, HashMap>& out ) +{ + for( const Binding& b : ing.bindings ) + { + if( b.kind != LocalBindKind::RubyClassMixin || b.var != "self" || b.fileId >= bases.ix.opensByFile.size() ) + { + continue; + } + const std::vector& opens = bases.ix.opensByFile[ b.fileId ]; + const std::uint32_t o = rubyInnermostOpen( opens, b.startByte ); + if( o != kNoFile ) + { + out[ opens[ o ].fqn ].push_back( { opens[ o ].fqn, RubyAncestry::Extend } ); + } + } +} + +// The ancestry by fully-qualified constant (RubyClassObjects::fqnSides): the inherit references (rubyFqnInherits), each +// module's `extend self` (rubyFqnSelfExtends), and each concern's nested `module ClassMethods`, which +// ActiveSupport::Concern extends onto every includer, from the concern that holds it. +inline void rubyFqnAncestry( const IngestResult& ing, const RubyBaseScope& bases, HashMap>& out ) +{ + rubyFqnInherits( ing, bases, out ); + rubyFqnSelfExtends( ing, bases, out ); + for( const Symbol& s : ing.symbols ) + { + const std::string* fqn = rubyIsConcernClassMethods( s ) ? bases.fqnOfSymbol( s ) : nullptr; + const std::size_t cut = fqn != nullptr ? fqn->rfind( "::" ) : std::string::npos; + if( cut != std::string::npos ) + { + out[ fqn->substr( 0, cut ) ].push_back( { *fqn, RubyAncestry::ClassMethods } ); + } + } + const auto key = []( const RubyAncestorEdge& e ) { return std::tie( e.fqn, e.how ); }; + for( auto& [ k, v ] : out ) + { + std::ranges::sort( v, {}, key ); + const auto dup = std::ranges::unique( v, {}, key ); + v.erase( dup.begin(), dup.end() ); + } +} + +// The ancestry's names alone (RubyClassObjects::fqnUp): each constant's ancestors in fqnSides, one per constant, in order. +inline void rubyFqnNames( const HashMap>& sides, HashMap>& up ) +{ + for( const auto& [ k, v ] : sides ) + { + std::vector& names = up[ k ]; + for( const RubyAncestorEdge& e : v ) // sorted by constant first + { + if( names.empty() || names.back() != e.fqn ) + { + names.push_back( e.fqn ); + } + } + } +} + +// Every Ruby class whose superclass is written as a forwarding base (kRubyInstanceForwardingBases): the superclass +// directive sits at the class's own start byte (ingest_relations.h), which is the class symbol's. +inline void rubyInstanceForwarders( const IngestResult& ing, HashMap& out ) +{ + HashMap sites; // (fileId << 32) | the class's start byte + for( const Include& inc : ing.includes ) + { + const std::string_view written = std::string_view( inc.target ).substr( rubyConstIsAbsolute( inc.target ) ? 2 : 0 ); + if( inc.isSymbolic && !inc.isValueUse && std::ranges::find( kRubyInstanceForwardingBases, written ) != std::end( kRubyInstanceForwardingBases ) ) + { + sites.try_emplace( ( std::uint64_t( inc.fileId ) << 32 ) | inc.byte, 1 ); + } + } + if( sites.empty() ) + { + return; + } + for( const Symbol& s : ing.symbols ) + { + if( s.lang == Lang::Ruby && s.kind == SymKind::Class && sites.find( ( std::uint64_t( s.fileId ) << 32 ) | s.sigStartByte ) != sites.end() ) + { + out.try_emplace( s.name, 1 ); + } + } +} + +// Every FactoryBot factory the tree defines → the class it builds (ingest_binds.h captureRubyFactories); a name two +// definitions give different classes is a tombstone: it types nothing. Every trailing path of every constant the tree +// opens, which a qualified constant must be. And which of RSpec's builders the tree defines a method of the name of. +inline RubyTypedReceivers buildRubyTypedReceivers( const IngestResult& ing, const RubySelfReach& reach, const RubyBaseScope& bases, + const HashMap>& up, const Narrower& narrower ) +{ + RubyTypedReceivers out{ .reach = reach, .up = up, .narrower = narrower }; + rubyOpenedPaths( ing, bases, out.openedPaths ); + rubyDefinedBuilders( ing, out.builderDefined ); + for( const Binding& b : ing.bindings ) + { + if( b.kind != LocalBindKind::RubyFactory || b.var.empty() ) + { + continue; + } + const auto [ it, inserted ] = out.factoryClass.try_emplace( b.var, b.typeName ); + if( !inserted && it->second != b.typeName ) + { + it->second.clear(); + } + } + return out; +} + +// Ruby's lookup on a class object over one graph (RubyClassObjects): beside the classes it reads the SCREAMING constants the +// tree assigns (a call on one is on a value, left to the name ladder), every class or module defining a method_missing (a +// nested ClassMethods by the concern that holds it, as the overlay keys it), the defs a class object answers from +// (rubyClassObjectDefs), the classes below a forwarding base, and the ancestry by fully-qualified constant. +inline RubyClassObjects buildRubyClassObjects( const IngestResult& ing, const RubyTypedReceivers& typed, const RubyBaseScope& bases ) +{ + RubyClassObjects out{ .typed = typed, .ing = ing, .bases = bases }; + for( const Symbol& s : ing.symbols ) + { + const auto q = s.lang == Lang::Ruby && s.name == "method_missing" ? typed.reach.qualifiedOwner.find( s.id ) : typed.reach.qualifiedOwner.end(); + if( s.lang == Lang::Ruby && s.kind == SymKind::Var && rubyIsConstantName( s.name ) ) + { + out.constantValues.try_emplace( s.name, 1 ); + } + else if( s.lang == Lang::Ruby && s.name == "method_missing" && !s.scope.empty() ) + { + out.missingOwners.try_emplace( q != typed.reach.qualifiedOwner.end() ? q->second : s.scope, 1 ); + } + } + rubyClassObjectDefs( ing, out ); + rubyInstanceForwarders( ing, out.forwarders ); + rubyFqnAncestry( ing, bases, out.fqnSides ); + rubyFqnNames( out.fqnSides, out.fqnUp ); + out.initializeCall.lang = Lang::Ruby; + out.initializeCall.calleeName = kRubyInitialize; + return out; +} + +// The gate's evidence, built once per graph. Deterministic: class ids follow symbol-id order, and each file's list is +// sorted and deduplicated before any lookup. Files with no symbol in a gated language get no list (their calls are +// never gated), which keeps a mostly-C++ corpus from paying for a Python corner of it. +// +// collectFileClassEvidence: every class NAME each gated file mentions — the classes it defines, any class a reference +// there names as callee, receiver or qualifier, or a binding names as type, variable or imported name — and every class +// an ES import binding in that file resolves to (a default or renamed import names it under another spelling). +// One class NAME a gated file mentions. A DEFINITION (the class-like symbol, or the bare declared name its statement +// leaves) names it in fileClasses only; any other mention names it in fileRefClasses too — the list a Python call on a +// receiver other than self/cls reads (BuiltinMethodGate::judge). +inline void noteFileClass( BuiltinMethodGate& gate, const std::vector& fileGated, std::uint32_t fileId, const std::string& name, bool definition ) +{ + if( fileId >= fileGated.size() || fileGated[ fileId ] == 0 || name.empty() ) + { + return; + } + const auto it = gate.classId.find( name ); + if( it == gate.classId.end() ) + { + return; + } + gate.fileClasses[ fileId ].push_back( it->second ); + if( !definition ) + { + gate.fileRefClasses[ fileId ].push_back( it->second ); + } +} + +// The class names one binding mentions: its type, its variable and its imported name. A bare declared NAME with no type +// and no import is how Python records the `class Pool:` statement's own module-level name (a VarDecl) — the definition +// again, not a use of the class — so that variable names it as a definition. +inline void noteBindingClasses( BuiltinMethodGate& gate, const std::vector& fileGated, const Binding& b ) +{ + const bool bareDeclaration = b.kind == LocalBindKind::VarDecl && b.typeName.empty() && b.importedName.empty(); + noteFileClass( gate, fileGated, b.fileId, b.typeName, false ); + noteFileClass( gate, fileGated, b.fileId, b.var, bareDeclaration ); + noteFileClass( gate, fileGated, b.fileId, b.importedName, false ); +} + +inline void collectFileClassEvidence( const IngestResult& ing, const std::vector& fileGated, const JsImportTables& jsImports, BuiltinMethodGate& gate ) +{ + gate.fileClasses.assign( ing.files.size(), {} ); + gate.fileRefClasses.assign( ing.files.size(), {} ); + const auto note = [ & ]( std::uint32_t fileId, const std::string& name, bool definition = false ) + { + noteFileClass( gate, fileGated, fileId, name, definition ); + }; + for( const Symbol& s : ing.symbols ) + { + if( BuiltinMethodGate::isClassLike( s ) ) + { + note( s.fileId, s.name, /*definition=*/true ); // defined here: every call in the file may be on an instance of it + } + } + for( const Reference& r : ing.references ) + { + if( !r.isDocLink ) // a backtick mention in prose is not code evidence + { + note( r.fileId, r.calleeName ); + note( r.fileId, r.recvVar ); + note( r.fileId, r.qualifier ); + } + } + for( const Binding& b : ing.bindings ) + { + noteBindingClasses( gate, fileGated, b ); + } + for( const auto& [ importKey, bound ] : jsImports.targets ) // key "#" (jsImportKey) + { + if( bound.outcome != JsImportOutcome::Pinned || bound.node >= ing.symbols.size() || !BuiltinMethodGate::isClassLike( ing.symbols[ bound.node ] ) ) + { + continue; + } + std::uint32_t fileId = 0; + const auto [ end, ec ] = std::from_chars( importKey.data(), importKey.data() + importKey.size(), fileId ); + if( ec == std::errc() && end != importKey.data() ) + { + note( fileId, ing.symbols[ bound.node ].name ); // order-free: every list is sorted below + } + } + for( std::vector& named : gate.fileClasses ) + { + std::sort( named.begin(), named.end() ); + named.erase( std::unique( named.begin(), named.end() ), named.end() ); + } + for( std::vector& named : gate.fileRefClasses ) + { + std::sort( named.begin(), named.end() ); + named.erase( std::unique( named.begin(), named.end() ), named.end() ); + } +} + +inline BuiltinMethodGate buildBuiltinMethodGate( const IngestResult& ing, const ExternalVeto& veto, const ExternalVetoTables& vetoTables, + const std::vector>& directIncludes, const JsImportTables& jsImports ) +{ + PROFILE_SCOPE_DESCRIBE( "buildGraph/2j: builtin-method name gate" ); + BuiltinMethodGate gate{ ing, veto, vetoTables, directIncludes, jsImports, {}, {}, {}, {}, {}, {}, {}, {}, {}, false }; + std::vector fileGated( ing.files.size(), 0 ); + for( const Symbol& s : ing.symbols ) + { + if( s.fileId < fileGated.size() && !BuiltinMethodGate::tableFor( s.lang ).empty() ) + { + fileGated[ s.fileId ] = 1; + gate.active = true; + } + } + if( !gate.active ) + { + return gate; + } + for( const Symbol& s : ing.symbols ) + { + if( BuiltinMethodGate::isClassLike( s ) && gate.classId.try_emplace( s.name, std::uint32_t( gate.className.size() ) ).second ) + { + gate.className.push_back( s.name ); + } + } + // Containers only in gated files: a gated call's targets are language-compatible with it, so they live there too. + gate.containersByFile = symbolsByFileInIdOrder( ing, [ & ]( const Symbol& s ) + { return fileGated[ s.fileId ] != 0 && ( BuiltinMethodGate::isClassLike( s ) || s.kind == SymKind::Function || s.kind == SymKind::Method ); } ); + collectFileClassEvidence( ing, fileGated, jsImports, gate ); + return gate; +} + +// Counts one call reference's disposition when its resolve-loop iteration ENDS — on every exit, each `continue` +// included — so no exit needs a counter of its own, only the assignment naming what happened. A reference that +// leaves naming nothing is counted Unaccounted, and the conservation line cannot balance silently. +struct DispositionTally +{ + const CallDisposition& disposition; + CallDispositionCounts& counts; + ~DispositionTally() + { + ++counts[ std::size_t( disposition ) ]; + } +}; + +// `census` arms the eval-only S6-C silent-pin census (src/pincensus.h): every DECIDED call site records +// which narrowing stage committed it and to which canonical target. It adds rows to g.pinCensus and +// changes NOTHING else — no candidate is admitted, dropped or reordered by it, so the emitted map is +// byte-identical armed or not (test/pincensuscheck.sh arm (E) is the executable form of that sentence). + +inline void internDeclinedList( Graph& g, HashMap>& listsByHash, std::span cand ); // defined beside declinedCallsNaming, its one reader +inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = nullptr, bool census = false ) +{ + PROFILE_SCOPE_DESCRIBE( "buildGraph: resolve refs + build CSR" ); + const std::size_t N = ing.symbols.size(); + Graph g; + g.wOutDeg.assign( N, 0.0 ); + g.ambOut.assign( N, 0u ); // counted in the resolve loop: calls that stay split across >1 def after narrowing + g.locPinOut.assign( N, 0u ); // counted in the resolve loop: calls the S6-C locality tie-break alone pinned to one def + g.unresolvedOut.assign( N, 0u ); // counted in the resolve loop: calls whose in-repo defs were all lang-filtered + g.declinedOut.assign( N, 0u ); // counted in the resolve loop: calls tier 3 declined to guess at (no edge) + if( scip ) { g.scipDocsSeen = scip->documentsSeen; g.scipEdgesPinned = scip->edgesPinned; } + // #66: carry the crawl's own unindexed-extension roll-up onto the graph, so the verbs that answer off + // this CSR can disclose the same gap the map header already prints. Summed HERE, from the identical + // vector the header reads (ing.crawlSkips.unindexedExts), because a second traversal elsewhere is + // exactly the two-derivations-of-one-number defect M15 fixed for the gauge pair. + for( const UnindexedExt& ue : ing.crawlSkips.unindexedExts ) { g.unindexedFiles += ue.files; } + + // S6-C canonical ids: `path::scope::name` per symbol (bare name when no scope). Computed once here so the + // resolution locality tie-break (below) and serialize's `id=` attribute share one definition. Deterministic. + g.canonId.resize( N ); + g.localityKey.resize( N ); + { + PROFILE_SCOPE_DESCRIBE( "buildGraph/1a: canonId + localityKey (per symbol)" ); + for( const Symbol& s : ing.symbols ) + { + g.canonId[ s.id ] = canonicalId( ing.files[ s.fileId ], s.scope, s.name ); + g.localityKey[ s.id ] = localityKeyOf( ing.files[ s.fileId ], s.scope, s.name ); // == canonId when scoped + } + } + + // A4-R5 JNI: decode every `Java_pkg_Cls_method` C/C++ def to its readable dotted Java name and stash it as + // the symbol's binding label. No ingest capture / cache change — it is a pure function of the def NAME. The + // vector stays EMPTY (no allocation) when the tree holds no JNI export, so a JNI-free corpus is unaffected. + { + PROFILE_SCOPE_DESCRIBE( "buildGraph/1b: JNI name decode (per symbol)" ); + for( const Symbol& s : ing.symbols ) + { + if( ( s.lang != Lang::Cpp && s.lang != Lang::ObjC ) || s.name.size() <= 5 || s.name.compare( 0, 5, "Java_" ) != 0 ) + { + continue; + } + std::string readable = decodeJniName( s.name ); + if( readable.empty() ) + { + continue; + } + if( g.bindLabel.empty() ) + { + g.bindLabel.assign( N, std::string() ); + } + g.bindLabel[ s.id ] = std::move( readable ); + } + } + + // ── Multi-root workspace: name-based resolution NEVER crosses roots. Every + // name-tier candidate below (qualified canonical, Rule 1/2, the name-based-fallback byName fill, inheritance, doc + // mentions, HAS-A) is filtered to the REFERENCE's own root; cross-root edges enter ONLY via evidence + // (the path-resolved include set feeding Rule 3, and the FFI binding tables — both left unfiltered by + // design). fileRoot is EMPTY on a single-root run, so sameRoot is constant-true and the resolved graph + // is byte-identical to today (the G5 quarantine). + const bool multiRoot = !ing.fileRoot.empty(); + const auto sameRoot = [ & ]( NodeId candSym, std::uint32_t refFileId ) noexcept -> bool + { + return !multiRoot || ing.fileRoot[ ing.symbols[ candSym ].fileId ] == ing.fileRoot[ refFileId ]; + }; + + // file → directory id (path up to the last '/') + HashMap dirIds; + std::vector fileDir( ing.files.size(), 0 ); + { + PROFILE_SCOPE_DESCRIBE( "buildGraph/1c: fileDir (dir interning)" ); + for( std::size_t f = 0; f < ing.files.size(); ++f ) + { + std::string_view p = ing.files[f]; + const std::size_t sl = p.rfind( '/' ); + std::string dir = ( sl == std::string_view::npos ) ? std::string() : std::string( p.substr( 0, sl ) ); + fileDir[f] = dirIds.emplace( std::move( dir ), std::uint32_t( dirIds.size() ) ).first->second; + } + } + + // name → candidate definition ids. rw::svector<,2>: most names define 1-2 symbols, so the id-list is + // inline (no per-name malloc) and size() is branch-free — the measured-best value type for this + // write-once (here) / read-hot (resolve below) shape (see bench/bench_svector3.cpp). Iterates in + // insertion order exactly like std::vector, so the resolved graph — and the output — is unchanged. + HashMap> byName; + byName.reserve( N ); // ≤ one entry per symbol → skip the rehash cascade + { + PROFILE_SCOPE_DESCRIBE( "buildGraph/1d: byName (name -> def ids)" ); + for( const Symbol& s : ing.symbols ) + { + byName[ s.name ].push_back( s.id ); + } + } + + // decl/def collapse (adversarial-review #1) — collapseDeclarationsOfName carries the rule, its per-root and per-family + // keys, and why a Kotlin body never evicts a Java declaration (or the reverse). + { + PROFILE_SCOPE_DESCRIBE( "buildGraph/1e: decl/def collapse" ); + for( auto& [ name, ids ] : byName ) + { + collapseDeclarationsOfName( ing, multiRoot, ids ); + } + } + + // canonical scope::name → definition ids (E#4): lets a qualified call `A::b()` resolve to the `b` whose + // enclosing scope is `A`, BEFORE the bare-name spray — the deterministic [AST] cut to call-graph // ambiguity. Definitions only (body present); the obj.method()/unqualified halves stay bare-name (and // keep their honest `amb`). C++ only (scope is populated for Lang::Cpp). HashMap> canonByName; @@ -3196,14 +4910,19 @@ inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = null { continue; // a Ruby base the tree never opens: nothing in-tree to walk to } + if( const std::size_t at = std::size_t( &ir - ing.references.data() ); at < rubyBases.mixinRefs.size() && rubyBases.mixinRefs[ at ] != 0 ) + { + continue; // a Ruby mixin: only the Ruby overlay (buildRubyCha) carries it + } std::string_view derivedName; if( !ir.qualifier.empty() ) { // Rust impl → derived type name in qualifier derivedName = ir.qualifier; } - else if( ir.fromSymbol != kNoNode && isClassLikeK( ing.symbols[ ir.fromSymbol ].kind ) ) + else if( ir.fromSymbol != kNoNode && ( isClassLikeK( ing.symbols[ ir.fromSymbol ].kind ) + || BuiltinMethodGate::isClassLike( ing.symbols[ ir.fromSymbol ] ) ) ) { - derivedName = ing.symbols[ ir.fromSymbol ].name; // the ref sits inside the derived class header + derivedName = ing.symbols[ ir.fromSymbol ].name; // the derived class header — or a Ruby module's mixin } if( derivedName.empty() || ir.calleeName.empty() ) { @@ -3221,11 +4940,18 @@ inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = null for( auto& [ k, v ] : chaUp ) { std::sort( v.begin(), v.end() ); v.erase( std::unique( v.begin(), v.end() ), v.end() ); } for( auto& [ k, v ] : chaDown ) { std::sort( v.begin(), v.end() ); v.erase( std::unique( v.begin(), v.end() ), v.end() ); } } + const RubyCha rubyCha = buildRubyCha( ing, rubyBases, chaUp, chaDown ); + const ChaUpNames& rubyUp = rubyCha.merged ? rubyCha.up : chaUp; // a Ruby call to self reads mixins too + const RubySelfReach rubyReach = buildRubySelfReach( ing, rubyUp, rubyCha.merged ? rubyCha.down : chaDown, rubyBases ); + const RubyTypedReceivers rubyTypes = buildRubyTypedReceivers( ing, rubyReach, rubyBases, rubyUp, narrower ); // a receiver the code builds + const RubyClassObjects rubyClasses = buildRubyClassObjects( ing, rubyTypes, rubyBases ); // a receiver that is a class + const RubyTopSelf rubyTop = buildRubyTopSelf( ing, rubyReach ); // a task file's or a template's self outside any class const std::vector specializationsWithBases = sortedSpecializationNames( chaUp ); // resolve.h: what a C++ specialization inherits ChaConeMemo chaCones( chaUp, chaDown ); // one cone per receiver type, computed on first use (see the type) const ClassIdentity classIds = buildClassIdentity( ing, chaUp ); // Rule 2's class identity: nesting, owners, real inheritance (resolve.h) std::vector filtScratch; // reused per-call survivor buffer for CHA-lite / arity filtering rw::SmallVec reachScratch; // reused per-call buffer: the candidates a call can reach by name (reachableByName) + rw::SmallVec rubyReachable; // reused per-call buffer: the candidates a Ruby call to self can reach (RubySelfReach) // ---- census arming + the ORACLE side (eval-only; src/pincensus.h) ------------------------------ // The oracle rows are a straight transcription of the overlay's own (from, calleeName) → target table @@ -3597,23 +5323,87 @@ inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = null // otherwise narrowed stays false and we fall through to the unchanged name-based-fallback ladder. Skipped when the call // was already pinned by an explicit `A::` qualifier (canonical) — that is the more specific signal. // (`narrowed` is declared above the L3 fn-binding block, which fires ahead of Rule 1.) - if( !scipPinned && !canonical && !narrowed ) + // Ruby: a call to self inside a def answers from what self is there — the class object in a singleton method, an + // instance in a class's other defs (RubyClassObjects::selfCall; parser version 145). Rule 1 and its base walk, which + // read a class's defs of both sides alike, step aside for it. + const RubySideAnswer rubySelfCall = !scipPinned && !canonical && !narrowed ? rubyClasses.selfCall( r ) : RubySideAnswer {}; + bool rubySideNarrowed = false; // the side-aware lookup chose the candidates: right by construction + if( rubySelfCall.side ) + { + narrowed = narrowTo( rubySelfCall.hit, r, cand ); + rubySideNarrowed = narrowed; + } + // A bare `new` in a class method is Class#new: it runs the first initialize self's instances find, and self may be a + // class below this one. None in the class's lookup and none in reach below it — Ruby's is outside the tree, as for + // `C.new`; one below it — the call declines. Never another class's `def self.new` by name. + if( rubySelfCall.side == RubySide::ClassObject && rubySelfCall.hit == nullptr && !rubySelfCall.open && r.calleeName == kRubyNew ) + { + const auto inits = byName.find( std::string( kRubyInitialize ) ); + rubyReachable.clear(); + if( inits != byName.end() ) + { + rubyTop.reachableOf( rubyReach.selfOf( ing, r ), inits->second, rubyReachable ); + } + if( rubyReachable.empty() ) + { + disposition = vetoExternal( r ); + continue; + } + ++g.declinedOut[ r.fromSymbol ]; + internDeclinedList( g, declinedListsByHash, std::span( rubyReachable.data(), rubyReachable.size() ) ); + disposition = CallDisposition::Declined; + continue; + } + if( !scipPinned && !canonical && !narrowed && !rubySelfCall.side ) { - narrowed = narrowTo( narrower.rule1ClassMember( r, ing.symbols[ r.fromSymbol ].scope ), r, cand ); + narrowed = narrowTo( narrower.rule1ClassMember( r, callerSelfScope( ing, r ) ), r, cand ); } // Phase 5 (docs/EVALS.md "Phase 5", mechanism 2): Rule 1's shapes walk the enclosing class's BASES when the // class itself defines no `m` — and `super().m()` (RecvKind::SuperObj) walks the bases ONLY. See // Narrower::rule1BaseWalk. A `super()` miss is a VETO: the MRO left the indexed tree, and the spray below // would hand the site to the caller's own class — the one class `super()` skips by definition. - if( !scipPinned && !canonical && !narrowed ) + if( !scipPinned && !canonical && !narrowed && !rubySelfCall.side ) { - narrowed = narrowTo( narrower.rule1BaseWalk( r, ing.symbols[ r.fromSymbol ].scope, chaUp, chaUpDeclared ), r, cand ); + narrowed = narrowTo( narrower.rule1BaseWalk( r, callerSelfScope( ing, r ), r.lang == Lang::Ruby ? rubyUp : chaUp, chaUpDeclared ), r, cand ); } if( !scipPinned && !canonical && !narrowed && r.recv == RecvKind::SuperObj && bindingTier.empty() ) { disposition = vetoExternal( r ); continue; } + // Ruby: a receiver the code builds is an instance, and answers from its class's instances' lookup — by + // fully-qualified constant where the tree keeps an index (RubyClassObjects::typedLookup) and that finds the method; + // else by name (RubyTypedReceivers), only its instance methods when the class is known: a type written as its final + // segment alone (`described_class`'s, which ingest keeps so) may name another class from the call site. When that + // defines nothing too, from what the class's instances reach (the RubySelfReach cut below, read with the class as + // self). + const std::string* const rubyTyped = !scipPinned && !canonical && !narrowed ? rubyTypes.classOf( r ) : nullptr; + RubySideAnswer rubyBuilt; + if( rubyTyped != nullptr ) + { + rubyBuilt = rubyClasses.typedLookup( r, *rubyTyped ); + rubySideNarrowed = rubyBuilt.side && narrowTo( rubyBuilt.hit, r, cand ); + narrowed = rubySideNarrowed + || narrowTo( rubyBuilt.side ? rubyClasses.onSide( rubyTypes.lookup( r, *rubyTyped ), *rubyBuilt.side ) : rubyTypes.lookup( r, *rubyTyped ), r, cand ); + } + else if( const RubyClosedLookup closed = !scipPinned && !canonical && !narrowed ? rubyClosedLookup( r, rubyTypes, rubyClasses ) : RubyClosedLookup {}; closed.closed ) + { + if( !narrowTo( closed.hit, r, cand ) ) + { + disposition = vetoExternal( r ); // RSpec's own target, a class whose lookup leaves the tree: the method Ruby runs is outside it + continue; + } + narrowed = true; + } + // the side self stands on, for the RubySelfReach cuts below: a built instance's, or the call to self's, wherever the + // side-aware lookup could be walked. None once that lookup chose the candidates (C.new's initialize and a forwarding + // base's instance method are right there, on the other side), nor where a method_missing may answer what it misses. + const RubySideAnswer& rubyAsked = rubyTyped != nullptr ? rubyBuilt : rubySelfCall; + const std::optional rubySide = rubySideNarrowed || rubyAsked.open ? std::nullopt : rubyAsked.side; + const std::string* const rubySelf = rubyTyped != nullptr ? rubyTyped + : r.lang == Lang::Ruby && r.fromSymbol != kNoNode && RubySelfReach::judges( r, callerSelfScope( ing, r ) ) + ? &rubyReach.selfOf( ing, r ) + : rubyTop.selfOf( r ); // P2-D Rule 2 (receiver-variable type): a named-receiver call `x.m()` / `x->m()` resolves to the method on the VARIABLE's type (`Foo::m` // for `Foo x;`), BEFORE the bare-name spray — the other half of the [TYPE] cut — read through class identity (resolve.h identityNarrow: // nested namesakes dropped, an inherited body, an interface's dispatch split); otherwise narrowed stays false and the name-based fallback @@ -3709,6 +5499,27 @@ inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = null && reachableByName( ing, it->second, r, reachScratch ) && !( r.role == RefRole::Call && r.qualifier.empty() && bindingTier.empty() && externalVeto.isExternalBound( r ) ); const rw::SmallVec* nameIds = localsYield ? &reachScratch : ( it != byName.end() ? &it->second : nullptr ); + // Ruby: Rule 3 and the ladder choose only among what self's method lookup can reach (RubySelfReach), on the side + // self stands on when that is known — a narrow to an unreachable namesake must not hide a reachable one. + const RubyCut rubyCut = !scipPinned && !canonical && !narrowed && nameIds != nullptr && rubySelf != nullptr + ? rubyClasses.reachableOnSide( rubyTop, *rubySelf, rubySide, *nameIds, rubyReachable ) + : RubyCut::Unchanged; + if( rubyCut == RubyCut::Declined ) + { + ++g.declinedOut[ r.fromSymbol ]; // in reach, but on the other side of self: no edge, and no claim it is outside + internDeclinedList( g, declinedListsByHash, std::span( rubyReachable.data(), rubyReachable.size() ) ); + disposition = CallDisposition::Declined; + continue; + } + if( rubyCut == RubyCut::Cut ) + { + if( rubyReachable.empty() ) + { + disposition = vetoExternal( r ); // every namesake is out of reach: the method Ruby runs is outside the tree + continue; + } + nameIds = &rubyReachable; + } if( !scipPinned && !canonical && !narrowed && it != byName.end() ) { if( narrower.rule3IncludeFile( *nameIds, r.fileId, rule3Out, localsYield ? 1u : 2u ) @@ -3794,6 +5605,27 @@ inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = null } } + // ---- Ruby: the same cut over the assembled set, for the paths above that re-read the whole name list ---------- + if( const RubyCut cut = !scipPinned && !canonical && !cand.empty() && rubySelf != nullptr + ? rubyClasses.reachableOnSide( rubyTop, *rubySelf, rubySide, cand, filtScratch ) + : RubyCut::Unchanged; + cut != RubyCut::Unchanged ) + { + cand.swap( filtScratch ); + if( cut == RubyCut::Declined ) + { + ++g.declinedOut[ r.fromSymbol ]; + internDeclinedList( g, declinedListsByHash, cand ); + disposition = CallDisposition::Declined; + continue; + } + if( cand.empty() ) + { + disposition = vetoExternal( r ); + continue; + } + } + // ---- namespace gate: drop any candidate whose KIND the reference's ROLE cannot mean -------------- // Applied ONCE over the assembled candidate set rather than `&&`-ed into each of the six admission // sites above: it is a stable in-place filter that preserves candidate order and can only remove, so @@ -3950,7 +5782,7 @@ inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = null // target; it drops only same-name methods of UNRELATED classes. Empty intersection ⇒ degrade. if( tier.size() > 1 && !identityClaim ) // a class-identity claim is type-verified; the cone cannot name `TBase` { - const std::string_view recvType = narrower.receiverStaticType( r, ing.symbols[ r.fromSymbol ].scope ); + const std::string_view recvType = narrower.receiverStaticType( r, callerSelfScope( ing, r ) ); if( !recvType.empty() ) { // The strict cone = {recvType} ∪ ANCESTORS ∪ DESCENDANTS, two fully-independent directional diff --git a/src/ingest_binds.h b/src/ingest_binds.h index ad7c987a5..a242c94a7 100644 --- a/src/ingest_binds.h +++ b/src/ingest_binds.h @@ -167,10 +167,13 @@ inline bool rubyDefinesDescribedClassMethod( std::string_view src ) noexcept return false; } -// How a Ruby node kind takes part in LOCAL scoping, for rubyDescribedClassIsLocal. A Closure (a block, a lambda) keeps -// its locals in but sees the enclosing ones; a Wall (a def, a class, a module) sees no outer local; the Binds* kinds -// bind a local — the `left:` child, the `name:` child (not a parameter's default value), or any child identifier. -enum class RubyLocalRole : std::uint8_t { Closure, Wall, BindsLeft, BindsName, BindsChildren }; +// How a Ruby node kind takes part in LOCAL scoping — Ruby's own parse-time rule, read by rubyDescribedClassIsLocal and by +// captureRubyBareCalls. A Closure (a block, a lambda) keeps its locals in but sees the enclosing ones; a Wall (a def, a +// class, a module) sees no outer local; the Binds* kinds bind a local — the `left:` child, the `name:` child (not a +// parameter's default value), any child identifier, the `pattern:` child (`for x in`, `in x`, `v => x`), a hash +// pattern's key (`in { x: }` binds x; `in { k: x }` binds x), or the named groups of a regex literal matched with `=~` +// (`/(?…)/ =~ s` — Ruby binds them only for a literal on the LEFT that holds no interpolation). +enum class RubyLocalRole : std::uint8_t { Closure, Wall, BindsLeft, BindsName, BindsChildren, BindsPattern, BindsKey, BindsRegexGroups }; struct RubyLocalKind { std::string_view kind; @@ -188,7 +191,11 @@ inline constexpr RubyLocalKind kRubyLocalKinds[] = { { "destructured_left_assignment", RubyLocalRole::BindsChildren }, { "rest_assignment", RubyLocalRole::BindsChildren }, { "destructured_parameter", RubyLocalRole::BindsChildren }, { "splat_parameter", RubyLocalRole::BindsChildren }, { "hash_splat_parameter", RubyLocalRole::BindsChildren }, { "block_parameter", RubyLocalRole::BindsChildren }, - { "exception_variable", RubyLocalRole::BindsChildren }, + { "exception_variable", RubyLocalRole::BindsChildren }, { "for", RubyLocalRole::BindsPattern }, + { "in_clause", RubyLocalRole::BindsPattern }, { "match_pattern", RubyLocalRole::BindsPattern }, + { "test_pattern", RubyLocalRole::BindsPattern }, { "array_pattern", RubyLocalRole::BindsChildren }, + { "find_pattern", RubyLocalRole::BindsChildren }, { "as_pattern", RubyLocalRole::BindsName }, + { "keyword_pattern", RubyLocalRole::BindsKey }, { "binary", RubyLocalRole::BindsRegexGroups }, }; inline std::optional rubyLocalRole( const char* t ) noexcept @@ -197,116 +204,1601 @@ inline std::optional rubyLocalRole( const char* t ) noexcept return it == std::end( kRubyLocalKinds ) ? std::nullopt : std::optional( it->role ); } +// The one field a BindsLeft / BindsName / BindsPattern node binds through, indexed by RubyLocalRole; Count for the rest. +inline constexpr NodeField kRubyBindingField[] = { NodeField::Count, NodeField::Count, NodeField::Left, NodeField::Name, + NodeField::Count, NodeField::Pattern, NodeField::Count, NodeField::Count }; +static_assert( std::size( kRubyBindingField ) == std::size_t( RubyLocalRole::BindsRegexGroups ) + 1, "one row per RubyLocalRole" ); + +// The `key:` of a hash pair or hash-pattern pair written with no value — `{ payload: }`, `in { x: }` — or null. Ruby +// reads that key as a name: the hash shorthand reads `payload`, the pattern binds `x`. +inline TSNode rubyShorthandKey( TSNode pair ) noexcept +{ + const TSNode key = fieldChild( pair, NodeField::Key ); + const bool bare = ts_node_is_null( fieldChild( pair, NodeField::Value ) ) && !ts_node_is_null( key ) && kindIs( ts_node_type( key ), "hash_key_symbol" ); + return bare ? key : TSNode {}; +} + +// `/(?…)/ =~ s` binds each named group `(?…)` as a local — Ruby's rule, for a regex LITERAL on the left of `=~` +// that holds no interpolation. Each name is handed to `onName`; `(?<=` / `(? +inline void rubyRegexMatchGroups( TSNode binary, std::string_view src, OnName&& onName ) +{ + const TSNode left = fieldChild( binary, NodeField::Left ); + if( ts_node_is_null( left ) || !kindIs( ts_node_type( left ), "regex" ) || nodeFieldText( binary, NodeField::Operator, src ) != "=~" ) + { + return; + } + for( std::uint32_t i = 0, cc = ts_node_named_child_count( left ); i < cc; ++i ) + { + if( kindIs( ts_node_type( ts_node_named_child( left, i ) ), "interpolation" ) ) + { + return; + } + } + const std::string_view re = pattern::nodeText( left, src ); + for( std::size_t at = re.find( "(?<" ); at != std::string_view::npos; at = re.find( "(?<", at + 3 ) ) + { + std::size_t end = at + 3; + while( end < re.size() && namesplit::isIdentChar( re[ end ] ) ) + { + ++end; + } + if( end > at + 3 && end < re.size() && re[ end ] == '>' ) + { + onName( re.substr( at + 3, end - at - 3 ) ); + } + } +} + +// Every local `n` (whose kind plays `role`) binds: `onNode( identifier )` for a binding spelled by an (identifier) child — +// that node is the binding site, never a read — and `onText( name )` for one spelled by no identifier node (a hash +// pattern's `key:` shorthand, a regex named group). The one reading of kRubyLocalKinds' Binds* roles. +template +inline void rubyForEachBinding( TSNode n, RubyLocalRole role, std::string_view src, OnNode&& onNode, OnText&& onText ) +{ + const auto bindIdentifier = [ & ]( TSNode c ) + { + if( !ts_node_is_null( c ) && kindIs( ts_node_type( c ), "identifier" ) ) + { + onNode( c ); + } + }; + if( const NodeField f = kRubyBindingField[ std::size_t( role ) ]; f != NodeField::Count ) + { + bindIdentifier( fieldChild( n, f ) ); + } + else if( role == RubyLocalRole::BindsChildren ) + { + for( std::uint32_t i = 0, cc = ts_node_named_child_count( n ); i < cc; ++i ) + { + bindIdentifier( ts_node_named_child( n, i ) ); + } + } + else if( role == RubyLocalRole::BindsKey ) + { + bindIdentifier( fieldChild( n, NodeField::Value ) ); // `in { k: x }` + if( const TSNode key = rubyShorthandKey( n ); !ts_node_is_null( key ) ) + { + onText( pattern::nodeText( key, src ) ); // `in { x: }` — the key is the local + } + } + else if( role == RubyLocalRole::BindsRegexGroups ) + { + rubyRegexMatchGroups( n, src, onText ); + } +} + // True when `n`, whose kind plays `role`, itself BINDS the local described_class. inline bool rubyNodeBindsDescribedClass( TSNode n, std::optional role, std::string_view src ) { - const auto named = [ & ]( TSNode c ) { return !ts_node_is_null( c ) && kindIs( ts_node_type( c ), "identifier" ) && pattern::nodeText( c, src ) == "described_class"; }; - if( role == RubyLocalRole::BindsLeft || role == RubyLocalRole::BindsName ) + if( !role ) + { + return false; + } + static constexpr std::string_view kName = "described_class"; + bool binds = false; + rubyForEachBinding( n, *role, src, [ & ]( TSNode c ) { binds = binds || pattern::nodeText( c, src ) == kName; }, + [ & ]( std::string_view name ) { binds = binds || name == kName; } ); + return binds; +} + +// True when a child of `n` that ENDS before byte `at` — an earlier statement, a block's or a def's parameters — binds +// described_class, searched without entering a nested block, lambda, def, class or module, whose locals never reach +// past it. `stack` is the caller's scratch, so a hostile nesting depth costs heap, not stack. +inline bool rubyEarlierChildBindsDescribedClass( TSNode n, std::uint32_t at, std::string_view src, std::vector& stack ) +{ + stack.clear(); + const std::uint32_t cc = ts_node_named_child_count( n ); + for( std::uint32_t i = 0; i < cc && ts_node_end_byte( ts_node_named_child( n, i ) ) <= at; ++i ) + { + stack.push_back( ts_node_named_child( n, i ) ); + } + while( !stack.empty() ) + { + const TSNode cur = stack.back(); + stack.pop_back(); + const std::optional role = rubyLocalRole( ts_node_type( cur ) ); + if( rubyNodeBindsDescribedClass( cur, role, src ) ) + { + return true; + } + if( role == RubyLocalRole::Closure || role == RubyLocalRole::Wall ) + { + continue; // a nested scope's locals never reach the site + } + const std::uint32_t kids = ts_node_named_child_count( cur ); + for( std::uint32_t i = 0; i < kids; ++i ) + { + stack.push_back( ts_node_named_child( cur, i ) ); + } + } + return false; +} + +// True when the (identifier) `site` reads a LOCAL named described_class, not RSpec's method — Ruby's own rule: a local +// is visible after its binding, in its own scope and in the blocks nested inside it. So the walk climbs from the site, +// searching each enclosing node's earlier children, and stops after the first def, class or module. An enclosing +// assignment counts too: in `described_class = described_class.m` the right side already reads the local. Floor (e). +inline bool rubyDescribedClassIsLocal( TSNode site, std::string_view src ) +{ + const std::uint32_t at = ts_node_start_byte( site ); + std::vector stack; + for( TSNode n = ts_node_parent( site ); !ts_node_is_null( n ); n = ts_node_parent( n ) ) + { + const std::optional role = rubyLocalRole( ts_node_type( n ) ); + if( rubyNodeBindsDescribedClass( n, role, src ) || rubyEarlierChildBindsDescribedClass( n, at, src, stack ) ) + { + return true; + } + if( role == RubyLocalRole::Wall ) + { + break; // a def, class or module: no outer local reaches in + } + } + return false; +} + +// RSpec's `described_class` — a bare (identifier) receiver no binding names — read as the constant it IS. RSpec's +// own rule (rspec-core 3.13, Metadata::ExampleGroupHash#described_class): a group's described class is its FIRST +// description argument unless that is nil or a String; otherwise it is the parent group's. So the walk climbs the +// enclosing calls whose BLOCK holds the site (the child it came from is a do_block/block), and the innermost example +// group with a constant first argument answers. A string or absent argument passes outward; any other argument +// (`describe :sym`, a variable) and a shared group stop the walk with no answer, and the site is left exactly as it +// was. test/rubydescribedclasscheck.sh. rspecDescribingGroup is the climb: the group that answers, or a null node. +inline TSNode rspecDescribingGroup( TSNode node, std::string_view src ) +{ + TSNode prev = node; + for( TSNode n = ts_node_parent( node ); !ts_node_is_null( n ); prev = n, n = ts_node_parent( n ) ) + { + const char* pt = ts_node_type( prev ); + if( !kindIs( ts_node_type( n ), "call" ) || !( kindIs( pt, "do_block" ) || kindIs( pt, "block" ) ) ) + { + continue; // not a call, or the site is in its receiver/arguments rather than its block + } + const int group = rspecGroupKind( n, src ); + if( group == 2 ) + { + return TSNode {}; // a shared group: its body runs in whichever group includes it + } + if( group == 1 && rspecGroupArgument( n, src ) ) + { + return n; + } + } + return TSNode {}; +} + +inline std::string_view rspecDescribedClass( TSNode node, std::string_view src ) +{ + const TSNode group = rspecDescribingGroup( node, src ); + return ts_node_is_null( group ) ? std::string_view {} : rspecGroupArgument( group, src ).value_or( std::string_view {} ); +} + +// The described class AS WRITTEN (parser version 137): the describing group's constant argument whole +// (`BankIntegration::Sync`, where rspecDescribedClass answers its final segment); empty when that answers none. +inline std::string_view rspecDescribedPath( TSNode node, std::string_view src ) +{ + const TSNode group = rspecDescribingGroup( node, src ); + const TSNode args = ts_node_is_null( group ) ? group : fieldChild( group, NodeField::Arguments ); + const TSNode first = ts_node_is_null( args ) ? args : ts_node_named_child( args, 0 ); + return !ts_node_is_null( first ) && rubyIsConstantChain( first ) ? pattern::nodeText( first, src ) : std::string_view {}; +} + +// ─── Ruby's BARE-WORD call (test/rubybarecallcheck.sh) ──────────────────────────────────────────────────────────────── +// `full_name`, `render_profile`, the `items` of `items.sum`: an identifier with no receiver, no arguments and no +// parentheses parses as a plain (identifier) — the node a local read is — so queries/ruby/tags.scm cannot capture it, +// and one Ruby call site in five minted no reference at all. Ruby's parser tells the two apart LEXICALLY, and +// captureRubyBareCalls applies that rule: an identifier is a local exactly when a binding of its name (kRubyLocalKinds' +// Binds* roles) precedes it in its own scope or in an enclosing Closure's, up to the nearest Wall; every other bare +// identifier is a method call on self (Prism: CallNode#variable_call?). It becomes a receiver-less call reference, the +// shape resolve.h's Rule 1 and its base walk already read as an implicit-self send. +// +// The bindings land when the walk reaches the BINDING node, before its children, so `x = x` reads the local on its +// right (Ruby's own reading) and a parameter's default value sees the parameters bound before it. A hash or keyword- +// argument shorthand (`{ payload: }`) reads `payload` and is a call when no local has that name. Inside a block or +// lambda with no parameter list, `it` and `_1`…`_9` are its implicit parameters (Ruby 3.4). +// +// RSpec: the let family DEFINES a method on its example group, visible to the whole group — before the line that +// defines it, in nested groups, and through a `def` inside the group (a method calling a method) — and every example +// group defines `subject`. Those names bind like locals over the group's block, stopped only by a class/module wall, so a +// spec's `user` never reaches an application `def user`. They are not minted as definitions: a let is reachable only +// from its group, and a definition application code could name would turn unique application calls into declines. +// +// Floors, pinned by the gate: `defined?( x )` calls nothing and is not walked; a shared context's lets are unknown in +// the including file; run-time locals (`binding.local_variable_set`, `eval`) are invisible; and inside +// `instance_eval`/`class_eval` blocks the call still resolves against the lexical class, as a parenthesised `m()` does. +inline constexpr std::string_view kRubyLetFamily[] = { "let", "let!", "subject", "subject!", "let_it_be", "let_it_be_with_reload", "let_it_be_with_refind" }; + +// The local names visible at the walk's position: one frame per open Ruby scope, each name a stack of the frames that +// bind it (deepest last), so a lookup is one hash probe whatever the scope's width. +struct RubyLocalScopes +{ + struct Frame + { + std::uint32_t endByte; // the scope node's end: the frame closes at the first node starting at or past it + std::uint32_t wall; // index of the innermost Wall frame — this one, when it is a Wall + std::uint32_t classWall; // index of the innermost class/module/`class <<`/file frame: where a let stops + std::uint32_t firstBound; // this frame's names in `bound` start here + bool implicitParams; // a block or lambda with no parameter list + }; + struct Entry { std::uint32_t depth; bool let; }; // one frame binding a name: a local, or a let + std::vector frames; + std::vector bound; + HashMap> names; + + void push( std::uint32_t endByte, bool wall, bool classLike, bool implicitParams ) + { + const std::uint32_t depth = std::uint32_t( frames.size() ); + const Frame outer = frames.empty() ? Frame {} : frames.back(); + frames.push_back( Frame { endByte, wall ? depth : outer.wall, classLike ? depth : outer.classWall, std::uint32_t( bound.size() ), implicitParams } ); + } + + // close every frame whose node ended at or before `atByte` — never the file's own + void closeEndedBy( std::uint32_t atByte ) + { + while( frames.size() > 1 && frames.back().endByte <= atByte ) + { + for( std::size_t i = frames.back().firstBound; i < bound.size(); ++i ) + { + names.find( bound[ i ] )->second.pop_back(); // a name's deepest entry is the innermost frame's + } + bound.resize( frames.back().firstBound ); + frames.pop_back(); + } + } + + void bind( std::string_view name, bool let ) + { + const std::uint32_t depth = std::uint32_t( frames.size() - 1 ); + rw::SmallVec& entries = names[ name ]; + if( entries.empty() || entries.back().depth != depth || entries.back().let != let ) + { + entries.push_back( Entry { depth, let } ); + bound.push_back( name ); + } + } + + // a local up to the innermost wall, a let up to the innermost class wall, or a block's implicit parameter + bool visible( std::string_view name ) const + { + const Frame& top = frames.back(); + if( const auto it = names.find( name ); it != names.end() ) + { + for( auto e = it->second.rbegin(); e != it->second.rend() && e->depth >= top.classWall; ++e ) + { + if( e->let || e->depth >= top.wall ) + { + return true; + } + } + } + const bool implicitName = name == "it" || ( name.size() == 2 && name[ 0 ] == '_' && name[ 1 ] >= '1' && name[ 1 ] <= '9' ); // `it`, `_1`…`_9` + return implicitName && std::any_of( frames.begin() + top.wall, frames.end(), []( const Frame& f ) { return f.implicitParams; } ); + } +}; + +// The first argument of a call when it is a plain symbol — `:user` → user — or empty. +inline std::string_view rubyFirstSymbolArgument( TSNode call, std::string_view src ) +{ + const TSNode args = fieldChild( call, NodeField::Arguments ); + const TSNode first = ts_node_is_null( args ) ? args : ts_node_named_child( args, 0 ); + if( ts_node_is_null( first ) || !kindIs( ts_node_type( first ), "simple_symbol" ) ) + { + return {}; + } + const std::string_view sym = pattern::nodeText( first, src ); + return sym.substr( sym.empty() ? 0 : 1 ); +} + +// The name one statement of an example group's body defines, or empty: a receiver-less let-family call, named by its +// first simple_symbol argument — a nameless `subject { … }` defines `subject`. +inline std::string_view rubyLetFamilyName( TSNode c, std::string_view src ) +{ + if( !kindIs( ts_node_type( c ), "call" ) || !ts_node_is_null( fieldChild( c, NodeField::Receiver ) ) ) + { + return {}; + } + const std::string_view m = fieldChildTextOfKind( c, NodeField::Method, "identifier", src ); + if( std::ranges::find( kRubyLetFamily, m ) == std::end( kRubyLetFamily ) ) + { + return {}; + } + const std::string_view name = rubyFirstSymbolArgument( c, src ); + return name.empty() && m.starts_with( "subject" ) ? std::string_view( "subject" ) : name; +} + +// The block a Ruby call is given — its last named child (tree-sitter-ruby's `block:` field) — or null. +inline TSNode rubyCallBlock( TSNode call ) +{ + const std::uint32_t cc = ts_node_named_child_count( call ); + const TSNode last = cc == 0 ? TSNode {} : ts_node_named_child( call, cc - 1 ); + return !ts_node_is_null( last ) && ( kindIs( ts_node_type( last ), "do_block" ) || kindIs( ts_node_type( last ), "block" ) ) ? last : TSNode {}; +} + +// The block of an RSpec example-group call, or null. +inline TSNode rspecGroupBlock( TSNode call, std::string_view src ) +{ + return rspecGroupKind( call, src ) == 0 ? TSNode {} : rubyCallBlock( call ); +} + +// Is `c` a child of `n` that NAMES something — a method, an alias — rather than reading a value? `call`'s `method:` is +// the tags query's capture already; a def's `name:` and `object:`, a setter's name and alias/undef operands read nothing. +inline bool rubyNamingChild( TSNode n, const char* t, TSNode c ) noexcept +{ + if( kindIs( t, "call" ) ) + { + return ts_node_eq( c, fieldChild( n, NodeField::Method ) ); + } + if( kindIs( t, "method" ) || kindIs( t, "singleton_method" ) ) + { + return ts_node_eq( c, fieldChild( n, NodeField::Name ) ) || ts_node_eq( c, fieldChild( n, NodeField::Object ) ); + } + return kindIs( t, "setter" ) || kindIs( t, "alias" ) || kindIs( t, "undef" ); +} + +// ─── Ruby's TYPED receiver (test/rubytypedrecvcheck.sh, parser version 132) ─────────────────────────────────────────── +// A receiver the code BUILDS is an instance of one class, and Ruby's method lookup on that class decides which `def` a call +// on it reaches (graph.h rubyTypedReceiver). What builds one is read from what the expression IS: `Const.new( … )` and the +// Active Record finders on a constant (kRubyTypedFinders — `User.find_by( … )` returns a User), `described_class.new` +// inside an example group that describes a constant, and a FactoryBot build — `create`/`build`/`build_stubbed( :user, … )`, +// receiver-less inside an example group or on `FactoryBot` anywhere — whose class only the tree's factory definitions +// know (captureRubyFactories). A call's receiver is typed when it is one of those written in place, a local whose every +// binding in its scope builds the same one, or the let/subject RSpec runs for that name there — the innermost definition, +// a nested group's override included — where a group with no subject of its own has its parent's, and at the top +// `described_class.new`. RubyBareCallWalk reads all of it on the walk it already makes, beside the locals it tracks, and +// typeReceivers writes each type into its call reference (model.h rubyTypedRecvToken). Floors (test/rubytypedrecvcheck.sh): +// control flow is not read (one binding of another shape anywhere in the scope, a parameter or a block parameter +// included, leaves the local untyped); instance variables, return values and a shared context's lets are not typed. +// RSpec's own builders are read the same way (test/rubyrspectargetcheck.sh, parser version 133): inside an example +// group a receiver-less `expect( v )`, `allow( v )`, `expect_any_instance_of( K )`, `allow_any_instance_of( K )` and a +// bare `is_expected` build one of RSpec's target classes (model.h kRspecTargets; `expect { … }` a BlockExpectationTarget), +// so `expect( v ).to` is a call on RSpec's object. Not read: outside a group (a helper module a `config.include` mixes +// in). A matcher builder (`receive`, `change`, …) is one of them too, and a call on a chain rooted at one — the `to` of +// `change { }.from( a ).to( b )`, `.and_return` after `receive( :m ).with( 1 )` — is typed as the root, through at most +// kRubyMatcherChainLinks links (rubyMatcherChainType, parser version 136). +inline constexpr std::string_view kRubyTypedFinders[] = { "create", "create!", "find", "find_by", "find_by!", "find_or_create_by", "find_or_create_by!", + "find_or_initialize_by", "find_sole_by", "first", "first!", "last", "last!", "new", "sole", + "take", "take!" }; +inline constexpr std::string_view kRubyFactoryBuilds[] = { "build", "build_stubbed", "create" }; +inline constexpr std::size_t kRubyMatcherChainLinks = 8; // a chain is short; a hostile one costs a bounded walk + +// What one value builds: a class's constant as written, or a factory name (`factory`), and the method that built it. Empty +// `type` = untyped. Views into the file's source. +struct RubyValueType +{ + std::string_view type; + std::string_view via; + bool factory = false; + bool operator==( const RubyValueType& ) const = default; +}; + +// Where a value is read: inside an example group's block (a receiver-less `create( :user )` is FactoryBot's there), and +// the class `described_class` names at the site — empty when no constant-described group encloses it or the file +// redefines the name. The walk keeps both per frame, so neither costs a climb up the tree. +struct RubyValueSite +{ + bool inGroup = false; + std::string_view described; +}; + +// What a FactoryBot build `v` to `m` builds — its factory, named by its first symbol argument — when `m` is a build +// (kRubyFactoryBuilds). +inline RubyValueType rubyFactoryBuildType( TSNode v, std::string_view m, std::string_view src ) +{ + const bool build = std::ranges::find( kRubyFactoryBuilds, m ) != std::end( kRubyFactoryBuilds ); + const std::string_view factory = build ? rubyFirstSymbolArgument( v, src ) : std::string_view {}; + return factory.empty() ? RubyValueType {} : RubyValueType { factory, m, true }; +} + +// What a receiver-less call `v` to `m` builds inside an example group: one of RSpec's targets (model.h kRspecTargets — +// `expect { … }`, a block and no argument, is the block form), else a FactoryBot build (`create( :user )`). A bare word +// (`is_expected`) is such a call with no arguments. +inline RubyValueType rubyGroupBuildType( TSNode v, std::string_view m, std::string_view src ) +{ + const auto target = std::ranges::find( kRspecTargets, m, &RspecTarget::builder ); + if( target == std::end( kRspecTargets ) ) + { + return rubyFactoryBuildType( v, m, src ); + } + const bool block = kindIs( ts_node_type( v ), "call" ) && ts_node_is_null( fieldChild( v, NodeField::Arguments ) ) && !ts_node_is_null( rubyCallBlock( v ) ); + return RubyValueType { m == "expect" && block ? kRspecBlockTarget : target->cls, m, false }; +} + +// What a matcher chain `v` inside an example group builds: the class of the RSpec builder at its root — the receiver-less +// call fewer than kRubyMatcherChainLinks receivers below `v`, so a call on `v` is at most that many links from it — whose +// every link is RSpec's; untyped for any other root (a FactoryBot build's methods return anything). +inline RubyValueType rubyMatcherChainType( TSNode v, std::string_view src ) +{ + TSNode link = v; + for( std::size_t depth = 0; depth < kRubyMatcherChainLinks && !ts_node_is_null( link ) && kindIs( ts_node_type( link ), "call" ); ++depth ) + { + const TSNode recv = fieldChild( link, NodeField::Receiver ); + if( ts_node_is_null( recv ) ) + { + const RubyValueType root = rubyGroupBuildType( link, fieldChildTextOfKind( link, NodeField::Method, "identifier", src ), src ); + return root.factory ? RubyValueType {} : root; + } + link = recv; + } + return {}; +} + +// What a call `v` to `m` on `recv` builds, read at `site`: a FactoryBot build on `FactoryBot`, a finder on any other +// constant — the constant AS WRITTEN (`OpenSSL::Cipher`): graph.h reads a qualified one only when the tree opens that +// path — `described_class.new`, or a link of a matcher chain in an example group. +inline RubyValueType rubyReceiverBuildType( TSNode v, TSNode recv, std::string_view m, std::string_view src, const RubyValueSite& site ) +{ + const std::string_view on = isRubyConstantNode( recv ) ? rubyFinalConstant( recv, src ) : std::string_view {}; + if( ( on == "FactoryBot" || on == "FactoryGirl" ) && std::ranges::find( kRubyFactoryBuilds, m ) != std::end( kRubyFactoryBuilds ) ) + { + return rubyFactoryBuildType( v, m, src ); + } + if( !on.empty() ) + { + const bool finder = std::ranges::find( kRubyTypedFinders, m ) != std::end( kRubyTypedFinders ); + return finder ? RubyValueType { pattern::nodeText( recv, src ), m, false } : RubyValueType {}; + } + const bool describedNew = m == "new" && kindIs( ts_node_type( recv ), "identifier" ) && pattern::nodeText( recv, src ) == "described_class"; + if( describedNew ) + { + return site.described.empty() ? RubyValueType {} : RubyValueType { site.described, m, false }; + } + return site.inGroup ? rubyMatcherChainType( v, src ) : RubyValueType {}; +} + +// What the value expression `v` builds (the section note above), read at `site`. +inline RubyValueType rubyValueType( TSNode v, std::string_view src, const RubyValueSite& site ) +{ + const bool word = !ts_node_is_null( v ) && kindIs( ts_node_type( v ), "identifier" ); // a bare `is_expected` + if( !word && ( ts_node_is_null( v ) || !kindIs( ts_node_type( v ), "call" ) ) ) + { + return {}; + } + const std::string_view m = word ? pattern::nodeText( v, src ) : fieldChildTextOfKind( v, NodeField::Method, "identifier", src ); + const TSNode recv = word ? TSNode {} : fieldChild( v, NodeField::Receiver ); + if( ts_node_is_null( recv ) ) + { + return site.inGroup ? rubyGroupBuildType( v, m, src ) : RubyValueType {}; + } + return rubyReceiverBuildType( v, recv, m, src, site ); +} + +// What a let/subject call's block returns — its body's last statement — or null. +inline TSNode rubyLetValue( TSNode letCall ) +{ + const TSNode block = rubyCallBlock( letCall ); + const TSNode body = ts_node_is_null( block ) ? block : fieldChild( block, NodeField::Body ); + TSNode last {}; + if( !ts_node_is_null( body ) ) + { + ChildCursor cursor( body ); // O(children): a body is as wide as its comments (src/infra/tschildren.h) + forEachNamedChild( body, cursor.cur, [ & ]( TSNode c ) { last = kindIs( ts_node_type( c ), "comment" ) ? last : c; return true; } ); + } + return last; +} + +// ─── Ruby's DECLARED calls (test/rubydeclrefcheck.sh, parser version 134) ───────────────────────────────────────────── +// A Rails declaration names, by SYMBOL, a method the framework calls later on the class's instances: `before_action +// :authenticate`, `after_save :reindex`, `validate :name_present`, an `if:` / `unless:` condition, `rescue_from …, with: +// :not_found`. Nothing in the source calls the method, so each symbol is read as a call to it — a receiver-less call from +// the class body, whose self's lookup is the class's (graph.h RubySelfReach) — when the macro sits at class-body position +// (receiver-less, the innermost def-or-class wall a class or module body: a concern's `included do` block counts, a def +// does not). `send`/`public_send`/`__send__`/`try`/`try!`/`method` with a literal symbol call that method on the same +// receiver: the call's own reference, renamed (RubyBareCallWalk::sendTargets). Floors (test/rubydeclrefcheck.sh): option +// values that are not calls (`only:`, `except:`, `on:`), `validates`' attribute names, `skip_*_action`, `set_callback`, a +// string or computed name, and a callback macro inside a def make none. +inline constexpr std::string_view kRubyCallbackMacros[] = { + // ActionController, ActionMailer, AbstractController + "before_action", "after_action", "around_action", "prepend_before_action", "prepend_after_action", "prepend_around_action", + "append_before_action", "append_after_action", "append_around_action", "before_filter", "after_filter", "around_filter", + // ActiveModel, ActiveRecord + "before_validation", "after_validation", "before_save", "around_save", "after_save", "before_create", "around_create", + "after_create", "before_update", "around_update", "after_update", "before_destroy", "around_destroy", "after_destroy", + "after_commit", "after_rollback", "after_create_commit", "after_update_commit", "after_destroy_commit", "after_save_commit", + "after_initialize", "after_find", "after_touch", "validate", + // ActiveJob, ActionMailer delivery, ActionCable channels + "before_enqueue", "around_enqueue", "after_enqueue", "before_perform", "around_perform", "after_perform", "after_discard", + "before_deliver", "around_deliver", "after_deliver", "before_subscribe", "after_subscribe", "before_unsubscribe", + "after_unsubscribe" }; +inline constexpr std::string_view kRubySendCalls[] = { "__send__", "method", "public_send", "send", "try", "try!" }; + +// What a declaration's symbols name: a callback's positional symbols and its conditions, only the conditions (a +// validation's positional symbols are attributes), or rescue_from's `with:` handler. +enum class RubyDeclared : std::uint8_t { None, Callback, Conditions, Rescue }; + +inline RubyDeclared rubyDeclaredKind( std::string_view m ) noexcept +{ + if( std::ranges::find( kRubyCallbackMacros, m ) != std::end( kRubyCallbackMacros ) ) + { + return RubyDeclared::Callback; + } + if( m == "validates" || m == "validates_with" || ( m.starts_with( "validates_" ) && m.ends_with( "_of" ) ) ) + { + return RubyDeclared::Conditions; + } + return m == "rescue_from" ? RubyDeclared::Rescue : RubyDeclared::None; +} + +// Is the option key `key` (`if:` or the older `:if =>`) one whose value a `kind` declaration calls? +inline bool rubyDeclaredOption( TSNode key, RubyDeclared kind, std::string_view src ) noexcept +{ + std::string_view k = ts_node_is_null( key ) ? std::string_view {} : pattern::nodeText( key, src ); + k = k.substr( k.starts_with( ':' ) ? 1 : 0 ); + k = k.substr( 0, k.size() - ( k.ends_with( ':' ) ? 1 : 0 ) ); + return k == "if" || k == "unless" || ( kind == RubyDeclared::Rescue && k == "with" ); +} + +// Each method symbol a `kind` declaration's arguments name, in source order (the section note above), handed to `fn`. +template +inline void rubyForEachDeclaredSymbol( TSNode call, RubyDeclared kind, std::string_view src, Fn&& fn ) +{ + const TSNode args = fieldChild( call, NodeField::Arguments ); + if( ts_node_is_null( args ) ) + { + return; + } + const auto symbol = []( TSNode n ) { return kindIs( ts_node_type( n ), "simple_symbol" ); }; + ChildCursor cursor( args ); // O(children) (src/infra/tschildren.h) + forEachNamedChild( args, cursor.cur, [ & ]( TSNode a ) + { + const TSNode value = kindIs( ts_node_type( a ), "pair" ) && rubyDeclaredOption( fieldChild( a, NodeField::Key ), kind, src ) + ? fieldChild( a, NodeField::Value ) : TSNode {}; + if( kind == RubyDeclared::Callback && symbol( a ) ) + { + fn( a ); + } + else if( !ts_node_is_null( value ) && symbol( value ) ) + { + fn( value ); + } + else if( !ts_node_is_null( value ) && kindIs( ts_node_type( value ), "array" ) ) + { + ChildCursor inner( value ); + forEachNamedChild( value, inner.cur, [ & ]( TSNode e ) { if( symbol( e ) ) { fn( e ); } return true; } ); + } + return true; + } ); +} + +// One call whose receiver is typed (RubyBareCallWalk): the call node's start (its reference's startByte) and method, +// whether the receiver is an (identifier) — the reference's NamedVar — or a value, and what it builds; for a local, the +// scope and name whose binding types are read once the walk has seen every binding (`local` non-empty, `type` filled by +// typeReceivers). +struct RubyTypedSite +{ + std::uint32_t callStart; + std::string_view method; + bool identReceiver; + RubyValueType type; + std::uint32_t wall; + std::string_view local; +}; + +// One `send`-family call naming its method by a literal symbol (RubyBareCallWalk::sendTargets): the call node's start +// and method, and the symbol. +struct RubySendSite +{ + std::uint32_t callStart; + std::string_view method; + TSNode symbol; + std::size_t ref = 0; // the call's reference, when exactly one matched (`hits` == 1) + std::uint32_t hits = 0; +}; + +// One file's bare-word calls (see the section note above): one iterative pre-order walk in source order — an explicit +// stack, no recursion, so a hostile nesting depth costs heap, not stack. +class RubyBareCallWalk +{ +public: + RubyBareCallWalk( std::uint32_t fileId, std::string_view src, std::vector& refs ) : fileId_( fileId ), src_( src ), refs_( refs ) + { + scopes_.names.reserve( 64 ); + stack_.reserve( 64 ); + kids_.reserve( 16 ); + } + void run( TSNode root ); + void typeReceivers( std::size_t firstRef, std::size_t endRef ); // after run: write each typed receiver into its call reference + void sendTargets( std::size_t firstRef, std::size_t endRef ); // after typeReceivers: each `send( :m )`'s call to m + +private: + struct Item + { + TSNode node; + bool naming; // an identifier in a naming or binding position: it reads nothing + bool groupBlock; // the block of an RSpec example-group call + }; + void visit( const Item& item, ChildCursor& cursor ); + void enter( TSNode n, const char* t, RubyLocalRole role, bool groupBlock ); + void bindGroupLets( TSNode block, bool groupBlock ); + void pushChildren( TSNode n, const char* t, ChildCursor& cursor ); + void emit( TSNode at, std::string_view name ); + void noteFrame( bool groupBlock, TSNode block ); + RubyValueSite valueSite() const; + void noteLocalTypes( TSNode n, RubyLocalRole role ); + void noteLetType( TSNode letCall, std::string_view name ); + void noteImplicitSubject(); + void noteTypedCall( TSNode call ); + void noteDeclaredCall( TSNode call ); + void noteNamedReceiver( std::uint32_t at, std::string_view method, std::string_view name ); + const RubyTypedSite* siteOf( const RawRef& r ) const; + const std::string& typeKey( std::uint32_t frame, std::string_view name ); + + std::uint32_t fileId_; + std::string_view src_; + std::vector& refs_; + RubyLocalScopes scopes_; + std::vector stack_; + std::vector kids_; + std::vector binding_; // the visited node's binding identifiers: its children marked `naming` + std::vector sendSites_; + + // typed receivers (the section note above): per open frame depth, a serial naming that frame and whether it is inside + // an example group; "#" → what a let defines in that group / what every binding of a local in that + // scope builds (a tombstone once two disagree); the group frames whose own subject is explicit; the typed call sites + std::vector frameSerial_; + std::vector frameInGroup_; + std::vector frameDescribed_; // the class described_class names in the frame (rspecDescribedClass's rule) + bool describedRedefined_ = false; // the file defines a method named described_class + std::uint32_t nextSerial_ = 0; + HashMap letTypes_; + HashMap localTypes_; + HashMap explicitSubjects_; + std::vector typedSites_; + std::string key_; +}; + +inline void RubyBareCallWalk::run( TSNode root ) +{ + ChildCursor cursor( root ); + scopes_.push( ts_node_end_byte( root ), true, true, false ); + describedRedefined_ = rubyDefinesDescribedClassMethod( src_ ); + noteFrame( false, root ); + stack_.push_back( Item { root, false, false } ); + while( !stack_.empty() ) + { + const Item item = stack_.back(); + stack_.pop_back(); + visit( item, cursor ); + } +} + +inline void RubyBareCallWalk::visit( const Item& item, ChildCursor& cursor ) +{ + const TSNode n = item.node; + const char* t = ts_node_type( n ); + scopes_.closeEndedBy( ts_node_start_byte( n ) ); + if( kindIs( t, "identifier" ) ) + { + if( !item.naming ) + { + emit( n, pattern::nodeText( n, src_ ) ); + } + return; + } + if( kindIs( t, "unary" ) && pattern::nodeText( n, src_ ).starts_with( "defined?" ) ) + { + return; // floor (a): defined?( x ) asks, and calls nothing + } + if( const TSNode key = kindIs( t, "pair" ) ? rubyShorthandKey( n ) : TSNode {}; !ts_node_is_null( key ) ) + { + emit( key, pattern::nodeText( key, src_ ) ); // `{ payload: }` reads payload + } + if( kindIs( t, "call" ) ) + { + noteTypedCall( n ); + noteDeclaredCall( n ); + } + binding_.clear(); + if( const std::optional role = rubyLocalRole( t ) ) + { + enter( n, t, *role, item.groupBlock ); + } + pushChildren( n, t, cursor ); +} + +// a scope node opens its frame (and an example group's block binds its lets); a binding node binds its names +inline void RubyBareCallWalk::enter( TSNode n, const char* t, RubyLocalRole role, bool groupBlock ) +{ + if( role == RubyLocalRole::Wall || role == RubyLocalRole::Closure ) + { + const bool closure = role == RubyLocalRole::Closure; + scopes_.push( ts_node_end_byte( n ), !closure, kindIs( t, "class" ) || kindIs( t, "module" ) || kindIs( t, "singleton_class" ), + closure && ts_node_is_null( fieldChild( n, NodeField::Parameters ) ) ); + noteFrame( groupBlock, n ); + if( closure && !kindIs( t, "lambda" ) ) + { + bindGroupLets( n, groupBlock ); + } + return; + } + rubyForEachBinding( n, role, src_, [ & ]( TSNode c ) { binding_.push_back( c ); scopes_.bind( pattern::nodeText( c, src_ ), false ); }, + [ & ]( std::string_view name ) { scopes_.bind( name, false ); } ); + noteLocalTypes( n, role ); +} + +// every let the block's statements define, and `subject` when the block is an example group's +inline void RubyBareCallWalk::bindGroupLets( TSNode block, bool groupBlock ) +{ + if( groupBlock ) + { + scopes_.bind( "subject", true ); + noteImplicitSubject(); + } + const TSNode body = fieldChild( block, NodeField::Body ); + for( std::uint32_t i = 0, cc = ts_node_is_null( body ) ? 0 : ts_node_named_child_count( body ); i < cc; ++i ) + { + if( const std::string_view name = rubyLetFamilyName( ts_node_named_child( body, i ), src_ ); !name.empty() ) + { + scopes_.bind( name, true ); + noteLetType( ts_node_named_child( body, i ), name ); + } + } +} + +inline void RubyBareCallWalk::pushChildren( TSNode n, const char* t, ChildCursor& cursor ) +{ + const TSNode groupBlock = kindIs( t, "call" ) ? rspecGroupBlock( n, src_ ) : TSNode {}; + collectChildren( n, cursor.cur, kids_ ); + for( std::size_t i = kids_.size(); i > 0; --i ) + { + const TSNode c = kids_[ i - 1 ]; + const bool naming = rubyNamingChild( n, t, c ) || std::ranges::any_of( binding_, [ & ]( TSNode b ) { return ts_node_eq( b, c ); } ); + stack_.push_back( Item { c, naming, !ts_node_is_null( groupBlock ) && ts_node_eq( c, groupBlock ) } ); + } +} + +// receiver-less: RecvKind::None, the implicit-self shape Rule 1 reads +inline void RubyBareCallWalk::emit( TSNode at, std::string_view name ) +{ + if( !name.empty() && !scopes_.visible( name ) ) + { + refs_.push_back( rawRefAt( at, fileId_, Lang::Ruby, RefRole::Call, std::string( name ) ) ); + } +} + +// a new frame at the top of scopes_: its serial, whether it sits in an example group, and the class described_class names +// in it — rspecDescribedClass's rule, kept per frame: an example group's block takes its group's constant argument, a +// String or absent one passes the parent's through, any other argument and a shared group name none; any other frame +// inherits its parent's +inline void RubyBareCallWalk::noteFrame( bool groupBlock, TSNode block ) +{ + const std::size_t depth = scopes_.frames.size() - 1; + if( frameSerial_.size() <= depth ) + { + frameSerial_.resize( depth + 1 ); + frameInGroup_.resize( depth + 1 ); + frameDescribed_.resize( depth + 1 ); + } + const std::string_view parent = depth > 0 ? frameDescribed_[ depth - 1 ] : std::string_view {}; + const TSNode group = groupBlock ? ts_node_parent( block ) : TSNode {}; + const int kind = ts_node_is_null( group ) ? 0 : rspecGroupKind( group, src_ ); + const std::optional argument = kind == 1 ? rspecGroupArgument( group, src_ ) : std::nullopt; + frameSerial_[ depth ] = nextSerial_++; + frameInGroup_[ depth ] = char( groupBlock || ( depth > 0 && frameInGroup_[ depth - 1 ] != 0 ) ); + frameDescribed_[ depth ] = kind == 2 ? std::string_view {} : argument ? *argument : parent; +} + +// the value site at the walk's position: its group, and what described_class names there unless the file or a visible +// local or let redefines it +inline RubyValueSite RubyBareCallWalk::valueSite() const +{ + const std::size_t top = scopes_.frames.size() - 1; + const bool redefined = describedRedefined_ || scopes_.visible( "described_class" ); + return RubyValueSite { frameInGroup_[ top ] != 0, redefined ? std::string_view {} : frameDescribed_[ top ] }; +} + +inline const std::string& RubyBareCallWalk::typeKey( std::uint32_t frame, std::string_view name ) +{ + key_.clear(); + key_.append( std::to_string( frame ) ).push_back( '#' ); + key_.append( name ); + return key_; +} + +// each local `n` binds, folded into its scope's type: what an assignment (or `||=`) builds; nothing, for every other binding +inline void RubyBareCallWalk::noteLocalTypes( TSNode n, RubyLocalRole role ) +{ + const std::uint32_t wall = frameSerial_[ scopes_.frames.back().wall ]; + const TSNode left = role == RubyLocalRole::BindsLeft ? fieldChild( n, NodeField::Left ) : TSNode {}; + const bool assigns = !ts_node_is_null( left ) && kindIs( ts_node_type( left ), "identifier" ) + && ( kindIs( ts_node_type( n ), "assignment" ) || nodeFieldText( n, NodeField::Operator, src_ ) == "||=" ); + const RubyValueType built = assigns ? rubyValueType( fieldChild( n, NodeField::Right ), src_, valueSite() ) : RubyValueType {}; + const auto fold = [ & ]( std::string_view name, const RubyValueType& type ) + { + const auto [ it, inserted ] = localTypes_.try_emplace( typeKey( wall, name ), type ); + if( !inserted && !( it->second == type ) ) + { + it->second = {}; // two bindings build different things: untyped in the whole scope + } + }; + rubyForEachBinding( n, role, src_, [ & ]( TSNode c ) { fold( pattern::nodeText( c, src_ ), assigns && ts_node_eq( c, left ) ? built : RubyValueType {} ); }, + [ & ]( std::string_view name ) { fold( name, {} ); } ); +} + +// a let/subject statement of the group whose frame is on top: what its block builds; a named subject is the group's subject too +inline void RubyBareCallWalk::noteLetType( TSNode letCall, std::string_view name ) +{ + const std::uint32_t group = frameSerial_[ scopes_.frames.size() - 1 ]; + const RubyValueType built = rubyValueType( rubyLetValue( letCall ), src_, valueSite() ); + letTypes_[ typeKey( group, name ) ] = built; + if( fieldChildTextOfKind( letCall, NodeField::Method, "identifier", src_ ).starts_with( "subject" ) ) + { + letTypes_[ typeKey( group, "subject" ) ] = built; + explicitSubjects_[ group ] = built; + } +} + +// an example group's implicit subject: the nearest enclosing group's explicit one — RSpec looks it up through the parent +// groups — else `described_class.new` +inline void RubyBareCallWalk::noteImplicitSubject() +{ + const std::size_t top = scopes_.frames.size() - 1; + RubyValueType built = {}; + bool found = false; + for( std::size_t d = top; d > scopes_.frames[ top ].classWall && !found; --d ) + { + const auto it = explicitSubjects_.find( frameSerial_[ d - 1 ] ); + found = it != explicitSubjects_.end(); + built = found ? it->second : built; + } + const std::string_view described = found ? std::string_view {} : valueSite().described; + letTypes_[ typeKey( frameSerial_[ top ], "subject" ) ] = described.empty() ? built : RubyValueType { described, "new", false }; +} + +// a call whose receiver is typed: a value written in place, a let RSpec runs there, or a local of the scope (read at the end) +inline void RubyBareCallWalk::noteTypedCall( TSNode call ) +{ + const TSNode recv = fieldChild( call, NodeField::Receiver ); + const std::string_view method = fieldChildTextOfKind( call, NodeField::Method, "identifier", src_ ); + if( ts_node_is_null( recv ) || method.empty() ) + { + return; + } + const std::uint32_t at = ts_node_start_byte( call ); + const bool ident = kindIs( ts_node_type( recv ), "identifier" ); + if( !ident || !scopes_.visible( pattern::nodeText( recv, src_ ) ) ) + { + // a value — or a bare word no local or let binds, which is a call on self (`is_expected`) + const RubyValueType built = rubyValueType( recv, src_, valueSite() ); + if( !built.type.empty() ) + { + typedSites_.push_back( RubyTypedSite { at, method, ident, built, 0, {} } ); + } + return; + } + noteNamedReceiver( at, method, pattern::nodeText( recv, src_ ) ); +} + +// a call whose receiver is the bare name `name`: the innermost definition of it visible here — a let, or a local of this +// scope (whose type is read at the end, once every binding is seen) — decides, typed or not +inline void RubyBareCallWalk::noteNamedReceiver( std::uint32_t at, std::string_view method, std::string_view name ) +{ + const auto it = scopes_.names.find( name ); + if( it == scopes_.names.end() ) + { + return; // no binding: a call on self, which no let or local types + } + const RubyLocalScopes::Frame& top = scopes_.frames.back(); + const auto visible = std::find_if( it->second.rbegin(), it->second.rend(), + [ & ]( const RubyLocalScopes::Entry& e ) { return e.depth < top.classWall || e.let || e.depth >= top.wall; } ); + if( visible == it->second.rend() || visible->depth < top.classWall ) + { + return; + } + const bool local = !visible->let; + const std::uint32_t frame = frameSerial_[ local ? top.wall : visible->depth ]; + const auto& table = local ? localTypes_ : letTypes_; + const auto found = table.find( typeKey( frame, name ) ); + if( found != table.end() && !found->second.type.empty() ) + { + typedSites_.push_back( RubyTypedSite { at, method, true, found->second, frame, local ? name : std::string_view {} } ); + } +} + +// Each typed site's local read now that every binding is seen, then each written into the call reference the tags query +// made for it — the one in [firstRef, endRef) at the call's start, with its method, whose receiver kind is the site's +// (an identifier receiver is NamedVar, a value receiver FieldOfVar). +inline void RubyBareCallWalk::typeReceivers( std::size_t firstRef, std::size_t endRef ) +{ + for( RubyTypedSite& site : typedSites_ ) + { + if( !site.local.empty() ) + { + const auto it = localTypes_.find( typeKey( site.wall, site.local ) ); + site.type = it == localTypes_.end() ? RubyValueType {} : it->second; + } + } + std::erase_if( typedSites_, []( const RubyTypedSite& site ) { return site.type.type.empty(); } ); + std::stable_sort( typedSites_.begin(), typedSites_.end(), []( const RubyTypedSite& a, const RubyTypedSite& b ) { return a.callStart < b.callStart; } ); + for( std::size_t i = firstRef; i < endRef && i < refs_.size() && !typedSites_.empty(); ++i ) + { + if( const RubyTypedSite* site = siteOf( refs_[ i ] ) ) + { + refs_[ i ].recvVar = rubyTypedRecvToken( site->type.type, site->type.via, site->type.factory ); + } + } +} + +// A declaration's method symbols (the section note above): a callback macro's at class-body position, each a call +// reference at its symbol; a `send`-family call's symbol, recorded for sendTargets +inline void RubyBareCallWalk::noteDeclaredCall( TSNode call ) +{ + const std::string_view m = fieldChildTextOfKind( call, NodeField::Method, "identifier", src_ ); + const bool bare = ts_node_is_null( fieldChild( call, NodeField::Receiver ) ); + const RubyLocalScopes::Frame& top = scopes_.frames.back(); + const RubyDeclared kind = bare && top.wall == top.classWall ? rubyDeclaredKind( m ) : RubyDeclared::None; + if( kind != RubyDeclared::None ) + { + rubyForEachDeclaredSymbol( call, kind, src_, [ & ]( TSNode sym ) + { + const std::string_view text = pattern::nodeText( sym, src_ ); + refs_.push_back( rawRefAt( sym, fileId_, Lang::Ruby, RefRole::Call, std::string( text.substr( text.empty() ? 0 : 1 ) ) ) ); + } ); + return; + } + if( !m.empty() && std::ranges::find( kRubySendCalls, m ) != std::end( kRubySendCalls ) ) + { + const TSNode args = fieldChild( call, NodeField::Arguments ); + const TSNode first = ts_node_is_null( args ) ? args : ts_node_named_child( args, 0 ); + if( !ts_node_is_null( first ) && kindIs( ts_node_type( first ), "simple_symbol" ) ) + { + sendSites_.push_back( RubySendSite { ts_node_start_byte( call ), m, first } ); + } + } +} + +// Each `send`-family call's named method, as a call on the same receiver: a copy of the call's own reference — the one +// in [firstRef, endRef) at the call's start that names it, typed receiver and all — renamed and placed at the symbol. Two +// same-named calls starting at one byte (`a.send( :x ).send( :y )`) are ambiguous, and neither is copied. +inline void RubyBareCallWalk::sendTargets( std::size_t firstRef, std::size_t endRef ) +{ + std::stable_sort( sendSites_.begin(), sendSites_.end(), []( const RubySendSite& a, const RubySendSite& b ) { return a.callStart < b.callStart; } ); + const auto byStart = []( const RubySendSite& site, std::uint32_t at ) { return site.callStart < at; }; + for( std::size_t i = firstRef; i < endRef && i < refs_.size() && !sendSites_.empty(); ++i ) + { + const RawRef& r = refs_[ i ]; + for( auto site = std::lower_bound( sendSites_.begin(), sendSites_.end(), r.startByte, byStart ); + r.role == RefRole::Call && site != sendSites_.end() && site->callStart == r.startByte; ++site ) + { + site->ref = site->method == r.name ? i : site->ref; + site->hits += site->method == r.name ? 1u : 0u; + } + } + for( const RubySendSite& site : sendSites_ ) + { + if( site.hits == 1 ) + { + RawRef copy = refs_[ site.ref ]; + const std::string_view text = pattern::nodeText( site.symbol, src_ ); + copy.name.assign( text.substr( text.empty() ? 0 : 1 ) ); + copy.startByte = ts_node_start_byte( site.symbol ); + copy.line = ts_node_start_point( site.symbol ).row + 1; + refs_.push_back( std::move( copy ) ); + } + } +} + +// the typed site a call reference was made for: the one at its start whose method it names (a setter `x.name =` names +// `name=`) and whose receiver kind it records — an (identifier) receiver is NamedVar, a value receiver FieldOfVar +inline const RubyTypedSite* RubyBareCallWalk::siteOf( const RawRef& r ) const +{ + const bool ident = r.recv == RecvKind::NamedVar; + if( r.lang != Lang::Ruby || r.role != RefRole::Call || ( !ident && r.recv != RecvKind::FieldOfVar ) ) + { + return nullptr; + } + const std::string_view name = r.name; + const std::string_view bare = name.ends_with( '=' ) ? name.substr( 0, name.size() - 1 ) : name; + const auto byStart = []( const RubyTypedSite& site, std::uint32_t at ) { return site.callStart < at; }; + for( auto site = std::lower_bound( typedSites_.begin(), typedSites_.end(), r.startByte, byStart ); + site != typedSites_.end() && site->callStart == r.startByte; ++site ) + { + if( site->identReceiver == ident && ( site->method == name || site->method == bare ) ) + { + return &*site; + } + } + return nullptr; +} + +// One file's bare-word calls, and the type of each call receiver the file builds — written into the file's references +// already in `refs` (the tags query's: the trailing run carrying this fileId); the walk's own references follow them. +inline void captureRubyBareCalls( TSNode root, std::uint32_t fileId, std::string_view src, std::vector& refs ) +{ + const std::size_t queryRefsEnd = refs.size(); + std::size_t firstRef = queryRefsEnd; + while( firstRef > 0 && refs[ firstRef - 1 ].fileId == fileId ) + { + --firstRef; + } + RubyBareCallWalk walk( fileId, src, refs ); + walk.run( root ); + walk.typeReceivers( firstRef, queryRefsEnd ); + walk.sendTargets( firstRef, queryRefsEnd ); +} + +// ─── A Jbuilder template's `json`, and `helper_method` (test/rubyrakejbuildercheck.sh, parser version 135) ───────────── +// graph.h RubyTopSelf reads a template's self outside any class as the view. The template handler binds `json` there to the +// view's JbuilderTemplate (model.h kJbuilderTemplate), whose method_missing makes every name a key: a read of it is no +// call, and a call on it is typed that class. A `helper_method :a, :b` at class-body position names methods of the class +// its views call, recorded as LocalBindKind::RubyHelperMethod bindings against the declaring class or module (a concern's +// `included do` block declares for the concern, whose def it is). + +// Over a template's references — the trailing run carrying this fileId: the bare-word `json` reads dropped, and each call +// on `json` typed JbuilderTemplate. +inline void typeRubyJbuilderLocal( std::uint32_t fileId, std::vector& refs ) +{ + std::size_t first = refs.size(); + while( first > 0 && refs[ first - 1 ].fileId == fileId ) { - return named( fieldChild( n, role == RubyLocalRole::BindsLeft ? NodeField::Left : NodeField::Name ) ); + --first; } - if( role != RubyLocalRole::BindsChildren ) + const auto read = []( const RawRef& r ) { return r.role == RefRole::Call && r.recv == RecvKind::None && r.name == kJbuilderLocal; }; + refs.erase( std::remove_if( refs.begin() + std::ptrdiff_t( first ), refs.end(), read ), refs.end() ); + for( std::size_t i = first; i < refs.size(); ++i ) { - return false; + if( refs[ i ].role == RefRole::Call && refs[ i ].recv == RecvKind::NamedVar && refs[ i ].recvVar == kJbuilderLocal ) + { + refs[ i ].recvVar = rubyTypedRecvToken( kJbuilderTemplate, kJbuilderLocal, false ); + } } - const std::uint32_t cc = ts_node_named_child_count( n ); - for( std::uint32_t i = 0; i < cc; ++i ) +} + +// Does `call` sit at class-body position — its nearest enclosing def-or-class a class or module, any blocks between? +inline bool rubyAtClassBody( TSNode call ) noexcept +{ + for( TSNode p = ts_node_parent( call ); !ts_node_is_null( p ); p = ts_node_parent( p ) ) { - if( named( ts_node_named_child( n, i ) ) ) + const char* t = ts_node_type( p ); + if( kindIs( t, "class" ) || kindIs( t, "module" ) ) { return true; } + if( kindIs( t, "method" ) || kindIs( t, "singleton_method" ) || kindIs( t, "singleton_class" ) ) + { + return false; + } } return false; } -// True when a child of `n` that ENDS before byte `at` — an earlier statement, a block's or a def's parameters — binds -// described_class, searched without entering a nested block, lambda, def, class or module, whose locals never reach -// past it. `stack` is the caller's scratch, so a hostile nesting depth costs heap, not stack. -inline bool rubyEarlierChildBindsDescribedClass( TSNode n, std::uint32_t at, std::string_view src, std::vector& stack ) +// One binding per symbol argument in `args` (`helper_method :a, :b`, `attr_accessor :c`): `like` with the symbol's own start +// byte and its name (the colon dropped). +inline void rubyNoteSymbolArguments( TSNode args, const RawBind& like, std::string_view src, std::vector& binds ) { - stack.clear(); - const std::uint32_t cc = ts_node_named_child_count( n ); - for( std::uint32_t i = 0; i < cc && ts_node_end_byte( ts_node_named_child( n, i ) ) <= at; ++i ) + if( ts_node_is_null( args ) ) { - stack.push_back( ts_node_named_child( n, i ) ); + return; + } + ChildCursor cursor( args ); // O(children) (src/infra/tschildren.h) + forEachNamedChild( args, cursor.cur, [ & ]( TSNode a ) + { + if( kindIs( ts_node_type( a ), "simple_symbol" ) ) + { + RawBind b = like; + b.startByte = ts_node_start_byte( a ); + b.var = std::string( pattern::nodeText( a, src ).substr( 1 ) ); + binds.push_back( std::move( b ) ); + } + return true; + } ); +} + +// Each symbol of the `helper_method` call `call`, a binding against the class or module around it. +inline void rubyNoteHelperMethods( TSNode call, std::uint32_t fileId, std::string_view src, std::vector& binds ) +{ + const std::string owner = rubyEnclosingScopeOf( call, src ); + if( !owner.empty() ) + { + rubyNoteSymbolArguments( fieldChild( call, NodeField::Arguments ), + RawBind { .fileId = fileId, .lang = Lang::Ruby, .kind = LocalBindKind::RubyHelperMethod, .typeName = owner }, src, binds ); } +} + +// Every node of a Ruby file's tree, handed to `take` in turn; a node `take` returns true for is taken, and its subtree is +// not walked. Iterative, so a hostile nesting depth costs heap, not stack. +template +inline void rubyWalkUntaken( TSNode root, Take&& take ) +{ + std::vector stack { root }; + std::vector kids; + ChildCursor cursor( root ); while( !stack.empty() ) { - const TSNode cur = stack.back(); + const TSNode n = stack.back(); stack.pop_back(); - const std::optional role = rubyLocalRole( ts_node_type( cur ) ); - if( rubyNodeBindsDescribedClass( cur, role, src ) ) + if( !take( n ) ) + { + collectChildren( n, cursor.cur, kids ); + stack.insert( stack.end(), kids.begin(), kids.end() ); + } + } +} + +// One file's `helper_method` declarations, into `binds` — a file that never spells the word is not walked (rubyWalkUntaken). +inline void captureRubyHelperMethods( TSNode root, std::uint32_t fileId, std::string_view src, std::vector& binds ) +{ + if( src.find( "helper_method" ) == std::string_view::npos ) + { + return; + } + rubyWalkUntaken( root, [ & ]( TSNode n ) + { + const bool declares = kindIs( ts_node_type( n ), "call" ) && ts_node_is_null( fieldChild( n, NodeField::Receiver ) ) + && fieldChildTextOfKind( n, NodeField::Method, "identifier", src ) == "helper_method"; + if( declares && rubyAtClassBody( n ) ) { + rubyNoteHelperMethods( n, fileId, src, binds ); return true; } - if( role == RubyLocalRole::Closure || role == RubyLocalRole::Wall ) + return false; + } ); +} + +// Parser version 137 (test/rubyclassrecvcheck.sh): is the Ruby def `defNode` a SINGLETON method — `def self.m`, or a def +// inside `class << self` — which a call on a class object answers and no instance's lookup reaches? The nearest class, +// module or def above a plain `def` decides; a block between them (`class_methods do`, `included do`) is walked through. +// rubyDefSide below tells whose class object; captureRubyClassMixins reads a directive in `class << self` with this. +inline bool rubyIsSingletonDef( TSNode defNode ) noexcept +{ + if( kindIs( ts_node_type( defNode ), "singleton_method" ) ) + { + const TSNode object = fieldChild( defNode, NodeField::Object ); + return !ts_node_is_null( object ) && kindIs( ts_node_type( object ), "self" ); + } + for( TSNode n = ts_node_parent( defNode ); !ts_node_is_null( n ); n = ts_node_parent( n ) ) + { + const char* t = ts_node_type( n ); + if( kindIs( t, "singleton_class" ) ) { - continue; // a nested scope's locals never reach the site + const TSNode value = fieldChild( n, NodeField::Value ); + return !ts_node_is_null( value ) && kindIs( ts_node_type( value ), "self" ); } - const std::uint32_t kids = ts_node_named_child_count( cur ); - for( std::uint32_t i = 0; i < kids; ++i ) + if( kindIs( t, "class" ) || kindIs( t, "module" ) || kindIs( t, "method" ) || kindIs( t, "singleton_method" ) ) { - stack.push_back( ts_node_named_child( cur, i ) ); + return false; } } return false; } -// True when the (identifier) `site` reads a LOCAL named described_class, not RSpec's method — Ruby's own rule: a local -// is visible after its binding, in its own scope and in the blocks nested inside it. So the walk climbs from the site, -// searching each enclosing node's earlier children, and stops after the first def, class or module. An enclosing -// assignment counts too: in `described_class = described_class.m` the right side already reads the local. Floor (e). -inline bool rubyDescribedClassIsLocal( TSNode site, std::string_view src ) +// What a Ruby def is to a class object (parser versions 137 and 145; test/rubyclassrecvcheck.sh). +enum class RubyDefSide : std::uint8_t { - const std::uint32_t at = ts_node_start_byte( site ); - std::vector stack; - for( TSNode n = ts_node_parent( site ); !ts_node_is_null( n ); n = ts_node_parent( n ) ) + Instance, // an instance method: no call on a class object reaches it + Singleton, // a singleton method of the class or module that holds it — `def self.m`, a def or an accessor in `class << self` + Includer, // a class method of each class INCLUDING its module, defined on the includer itself: a singleton def inside a + // concern's `included do`, which runs on the includer + ClassMethods, // a class method of each includer from a module extended onto it: a def inside a concern's `class_methods + // do`, which ActiveSupport::Concern makes the concern's ClassMethods +}; + +// The two blocks of an ActiveSupport::Concern whose defs can be its includer's class methods (RubyDefSide::Includer, +// ClassMethods). +inline constexpr std::array kRubyIncluderBlocks = { "included", "class_methods" }; + +// The concern hook (kRubyIncluderBlocks) whose block `n` is, written at class-body level, or empty. +inline std::string_view rubyIncluderHook( TSNode n, std::string_view src ) noexcept +{ + const TSNode owner = kindIs( ts_node_type( n ), "do_block" ) || kindIs( ts_node_type( n ), "block" ) ? ts_node_parent( n ) : TSNode {}; + if( ts_node_is_null( owner ) || !kindIs( ts_node_type( owner ), "call" ) ) { - const std::optional role = rubyLocalRole( ts_node_type( n ) ); - if( rubyNodeBindsDescribedClass( n, role, src ) || rubyEarlierChildBindsDescribedClass( n, at, src, stack ) ) + return {}; + } + const std::string_view hook = rubyNamedDirective( owner, src, kRubyIncluderBlocks ); + return !hook.empty() && rubyAttrAtClassBodyLevel( owner, src ) ? hook : std::string_view {}; +} + +// What a def written in a concern's `hook` block is: in `included do`, a singleton def is the includer's own and a plain +// one an instance method; in `class_methods do`, a plain def is the includer's class method. +inline RubyDefSide rubyHookSide( std::string_view hook, bool singleton ) noexcept +{ + if( hook == "included" ) + { + return singleton ? RubyDefSide::Includer : RubyDefSide::Instance; + } + return singleton ? RubyDefSide::Instance : RubyDefSide::ClassMethods; +} + +// What the Ruby def — or `class << self` accessor — at `defNode` is to a class object. A `def self.m`, or a `class << self` +// above it, makes it a singleton method, and the nearest class, module or def above decides whose — unless a concern's +// `included do` (for a singleton def) or `class_methods do` (for a plain one) at class-body level comes first, which makes +// it the includer's: defined on the includer itself, or in the module the concern extends onto it. Any other block between +// is walked through. +inline RubyDefSide rubyDefSide( TSNode defNode, std::string_view src ) noexcept +{ + const bool selfDef = kindIs( ts_node_type( defNode ), "singleton_method" ); + if( selfDef && fieldChildTextOfKind( defNode, NodeField::Object, "self", src ).empty() ) + { + return RubyDefSide::Instance; // `def obj.m` is another object's + } + bool singleton = selfDef; + for( TSNode n = ts_node_parent( defNode ); !ts_node_is_null( n ); n = ts_node_parent( n ) ) + { + const char* t = ts_node_type( n ); + if( kindIs( t, "singleton_class" ) ) { - return true; + if( fieldChildTextOfKind( n, NodeField::Value, "self", src ).empty() ) + { + return RubyDefSide::Instance; // `class << Other` opens another object's singleton + } + singleton = true; + continue; } - if( role == RubyLocalRole::Wall ) + if( kindIs( t, "class" ) || kindIs( t, "module" ) || kindIs( t, "method" ) || kindIs( t, "singleton_method" ) ) { - break; // a def, class or module: no outer local reaches in + break; + } + if( const std::string_view hook = rubyIncluderHook( n, src ); !hook.empty() ) + { + return rubyHookSide( hook, singleton ); } } - return false; + return singleton ? RubyDefSide::Singleton : RubyDefSide::Instance; } -// RSpec's `described_class` — a bare (identifier) receiver no binding names — read as the constant it IS. RSpec's -// own rule (rspec-core 3.13, Metadata::ExampleGroupHash#described_class): a group's described class is its FIRST -// description argument unless that is nil or a String; otherwise it is the parent group's. So the walk climbs the -// enclosing calls whose BLOCK holds the site (the child it came from is a do_block/block), and the innermost example -// group with a constant first argument answers. A string or absent argument passes outward; any other argument -// (`describe :sym`, a variable) and a shared group stop the walk with no answer, and the site is left exactly as it -// was. test/rubydescribedclasscheck.sh. -inline std::string_view rspecDescribedClass( TSNode node, std::string_view src ) +// The importedName a Ruby class-object binding carries for a def on `side` (model.h LocalBindKind::RubySingletonDef). +inline std::string rubySideMark( RubyDefSide side ) { - TSNode prev = node; - for( TSNode n = ts_node_parent( node ); !ts_node_is_null( n ); prev = n, n = ts_node_parent( n ) ) + return std::string( side == RubyDefSide::Includer ? kRubyIncluderMark : side == RubyDefSide::ClassMethods ? kRubyClassMethodsMark : std::string_view {} ); +} + +// A LocalBindKind::RubySingletonDef binding for the def at `defNode` named `name`, when a class object answers it: a +// singleton method (parser version 137), or a class method of each includer (parser version 145). +inline void rubyNoteClassSideDef( TSNode defNode, std::uint32_t fileId, std::string_view name, std::string_view src, std::vector& binds ) +{ + const RubyDefSide side = rubyDefSide( defNode, src ); + if( side != RubyDefSide::Instance ) { - const char* pt = ts_node_type( prev ); - if( !kindIs( ts_node_type( n ), "call" ) || !( kindIs( pt, "do_block" ) || kindIs( pt, "block" ) ) ) + binds.push_back( RawBind { .fileId = fileId, .startByte = ts_node_start_byte( defNode ), .lang = Lang::Ruby, .kind = LocalBindKind::RubySingletonDef, + .var = std::string( name ), .importedName = rubySideMark( side ) } ); + } +} + +// The bindings for what the `class << self` call `n` declares on the class object's `side` — an accessor, or a +// delegation-DSL call (captureRubySingletonAccessors below). An accessor's binding sits at its symbol, +// where captureRubyAttrDefs starts its def (no owner), and carries the side's mark (one a concern's `included do` +// declares is the includer's); a delegated name has no def, and is read against the class that owns the singleton. +inline void rubyNoteSingletonDecl( TSNode n, RubyDefSide side, std::uint32_t fileId, std::string_view src, std::vector& binds ) +{ + const std::string_view delegation = rubyNamedDirective( n, src, kRubyDelegationCalls ); + if( delegation.empty() ) + { + rubyNoteSymbolArguments( fieldChild( n, NodeField::Arguments ), + RawBind { .fileId = fileId, .lang = Lang::Ruby, .kind = LocalBindKind::RubySingletonDef, .importedName = rubySideMark( side ) }, + src, binds ); + return; + } + const std::string owner = rubyEnclosingScopeOf( n, src ); + if( delegation == "delegate_missing_to" ) + { + binds.push_back( RawBind { .fileId = fileId, .startByte = ts_node_start_byte( n ), .lang = Lang::Ruby, .kind = LocalBindKind::RubySingletonDef, .typeName = owner } ); + return; + } + rubyNoteSymbolArguments( fieldChild( n, NodeField::Arguments ), RawBind { .fileId = fileId, .lang = Lang::Ruby, .kind = LocalBindKind::RubySingletonDef, .typeName = owner }, + src, binds ); +} + +// What a `class << self` declares beside its defs is the class object's too (parser version 137): each accessor +// (`attr_accessor :config`, one RubySingletonDef binding per name, where captureRubyAttrDefs starts its def) and each name +// the delegation DSL defines (model.h kRubyDelegationCalls — `delegate :reset, to: :instance`: a binding per name against +// the owning class; `delegate_missing_to` one with no name, as it answers every name). A file that never writes `<<` is not +// walked; the walk is iterative, so a hostile nesting depth costs heap, not stack. +inline void captureRubySingletonAccessors( TSNode root, std::uint32_t fileId, std::string_view src, std::vector& binds ) +{ + if( src.find( "<<" ) == std::string_view::npos ) + { + return; + } + rubyWalkUntaken( root, [ & ]( TSNode n ) + { + const bool call = kindIs( ts_node_type( n ), "call" ); + const bool declares = call && ( !rubyNamedDirective( n, src, kRubyAttrFamilyNames ).empty() || !rubyNamedDirective( n, src, kRubyDelegationCalls ).empty() ); + const RubyDefSide side = declares && rubyAttrAtClassBodyLevel( n, src ) ? rubyDefSide( n, src ) : RubyDefSide::Instance; + if( side == RubyDefSide::Instance ) { - continue; // not a call, or the site is in its receiver/arguments rather than its block + return false; } - const int group = rspecGroupKind( n, src ); - if( group == 2 ) + rubyNoteSingletonDecl( n, side, fileId, src, binds ); + return true; + } ); +} + +// Parser version 145 (test/rubyclassrecvcheck.sh): each mixin the CLASS OBJECT's lookup reaches rather than its +// instances' — the constants of an `extend M`, and of an `include`/`prepend` inside `class << self` — as a +// LocalBindKind::RubyClassMixin binding at the constant's own start byte, where captureRubyMixinBases puts its inherit +// reference, read where that reads the directive (class-body level, or a concern's `included do`). One a concern's +// `included do` writes extends the includer, not the concern: importedName kRubyIncluderMark. A class body's `extend self` +// names no constant: its binding sits at the directive, var "self". A file that never writes `extend` or `<<` is not +// walked, and the walk enters no def, whose body holds no directive. +// Is the mixin directive `n` an `extend self` — its one argument `self`? +inline bool rubyExtendsSelf( TSNode n ) noexcept +{ + const TSNode args = fieldChild( n, NodeField::Arguments ); + return !ts_node_is_null( args ) && ts_node_named_child_count( args ) == 1 && kindIs( ts_node_type( ts_node_named_child( args, 0 ) ), "self" ); +} + +// The RubyClassMixin bindings of the mixin directive `n` (an `extend` when `extend`), read where captureRubyMixinBases reads +// it: one per constant a class object's lookup reaches, and a module body's `extend self`. +inline void rubyNoteClassMixins( TSNode n, bool extend, std::uint32_t fileId, std::string_view src, std::vector& binds ) +{ + const bool inIncluded = rubyInConcernIncludedBlock( n, src ); + if( !inIncluded && !rubyAttrAtClassBodyLevel( n, src ) ) + { + return; + } + const bool singleton = rubyIsSingletonDef( n ); // the directive sits in `class << self` + if( extend && !inIncluded && !singleton && rubyExtendsSelf( n ) ) + { + binds.push_back( RawBind { .fileId = fileId, .startByte = ts_node_start_byte( n ), .lang = Lang::Ruby, .kind = LocalBindKind::RubyClassMixin, .var = "self" } ); + } + if( extend == singleton ) + { + return; // an include joins the instances' lookup, an extend inside `class << self` the singleton's own + } + const bool includer = inIncluded || ( singleton && rubyDefSide( n, src ) == RubyDefSide::Includer ); + for( const TSNode c : rubyMixinTargets( n, src ) ) + { + binds.push_back( RawBind { .fileId = fileId, .startByte = ts_node_start_byte( c ), .lang = Lang::Ruby, .kind = LocalBindKind::RubyClassMixin, + .importedName = includer ? std::string( kRubyIncluderMark ) : std::string {} } ); + } +} + +inline void captureRubyClassMixins( TSNode root, std::uint32_t fileId, std::string_view src, std::vector& binds ) +{ + if( src.find( "extend" ) == std::string_view::npos && src.find( "<<" ) == std::string_view::npos ) + { + return; + } + rubyWalkUntaken( root, [ & ]( TSNode n ) + { + const char* t = ts_node_type( n ); + if( kindIs( t, "method" ) || kindIs( t, "singleton_method" ) ) + { + return true; // a def body holds no directive + } + const std::string_view verb = kindIs( t, "call" ) ? rubyNamedDirective( n, src, kRubyConstantDirectives ) : std::string_view {}; + if( verb.empty() || verb == "autoload" ) + { + return false; + } + rubyNoteClassMixins( n, verb == "extend", fileId, src, binds ); + return true; + } ); +} + +// ─── FactoryBot's factory definitions (parser version 132, test/rubytypedrecvcheck.sh) ───────────────────────────────── +// `factory :user do … end` inside a `FactoryBot.define` (or `FactoryGirl.define`) block defines what `create( :user )` +// builds, and its class is FactoryBot's own rule: the `class:` option as written — a constant, or a string naming one (its +// final segment) — else the enclosing factory's (a nested factory builds its parent's class), else the `parent:` factory's +// when the file defined it above, else the name camelised (`:user_profile` → UserProfile). One RubyFactory binding per name +// and per `aliases:` entry; graph.h rubyFactoryClasses joins them over the tree, where two classes for one name type nothing. +// A `class:` of any other spelling (a symbol, an expression) records no factory: its class is not known. + +// `user_profile` → UserProfile, `admin/user` → User: the final segment of the constant FactoryBot camelises the name into +inline std::string rubyFactoryDefaultClass( std::string_view name ) +{ + const std::size_t slash = name.rfind( '/' ); + std::string out; + bool upper = true; + for( const char c : name.substr( slash == std::string_view::npos ? 0 : slash + 1 ) ) + { + if( c == '_' ) { - return {}; // a shared group: its body runs in whichever group includes it + upper = true; + continue; } - if( group == 1 ) + out.push_back( upper && c >= 'a' && c <= 'z' ? char( c - 'a' + 'A' ) : c ); + upper = false; + } + return out; +} + +// The value of the `key:` option among a call's arguments (`class: "User"`, `parent: :user`), or null. +inline TSNode rubyCallOption( TSNode call, std::string_view key, std::string_view src ) +{ + const TSNode args = fieldChild( call, NodeField::Arguments ); + TSNode value {}; + if( !ts_node_is_null( args ) ) + { + ChildCursor cursor( args ); + forEachNamedChild( args, cursor.cur, [ & ]( TSNode c ) + { + const TSNode k = kindIs( ts_node_type( c ), "pair" ) ? fieldChild( c, NodeField::Key ) : TSNode {}; + const bool named = !ts_node_is_null( k ) && kindIs( ts_node_type( k ), "hash_key_symbol" ) && pattern::nodeText( k, src ) == key; + value = named ? fieldChild( c, NodeField::Value ) : value; + return !named; + } ); + } + return value; +} + +// The class a `class:` option names: a constant's final segment, or a plain string's (`"Admin::User"` → User); else empty. +inline std::string_view rubyFactoryClassOption( TSNode value, std::string_view src ) +{ + if( ts_node_is_null( value ) ) + { + return {}; + } + if( isRubyConstantNode( value ) ) + { + return rubyFinalConstant( value, src ); + } + if( !kindIs( ts_node_type( value ), "string" ) || ts_node_named_child_count( value ) != 1 + || !kindIs( ts_node_type( ts_node_named_child( value, 0 ) ), "string_content" ) ) + { + return {}; + } + const std::string_view text = pattern::nodeText( ts_node_named_child( value, 0 ), src ); + const std::size_t cut = text.rfind( "::" ); + return cut == std::string_view::npos ? text : text.substr( cut + 2 ); +} + +class RubyFactoryWalk +{ +public: + RubyFactoryWalk( std::uint32_t fileId, std::string_view src, std::vector& binds ) : fileId_( fileId ), src_( src ), binds_( binds ) {} + void run( TSNode root ); + +private: + struct Item + { + TSNode node; + bool inDefine; // inside a `FactoryBot.define` block + std::uint32_t parent; // index into classes_ of the enclosing factory's class; kNone outside one + }; + static constexpr std::uint32_t kNone = 0xFFFFFFFFu; + void define( TSNode call, const Item& item ); + std::string classOf( TSNode call, std::string_view name, const Item& item ) const; + void emit( TSNode call, std::string_view name, const std::string& cls ); + + std::uint32_t fileId_; + std::string_view src_; + std::vector& binds_; + std::vector stack_; + std::vector kids_; + std::vector classes_; + HashMap byName_; // a factory defined above in this file → its class +}; + +inline void RubyFactoryWalk::run( TSNode root ) +{ + ChildCursor cursor( root ); + stack_.push_back( Item { root, false, kNone } ); + while( !stack_.empty() ) + { + Item item = stack_.back(); + stack_.pop_back(); + const TSNode n = item.node; + if( kindIs( ts_node_type( n ), "call" ) ) { - if( const std::optional arg = rspecGroupArgument( n, src ) ) + const TSNode recv = fieldChild( n, NodeField::Receiver ); + const std::string_view on = !ts_node_is_null( recv ) && isRubyConstantNode( recv ) ? rubyFinalConstant( recv, src_ ) : std::string_view {}; + const std::string_view m = fieldChildTextOfKind( n, NodeField::Method, "identifier", src_ ); + item.inDefine = item.inDefine || ( ( on == "FactoryBot" || on == "FactoryGirl" ) && m == "define" ); + if( item.inDefine && ts_node_is_null( recv ) && m == "factory" ) { - return *arg; + define( n, item ); + continue; } } + collectChildren( n, cursor.cur, kids_ ); + for( std::size_t i = kids_.size(); i > 0; --i ) + { + stack_.push_back( Item { kids_[ i - 1 ], item.inDefine, item.parent } ); + } } - return {}; +} + +// one `factory :name, …` call: its class (the section note above), a binding per name and alias, then its block's +// factories, which build this class unless they say otherwise +inline void RubyFactoryWalk::define( TSNode call, const Item& item ) +{ + const std::string_view name = rubyFirstSymbolArgument( call, src_ ); + std::string cls = name.empty() ? std::string {} : classOf( call, name, item ); + if( cls.empty() ) + { + return; // no symbol name, or a class this file does not say: no factory is recorded, and none nested in it + } + const std::uint32_t index = std::uint32_t( classes_.size() ); + classes_.push_back( std::move( cls ) ); + byName_.try_emplace( name, index ); + emit( call, name, classes_[ index ] ); + if( const TSNode aliases = rubyCallOption( call, "aliases", src_ ); !ts_node_is_null( aliases ) && kindIs( ts_node_type( aliases ), "array" ) ) + { + ChildCursor cursor( aliases ); + forEachNamedChild( aliases, cursor.cur, [ & ]( TSNode a ) + { + const std::string_view alias = kindIs( ts_node_type( a ), "simple_symbol" ) ? pattern::nodeText( a, src_ ).substr( 1 ) : std::string_view {}; + if( !alias.empty() ) + { + emit( call, alias, classes_[ index ] ); + } + return true; + } ); + } + if( const TSNode block = rubyCallBlock( call ); !ts_node_is_null( block ) ) + { + stack_.push_back( Item { block, true, index } ); + } +} + +// the class `factory :name, …` builds, by FactoryBot's rule (the section note above); empty when this file does not say +inline std::string RubyFactoryWalk::classOf( TSNode call, std::string_view name, const Item& item ) const +{ + if( const TSNode opt = rubyCallOption( call, "class", src_ ); !ts_node_is_null( opt ) ) + { + return std::string( rubyFactoryClassOption( opt, src_ ) ); + } + if( const TSNode parent = rubyCallOption( call, "parent", src_ ); !ts_node_is_null( parent ) ) + { + const bool symbol = kindIs( ts_node_type( parent ), "simple_symbol" ); + const auto named = symbol ? byName_.find( pattern::nodeText( parent, src_ ).substr( 1 ) ) : byName_.end(); + return named == byName_.end() ? std::string {} : classes_[ named->second ]; + } + return item.parent != kNone ? classes_[ item.parent ] : rubyFactoryDefaultClass( name ); +} + +inline void RubyFactoryWalk::emit( TSNode call, std::string_view name, const std::string& cls ) +{ + RawBind b; + b.fileId = fileId_; + b.startByte = ts_node_start_byte( call ); + b.lang = Lang::Ruby; + b.kind = LocalBindKind::RubyFactory; + b.var = std::string( name ); + b.typeName = cls; + binds_.push_back( std::move( b ) ); +} + +// One file's FactoryBot factories, into `binds` — a file with no `FactoryBot.define` / `FactoryGirl.define` is not walked. +inline void captureRubyFactories( TSNode root, std::uint32_t fileId, std::string_view src, std::vector& binds ) +{ + if( src.find( "FactoryBot.define" ) != std::string_view::npos || src.find( "FactoryGirl.define" ) != std::string_view::npos ) + { + RubyFactoryWalk( fileId, src, binds ).run( root ); + } +} + +// `described_class::Worker` — a constant path whose head is RSpec's described_class — spelled through the path the +// described class is written as (`Log::Worker`, parser version 137); empty for any other head, and where the file +// redefines described_class or a local of that name reaches the head. +inline std::string rubyDescribedRelativePath( TSNode node, std::string_view src ) +{ + TSNode head = node; + for( TSNode scope = fieldChild( head, NodeField::Scope ); kindIs( ts_node_type( head ), "scope_resolution" ) && !ts_node_is_null( scope ); + scope = fieldChild( head, NodeField::Scope ) ) + { + head = scope; // the leftmost segment, one step at a time: a deep chain costs a loop, never a frame + } + const bool described = kindIs( ts_node_type( head ), "identifier" ) && pattern::nodeText( head, src ) == "described_class" + && !rubyDefinesDescribedClassMethod( src ) && !rubyDescribedClassIsLocal( head, src ); + const std::string_view path = described ? rspecDescribedPath( head, src ) : std::string_view {}; + if( path.empty() ) + { + return {}; + } + return std::string( path ).append( pattern::nodeText( node, src ).substr( ts_node_end_byte( head ) - ts_node_start_byte( node ) ) ); } // nullopt when the node is neither (classifyReceiver's shared arms decide it); otherwise the answer, which is empty for a @@ -670,6 +2162,31 @@ inline RecvShape receiverOf( TSNode nameNode, Lang lang, std::string_view src ) return rs; } +// The constant a Ruby call's receiver names, AS WRITTEN, when that differs from `final` — the final segment recvVar keeps +// (parser version 137: `Stripe::Customer` is not the tree's Customer unless the tree opens that path, graph.h +// RubyClassObjects). A qualified constant's whole path; the path through the described class for `described_class` and +// for `described_class::Worker`; empty for a final segment alone, and for a path with any other head that is no constant +// (`self.class::Foo`). +inline std::string rubyReceiverWrittenPath( TSNode nameNode, std::string_view final, std::string_view src ) +{ + const TSNode call = calleeAccessParent( nameNode ); + const TSNode recv = ts_node_is_null( call ) || !kindIs( ts_node_type( call ), "call" ) ? TSNode {} : fieldChild( call, NodeField::Receiver ); + std::string written; + if( ts_node_is_null( recv ) ) + { + return written; + } + if( kindIs( ts_node_type( recv ), "identifier" ) && pattern::nodeText( recv, src ) == "described_class" ) + { + written = rspecDescribedPath( recv, src ); // classifyRubyReceiver named the group's constant: its path beside it + } + else if( isRubyConstantNode( recv ) ) + { + written = rubyIsConstantChain( recv ) ? std::string( pattern::nodeText( recv, src ) ) : rubyDescribedRelativePath( recv, src ); + } + return written == final ? std::string {} : written; +} + // Java method references are not ordinary member-access nodes. The pinned grammar's first named // child is the receiver and deliberately uses the SAME `identifier` node for a simple type and a // variable. Preserve that uncertainty for graph.h: simple and dotted type candidates are retained diff --git a/src/ingest_cache.h b/src/ingest_cache.h index 898e015cb..310de224d 100644 --- a/src/ingest_cache.h +++ b/src/ingest_cache.h @@ -296,7 +296,82 @@ constexpr std::uint32_t kCacheVersion = 27; // 27: same record layout // (Py `pkg.mod`, TS `./x`, Rust `crate::a::b`/`mod:x`) — // a target FORMAT change → old caches must be rejected. // 4: Include gained a `bool isAngle` (quote/angle) field -constexpr std::uint32_t kParserVer = 133; // bump on any grammar/.scm/extraction change +constexpr std::uint32_t kParserVer = 145; // bump on any grammar/.scm/extraction change + // 145 = 2026-10-04 (test/rubyclassrecvcheck.sh, PR #373 review): a + // Ruby mixin the class object's lookup reaches — an `extend`, an + // `include` in `class << self` — mints a LocalBindKind:: + // RubyClassMixin binding at its constant (an appended kind), and a + // RubySingletonDef binding carries importedName "included" when + // its def is each includer's class method (a singleton def in a + // concern's `included do`, a def in `class_methods do`, which now + // mints one). Above main's 143 and 144. Same layout: kCacheVersion + // stays 27, kQSnapCacheScheme stays 16. A Ruby cache written at 142 + // lacks them and must re-parse. + // 142 = 2026-10-04 (the Ruby method-lookup branch rebased onto main 2720d1c5): + // cache-key hygiene, not an extraction change. The branch's eight + // Ruby extraction steps carried 130–137 (the notes below keep + // them); main took 130–133 for other lanes, and train 25 uses + // 134, 135, 140 and 141. 142 is above all of them, so no cache + // another build wrote is read as this build's. Same layout: + // kCacheVersion stays 27, kQSnapCacheScheme stays 16. + // 137 = 2026-10-02 (test/rubyclassrecvcheck.sh): a Ruby call whose + // receiver is a constant written with a path keeps the path in + // RawRef::fieldName (`Billing::Invoice.issue` → "Billing:: + // Invoice"; recvVar keeps "Invoice"), and a singleton method — + // `def self.m`, a def or an accessor in `class << self` — mints a + // LocalBindKind::RubySingletonDef binding (an appended kind), both + // read by graph.h's class-object lookup. Same layout: kCacheVersion + // unchanged (27). A Ruby cache written at 136 lacks them and must + // re-parse. + // 136 = 2026-10-02 (test/rubyrspectargetcheck.sh): RSpec's matcher + // builders (`receive`, `change`, … model.h kRspecTargets) type + // their receivers in an example group, and a call on a chain + // rooted at one is typed as the root (ingest_binds.h + // rubyMatcherChainType). Same layout: kCacheVersion unchanged + // (27). A Ruby cache written at 135 holds the untyped chains. + // 135 = 2026-10-02 (test/rubyrakejbuildercheck.sh): `.rake` and + // `.jbuilder` files index as Ruby; a Jbuilder template's `json` + // is a local typed JbuilderTemplate, and `helper_method :m` at + // class-body position mints a LocalBindKind::RubyHelperMethod + // binding (an appended kind). New records, same layout: + // kCacheVersion unchanged (27). A Ruby cache written at 134 + // lacks them and must re-parse. + // 134 = 2026-10-02 (test/rubydeclrefcheck.sh): a Rails callback, + // validation condition or rescue_from handler named by SYMBOL + // mints a Ruby call RawRef at the symbol, and `send( :m )` and + // its kin a renamed copy of their own call RawRef + // (ingest_binds.h noteDeclaredCall, sendTargets). New records, + // same layout: kCacheVersion unchanged (27). A Ruby cache + // written at 133 lacks them and must re-parse. + // 133 = 2026-10-02 (test/rubyrspectargetcheck.sh): inside an + // RSpec example group, a receiver built by `expect`, `allow`, + // `expect_any_instance_of`, `allow_any_instance_of` or a bare + // `is_expected` carries RSpec's target class as its type + // (model.h kRspecTargets). Same layout: kCacheVersion + // unchanged (27). A Ruby cache written at 132 holds those + // receivers untyped and must re-parse. + // 132 = 2026-10-02 (test/rubytypedrecvcheck.sh): a Ruby call + // receiver the file BUILDS (`Client.new`, a finder, a FactoryBot + // build, a let or local holding one) carries its type in the + // call RawRef's recvVar (model.h rubyTypedRecvToken), and each + // FactoryBot `factory` mints a RawBind of the APPENDED kind + // LocalBindKind::RubyFactory (ingest_binds.h). Same layout: + // kCacheVersion unchanged (27). A Ruby cache written at 131 + // holds untyped receivers and no factories and must re-parse. + // 131 = 2026-10-02 (test/rubyreachcheck.sh): a Ruby MIXIN is an + // ancestor — `include`/`extend`/`prepend` at class-DSL + // position mints one inherit RawRef per constant, at the + // constant's byte (ingest_relations.h captureRubyMixinBases). + // New records, same layout: kCacheVersion unchanged (27). A + // Ruby cache written at 130 holds none of them and must re-parse. + // 130 = 2026-10-01 (test/rubybarecallcheck.sh): Ruby's BARE-WORD call — an + // identifier with no receiver, arguments or parentheses, which + // tree-sitter-ruby parses as the same (identifier) a local read + // is — mints a receiver-less call RawRef wherever Ruby's own + // lexical rule says it is not a local (ingest_binds.h + // captureRubyBareCalls). New records, same layout: + // kCacheVersion unchanged (27). A Ruby cache written at 129 + // holds none of them and must re-parse. // 133 = 2026-10-02 (train 24): cache-key hygiene above every branch build's number. Two // merged lanes changed extraction under their own numbers: 130 (a body-less C/C++ // enum/struct/union/class specifier in a function signature keeps its own span, so diff --git a/src/ingest_crawl.h b/src/ingest_crawl.h index 6a5c10a32..88420139f 100644 --- a/src/ingest_crawl.h +++ b/src/ingest_crawl.h @@ -61,6 +61,10 @@ struct LangEntry // Named once so the table row and ingest_sidecap.h's restrictAstroToFrontmatter cannot drift apart. inline constexpr std::string_view kAstroExt = ".astro"; +// The Ruby extension whose top level is a view (a Jbuilder template), named once for the row and ingest_sidecap.h's +// captureRubyBareCalls call, which binds the template's `json` there. +inline constexpr std::string_view kJbuilderExt = ".jbuilder"; + // Order does not matter (linear scan); kept grouped by language for readability. // The extent is EXACT, not headroom: it was 32 with 32 rows, .toml made it 33, .pyi made it 34 and the // .yml/.yaml pair made it 36, the .php/.phtml/.lua trio made it 40, the .ex/.exs pair made it 42, the @@ -93,7 +97,7 @@ inline constexpr std::string_view kAstroExt = ".astro"; // the latter a list item), so those files carry the file-level node alone and serve as ONE whole-file // unit. A heading detector per format is a later lane with its own measurement. `.mdx` is markdown with // JSX, which the block grammar already reads as html blocks (opaque). Gate: test/textdocscheck.sh. -constexpr std::array kLangTable = {{ +constexpr std::array kLangTable = {{ { ".cpp", Lang::Cpp, &tree_sitter_cpp, "cpp" }, { ".cc", Lang::Cpp, &tree_sitter_cpp, "cpp" }, { ".cxx", Lang::Cpp, &tree_sitter_cpp, "cpp" }, @@ -185,6 +189,11 @@ constexpr std::array kLangTable = {{ { ".zsh", Lang::Bash, &tree_sitter_bash, "bash" }, // zsh — parsed with the bash grammar (superset-ish; partial) { ".java", Lang::Java, &tree_sitter_java, "java" }, // Java — classes/interfaces/enums/methods/ctors + calls { ".rb", Lang::Ruby, &tree_sitter_ruby, "ruby" }, // Ruby — class/module/def + method calls + require + // A Rake task file and a Jbuilder view are plain Ruby (test/rubyrakejbuildercheck.sh, parser version 135): what a + // task or a JSON view calls is a caller like any other. Outside any class their self is known — Rake's `main`, the + // view — and graph.h RubyTopSelf reads it; a `.jbuilder` file's `json` is the template's (ingest_binds.h). + { ".rake", Lang::Ruby, &tree_sitter_ruby, "ruby" }, + { kJbuilderExt, Lang::Ruby, &tree_sitter_ruby, "ruby" }, { ".json", Lang::Json, &tree_sitter_json, "json" }, // JSON — top-level + 2nd-level object keys as t="sec"; DATA, no call edges { ".toml", Lang::Toml, &tree_sitter_toml, "toml" }, // TOML — [table] headers + their keys as t="sec"; DATA, no call edges { ".yml", Lang::Yaml, &tree_sitter_yaml, "yaml" }, // YAML — mapping keys (mdepth<=2, seqs transparent) as t="sec"; DATA, no call edges diff --git a/src/ingest_relations.h b/src/ingest_relations.h index 70ddafbf2..b3f1e9ea5 100644 --- a/src/ingest_relations.h +++ b/src/ingest_relations.h @@ -56,6 +56,21 @@ inline bool isBaseTypeNodeIn( const char* nt, Lang lang ) noexcept return lang == Lang::Ruby ? ( kindIs( nt, "constant" ) || kindIs( nt, "scope_resolution" ) ) : isBaseTypeNode( nt ); } +// One use-site RawRef sited at node `at`: its start byte (the byte-span sweep attributes the ref to the enclosing def) +// and its 1-based line (ABS-3: --uses p="file:line"), with the role and name the caller decides; every other field keeps +// RawRef's default. +inline RawRef rawRefAt( TSNode at, std::uint32_t fileId, Lang lang, RefRole role, std::string name ) +{ + RawRef r; + r.fileId = fileId; + r.startByte = ts_node_start_byte( at ); + r.line = ts_node_start_point( at ).row + 1; + r.lang = lang; + r.role = role; + r.name = std::move( name ); + return r; +} + // Emit one inherit RawRef (derived → base) for a base-type node. startByte sits inside the class header // (the type node's own start), so the byte-span enclosing attribution binds fromSymbol = the derived class. inline void emitBaseRef( TSNode typeNode, std::uint32_t fileId, Lang lang, std::string_view src, std::vector& refs ) @@ -65,14 +80,8 @@ inline void emitBaseRef( TSNode typeNode, std::uint32_t fileId, Lang lang, std:: { return; } - RawRef r; - r.fileId = fileId; - r.startByte = a; // inside the class header → attributes to the derived class - r.line = ts_node_start_point( typeNode ).row + 1; // ABS-3: 1-based use-site line for --uses - r.lang = lang; + RawRef r = rawRefAt( typeNode, fileId, lang, RefRole::Extends, finalSegment( src.substr( a, b - a ) ) ); // a base-class / interface use-site r.isInherit = true; - r.role = RefRole::Extends; // ABS-3: a base-class / interface use-site (derived → base) - r.name = finalSegment( src.substr( a, b - a ) ); refs.push_back( std::move( r ) ); } @@ -441,9 +450,8 @@ void captureMacroBodyCalls( TSNode defineNode, std::uint32_t fileId, Lang lang, // attribution assigns fromSymbol = the derived class. Explicit-syntax langs: C++/TS/JS/Java/Python/Swift/ // C#/PHP/Kotlin/Ruby. Ruby reuses the `superclass` clause name Java's extends clause already has, and // names its base with (constant)/(scope_resolution) — see isBaseTypeNodeIn. A Ruby MIXIN -// (`include M` / `extend M` / `prepend M`) is NOT captured here, for the same reason the PHP note below -// gives: it is a call in the class BODY, not a clause. Ruby's ancestor chain does hold included modules, -// so that is a stated residue (test/rubyinheritcheck.sh floor (b)), not a claim it is not inheritance. Lua is deliberately absent and it is a DISCLOSED non-goal, not an omission: Lua inheritance IS +// (`include M` / `extend M` / `prepend M`) is a call in the class BODY, not a clause, so it is captured where +// the call is read instead: captureRubyMixinBases (parser version 131; test/rubyreachcheck.sh). Lua is deliberately absent and it is a DISCLOSED non-goal, not an omission: Lua inheritance IS // `setmetatable( Derived, { __index = Base } )`, an ordinary runtime call over an ordinary table, so there // is no syntax to read and a Lua corpus correctly reports no inheritance edges at all. // @@ -1465,11 +1473,12 @@ inline std::string_view rubyNamedDirective( TSNode n, std::string_view src, std: // `include A, B` / `extend M` / `prepend P` — ONE directive naming N constants and therefore N Include // records, in SOURCE order (the same shape as elixirAliasGroup). A non-constant argument (`include -// Object.const_get(:X)`, `include mod`) contributes nothing; the constant ones beside it still do. -inline std::vector rubyMixinTargets( TSNode n, std::string_view src ) +// Object.const_get(:X)`, `include mod`) contributes nothing; the constant ones beside it still do. The constant +// NODES: captureIncludes reads each one's text for its Include record, and captureRubyMixinBases its byte. +inline std::vector rubyMixinTargets( TSNode n, std::string_view src ) { - std::vector out; - const std::string_view m = rubyNamedDirective( n, src, kRubyConstantDirectives ); + std::vector out; + const std::string_view m = rubyNamedDirective( n, src, kRubyConstantDirectives ); if( m.empty() || m == "autoload" ) { return out; @@ -1482,9 +1491,9 @@ inline std::vector rubyMixinTargets( TSNode n, std::string_view src ChildCursor cursor( args ); // O(children): `include A, # … B` — 58x at 16 000 (childwalkscalecheck B20) forEachNamedChild( args, cursor.cur, [ & ]( TSNode a ) { - if( std::string c = rubyConstantText( a, src ); !c.empty() ) + if( isBaseTypeNodeIn( ts_node_type( a ), Lang::Ruby ) ) // a constant or a constant path: rubyConstantText's set { - out.push_back( std::move( c ) ); + out.push_back( a ); } return true; } ); @@ -1554,6 +1563,44 @@ inline bool rubyAttrAtClassBodyLevel( TSNode n, std::string_view src ) noexcept return false; } +// A Ruby MIXIN is an ancestor (parser version 131): `include A, B` / `extend M` / `prepend P` at class-DSL position +// (rubyAttrAtClassBodyLevel) puts each constant in the class's ancestor chain, so each emits one inherit reference at the +// CONSTANT's own byte — the shape a superclass has, so resolve.h's RubyBaseScope scopes it by Ruby's constant lookup and +// graph.h's inheritance graph and base walk read it unchanged. `extend` reaches the class's singleton, not its +// instances; the graph keeps one method namespace per class (instance and class methods alike), so it is an ancestor +// there too. So does one inside a module body's `included do … end` (rubyInConcernIncludedBlock). A directive anywhere +// else — a method body, another block, a file's top level — emits none. +// Is `call` a statement of an ActiveSupport::Concern `included do … end` block written at a module's body level? When a +// class includes the concern the block runs in that class, so a mixin there joins the INCLUDER's ancestors; read as an +// ancestor of the concern itself, it reaches the includer through the concern — activerecord's AttributeMethods +// assembles its Read, Write and PrimaryKey modules exactly so. +inline bool rubyInConcernIncludedBlock( TSNode call, std::string_view src ) noexcept +{ + TSNode up = ts_node_parent( call ); + if( !ts_node_is_null( up ) && ( kindIs( ts_node_type( up ), "body_statement" ) || kindIs( ts_node_type( up ), "block_body" ) ) ) + { + up = ts_node_parent( up ); // do … end wraps its statements in body_statement, { } in block_body + } + const TSNode block = up; + const TSNode owner = ts_node_is_null( block ) ? block : ts_node_parent( block ); + return !ts_node_is_null( owner ) && ( kindIs( ts_node_type( block ), "do_block" ) || kindIs( ts_node_type( block ), "block" ) ) + && kindIs( ts_node_type( owner ), "call" ) && rubyNamedDirective( owner, src, std::array{ "included" } ) == "included" + && rubyAttrAtClassBodyLevel( owner, src ); +} + +inline void captureRubyMixinBases( TSNode call, const std::vector& constants, std::uint32_t fileId, std::string_view src, + std::vector& refs ) +{ + if( constants.empty() || !( rubyAttrAtClassBodyLevel( call, src ) || rubyInConcernIncludedBlock( call, src ) ) ) + { + return; + } + for( const TSNode a : constants ) + { + emitBaseRef( a, fileId, Lang::Ruby, src, refs ); + } +} + // A CONSTANT CHAIN: `Name`, `A::B::C`, `::A::B` — every segment a constant, the head a constant or absent (`::A`). // rubyConstantText above accepts any scope_resolution and is right for the positions Ruby's grammar already // restricts to constants (a class name, a superclass, a mixin argument); a RECEIVER is not such a position — @@ -2556,10 +2603,12 @@ void captureIncludes( TSNode root, Lang lang, std::uint32_t fileId, std::string_ { // `include A, B` / `extend M` / `prepend P` (parser version 82): N constants off one directive node, in // SOURCE order, each a symbolic Include — the Ruby twin of the Elixir alias group below. - for( std::string& member : rubyMixinTargets( n, src ) ) + const std::vector mixins = rubyMixinTargets( n, src ); + for( const TSNode member : mixins ) { - emitDirective( std::move( member ), true, isLazy, false ); + emitDirective( rubyConstantText( member, src ), true, isLazy, false ); } + captureRubyMixinBases( n, mixins, fileId, src, refs ); // parser version 131: and each is an ancestor } else if( lang == Lang::Ruby && kindIs( t, "argument_list" ) ) { diff --git a/src/ingest_sidecap.h b/src/ingest_sidecap.h index 466ddd5ac..461ae88d9 100644 --- a/src/ingest_sidecap.h +++ b/src/ingest_sidecap.h @@ -2226,6 +2226,10 @@ void captureTagsFacts( TSQueryCursor* cursor, const LangEntry& le, std::uint32_t else if( le.lang == Lang::Ruby ) { // enclosing class/module → id= addressability, per-class overload sets, editCheckImplicitReceiver d.scope = rubyEnclosingScopeOf( nameNode, src ); // (test/rubyscopecheck.sh) + if( kind == SymKind::Method || kind == SymKind::Function ) + { + rubyNoteClassSideDef( defNode, fileId, d.name, src, binds ); // a class object's method (test/rubyclassrecvcheck.sh) + } } else if( le.lang == Lang::Kotlin ) { // enclosing class/object/companion-object → same P2-D Rule-1 narrowing Python/Ruby get; @@ -2345,6 +2349,10 @@ void captureTagsFacts( TSQueryCursor* cursor, const LangEntry& le, std::uint32_t r.recv = rs.kind; r.recvVar = std::move( rs.var ); // → one-hop narrowing in resolve.h r.fieldName = std::move( rs.field ); // depth-2 intermediate field; "" otherwise r.viaArrow = rs.viaArrow; // `p->m()`: Rule 2b's smart-pointer pointee needs it + if( le.lang == Lang::Ruby && r.recv == RecvKind::NamedVar ) + { + r.fieldName = rubyReceiverWrittenPath( nameNode, r.recvVar, src ); // parser version 137: a constant's path as written + } auto [ ac, ak ] = callArity( nameNode, le.lang, src ); // B2.2: call-site positional arg count r.argCount = ac; r.argCountKnown = ak; // → arity filter in graph.h } @@ -2392,9 +2400,28 @@ void captureTagsFacts( TSQueryCursor* cursor, const LangEntry& le, std::uint32_t // Parser version 121 (test/rubyattrscheck.sh): the Ruby attr family's Var defs. Appended after the // dead/field folds — neither touches Ruby (no preprocessor; Var is not a Field kind) — and inside the // same defs window, so the lex build and cache round-trip treat these defs like captured ones. + // Parser version 130 (test/rubybarecallcheck.sh): Ruby's bare-word calls, by Ruby's own local rule, into the + // same refs window — the tags query cannot tell a bare call from a local read (both are an (identifier)). + // Parser version 132 (test/rubytypedrecvcheck.sh): the same walk types each call receiver the file builds, written + // into the call references this window holds; and FactoryBot's factory definitions, into binds. + // Parser version 135 (test/rubyrakejbuildercheck.sh): `helper_method` declarations into binds, and a Jbuilder + // template's `json` — a local, typed JbuilderTemplate — in the refs window. + // Parser version 137 (test/rubyclassrecvcheck.sh): the accessors a `class << self` declares, into binds as singleton + // methods — beside each singleton def, which the def capture above notes. + // Parser version 145 (test/rubyclassrecvcheck.sh): the mixins the class object's lookup reaches — an `extend`, an + // `include` in `class << self` — into binds, beside the inherit reference each mixin already is. if( le.lang == Lang::Ruby ) { captureRubyAttrDefs( root, fileId, src, defs ); + captureRubyBareCalls( root, fileId, src, refs ); + captureRubyFactories( root, fileId, src, binds ); + captureRubyHelperMethods( root, fileId, src, binds ); + captureRubySingletonAccessors( root, fileId, src, binds ); + captureRubyClassMixins( root, fileId, src, binds ); + if( le.ext == kJbuilderExt ) + { + typeRubyJbuilderLocal( fileId, refs ); + } } } diff --git a/src/lintrules.h b/src/lintrules.h index 845a83636..5c7642931 100644 --- a/src/lintrules.h +++ b/src/lintrules.h @@ -160,7 +160,7 @@ inline constexpr LintExtRow kLintExtRows[] = { { ".js", Lang::JavaScript }, { ".jsx", Lang::JavaScript }, { ".mjs", Lang::JavaScript }, { ".cjs", Lang::JavaScript }, { ".sh", Lang::Bash }, { ".bash", Lang::Bash }, { ".zsh", Lang::Bash }, { ".java", Lang::Java }, - { ".rb", Lang::Ruby }, + { ".rb", Lang::Ruby }, { ".rake", Lang::Ruby }, { ".jbuilder", Lang::Ruby }, { ".cs", Lang::CSharp }, { ".php", Lang::Php }, { ".phtml", Lang::Php }, { ".lua", Lang::Lua }, diff --git a/src/model.h b/src/model.h index 3ff2b953a..ec96b05a3 100644 --- a/src/model.h +++ b/src/model.h @@ -18,6 +18,7 @@ #include // std::tie — lessUnindexedExt's mixed-direction compare #include // Symbol::evWhy — the fixed-size ev_why tag counters #include +#include // rubyTypedRecvOf — a Ruby reference's built receiver type, or none #include #include #include // std::is_trivially_copyable_v — the VarSpan layout pin below @@ -614,10 +615,15 @@ struct FnLocalScope // override. Consumers that MEASURE or SERVE a body (clones, complexity, --readability units, lexical.h's route anchor) // keep the plain span test: a class with no body is a definition, but it has no volume. Gate: test/kotlincheck.sh §11 // (the PR's bodyless interface) and §13. +// Ruby breaks it for EVERY kind: the language has no declarations at all — no prototype, no abstract member, no +// interface — so each class, module, def, attr accessor and constant it indexes is the definition of its name. An +// `attr_reader :aliases` accessor and an empty `def on_event; end` hook own no body span, and read as declarations the +// collapse evicted them for any same-named bodied def in the family: a call inside Tracker to its own `aliases` bound to +// an unrelated Join#aliases. Gate: test/rubybarecallcheck.sh ("Ruby has no declarations"). inline bool isDefinitionNotDeclaration( const Symbol& s ) noexcept { const bool kotlinType = s.lang == Lang::Kotlin && ( s.kind == SymKind::Class || s.kind == SymKind::Struct || s.kind == SymKind::Interface ); - return s.endByte > s.sigEndByte || kotlinType; + return s.endByte > s.sigEndByte || kotlinType || s.lang == Lang::Ruby; } // local-variable-indexing plan Phase 1 MVP scope (docs/LOCALS_INDEXING.md): C/C++ only — highest @@ -712,14 +718,109 @@ struct Reference std::string calleeName; // referenced name (final identifier segment) std::string qualifier; // explicit scope at the call site (`A` in `A::b()`); "" if bare/method — for canonical resolve std::string recvVar; // receiver variable identifier when recv==NamedVar/FieldOfVar (`x` in `x->m()`); "" otherwise — for Rule 2 + // A Ruby receiver the code BUILDS carries its type here instead (rubyTypedRecvOf below). std::string fieldName; // member variable name when isCompose (e.g. "m_pool"); ALSO the INTERMEDIATE // field of a depth-2 chained receiver when recv is FieldOfThis/FieldOfVar // (`this->m_pool.run()` → "m_pool") — the two are mutually exclusive // (a compose ref is never a call ref, and the compose readers all gate - // on isCompose), so one slot carries both; "" otherwise + // on isCompose), so one slot carries both; "" otherwise. A third reading, + // on a Ruby call whose receiver is a constant written with a path (recv + // NamedVar, parser version 137): the path as written (`Billing::Invoice.issue` + // → "Billing::Invoice"; `described_class` → the group's constant whole), + // recvVar keeping the final segment every class is keyed by (graph.h + // RubyClassObjects::ofIndexed); no chain or compose reader reads a NamedVar's std::string composeRel; // "creates" (value/inline) or "uses" (reference/pointer) when isCompose; "" otherwise }; +// A Ruby call whose RECEIVER the code builds (parser version 132, test/rubytypedrecvcheck.sh): `c = Client.new` then `c.get`, +// `let( :user ) { create( :user ) }` then `user.activate!`, `User.find_by( … ).activate!`. ingest_binds.h reads the shape +// by Ruby's own local rule and writes the type into Reference::recvVar as "." — the class's constant as written +// and the method that built it (`User.find_by`, `OpenSSL::Cipher.new`) — or ":." for a FactoryBot build (`:user.create`), whose class +// only the tree's factory definitions know (graph.h rubyFactoryClasses). The receiver KIND is left as it was (NamedVar, +// FieldOfVar), so every rule that does not read the type sees the call exactly as before; and a `.` is in no Ruby +// identifier or constant segment, so no receiver the tool recorded before can read as a type. +inline constexpr char kRubyTypedRecvSep = '.'; + +struct RubyTypedRecv +{ + std::string_view type; // the class's constant as written (`User`, `Admin::User`), or the factory name + std::string_view via; // the method that built it: `new`, a finder, a factory build + bool factory = false; // `type` names a FactoryBot factory, not a class +}; + +inline std::string rubyTypedRecvToken( std::string_view type, std::string_view via, bool factory ) +{ + return std::string( factory ? ":" : "" ).append( type ).append( 1, kRubyTypedRecvSep ).append( via ); +} + +// the type a Ruby reference's receiver was built as, or nullopt for every other reference +inline std::optional rubyTypedRecvOf( const Reference& r ) noexcept +{ + const std::size_t sep = r.lang == Lang::Ruby ? r.recvVar.find( kRubyTypedRecvSep ) : std::string::npos; + if( sep == std::string::npos ) + { + return std::nullopt; + } + const std::string_view token = r.recvVar; + const bool factory = token.starts_with( ':' ); + return RubyTypedRecv { token.substr( factory ? 1 : 0, sep - ( factory ? 1 : 0 ) ), token.substr( sep + 1 ), factory }; +} + +// RSpec's targets (test/rubyrspectargetcheck.sh, parser version 133): inside an example group, the receiver-less builder +// `builder` — or the bare `is_expected`, which is `expect( subject )` — returns an instance of `cls`, whose `to`/`not_to`/ +// `to_not` are RSpec's. ingest_binds.h rubyValueType types the value; graph.h RubyTypedReceivers answers a call on it. +// The matchers whose chain methods are RSpec's join them (parser version 136): `receive( :m ).with( 1 )`, the `to` of +// `change { }.from( 1 ).to( 2 )` — a call on a chain rooted at one is typed as the root (ingest_binds.h +// rubyMatcherChainType); a link's own class (Change's `from` returns a ChangeFromValue) is RSpec's all the same. +struct RspecTarget +{ + std::string_view builder; + std::string_view cls; +}; +inline constexpr RspecTarget kRspecTargets[] = { + { "expect", "RSpec::Expectations::ValueExpectationTarget" }, // `expect { … }`: kRspecBlockTarget + { "is_expected", "RSpec::Expectations::ValueExpectationTarget" }, + { "allow", "RSpec::Mocks::AllowanceTarget" }, + { "expect_any_instance_of", "RSpec::Mocks::AnyInstanceExpectationTarget" }, + { "allow_any_instance_of", "RSpec::Mocks::AnyInstanceAllowanceTarget" }, + { "receive", "RSpec::Mocks::Matchers::Receive" }, + { "have_received", "RSpec::Mocks::Matchers::HaveReceived" }, + { "receive_messages", "RSpec::Mocks::Matchers::ReceiveMessages" }, + { "receive_message_chain", "RSpec::Mocks::Matchers::ReceiveMessageChain" }, + { "change", "RSpec::Matchers::BuiltIn::Change" }, + { "raise_error", "RSpec::Matchers::BuiltIn::RaiseError" }, + { "raise_exception", "RSpec::Matchers::BuiltIn::RaiseError" }, + { "output", "RSpec::Matchers::BuiltIn::Output" }, + { "be_within", "RSpec::Matchers::BuiltIn::BeWithin" }, + { "yield_control", "RSpec::Matchers::BuiltIn::YieldControl" }, +}; +inline constexpr std::string_view kRspecBlockTarget = "RSpec::Expectations::BlockExpectationTarget"; + +// The RSpec target a receiver typed `cls` (leading `::` dropped) by the builder `via` is — `expect { … }`'s block form +// included — or nullptr: a class reached any other way (`ValueExpectationTarget.new`) is an ordinary constant. +inline const RspecTarget* rspecTargetOf( std::string_view cls, std::string_view via ) noexcept +{ + for( const RspecTarget& t : kRspecTargets ) + { + if( t.builder == via ) + { + return t.cls == cls || ( via == "expect" && cls == kRspecBlockTarget ) ? &t : nullptr; + } + } + return nullptr; +} + +// Ruby's delegation DSL: a class that writes one defines methods the tree indexes nowhere in it (ActiveSupport's `delegate` +// and `delegate_missing_to`, Forwardable's `def_delegator(s)`), so it may answer a name no lookup over the tree finds (graph.h +// RubySelfReach); one a `class << self` writes defines the class object's (ingest_binds.h captureRubySingletonAccessors). +inline constexpr std::string_view kRubyDelegationCalls[] = { "def_delegator", "def_delegators", "delegate", "delegate_missing_to" }; + +// A Jbuilder template's `json` (test/rubyrakejbuildercheck.sh, parser version 135): the template handler binds it to the +// view's JbuilderTemplate, a BasicObject whose method_missing makes every call on it a key. ingest_binds.h types the local +// in a `.jbuilder` file; graph.h RubyTypedReceivers refuses a call on it unless the tree opens the class. +inline constexpr std::string_view kJbuilderTemplate = "JbuilderTemplate"; +inline constexpr std::string_view kJbuilderLocal = "json"; + // A physical dependency: one #include / import directive (file → target). The target is the raw // include path / module name (resolved to a file id later, for the file→file dependency graph). struct Include @@ -860,9 +961,39 @@ enum class LocalBindKind : std::uint8_t ElixirDefault, // var=callable name/arity, importedName=full name/arity, typeName=module; no synthetic symbol. ElixirImport, // typeName=module, var=all/only/except/functions/macros; importedName=newline-delimited name/arities. // spanStart/spanEnd delimit lexical visibility, starting after the directive. + RubyFactory, // parser version 132: a FactoryBot `factory :name` definition — var=the factory name (or one of its + // `aliases:`), typeName=the final segment of the class it builds (ingest_binds.h captureRubyFactories). + // Read by graph.h rubyFactoryClasses ONLY; every other reader skips it by kind or finds no + // variable of that name. APPENDED for the same cache reason as VarDecl. + RubyHelperMethod, // parser version 135: a `helper_method :name` declaration at class-body position — var=the method, + // typeName=the final segment of the class or module that declares it (ingest_binds.h + // RubyBareCallWalk::noteDeclaredCall). Read by graph.h RubyTopSelf ONLY. APPENDED, as above. + RubySingletonDef, // parser version 137: a Ruby SINGLETON method — `def self.m`, a def or an accessor inside `class << self` + // — at startByte, the def's own start (Symbol::sigStartByte); var=the method, typeName empty + // (ingest_binds.h rubyNoteClassSideDef, captureRubySingletonAccessors). A name the delegation DSL + // defines in a `class << self` has no def: typeName=the owning class, var=the name (empty for + // `delegate_missing_to`, every name). Read by graph.h's class-object lookup ONLY + // (rubyClassObjectDefs). APPENDED, as above. Parser version 145: a def that is a class method of + // each class INCLUDING its module rather than one of the module carries importedName + // kRubyIncluderMark — a singleton def inside a concern's `included do … end`, which defines it on + // the includer itself — or kRubyClassMethodsMark — a def inside its `class_methods do … end`, which + // ActiveSupport::Concern extends onto the includer (ingest_binds.h rubyDefSide). + RubyClassMixin, // parser version 145: a mixin the CLASS OBJECT's lookup reaches rather than its instances' — the constant + // of an `extend M`, or of an `include`/`prepend` inside `class << self` — at startByte, the + // constant's own start, where ingest_relations.h captureRubyMixinBases puts its inherit reference; + // var and typeName empty; importedName=kRubyIncluderMark when the directive sits in a concern's + // `included do … end`, which extends the includer, not the concern. A module body's `extend self` + // has no constant: var="self", at the directive's own start byte (ingest_binds.h + // captureRubyClassMixins). Read by graph.h's class-object lookup ONLY (rubyFqnAncestry). APPENDED, + // as above. }; // The number of LocalBindKind enumerators — the bound readBind validates a cached kind byte against (see kSymKindCount). -inline constexpr std::size_t kLocalBindKindCount = static_cast( LocalBindKind::ElixirImport ) + 1; +inline constexpr std::size_t kLocalBindKindCount = static_cast( LocalBindKind::RubyClassMixin ) + 1; +// The importedName of a Ruby class-object binding that acts on each class INCLUDING the module, not on the module +// (LocalBindKind::RubySingletonDef, RubyClassMixin; parser version 145): written in a concern's `included do`, or a def +// in its `class_methods do`. +inline constexpr std::string_view kRubyIncluderMark = "included"; +inline constexpr std::string_view kRubyClassMethodsMark = "class_methods"; static_assert( enumCountIsExact(), "kLocalBindKindCount must name the LAST LocalBindKind enumerator — move it with the append" ); inline constexpr const char* kFnBindLambdaTarget = "(lambda)"; // parens are illegal in identifiers, so diff --git a/src/quality.h b/src/quality.h index 4b6d81d68..95252d777 100644 --- a/src/quality.h +++ b/src/quality.h @@ -2179,7 +2179,18 @@ inline std::string cacheRootKeyHex( const std::string& root ) // not include this header; it relies on ingest.cpp including quality.h (line 13) before ingest_cache.h, and a reorder // that broke that fails the build on the undeclared name rather than passing. constexpr std::uint32_t kIngestCacheVersionMirror = 27; // MUST equal ingest.cpp's kCacheVersion (gated); 27 = corrected fnScope values, 26 = function-local def scope span (25 = #157 + #150) -constexpr std::uint32_t kIngestParserVerMirror = 133; // MUST equal ingest.cpp's kParserVer (gated) +constexpr std::uint32_t kIngestParserVerMirror = 145; // MUST equal ingest.cpp's kParserVer (gated) + // 145 = 2026-10-04 (Ruby include and extend sides, see kParserVer + // note; kIngestCacheVersionMirror stays 27) + // 142 = 2026-10-04 (the Ruby method-lookup branch on main 2720d1c5: + // above main's 133, train 25's 141 and the branch's carried 130–137, + // see kParserVer note; kIngestCacheVersionMirror stays 27) + // 132 = 2026-10-02 (Ruby typed receivers, see kParserVer note; + // kIngestCacheVersionMirror stays 27) + // 131 = 2026-10-02 (Ruby mixins are ancestors, see kParserVer note; + // kIngestCacheVersionMirror stays 27) + // 130 = 2026-10-01 (Ruby bare-word calls, see kParserVer note; + // kIngestCacheVersionMirror stays 27) // 133 = 2026-10-02 (train 24: above the merged lanes' 130 and 131 extraction // changes and train 23's 132, see kParserVer note; kIngestCacheVersionMirror stays 27) // 132 = 2026-10-02 (train 23: cache-key hygiene above every branch diff --git a/src/resolve.h b/src/resolve.h index 653e31787..216e8bc42 100644 --- a/src/resolve.h +++ b/src/resolve.h @@ -208,7 +208,7 @@ inline IncludeLang includeLangOf( std::string_view path ) noexcept // never a basename fallback); none is deferred, because unlike a Go package path or a C# namespace, // each of these four names a FILE by a rule this tool can evaluate without a build system. { ".sh", IncludeLang::Bash }, { ".bash", IncludeLang::Bash }, { ".zsh", IncludeLang::Bash }, - { ".rb", IncludeLang::Ruby }, + { ".rb", IncludeLang::Ruby }, { ".rake", IncludeLang::Ruby }, { ".jbuilder", IncludeLang::Ruby }, { ".lua", IncludeLang::Lua }, { ".ex", IncludeLang::Elixir }, { ".exs", IncludeLang::Elixir }, // B6.2: `.cs` has NO entry here — it falls through to IncludeLang::Other below, DEFERRED like @@ -1576,20 +1576,13 @@ inline std::uint32_t rubyInnermostOpen( const std::vector& opens, s return i; } -inline RubyConstantIndex buildRubyConstantIndex( const IngestResult& ing ) +// The constant index of every class/module the tree opens, whether or not anything names one (buildRubyConstantIndex +// below is the gated form the resolver reads; graph.h buildRubyTypedReceivers reads this one when that one is empty). +inline RubyConstantIndex rubyConstantIndexOf( const IngestResult& ing ) { PROFILE_SCOPE_DESCRIBE( "resolve/ruby: constant index" ); RubyConstantIndex ix; - bool anySymbolic = false; - for( const Include& inc : ing.includes ) - { - if( inc.isSymbolic ) - { - anySymbolic = true; - break; - } - } - if( !anySymbolic || ing.constOpens.empty() ) + if( ing.constOpens.empty() ) { return ix; } @@ -1706,6 +1699,13 @@ inline RubyConstantIndex buildRubyConstantIndex( const IngestResult& ing ) return ix; } +// The constant index the resolver reads: empty unless a symbolic directive exists (a tree that names no constant needs none). +inline RubyConstantIndex buildRubyConstantIndex( const IngestResult& ing ) +{ + const bool anySymbolic = std::any_of( ing.includes.begin(), ing.includes.end(), []( const Include& inc ) noexcept { return inc.isSymbolic; } ); + return anySymbolic ? rubyConstantIndexOf( ing ) : RubyConstantIndex {}; +} + // Resolve one symbolic Ruby directive to its (offset, count) run in `ix.files` — {0,0} when nothing in the // tree defines it. `memo` is keyed by (the WHOLE nesting chain, written target): two sites with the same // Module.nesting and spelling — the whole of a Rails controller's `include`s, every model's `< ApplicationRecord` @@ -1812,6 +1812,8 @@ struct RubyBaseScope HashMap baseFqn; // reference index → resolved base constant (EMPTY // when the tree opens no such constant); find-only HashMap> classesByFqn; // constant → the Ruby symbols whose own open it is + std::vector mixinRefs; // per reference: 1 ⇒ read off an include/extend/prepend + // (parser version 131); empty when the tree has none // nullptr: not a scoped Ruby base — the byName rule stands. Otherwise the resolved constant (empty = none). const std::string* resolvedBase( std::size_t refIdx ) const noexcept @@ -1885,15 +1887,17 @@ inline HashMap rubySuperclassSites( const Ing // Ruby's lookup of a written base from the derived class open `derived`: `::X` absolute, else innermost-first // along the ENCLOSING nesting (the superclass is evaluated before the class opens), then the top level. Returns -// the constant the tree opens, or empty when it opens none. +// the constant the tree opens, or empty when it opens none — passing over `skip`: the superclass join names the derived +// class's own constant there, as no class is its own superclass (`class SchemaCreation < SchemaCreation` inside +// `module MySQL` names the outer SchemaCreation; parser version 145, test/rubyclassrecvcheck.sh). // Stated floor — a QUALIFIED name is looked up whole: `class X < Mod::B` tries `::Mod::B` innermost-first, // then `Mod::B`. Ruby resolves only the FIRST segment lexically and the rest strictly inside it, so where an // enclosing `Outer::Mod` exists without a `B`, Ruby raises NameError while this falls through to a top-level // `Mod::B`. That code cannot load, so the difference only shows in a tree that is already broken. inline std::string rubyResolveBaseConstant( const HashMap& opened, const std::vector& opens, - std::uint32_t derived, std::string_view written ) + std::uint32_t derived, std::string_view written, std::string_view skip = {} ) { - const auto isOpened = [ &opened ]( const std::string& c ) { return opened.find( c ) != opened.end(); }; + const auto isOpened = [ &opened, skip ]( const std::string& c ) { return c != skip && opened.find( c ) != opened.end(); }; if( rubyConstIsAbsolute( written ) ) { std::string abs( written.substr( 2 ) ); @@ -1928,6 +1932,105 @@ inline bool rubyBaseUnscopedTestSeam() noexcept return on; } +// One symbolic directive's site: its byte and its constant as written. +struct RubyDirectiveSite +{ + std::uint32_t byte = 0; + const std::string* target = nullptr; +}; + +// Every symbolic directive that is not a value use, per file in byte order — the records a MIXIN reference is joined to +// (parser version 131: ingest_relations.h captureRubyMixinBases puts the reference at its constant, the directive sits at +// its `include` call). Stable, so two constants of one `include A, B` keep their source order at their shared byte. +inline std::vector> rubyDirectiveSitesByFile( const IngestResult& ing, const RubyConstantIndex& ix ) +{ + std::vector> byFile( ix.opensByFile.size() ); + for( const Include& inc : ing.includes ) + { + if( inc.isSymbolic && !inc.isValueUse && inc.fileId < byFile.size() ) + { + byFile[ inc.fileId ].push_back( { inc.byte, &inc.target } ); + } + } + for( std::vector& v : byFile ) + { + std::stable_sort( v.begin(), v.end(), []( const RubyDirectiveSite& a, const RubyDirectiveSite& b ) noexcept { return a.byte < b.byte; } ); + } + return byFile; +} + +// The mixin directive a Ruby inherit reference at `r` was read off, or nullptr when it is no mixin (a superclass): the +// nearest directive at or before the reference, strictly inside the same open `derived` (a superclass's directive sits AT +// the open's own start byte, so the strict bound leaves it out), whose constant ends in the reference's name. +inline const std::string* rubyMixinDirectiveOf( const std::vector& sites, const std::vector& opens, + std::uint32_t derived, const Reference& r ) noexcept +{ + auto it = std::upper_bound( sites.begin(), sites.end(), r.startByte, + []( std::uint32_t b, const RubyDirectiveSite& s ) noexcept { return b < s.byte; } ); + while( it != sites.begin() ) + { + --it; + if( it->byte <= opens[ derived ].startByte ) + { + return nullptr; + } + const std::string_view w = *it->target; + const std::size_t cut = w.rfind( "::" ); + if( rubyInnermostOpen( opens, it->byte ) == derived && ( cut == std::string_view::npos ? w : w.substr( cut + 2 ) ) == r.calleeName ) + { + return it->target; + } + } + return nullptr; +} + +// Ruby's lookup of a MIXIN's constant: an `include M` in a class body is evaluated INSIDE the class, so the class's own +// scope is searched first (`Outer::M` for `class Outer; include M`), then the enclosing nesting and the top level +// exactly as a superclass's (rubyResolveBaseConstant). +inline std::string rubyResolveMixinConstant( const HashMap& opened, const std::vector& opens, + std::uint32_t derived, std::string_view written ) +{ + if( !rubyConstIsAbsolute( written ) ) + { + std::string inside = opens[ derived ].fqn; + inside += "::"; + inside += written; + if( opened.find( inside ) != opened.end() ) + { + return inside; + } + } + return rubyResolveBaseConstant( opened, opens, derived, written ); +} + +// The mixin half of the join (parser version 131), run before the superclass half below: each Ruby inherit reference read +// off an `include`/`extend`/`prepend` gets its constant from rubyResolveMixinConstant. The superclass half then skips the +// references this one scoped. The test seam leaves every reference to the superclass half, which discloses it. +inline void rubyScopeMixinReferences( RubyBaseScope& sc, const IngestResult& ing ) +{ + if( rubyBaseUnscopedTestSeam() ) + { + return; + } + const std::vector> sites = rubyDirectiveSitesByFile( ing, sc.ix ); + for( std::size_t i = 0; i < ing.references.size(); ++i ) + { + const Reference& r = ing.references[ i ]; + if( !r.isInherit || r.lang != Lang::Ruby || r.fileId >= sc.ix.opensByFile.size() ) + { + continue; + } + const std::vector& opens = sc.ix.opensByFile[ r.fileId ]; + const std::uint32_t derived = rubyInnermostOpen( opens, r.startByte ); + if( const std::string* mixin = ( derived == kNoFile ) ? nullptr : rubyMixinDirectiveOf( sites[ r.fileId ], opens, derived, r ) ) + { + sc.baseFqn.emplace( std::uint32_t( i ), rubyResolveMixinConstant( sc.opened, opens, derived, *mixin ) ); + sc.mixinRefs.resize( ing.references.size(), 0 ); // sized once, on the first mixin + sc.mixinRefs[ i ] = 1; + } + } +} + // Joins each Ruby inherit reference to the superclass directive at its derived class's open and records the // resolved base constant in `sc.baseFqn`. A reference the join cannot place is left out — the byName rule stands. inline void rubyScopeBaseReferences( RubyBaseScope& sc, const IngestResult& ing ) @@ -1940,6 +2043,10 @@ inline void rubyScopeBaseReferences( RubyBaseScope& sc, const IngestResult& ing { continue; } + if( sc.baseFqn.find( std::uint32_t( i ) ) != sc.baseFqn.end() ) + { + continue; // a mixin, scoped by rubyScopeMixinReferences + } const std::vector& opens = sc.ix.opensByFile[ r.fileId ]; const std::uint32_t derived = rubyInnermostOpen( opens, r.startByte ); const auto site = ( derived == kNoFile || rubyBaseUnscopedTestSeam() ) ? sites.end() : sites.find( ( std::uint64_t( r.fileId ) << 32 ) | opens[ derived ].startByte ); @@ -1949,7 +2056,7 @@ inline void rubyScopeBaseReferences( RubyBaseScope& sc, const IngestResult& ing "Ruby inherit reference with no superclass directive at its class open: base left on the byName rule" ); continue; } - sc.baseFqn.emplace( std::uint32_t( i ), rubyResolveBaseConstant( sc.opened, opens, derived, *site->second ) ); + sc.baseFqn.emplace( std::uint32_t( i ), rubyResolveBaseConstant( sc.opened, opens, derived, *site->second, opens[ derived ].fqn ) ); } } @@ -1978,6 +2085,7 @@ inline RubyBaseScope buildRubyBaseScope( const IngestResult& ing ) sc.classesByFqn[ *fqn ].push_back( s.id ); // ids ascending → each list is in id order } } + rubyScopeMixinReferences( sc, ing ); // parser version 131: mixins first, then the superclass half skips them rubyScopeBaseReferences( sc, ing ); return sc; } @@ -6080,8 +6188,9 @@ struct Narrower const bool isThisSelf = ( r.recv == RecvKind::ThisObj ); // Ruby rides the bare arm too: a receiver-less `m(args)` inside a method is an implicit-self send — - // the language has no other reading of it (a bare `m` with neither receiver nor parens is a local - // read and is never captured; queries/ruby/tags.scm). test/rubyscopecheck.sh, Rule 1 arms. + // the language has no other reading of it — and so is a bare `m` with neither receiver nor parens + // wherever Ruby's lexical rule says it is no local (ingest_binds.h captureRubyBareCalls, parser + // version 130). test/rubyscopecheck.sh, Rule 1 arms; test/rubybarecallcheck.sh. const bool isCish = ( r.lang == Lang::Cpp || r.lang == Lang::ObjC || r.lang == Lang::Ruby ); const bool bareCish = isCish && ( r.recv == RecvKind::None ); // C++ unqualified member-or-namespace lookup; Ruby implicit self if( !isThisSelf && !bareCish ) diff --git a/src/taskroute.h b/src/taskroute.h index 9e7110cb4..cc49a51cb 100644 --- a/src/taskroute.h +++ b/src/taskroute.h @@ -553,7 +553,7 @@ inline constexpr std::string_view kCodeExtensions[] = { // crawl itself admits it (src/ingest_crawl.h's kLangTable) — a task description naming a .hxx FILE:LINE // seed used to be recognized nowhere even though the file it names now indexes fine. ".cpp", ".cc", ".cxx", ".h", ".hpp", ".hh", ".hxx", ".c", ".py", ".ts", ".tsx", ".js", ".jsx", ".go", ".rs", - ".java", ".rb", ".swift", ".cs", ".m", ".mm", ".cu", ".cuh", ".metal", ".kt", + ".java", ".rb", ".rake", ".jbuilder", ".swift", ".cs", ".m", ".mm", ".cu", ".cuh", ".metal", ".kt", }; inline bool looksLikeFileToken( std::string_view token ) noexcept diff --git a/test/builtinbindcheck.sh b/test/builtinbindcheck.sh index 65dcad189..3c504faa4 100755 --- a/test/builtinbindcheck.sh +++ b/test/builtinbindcheck.sh @@ -249,8 +249,10 @@ echo "=== (I) the header, its legend and the census agree ===" HDR="$( stats "$TMP/map.xml" )" [ "$( gauge "$HDR" declined )" = 15 ] && ok "(I) header declined=15 (8 gate declines outside mono/: four Python .get, JS Map.get and Array.push, TS Map.get, Ruby Hash#fetch; mono/: 4 ladder declines kept + pop, popitem, shift)" \ || no "(I) header declined= should be 15: ${HDR:-no stats comment}" -[ "$( gauge "$HDR" external )" = 1 ] && ok "(I) header external=1: raw.decode(\"utf-8\") can reach no in-repo definition (the only one is a closure)" \ - || no "(I) header external= should be 1: ${HDR:-no stats comment}" +# external=3 since parser version 137 (test/rubyclassrecvcheck.sh): RbPool.new and LinkedList.new are Class#new on a class +# the tree defines no initialize for, which runs BasicObject's — outside the tree +[ "$( gauge "$HDR" external )" = 3 ] && ok "(I) header external=3: raw.decode(\"utf-8\") can reach no in-repo definition (the only one is a closure); RbPool.new and LinkedList.new reach no in-tree initialize" \ + || no "(I) header external= should be 3: ${HDR:-no stats comment}" legend_of "$TMP/map.xml" | grep -q 'hdr:declined=also-counts-builtin-type-method-calls' \ && ok "(I) the full map legend defines hdr:declined= including the builtin-name clause" \ || no "(I) the full map legend does not name the builtin-name decline under hdr:declined=" diff --git a/test/callformcheck.sh b/test/callformcheck.sh index d7ce95db7..dcab74b86 100755 --- a/test/callformcheck.sh +++ b/test/callformcheck.sh @@ -407,13 +407,20 @@ uses ruby member_fn 1 "2. method call through a receiver" uses ruby receiver_fn 1 "3. module-function call, dot receiver" uses ruby colon_fn 1 "4. :: used as the method-call operator" uses ruby deep_fn 1 "5. scope_resolution receiver, then a dot call" -# 6. ABSENT BY DESIGN, stated in queries/ruby/tags.scm's own header: a bare, receiver-less, -# paren-less call is textually indistinguishable from a local-variable read. -# The presence guard here is the one V5 defeated: it deleted this very line and both arms below -# stayed green. Anchored so it matches the CALL inside caller_absent, never the `def` on line 10. -fixtureHasRe ruby/main.rb '^ bare_noparen_fn$' "6. the bare paren-less CALL is still WRITTEN (V5's deletion target)" -uses ruby bare_noparen_fn 0 "6. ABSENT BY DESIGN: bare paren-less call, pinned at literal 0" -probeBlind ruby caller_absent bare_noparen_fn "6. …and absent at EXTRACTION, by the grammar's own disclosure" +# 6. A bare, receiver-less, paren-less name parses as an (identifier), the same node a local read +# is. Until parser 130 it was ABSENT BY DESIGN here (pinned at literal 0); ingest_binds.h:: +# captureRubyBareCalls now applies Ruby's own lexical rule, so with no binding before it, it is a call. +# 7. The same name after `bare_noparen_fn = 7` in the same def is a local read: ABSENT BY DESIGN, and +# caller_absent's reference list is legitimately empty. The presence guard is the one V5 defeated +# (it deleted the bare line and both absence arms stayed green); anchored on the indentation, it +# matches the bare line in BOTH callers and never the `def` on line 11 — both are counted. +fixtureHasRe ruby/main.rb '^ bare_noparen_fn = 7$' "7. the binding that makes row 7 a local is still WRITTEN" +[ "$( grep -cE '^ bare_noparen_fn$' "$FIX/ruby/main.rb" )" = 2 ] \ + && ok "[fixture] ruby/main.rb still spells the bare \`bare_noparen_fn\` twice — 6. the call and 7. the local read" \ + || no "[fixture] ruby/main.rb no longer spells the bare \`bare_noparen_fn\` twice — row 6 or row 7 is now VACUOUS" +uses ruby bare_noparen_fn 1 "6. bare paren-less call, one edge — caller_bare's; caller_absent's read is a local" +probeSees ruby caller_bare bare_noparen_fn "6. …and it EXTRACTS as a receiver-less call" +probeBlind ruby caller_absent bare_noparen_fn "7. ABSENT BY DESIGN: a read after an assignment binds the name is a local" echo echo "=== Swift — test/callformfix/swift/main.swift ===" diff --git a/test/callformfix/ruby/main.rb b/test/callformfix/ruby/main.rb index 735104b75..5f1ffaa6c 100644 --- a/test/callformfix/ruby/main.rb +++ b/test/callformfix/ruby/main.rb @@ -1,7 +1,8 @@ # RUBY CALL-FORM MATRIX fixture — one line per call SPELLING the grammar distinguishes. -# Expected counts are literals read off this file. The ABSENT row is absent BY DESIGN, stated in -# queries/ruby/tags.scm's own header: a bare, receiver-less, parenthesis-less call is -# indistinguishable from a local-variable read, so it is deliberately not captured. +# Expected counts are literals read off this file. A bare, receiver-less, parenthesis-less name is +# a call or a local read by Ruby's LEXICAL rule (ingest_binds.h::captureRubyBareCalls): it is a +# local exactly when an assignment earlier in the same scope binds it. Row 6 is the call; row 7 is +# the same name after such an assignment, and is absent BY DESIGN. def bare_paren_fn 1 @@ -43,8 +44,13 @@ def caller a end +def caller_bare + # 6. a bare, receiver-less, paren-less call — no binding of the name precedes it in this def + bare_noparen_fn +end + def caller_absent - # 6. ABSENT BY DESIGN — a bare, receiver-less, paren-less call parses as an identifier and is - # indistinguishable from a local-variable read. Capturing it would spray every local name. + # 7. ABSENT BY DESIGN — the assignment makes the name a local, so the second line is a read + bare_noparen_fn = 7 bare_noparen_fn end diff --git a/test/qschemetrip.hash b/test/qschemetrip.hash index 7c9b97d3d..751d7a351 100644 --- a/test/qschemetrip.hash +++ b/test/qschemetrip.hash @@ -1 +1 @@ -a7f2ec25d4432882c7d7f41c95091316329d11df25a45d70ed4be3581c9388db +2aaf4390cb877cdcdd4dab2d516218aa2c46350f2cafdf582d1fe33d4f692eeb diff --git a/test/qschemetripcheck.sh b/test/qschemetripcheck.sh index 725e18f98..3395c780c 100755 --- a/test/qschemetripcheck.sh +++ b/test/qschemetripcheck.sh @@ -34,6 +34,14 @@ SRC="$ROOT/src/quality.h" ING="$ROOT/src/ingest_cache.h" # extraction-identity constants moved here (2026-08-29 ingest.cpp section split); the hashed CONCAT label keeps its historical spelling so the pin holds PIN="$ROOT/test/qschemetrip.hash" # RE-PIN LOG (the pin is a bare hash, so its justification has to live here). +# 2026-10-04, feat/ruby-method-lookup (PR #373 review: include and extend sides): RE-PINNED with UPDATE_GOLDEN=1 (hash +# 2aaf4390cb…4f692eeb). kParserVer 142 -> 145: a Ruby extraction change (RubyClassMixin bindings, the includer mark), above +# main's 143 and 144; kCacheVersion stays 27, kQSnapCacheScheme stays 16; quality.h's kIngestParserVerMirror moves with +# it. The train may renumber. +# 2026-10-04, feat/ruby-method-lookup rebased onto main 2720d1c5: RE-DERIVED ONCE with UPDATE_GOLDEN=1 (hash +# 494bccc1ca…be6c147e). kParserVer 133 -> 142: the branch's eight Ruby extraction steps carried 130–137, and 142 is +# above those, main's 133 and train 25's 141; kCacheVersion stays 27, kQSnapCacheScheme stays 16; quality.h's +# kIngestParserVerMirror moves with it. The train may renumber. # 2026-10-02, train 24 (recall, answer-honesty, #368, contrib-checklist, hygiene-orphans, readme-terminality merged): # RE-DERIVED ONCE on the merged tree with UPDATE_GOLDEN=1 (hash a7f2ec25d4…9388db). kParserVer 132 -> 133: above the two # merged lanes' extraction changes (130 body-less C/C++ type specifier span; 131 TS/TSX await/unary type-argument diff --git a/test/regression.sh b/test/regression.sh index 6c51ed8af..a409e334d 100755 --- a/test/regression.sh +++ b/test/regression.sh @@ -277,7 +277,7 @@ else RIPWIRE_BIN="$BIN" bash "$ROOT/test/codexdoctorcheck.sh" 2>&1 | sed 's/^/ | /' fi # retired: cacheexclkeycheck — the per-configuration auto-cache key it pinned is a registered NEGATIVE (docs/EVALS.md, "The auto-cache key ignores --exclude", RUN 2026-09-03: a 158K-file root with >= 12 gate configurations thrashed the 2 GiB sweep); the retry design keeps ONE superset blob per root and will bring its own gate -for _g in a9disclosurecheck abicheck accessshapecheck ackonlycheck adaptivecheck adaptivecutshapecheck affectedcheck agentloopclaudecheck agentloopcodexcheck agentloopeditsuitecheck agentloopfollowupcheck agentloopgradercheck agentlooplockcheck agentloopopencodecheck agentsurfacecheck agenttablecheck aiderbytescheck anchorbodycheck anchorcheck archcheck archmetricscheck argvdiffcheck arisefollowupcheck ariseshimcheck aritycheck artifactcheck astqueryregexcheck astrocheck atcheck atomscheck attrvocabcheck baselinecheck baselinedirtycheck baselineportcheck bashsourcecheck batchcheck binoverridecheck blindspotcheck bm25boundcheck bm25check bodiesshowncheck bodydialectcheck budgetpolicycheck buildtypestampcheck builtinbindcheck bundleidcheck cachefuzzcheck cachehashcheck cacheidentitycheck cacheisolationcheck cachelintcheck cacheoffsetcheck cachereservecheck cachesplitcheck callerscheck callformcheck callsrankordercheck candheadcheck candidatescheck canoncheck capdisclosurecheck capsweepcheck ccheck ccjsoncheck ceilingverdictcheck chacheck chaconecheck chainguardcheck chainidcheck childwalkscalecheck churndecaycheck churnjoincheck churnjsonstampcheck claudeconfigdircheck clicheck clonebandcheck clonecachecheck clonededupcheck cloneidiomcheck clonelexcheck clsrecvcheck cochangeboostcheck cochangecliocheck cochangesurprisecheck codexinstallhonestycheck codexplugincheck codexwrapcheck collectioncapcheck columnarattrcheck columnarcheck columnarcommacheck commentcoherencecheck commenttokencheck communitydrillcheck communitylabelcheck compactlegendcheck compactroutecheck completecheck composelangcheck connectcheck connectcorecheck connectjoincheck constcheck contextratiocheck coplintcheck cppbenchcheck cppoperatorcheck cppqualcheck cpptmplscopecheck crashsweepcheck crawlescapecheck crossdirincludecheck crossrefcheck crossrefdegradecheck csharpcheck csharpcondcheck cudacheck cyclecutcheck dartcheck deadcheck deadfiltercheck deadprecisioncheck deckcheck deckclaimcheck declinecheck declinedlistcheck decltodefcheck deeptailcheck defaultceilingcheck defoverdeclcheck degradedhintcheck dependencypincheck deplangscheck depsprecisecheck detailcheck diagnoticecheck didyoumeancheck dispatchordercheck dmmcheck docanchorcheck docdemotecheck docdriftcheck docdriftcommentcheck docmdcachecheck docmentioncheck docscommandscheck doctorcheck donelegendcheck droppedpositivecheck duprowcheck dynmapsimdcheck editcheckanswercheck editcheckcheck editcheckdeclcheck editchecknotecheck edithandlehintcheck editpayloadbinarycheck editplancheck editplanpayloadconfinecheck editplanrecheckcheck editplanrollbackmsgcheck editpreviewcheck editroundtripcheck edittargetfileabscheck eliximportcheck elixircheck elixirnamearitycheck elixirsemanticcheck emitescapecheck emittertruthcheck emptycorpuscheck emptyvaluerefusecheck ensembleavailcheck ensemblecheck enumtablecheck essentialcxcheck estchargecheck evalcheck evictioncheck exemplarcheck exemplarconfcheck exercisescheck expandbodyfirstcheck expandcallscheck expandmodecheck expandrangecheck expandsibscheck expandtokencheck expandtopk0check extentcheck externalvetocheck fficheck fieldaffinitycheck fieldidcheck fieldnarrowcheck fieldusescheck filerootcheck fileselectorrefusecheck fillordercheck fixedbufsweep flagscheck flagsnoisecheck flagsurfacecheck flagtablecheck flipcheck floormarkcheck fnliteralcheck fnptrcheck forautobodycheck forblowupcheck forbudgetmonotoncheck forcalibfactscheck forcompresscheck fordisclosurecheck forhdrshapecheck forlenscheck formatgatecheck formaxtokenscheck fornotesbudgetcheck fornotesjsoncheck forrankordercheck forrootlegendcheck forsectioncollapsecheck forwidencheck freshclonecheck freshnesscheck g1configcheck gateabilitycheck gatecountcheck gateexitcheck gdscriptcheck genrecallcheck gitenvhermeticcheck githardencheck gitignorecheck gitquotepathcheck gitstampcheck goinstcheck gointerfacecheck graphlegendbudgetcheck graphqueryrefusecheck grepanchorcheck grepandcheck grepbytescheck grepcheck grepcontextcheck grepcorpuscheck grepfastcheck grepfollowupcheck grepignorecheck grepscancheck grepseamcheck greptiercheck guardmsgcheck hasacheck hazardpatterncheck headbinstagecheck headsnapcachecheck helpbudgetcheck hermesinstallcheck historyoraclecheck hookcheck hostilecheck hotspotsincecheck htmlcolorcheck htmlhostcheck htmlrendercheck identitycheck impactdepthcheck impactimportcheck impactpartitioncheck importcapcheck importnarrowcheck includeanglecheck includeprecisecheck indexoutcheck infraportcheck isolateprovenancecheck javamethodrefcheck javarubycheck jslangcheck jsmetricscheck jsnestedcheck jsoncheck jsonlangcheck jsonparitycheck jsonredactcheck jsonrefusallegendcheck jsonwalkcheck jsshapecheck jsverbscheck jsxcallcheck knownitemcheck kotlincheck landingcheck langcensuscheck langcheck layerquerycheck layoutcheck lb3namecheck legendcostcheck legendcoveragecheck legenddriftcheck legendrefcheck legobundlecheck legocheck liftdisclosurecheck limitstablecheck lintbudgetcheck lintcatalogcheck lintcheck lintdedupcheck lintpayloadcapcheck lintprecisioncheck lintrulescheck lintscopecheck lintselectcheck listingpagingcheck localitycheck localscountcheck loopconservationcheck lpincheck luacheck luarequirecheck macroedgecheck macroreparsecheck manifestcheck mapdatasectioncheck mapdiffcheck matchcapturecheck matchgrammarcheck maxfilesizecheck mcpattrparitycheck mcpaudit4hardencheck mcpclidiffcheck mcpcodexmetacheck mcpcontractcheck mcpdegradedhintcheck mcpeditcheck mcpeditkindcheck mcpeditmodecheck mcpeditpresencecheck mcpeditracecheck mcpflagshipcheck mcpforparitycheck mcpframehonestycheck mcpgrepdegradedcheck mcphandlecheck mcpincrementalcheck mcpmanifestcheck mcprangeedgecheck mcpreadloopcheck mcpredactcheck mcpreloadcheck mcpremotecheck mcprobustcheck mcpslicecheck mcpstalecheck mcpstdiolinecapcheck mcpstrictschemacheck mcptoolprunecheck mcptoolsubsetcheck mcptranchecheck mcptwinclaimscheck mcpverbscheck mcpw2fixcheck mcpw3fixcheck mcpwatchercheck mdembedcheck mdsectioncheck memguardcheck mentioncapcheck mentioncheck mentionsverbcheck mergechurncheck mergescoutcheck mergescoutlonglinecheck metalcheck meterdisclosurecheck metricscheck modifierguardcheck moduleconstcheck morecontractcheck mrowalkcheck multirootcheck multiswecheck namedfileinputcheck nameinfocheck namingcalibrationcheck namingconsistencycheck naminglenscheck naminglocalscheck narrowcheck narrowlangcheck neighbourcapcheck nestedimportcheck nestedqualcheck nestprofilecheck nextverbcheck noaliascheck nodekindcheck nongitqmetricscheck nonlocalstatecheck notecanoncheck notescheck notesdegradecheck nsfiltercheck nulbytecheck numericrefusecheck objcfieldcheck objcsniffcheck opencodewrapcheck optremarkscheck optremarkshotcheck ordercheck osswitchcheck oswin32logiccheck outlinecheck overbudgetcommentcheck ownerscheck packcallersharecheck packtaskcheck packtaskmonotoncheck packtaskquotacheck padscalecheck paginationcheck pagingsweepcheck panellegendcheck pargatescheck parsehealthcheck partitioncheck patterncheck perfharnesscheck phpcheck pincensuscheck planlanescheck planlintcheck pmccheck portablebuildcheck portablecachecheck postingscheck ppaltcheck ppdeadrolescheck pranchorcheck prbudgetcheck prcheck prcontextcheck prconvergecheck precedencecheck preproccondcheck preprocdeadscalecheck prmaskanchorcheck prnestedcapcheck probecheck propcostcheck prrefsafecheck prrenamecheck pyimportprecisecheck pymodulealiascheck pyshapecheck qackconcurrencycheck qackorigincheck qbaselineproducercheck qchurncheck qchurnmemocheck qddialscheck qdrefpaircheck qextractionkeycheck qoriginoraclecheck qrevtokencheck qrowlocatorcheck qschemetripcheck qsnapcachecheck qsnapprefetchcheck qsnapproducercheck qualifiedresolvecheck qualitycheck qualitycrosslangcheck qualityexcludecheck qualitykeycheck qualitykindscheck qualityorigincheck qualitypanelcheck qualityscopecheck qualitysignalcheck qualitystalecheck qualitysymcheck qualnewcheck querycheck queryfilescancheck racymtimecheck radixsimdcheck rangecomposecheck rankbycheck reachcheck readabilitycheck readmedriftcheck readmeexamplecheck recallanchorcheck recallboundarycheck recallbudgetcheck recallbufcheck recallevalcheck recallparitycheck recallpassagecheck recallrankdepthcheck recallrelcheck recalltablecheck recalltotalcheck receiptpostcheck recentscopecheck redactcheck redactfixcheck refusaltailcheck regexbombcheck regexcheck regexguardcheck regexrefusecheck registermacrocheck relevancefloorcheck relinkcheck reportcheck resolvecheck resolverhonestycheck retrievalqualitycheck reusefirstworkflowcheck ripwirepubliccheck rootrelcheck rootrelemitcheck rootspellingcheck routecheck routeedgecheck routehookcheck routeoncecheck routingreportcheck rubyargcheck rubyattrscheck rubyconstcheck rubydescribedclasscheck rubyinheritcheck rubymetricscheck rubyrecvcheck rubyrecvnarrowcheck rubyrequirecheck rubyscopecheck rubysettercheck runhintcheck runtracecheck rustanccheck rustimportprecisecheck rustqualcheck safedeletecheck sarifcheck savecachecheck scipcheck scipjoincheck scorecardcheck scoutheadconflictcheck scoutkeycheck scroundtripcheck seedboundscheck selectorchaincheck selectorhonestycheck selectorrefusecheck selectorscopecheck selfcheckcheck selfcontainedcheck shadowcheck shallowhistorycheck shapingflagcheck shellgateindexcheck showcasecapturecheck sibliftcheck sidecarsymlinkcheck sigredactcheck sincecheck sincecochangecheck sincewindowcheck singledefcheck situdiffcheck situshapecheck skilldescbudgetcheck skillevalcheck skillevalsplitcheck skillinstallcheck skillroutingjudgedcheck skillscanreadcheck skilltruthcheck skipclassifycheck skippedcheck skipreasoncheck slicecheck slicediffcheck sliceflowcheck sliceflowsenscheck spectimingcheck staleackcheck statgatecheck stdqualcheck strkerncheck structlayoutcheck sublistcountcheck substrfiltercheck subtokencheck svectorcheck swiftcheck swiftmemberscheck swiftshapecheck taskechocheck tempfilesymlinkcheck termmargincheck testedreachcheck testgatecheck testgatelegendbudgetcheck testgatepagecheck testgaterefusecheck testmacrocheck testrowruncheck testscopecheck textdocscheck timsortcheck tokenbudgetcheck tomllangcheck toolcallroutecheck tornreadcheck traceasanlinearcheck tracecheck tracehandoffcapcheck tracehopcheck traceminecheck treecheck truncvocabcheck tsimportprecisecheck tsshapecheck type3check type3clonecheck typerefcheck unreachablecheck unresolvedcheck usescheck usesselectorcheck usingdeclcheck utf8scrubcheck vendoredassetcheck vendoredbundlecheck vendorpatchcheck verifycheck versioncheck w2verbscheck w3fixbudgetcheck w3fixlegendcheck weaksignalcheck withgraphcheck withprofilecheck worktreeleakcheck wrapverbscheck writetargetcheck xmlwellformed yamllangcheck zonecheck zoneconsistencycheck zoomcheck; do +for _g in a9disclosurecheck abicheck accessshapecheck ackonlycheck adaptivecheck adaptivecutshapecheck affectedcheck agentloopclaudecheck agentloopcodexcheck agentloopeditsuitecheck agentloopfollowupcheck agentloopgradercheck agentlooplockcheck agentloopopencodecheck agentsurfacecheck agenttablecheck aiderbytescheck anchorbodycheck anchorcheck archcheck archmetricscheck argvdiffcheck arisefollowupcheck ariseshimcheck aritycheck artifactcheck astqueryregexcheck astrocheck atcheck atomscheck attrvocabcheck baselinecheck baselinedirtycheck baselineportcheck bashsourcecheck batchcheck binoverridecheck blindspotcheck bm25boundcheck bm25check bodiesshowncheck bodydialectcheck budgetpolicycheck buildtypestampcheck builtinbindcheck bundleidcheck cachefuzzcheck cachehashcheck cacheidentitycheck cacheisolationcheck cachelintcheck cacheoffsetcheck cachereservecheck cachesplitcheck callerscheck callformcheck callsrankordercheck candheadcheck candidatescheck canoncheck capdisclosurecheck capsweepcheck ccheck ccjsoncheck ceilingverdictcheck chacheck chaconecheck chainguardcheck chainidcheck childwalkscalecheck churndecaycheck churnjoincheck churnjsonstampcheck claudeconfigdircheck clicheck clonebandcheck clonecachecheck clonededupcheck cloneidiomcheck clonelexcheck clsrecvcheck cochangeboostcheck cochangecliocheck cochangesurprisecheck codexinstallhonestycheck codexplugincheck codexwrapcheck collectioncapcheck columnarattrcheck columnarcheck columnarcommacheck commentcoherencecheck commenttokencheck communitydrillcheck communitylabelcheck compactlegendcheck compactroutecheck completecheck composelangcheck connectcheck connectcorecheck connectjoincheck constcheck contextratiocheck coplintcheck cppbenchcheck cppoperatorcheck cppqualcheck cpptmplscopecheck crashsweepcheck crawlescapecheck crossdirincludecheck crossrefcheck crossrefdegradecheck csharpcheck csharpcondcheck cudacheck cyclecutcheck dartcheck deadcheck deadfiltercheck deadprecisioncheck deckcheck deckclaimcheck declinecheck declinedlistcheck decltodefcheck deeptailcheck defaultceilingcheck defoverdeclcheck degradedhintcheck dependencypincheck deplangscheck depsprecisecheck detailcheck diagnoticecheck didyoumeancheck dispatchordercheck dmmcheck docanchorcheck docdemotecheck docdriftcheck docdriftcommentcheck docmdcachecheck docmentioncheck docscommandscheck doctorcheck donelegendcheck droppedpositivecheck duprowcheck dynmapsimdcheck editcheckanswercheck editcheckcheck editcheckdeclcheck editchecknotecheck edithandlehintcheck editpayloadbinarycheck editplancheck editplanpayloadconfinecheck editplanrecheckcheck editplanrollbackmsgcheck editpreviewcheck editroundtripcheck edittargetfileabscheck eliximportcheck elixircheck elixirnamearitycheck elixirsemanticcheck emitescapecheck emittertruthcheck emptycorpuscheck emptyvaluerefusecheck ensembleavailcheck ensemblecheck enumtablecheck essentialcxcheck estchargecheck evalcheck evictioncheck exemplarcheck exemplarconfcheck exercisescheck expandbodyfirstcheck expandcallscheck expandmodecheck expandrangecheck expandsibscheck expandtokencheck expandtopk0check extentcheck externalvetocheck fficheck fieldaffinitycheck fieldidcheck fieldnarrowcheck fieldusescheck filerootcheck fileselectorrefusecheck fillordercheck fixedbufsweep flagscheck flagsnoisecheck flagsurfacecheck flagtablecheck flipcheck floormarkcheck fnliteralcheck fnptrcheck forautobodycheck forblowupcheck forbudgetmonotoncheck forcalibfactscheck forcompresscheck fordisclosurecheck forhdrshapecheck forlenscheck formatgatecheck formaxtokenscheck fornotesbudgetcheck fornotesjsoncheck forrankordercheck forrootlegendcheck forsectioncollapsecheck forwidencheck freshclonecheck freshnesscheck g1configcheck gateabilitycheck gatecountcheck gateexitcheck gdscriptcheck genrecallcheck gitenvhermeticcheck githardencheck gitignorecheck gitquotepathcheck gitstampcheck goinstcheck gointerfacecheck graphlegendbudgetcheck graphqueryrefusecheck grepanchorcheck grepandcheck grepbytescheck grepcheck grepcontextcheck grepcorpuscheck grepfastcheck grepfollowupcheck grepignorecheck grepscancheck grepseamcheck greptiercheck guardmsgcheck hasacheck hazardpatterncheck headbinstagecheck headsnapcachecheck helpbudgetcheck hermesinstallcheck historyoraclecheck hookcheck hostilecheck hotspotsincecheck htmlcolorcheck htmlhostcheck htmlrendercheck identitycheck impactdepthcheck impactimportcheck impactpartitioncheck importcapcheck importnarrowcheck includeanglecheck includeprecisecheck indexoutcheck infraportcheck isolateprovenancecheck javamethodrefcheck javarubycheck jslangcheck jsmetricscheck jsnestedcheck jsoncheck jsonlangcheck jsonparitycheck jsonredactcheck jsonrefusallegendcheck jsonwalkcheck jsshapecheck jsverbscheck jsxcallcheck knownitemcheck kotlincheck landingcheck langcensuscheck langcheck layerquerycheck layoutcheck lb3namecheck legendcostcheck legendcoveragecheck legenddriftcheck legendrefcheck legobundlecheck legocheck liftdisclosurecheck limitstablecheck lintbudgetcheck lintcatalogcheck lintcheck lintdedupcheck lintpayloadcapcheck lintprecisioncheck lintrulescheck lintscopecheck lintselectcheck listingpagingcheck localitycheck localscountcheck loopconservationcheck lpincheck luacheck luarequirecheck macroedgecheck macroreparsecheck manifestcheck mapdatasectioncheck mapdiffcheck matchcapturecheck matchgrammarcheck maxfilesizecheck mcpattrparitycheck mcpaudit4hardencheck mcpclidiffcheck mcpcodexmetacheck mcpcontractcheck mcpdegradedhintcheck mcpeditcheck mcpeditkindcheck mcpeditmodecheck mcpeditpresencecheck mcpeditracecheck mcpflagshipcheck mcpforparitycheck mcpframehonestycheck mcpgrepdegradedcheck mcphandlecheck mcpincrementalcheck mcpmanifestcheck mcprangeedgecheck mcpreadloopcheck mcpredactcheck mcpreloadcheck mcpremotecheck mcprobustcheck mcpslicecheck mcpstalecheck mcpstdiolinecapcheck mcpstrictschemacheck mcptoolprunecheck mcptoolsubsetcheck mcptranchecheck mcptwinclaimscheck mcpverbscheck mcpw2fixcheck mcpw3fixcheck mcpwatchercheck mdembedcheck mdsectioncheck memguardcheck mentioncapcheck mentioncheck mentionsverbcheck mergechurncheck mergescoutcheck mergescoutlonglinecheck metalcheck meterdisclosurecheck metricscheck modifierguardcheck moduleconstcheck morecontractcheck mrowalkcheck multirootcheck multiswecheck namedfileinputcheck nameinfocheck namingcalibrationcheck namingconsistencycheck naminglenscheck naminglocalscheck narrowcheck narrowlangcheck neighbourcapcheck nestedimportcheck nestedqualcheck nestprofilecheck nextverbcheck noaliascheck nodekindcheck nongitqmetricscheck nonlocalstatecheck notecanoncheck notescheck notesdegradecheck nsfiltercheck nulbytecheck numericrefusecheck objcfieldcheck objcsniffcheck opencodewrapcheck optremarkscheck optremarkshotcheck ordercheck osswitchcheck oswin32logiccheck outlinecheck overbudgetcommentcheck ownerscheck packcallersharecheck packtaskcheck packtaskmonotoncheck packtaskquotacheck padscalecheck paginationcheck pagingsweepcheck panellegendcheck pargatescheck parsehealthcheck partitioncheck patterncheck perfharnesscheck phpcheck pincensuscheck planlanescheck planlintcheck pmccheck portablebuildcheck portablecachecheck postingscheck ppaltcheck ppdeadrolescheck pranchorcheck prbudgetcheck prcheck prcontextcheck prconvergecheck precedencecheck preproccondcheck preprocdeadscalecheck prmaskanchorcheck prnestedcapcheck probecheck propcostcheck prrefsafecheck prrenamecheck pyimportprecisecheck pymodulealiascheck pyshapecheck qackconcurrencycheck qackorigincheck qbaselineproducercheck qchurncheck qchurnmemocheck qddialscheck qdrefpaircheck qextractionkeycheck qoriginoraclecheck qrevtokencheck qrowlocatorcheck qschemetripcheck qsnapcachecheck qsnapprefetchcheck qsnapproducercheck qualifiedresolvecheck qualitycheck qualitycrosslangcheck qualityexcludecheck qualitykeycheck qualitykindscheck qualityorigincheck qualitypanelcheck qualityscopecheck qualitysignalcheck qualitystalecheck qualitysymcheck qualnewcheck querycheck queryfilescancheck racymtimecheck radixsimdcheck rangecomposecheck rankbycheck reachcheck readabilitycheck readmedriftcheck readmeexamplecheck recallanchorcheck recallboundarycheck recallbudgetcheck recallbufcheck recallevalcheck recallparitycheck recallpassagecheck recallrankdepthcheck recallrelcheck recalltablecheck recalltotalcheck receiptpostcheck recentscopecheck redactcheck redactfixcheck refusaltailcheck regexbombcheck regexcheck regexguardcheck regexrefusecheck registermacrocheck relevancefloorcheck relinkcheck reportcheck resolvecheck resolverhonestycheck retrievalqualitycheck reusefirstworkflowcheck ripwirepubliccheck rootrelcheck rootrelemitcheck rootspellingcheck routecheck routeedgecheck routehookcheck routeoncecheck routingreportcheck rubyargcheck rubyattrscheck rubybarecallcheck rubyclassrecvcheck rubyconstcheck rubydeclrefcheck rubydescribedclasscheck rubyinheritcheck rubymetricscheck rubyrakejbuildercheck rubyreachcheck rubyrecvcheck rubyrecvnarrowcheck rubyrequirecheck rubyrspectargetcheck rubyscopecheck rubysettercheck rubytypedrecvcheck runhintcheck runtracecheck rustanccheck rustimportprecisecheck rustqualcheck safedeletecheck sarifcheck savecachecheck scipcheck scipjoincheck scorecardcheck scoutheadconflictcheck scoutkeycheck scroundtripcheck seedboundscheck selectorchaincheck selectorhonestycheck selectorrefusecheck selectorscopecheck selfcheckcheck selfcontainedcheck shadowcheck shallowhistorycheck shapingflagcheck shellgateindexcheck showcasecapturecheck sibliftcheck sidecarsymlinkcheck sigredactcheck sincecheck sincecochangecheck sincewindowcheck singledefcheck situdiffcheck situshapecheck skilldescbudgetcheck skillevalcheck skillevalsplitcheck skillinstallcheck skillroutingjudgedcheck skillscanreadcheck skilltruthcheck skipclassifycheck skippedcheck skipreasoncheck slicecheck slicediffcheck sliceflowcheck sliceflowsenscheck spectimingcheck staleackcheck statgatecheck stdqualcheck strkerncheck structlayoutcheck sublistcountcheck substrfiltercheck subtokencheck svectorcheck swiftcheck swiftmemberscheck swiftshapecheck taskechocheck tempfilesymlinkcheck termmargincheck testedreachcheck testgatecheck testgatelegendbudgetcheck testgatepagecheck testgaterefusecheck testmacrocheck testrowruncheck testscopecheck textdocscheck timsortcheck tokenbudgetcheck tomllangcheck toolcallroutecheck tornreadcheck traceasanlinearcheck tracecheck tracehandoffcapcheck tracehopcheck traceminecheck treecheck truncvocabcheck tsimportprecisecheck tsshapecheck type3check type3clonecheck typerefcheck unreachablecheck unresolvedcheck usescheck usesselectorcheck usingdeclcheck utf8scrubcheck vendoredassetcheck vendoredbundlecheck vendorpatchcheck verifycheck versioncheck w2verbscheck w3fixbudgetcheck w3fixlegendcheck weaksignalcheck withgraphcheck withprofilecheck worktreeleakcheck wrapverbscheck writetargetcheck xmlwellformed yamllangcheck zonecheck zoneconsistencycheck zoomcheck; do [ -f "$ROOT/test/$_g.sh" ] || continue if RIPWIRE_BIN="$BIN" bash "$ROOT/test/$_g.sh" >/dev/null 2>&1; then ok "absorb gate ($_g.sh)" diff --git a/test/rubybarecallcheck.sh b/test/rubybarecallcheck.sh new file mode 100755 index 000000000..da2658e63 --- /dev/null +++ b/test/rubybarecallcheck.sh @@ -0,0 +1,597 @@ +#!/usr/bin/env bash +# rubybarecallcheck.sh — gate for Ruby's BARE-WORD call: `full_name` / `render_profile` / `items.sum`, an identifier +# with no receiver, no arguments and no parentheses. tree-sitter-ruby parses it as a plain (identifier) — the same node +# as a local-variable read — so queries/ruby/tags.scm never captured it, and one Ruby call site in five minted no +# reference at all (measured with Prism's `variable_call?` on activerecord 7.2.3.2 lib: 4,157 of 28,584 call sites; on +# two Rails applications 50,893 of 250,320 and 56,897 of 295,879). The resolver already reads a receiver-less Ruby call +# as an implicit-self send (resolve.h Rule 1 and its base walk); it was never handed this one. +# +# The rule is Ruby's own, not a heuristic. The Ruby parser decides it LEXICALLY: an identifier is a local variable +# exactly when an assignment, a parameter or a pattern binds that name EARLIER in the same scope; every other bare +# identifier is a method call on self (Prism: CallNode#variable_call?). Scopes are Ruby's: a `def`, `class`, `module` +# and `class << x` body sees no outer local; a block or lambda sees the enclosing locals and keeps its own. So: +# * a read AFTER `x = …` / `x ||= …` / `a, (x, b) = …` / `*x = …`, a method, block or lambda parameter, a block-local +# (`|i; x|`), `rescue => x`, `for x in`, a pattern binding (`in [x]`, `in {x:}`, `=> x`, `in x`), a regex named +# capture (`/(?…)/ =~ s`) — and the right side of its own assignment (`x = x`) — is a LOCAL: no reference; +# * the same name BEFORE its binding, in a SIBLING block, or across a `def` wall is a CALL; +# * inside a block with no parameters, `it` and `_1`…`_9` are the block's implicit parameters (Ruby 3.4); +# * a hash or keyword-argument shorthand (`{ payload: }`) reads `payload` — a call when no local has that name. +# RSpec: `let`/`let!`/`subject`/`subject!` (and test-prof's `let_it_be` family) define a METHOD on the example group, +# visible to the whole group — before the line that defines it, in nested groups and in a `def` inside the group — and +# every example group defines `subject` implicitly. Those names bind like locals over the group's block, so a spec's +# `user` never reaches an unrelated application `def user`. They are NOT minted as definitions: a let is reachable +# only from its group, and a definition the resolver can see from application code would turn unique application +# calls into declines (measured: 1,818 and 1,036 application call sites to a uniquely-defined method share a name with a +# let on the two Rails applications). +# +# Ruby has NO DECLARATIONS. Every class, module, def, attr accessor and constant it indexes is a definition — there is +# no prototype for a body to replace — so a body-less one (an `attr_reader` accessor, an empty `def on_event; end` hook) +# is the answer for a call that names it, not a declaration the decl/def collapse (graph.h collapseDeclarationsOfName) +# may evict in favour of a same-named bodied def anywhere in the tree. Measured as a lower bound on the bare calls this +# round adds: 367 of 2,952 on activerecord named an attr accessor of the caller's own file and landed in another file +# (activesupport 26 of 721; the two applications 402 of 12,033 and 288 of 8,339). The receiver forms `self.x` and `x()` +# were already misrouted the same way. Pinned below (model.h isDefinitionNotDeclaration, the Kotlin-type precedent). +# And a Ruby definition is no other language's body: a C prototype with no C definition (an extension's header) stays +# its C callers' best-available target whatever Ruby defines under that name (graph.h's collapse gives Ruby its own +# family, as it gives Kotlin one). Before, a Ruby `def` evicted it and the C call read unresolved. +# +# Stated floors, each pinned below: +# (a) `defined?( name )` asks whether `name` exists; it calls nothing — no reference. +# (b) a `let` that a SHARED context defines (`shared_context` … `include_context`) is not known in the including file: +# there the name reads as Ruby reads it with no let in sight — a call. +# (c) locals made at run time (`binding.local_variable_set`, `eval`) are invisible to a lexical rule. +# (d) inside `instance_eval`/`class_eval`/`instance_exec` blocks self changes; a bare call there resolves against the +# lexical class, exactly as a parenthesised `m()` there always has. +# (e) the RSpec DSL call itself — `subject( :x ) { … }`, `let( :x ) { … }`, `it "…" do` — is a call with arguments, +# captured before this round, and binds by name like any call when the tree defines that name; only the bare +# READS of the names they define are this round's. +# +# Usage: test/rubybarecallcheck.sh | RIPWIRE_BIN=asan/ripwire test/rubybarecallcheck.sh +# Exits non-zero on any failure. Does NOT edit test/regression.sh. Self-contained via mktemp. +set -u +ROOT="$( cd "$( dirname "$0" )/.." && pwd )" +BIN="${1:-${RIPWIRE_BIN:-$ROOT/build/ripwire}}" +[ "${BIN#/}" = "$BIN" ] && BIN="$ROOT/$BIN" +fail=0 +ok(){ echo " PASS $1" || { fail=1; echo " FAIL could not write the PASS line for: $1"; }; return 0; } +no(){ echo " FAIL $1"; fail=1; } + +[ -x "$BIN" ] || { echo "no ripwire binary at $BIN — build first (cmake --build build -j)"; exit 2; } + +DIR="$( mktemp -d )"; trap 'rm -rf "$DIR"' EXIT +FIX="$DIR/fix"; mkdir -p "$FIX/lib/app" "$FIX/lib/base" "$FIX/lib/other" "$FIX/lib/decoys" "$FIX/spec" + +# Bare calls inside one class: Rule 1 (the caller's own class) answers each of them. +cat > "$FIX/lib/app/person.rb" <<'RUBY' +class Person + attr_reader :title + + def full_name + first_name + " " + last_name + end + + def first_name + "a" + end + + def last_name + "b" + end + + def show + render_profile + end + + def render_profile + "#{greeting}!" + end + + def greeting + "hi" + end + + def total + items.sum + end + + def items + [1] + end + + def logs + log( message ) + end + + def log( m ) + m + end + + def message + "m" + end + + def shorthand + { payload: } + end + + def payload + 1 + end + + def endless = endless_target + + def endless_target + 2 + end + + def each_item + items.each { |i| process_one } + end + + def process_one + 3 + end + + def label + title + end + + def helper + 1 + end + + def run + helper + end +end +RUBY + +# Same-named methods elsewhere: a bare call must pin to the caller's own class or its superclass, never split. +cat > "$FIX/lib/other/robot.rb" <<'RUBY' +class Robot + def helper + 2 + end + + def inherited_helper + 2 + end +end +RUBY + +cat > "$FIX/lib/base/parent.rb" <<'RUBY' +class Parent + def inherited_helper + 1 + end +end +RUBY + +cat > "$FIX/lib/app/child.rb" <<'RUBY' +class Child < Parent + def go + inherited_helper + end +end +RUBY + +cat > "$FIX/lib/app/boot.rb" <<'RUBY' +def boot_now + 1 +end + +boot_now +RUBY + +# Every name a local or an RSpec let binds is ALSO a method of module Decoy, in another directory, which the calling +# classes (Locals, Scoping) include — so a binding misread as a call would reach Decoy through the base walk (and, from a +# spec's example group, which is in no class, as the unique global) and show as a caller of it. A module, not a class: +# an unrelated class's method is out of a bare call's reach (test/rubyreachcheck.sh), so it could show nothing. +cat > "$FIX/lib/decoys/decoy.rb" <<'RUBY' +module Decoy + def x_asg; end + def x_op; end + def x_m2; end + def x_rest; end + def x_req; end + def x_opt; end + def x_dflt; end + def x_splat; end + def x_kw; end + def x_kwopt; end + def x_dsplat; end + def x_blk; end + def x_bp; end + def x_dp; end + def x_bl; end + def x_lam; end + def x_err; end + def x_for; end + def x_ap; end + def x_fp; end + def x_hp; end + def x_as; end + def x_in; end + def x_rw; end + def x_tp; end + def x_rx; end + def x_self; end + def x_mod; end + def x_outer; end + def x_defined; end + def it; end + def pre_bound; end + def sib_local; end + def wall_x; end + def other_param; end + def spec_user; end + def named_subj; end + def subject; end + def later_let; end + def spec_helper_q; end +end +RUBY + +cat > "$FIX/lib/app/locals.rb" <<'RUBY' +class Locals + include Decoy + def assign + x_asg = 1 + x_asg + end + + def op_assign + x_op ||= 1 + x_op + end + + def multi + x_m1, ( x_m2, x_m3 ) = 1, [ 2, 3 ] + x_m2 + end + + def rest + a, *x_rest = 1, 2 + x_rest + end + + def params( x_req, x_opt = 1, x_dflt = x_req, *x_splat, x_kw:, x_kwopt: 2, **x_dsplat, &x_blk ) + [ x_req, x_opt, x_dflt, x_splat, x_kw, x_kwopt, x_dsplat, x_blk ] + end + + def block_params + [ 1 ].each { |x_bp| x_bp } + { a: 1 }.each { |( x_dp, v ), w| x_dp } + [ 1 ].each { |i; x_bl| x_bl = i; x_bl } + end + + def implicit_params + [ 1 ].map { it } + [ 1 ].map { _1 } + end + + def lambdas + l = ->( x_lam ) { x_lam } + l + end + + def rescues + begin + 1 + rescue => x_err + x_err + end + end + + def fors + for x_for in [ 1 ] + x_for + end + end + + def patterns( v ) + case v + in [ x_ap, *rest ] then x_ap + in [ *, x_fp, * ] then x_fp + in { x_hp: } then x_hp + in Integer => x_as then x_as + in x_in then x_in + end + end + + def rightward( h, v ) + h => { x_rw: } + x_rw + v in Integer => x_tp + x_tp + end + + def named_captures( s ) + /(?\d+)/ =~ s + x_rx + end + + def self_read + x_self = x_self + end + + def modifier + x_mod = 1 if x_mod + end + + def nested + x_outer = 1 + [ 1 ].each { x_outer } + end + + def defined_check + defined?( x_defined ) + end +end +RUBY + +# Ruby's scoping where the SAME spelling is a call: before the binding, in a sibling block, across a def wall. +cat > "$FIX/lib/app/scoping.rb" <<'RUBY' +class Scoping + include Decoy + wall_x = 1 + + def before_bind + pre_bound + pre_bound = 1 + pre_bound + end + + def sibling + [ 1 ].each { sib_local = 1 } + sib_local + end + + def through_wall + wall_x + end + + def param_elsewhere( other_param ) + other_param + end + + def no_param_here + other_param + end +end +RUBY + +cat > "$FIX/spec/person_spec.rb" <<'RUBY' +RSpec.describe Person do + let( :spec_user ) { Person.new } + subject( :named_subj ) { Person.new } + + it "reads its group's lets" do + spec_user.full_name + named_subj.full_name + subject.full_name + later_let + spec_helper_q + end + + def group_helper + spec_user + end + + def spec_helper_q + 1 + end + + context "nested" do + it { spec_user } + end + + let!( :later_let ) { 1 } +end +RUBY + +# No declarations: the caller's own private attr_reader, and an inherited empty hook, against bodied defs elsewhere. +cat > "$FIX/lib/app/tracker.rb" <<'RUBY' +class Tracker + def t_bare + aliases + end + + def t_self + self.aliases + end + + def t_paren + aliases() + end + + private + attr_reader :aliases +end +RUBY + +cat > "$FIX/lib/other/join.rb" <<'RUBY' +class Join + def aliases + 1 + end + + def on_event + 2 + end +end +RUBY + +cat > "$FIX/lib/base/hooks.rb" <<'RUBY' +class Hooks + def on_event; end +end +RUBY + +cat > "$FIX/lib/app/hooked.rb" <<'RUBY' +class Hooked < Hooks + def fire + on_event + end +end +RUBY + +# A mixed tree: a C extension's header declares widget_count/widget_total with no C body; Ruby defines both names. +mkdir -p "$FIX/ext" +cat > "$FIX/ext/widget.h" <<'C' +int widget_count( void ); +int widget_total( void ); +C +cat > "$FIX/ext/widget.c" <<'C' +#include "widget.h" +int widget_report( void ) { return widget_count() + widget_total(); } +C +cat > "$FIX/lib/app/widget.rb" <<'RUBY' +class Widget + attr_reader :widget_total + + def widget_count + 1 + end +end +RUBY + +cat > "$FIX/spec/implicit_spec.rb" <<'RUBY' +RSpec.describe Person do + it { subject.full_name } +end +RUBY + +MAP="$DIR/map.xml" +"$BIN" "$FIX" --no-cache >"$MAP" 2>"$DIR/map.err" +if [ $? -eq 0 ]; then ok "default map exits 0"; else no "default map exited non-zero: $( cat "$DIR/map.err" )"; fi +[ -s "$DIR/map.err" ] && no "unexpected stderr: $( head -3 "$DIR/map.err" )" || ok "clean stderr" +command -v xmllint >/dev/null 2>&1 && { if xmllint --noout "$MAP"; then ok "xmllint --noout"; else no "xmllint failed"; fi; } + +# callers ROOT SYM — writes the --callers=SYM caller rows (one per line) to $DIR/c.rows; returns 2 when the run failed, +# so an absence arm can never read a crashed run as "no caller". +callers(){ + if ! "$BIN" "$1" --no-cache --callers="$2" >"$DIR/c.out" 2>"$DIR/c.err" + then + no "--callers=$2 exited non-zero: $( head -3 "$DIR/c.err" )" + return 2 + fi + sed 's/>\n"$DIR/ce.out" 2>"$DIR/ce.err" + then + no "--callees=$1 exited non-zero: $( head -3 "$DIR/ce.err" )" + return + fi + sed 's/>\n\n"$DIR/b.xml" 2>/dev/null +cmp -s "$MAP" "$DIR/b.xml" && ok "byte-identical across two --no-cache runs" \ + || no "output differs across runs" +if ! "$BIN" "$FIX" --cache="$DIR/c.bin" >"$DIR/cold.xml" 2>"$DIR/cold.err" +then + no "the cold cache run exited non-zero: $( head -3 "$DIR/cold.err" )" +fi +if ! "$BIN" "$FIX" --cache="$DIR/c.bin" >"$DIR/warm.xml" 2>"$DIR/warm.err" +then + no "the warm cache run exited non-zero: $( head -3 "$DIR/warm.err" )" +fi +cmp -s "$DIR/cold.xml" "$DIR/warm.xml" && ok "warm run == cold run" \ + || no "the warm cache disagrees with the cold run" + +echo "=== mutation: rename the binding → the read becomes a call ===" +MUT="$DIR/mut"; cp -R "$FIX" "$MUT" +sed 's/^ x_asg = 1$/ y_asg = 1/' "$FIX/lib/app/locals.rb" >"$MUT/lib/app/locals.rb" +grep -q '^ y_asg = 1$' "$MUT/lib/app/locals.rb" || no "mutation did not apply" +if callers "$MUT" "Decoy::x_asg" +then + grep -q ' n="assign"' "$DIR/c.rows" && ok "mutation: with the binding renamed, assign → Decoy::x_asg (the included module's method)" \ + || no "mutation: renamed binding, but assign still does not call Decoy::x_asg" +fi + +echo +[ "$fail" -eq 0 ] && echo "ALL PASS" || { echo "SOME CHECKS FAILED"; exit 1; } diff --git a/test/rubyclassrecvcheck.sh b/test/rubyclassrecvcheck.sh new file mode 100755 index 000000000..b65357d44 --- /dev/null +++ b/test/rubyclassrecvcheck.sh @@ -0,0 +1,1321 @@ +#!/usr/bin/env bash +# rubyclassrecvcheck.sh — gate for a Ruby call on a CLASS: `Report.generate`, `User.where( … )`, `Report.new( 1 )`, +# `JSON.parse( s )`. The receiver is a constant, so the call is sent to the class object, and Ruby's lookup on it answers. +# Before this round such a call bound to whichever in-tree method had the name once the class itself missed it: Rails +# apps' `Hash.new`, `Date.new` and `Service.new` all bound to one controller's `new` action, a model's `where` to a mailer +# preview's mock. +# THE RULE (graph.h RubyTypedReceivers::closedLookup; ingest_binds.h classifyRubyReceiver; parser versions 137 and 145): +# * the class the tree opens at the constant answers from its lookup: the class's singleton methods (`def self.m`, a +# def or an accessor in `class << self`), what each module it includes defines on its includer (a `def self.m` or an +# `extend` in a concern's `included do`, a concern's `class_methods do`, a nested `module ClassMethods`), the +# instance methods of the modules it extends (or includes inside `class << self`), its superclasses' singleton +# methods, then a reopened Module, Class, Object, Kernel or BasicObject. The shallowest that defines the name +# decides; what an included module defines on its includer stands at the includer's own level. A class's instance +# method is its instances' alone, and so is an included module's; a module's own singleton method is the module's. +# Ruby's order: the class's singleton (with what its included modules define there), then the modules it extends — +# a `class_methods do` block is one — and only then its superclass's; `extend self` puts a module, and what it +# includes, on its own class object; +# * `C.new` is Class#new unless a `def self.new` answers first, and Class#new runs the first `initialize` an instance's +# lookup finds — never an instance method named `new` (a controller's action); +# * when nothing there defines the name, Ruby's answer is outside the tree — ActiveRecord::Base's `where`, +# StandardError's initialize — and the call is refused as external, never bound to an unrelated namesake; +# * a constant the tree never opens (JSON, Time, RSpec, `Point = Struct.new( … )`) answers only from a reopened root; +# a QUALIFIED constant names an in-tree class only when the tree opens that path (`Stripe::Customer` is not the +# app's Customer); +# * a class written below a base that forwards a class object's call to a new instance — a mailer's +# (ActionMailer::Base, Devise::Mailer), ActiveSupport::CurrentAttributes — answers a name it lacks from its +# instances' lookup: `UserMailer.welcome( u )` runs UserMailer#welcome; +# * a method_missing in a class's lookup answers only what nothing in that lookup defines (Ruby calls it on a miss): such +# a name is left as before. So is a name a `class << self` delegates (`delegate :reset, to: :instance`), a class +# method the tree indexes no def of; +# * the constant is read as Ruby reads it from the call site, by fully-qualified constant (resolve.h's constant index): +# lexically — the innermost class or module open around the site, its enclosing ones, the top level — then through +# the ancestors of that open or of a qualified constant's head (`Sub::Failure` for a Failure Sub's superclass nests). +# The lookup then walks what the class's superclasses and mixins resolve to, so two classes or modules of one name no +# longer share it (`BankIntegration.process_event` reaches BankIntegration::HookMethods, not MailIntegration's; +# `Result.new` inside `module Billing`, Billing::Result's initialize). A constant that names nothing the tree opens +# from the site is outside it (`Row = Struct.new( … )` beside an unrelated in-tree Row); +# * a namespace Zeitwerk makes of a directory is opened by no file: a constant whose last two segments or more are one +# the tree opens names it (`Alerts::Senders::Pay` is the tree's `Senders::Pay`). +# +# Stated floors, each pinned below: +# (a) a constant the tree assigns (`DEFAULT_LIMITS = Limits.new`) is a value of a class the tree does not type: a +# call on it binds by name as before. +# (b) LIFTED at parser version 145: the tree tells `include` from `extend`, so an included module's instance method no +# longer answers a call on the class (`Report.print_me` is refused). +# (c) a module's every instance method answers a call on the module: `module_function` is not read, so a method a +# module neither copies with it nor gets from `extend self` answers too. `extend self` is read (parser version 145): +# with it, what the module includes answers as well. +# (d) a tree with no Ruby superclass, mixin or constant reference keeps no constant index: there a constant is read by +# its final segment, a qualified one only where the tree opens that path, and two classes of one name share a lookup. +# (e) a module mixed in by a call rather than a class-body directive — `Tool.extend( Tool::Ext )`, or +# `singleton_class.prepend EscapeExt` in the body (activesupport's ERB::Util) — is in no lookup: a call it answers +# is refused. +# (f) a class `Struct.new( … ) do … end` or `Data.define( … ) do … end` builds is opened by no file: a def in its block +# (`def self.from`) answers no call on its constant, which is refused. +# (g) `class << Clock` opens another object's singleton: its defs read as top-level defs, which no call with a receiver +# reaches, so `Clock.tick` is refused (activesupport's `class << Benchmark; def ms`). +# (h) LIFTED at parser version 145: a mixin's own singleton method (`def self.x` in the module's body) stays the +# module's (`Speaker.shout` is refused, `Shouting.shout` answers), while one written inside a concern's +# `included do … end` block, which runs on the includer, is the includer's (`Engine.configurations`). +# (i) LIFTED (parser version 145, a resolver change): a call to self answers from what self is — the class object in a +# singleton method, an instance in a class's other defs — and a receiver the code builds is an instance: +# `svc = Service.new( 9 ); svc.call` reaches the instance `call`, not `def self.call`, and a bare `new` in a class +# method reaches its class's initialize (refused when neither the class nor one below it has one). A name only the +# other side defines declines: self may be another's in a block. Left as before: an instance method of a module, a +# def a concern defines for its includers, a class body, a module or a FactoryBot build as a receiver, and what a +# lookup holding a method_missing misses. The superclass a class writes is never the class itself (`class Creation < +# Creation` inside `module MySQL`). +# (j) a method_missing that only an instance answers (an instance method of the class or of a module it includes) still +# leaves a call on the class object to the name ladder when the lookup misses: `Proxy.anything` binds by name, where +# Ruby raises NoMethodError. +# (k) two modules extended at one level answer their union: `Dual.pick` splits between First and Second, where Ruby +# takes the later `extend`. +# +# Usage: test/rubyclassrecvcheck.sh | RIPWIRE_BIN=asan/ripwire test/rubyclassrecvcheck.sh +# Exits non-zero on any failure. Self-contained via mktemp. +set -u +ROOT="$( cd "$( dirname "$0" )/.." && pwd )" +BIN="${1:-${RIPWIRE_BIN:-$ROOT/build/ripwire}}" +[ "${BIN#/}" = "$BIN" ] && BIN="$ROOT/$BIN" +fail=0 +ok(){ echo " PASS $1" || { fail=1; echo " FAIL could not write the PASS line for: $1"; }; return 0; } +no(){ echo " FAIL $1"; fail=1; } + +[ -x "$BIN" ] || { echo "no ripwire binary at $BIN — build first (cmake --build build -j)"; exit 2; } + +DIR="$( mktemp -d )"; trap 'rm -rf "$DIR"' EXIT +FIX="$DIR/fix" +for d in app/mailers app/models app/services app/services/alerts/senders lib/adapters lib/billing lib/shipping lib/mail_integration lib/bank_integration spec +do + mkdir -p "$FIX/$d" +done + +cat > "$FIX/lib/report.rb" <<'RUBY' +class Report + extend Finders + include Searchable + include Taggable + include Archivable + include Printable + include Strikes + + def self.generate( n ) + n + end + + def initialize( n ) + @n = n + end + + def render + "r" + end +end +RUBY + +cat > "$FIX/lib/child_report.rb" <<'RUBY' +class ChildReport < Report +end +RUBY + +cat > "$FIX/lib/finders.rb" <<'RUBY' +module Finders + def lookup( id ) + id + end + + def self.finder_own + 1 + end +end +RUBY + +cat > "$FIX/lib/searchable.rb" <<'RUBY' +module Searchable + extend ActiveSupport::Concern + + class_methods do + def search( q ) + q + end + end +end +RUBY + +cat > "$FIX/lib/taggable.rb" <<'RUBY' +module Taggable + def self.included( base ) + base.extend ClassMethods + end + + module ClassMethods + def tagged_with( t ) + t + end + end +end +RUBY + +# ActiveSupport::Concern extends a concern's nested ClassMethods onto every includer without a written `extend` +cat > "$FIX/lib/archivable.rb" <<'RUBY' +module Archivable + extend ActiveSupport::Concern + + module ClassMethods + def archived + [] + end + end +end +RUBY + +# another concern's ClassMethods of the same name, which Report never includes +cat > "$FIX/lib/purgeable.rb" <<'RUBY' +module Purgeable + extend ActiveSupport::Concern + + module ClassMethods + def archived + [] + end + end +end +RUBY + +cat > "$FIX/app/mailers/application_mailer.rb" <<'RUBY' +class ApplicationMailer < ActionMailer::Base +end +RUBY + +cat > "$FIX/app/mailers/user_mailer.rb" <<'RUBY' +class UserMailer < ApplicationMailer + def welcome( u ) + u + end +end +RUBY + +cat > "$FIX/app/models/current.rb" <<'RUBY' +class Current < ActiveSupport::CurrentAttributes + attribute :user +end +RUBY + +cat > "$FIX/app/models/application_record.rb" <<'RUBY' +class ApplicationRecord < ActiveRecord::Base + self.abstract_class = true +end +RUBY + +cat > "$FIX/app/models/user.rb" <<'RUBY' +class User < ApplicationRecord + def self.admins + where( admin: true ) + end +end +RUBY + +cat > "$FIX/app/models/customer.rb" <<'RUBY' +class Customer < ApplicationRecord + def self.create_from( params ) + params + end +end +RUBY + +cat > "$FIX/lib/billing/invoice.rb" <<'RUBY' +module Billing + class Invoice + def self.issue( n ) + n + end + + def void + true + end + end +end +RUBY + +# a top-level class of the described class's final name: described_class keeps only `Invoice`, which names this one +cat > "$FIX/lib/invoice.rb" <<'RUBY' +class Invoice + def self.void + false + end +end +RUBY + +cat > "$FIX/lib/errors.rb" <<'RUBY' +class ValidationFailed < StandardError +end + +class Base + class Failure < StandardError + def initialize( m ) + super + end + end +end + +class Sub < Base +end +RUBY + +cat > "$FIX/lib/core_ext.rb" <<'RUBY' +class Module + def cached_name + name + end +end + +class Time + def self.zone_now + now + end +end +RUBY + +cat > "$FIX/lib/limits.rb" <<'RUBY' +class Limits + def fetch_limit( k ) + k + end +end + +DEFAULT_LIMITS = Limits.new +RUBY + +cat > "$FIX/lib/dynamic.rb" <<'RUBY' +class Dynamic + def self.method_missing( name, *args ) + name + end +end +RUBY + +cat > "$FIX/lib/builder.rb" <<'RUBY' +class Builder + def self.new( *args ) + super + end + + def initialize( x ) + @x = x + end +end +RUBY + +# a class method and an instance method of one name, the instance half in a reopening +cat > "$FIX/lib/service.rb" <<'RUBY' +class Service + def self.call( x ) + new( x ).call + end +end +RUBY + +cat > "$FIX/lib/service_instance.rb" <<'RUBY' +class Service + def initialize( x ) + @x = x + end + + def call + @x + end +end +RUBY + +cat > "$FIX/lib/settings.rb" <<'RUBY' +class Settings + class << self + include Defaults + attr_accessor :config + + def build_all + [] + end + end + + attr_reader :label +end +RUBY + +# an instance method named new, as a controller's action is +cat > "$FIX/lib/pipeline.rb" <<'RUBY' +class Pipeline + def initialize( steps ) + @steps = steps + end + + def new + "an action named new" + end +end +RUBY + +# a concern with an instance method and a ClassMethods method of one name +cat > "$FIX/lib/strikes.rb" <<'RUBY' +module Strikes + extend ActiveSupport::Concern + + def strike? + false + end + + module ClassMethods + def strike? + true + end + end +end +RUBY + +cat > "$FIX/lib/printable.rb" <<'RUBY' +module Printable + def print_me + "p" + end +end +RUBY + +cat > "$FIX/lib/defaults.rb" <<'RUBY' +module Defaults + def preset_names + [] + end +end +RUBY + +cat > "$FIX/lib/util.rb" <<'RUBY' +module Util + def helper + 1 + end +end +RUBY + +cat > "$FIX/lib/registry.rb" <<'RUBY' +class Registry + class << self + delegate :reset, to: :instance + + def instance + @instance ||= new + end + end + + def reset + true + end +end +RUBY + +# Zeitwerk: app/services/alerts/senders/ is the namespace Alerts::Senders, which no file opens +cat > "$FIX/app/services/alerts/senders/pay.rb" <<'RUBY' +module Alerts + class Senders::Pay + def self.done?( x ) + x + end + end +end +RUBY + +# two modules of one name, each extended by its own integration +for m in Mail Bank +do + f="$( printf '%s' "$m" | tr 'A-Z' 'a-z' )_integration" + printf 'module %sIntegration\n extend HookMethods\nend\n' "$m" > "$FIX/lib/$f.rb" + printf 'module %sIntegration\n module HookMethods\n def process_event( id )\n id\n end\n end\nend\n' "$m" > "$FIX/lib/$f/hook_methods.rb" +done + +# two classes of one name; a call inside `module Billing` names Billing's +cat > "$FIX/lib/billing/result.rb" <<'RUBY' +module Billing + class Result + def initialize( a ) + @a = a + end + end +end +RUBY + +cat > "$FIX/lib/shipping/result.rb" <<'RUBY' +module Shipping + class Result + def initialize( b ) + @b = b + end + end +end +RUBY + +cat > "$FIX/lib/billing/charge.rb" <<'RUBY' +module Billing + class Charge + def settle + Result.new( 1 ) # @lexical_new + end + end +end +RUBY + +# a Struct constant local to a class, beside an unrelated in-tree class of that name +cat > "$FIX/lib/exporter.rb" <<'RUBY' +class Exporter + Row = Struct.new( :a ) + + def run + Row.new( 1 ) # @struct_local + end +end +RUBY + +cat > "$FIX/lib/reports_row.rb" <<'RUBY' +module Reports + class Row + def initialize( x ) + @x = x + end + end +end +RUBY + +# floor (e): a module extended at run time +cat > "$FIX/lib/tool.rb" <<'RUBY' +module Tool + module Ext + def tool_reset + true + end + end +end + +Tool.extend( Tool::Ext ) +RUBY + +# floor (f): a def in a Data.define block +cat > "$FIX/lib/filters.rb" <<'RUBY' +class Selection + Filters = Data.define( :search ) do + def self.from( h ) + new( search: h[:search] ) + end + end + + def build( h ) + Filters.from( h ) # @data_block_floor + end +end +RUBY + +# floor (g): another object's singleton class, opened from outside it +cat > "$FIX/lib/clock_ext.rb" <<'RUBY' +class << Clock + def tick + 1 + end +end +RUBY + +# what a module a class includes defines on the class object: a def self.x in a concern's included block (it runs on the +# includer) and an extend there — but not a module-body def self.x or extend, which stay the module's own +cat > "$FIX/lib/configurable.rb" <<'RUBY' +module Configurable + extend ActiveSupport::Concern + extend Tunables + + included do + extend Knobs + + def self.configurations + {} + end + end + + module ClassMethods + def configure + 1 + end + end +end + +module Tunables + def tune + 1 + end +end + +module Knobs + def knob + 1 + end +end + +class Engine + include Configurable +end + +module Shouting + def self.shout + 1 + end +end + +class Speaker + include Shouting +end + +# a method_missing answers only what nothing in the lookup defines (ActiveRecord::Base extends DynamicMatchers) +module Matchers + def method_missing( name, *args ) + super + end +end + +class Store + extend Matchers + include Configurable +end + +# a concern that includes a concern: the inner one's included block runs on the outer one's includer, whose own +# singleton it then defines — ahead of a module the includer extends +module Outer + extend ActiveSupport::Concern + include Configurable +end + +module Presets + def configurations + {} + end +end + +class Rig + extend Presets + include Outer +end + +# an extended module's initialize is the class object's method, never what Class#new runs +module Assembly + def initialize( *args ) + super + end +end + +class Gadget + extend Assembly +end + +# Ruby's order on a class object: the class's own singleton, then the modules it extends (a concern's ClassMethods and +# class_methods block among them), and only then its superclass's +module Ranked + extend ActiveSupport::Concern + + class_methods do + def rank + 1 + end + + def tier + 1 + end + end + + module ClassMethods + def grade + 1 + end + end +end + +class Listing + def self.grade + 0 + end +end + +class Featured < Listing + include Ranked + + def self.rank + 0 + end +end + +# extend self: a module's own methods, and those of the modules it includes, answer a call on it +module Codec + def encode( x ) + x + end +end + +module JsonCodec + include Codec + extend self +end + +# floor (e): a singleton_class.prepend is a call, not a class-body directive +module EscapeExt + def escape_text( s ) + s + end +end + +module Escaper + singleton_class.prepend EscapeExt +end + +# floor (j): a method_missing only an instance answers +class Proxy + def method_missing( name, *args ) + super + end +end + +# floor (k): two modules extended at one level +module First + def pick + 1 + end +end + +module Second + def pick + 2 + end +end + +class Dual + extend First + extend Second +end +RUBY + +# a call to self answers from what self is: the class object in a singleton method, an instance in any other def +cat > "$FIX/lib/job.rb" <<'RUBY' +class Job + def self.run + 2 + end + + def self.prepare + 0 + end +end +RUBY + +# the instance half in a reopening, so each arm names its def by file +cat > "$FIX/lib/job_instance.rb" <<'RUBY' +class Job + def initialize( x ) + @x = x + end + + def run + 1 + end + + def perform + 1 + end +end +RUBY + +# a class whose initialize is outside the tree, building itself in a class method; its delegation DSL leaves step 2's +# reach open (rubyreachcheck floor (b)), so Builder's def self.new is a class-side namesake in reach +cat > "$FIX/lib/gauge.rb" <<'RUBY' +class Gauge + include ActiveModel::Model + delegate :to_s, to: :value + + def self.build( v ) + new( value: v ) # @class_self_new_out_of_tree + end +end +RUBY + +# a class method building self: a subclass's initialize runs when self is the subclass; with none anywhere, Object's +cat > "$FIX/lib/card.rb" <<'RUBY' +class CardPresenter + def self.present( list ) + list.map { |x| new( x ) } # @class_self_new_cone + end +end + +class WideCardPresenter < CardPresenter + def initialize( x ) + @x = x + end +end + +class Blank + def self.make + new # @class_self_new_no_initialize + end +end +RUBY + +# self has no one side in a module's instance method (ActiveRecord's ConnectionHandling, extended onto Base): its call to +# a class method the includer's concern defines stays where the ladder had it +cat > "$FIX/lib/handling.rb" <<'RUBY' +module Sharding + extend ActiveSupport::Concern + + included do + def self.current_shard + :default + end + end +end + +module Handling + def connected? + current_shard # @module_method_self + end +end + +class Store2 + extend Handling + include Sharding +end +RUBY + +# a singleton method_missing forwards a class's call to an instance (ActiveRecord::Migration): a bare call in a class +# method that the class object's lookup misses is left as before, not declined for its side +cat > "$FIX/lib/migration_like.rb" <<'RUBY' +class Migrator + class << self + def method_missing( name, *args ) + new.public_send( name, *args ) + end + + def maintain + quiet # @singleton_missing_miss + end + end + + def quiet + 1 + end +end +RUBY + +# no class is its own superclass: `class Creation < Creation` inside `module MySQL` names the outer Creation +cat > "$FIX/lib/adapters/creation.rb" <<'RUBY' +module Adapters + class Creation + def column_options( o ) + o + end + end +end +RUBY + +cat > "$FIX/lib/adapters/mysql_creation.rb" <<'RUBY' +module Adapters + module MySQL + class Creation < Creation + def visit_add( o ) + column_options( o ) # @superclass_of_own_name + end + end + end +end +RUBY + +cat > "$FIX/lib/adapters/postgres_creation.rb" <<'RUBY' +module Adapters + module Postgres + class Creation < Creation + def column_options( o ) + super + end + end + end +end +RUBY + +# the callers in a third reopening: neither run is this file's own +cat > "$FIX/lib/job_calls.rb" <<'RUBY' +class Job + def self.perform( x ) + run # @class_self_run + new( x ) # @class_self_new + end + + class << self + def enqueue + run # @singleton_class_self_run + end + end + + def work + run # @instance_self_run + self.run # @instance_self_dot_run + prepare # @instance_self_class_only + end +end +RUBY + +cat > "$FIX/lib/batch.rb" <<'RUBY' +class Batch < Job + def self.run + 3 + end + + def perform + run # @inherited_instance_run + end +end +RUBY + +cat > "$FIX/lib/helpers.rb" <<'RUBY' +def format_amount( n ) + n.to_s +end +RUBY + +# Every name the fixture's calls might bind to by name alone: a Decoy target is a wrong edge. +cat > "$FIX/lib/decoy.rb" <<'RUBY' +class Decoy + def where( *a ); end + def find_by( *a ); end + def parse( s ); end + def now; end + def new( *a ); end + def create_from( p ); end + def lookup( id ); end + def search( q ); end + def tagged_with( t ); end + def describe( *a ); end + def anything; end + def format_amount( n ); end + def cached_name; end + def zone_now; end + def archived; end + def render; end + def label; end + def welcome( u ); end + def user=( u ); end + def done?( x ); end + def tool_reset; end + def from( h ); end + def configurations; end + def shout; end + def print_me; end + def finder_own; end + def preset_names; end + def knob; end + def tune; end + def find_by_name; end + def rank; end + def tier; end + def grade; end + def encode( x ); end + def escape_text( s ); end + def pick; end + def prepare; end + def perform; end +end +RUBY + +CALLER=app/services/caller.rb +cat > "$FIX/$CALLER" <<'RUBY' +class Caller + def run( s ) + Report.generate( 1 ) # @own + ChildReport.generate( 2 ) # @inherited + Report.lookup( 3 ) # @extended + Report.search( "q" ) # @concern + Report.tagged_with( "t" ) # @class_methods + Report.archived # @concern_class_methods + Billing::Invoice.issue( 4 ) # @qualified_opened + Customer.create_from( {} ) # @customer + Report.new( 5 ) # @new + ChildReport.new( 6 ) # @new_inherited + ValidationFailed.new( "x" ) # @new_out_of_tree + Point.new( 1, 2 ) # @struct + User.where( admin: true ) # @ar_where + User.find_by( id: 1 ) # @ar_find_by + Stripe::Customer.create_from( {} ) # @qualified_not_opened + JSON.parse( s ) # @json + Time.now # @time + Time.zone_now # @time_reopened + Report.cached_name # @root_module + JSON.cached_name # @root_external + Report.format_amount( 7 ) # @private_top + Dynamic.anything # @missing + DEFAULT_LIMITS.fetch_limit( :a ) # @value_floor + Service.call( 9 ) # @singleton + Settings.config # @singleton_accessor + Settings.build_all # @singleton_class + Settings.label # @instance_accessor + Report.render # @instance + Pipeline.new( [] ) # @new_action + Builder.new( 8 ) # @self_new + Report.print_me # @include_instance + Util.helper # @module_floor + UserMailer.welcome( 10 ) # @mailer + Current.user = 11 # @current + Registry.reset # @class_delegate + Alerts::Senders::Pay.done?( 12 ) # @zeitwerk + BankIntegration.process_event( 13 ) # @fqn_mixin + Billing::Result.new( 14 ) # @qualified_new + Tool.tool_reset # @runtime_extend_floor + Report.strike? # @concern_precedence + Clock.tick # @foreign_singleton_floor + Sub::Failure.new( "m" ) # @ancestor_const + Engine.configurations # @included_singleton + Speaker.shout # @module_singleton + Shouting.shout # @module_own + Report.finder_own # @extended_own_singleton + Settings.preset_names # @singleton_include + Engine.knob # @included_extend + Engine.tune # @module_extend + Configurable.tune # @module_extend_own + Configurable.configurations # @included_on_concern + Searchable.search # @class_methods_on_concern + Archivable.archived # @nested_class_methods_on_concern + Store.configurations # @missing_after_lookup + Store.find_by_name # @missing_miss + Rig.configurations # @concern_dependency + Gadget.new # @new_extended + Featured.rank # @own_before_class_methods_block + Featured.tier # @class_methods_block + Featured.grade # @class_methods_before_superclass + JsonCodec.encode( 1 ) # @extend_self_include + Escaper.escape_text( "s" ) # @singleton_prepend_floor + Proxy.anything # @instance_missing_floor + Dual.pick # @same_level_floor + svc = Service.new( 9 ) + svc.call # @built_instance_call + job = Job.new( 1 ) + job.run # @built_instance_run + job.perform # @built_instance_perform + end +end + +Point = Struct.new( :x, :y ) +RUBY + +SPEC=spec/report_spec.rb +cat > "$FIX/$SPEC" <<'RUBY' +RSpec.describe Report do # @rspec_describe + it "generates" do + described_class.generate( 1 ) # @described + end +end + +RSpec.describe Billing::Invoice do + it "issues" do + described_class.issue( 2 ) # @described_qualified + end +end + +RSpec.describe Billing::Invoice do + subject( :invoice ) { described_class.new } + + it "voids" do + invoice.void # @described_new_namesake + end +end + +RSpec.describe Billing do + it "issues through the namespace" do + described_class::Invoice.issue( 3 ) # @described_path + end +end +RUBY + +echo "=== the map is well-formed ===" +MAP="$DIR/a.xml" +if ! "$BIN" "$FIX" --no-cache >"$MAP" 2>"$DIR/a.err" +then + no "the default map exited non-zero: $( head -3 "$DIR/a.err" )" +fi +if command -v xmllint >/dev/null 2>&1 +then + if xmllint --noout "$MAP"; then ok "xmllint --noout"; else no "xmllint failed"; fi +fi + +# The census (src/pincensus.h): one `C` row per call site the resolver decided or refused, so each arm reads ONE line. +census(){ + if ! "$BIN" "$1" --no-cache --pin-census="$DIR/census.tsv" >/dev/null 2>"$DIR/census.err" + then + no "--pin-census on $1 exited non-zero: $( head -3 "$DIR/census.err" )" + return 2 + fi + return 0 +} +# line FILE MARK — the line number carrying `# @MARK` (empty when the marker is missing) +line(){ + grep -n "# @$2\$" "$FIX/$1" | head -1 | cut -d: -f1 +} +# rows FILE LINE CALLEE — "mechanismtargets" of each CALLEE call the census records on FILE:LINE +rows(){ + awk -F'\t' -v f="$1" -v l="$2" -v c="$3" '$1 == "C" && $9 == l && $7 == c && index( $6, f "::" ) == 1 { print $2 "\t" $8 }' "$DIR/census.tsv" +} +# reaches CALLEE SYM FILE MARK WHY — the CALLEE call on the marked line resolves to SYM, and to no Decoy +reaches(){ + local n r; n="$( line "$3" "$4" )" + [ -n "$n" ] || { no "fixture marker @$4 missing in $3"; return; } + r="$( rows "$3" "$n" "$1" )" + if printf '%s\n' "$r" | grep -qF "::$2#" && ! printf '%s\n' "$r" | grep -qF "::Decoy::" + then + ok "@$4 :$1 → $2 ($5)" + else + no "@$4 ($3:$n) :$1 does not reach $2 alone ($5); census: $( printf '%s' "$r" | tr '\t\n' ' ;' )" + fi +} +# only CALLEE SYM FILE MARK WHY — the CALLEE call on the marked line has one target, SYM (a trailing id path, or a whole +# `file::id` one) +only(){ + local n r t; n="$( line "$3" "$4" )" + [ -n "$n" ] || { no "fixture marker @$4 missing in $3"; return; } + r="$( rows "$3" "$n" "$1" )" + t="$( printf '%s\n' "$r" | cut -f2 | tr '|' '\n' | grep . | sed 's/#[0-9]*$//' )" + if [ "$( printf '%s\n' "$t" | grep -c . )" -eq 1 ] && { [ "$t" = "$2" ] || [ "${t%::$2}" != "$t" ]; } + then + ok "@$4 :$1 → $2 alone ($5)" + else + no "@$4 ($3:$n) :$1 does not reach $2 alone ($5); census: $( printf '%s' "$r" | tr '\t\n' ' ;' )" + fi +} +# refused CALLEE FILE MARK WHY — the CALLEE call on the marked line is recorded and refused as external +refused(){ + local n r; n="$( line "$2" "$3" )" + [ -n "$n" ] || { no "fixture marker @$3 missing in $2"; return; } + r="$( rows "$2" "$n" "$1" )" + if [ -n "$r" ] && ! printf '%s\n' "$r" | grep -qv "^external \$" + then + ok "@$3 :$1 refused as external ($4)" + else + no "@$3 ($2:$n) :$1 is not refused ($4); census: $( printf '%s' "$r" | tr '\t\n' ' ;' )" + fi +} + +census "$FIX" || exit 1 + +echo "=== the class the tree opens answers: its methods, its mixins', its superclasses' ===" +only generate Report::generate $CALLER own "the class's own def self.generate" +only generate Report::generate $CALLER inherited "a superclass's class method" +only lookup Finders::lookup $CALLER extended "a module the class extends" +only search Searchable::search $CALLER concern "a concern's class_methods block" +only tagged_with ClassMethods::tagged_with $CALLER class_methods "a nested module ClassMethods the concern extends onto its includer" +only strike? ClassMethods::strike? $CALLER concern_precedence "a concern's ClassMethods, not its instance method of the name" +only archived lib/archivable.rb::ClassMethods::archived $CALLER concern_class_methods "the ClassMethods of the concern Report includes, not another concern's" +only issue Invoice::issue $CALLER qualified_opened "a qualified constant whose path the tree opens" +only process_event lib/bank_integration/hook_methods.rb::HookMethods::process_event $CALLER fqn_mixin "BankIntegration's own HookMethods, not MailIntegration's of that name" +only new lib/billing/result.rb::Result::initialize lib/billing/charge.rb lexical_new "Result inside module Billing is Billing::Result, not Shipping's" +only new lib/billing/result.rb::Result::initialize $CALLER qualified_new "Billing::Result, from outside the namespace" +only done? Pay::done? $CALLER zeitwerk "Alerts::Senders::Pay ends in Senders::Pay, a path the tree opens" +only create_from Customer::create_from $CALLER customer "the app's own Customer" +only generate Report::generate $SPEC described "described_class is the group's class" +only issue Invoice::issue $SPEC described_qualified "a qualified described class is read whole" +only issue Invoice::issue $SPEC described_path "described_class::Invoice is Billing::Invoice" + +echo "=== a class's instance method is its instances' alone ===" +only call lib/service.rb::Service::call $CALLER singleton "def self.call, not the instance call a reopening defines" +only config Settings::config $CALLER singleton_accessor "an accessor class << self declares" +only build_all Settings::build_all $CALLER singleton_class "a def in class << self" +refused label $CALLER instance_accessor "an instance attr_reader is no method of the class object" +refused render $CALLER instance "Report#render is Report's instances'" + +echo "=== include is the instances' side, extend the class object's (floors (b) and (h) lifted, parser version 145) ===" +refused print_me $CALLER include_instance "an included module's instance method is the instances', not the class object's" +refused shout $CALLER module_singleton "Shouting's module-body def self.shout stays Shouting's: Speaker.shout raises NoMethodError" +only shout Shouting::shout $CALLER module_own "a module-body def self.shout answers a call on the module" +refused finder_own $CALLER extended_own_singleton "an extended module's own def self.x is not among the instance methods extend adds" +only preset_names Defaults::preset_names $CALLER singleton_include "an include inside class << self extends the class object" +only configurations Configurable::configurations $CALLER included_singleton "a def self.x in a concern's included block runs on the includer" +only knob Knobs::knob $CALLER included_extend "an extend in a concern's included block extends the includer" +refused tune $CALLER module_extend "a concern's body-level extend is the concern's own, not its includer's" +only tune Tunables::tune $CALLER module_extend_own "a module's body-level extend answers a call on the module" +refused configurations $CALLER included_on_concern "an included block runs on each includer, never on the concern" +refused search $CALLER class_methods_on_concern "a class_methods block extends each includer, never the concern" +refused archived $CALLER nested_class_methods_on_concern "a nested module ClassMethods extends each includer, never the concern" +only configurations Configurable::configurations $CALLER concern_dependency "an inner concern's included block runs on the outer one's includer, ahead of a module it extends" +refused new $CALLER new_extended "an extended module's initialize is no instance's: Class#new never runs it" +only rank Featured::rank $CALLER own_before_class_methods_block "the class's own def self.rank answers before a class_methods block, which is extended" +only tier Ranked::tier $CALLER class_methods_block "a class_methods block answers its includer" +only grade ClassMethods::grade $CALLER class_methods_before_superclass "an extended module answers before the superclass's singleton" +only encode Codec::encode $CALLER extend_self_include "extend self: what the module includes answers a call on it" + +echo "=== a call to self, or on a built instance, answers from its own side (floor (i) lifted, parser version 145) ===" +JOB=lib/job.rb; JOBI=lib/job_instance.rb; JOBC=lib/job_calls.rb +only run $JOB::Job::run $JOBC class_self_run "a bare call in def self.perform is the class object's: def self.run" +only new Job::initialize $JOBC class_self_new "a bare new in a singleton method is Class#new: it runs initialize" +only run $JOB::Job::run $JOBC singleton_class_self_run "a def in class << self is the class object's too" +GAUGEN="$( line lib/gauge.rb class_self_new_out_of_tree )" +GAUGE="$( rows lib/gauge.rb "$GAUGEN" new )" +if [ -z "$GAUGEN" ] +then + no "fixture marker @class_self_new_out_of_tree missing in lib/gauge.rb" +elif ! printf '%s\n' "$GAUGE" | grep -q "::new#" +then + ok "@class_self_new_out_of_tree :new binds no class's def self.new by name: Class#new runs an initialize" +else + no "@class_self_new_out_of_tree :new binds a def self.new by name; census: $( printf '%s' "$GAUGE" | tr '\t\n' ' ;' )" +fi +refused new lib/card.rb class_self_new_no_initialize "a bare new with no initialize in the class or below it runs Object's: outside the tree" +CARDN="$( line lib/card.rb class_self_new_cone )" +CARD="$( rows lib/card.rb "$CARDN" new )" +if [ -n "$CARDN" ] && ! printf '%s\n' "$CARD" | grep -q "^external" +then + ok "@class_self_new_cone :new is not refused: self may be a class below, whose initialize runs" +else + no "@class_self_new_cone :new is refused, or its marker is missing; census: $( printf '%s' "$CARD" | tr '\t\n' ' ;' )" +fi +only run $JOBI::Job::run $JOBC instance_self_run "a bare call in an instance method is the instance's: def run" +only run $JOBI::Job::run $JOBC instance_self_dot_run "self.run in an instance method is the instance's" +PREPN="$( line $JOBC instance_self_class_only )" +PREP="$( rows $JOBC "$PREPN" prepare )" +if [ -z "$PREPN" ] +then + no "fixture marker @instance_self_class_only missing in $JOBC" +elif ! printf '%s\n' "$PREP" | grep -q "::Job::prepare#" +then + ok "@instance_self_class_only :prepare reaches no def self.prepare: an instance answers none (declined, not refused: self may be another's in a block)" +else + no "@instance_self_class_only :prepare reaches def self.prepare from an instance method; census: $( printf '%s' "$PREP" | tr '\t\n' ' ;' )" +fi +only run $JOBI::Job::run lib/batch.rb inherited_instance_run "Batch's def self.run hides nothing from an instance: Job#run answers" +only call lib/service_instance.rb::Service::call $CALLER built_instance_call "svc = Service.new: the instance call, not def self.call" +only run $JOBI::Job::run $CALLER built_instance_run "a built Job is an instance: def run" +only perform $JOBI::Job::perform $CALLER built_instance_perform "a built Job is an instance: def perform, not def self.perform (job_calls.rb)" + +only current_shard Sharding::current_shard lib/handling.rb module_method_self "a module's instance method has no one self: its call keeps the includer's class method" +only quiet Migrator::quiet lib/migration_like.rb singleton_missing_miss "a singleton method_missing may forward a miss to an instance: left as before" +only column_options lib/adapters/creation.rb::Creation::column_options lib/adapters/mysql_creation.rb superclass_of_own_name "class Creation < Creation in module MySQL names the outer Creation, not the sibling Postgres one" +only void lib/billing/invoice.rb::Invoice::void $SPEC described_new_namesake "described_class.new is a Billing::Invoice, not the top-level Invoice's def self.void" + +echo "=== a base that forwards to an instance: the instances' lookup answers ===" +only welcome UserMailer::welcome $CALLER mailer "ActionMailer::Base's class object runs the instance method" +only user= Current::user= $CALLER current "ActiveSupport::CurrentAttributes forwards to the instance" + +echo "=== C.new runs initialize ===" +only new Report::initialize $CALLER new "Report.new reaches Report#initialize, never a method named new" +only new Report::initialize $CALLER new_inherited "a subclass with no initialize reaches its superclass's" +refused new $CALLER new_out_of_tree "StandardError's initialize is outside the tree" +refused new $CALLER struct "Point = Struct.new: the tree never opens Point" +refused new lib/exporter.rb struct_local "Exporter::Row is a Struct: Reports::Row is no constant Exporter names" +only new Pipeline::initialize $CALLER new_action "an instance method named new (a controller's action) is no Class#new" +only new Builder::new $CALLER self_new "a def self.new answers first" + +echo "=== a lookup that leaves the tree is refused, never handed to a namesake ===" +refused where $CALLER ar_where "User.where is ActiveRecord::Base's" +refused find_by $CALLER ar_find_by "User.find_by is ActiveRecord::Base's" +refused create_from $CALLER qualified_not_opened "Stripe::Customer is not the app's Customer: the tree never opens that path" +refused parse $CALLER json "JSON is never opened" +refused now $CALLER time "Time is opened, and its opening defines no now" +only zone_now Time::zone_now $CALLER time_reopened "a class the tree reopens answers from the reopening" +refused describe $SPEC rspec_describe "RSpec is never opened" +refused format_amount $CALLER private_top "a top-level def is a private method of Object: no receiver may call it" + +echo "=== a reopened root answers every class object ===" +only cached_name Module::cached_name $CALLER root_module "a class is a Module" +only cached_name Module::cached_name $CALLER root_external "so is a class the tree never opens" + +echo "=== a class << self delegation answers the name, and a method_missing what the lookup lacks: left as before ===" +DELEG="$( rows $CALLER "$( line $CALLER class_delegate )" reset )" +if printf '%s\n' "$DELEG" | grep -qF "::Registry::reset#" +then + ok "@class_delegate :reset binds by name as before (a class << self delegates it)" +else + no "@class_delegate :reset no longer binds by name; census: $( printf '%s' "$DELEG" | tr '\t\n' ' ;' )" +fi +MISSING="$( rows $CALLER "$( line $CALLER missing )" anything )" +if printf '%s\n' "$MISSING" | grep -qF "::Decoy::anything#" +then + ok "@missing :anything binds by name as before (Dynamic answers any name)" +else + no "@missing :anything no longer binds by name; census: $( printf '%s' "$MISSING" | tr '\t\n' ' ;' )" +fi +MISS="$( rows $CALLER "$( line $CALLER missing_miss )" find_by_name )" +if printf '%s\n' "$MISS" | grep -qF "::Decoy::find_by_name#" +then + ok "@missing_miss :find_by_name binds by name as before (nothing in Store's lookup defines it, so Matchers' method_missing may)" +else + no "@missing_miss :find_by_name no longer binds by name; census: $( printf '%s' "$MISS" | tr '\t\n' ' ;' )" +fi +only configurations Configurable::configurations $CALLER missing_after_lookup "a def in the lookup answers before any method_missing does: Ruby calls it only on a miss" + +echo "=== stated floors ===" +only fetch_limit Limits::fetch_limit $CALLER value_floor "floor (a): a constant the tree assigns binds by name" +refused tool_reset $CALLER runtime_extend_floor "floor (e): Tool.extend( Tool::Ext ) at run time is in no lookup" +refused escape_text $CALLER singleton_prepend_floor "floor (e): singleton_class.prepend EscapeExt is a call, in no lookup" +refused from lib/filters.rb data_block_floor "floor (f): Filters = Data.define do … end is opened by no file" +refused tick $CALLER foreign_singleton_floor "floor (g): class << Clock is no open of Clock" +only helper Util::helper $CALLER module_floor "floor (c): a module's method answers a call on the module" +PROXY="$( rows $CALLER "$( line $CALLER instance_missing_floor )" anything )" +if printf '%s\n' "$PROXY" | grep -qF "::Decoy::anything#" +then + ok "floor (j) pinned: Proxy.anything binds by name, though only Proxy's instances answer any name" +else + no "floor (j): Proxy.anything no longer binds by name; census: $( printf '%s' "$PROXY" | tr '\t\n' ' ;' )" +fi +DUAL="$( rows $CALLER "$( line $CALLER same_level_floor )" pick )" +if printf '%s\n' "$DUAL" | grep -qF "::First::pick#" && printf '%s\n' "$DUAL" | grep -qF "::Second::pick#" && ! printf '%s\n' "$DUAL" | grep -qF "::Decoy::" +then + ok "floor (k) pinned: Dual.pick splits between the two modules Dual extends" +else + no "floor (k): Dual.pick no longer splits between First and Second; census: $( printf '%s' "$DUAL" | tr '\t\n' ' ;' )" +fi +only new Failure::initialize $CALLER ancestor_const "Sub::Failure is Base::Failure, through Sub's superclass" + +# floor (d): a tree with no superclass, mixin or constant reference keeps no constant index +FIX2="$DIR/noindex"; mkdir -p "$FIX2" +printf 'module Alpha\n class Row\n def initialize( a )\n @a = a\n end\n end\nend\n' > "$FIX2/alpha.rb" +printf 'module Beta\n class Row\n def initialize( b )\n @b = b\n end\n end\nend\n' > "$FIX2/beta.rb" +printf 'class Caller\n def run\n Alpha::Row.new( 1 )\n end\nend\n' > "$FIX2/caller.rb" +if "$BIN" "$FIX2" --no-cache --pin-census="$DIR/noindex.tsv" >/dev/null 2>&1 +then + NOIX="$( awk -F'\t' '$1 == "C" && $7 == "new" && index( $6, "caller.rb::" ) == 1 { print $8 }' "$DIR/noindex.tsv" )" + if printf '%s' "$NOIX" | grep -qF "alpha.rb::Row::initialize#" && printf '%s' "$NOIX" | grep -qF "beta.rb::Row::initialize#" + then + ok "floor (d) pinned: with no constant index, Alpha::Row.new splits over both Rows' initialize" + else + no "floor (d): with no constant index, Alpha::Row.new no longer splits over both Rows; census: $NOIX" + fi +else + no "--pin-census on the no-index fixture exited non-zero" +fi + +echo "=== determinism and warm == cold ===" +"$BIN" "$FIX" --no-cache >"$DIR/b.xml" 2>/dev/null +if cmp -s "$MAP" "$DIR/b.xml" +then + ok "byte-identical across two --no-cache runs" +else + no "output differs across runs" +fi +if ! "$BIN" "$FIX" --cache="$DIR/c.bin" >"$DIR/cold.xml" 2>"$DIR/cold.err" +then + no "the cold cache run exited non-zero: $( head -3 "$DIR/cold.err" )" +fi +if ! "$BIN" "$FIX" --cache="$DIR/c.bin" >"$DIR/warm.xml" 2>"$DIR/warm.err" +then + no "the warm cache run exited non-zero: $( head -3 "$DIR/warm.err" )" +fi +if cmp -s "$DIR/cold.xml" "$DIR/warm.xml" +then + ok "warm run == cold run" +else + no "the warm cache disagrees with the cold run" +fi + +echo "=== --callers: Report.new is a caller of Report#initialize ===" +"$BIN" "$FIX" --no-cache --callers=Report::initialize >"$DIR/callers.xml" 2>/dev/null +if grep -q 'caller.rb' "$DIR/callers.xml" +then + ok "--callers=Report::initialize names app/services/caller.rb" +else + no "--callers=Report::initialize does not name the caller: $( head -c 400 "$DIR/callers.xml" )" +fi + +echo +[ "$fail" -eq 0 ] && echo "ALL PASS" || { echo "SOME CHECKS FAILED"; exit 1; } diff --git a/test/rubydeclrefcheck.sh b/test/rubydeclrefcheck.sh new file mode 100755 index 000000000..0cd7b4bd5 --- /dev/null +++ b/test/rubydeclrefcheck.sh @@ -0,0 +1,370 @@ +#!/usr/bin/env bash +# rubydeclrefcheck.sh — gate for a Ruby method named by a SYMBOL that Rails or Ruby later calls. `before_action +# :authenticate`, `after_save :reindex`, `validate :name_present`, `rescue_from …, with: :not_found`, an `if:` / `unless:` +# condition, and `send( :render_user )` / `try( :assist )` / `method( :log_one )`: each names a method the framework calls +# on that object, but nothing in the source CALLS it, so before this round every callback method in a Rails app had no +# caller — it looked dead, ranked as a leaf, and `--callers` answered nothing. +# THE RULE (ingest_binds.h RubyBareCallWalk::noteDeclaredCalls; parser version 134): the symbol is a call to that method, +# * for a callback macro at class-body position — receiver-less, its nearest enclosing def-or-class wall a class or +# module body (an `included do` block inside a concern counts) — its positional symbols, and the symbols of its `if:`, +# `unless:` (a symbol or an array of them) and, for `rescue_from`, `with:` options; for `validates`/`validates_*_of` +# only the `if:`/`unless:` options (its positional symbols are attributes). Ruby's self there is the class, and the +# callback runs on its instances, whose lookup is the class's: the call resolves like any call to self from the class +# body (rubyreachcheck); +# * for `send`/`public_send`/`__send__`/`try`/`try!`/`method` whose first argument is a literal symbol — a call to that +# method on the same receiver, typed or not (`helper.try( :assist )` answers from Helper, rubytypedrecvcheck). +# +# Stated floors, each pinned below: +# (a) option values that are not calls make none: `only:`/`except:` (action names), `on:` (an event), `prepend:`; and +# `validates`' attribute names make none. +# (b) `skip_before_action :x` removes a callback and calls nothing: no reference. +# (c) `set_callback :save, :before, :x` and other ActiveSupport::Callbacks plumbing are not read. +# (d) a string or computed name (`send( "render_user" )`, `send( name )`) is not read. +# (e) a callback macro inside a method body is not read (it is not at class-body position). +# +# Usage: test/rubydeclrefcheck.sh | RIPWIRE_BIN=asan/ripwire test/rubydeclrefcheck.sh +# Exits non-zero on any failure. Self-contained via mktemp. +set -u +ROOT="$( cd "$( dirname "$0" )/.." && pwd )" +BIN="${1:-${RIPWIRE_BIN:-$ROOT/build/ripwire}}" +[ "${BIN#/}" = "$BIN" ] && BIN="$ROOT/$BIN" +fail=0 +ok(){ echo " PASS $1" || { fail=1; echo " FAIL could not write the PASS line for: $1"; }; return 0; } +no(){ echo " FAIL $1"; fail=1; } + +[ -x "$BIN" ] || { echo "no ripwire binary at $BIN — build first (cmake --build build -j)"; exit 2; } + +DIR="$( mktemp -d )"; trap 'rm -rf "$DIR"' EXIT +FIX="$DIR/fix" +for d in app/controllers app/models/concerns app/jobs app/helpers lib +do + mkdir -p "$FIX/$d" +done + +cat > "$FIX/app/controllers/application_controller.rb" <<'RUBY' +class ApplicationController < ActionController::Base + before_action :authenticate # @before_action + rescue_from ActiveRecord::RecordNotFound, with: :not_found # @rescue_with + + private + + def authenticate + true + end + + def ensure_admin + true + end + + def not_found + :not_found + end +end +RUBY +cat > "$FIX/app/controllers/users_controller.rb" <<'RUBY' +class UsersController < ApplicationController + before_action :load_user, only: [ :show ] # @only_floor + before_action :ensure_admin # @inherited_cb + before_action :authenticate_user! # @generated + after_action :track, if: :tracking? # @after_if + around_action :with_locale, except: :index # @around + skip_before_action :authenticate # @skip_floor + set_callback :process_action, :before, :set_trace # @set_callback_floor + + def show + send( :render_user ) # @send_bare + send( "render_user" ) # @send_string_floor + helper = Helper.new + helper.try( :assist ) # @try_typed + [ 1 ].each( &method( :log_one ) ) # @method_ref + end + + def self.configure + before_action :late_hook # @in_def_floor + end + + def index + :index + end + + private + + def load_user + true + end + + def track + true + end + + def tracking? + true + end + + def with_locale + yield + end + + def render_user + true + end + + def log_one( x ) + x + end + + def set_trace + true + end + + def late_hook + true + end +end +RUBY +cat > "$FIX/app/helpers/helper.rb" <<'RUBY' +class Helper + def assist + :helper + end +end +RUBY +cat > "$FIX/app/models/application_record.rb" <<'RUBY' +class ApplicationRecord < ActiveRecord::Base + self.abstract_class = true +end +RUBY +cat > "$FIX/app/models/user.rb" <<'RUBY' +class User < ApplicationRecord + include Auditable + before_save :normalize_name # @before_save + after_commit :reindex, on: :create # @on_floor + validate :name_present # @validate + validates :email, presence: true, unless: :guest? # @validates_unless + before_validation :trim_name, :lower_email # @two_symbols + after_create_commit :notify_team, if: [ :active?, :confirmed? ] # @if_array + + def normalize_name + true + end + + def reindex + true + end + + def name_present + true + end + + def guest? + false + end + + def trim_name + true + end + + def lower_email + true + end + + def notify_team + true + end + + def active? + true + end + + def confirmed? + true + end +end +RUBY +cat > "$FIX/app/models/concerns/auditable.rb" <<'RUBY' +module Auditable + extend ActiveSupport::Concern + + included do + after_save :write_audit # @concern_cb + end + + def write_audit + true + end +end +RUBY +cat > "$FIX/app/jobs/sync_job.rb" <<'RUBY' +class SyncJob < ApplicationJob + before_perform :check_lock # @job_cb + + def perform + true + end + + def check_lock + true + end +end +RUBY +# the namesakes a wrong reading would bind to: every name above, on a class outside every caller's reach +cat > "$FIX/lib/decoy.rb" <<'RUBY' +class Decoy + def authenticate; end + def not_found; end + def ensure_admin; end + def authenticate_user!; end + def load_user; end + def track; end + def tracking?; end + def with_locale; end + def render_user; end + def assist; end + def log_one( x ); end + def set_trace; end + def late_hook; end + def normalize_name; end + def reindex; end + def name_present; end + def guest?; end + def trim_name; end + def lower_email; end + def notify_team; end + def active?; end + def confirmed?; end + def write_audit; end + def check_lock; end + def show; end + def index; end + def create; end + def email; end +end +RUBY + +MAP="$DIR/a.xml" +"$BIN" "$FIX" --no-cache >"$MAP" 2>"$DIR/map.err" +if [ $? -eq 0 ]; then ok "default map exits 0"; else no "default map exited non-zero: $( cat "$DIR/map.err" )"; fi +if [ -s "$DIR/map.err" ]; then no "unexpected stderr: $( head -3 "$DIR/map.err" )"; else ok "clean stderr"; fi +if command -v xmllint >/dev/null 2>&1 +then + if xmllint --noout "$MAP"; then ok "xmllint --noout"; else no "xmllint failed"; fi +fi + +# The census (src/pincensus.h): one `C` row per call site the resolver decided or refused, so each arm reads ONE line. +census(){ + if ! "$BIN" "$1" --no-cache --pin-census="$DIR/census.tsv" >/dev/null 2>"$DIR/census.err" + then + no "--pin-census on $1 exited non-zero: $( head -3 "$DIR/census.err" )" + return 2 + fi + return 0 +} +# line FILE MARK — the line number carrying `# @MARK` (empty when the marker is missing) +line(){ + grep -n "# @$2\$" "$FIX/$1" | head -1 | cut -d: -f1 +} +# rows FILE LINE CALLEE — "mechanismtargets" of each CALLEE call the census records on FILE:LINE +rows(){ + awk -F'\t' -v f="$1" -v l="$2" -v c="$3" '$1 == "C" && $9 == l && $7 == c && index( $6, f "::" ) == 1 { print $2 "\t" $8 }' "$DIR/census.tsv" +} +# reaches CALLEE SYM FILE MARK WHY — the CALLEE call on the marked line resolves to SYM, and to no Decoy +reaches(){ + local n r; n="$( line "$3" "$4" )" + [ -n "$n" ] || { no "fixture marker @$4 missing in $3"; return; } + r="$( rows "$3" "$n" "$1" )" + if printf '%s\n' "$r" | grep -qF "::$2#" && ! printf '%s\n' "$r" | grep -qF "::Decoy::" + then + ok "@$4 :$1 → $2 ($5)" + else + no "@$4 ($3:$n) :$1 does not reach $2 alone ($5); census: $( printf '%s' "$r" | tr '\t\n' ' ;' )" + fi +} +# refused CALLEE FILE MARK WHY — the CALLEE call on the marked line is recorded and refused as external +refused(){ + local n r; n="$( line "$2" "$3" )" + [ -n "$n" ] || { no "fixture marker @$3 missing in $2"; return; } + r="$( rows "$2" "$n" "$1" )" + if [ -n "$r" ] && ! printf '%s\n' "$r" | grep -qv "^external \$" + then + ok "@$3 :$1 refused as external ($4)" + else + no "@$3 ($2:$n) :$1 is not refused ($4); census: $( printf '%s' "$r" | tr '\t\n' ' ;' )" + fi +} +# absent CALLEE FILE MARK WHY — no CALLEE call is recorded on the marked line (a Decoy namesake would catch one) +absent(){ + local n r; n="$( line "$2" "$3" )" + [ -n "$n" ] || { no "fixture marker @$3 missing in $2"; return; } + r="$( rows "$2" "$n" "$1" )" + if [ -z "$r" ] + then + ok "@$3 makes no :$1 call ($4)" + else + no "@$3 ($2:$n) records a :$1 call ($4); census: $( printf '%s' "$r" | tr '\t\n' ' ;' )" + fi +} + +census "$FIX" || exit 1 + +AC=app/controllers/application_controller.rb +UC=app/controllers/users_controller.rb +UM=app/models/user.rb + +echo "=== a callback names a method the framework calls on the class's instances ===" +reaches authenticate ApplicationController::authenticate $AC before_action "before_action :authenticate — the class's own method" +reaches not_found ApplicationController::not_found $AC rescue_with "rescue_from …, with: :not_found" +reaches load_user UsersController::load_user $UC only_floor "before_action :load_user" +reaches ensure_admin ApplicationController::ensure_admin $UC inherited_cb "a callback in a subclass reaches the superclass's method" +refused authenticate_user! $UC generated "a method the tree does not define in reach (Devise's) is refused, not bound to Decoy's" +reaches track UsersController::track $UC after_if "after_action :track" +reaches tracking? UsersController::tracking? $UC after_if "if: :tracking? — the condition is called too" +reaches with_locale UsersController::with_locale $UC around "around_action :with_locale" +reaches normalize_name User::normalize_name $UM before_save "before_save :normalize_name" +reaches reindex User::reindex $UM on_floor "after_commit :reindex" +reaches name_present User::name_present $UM validate "validate :name_present" +reaches guest? User::guest? $UM validates_unless "validates …, unless: :guest?" +reaches trim_name User::trim_name $UM two_symbols "before_validation's first symbol" +reaches lower_email User::lower_email $UM two_symbols "… and its second" +reaches notify_team User::notify_team $UM if_array "after_create_commit :notify_team" +reaches active? User::active? $UM if_array "if: [ :active?, … ] — each symbol of the array" +reaches confirmed? User::confirmed? $UM if_array "… and the next" +reaches write_audit Auditable::write_audit app/models/concerns/auditable.rb concern_cb "a concern's included do block runs in the including class" +reaches check_lock SyncJob::check_lock app/jobs/sync_job.rb job_cb "before_perform :check_lock" + +echo "=== send and its kin call the named method on the same receiver ===" +reaches render_user UsersController::render_user $UC send_bare "send( :render_user ) is a call to self" +reaches assist Helper::assist $UC try_typed "helper.try( :assist ) on a Helper — the typed receiver answers" +reaches log_one UsersController::log_one $UC method_ref "method( :log_one ) names self's log_one" + +echo "=== floors ===" +absent show $UC only_floor "floor (a): only: [ :show ] names actions, calls none" +absent index $UC around "floor (a): except: :index names an action" +absent create $UM on_floor "floor (a): on: :create is an event" +absent email $UM validates_unless "floor (a): validates' :email is an attribute" +absent authenticate $UC skip_floor "floor (b): skip_before_action removes a callback" +absent set_trace $UC set_callback_floor "floor (c): set_callback is not read" +absent render_user $UC send_string_floor "floor (d): a string name is not read" +absent late_hook $UC in_def_floor "floor (e): a callback macro inside a def is not at class-body position" + +echo "=== determinism and warm == cold ===" +"$BIN" "$FIX" --no-cache >"$DIR/b.xml" 2>/dev/null +cmp -s "$MAP" "$DIR/b.xml" && ok "byte-identical across two --no-cache runs" \ + || no "output differs across runs" +if ! "$BIN" "$FIX" --cache="$DIR/c.bin" >"$DIR/cold.xml" 2>"$DIR/cold.err" +then + no "the cold cache run exited non-zero: $( head -3 "$DIR/cold.err" )" +fi +if ! "$BIN" "$FIX" --cache="$DIR/c.bin" >"$DIR/warm.xml" 2>"$DIR/warm.err" +then + no "the warm cache run exited non-zero: $( head -3 "$DIR/warm.err" )" +fi +cmp -s "$DIR/cold.xml" "$DIR/warm.xml" && ok "warm run == cold run" \ + || no "the warm cache disagrees with the cold run" + +echo "=== --callers: a callback method has its class as a caller ===" +"$BIN" "$FIX" --no-cache --callers=User::normalize_name >"$DIR/callers.xml" 2>/dev/null +grep -q 'n="User"' "$DIR/callers.xml" && ok "--callers=User::normalize_name names User" \ + || no "--callers=User::normalize_name does not name User: $( head -c 400 "$DIR/callers.xml" )" + +echo +[ "$fail" -eq 0 ] && echo "ALL PASS" || { echo "SOME CHECKS FAILED"; exit 1; } diff --git a/test/rubydescribedclasscheck.sh b/test/rubydescribedclasscheck.sh index fb70190d7..2853127ec 100755 --- a/test/rubydescribedclasscheck.sh +++ b/test/rubydescribedclasscheck.sh @@ -14,13 +14,15 @@ # with no receiver or the receiver `RSpec`, that carries a block. # # Stated floors, pinned below so each stays a decision: -# (a) a CHAINED receiver (`described_class.new.m_chain`) is untouched — the same one-hop bound as #267's -# `Calc.new.scale`: the receiver is a call, not a constant. +# (a) a CHAINED receiver (`described_class.new.m_chain`) was untouched — the same one-hop bound as #267's +# `Calc.new.scale`: the receiver is a call, not a constant — until parser version 132 typed a receiver the code +# builds (test/rubytypedrecvcheck.sh). Its arm below is kept, INVERTED: it now pins m_chain to Calc. # (b) a group with no constant (`RSpec.describe "no class"`, `RSpec.describe :sym`) names no class, so # `described_class` there is left exactly as it was. # (c) a `describe` call on any other receiver (`Docs.describe Calc do`) is not an RSpec example group. -# (d) `subject` — the implicit `described_class.new` — is NOT modeled in this round: an explicit `subject { … }` -# can be anything, and telling the two apart is its own round. +# (d) `subject` — the implicit `described_class.new` — is NOT modeled by this rule: an explicit `subject { … }` +# can be anything. test/rubytypedrecvcheck.sh (parser version 132) types a group's subject, explicit or +# implicit, as a call RECEIVER from what its block builds. # (e) a REDEFINED `described_class` declines: the redefinition names what it means, and this rule does not read # it, so it answers nothing rather than the group's constant. A METHOD of that name — any `:described_class` # symbol (`let( :described_class ) { … }`) or `def described_class` — declines every site in the file, since a @@ -477,8 +479,10 @@ pins calc_spec.rb m_inner Tally Calc "a nested describe Tally is the innermo pins engine_spec.rb e_run Engine Other "a bare describe, and a scope_resolution constant named by its final segment" pins calc_spec.rb m_nil Calc Tally "describe nil passes its parent's class through — RSpec reads nil like a String" -echo "=== floors (a) chained, (b) no constant, (c) not RSpec, (e) redefined — each left exactly as it was ===" -untouched calc_spec.rb m_chain "described_class.new.m_chain — the receiver is a call, the #267 one-hop bound (floor (a), stated)" +echo "=== floor (a) lifted: described_class.new is a typed receiver (parser version 132) ===" +pins calc_spec.rb m_chain Calc Tally "described_class.new.m_chain — the receiver the code builds is a Calc (test/rubytypedrecvcheck.sh)" + +echo "=== floors (b) no constant, (c) not RSpec, (e) redefined — each left exactly as it was ===" untouched noclass_spec.rb m_none "RSpec.describe \"no class\" names no class (floor (b), stated)" untouched noclass_spec.rb m_sym "RSpec.describe :sym names no class (floor (b), stated)" untouched docs_spec.rb m_docs "Docs.describe Calc is not an RSpec example group (floor (c), stated)" diff --git a/test/rubyinheritcheck.sh b/test/rubyinheritcheck.sh index 0295c3d38..f29d5c91e 100755 --- a/test/rubyinheritcheck.sh +++ b/test/rubyinheritcheck.sh @@ -25,11 +25,11 @@ # `extends_clause`, C#'s `base_list`), and in Ruby that descent landed on `Struct`, minting an edge that # only read as absent because the fixture defines no Struct. `class Built < Factory.fabricate( :x )` against # an in-tree `Factory` pins it. -# (b) a MIXIN (`include Helper` / `extend` / `prepend`) is NOT an inheritance edge in this round. It is -# a receiver-less call in the class BODY, not a clause — the same shape, and the same decision, as -# PHP's in-body `use SomeTrait;` (captureBases' own header). Ruby's ancestor chain really does hold -# included modules, so this is a real residue and a later round's subject, not a claim that it is -# not inheritance. +# (b) a MIXIN (`include Helper` / `extend` / `prepend`) is an ANCESTOR for the call graph since parser version +# 131 — the base walk, the CHA cone and a bare call's reach read it (ingest_relations.h captureRubyMixinBases, +# test/rubyreachcheck.sh) — but it mints no IMPLEMENTOR row: --lego lists the classes below a CLASS, and the +# inheritance overlay takes only class-like bases (graph.h namespaceCompatible), which a module is not. So +# --lego=Helper still lists no Mixed; a module's implementor row is a separate decision, not taken here. # (c) the base WALK is keyed by NAME. A base is found by name (its final segment, as every language's is) # and then SCOPED: the superclass as written is looked up the way Ruby looks it up — Module.nesting # innermost first, then the top level, `::X` absolute — against every class/module the tree opens (the @@ -38,9 +38,9 @@ # `class Inner < Base` inside `module Beta` lands on Beta::Base alone, and a base the tree never opens # (`ActiveRecord::Base`) adds nothing to the CHA name graph — Rec's walk no longer reaches Space::Base's # methods. What stays name-keyed is the walk's METHOD probe (canonByName is keyed `Scope::method` with the -# IMMEDIATE scope), so two in-tree bases sharing a final name still share one probe: `UsesAlpha.beta_make` -# pins to Beta::Base#beta_make although UsesAlpha < Alpha::Base. Pinned below; lifting it needs a -# qualified scope on the symbol itself. +# IMMEDIATE scope), so two in-tree bases sharing a final name still share one probe — for a call to self. A call +# on a CLASS walks fully-qualified constants since parser version 137 (test/rubyclassrecvcheck.sh): the +# `UsesAlpha.beta_make` arm below is kept, INVERTED — UsesAlpha < Alpha::Base reaches no Beta::Base#beta_make. # (d) `Built = Class.new( Parent )` — with or without a block — makes Built < Parent at runtime, but it is a constant # ASSIGNMENT whose value is a call, not a `class` open: no class symbol, no superclass clause, no edge. The same # decision as (a): the tool reads a base off a class header, never off a computed value. @@ -333,10 +333,11 @@ CLS="$( "$BIN" "$FIX" --no-cache --callers=Parent::shared_helper 2>/dev/null )" echo "$CLS" | grep -q 'n="use_inherited"' && ok "--callers=Parent::shared_helper lists use_inherited" \ || no "--callers=Parent::shared_helper does not list use_inherited" +# Parser version 130 (test/rubybarecallcheck.sh) mints the bare, parenthesis-less call this arm once pinned as minting +# nothing; it now takes the same base walk as the parenthesised shared_helper( 1 ) above. UB="$( rowOf 'n="use_bare" ' )" -echo "$UB" | grep -q '"$DIR/b.xml" 2>/dev/null @@ -442,12 +458,21 @@ then || ok "mutation: Child is no longer an implementor of Parent" fi MQ="$( "$BIN" "$MUT" --no-cache 2>/dev/null | sed 's/>\n/dev/null )" -echo "$MCU" | grep -q 'n="call_inherited"' && ok "mutation: …and Unrelated::build is back among its callers" \ - || no "mutation: Unrelated::build does not list call_inherited: $( echo "$MCU" | grep -o ']*' )" +if echo "$MCU" | grep -q 'n="call_inherited"' +then + no "mutation: Unrelated::build lists call_inherited — a class whose lookup leaves the tree bound to a namesake" +else + ok "mutation: …and Unrelated::build is not among its callers either" +fi echo "=== #325 disclosure: a base left on the final-segment rule is counted as ruby_bases_unscoped= ===" # resolve.h's fallback — an inherit reference with no superclass directive at its class open keeps the byName rule — @@ -460,8 +485,12 @@ then && no "absence: --lego=Parent carries ruby_bases_unscoped= on a tree where every Ruby base was scoped" \ || ok "absence: no ruby_bases_unscoped= on --lego=Parent when every Ruby base was scoped (absent at zero)" fi -# The expected count: every `class X < Const` in the fixture (a computed superclass mints no inherit reference). -WANT="$( cat "$FIX"/*.rb | grep -cE '^[[:space:]]*class [A-Z][A-Za-z0-9_:]* < (::)?[A-Z][A-Za-z0-9_]*(::[A-Z][A-Za-z0-9_]*)*[[:space:]]*(;|#|$)' )" +# The expected count: every `class X < Const` in the fixture (a computed superclass mints no inherit reference) and, +# since parser version 131, every constant of a class-body `include`/`extend`/`prepend` — a mixin is an ancestor too +# (ingest_relations.h captureRubyMixinBases), so the seam drops its directive from the join as well. +SUPERS="$( cat "$FIX"/*.rb | grep -cE '^[[:space:]]*class [A-Z][A-Za-z0-9_:]* < (::)?[A-Z][A-Za-z0-9_]*(::[A-Z][A-Za-z0-9_]*)*[[:space:]]*(;|#|$)' )" +MIXINS="$( cat "$FIX"/*.rb | grep -E '^[[:space:]]+(include|extend|prepend) ' | grep -oE '(::)?[A-Z][A-Za-z0-9_]*(::[A-Z][A-Za-z0-9_]*)*' | grep -c . )" +WANT=$(( SUPERS + MIXINS )) if RIPWIRE_TEST_RUBY_BASE_UNSCOPED=1 runq SEAMED "$FIX" --no-cache --lego=Parent --legend=full then echo "$SEAMED" | grep -q " ruby_bases_unscoped=\"$WANT\"" \ diff --git a/test/rubyrakejbuildercheck.sh b/test/rubyrakejbuildercheck.sh new file mode 100755 index 000000000..1b275f08c --- /dev/null +++ b/test/rubyrakejbuildercheck.sh @@ -0,0 +1,299 @@ +#!/usr/bin/env bash +# rubyrakejbuildercheck.sh — gate for the two plain-Ruby file kinds a Rails application holds besides `.rb`: a Rake task +# file (`lib/tasks/*.rake`) and a Jbuilder view (`app/views/**/*.json.jbuilder`). Both are Ruby, parsed by the Ruby +# grammar, and before this round neither was crawled: a method only a task or a JSON view called had no caller. +# THE RULE (ingest_crawl.h kLangTable; ingest_binds.h RubyBareCallWalk; graph.h RubyTopSelf; parser version 135): +# * both extensions index as Ruby, with every Ruby rule the `.rb` files get (typed receivers, bare-word calls); +# * outside any class, Ruby's self is what runs the file. Rake loads a `.rake` file at top level and calls a task's +# block as written, so self there is `main`: a receiver-less call reaches a top-level def or a reopened Object, +# Kernel or BasicObject, and no in-tree class's method. ActionView compiles a template into a method of the view, so +# self in a `.jbuilder` template is the view: it also reaches each method of a helper module the application +# defines (Rails' `helper :all`: the modules of the files under app/helpers/) and each controller method a +# `helper_method` declaration names. Any other namesake — a partial's local (`badge` in `_badge.json.jbuilder`), +# a route helper, ActionView's own — is refused as external, and so is a def in test code (filter.h +# isTestSymbol: a def inside an example group's block has no class around it, and is the group's method); +# * `json` in a template is the template's own JbuilderTemplate (the handler binds it), not a call: a call on it +# (`json.summary …`, `json.partial! …`) is Jbuilder's — every key is a method_missing — and is refused as external +# unless the tree opens JbuilderTemplate. +# +# Stated floors, each pinned below: +# (a) `json.extract! user, :email` names attributes by symbol; they are not read as calls. +# (b) a `.rb` file's top level is left as before: a block there may run with another self (`describe`, +# `routes.draw`, `FactoryBot.define`), so a receiver-less call outside any class binds by name. +# +# Usage: test/rubyrakejbuildercheck.sh | RIPWIRE_BIN=asan/ripwire test/rubyrakejbuildercheck.sh +# Exits non-zero on any failure. Self-contained via mktemp. +set -u +ROOT="$( cd "$( dirname "$0" )/.." && pwd )" +BIN="${1:-${RIPWIRE_BIN:-$ROOT/build/ripwire}}" +[ "${BIN#/}" = "$BIN" ] && BIN="$ROOT/$BIN" +fail=0 +ok(){ echo " PASS $1" || { fail=1; echo " FAIL could not write the PASS line for: $1"; }; return 0; } +no(){ echo " FAIL $1"; fail=1; } + +[ -x "$BIN" ] || { echo "no ripwire binary at $BIN — build first (cmake --build build -j)"; exit 2; } + +DIR="$( mktemp -d )"; trap 'rm -rf "$DIR"' EXIT +FIX="$DIR/fix" +for d in app/controllers app/models app/helpers app/views/users lib/tasks script spec/support spec/views +do + mkdir -p "$FIX/$d" +done + +cat > "$FIX/app/controllers/users_controller.rb" <<'RUBY' +class UsersController < ApplicationController + helper_method :contact_line + + def show + @user = User.find( params[:id] ) + end + + private + + def contact_line + "contact" + end + + def secret_token + "token" + end +end +RUBY + +cat > "$FIX/app/models/user.rb" <<'RUBY' +class User < ApplicationRecord + def display_name + "name" + end +end +RUBY + +cat > "$FIX/app/helpers/avatar_helper.rb" <<'RUBY' +module AvatarHelper + def avatar_url_for( user ) + user.to_s + end +end +RUBY + +# a module named like a helper, outside app/helpers/: a spec's, which no view mixes in +cat > "$FIX/spec/support/api_helper.rb" <<'RUBY' +module ApiHelper + def json_response + {} + end +end +RUBY + +# a def inside an example group's block has no class around it, and is still the group's method: test code +cat > "$FIX/spec/views/show_spec.rb" <<'RUBY' +RSpec.describe "users/show" do + def page_title + "title" + end +end +RUBY + +cat > "$FIX/lib/site.rb" <<'RUBY' +class Object + def site_name + "site" + end +end +RUBY + +cat > "$FIX/lib/indexer.rb" <<'RUBY' +class Indexer + def rebuild + true + end + + def self.purge + true + end +end +RUBY + +# Every name the fixture's calls might bind to by name alone: a Decoy target is a wrong edge. +cat > "$FIX/lib/decoy.rb" <<'RUBY' +class Decoy + def json; end + def summary; end + def headline; end + def email; end + def badge; end + def json_response; end + def secret_token; end + def avatar_url_for( u ); end + def contact_line; end + def rebuild; end + def purge_stale; end + def cleanup; end + def report; end +end +RUBY + +SHOW=app/views/users/show.json.jbuilder +cat > "$FIX/$SHOW" <<'RUBY' +json.id @user.id # @json_local +json.summary "x" # @json_key +json.display @user.display_name # @dotted +json.avatar avatar_url_for( @user ) # @helper +json.contact contact_line # @helper_method +json.token secret_token # @not_helper_method +json.api json_response # @spec_helper +json.site site_name # @root +json.title page_title # @test_def +json.extract! @user, :email # @extract_floor +json.partial! "users/badge", badge: @user # @partial +json.posts [ 1 ] do |post| + json.headline post # @in_block +end +RUBY + +BADGE=app/views/users/_badge.json.jbuilder +cat > "$FIX/$BADGE" <<'RUBY' +json.label badge # @partial_local +RUBY + +RAKE=lib/tasks/search.rake +cat > "$FIX/$RAKE" <<'RUBY' +namespace :search do + desc "Rebuild the index" + task reindex: :environment do + Indexer.new.rebuild # @rake_typed + purge_stale # @rake_bare + cleanup # @rake_main + end +end + +def purge_stale + Indexer.purge +end +RUBY + +# floor (b): the same receiver-less call from a `.rb` file's top level still binds by name +cat > "$FIX/script/report.rb" <<'RUBY' +report # @rb_top +RUBY + +echo "=== the map: both file kinds are indexed, and the output is well-formed ===" +MAP="$DIR/a.xml" +if ! "$BIN" "$FIX" --no-cache >"$MAP" 2>"$DIR/a.err" +then + no "the default map exited non-zero: $( head -3 "$DIR/a.err" )" +fi +head -c 2000 "$MAP" | grep -o 'unindexed="[^"]*"' >"$DIR/unindexed" || true +if grep -qE 'rake|jbuilder' "$DIR/unindexed" +then + no "the header still lists .rake or .jbuilder files as unindexed: $( cat "$DIR/unindexed" )" +else + ok "neither .rake nor .jbuilder is in the header's unindexed= ($( cat "$DIR/unindexed" ))" +fi +if command -v xmllint >/dev/null 2>&1 +then + if xmllint --noout "$MAP"; then ok "xmllint --noout"; else no "xmllint failed"; fi +fi + +# The census (src/pincensus.h): one `C` row per call site the resolver decided or refused, so each arm reads ONE line. +census(){ + if ! "$BIN" "$1" --no-cache --pin-census="$DIR/census.tsv" >/dev/null 2>"$DIR/census.err" + then + no "--pin-census on $1 exited non-zero: $( head -3 "$DIR/census.err" )" + return 2 + fi + return 0 +} +# line FILE MARK — the line number carrying `# @MARK` (empty when the marker is missing) +line(){ + grep -n "# @$2\$" "$FIX/$1" | head -1 | cut -d: -f1 +} +# rows FILE LINE CALLEE — "mechanismtargets" of each CALLEE call the census records on FILE:LINE +rows(){ + awk -F'\t' -v f="$1" -v l="$2" -v c="$3" '$1 == "C" && $9 == l && $7 == c && index( $6, f "::" ) == 1 { print $2 "\t" $8 }' "$DIR/census.tsv" +} +# reaches CALLEE SYM FILE MARK WHY — the CALLEE call on the marked line resolves to SYM, and to no Decoy +reaches(){ + local n r; n="$( line "$3" "$4" )" + [ -n "$n" ] || { no "fixture marker @$4 missing in $3"; return; } + r="$( rows "$3" "$n" "$1" )" + if printf '%s\n' "$r" | grep -qF "::$2#" && ! printf '%s\n' "$r" | grep -qF "::Decoy::" + then + ok "@$4 :$1 → $2 ($5)" + else + no "@$4 ($3:$n) :$1 does not reach $2 alone ($5); census: $( printf '%s' "$r" | tr '\t\n' ' ;' )" + fi +} +# refused CALLEE FILE MARK WHY — the CALLEE call on the marked line is recorded and refused as external +refused(){ + local n r; n="$( line "$2" "$3" )" + [ -n "$n" ] || { no "fixture marker @$3 missing in $2"; return; } + r="$( rows "$2" "$n" "$1" )" + if [ -n "$r" ] && ! printf '%s\n' "$r" | grep -qv "^external \$" + then + ok "@$3 :$1 refused as external ($4)" + else + no "@$3 ($2:$n) :$1 is not refused ($4); census: $( printf '%s' "$r" | tr '\t\n' ' ;' )" + fi +} +# absent CALLEE FILE MARK WHY — no CALLEE call is recorded on the marked line (a Decoy namesake would catch one) +absent(){ + local n r; n="$( line "$2" "$3" )" + [ -n "$n" ] || { no "fixture marker @$3 missing in $2"; return; } + r="$( rows "$2" "$n" "$1" )" + if [ -z "$r" ] + then + ok "@$3 makes no :$1 call ($4)" + else + no "@$3 ($2:$n) records a :$1 call ($4); census: $( printf '%s' "$r" | tr '\t\n' ' ;' )" + fi +} + +census "$FIX" || exit 1 + +echo "=== a Rake task file is Ruby, and its top level's self is main ===" +reaches rebuild Indexer::rebuild $RAKE rake_typed "Indexer.new.rebuild in a task block — the typed receiver answers" +reaches purge_stale purge_stale $RAKE rake_bare "a top-level def of the file: a method of main" +refused cleanup $RAKE rake_main "an in-tree class's method is no method of main" + +echo "=== a Jbuilder template is Ruby, its json is Jbuilder's, and its self is the view ===" +absent json $SHOW json_local "json is the template's local: no call named json" +refused summary $SHOW json_key "json.summary is a Jbuilder key (method_missing), not Decoy's" +refused headline $SHOW in_block "inside a block, json is still the template's" +reaches display_name User::display_name $SHOW dotted "a call on a value reads as in any Ruby file" +reaches avatar_url_for AvatarHelper::avatar_url_for $SHOW helper "a helper module's method (app/helpers/): the view mixes it in" +reaches contact_line UsersController::contact_line $SHOW helper_method "a controller method helper_method declares" +refused secret_token $SHOW not_helper_method "a controller method no helper_method declares is not the view's" +refused json_response $SHOW spec_helper "a module outside app/helpers/ is not mixed into the view" +reaches site_name Object::site_name $SHOW root "a reopened Object answers every self" +refused badge $BADGE partial_local "a partial's local is no method of the view" +refused page_title $SHOW test_def "a def in test code (an example group's) is no method of the view" + +echo "=== stated floors ===" +absent email $SHOW extract_floor "floor (a): extract!'s symbols are attribute names, not read as calls" +RB_TOP="$( rows script/report.rb "$( line script/report.rb rb_top )" report )" +printf '%s\n' "$RB_TOP" | grep -qF "::Decoy::report#" && ok "floor (b) pinned: script/report.rb's top-level report → Decoy::report" \ + || no "floor (b): script/report.rb's top-level report no longer binds to Decoy::report; census: $( printf '%s' "$RB_TOP" | tr '\t\n' ' ;' )" + +echo "=== determinism and warm == cold ===" +"$BIN" "$FIX" --no-cache >"$DIR/b.xml" 2>/dev/null +cmp -s "$MAP" "$DIR/b.xml" && ok "byte-identical across two --no-cache runs" \ + || no "output differs across runs" +if ! "$BIN" "$FIX" --cache="$DIR/c.bin" >"$DIR/cold.xml" 2>"$DIR/cold.err" +then + no "the cold cache run exited non-zero: $( head -3 "$DIR/cold.err" )" +fi +if ! "$BIN" "$FIX" --cache="$DIR/c.bin" >"$DIR/warm.xml" 2>"$DIR/warm.err" +then + no "the warm cache run exited non-zero: $( head -3 "$DIR/warm.err" )" +fi +cmp -s "$DIR/cold.xml" "$DIR/warm.xml" && ok "warm run == cold run" \ + || no "the warm cache disagrees with the cold run" + +echo "=== --callers: a helper_method's caller is the template ===" +"$BIN" "$FIX" --no-cache --callers=UsersController::contact_line >"$DIR/callers.xml" 2>/dev/null +grep -q 'show.json.jbuilder' "$DIR/callers.xml" && ok "--callers=UsersController::contact_line names show.json.jbuilder" \ + || no "--callers=UsersController::contact_line does not name the template: $( head -c 400 "$DIR/callers.xml" )" + +echo +[ "$fail" -eq 0 ] && echo "ALL PASS" || { echo "SOME CHECKS FAILED"; exit 1; } diff --git a/test/rubyreachcheck.sh b/test/rubyreachcheck.sh new file mode 100755 index 000000000..6708abbe8 --- /dev/null +++ b/test/rubyreachcheck.sh @@ -0,0 +1,636 @@ +#!/usr/bin/env bash +# rubyreachcheck.sh — gate for what a Ruby BARE call can reach. A receiver-less `m` / `m( x )` inside a class or module +# is a send to self, and Ruby's method lookup decides which `def` answers it: the class, its included, extended and +# prepended modules, its superclass and theirs — and, because self may be any instance below the defining class, what +# a subclass or an includer adds (the template-method and concern idioms). A method of any OTHER class can never answer +# it; neither can an enclosing class's method (lexical nesting is constant lookup, not method lookup — `outer_helper` +# from inside `Outer::Inner` is a NoMethodError). Before this round three things broke that: +# * mixins were no ancestor: `include Hub::SignalMixin` put nothing in the caller's base walk, so `signal` +# split between the mixin's def and an unrelated `Hub#signal` in the same file, and a concern's call to its +# includer's method (`audit_target`) declined; +# * a class-BODY call ran against the enclosing namespace: `define_section :first` inside `Ns::Catalog` read Rule 1's +# self as `Ns` and bound to `Ns.define_section`, not `Catalog.define_section`; +# * a candidate no lookup can reach still bound by name: `render` in a controller to a component's `def render`, +# `request` to the `Rating#request` of a class the controller merely names, `errors` in an ActiveModel form to a +# stub's `def errors` — the method Ruby runs is the framework's, outside the tree. +# THE RULE (graph.h RubySelfReach, buildRubyCha; resolve.h rubyScopeMixinReferences): mixins are Ruby bases — one +# written inside a concern's `included do … end` too — scoped by Ruby's constant lookup like a superclass, and read +# only by a Ruby call to self, as is a concern's nested `module ClassMethods` (extended onto its includer); a class-body +# call's self is the class; and a bare call inside a class or module keeps only the candidates +# whose owner lies in that reach — the caller's ancestors, and the ancestors of every class below it — plus top-level +# defs (private methods of Object, reachable from anywhere). When none is left the call is refused as external (graph.h +# vetoExternal, counted in the header's external=, recorded as mechanism `external` by --pin-census): the method Ruby +# runs is outside the indexed tree, and the tool says so rather than naming a namesake. +# +# Stated floors, each pinned below where a fixture can show it: +# (a) a class whose lookup can leave the class — a `SimpleDelegator`/`Delegator` or Draper decorator in reach, a +# `method_missing` or a `delegate_missing_to` in reach — refuses nothing: its bare calls bind as before. A computed +# superclass (`< DelegateClass( User )`) names no base, so its class is not seen as one. +# (b) the delegation DSL — ActiveSupport's `delegate :a, to: :x` and `delegate_missing_to`, Forwardable's +# `def_delegator(s)` — defines methods the tree does not index, so a class that writes it (or one whose reach holds +# such a class) refuses nothing: its bare calls bind by name as before. Indexing the delegated names as methods +# is a later round's (they must answer a call to self only — a receiver call of that name is no evidence). +# (h) a top-level def is a private method of Object, and any ancestor answers before Object does: a class whose +# reach holds an out-of-tree superclass or mixin (ActionController::Base) does not reach a top-level def. +# (i) a mixin written inside a METHOD body runs when the method does (activerecord's `primary_key=` includes +# CompositePrimaryKey; `attr_readonly` includes HasReadonlyAttributes) and is not read: calls that need it are +# refused as external. +# (j) Rails mixes every app/helpers module into one view object at run time; the tree says nothing of it, so a helper +# module's bare call to another helper module's method is refused as external. +# (k) a core class's own ancestry is not modelled beyond every self's roots (BasicObject, Object, Kernel, and Module +# and Class): a reopened `class Array` calling a reopened `module Enumerable`'s method is refused as external. +# (c) PARTLY LIFTED at parser version 145 (test/rubyclassrecvcheck.sh floor (i)): a call to self in a class's def answers +# first from its own side's lookup (an instance's in an instance method, the class object's in a singleton method), +# and among the namesakes it reaches by name an instance admits no singleton method. Still: `extend M` and `include M` +# both put M in reach, so when that lookup misses, an instance method's call to an extended module's method is admitted. +# (d) inside `instance_eval`/`class_eval`/`instance_exec` blocks self changes; the rule reads the lexical self, as +# Rule 1 always has, so a DSL block run against another in-tree object has its calls there refused as external. +# (e) reach is read by class NAME, as the inheritance graph is keyed: two classes sharing a name share their reach, +# which refuses less, never more. +# (f) a call outside any class — a script's top level, an RSpec example group's blocks — is untouched by the rule. +# (g) `prepend M` is read as `include M`: a prepended override is not preferred over the class's own method. +# +# Usage: test/rubyreachcheck.sh | RIPWIRE_BIN=asan/ripwire test/rubyreachcheck.sh +# Exits non-zero on any failure. Self-contained via mktemp. +set -u +ROOT="$( cd "$( dirname "$0" )/.." && pwd )" +BIN="${1:-${RIPWIRE_BIN:-$ROOT/build/ripwire}}" +[ "${BIN#/}" = "$BIN" ] && BIN="$ROOT/$BIN" +fail=0 +ok(){ echo " PASS $1" || { fail=1; echo " FAIL could not write the PASS line for: $1"; }; return 0; } +no(){ echo " FAIL $1"; fail=1; } + +[ -x "$BIN" ] || { echo "no ripwire binary at $BIN — build first (cmake --build build -j)"; exit 2; } + +DIR="$( mktemp -d )"; trap 'rm -rf "$DIR"' EXIT +FIX="$DIR/fix" +for d in lib/coord lib/app lib/ns lib/dsl lib/other lib/services lib/tmpl lib/impls lib/concerns lib/orders lib/util lib/deco \ + lib/models lib/ghost lib/forms lib/stubs app/models spec lib/feed lib/feed/shared lib/feed/catalog lib/sql lib/quoting \ + lib/attrs lib/core_ext +do + mkdir -p "$FIX/$d" +done + +cat > "$FIX/app/models/application_record.rb" <<'RUBY' +class ApplicationRecord < ActiveRecord::Base + self.abstract_class = true +end +RUBY +cat > "$FIX/app/models/post.rb" <<'RUBY' +class Post < ApplicationRecord + scope :published, -> { where(published: true) } +end +RUBY +cat > "$FIX/lib/app/ctl.rb" <<'RUBY' +class Ctl + include Hub::SignalMixin + + def act + signal(:acted) + end +end +RUBY +cat > "$FIX/lib/app/outer.rb" <<'RUBY' +class Outer + def outer_helper + 6 + end + + class Inner + def m + outer_helper + end + end +end +RUBY +cat > "$FIX/lib/app/pages_controller.rb" <<'RUBY' +class PagesController < ActionController::Base + def show + render + request + Rating.check + end +end +RUBY +cat > "$FIX/lib/app/registry.rb" <<'RUBY' +class Registry + extend RegistryDsl + + register :a +end +RUBY +cat > "$FIX/lib/app/uses_top.rb" <<'RUBY' +class UsesTop + def m + top_helper + end +end +RUBY +cat > "$FIX/lib/concerns/auditable.rb" <<'RUBY' +module Auditable + def audit + audit_target + end +end +RUBY +cat > "$FIX/lib/coord/hub.rb" <<'RUBY' +class Hub + def signal(name) + name + end + + module SignalMixin + def signal(name) + name + end + end +end +RUBY +cat > "$FIX/lib/deco/user_deco.rb" <<'RUBY' +class UserDeco < SimpleDelegator + def label + display_name + end +end +RUBY +cat > "$FIX/lib/dsl/registry_dsl.rb" <<'RUBY' +module RegistryDsl + def register(name) + name + end +end +RUBY +cat > "$FIX/lib/forms/signup_form.rb" <<'RUBY' +class SignupForm + include ActiveModel::Model + + def check + errors + end +end +RUBY +cat > "$FIX/lib/ghost/ghost.rb" <<'RUBY' +class Ghost + def method_missing(name, *args) + name + end + + def respond_to_missing?(name, include_private = false) + true + end + + def m + phantom + end +end +RUBY +cat > "$FIX/lib/impls/impl.rb" <<'RUBY' +class Impl < BaseTemplate + def hook + 1 + end +end +RUBY +cat > "$FIX/lib/models/user_model.rb" <<'RUBY' +class UserModel + def display_name + "u" + end + + def phantom + "p" + end +end +RUBY +cat > "$FIX/lib/ns/catalog.rb" <<'RUBY' +module Ns + def self.define_section(name) + name + end + + class Catalog + def self.define_section(name) + name + end + + define_section :first + end +end +RUBY +cat > "$FIX/lib/orders/order.rb" <<'RUBY' +class Order + include Auditable + + def audit_target + 4 + end +end +RUBY +cat > "$FIX/lib/other/component.rb" <<'RUBY' +class Component + def render + "component" + end +end +RUBY +cat > "$FIX/lib/other/robot.rb" <<'RUBY' +class Robot + def register(name) + name + end +end +RUBY +cat > "$FIX/lib/other/spec_support.rb" <<'RUBY' +def spec_only_helper + 7 +end +RUBY +cat > "$FIX/lib/other/unrelated.rb" <<'RUBY' +class Unrelated + def hook + 2 + end + + def audit_target + 3 + end +end +RUBY +cat > "$FIX/lib/services/rating.rb" <<'RUBY' +class Rating + def self.check + true + end + + def request + nil + end +end +RUBY +cat > "$FIX/lib/stubs/link_stub.rb" <<'RUBY' +class LinkStub + def errors + end + + def where(conditions) + conditions + end +end +RUBY +cat > "$FIX/lib/tmpl/base_template.rb" <<'RUBY' +class BaseTemplate + def template + hook + end +end +RUBY +cat > "$FIX/lib/util/helpers.rb" <<'RUBY' +def top_helper + 5 +end +RUBY +cat > "$FIX/spec/page_spec.rb" <<'RUBY' +describe "pages" do + it "works" do + spec_only_helper + end +end +RUBY +# A concern's call reaches its includer's superclass (Renderer's attr_reader), while the one namesake in a file the +# concern REFERENCES (CommentsMod) is out of reach: Rule 3's include narrow must choose among the reachable only. +cat > "$FIX/lib/feed/renderer.rb" <<'RUBY' +class Renderer + attr_reader :story +end +RUBY +cat > "$FIX/lib/feed/shared/comments_mod.rb" <<'RUBY' +module CommentsMod + def self.version + 1 + end + + def story + nil + end +end +RUBY +cat > "$FIX/lib/feed/shared/group_content.rb" <<'RUBY' +module GroupContent + def summary + story + end + + def self.helper + CommentsMod.version + end +end +RUBY +cat > "$FIX/lib/feed/catalog/reacted.rb" <<'RUBY' +class Reacted < Renderer + include GroupContent +end +RUBY +# A class that writes the delegation DSL (ActiveSupport's `delegate`/`delegate_missing_to`, Forwardable's +# `def_delegator(s)`) defines methods the tree does not index, so it is exempt: its bare calls bind by name as before. +# The Quoting module and Pool class hold the only definitions of the delegated names, out of reach. +cat > "$FIX/lib/quoting/quoting.rb" <<'RUBY' +module Quoting + def quote_name(n) + n + end + + def owner_name + "o" + end +end +RUBY +cat > "$FIX/lib/quoting/pool.rb" <<'RUBY' +class Pool + def size + 0 + end + + def checkout + nil + end + + def forwarded_anywhere + nil + end + + def computed_name + nil + end +end +RUBY +cat > "$FIX/lib/sql/creation.rb" <<'RUBY' +class Creation + delegate :quote_name, to: :@conn, private: true + delegate :name, to: :owner, prefix: true + + def build + quote_name(:t) + end + + def label + owner_name + end +end +RUBY +cat > "$FIX/lib/sql/counter.rb" <<'RUBY' +class Counter + def_delegators :@pool, :checkout + + def count + checkout + end +end +RUBY +cat > "$FIX/lib/sql/top_ctl.rb" <<'RUBY' +class TopCtl < ActionController::Base + def display + top_helper + end +end +RUBY +cat > "$FIX/lib/sql/forwarder.rb" <<'RUBY' +class Forwarder + delegate_missing_to :target + + def go + forwarded_anywhere + end +end +RUBY +cat > "$FIX/lib/sql/computed.rb" <<'RUBY' +class Computed + NAMES = %i[computed_name].freeze + delegate( *NAMES, to: :inner ) + + def go + computed_name + end +end +RUBY +# A concern's `included do … end` runs in the INCLUDER, so a mixin written there joins the includer's ancestors — +# activerecord's AttributeMethods assembles Write and PrimaryKey this way; PrimaryKey#id= reaches Write through it. +cat > "$FIX/lib/attrs/attribute_methods.rb" <<'RUBY' +module AttributeMethods + extend ActiveSupport::Concern + + included do + include Write + include PrimaryKey + end +end +RUBY +cat > "$FIX/lib/attrs/write.rb" <<'RUBY' +module Write + def _write_attribute(name, value) + value + end +end +RUBY +cat > "$FIX/lib/attrs/primary_key.rb" <<'RUBY' +module PrimaryKey + def id=(value) + _write_attribute(:id, value) + end +end +RUBY +cat > "$FIX/lib/attrs/record_base.rb" <<'RUBY' +class RecordBase + include AttributeMethods +end +RUBY +cat > "$FIX/lib/other/writer_decoy.rb" <<'RUBY' +class WriterDecoy + def _write_attribute(name, value) + name + end +end +RUBY +# A reopened Object is every self's ancestor (activesupport's core_ext: Object#blank?, Kernel#silence_warnings). +cat > "$FIX/lib/core_ext/object.rb" <<'RUBY' +class Object + def blank_ish? + false + end +end +RUBY +cat > "$FIX/lib/app/uses_object.rb" <<'RUBY' +class UsesObject + def probe_blank + blank_ish? + end +end +RUBY +cat > "$FIX/lib/other/blank_decoy.rb" <<'RUBY' +class BlankDecoy + def blank_ish? + true + end +end +RUBY +# A concern's nested `module ClassMethods` is extended onto its includer (ActiveSupport::Concern): the includer's +# class-body DSL call reaches it. +cat > "$FIX/lib/concerns/callbacky.rb" <<'RUBY' +module Callbacky + extend ActiveSupport::Concern + + module ClassMethods + def define_hooks(*names) + names + end + end +end +RUBY +cat > "$FIX/lib/app/wrapper.rb" <<'RUBY' +class Wrapper + include Callbacky + + define_hooks :run +end +RUBY +cat > "$FIX/lib/other/hook_decoy.rb" <<'RUBY' +class HookDecoy + def define_hooks(*names) + names + end +end +RUBY + +MAP="$DIR/map.xml" +"$BIN" "$FIX" --no-cache >"$MAP" 2>"$DIR/map.err" +if [ $? -eq 0 ]; then ok "default map exits 0"; else no "default map exited non-zero: $( cat "$DIR/map.err" )"; fi +if [ -s "$DIR/map.err" ]; then no "unexpected stderr: $( head -3 "$DIR/map.err" )"; else ok "clean stderr"; fi +if command -v xmllint >/dev/null 2>&1 +then + if xmllint --noout "$MAP"; then ok "xmllint --noout"; else no "xmllint failed"; fi +fi + +# callers ROOT SYM — the --callers=SYM caller rows, one per line, in $DIR/c.rows; returns 2 when the run failed, so an +# absence arm can never read a crashed run as "no caller". +callers(){ + if ! "$BIN" "$1" --no-cache --callers="$2" >"$DIR/c.out" 2>"$DIR/c.err" + then + no "--callers=$2 exited non-zero: $( head -3 "$DIR/c.err" )" + return 2 + fi + sed 's/>\n/dev/null 2>"$DIR/census.err" +then + refusedFrom lib/app/pages_controller.rb::PagesController::show render "ActionController's render" + refusedFrom lib/app/pages_controller.rb::PagesController::show request "ActionController's request" + refusedFrom lib/forms/signup_form.rb::SignupForm::check errors "ActiveModel's errors" + refusedFrom app/models/post.rb::Post where "the relation's where" + refusedFrom lib/app/outer.rb::Inner::m outer_helper "no ancestor of Inner defines it" +else + no "--pin-census exited non-zero: $( head -3 "$DIR/census.err" )" +fi + +echo "=== floor (b): a class that delegates is exempt — its bare calls bind by name as before ===" +reaches Quoting::quote_name build "delegate :quote_name, to: :@conn — the delegated method is not indexed, Creation is exempt" +reaches Quoting::owner_name label "delegate … prefix: true — exempt the same way" +reaches Pool::checkout count "def_delegators :@pool, :checkout — Forwardable, exempt" +reaches Pool::forwarded_anywhere go "delegate_missing_to forwards any name, as method_missing does" +reaches Pool::computed_name go "delegate( *NAMES, … ) — the names are computed, the class is exempt" + +echo "=== a top-level def is reachable only while every ancestor in reach is in the tree ===" +misses top_helper display "TopCtl < ActionController::Base: the out-of-tree ancestor answers before Object's private method" + +echo "=== a reopened Object, Kernel, Module or Class is every self's ancestor ===" +reaches Object::blank_ish? probe_blank "class Object; def blank_ish? — any self reaches it" +misses BlankDecoy::blank_ish? probe_blank "an unrelated class's namesake stays out of reach" + +echo "=== what the rule leaves alone ===" +reaches top_helper m "a top-level def is a private method of Object, reachable from any self" +reaches UserModel::display_name label "floor (a): a SimpleDelegator subclass forwards what it lacks" +reaches UserModel::phantom m "floor (a): a class with method_missing answers any name" +reaches spec_only_helper "<file-scope>" "floor (f): an example group's block is in no class" + +echo "=== determinism and warm == cold ===" +"$BIN" "$FIX" --no-cache >"$DIR/b.xml" 2>/dev/null +cmp -s "$MAP" "$DIR/b.xml" && ok "byte-identical across two --no-cache runs" \ + || no "output differs across runs" +if ! "$BIN" "$FIX" --cache="$DIR/c.bin" >"$DIR/cold.xml" 2>"$DIR/cold.err" +then + no "the cold cache run exited non-zero: $( head -3 "$DIR/cold.err" )" +fi +if ! "$BIN" "$FIX" --cache="$DIR/c.bin" >"$DIR/warm.xml" 2>"$DIR/warm.err" +then + no "the warm cache run exited non-zero: $( head -3 "$DIR/warm.err" )" +fi +cmp -s "$DIR/cold.xml" "$DIR/warm.xml" && ok "warm run == cold run" \ + || no "the warm cache disagrees with the cold run" + +echo "=== mutation: drop the include → the concern no longer reaches Order ===" +MUT="$DIR/mut"; cp -R "$FIX" "$MUT" +grep -v '^ include Auditable$' "$FIX/lib/orders/order.rb" >"$MUT/lib/orders/order.rb" +grep -q 'include Auditable' "$MUT/lib/orders/order.rb" && no "mutation did not apply" +if callers "$MUT" "Order::audit_target" +then + grep -q ' n="audit"' "$DIR/c.rows" && no "mutation: without the include, audit still reaches Order::audit_target" \ + || ok "mutation: without the include, audit no longer reaches Order::audit_target" +fi + +echo +[ "$fail" -eq 0 ] && echo "ALL PASS" || { echo "SOME CHECKS FAILED"; exit 1; } diff --git a/test/rubyrecvnarrowcheck.sh b/test/rubyrecvnarrowcheck.sh index e23982cd1..a18956d8c 100755 --- a/test/rubyrecvnarrowcheck.sh +++ b/test/rubyrecvnarrowcheck.sh @@ -24,17 +24,23 @@ # maps to SymKind::Other (ingest_crawl.h::defKind) for every language, and in Ruby that kind is reached # by nothing else — queries/ruby/tags.scm emits class, module, method and constant, and the other three # have kinds of their own — so buildGraph's Rule 2c class-name set takes Ruby's SymKind::Other symbols. -# * A MISS never deletes an edge: the receiver naming no in-repo definition (`Time.now`), or naming one -# that does not define the callee, degrades to the unchanged honest ladder. +# * A receiver naming no in-repo definition (`Time.now`) mints no edge. One naming a class whose lookup does not +# define the callee degraded to the name ladder through parser version 136; since 137 Ruby's lookup on the class +# object decides it (test/rubyclassrecvcheck.sh): `Calc.report` reaches no Tally.report and is refused as +# external. That arm below is kept, INVERTED. # # Stated floors, pinned below so each stays a decision rather than an accident: # (a) LIFTED at parser version 98 by test/rubyinheritcheck.sh (`class Child < Parent` now mints an # inherit ref, so chaUp holds Ruby and the base walk runs). The arm below is kept, INVERTED: it # now asserts `Child.build` pins to Parent::build, and it is the tripwire that fires if Ruby ever # loses its inheritance edges again. -# (b) Matching is by FINAL SEGMENT, so two classes with the same last name in different namespaces -# both defining the callee keep BOTH candidates (an honest split), exactly as Rule 2c documents. -# (c) A variable receiver (`c.scale`) and a chained one (`Calc.new.scale`) are untouched by this round. +# (b) LIFTED at parser version 137 by test/rubyclassrecvcheck.sh: a call on a class reads the constant by its +# fully-qualified name through the constant index, so `Left::Shared.go` reaches Left's Shared alone where two +# classes of the last name `Shared` both define `go`. Through 136 that was an honest split; the arm below is kept, +# INVERTED. +# (c) A variable receiver (`c.scale`) is untouched by this round. A chained one (`Calc.new.scale`) was too, until +# parser version 132 typed a receiver the code builds (test/rubytypedrecvcheck.sh); its arm below is kept, +# INVERTED: it now asserts `Calc.new.scale` pins to Calc::scale alone. # (d) A constant receiver whose class defines BOTH `def self.x` and `def x` gets an honest two-way split that # includes the instance method (rails `Journey::Parser.parse`): a split, not a pin. Ruby's tags.scm gives # `method` and `singleton_method` one kind, so telling them apart is a later round. @@ -268,12 +274,20 @@ CPR="$( "$BIN" "$FIX" --no-cache --callers=Printer::format 2>/dev/null )" echo "$CPR" | grep -q 'count="0"' && ok "--callers=Printer::format reports count=\"0\"" \ || no "--callers=Printer::format did not report count=0: $( echo "$CPR" | grep -o ']*' )" -echo "=== a MISS never deletes an edge, and never invents one ===" +echo "=== a MISS invents no edge ===" E="$( rowOf 'n="external_call" ' )" echo "$E" | grep -q ' "$SEG/left.rb" +printf 'module Right\n class Shared\n def self.go\n 2\n end\n end\nend\n' > "$SEG/right.rb" +printf 'class Base\nend\n\nclass Caller < Base\n def run\n Left::Shared.go\n end\nend\n' > "$SEG/call.rb" +"$BIN" "$SEG" --no-cache --pin-census="$DIR/segment.tsv" >/dev/null 2>&1 +SEGT="$( awk -F'\t' '$1 == "C" && $7 == "go" { print $8 }' "$DIR/segment.tsv" )" +if [ "$( edgesTo "$SS" go )" -eq 1 ] && printf '%s' "$SEGT" | grep -qF "left.rb::Shared::go#" && ! printf '%s' "$SEGT" | grep -qF "right.rb::" +then + ok "Left::Shared.go → Left's Shared::go alone (one edge here; left.rb's def, not right.rb's, on the one-class-per-file fixture)" +else + no "Left::Shared.go is not Left's Shared::go alone ($( edgesTo "$SS" go ) go edges here; census on the per-file fixture: $SEGT)" +fi -echo "=== floor (c): variable and chained receivers are untouched by this round ===" +echo "=== floor (c): a variable receiver is untouched by this round; a chained Calc.new is typed (parser version 132) ===" IC="$( rowOf 'n="instance_call" ' )" [ "$( edgesTo "$IC" scale )" -eq 2 ] && ok "c.scale( 7 ) → unchanged 2-way split (a variable receiver has no type in Ruby; floor, stated)" \ || no "c.scale( 7 ) produced $( edgesTo "$IC" scale ) scale edges — this round must not move a variable receiver: $IC" CC="$( rowOf 'n="chain_call" ' )" -[ "$( edgesTo "$CC" scale )" -eq 2 ] && ok "Calc.new.scale( 8 ) → unchanged 2-way split (the receiver is a CALL, undecidable in one hop; floor, stated)" \ - || no "Calc.new.scale( 8 ) produced $( edgesTo "$CC" scale ) scale edges: $CC" +CCL="$( "$BIN" "$FIX" --no-cache --callers=Calc::scale 2>/dev/null )"; CCL_RC=$? +CTL="$( "$BIN" "$FIX" --no-cache --callers=Tally::scale 2>/dev/null )"; CTL_RC=$? +# The Tally half is an absence: a crash or refusal leaves CTL empty, which `! grep` alone reads as a PASS. +if [ "$CCL_RC" -ne 0 ] || [ "$CTL_RC" -ne 0 ] +then + no "--callers=Calc::scale / --callers=Tally::scale exited $CCL_RC / $CTL_RC — the absence half cannot be read off a failed run" +elif echo "$CCL" | grep -q 'n="chain_call"' && ! echo "$CTL" | grep -q 'n="chain_call"' +then + ok "Calc.new.scale( 8 ) → Calc::scale alone (floor (c) lifted: the receiver the code builds is a Calc — test/rubytypedrecvcheck.sh)" +else + no "Calc.new.scale( 8 ) does not pin to Calc::scale alone: $CC" +fi echo "$CC" | grep -q ' "$FIX/app/presenters/link_presenter.rb" <<'RUBY' +class LinkPresenter + def to + :presenter + end + + def not_to + :presenter + end + + def to_not + :presenter + end + + def with + :presenter + end + + def and_return + :presenter + end + + def twice + :presenter + end +end +RUBY +# a class's `def change` — every Rails migration has one — and an application module's are no methods of an example +# group: they shadow no builder +cat > "$FIX/db/migrate/001_add_name.rb" <<'RUBY' +class AddName < ActiveRecord::Migration[7.2] + def change + add_column :users, :name, :string + end +end +RUBY +cat > "$FIX/db/migration_mixin.rb" <<'RUBY' +module MigrationMixin + def change + :mixin + end +end +RUBY +cat > "$FIX/app/models/user.rb" <<'RUBY' +class User + def activate! + true + end +end +RUBY +cat > "$FIX/app/mailers/mailer.rb" <<'RUBY' +class Mailer + def address( email ) + email.to # @untyped + end +end +RUBY +cat > "$FIX/lib/core_ext/object.rb" <<'RUBY' +class Object + def to_widget + :widget + end +end +RUBY +cat > "$FIX/spec/support/helpers.rb" <<'RUBY' +module Helpers + def check_one( x ) + expect( x ).to eq( 1 ) # @support + end +end +RUBY +cat > "$FIX/spec/models/user_spec.rb" <<'RUBY' +RSpec.describe User do + let( :user ) { User.new } + + it "value" do + expect( user ).to eq( user ) # @expect_value + end + + it "block" do + expect { raise "x" }.to raise_error( RuntimeError ) # @expect_block + end + + it "negated" do + expect( user ).not_to be_nil # @not_to + expect( user ).to_not be_nil # @to_not + end + + it "mocks" do + allow( user ).to receive( :activate! ) # @allow + expect_any_instance_of( User ).to receive( :activate! ) # @any_expect + allow_any_instance_of( User ).to receive( :activate! ) # @any_allow + end + + it "local" do + target = expect( user ) + target.to eq( user ) # @local + end + + it { is_expected.to be_a( User ) } # @is_expected + + it "root" do + expect( user ).to_widget # @root + end + + context "nested" do + def nested_helper + expect( 1 ).to eq( 1 ) # @group_def + end + end + + shared_examples "shared" do + it { expect( 1 ).to eq( 1 ) } # @shared + end + + it "matcher chains" do + expect( user ).to receive( :activate! ).with( 1 ) # @receive_with + expect { 1 }.to change { 1 }.from( 1 ).to( 2 ) # @change_to + allow( user ).to receive( :activate! ).with( 1 ).and_return( 2 ) # @chain_link + expect( user ).to have_received( :activate! ).with( 1 ) # @have_received + allow( user ).to receive( :activate! ).with( 1 ).with( 2 ).with( 3 ).with( 4 ).with( 5 ).with( 6 ).with( 7 ).with( 8 ).twice # @chain_floor + end + + it "a receiver" do + helper_object.expect( 1 ).to eq( 1 ) # @recv_expect + end +end +RUBY + +# the shadow tree: it defines `expect` itself, so `expect( 1 )` is the tree's Wrapper, not RSpec's target +SHADOW="$DIR/shadow" +mkdir -p "$SHADOW/lib" "$SHADOW/spec/support" "$SHADOW/spec/models" +cat > "$SHADOW/lib/wrapper.rb" <<'RUBY' +class Wrapper + def initialize( v ) + @v = v + end + + def to( matcher ) + :wrapped + end +end +RUBY +cat > "$SHADOW/spec/support/shim.rb" <<'RUBY' +module Shim + def expect( v ) + Wrapper.new( v ) + end +end +RUBY +cat > "$SHADOW/spec/models/thing_spec.rb" <<'RUBY' +RSpec.describe "thing" do + it { expect( 1 ).to eq( 1 ) } # @shadow +end +RUBY + +MAP="$DIR/a.xml" +"$BIN" "$FIX" --no-cache >"$MAP" 2>"$DIR/map.err" +if [ $? -eq 0 ]; then ok "default map exits 0"; else no "default map exited non-zero: $( cat "$DIR/map.err" )"; fi +if [ -s "$DIR/map.err" ]; then no "unexpected stderr: $( head -3 "$DIR/map.err" )"; else ok "clean stderr"; fi +if command -v xmllint >/dev/null 2>&1 +then + if xmllint --noout "$MAP"; then ok "xmllint --noout"; else no "xmllint failed"; fi +fi + +# The census (src/pincensus.h): one `C` row per call site the resolver decided or refused, so each arm reads ONE line. +census(){ + if ! "$BIN" "$1" --no-cache --pin-census="$DIR/census.tsv" >/dev/null 2>"$DIR/census.err" + then + no "--pin-census on $1 exited non-zero: $( head -3 "$DIR/census.err" )" + return 2 + fi + return 0 +} +# line ROOT FILE MARK — the line number carrying `# @MARK` (empty when the marker is missing) +line(){ + grep -n "# @$3\$" "$1/$2" | head -1 | cut -d: -f1 +} +# targets FILE LINE — every target the census names for a call on FILE:LINE, one per line +targets(){ + awk -F'\t' -v f="$1" -v l="$2" '$1 == "C" && $9 == l && index( $6, f "::" ) == 1 { n = split( $8, t, "|" ); for( i = 1; i <= n; ++i ) print t[ i ] }' \ + "$DIR/census.tsv" +} +# rows FILE LINE CALLEE — "mechanismtargets" of each CALLEE call the census records on FILE:LINE +rows(){ + awk -F'\t' -v f="$1" -v l="$2" -v c="$3" '$1 == "C" && $9 == l && $7 == c && index( $6, f "::" ) == 1 { print $2 "\t" $8 }' "$DIR/census.tsv" +} +# reaches SYM FILE MARK WHY [ROOT] — a call on the marked line resolves to SYM +reaches(){ + local n; n="$( line "${5:-$FIX}" "$2" "$3" )" + [ -n "$n" ] || { no "fixture marker @$3 missing in $2"; return; } + targets "$2" "$n" | grep -qF "::$1#" && ok "@$3 → $1 ($4)" \ + || no "@$3 ($2:$n) does not reach $1 ($4); it reaches: $( targets "$2" "$n" | tr '\n' ' ' )" +} +# refused CALLEE FILE MARK WHY — every CALLEE call on the marked line is refused as external: no target, mechanism external +refused(){ + local n r; n="$( line "$FIX" "$2" "$3" )" + [ -n "$n" ] || { no "fixture marker @$3 missing in $2"; return; } + r="$( rows "$2" "$n" "$1" )" + if [ -n "$r" ] && ! printf '%s\n' "$r" | grep -qv "^external \$" + then + ok "@$3 .$1 refused as external ($4)" + else + no "@$3 ($2:$n) .$1 is not refused ($4); census: $( printf '%s' "$r" | tr '\t\n' ' ;' )" + fi +} + +census "$FIX" || exit 1 + +U=spec/models/user_spec.rb + +echo "=== a call on the target RSpec's builder returns is RSpec's ===" +refused to $U expect_value "expect( v ) is a ValueExpectationTarget — its to is RSpec's, not LinkPresenter#to" +refused to $U expect_block "expect { … } is a BlockExpectationTarget" +refused not_to $U not_to "not_to is the target's" +refused to_not $U to_not "to_not is the target's" +refused to $U allow "allow( v ) is an RSpec::Mocks::AllowanceTarget" +refused to $U any_expect "expect_any_instance_of( K ) is an AnyInstanceExpectationTarget" +refused to $U any_allow "allow_any_instance_of( K ) is an AnyInstanceAllowanceTarget" +refused to $U local "a local bound only to expect( v ) holds the target" +refused to $U is_expected "is_expected is expect( subject )" +refused to $U group_def "a def inside an example group runs in it" +refused to $U shared "a shared group's examples run in the including group" +reaches Object::to_widget $U root "a reopened Object answers on every instance, RSpec's targets included" + +echo "=== a matcher's chain is RSpec's: each link is called on the matcher its root builder returns ===" +refused with $U receive_with "receive( :m ) is an RSpec::Mocks::Matchers::Receive — its with is RSpec's" +refused to $U change_to "change { }.from( 1 ) is a Change matcher's chain — its to is RSpec's (a migration's def change, and its mixin's, shadow nothing)" +refused and_return $U chain_link "receive( … ).with( 1 ).and_return — two links from the builder" +refused with $U have_received "have_received( :m ).with" + +echo "=== floors: what is not RSpec's target, or not read as one ===" +reaches LinkPresenter::to app/mailers/mailer.rb untyped "an untyped receiver outside a spec binds by name as before" +reaches LinkPresenter::to spec/support/helpers.rb support "floor (a): a helper module outside a group binds by name as before" +reaches LinkPresenter::twice $U chain_floor "floor (c): a link more than 8 from its builder binds by name as before" +reaches LinkPresenter::to $U recv_expect "floor (d): helper.expect( 1 ) is not RSpec's builder — binds by name" + +census "$SHADOW" || exit 1 +reaches Wrapper::to spec/models/thing_spec.rb shadow "floor (b): the tree defines expect — expect( 1 ) is its Wrapper, bound by name" "$SHADOW" + +echo "=== determinism and warm == cold ===" +"$BIN" "$FIX" --no-cache >"$DIR/b.xml" 2>/dev/null +cmp -s "$MAP" "$DIR/b.xml" && ok "byte-identical across two --no-cache runs" \ + || no "output differs across runs" +if ! "$BIN" "$FIX" --cache="$DIR/c.bin" >"$DIR/cold.xml" 2>"$DIR/cold.err" +then + no "the cold cache run exited non-zero: $( head -3 "$DIR/cold.err" )" +fi +if ! "$BIN" "$FIX" --cache="$DIR/c.bin" >"$DIR/warm.xml" 2>"$DIR/warm.err" +then + no "the warm cache run exited non-zero: $( head -3 "$DIR/warm.err" )" +fi +cmp -s "$DIR/cold.xml" "$DIR/warm.xml" && ok "warm run == cold run" \ + || no "the warm cache disagrees with the cold run" + +echo "=== mutation: the tree opens RSpec's target class → it answers from the tree ===" +MUT="$DIR/mut"; cp -R "$FIX" "$MUT" +cat > "$MUT/spec/support/target_patch.rb" <<'RUBY' +module RSpec + module Expectations + class ValueExpectationTarget + def to_widget + :patched + end + end + end +end +RUBY +n="$( line "$MUT" "$U" root )" +m="$( line "$MUT" "$U" expect_value )" +if census "$MUT" +then + targets "$U" "$n" | grep -qF "::ValueExpectationTarget::to_widget#" && ok "mutation: the patched target's to_widget answers expect( v ).to_widget" \ + || no "mutation: with ValueExpectationTarget opened, expect( v ).to_widget does not reach it; it reaches: $( targets "$U" "$n" | tr '\n' ' ' )" + targets "$U" "$m" | grep -qF "::LinkPresenter::to#" && no "mutation: expect( v ).to reaches LinkPresenter#to with the target opened" \ + || ok "mutation: expect( v ).to still reaches no namesake — the opened class defines no to" +fi + +echo +[ "$fail" -eq 0 ] && echo "ALL PASS" || { echo "SOME CHECKS FAILED"; exit 1; } diff --git a/test/rubytypedrecvcheck.sh b/test/rubytypedrecvcheck.sh new file mode 100755 index 000000000..8cabc7875 --- /dev/null +++ b/test/rubytypedrecvcheck.sh @@ -0,0 +1,538 @@ +#!/usr/bin/env bash +# rubytypedrecvcheck.sh — gate for a Ruby call whose RECEIVER has a type the code states. `c = Client.new( url )` then +# `c.get`, `u = User.find( id )` then `u.display_name`, `Client.new.close`, and in a spec `let( :user ) { create( :user ) }` +# then `user.activate!`: the receiver is an instance of one class, and Ruby's method lookup on that class decides which +# `def` answers. Before this round every such call bound by NAME alone — same file, same directory, the one definer in +# the tree, else no edge — so `c.get` declined between Client#get and Other#get, `u.email` (a column Rails generates) +# bound to an unrelated Post#email, and in a spec that also builds a Post, every `user.…` call bound to Post's method +# through the include narrow (the file names Post as a receiver; it names User only as a describe argument). +# THE RULE (ingest_binds.h rubyTypeReceivers; graph.h rubyTypedReceiver): a receiver is TYPED when the code builds it in +# one of these shapes, read lexically by Ruby's own local rule — +# * a local whose every assignment in its def (or block, or file scope) is one of the shapes below and names one class; +# * an RSpec `let`/`let!`/`subject` whose block's value is one — the innermost visible definition, as RSpec runs it — +# and a group's implicit `subject`, which is `described_class.new`; +# * the receiver written directly: `Foo.new( … ).m`, `Foo.find_by( … ).m`, `create( :user ).m`. +# The shapes: `Const.new`, an Active Record finder on a constant (`find`, `find_by`, `create`, `first`, … — kRubyTypedFinders), +# `described_class.new`, and a FactoryBot build (`create`/`build`/`build_stubbed( :factory, … )` inside an example group), +# whose class is the factory's (`class:` as written, a nested factory's parent's, else the name camelised) read from the +# tree's `FactoryBot.define` blocks. A typed call answers from the class's lookup — the class, its mixins, its superclass +# chain, then a reopened Object/Kernel/BasicObject — and, when none of those defines it, from a class BELOW it (Active +# Record's STI hands back a subclass); with no definer in reach the method is outside the tree (a column, an association, +# a framework method) and the call is refused as external. +# +# Stated floors, each pinned below: +# (a) a type the tree does not define (`Net::HTTP.new`, `SimpleDelegator.new`, a `Point = Struct.new( … )` constant) +# changes nothing: the call binds by name as before — and neither does a QUALIFIED constant whose path the tree +# never opens (`Vendor::Client.new` beside an in-tree `Client`: classes are keyed by final segment everywhere else). +# (b) a class that defines its own `self.find` / `self.new` (or one of its ancestors does) may return anything: a local +# built that way is untyped. +# (c) a local assigned any other shape anywhere in its scope, or bound as a parameter or block parameter there, is +# untyped everywhere in that scope — the rule reads no control flow. +# (d) instance variables, method return values, `x.class`, and a `let` defined in a shared context in another file are +# not typed: those calls bind by name as before. +# (e) a class whose lookup can leave it (a delegator, `method_missing`, the delegation DSL — rubyreachcheck floor (a)/(b)) +# refuses nothing: its typed calls bind by name as before. +# (f) two factories sharing a name in different files, with different classes, type nothing. +# (g) a method defined only through `alias`/`alias_method` is no def the tree indexes, so a typed call to it finds no +# definer in reach and is refused — not bound to another class's namesake. +# +# Usage: test/rubytypedrecvcheck.sh | RIPWIRE_BIN=asan/ripwire test/rubytypedrecvcheck.sh +# Exits non-zero on any failure. Self-contained via mktemp. +set -u +ROOT="$( cd "$( dirname "$0" )/.." && pwd )" +BIN="${1:-${RIPWIRE_BIN:-$ROOT/build/ripwire}}" +[ "${BIN#/}" = "$BIN" ] && BIN="$ROOT/$BIN" +fail=0 +ok(){ echo " PASS $1" || { fail=1; echo " FAIL could not write the PASS line for: $1"; }; return 0; } +no(){ echo " FAIL $1"; fail=1; } + +[ -x "$BIN" ] || { echo "no ripwire binary at $BIN — build first (cmake --build build -j)"; exit 2; } + +DIR="$( mktemp -d )"; trap 'rm -rf "$DIR"' EXIT +FIX="$DIR/fix" +for d in app/models/concerns app/services lib/net lib/core_ext lib/misc spec/models spec/factories spec/support +do + mkdir -p "$FIX/$d" +done + +cat > "$FIX/app/models/application_record.rb" <<'RUBY' +class ApplicationRecord < ActiveRecord::Base + self.abstract_class = true + + def stamp_audit + :base + end +end +RUBY +cat > "$FIX/app/models/user.rb" <<'RUBY' +class User < ApplicationRecord + include Notifiable + + def display_name + "user" + end + + def activate! + true + end + + def title_line + display_name + end +end +RUBY +cat > "$FIX/app/models/super_user.rb" <<'RUBY' +class SuperUser < User + def power + 9 + end +end +RUBY +cat > "$FIX/app/models/post.rb" <<'RUBY' +class Post < ApplicationRecord + def display_name + "post" + end + + def activate! + false + end + + def email + "decoy" + end + + def notify! + :post + end + + def stamp_audit + :post + end +end +RUBY +cat > "$FIX/app/models/admin.rb" <<'RUBY' +class Admin + def display_name + "admin" + end +end +RUBY +cat > "$FIX/app/models/concerns/notifiable.rb" <<'RUBY' +module Notifiable + extend ActiveSupport::Concern + + def notify! + :notified + end +end +RUBY +cat > "$FIX/app/models/widget.rb" <<'RUBY' +class Widget + def self.find(id) + Gadget.new + end + + def spin + :widget + end +end +RUBY +cat > "$FIX/app/models/gadget.rb" <<'RUBY' +class Gadget + def spin + :gadget + end +end +RUBY +cat > "$FIX/app/models/user_presenter.rb" <<'RUBY' +class UserPresenter < SimpleDelegator +end +RUBY +cat > "$FIX/lib/net/client.rb" <<'RUBY' +class Client + def initialize(url = nil) + @url = url + end + + def get(path) + path + end + + def close + nil + end + + def wrap + Other.new + end + + alias_method :fetch, :get +end +RUBY +cat > "$FIX/lib/net/other.rb" <<'RUBY' +class Other + def get(path) + path + end + + def fetch(path) + path + end + + def close + nil + end +end +RUBY +cat > "$FIX/lib/net/pool.rb" <<'RUBY' +module Transport + class Pool + def drain + [] + end + end +end +RUBY +cat > "$FIX/lib/core_ext/object.rb" <<'RUBY' +class Object + def try_it + self + end +end +RUBY +cat > "$FIX/lib/misc/thing.rb" <<'RUBY' +class Thing + def try_it + :thing + end + + def power + 0 + end + + def drain + :thing + end +end +RUBY +cat > "$FIX/lib/misc/rating.rb" <<'RUBY' +class Rating + def request(req) + req + end +end +RUBY +cat > "$FIX/app/services/sync.rb" <<'RUBY' +class Sync + def run(cond, req) + c = Client.new("u") + c.get("/x") # @local_new + u = User.find(1) + u.display_name # @local_finder + u.email # @local_column + u.notify! # @local_mixin + u.stamp_audit # @local_super + u.power # @local_subclass + Client.new.close # @direct_new + User.find_by(id: 1).activate! # @direct_finder + c.try_it # @implicit_root + c.fetch("/a") # @alias_floor + r = Client.new + r = r.wrap + r.close # @reassigned + k = Other.new if cond + k = Client.new unless cond + k.get("/z") # @conflict + w = Widget.find(1) + w.spin # @own_finder + h = Net::HTTP.new("host") + h.request(req) # @out_of_tree + pr = UserPresenter.new(u) + pr.title_line # @delegator + pl = Transport::Pool.new + th = Thing.new + pl.drain # @qualified_in + ext = Vendor::Client.new + ext.get("/v") # @qualified_out + end + + def shadowed + c = Client.new + [1].each { |c| c.get("/q") } # @block_param + end + + def ivar_floor + @client = Client.new + @client.get("/i") # @ivar + end + + def client + Client.new + end + + def return_floor + client.get("/r") # @return + end +end +RUBY +cat > "$FIX/spec/factories/users.rb" <<'RUBY' +FactoryBot.define do + factory :user do + name { "x" } + + factory :admin do + admin { true } + end + end + + factory :editor, class: "User" + factory :twin, class: "User" +end +RUBY +cat > "$FIX/spec/factories/posts.rb" <<'RUBY' +FactoryBot.define do + factory :article, class: Post do + title { "t" } + end + + factory :twin, class: "Admin" +end +RUBY +cat > "$FIX/spec/support/shared_user.rb" <<'RUBY' +RSpec.shared_context "with a shared user" do + let(:shared_user) { create(:user) } +end +RUBY +cat > "$FIX/spec/models/user_spec.rb" <<'RUBY' +require "rails_helper" + +RSpec.describe User do + let(:user) { create(:user) } + let(:admin) { create(:admin) } + let(:editor) { build(:editor) } + let(:article) { create(:article) } + let(:plain) { Post.new } + let(:found) { User.find_by(id: 1) } + subject(:svc) { described_class.new } + let(:mystery) { make_thing } + let(:twin) { create(:twin) } + + it "types each let by what its block builds" do + user.display_name # @let_factory + admin.display_name # @let_nested_factory + editor.display_name # @let_class_string + article.display_name # @let_class_const + plain.display_name # @let_new + found.display_name # @let_finder + svc.display_name # @let_described + user.email # @let_column + mystery.title_line # @let_untyped + twin.email # @let_twin_factory + u2 = create(:user) + u2.display_name # @spec_local + end + + context "a nested group overrides the let" do + let(:user) { create(:article) } + + it { user.display_name } # @let_override + end + + it "a local shadows the let" do + user = Post.new + user.display_name # @local_shadows_let + end +end +RUBY +cat > "$FIX/spec/models/post_spec.rb" <<'RUBY' +require "rails_helper" + +RSpec.describe Post do + include_context "with a shared user" + + it { subject.display_name } # @implicit_subject + it { shared_user.title_line } # @shared_let +end +RUBY + +MAP="$DIR/map.xml" +"$BIN" "$FIX" --no-cache >"$MAP" 2>"$DIR/map.err" +if [ $? -eq 0 ]; then ok "default map exits 0"; else no "default map exited non-zero: $( cat "$DIR/map.err" )"; fi +if [ -s "$DIR/map.err" ]; then no "unexpected stderr: $( head -3 "$DIR/map.err" )"; else ok "clean stderr"; fi +if command -v xmllint >/dev/null 2>&1 +then + if xmllint --noout "$MAP"; then ok "xmllint --noout"; else no "xmllint failed"; fi +fi + +# The census (src/pincensus.h): one `C` row per call site the resolver decided or refused — mechanism, caller, callee, +# targets, line — so each arm reads ONE call site. (`--uses` cannot: it lists a caller's every same-named call under each +# target that caller reaches, and a spec's example blocks are all one caller, the file scope.) +census(){ + if ! "$BIN" "$1" --no-cache --pin-census="$DIR/census.tsv" >/dev/null 2>"$DIR/census.err" + then + no "--pin-census on $1 exited non-zero: $( head -3 "$DIR/census.err" )" + return 2 + fi + return 0 +} +# line ROOT FILE MARK — the line number carrying `# @MARK` (empty when the marker is missing) +line(){ + grep -n "# @$3\$" "$1/$2" | head -1 | cut -d: -f1 +} +# targets FILE LINE — every target the census names for the call on FILE:LINE, one per line +targets(){ + awk -F'\t' -v f="$1" -v l="$2" '$1 == "C" && $9 == l && index( $6, f "::" ) == 1 { n = split( $8, t, "|" ); for( i = 1; i <= n; ++i ) print t[ i ] }' \ + "$DIR/census.tsv" +} +# reaches SYM FILE MARK WHY — the call on the marked line resolves to SYM +reaches(){ + local n; n="$( line "$FIX" "$2" "$3" )" + [ -n "$n" ] || { no "fixture marker @$3 missing in $2"; return; } + targets "$2" "$n" | grep -qF "::$1#" && ok "@$3 → $1 ($4)" \ + || no "@$3 ($2:$n) does not reach $1 ($4); it reaches: $( targets "$2" "$n" | tr '\n' ' ' )" +} +# misses SYM FILE MARK WHY — the call on the marked line does NOT resolve to SYM +misses(){ + local n; n="$( line "$FIX" "$2" "$3" )" + [ -n "$n" ] || { no "fixture marker @$3 missing in $2"; return; } + targets "$2" "$n" | grep -qF "::$1#" && no "@$3 ($2:$n) reaches $1 — $4" || ok "@$3 does not reach $1 ($4)" +} + +census "$FIX" || exit 1 + +S=app/services/sync.rb +U=spec/models/user_spec.rb +P=spec/models/post_spec.rb + +echo "=== a local built by Const.new or a finder answers from its class ===" +reaches Client::get $S local_new "c = Client.new — Client#get, where the name alone declined" +misses Other::get $S local_new "Other is not c's class" +reaches User::display_name $S local_finder "u = User.find( 1 ) — an Active Record finder returns a User" +misses Post::display_name $S local_finder "Post is not u's class" +misses Post::email $S local_column "no class in User's lookup defines email: a column, outside the tree — refused" +reaches Notifiable::notify! $S local_mixin "User includes Notifiable: the mixin answers" +misses Post::notify! $S local_mixin "Post's namesake is not in User's lookup" +reaches ApplicationRecord::stamp_audit $S local_super "User < ApplicationRecord: the superclass answers" +misses Post::stamp_audit $S local_super "a sibling's override is not User's" +reaches SuperUser::power $S local_subclass "no ancestor defines power: a subclass may (STI hands one back)" +misses Thing::power $S local_subclass "Thing is neither above nor below User" +reaches Object::try_it $S implicit_root "a reopened Object is every instance's ancestor" +misses Thing::try_it $S implicit_root "Thing is not in Client's lookup" +reaches Pool::drain $S qualified_in "pl = Transport::Pool.new — the tree opens Transport::Pool" +misses Thing::drain $S qualified_in "Thing is not pl's class" + +echo "=== a receiver written as the construction answers the same way ===" +reaches Client::close $S direct_new "Client.new.close" +misses Other::close $S direct_new "Other is not the receiver's class" +reaches User::activate! $S direct_finder "User.find_by( … ).activate!" +misses Post::activate! $S direct_finder "Post is not the receiver's class" + +echo "=== an RSpec let answers from what its block builds ===" +reaches User::display_name $U let_factory "let( :user ) { create( :user ) } — factory :user builds a User" +misses Post::display_name $U let_factory "the include narrow's Post is not user's class" +reaches User::display_name $U let_nested_factory "factory :admin nests in factory :user: it builds a User" +misses Admin::display_name $U let_nested_factory "a nested factory's class is its parent's, not its camelised name" +reaches User::display_name $U let_class_string "factory :editor, class: \"User\"" +reaches Post::display_name $U let_class_const "factory :article, class: Post" +misses User::display_name $U let_class_const "article is a Post" +reaches Post::display_name $U let_new "let( :plain ) { Post.new }" +reaches User::display_name $U let_finder "let( :found ) { User.find_by( … ) }" +reaches User::display_name $U let_described "subject( :svc ) { described_class.new } inside describe User" +misses Post::email $U let_column "a User column is outside the tree — refused, not Post#email" +reaches User::display_name $U spec_local "u2 = create( :user ) inside an example" +reaches Post::display_name $U let_override "the nested group's let( :user ) { create( :article ) } is the one RSpec runs" +misses User::display_name $U let_override "the outer let is overridden" +reaches Post::display_name $U local_shadows_let "a local named user hides the let" +misses User::display_name $U local_shadows_let "the let is not what user reads there" +reaches Post::display_name $P implicit_subject "a group's implicit subject is described_class.new" + +echo "=== what the rule leaves alone ===" +misses Client::close $S reassigned "floor (c): r = r.wrap reassigns r — untyped" +misses Client::get $S conflict "floor (c): k is assigned an Other and a Client — untyped" +misses Other::get $S conflict "floor (c): neither class is k's" +misses Widget::spin $S own_finder "floor (b): Widget defines self.find — w is untyped" +reaches Rating::request $S out_of_tree "floor (a): Net::HTTP is outside the tree — the call binds by name as before" +misses Client::get $S qualified_out "floor (a): the tree opens no Vendor::Client — its final segment is no in-tree Client" +misses Other::fetch $S alias_floor "floor (g): Client's fetch is an alias_method, which the tree indexes as no def — refused, not Other#fetch" +reaches User::title_line $S delegator "floor (e): UserPresenter < SimpleDelegator forwards what it lacks" +misses Client::get $S block_param "floor (c): a block parameter c makes c untyped in its scope" +misses Client::get $S ivar "floor (d): an instance variable is not typed" +misses Client::get $S return "floor (d): a method's return value is not typed" +reaches User::title_line $U let_untyped "a let whose block builds nothing typed binds by name as before" +reaches Post::email $U let_twin_factory "floor (f): factory :twin names User in one file and Admin in another — untyped, binds by name" +reaches User::title_line $P shared_let "floor (d): a shared context's let is not seen in the including file — binds by name as before" + +echo "=== a qualified constant in a tree with no superclass, mixin or constant reference ===" +# The base scope and the resolver's constant index are built only for a tree that has those; the opens a qualified type +# is checked against must not depend on them. +FLAT="$DIR/flat"; mkdir -p "$FLAT/lib" +cat > "$FLAT/lib/pool.rb" <<'RUBY' +module Transport + class Pool + def drain + :pool + end + end +end +RUBY +cat > "$FLAT/lib/sink.rb" <<'RUBY' +class Sink + def drain + :sink + end + + def flush + :sink + end +end +RUBY +cat > "$FLAT/lib/run.rb" <<'RUBY' +class Runner + def go + pool = Transport::Pool.find( 1 ) + pool.drain # @flat_finder + pool.flush # @flat_refused + end +end +RUBY +n="$( line "$FLAT" lib/run.rb flat_finder )" +m="$( line "$FLAT" lib/run.rb flat_refused )" +if census "$FLAT" +then + targets lib/run.rb "$n" | grep -qF "::Pool::drain#" && ok "@flat_finder → Pool::drain (Transport::Pool is opened, though nothing else in the tree is scoped)" \ + || no "@flat_finder (lib/run.rb:$n) does not reach Pool::drain; it reaches: $( targets lib/run.rb "$n" | tr '\n' ' ' )" + targets lib/run.rb "$n" | grep -qF "::Sink::drain#" && no "@flat_finder reaches Sink::drain — Sink is not pool's class" \ + || ok "@flat_finder does not reach Sink::drain" + targets lib/run.rb "$m" | grep -qF "::Sink::flush#" && no "@flat_refused reaches Sink::flush — nothing in Pool's lookup defines flush" \ + || ok "@flat_refused does not reach Sink::flush (refused: no definer in Pool's lookup)" +fi + +echo "=== determinism and warm == cold ===" +"$BIN" "$FIX" --no-cache >"$DIR/b.xml" 2>/dev/null +cmp -s "$MAP" "$DIR/b.xml" && ok "byte-identical across two --no-cache runs" \ + || no "output differs across runs" +if ! "$BIN" "$FIX" --cache="$DIR/c.bin" >"$DIR/cold.xml" 2>"$DIR/cold.err" +then + no "the cold cache run exited non-zero: $( head -3 "$DIR/cold.err" )" +fi +if ! "$BIN" "$FIX" --cache="$DIR/c.bin" >"$DIR/warm.xml" 2>"$DIR/warm.err" +then + no "the warm cache run exited non-zero: $( head -3 "$DIR/warm.err" )" +fi +cmp -s "$DIR/cold.xml" "$DIR/warm.xml" && ok "warm run == cold run" \ + || no "the warm cache disagrees with the cold run" + +echo "=== mutation: the factory's class moves → the let follows it ===" +MUT="$DIR/mut"; cp -R "$FIX" "$MUT" +sed 's/factory :article, class: Post do/factory :article, class: "User" do/' "$FIX/spec/factories/posts.rb" >"$MUT/spec/factories/posts.rb" +grep -q 'class: "User" do' "$MUT/spec/factories/posts.rb" || no "mutation did not apply" +n="$( line "$MUT" "$U" let_class_const )" +if census "$MUT" +then + targets "$U" "$n" | grep -qF "::User::display_name#" && ok "mutation: factory :article, class: \"User\" — article.display_name now reaches User" \ + || no "mutation: with the factory's class changed, article.display_name does not reach User" +fi + +echo +[ "$fail" -eq 0 ] && echo "ALL PASS" || { echo "SOME CHECKS FAILED"; exit 1; }