feat(ingest): extract JS/TS arrow and function-expression definitions - #142
Merged
jmagly merged 3 commits intoAug 1, 2026
Merged
Conversation
…re not extracted Pins issue elder-plinius#141 in both directions: every named-value function form must be extracted with its name, parameters and span, and the shapes carrying no name-to-function binding (IIFE, destructured, HOC-wrapped) must stay unmatched. Also covers the end-to-end consequence — an arrow-bound sink ranking attack_surface, and an arrow-bound handler becoming an entry point. Fails on main: the .js/.ts/.tsx def-queries bind declarations only. Refs elder-plinius#141
The .js/.ts/.tsx def-queries bound declarations only, so every definition bound
to a value produced no CodeBlock and its sinks never reached classification or
ranking. Adds the name-to-function-value shapes to all three queries:
const f = (a) => {} / function (a) {} variable_declarator
const f = a => ... un-parenthesized single param
obj.f = ... / module.exports.handler assignment_expression
onmessage = e => ... bare-identifier assignment
{ f: (a) => {} } pair
class K { f = (a) => {} } field_definition / public_field_definition
Query-only — no parser, pipeline or CodeBlock-shape change. nodeToCodeBlock
already handles any @def/@name/@params triple and kindOf maps these to
'function'. Property-assigned and object-literal forms are named by the
property, consistent with how method_definition is named: that is the name
buildCallGraph and the entry-point heuristic match on.
Not matched, by design: HOC-wrapped values (const h = withAuth(fn)) and
anonymous callbacks (app.get('/x', (req,res) => ...)) have no name-to-function
binding, and inventing one is worse than honest absence. whitebox.ts caveat
block updated to match.
Refs elder-plinius#141
…k ids distinct
Two defects found by adversarial review of the arrow-extraction change:
- `class C { #handler = (x) => eval(x) }` yielded nothing: a private field's
name is a `private_property_identifier`, not the `property_identifier` the
class-field pattern required. A sink in a private handler is exactly as
dangerous as one in a public handler, so it is captured (name keeps the `#`).
- A block id is `path::name@line`, so two same-named definitions starting on the
same line collide, and buildCallGraph keys on that id — the two blocks silently
share one callers/callees entry even when only one carries a sink. Rare for
declarations, routine once value-bound functions are extracted (minified
bundles pack many `{ handler: … }` onto one line). The later block is now
disambiguated by column; the first keeps the plain id, so no existing id
changes and the Python path is untouched.
Refs elder-plinius#141
jmagly
approved these changes
Aug 1, 2026
jmagly
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed at 7f34c47b97b0605dfb7dad5edaff646659eccee2.
No blocking findings. The query patterns cover the named-value forms without broadening into anonymous or wrapped callbacks, the same-line ID disambiguation preserves existing first-occurrence IDs, and the regression suite exercises extraction, exclusions, security ranking, entry-point elevation, and reachability.
Verification on a synthetic merge into current main (a606adf):
npm run typecheck- focused regression suite: 33/33
- full suite: 740/740 Vitest, 11/11 ops preflight, 19/19 model matrix
- hosted CI: pass
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #141.
White-box ingest extracted JS/TS declarations only. Every definition bound to a value — the dominant shape in modern Node/TS — produced no
CodeBlock, so its body never reached sink classification, priority scoring, entry-point detection or the context pack. The file parsed fine and simply yielded nothing, so the hole was silent.This is the follow-up to #75, where the limit was documented in-code when the multi-language ingest landed.
The change
Query-only, in the
.js/.ts/.tsxdef-queries (src/recon/ts-grammars.ts). No parser, pipeline orCodeBlock-shape change:nodeToCodeBlockalready handles any captured@def/@name/@paramstriple andkindOfmaps these tofunction.const f = (a) => {}/= function (a) {}variable_declaratorconst f = a => …(un-parenthesized single param)variable_declaratorobj.f = …,module.exports.handler = …assignment_expression(member)onmessage = e => …(worker/global handler)assignment_expression(identifier){ f: (a) => {} }pairclass K { f = (a) => {} },#f = …field_definition/public_field_definitionProperty-assigned and object-literal forms are named by the property, consistent with how
method_definitionis named — that is the namebuildCallGraphand the entry-point heuristic match on. All three dialects share one pattern set; only the class-field node name differs (field_definition property:vspublic_field_definition name:).Deliberately not matched, because there is no name-to-function binding and inventing one is worse than honest absence:
const h = withAuth(async (req) => …)(the value is acall_expression)app.get('/x', (req, res) => …)The
whitebox.tscaveat block is updated in the same commit to say exactly this.Also fixed here (found by adversarial review of the change)
Same-line block-id collision. A block id is
path::name@line, so two same-named definitions starting on the same line collide — andbuildCallGraphkeys on that id, so the two silently share onecallers/calleesentry even when only one carries a sink. Rare for declarations; routine once value-bound functions are extracted, since minified/bundled JS packs many{ handler: … }onto one line. The later block is now disambiguated by column. The first keeps the plain id, so no existing id changes and the Python path is untouched.Verification
RED first — the test commit fails on
main(26 failures across all three dialects), passes after the query change.npm test— 708 passed, 65 files (was 675 before this branch; +33 new). Green.npm run test:coverage— 100% statements/branches/functions/lines on the three coverage-gated recon files, including the new branch ints-parse.ts.npm run typecheck— clean.eslinton every changed file — 0 errors.npm run doctor— PASS (warnings are pre-existing optional-tooling/offline-API items).The new suite pins both directions: each form extracted with the right name, params and span, and the excluded shapes (
const x = 42, IIFE, destructured bindings, HOC-wrapped) asserted absent. The full-set equality assertion catches a missed form and an over-matching pattern in one check. An end-to-end test runs the realingestRepositoryand asserts an arrow-bound sink ranksattack_surface, an arrow-boundhandlerbecomes an entry point, and the call-graph edge between them resolves.Scale check against this repo's own 1.24 MB / 21k-line SPA script (
docs/index.html), old query vs new, same parse tree:+119 real definitions recovered, no measurable match-time cost, and zero matches with an unbound
@name/@defcapture across that corpus — an unbound capture would throw and fail-open the whole file to[], so this was checked explicitly rather than assumed.Verified on macOS (Apple Silicon) only — that is the only platform I can test on. The change is a tree-sitter query string, so it carries no platform-specific behaviour, but I am not claiming Linux/Windows verification I did not do.
Commits
f8988e0test(ingest):RED baseline — fails onmainc8b0e20feat(ingest):the query change + caveat-block update7f34c47fix(ingest):#privatefields + same-line id disambiguation (review findings)