[rahulr85r]

TokenHack had ~20 hand-tuned scoring constants and no way to tell whether any of them helped. This adds the measurement, then acts on what it found.

The harness

tests/ — 72 questions across 6 OSS repos, each paired with the file that answers it. Built blind: questions and answers were derived by reading the repos only, with the router and its index explicitly off-limits, so the set isn't shaped around what this tool is good at. 45 of 72 are deliberately vocabulary-mismatch questions, where the question's words don't appear in the answer's name.

python3 tests/fetch_repos.py   # clones + indexes the pinned public repos
python3 tests/eval.py

Public repos are pinned to the commit their gold set was written against, so the numbers are reproducible. Every scoring constant now reads a TOKENHACK_<NAME> env override, so a change can be ablated instead of argued about.

First run: hit@5 = 0.19

Four questions in five got nothing useful into the top five. What that found:

1. graph_prop was 94% of the winning score

winner:  testsuite/.../SocketReadPendingTest.java        score 354.76
         bm25 14.72 | filename 2.0 | path 2.0 | def 4.0 | graph_prop 333.01
gold:    transport/.../AdaptiveRecvByteBufAllocator.java score 8.16  (rank 1,716)

Every real lexical signal summed to 22 and got shouted down 15-to-1. The filename gate meant to bound this is trivially cleared by a natural-language question — enough words that any well-connected file matches two. Propagation is evidence a neighbour matched, which is weaker than matching yourself: now always log-compressed and capped at the file's own query-derived score.

2. The highest-weighted channel in the formula was empty

ETA is the largest coefficient in the scorer, and only the Python adapter ever populated the prose channel:

netty               0 / 3,512 files with prose
spring-framework    0 / 9,487 files with prose
Signal-Android      2 / 5,534 files with prose

Java, Kotlin, Swift and JS returned nothing — and an empty channel doesn't error, it just contributes zero forever. On repos that are famously javadoc'd. Doc-comment extraction now ships for all adapters (80% coverage on Spring, 60% on netty), and with the channel actually populated ETA=1.2 was too low: 0.181 at ETA=0, 0.208 at 1.2, 0.264 at 2.4, flat above. Default raised to 2.4.

3. Tests outranked the code they test

Tests restate a concept's vocabulary more densely than implementations do, which is exactly what BM25 rewards. netty's pipeline question returned two testsuite files and a microbenchmark with DefaultChannelPipeline.java absent — and when the pairing did surface it, it printed the implementation as (test for ...) its own test. Test paths are now demoted unless the question is about tests.

4. Structural matches ignored IDF

filename_match and path_affinity counted 1.0 per token regardless of the token, so read + socket scored the same as idempotency + backoff. Now IDF-weighted (worth ~1.4 pts of hit@5 alone), and sub-threshold query tokens are dropped as corpus filler.

Result

	Before	After
hit@5	0.19	0.26
recall@5	0.16	0.19
MRR	0.13	0.17
hit@5, vocabulary mismatch	0.13	0.22

Baseline committed so --check catches regressions.

These are in-sample numbers on six repos, and most questions still miss. That's now a fact with a number attached rather than an unknown.

Also in this PR

• TypeScript/TSX adapter — the JS walk moved to walk_source(); typescript.py reuses it and adds interface/type-alias/enum declarations, dispatching .tsx to the JSX grammar. Biggest adoption blocker removed.
• Coverage warning — the router now says so when >25% of a repo's source files had no adapter. The indexer always recorded this and nothing read it, so an unsupported-language repo got confident rankings over a fragment of itself.
• Light plural folding on structural matches (bytes → ByteBuf, credentials → Credential).
• CI job (workflow_dispatch + on PRs touching the ranker) that clones the pinned repos and fails on regression.