[vdavamse]

Summary

Adds the SPRIG paper (arXiv:2602.23372v1, Democratizing GraphRAG: Linear, CPU-Only Graph Retrieval for Multi-Hop QA) to docs/ with an analysis summary in the existing convention, plus three research notes on the RAG backend.

Documentation and research only — no source changes.

Why SPRIG

It is the closest published work to CTRA's RAG-backend decision: a CPU-only, linear-time, token-free GraphRAG pipeline (lightweight NER -> entity co-occurrence graph -> Personalized PageRank), benchmarked against BM25, dense retrieval, RRF and cross-encoder reranking under a strict 4 GB memory budget.

Three findings bear directly on CTRA:

• Query-entity-seeded PPR alone scores R@10 0.464 / 0.357 - below plain BM25 (0.742 / 0.643). This is the closest published analogue to LinearRAG's retrieval mechanism; it suggests the graph is a re-scorer, not a retriever.
• GraphRRF - PPR seeded from a fused BM25+dense candidate list - is the best R@10 on both datasets (0.867 / 0.794). Meanwhile score-side fusion (RRF+PPR) is worse than not fusing at all. The gain comes from richer seeds feeding traversal.
• The cross-encoder buys MRR, not recall, and is the most expensive method overall.

Hardware measurements

research/rag-alternatives-hybrid-retrieval.md sections 1 and 3 replace the plan's extrapolations with numbers measured on the target machine, using 3,000 real Phase 2/3 trials pulled from the ClinicalTrials.gov v2 API and converted with the repo's own _ctg_to_passages:

Stage	Measured	Extrapolated to CTG (2.22M chunks)
MiniLM-L6 embed, GPU fp16	71.2 chunk/s	8.7 h
GLiNER-BioMed-large, GPU fp16	1.16 chunk/s	532 h (22 days)

GLiNER runs at 2.30 doc/s here against the 35 doc/s on T4 recorded in docs/ner-evaluation-guide.md - 15.2x slower, because the GTX 1650 Ti (Turing TU117) has no Tensor Cores. Two secondary findings change component choices: ONNX int8 runs slower than fp32 on this CPU (i7-10750H, AVX2 only, no AVX512-VNNI), and the MiniLM-L6 cross-encoder is unusable at 2.1 s/query, so the reranker must be TinyBERT-L2 (222 ms).

Benchmark scripts and raw JSON live in notes/bench_rag_hardware/ with a repro README. notes/ is gitignored, so they are not in this PR - say the word if they should be tracked alongside the docs that cite them.

Research notes

File	Contents
rag-alternatives-hybrid-retrieval.md	The hybrid (BM25 + dense + reranker) case. Also documents a query-time CPU wall in LinearRAG's calculate_passage_scores - an O(passages x activated_entities) Python str.count() scan - and an in-RAM embedding-store design that does not fit the corpus regardless of GPU. Includes a provenance audit confirming the vendored code is upstream LinearRAG, essentially unmodified.
rag-synthesis-sprig-schemagraph.md	How SPRIG and the schemagraph implementation would combine if the backend were rebuilt. Notes that _primekg_to_passages currently flattens 4M curated typed relations into prose, then spends GPU-days on NER to re-infer them from co-occurrence.
linearrag-improvements.md	Twelve in-place improvements to the existing LinearRAG implementation, keeping GLiNER - the option that preserves the current architecture. Includes a correctness fix: used_sentence_hash_ids is consumed in dict-iteration order, making retrieval non-deterministic.

Open items (not in this PR)

• The retrieval benchmark harness remains the blocking prerequisite. src/ctra/rag/eval/ is NER-only; the TREC-CT 2021 results quoted in docs/ner-evaluation-guide.md exist as prose, not code (grep -ri trec finds nothing under src/, scripts/ or tests/). Every quality claim in these notes is undecidable without it.
• A GLiNER bi-encoder benchmark is running separately. The GLiNER-BioMed paper reports the bi-encoder at +39-63% throughput with comparable or better F1 at small/base scale; docs/ner-evaluation-guide.md:143 lists this as supported but it appears never to have been run. Results will be posted here when available.

Caveats

• SPRIG's evaluation is Wikipedia multi-hop QA (HotpotQA, 2Wiki), not biomedical retrieval, and its graph uses generic en_core_web_sm NER. The numbers justify ordering the work, not settling it for CTRA.
• SPRIG's PPR uses max_iter=5 at 0.85 damping, so ~44% of initial error survives; its graph-only results are probably a pessimistic bound.
• No code link exists for SPRIG - the paper promises a release in future tense.

🤖 Generated with Claude Code

[vdavamse]

GLiNER variant sweep — results

The benchmark referenced under "Open items" has finished. It inverted the prediction in linearrag-improvements.md item 10, which is now corrected in place (1a6af6b).

120 real CTG chunks (mean 1,147 chars), GTX 1650 Ti fp16, batch 8, threshold 0.4, production call path (gliner_batched.py:249). "Agreement" = mean Jaccard of (text, label) entity sets against the production model over 40 docs — a cheap proxy, not F1.

Model	Labels	doc/s	vs prod	VRAM	CTG est	Agreement
uni-large (production)	16	1.13	1.00x	1,616 M	22.8 d	—
uni-large	6	1.22	1.08x	1,616 M	21.1 d	—
bi-large	16	1.43	1.27x	1,618 M	18.0 d	0.46
uni-base	16	3.95	3.50x	744 M	6.5 d	0.56
bi-base	16	3.40	3.02x	1,013 M	7.6 d	0.45
uni-small	16	6.35	5.63x	655 M	4.1 d	0.50
bi-small	16	4.84	4.29x	903 M	5.3 d	0.44

Three findings

1. The bi-encoder is not the win. It helps only at large (1.27x, below the paper's 39–63% band). At base and small it is slower than the uni-encoder (bi-base 0.86x of uni-base, bi-small 0.76x of uni-small) and uses more VRAM, consistent with carrying a second encoder. The paper's gains come from amortising cached label embeddings across many labels; at 16 labels and ~1,147-char chunks, text encoding dominates and the extra encoder is overhead. Their figures were RTX3090 / fp32 / "dataset-specific labels" — a different regime.

2. Model scale is the real lever. uni-small is 5.63x production (22.8 d → 4.1 d), uni-base 3.50x (→ 6.5 d). That is the 3–6x hoped for, from a different knob than expected.

3. ⚠️ The speedup is not free. Agreement with production is only 0.44–0.56. Even the closest variant shares barely half its entity set with the current model — these are materially different extractions, not cheaper approximations. Bi-encoder variants cluster low (0.44–0.46) at every scale, suggesting the architecture extracts systematically differently.

Also corrected: label count is not a cost lever

Cutting 16 → 6 labels gave 1.08x, not the superlinear saving item 11 predicted. The quadratic term in (|labels| + |text|) is real but irrelevant at ~1,147-char chunks, where |text| dominates. The paper's 92–568% figures were at 127 labels. Item 11 now rests on gazetteer precision, not speed, and drops down the priority list.

What this changes

Item 10 becomes the one decidable item on the list today: throughput is measured, and unlike the retrieval question, the harness already exists — src/ctra/rag/eval/ (NERExperiment, CHIA/BioNLP benchmarks, threshold sweeps) is exactly the tool for adjudicating whether uni-base or uni-small holds up on F1. A 3–6x faster index would also make every later experiment cheaper.

Scripts and raw JSON are in notes/bench_rag_hardware/ (untracked — notes/ is gitignored).