[yodablocks]

What

An experiment, not a change to the package: TypeSafe's Jev as a drop-in re-ranker (one Noul per question and passage pair), compared with the production bge-reranker-v2-m3 cross-encoder on the 38-question gold set. Everything is under experiments/jev_rerank/. Nothing in quarq/ changes.

Decision bands were committed before the first run (see the experiment README).

Result (one run, then a wording-sensitivity run)

Arm	Hit@1	Hit@5	MRR	Query latency p50
embedding only	23/38	35/38	0.74	50 ms
bge (production)	33/38	36/38	0.90	2,079 ms (Apple GPU)
Jev	36/38	36/38	0.95	862 ms (12 parallel calls)

• Jev fixed 3 questions and broke none. That is exactly the "beats" threshold.
• Three wordings of the Noul question all scored 36/38 with identical per-question results.
• 380 calls cost about $0.014. Per-call latency p50 520 ms, p95 about 810 ms.
• Both re-rankers miss the same 2 questions (b1-020, b2-011): the gold page is not in the embedding top 40, so no re-ranker can fix them.

Caveats

• 38 questions: one question is 2.6 points, and the win is +3. Read it as "at least as good", not a proven win.
• 28 of the 38 questions were drafted from the chunks being searched, which flatters retrieval.
• Identical results across wordings may mean the set is too easy to separate them.
• Jev is a cloud API: the question and passage text leave the machine. The corpus is public documents, but quarq's local-handling promise does not hold for this arm.
• Latency is not a fair comparison: bge ran locally, Jev over the network.
• Not measured: run-to-run variation of one fixed wording (responses are cached), French-only subset, local alternatives.

Reproduce

export TYPESAFE_API_KEY=...
python3 experiments/jev_rerank/run.py --arms embedding,bge,jev
python3 experiments/jev_rerank/run.py --arms jev:v2,jev:v3

Responses are cached in experiments/jev_rerank/.cache/ (gitignored). pytest tests/: 247 passed, unchanged.