Reduce repeated prompt tokens in listwise relevance scoring

[hotchpotch]

Summary

The default listwise relevance prompt repeated its full instructions for every document. Store the evaluation rubric once per request and use shorter per-document references with explicit true/false criteria.

The updated preset considers both evidence usefulness and retrieval relevance, including partial answers and supporting facts. Existing library calls require no new options; the default threshold remains 0.2. Scores and filtering decisions may change.

• Include the rubric in request-size estimates and retain the complete submitted prompt in execution details.
• Preserve existing request formats for ordinary reranking, pointwise relevance, and custom prompts.
• Let the evaluation script use all dataset queries by default, with optional --query-limit, --split-state-budget, and --split-request-budget controls. Document Gemma tokenizer usage with explicit budgets.
• Update usage documentation and add regression coverage for request sizing, splitting, document mapping, filtering, prompt overrides, and evaluation CLI settings.

Validation

• uv run --locked tox: 209 tests passed, 52 live tests skipped; lint and type checks passed.
• uv run --locked pytest tests/test_instructions.py: 60 passed.
• git diff --cached --check: passed before committing.