+
Benchmark · Annotation
+
Inspect evaluation answers and mark evidence quality.
+
+
+
+
+
+
+
+
+
+
Evaluation
+
+
ClimRetrieve v1 · Ranking
+
Question
+
+ - TCFD-01
+ - TCFD-02
+ - TCFD-03
+
+
12 / 16 evaluated · 3 annotations saved
+
+
+
Question
+
How does the board oversee climate-related risks?
+
Reference type: Governance · Expected chunks: 3
+
+
Retrieved evidence
+
+
+
+ | Rank |
+ Relevance |
+ Source passage |
+ Mark |
+
+
+
+
+ | 1 |
+ [Relevant v] |
+ “The Sustainability Committee meets quarterly to review climate risk…” · p.14 |
+ [x] |
+
+
+ | 2 |
+ [Partly relevant v] |
+ “The CRO reports climate metrics to the audit committee…” · p.18 |
+ [x] |
+
+
+ | 3 |
+ [Not relevant v] |
+ “Scenario analysis is presented annually to the full board…” · p.22 |
+ [ ] |
+
+
+
+
Relevance scale: 0 Not relevant · 1 Partly relevant · 2 Relevant
+
Annotation note
+
+
+
+
+
+
+
+
+
Evaluation details
+
Model answer: The board’s Sustainability Committee reviews climate risks quarterly, with reporting from the CRO.
+
Matched benchmark chunks: 2 / 3 · Evidence coverage: 67%
+
Annotation history: 3 saved · Last edited by you just now
+
+
+
+
+
+
+
+
+
Benchmark · Evaluation
+
Run a ranking evaluation against a shared benchmark.
+
+
+
+
+
+
+
+
+
+
Evaluation mode
+
+
Ranking compares the order of retrieved chunks. Classification compares one predicted label per question.
+
+
+
Benchmark set
+
+
16 questions · 4 question types · shared reference set
+
+
+
Your dataset
+
+
Saved from Upload & map · ranking format validated
+
+
+
Question matching
+
12 / 16 questions will be evaluated
+
███████████████░░░ 75%
+
12 questions have matching IDs and compatible question types. 4 questions are not present in your dataset and will be excluded.
+
+
+
+ | Match status |
+ Questions |
+ Action |
+
+
+
+
+ | Ready to evaluate |
+ 12 |
+ Included |
+
+
+ | Missing from dataset |
+ 3 |
+ Excluded |
+
+
+ | Type mismatch |
+ 1 |
+ Excluded |
+
+
+
+
Higher question coverage generally gives a more reliable and accurate benchmark result.
+
+
+
Ranking configuration
+
+
+
+
+
K values:
+
Evaluation name:
+
+
+
+
+
+
+
+
+
+
+
+
+
Benchmark · Results
+
Review metrics and compare saved evaluations.
+
+
+
+
+
+
+
+
+
+
Microsoft retrieval · v2
+
ClimRetrieve v1 · Ranking · Completed 02 Sep 2026, 14:32
+
12 / 16 questions evaluated · 75% coverage · 4 excluded
+
Higher values indicate that relevant chunks were ranked earlier and the result is more accurate against the reference set.
+
+
Result
+
+
+
+ | MAP |
+ MRR |
+ Precision@5 |
+ Recall@10 |
+
+
+
+
+ | 0.742 |
+ 0.833 |
+ 0.780 |
+ 0.910 |
+
+
+
+
+
Compare evaluations
+
Select saved runs to compare side by side.
+
+
+
+ | Evaluation |
+ Dataset |
+ Questions |
+ MAP |
+ MRR |
+ Precision@5 |
+
+
+
+
+ | [x] Microsoft v2 |
+ Microsoft retrieval · v2 |
+ 12/16 |
+ 0.742 |
+ 0.833 |
+ 0.780 |
+
+
+ | [x] Northwind v1 |
+ Northwind retrieval · v1 |
+ 16/16 |
+ 0.681 |
+ 0.750 |
+ 0.702 |
+
+
+ | [ ] Chunks baseline |
+ chunks_data.csv |
+ 10/16 |
+ 0.554 |
+ 0.600 |
+ 0.571 |
+
+
+
+
Metric comparison
+
+
+
+ | Metric |
+ Microsoft v2 |
+ Northwind v1 |
+ Difference |
+
+
+
+
+ | MAP |
+ 0.742 |
+ 0.681 |
+ +0.061 |
+
+
+ | MRR |
+ 0.833 |
+ 0.750 |
+ +0.083 |
+
+
+ | Precision@5 |
+ 0.780 |
+ 0.702 |
+ +0.078 |
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Benchmark · Upload and map
+
Prepare a dataset for evaluation.
+
+
+
+
+
+
+
+
+
+
1. Upload · 2. Align · 3. Review mapping
+
Upload your system output, align it with the benchmark format, then confirm the detected fields before evaluation.
+
+
+
Upload dataset
+
+
+
+
+
Selected file: chunks_Microsoft_2024.csv · 1,240 rows · Uploaded just now
+
Supported ranking fields: query/item ID, report ID, chunk ID, position/rank, optional score.
+
+
+
Benchmark format
+
Reference set:
+
+ - 16 benchmark questions
+ - 4 expected question types
+ - Ground-truth labels managed by the benchmark owner
+
+
+
+
+
Align columns
+
+
+
+ | Your column |
+ Benchmark field |
+ Status |
+
+
+
+
+ | [query_id________v] |
+ Query / question ID |
+ ✓ matched |
+
+
+ | [report_id_______v] |
+ Report ID |
+ ✓ matched |
+
+
+ | [chunk_id________v] |
+ Retrieved chunk ID |
+ ✓ matched |
+
+
+ | [position________v] |
+ Rank / position |
+ ✓ matched |
+
+
+ | [score___________v] |
+ Relevance score |
+ Optional |
+
+
+
+
+
+
+
+
+
+
Mapping preview
+
16 / 16 benchmark questions found · 1,240 / 1,240 rows valid · 0 errors
+
+
+
+ | Question ID |
+ Report |
+ Top retrieved chunks |
+ Question type |
+
+
+
+
+ | TCFD-01 |
+ Microsoft 2024 |
+ chunk_014, chunk_088, chunk_102 |
+ Governance |
+
+
+ | TCFD-02 |
+ Microsoft 2024 |
+ chunk_031, chunk_044, chunk_090 |
+ Strategy |
+
+
+ | TCFD-03 |
+ Microsoft 2024 |
+ chunk_052, chunk_067, chunk_071 |
+ Risk |
+
+
+
+
Dataset name
+
+
+
+
+
+
+
+
+
+
+
+