Skip to content

docs: fix benchmark repository references - #10

Merged
zhanghanduo merged 2 commits into
mainfrom
docs/fix-benchmark-links
Aug 25, 2026
Merged

zhanghanduo merged 2 commits into
mainfrom
docs/fix-benchmark-links

Conversation

@ntudy

@ntudy ntudy commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • update benchmark documentation to use the bundled FrontierAgent paths
  • remove stale internal-repository and Git-subtree references
  • align FrontierSearchBench scorer attribution and FrontierChallenge site documentation with the public repository

Validation

  • checked all affected local Markdown links and image targets
  • verified benchmark registry entries and documented task/query counts
  • ran Python syntax checks and git diff --check
  • completed a defect-first review with no findings

Test notes

  • full pytest suite was not run because the local project virtual environment is not configured

@ntudy
ntudy requested review from zhanghanduo and a balanced review from Copilot August 25, 2026 09:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates benchmark documentation and scorer attribution to reflect bundled FrontierSearchBench and FrontierChallenge paths.

Changes:

  • Removes stale standalone-repository and Git-subtree references.
  • Updates benchmark paths, ownership notes, and site deployment guidance.
  • Points scorer attribution headers to the FrontierAgent repository.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
README.md Describes bundled benchmarks.
docs/framework.md Updates architecture tree.
docs/eval-frontier-search.md Revises benchmark ownership and workflow references.
benchmarks/README.md Updates benchmark descriptions and registry terminology.
benchmarks/public/runner/score_frontier_search.py Updates scorer comment terminology.
benchmarks/public/README.md Documents bundled sibling benchmark.
benchmarks/frontierchallenge/site/README.md Corrects site build and deployment instructions.
benchmarks/frontier_search_bench/UPSTREAM.md Replaces obsolete subtree provenance.
benchmarks/frontier_search_bench/README.md Documents in-repository paths and setup.
benchmarks/frontier_search_bench/eval/verifiable/scorers/query_38/auto_scorer.py Updates HTTP attribution.
benchmarks/frontier_search_bench/eval/verifiable/scorers/query_15/auto_scorer.py Updates HTTP attribution.
benchmarks/frontier_search_bench/eval/verifiable/README.md Updates directory terminology; credential location remains inaccurate.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread benchmarks/frontier_search_bench/eval/verifiable/README.md Outdated

@zhanghanduo zhanghanduo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zhanghanduo
zhanghanduo merged commit 9c64f25 into main Aug 25, 2026
7 checks passed
@zhanghanduo
zhanghanduo deleted the docs/fix-benchmark-links branch August 25, 2026 09:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants