agentbench: 改为接标准基准的薄入口（HumanEval / SWE-bench） - #70
Merged
Merged
Conversation
不再自带题目。agentbench 现在是调用业界两套标准基准（Codex / Claude 评测同款）的入口，把 byteseek 当作被测 agent 接进去： - 新增 bench（bash 入口，exec 到 BENCH_HOME 的 venv） - 新增 bench.py：题库 → agent → predictions → 官方评测 suite: humaneval | humaneval+ | swebench-verified | swebench-lite agent: gold（题库自带答案，验管道）| byteseek 退出码：0 通过 / 1 未通过 / 2 前置条件不满足 / 3 参数错误 - 删除自出的 00–10 难度题库（64 个 .question/.answer 对） - README 重写：新定位、命令、suite 说明 Co-Authored-By: Claude Code <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
变更
agentbench 不再自带题目，改为调用业界两套标准基准（Codex / Claude 评测同款），把 byteseek 当作被测 agent 接进去。
新增
agentbench/bench—— bash 入口（exec 到BENCH_HOME的 venv）agentbench/bench.py（368 行）—— 题库 → agent → predictions → 官方评测humaneval|humaneval+|swebench-verified|swebench-litegold（题库自带答案，验管道）|byteseek（驱动 byteseek 代码脑）0通过 /1未通过 /2前置条件不满足（缺题库、缺 docker、byteseek 不可用）/3参数错误删除
00–10难度题库（64 个.question/.answer对）修改
agentbench/README.md重写：新定位、命令、suite 说明说明
本 PR 不含工作区里另一处改动（
tests/selftest.kv的删除）—— 该文件仍是make test与 CIselftestjob 的入口，删除会让 CI 必挂且 Makefile/CI 未同步。已从本次提交撤回，另行讨论。🤖 Generated with Claude Code