Skip to content

feat(klx): deepeyes v2 agentic run through and raw_reward match - #309

Open
grootZ wants to merge 1 commit into
redai-studio:mainfrom
grootZ:mr_master
Open

feat(klx): deepeyes v2 agentic run through and raw_reward match#309
grootZ wants to merge 1 commit into
redai-studio:mainfrom
grootZ:mr_master

Conversation

@grootZ

@grootZ grootZ commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

What

镜像:iregistry.baidu-int.com/xpu/xrelax_torch29_ubuntu2204_xsgl0510_dev:20260831_3
启动命令:bash examples/deepeyes_v2_agentic/run_deepeyes_v2_agentic_klx.sh
目前能够跑通,已完成200步并且raw_reward对齐

Why

How

Testing

  • pre-commit run --all-files passes
  • Tests pass (pytest tests/)
  • New tests added (if applicable)
  • Documentation updated (if applicable)

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Refactoring (no functional changes)
  • Performance improvement
  • CI/CD or build changes

Screenshots / Logs

@grootZ
grootZ requested a review from yxyOo as a code owner September 10, 2026 08:39

@rai-studio-bot rai-studio-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

发现需要在合入前修复的 agent 启动问题,另有并发参数未生效的问题;细节和修复建议见行级评论。Bash 语法检查通过;CI 的 pre-commit 已通过,Python 测试仍在运行。当前无可用昆仑芯训练集群,未执行硬件集成测试。

  • P3 优先级:P3 非行级:PR 描述仍为空模板。请补充昆仑芯硬件与软件镜像版本、实际启动命令、已完成的 rollout/训练步数,以及标题中 “match” 的对齐指标和结果,便于复现和判断验证范围。
Powered by Nyanpasu with gpt-6-astra medium, please check the suggestions carefully.

# agent process can find apptainer / search cache.
# SANDBOX_CONFIG_PATH is required — the agent reads it in _build_executor
# to find the apptainer backend YAML config (image path, bind paths, etc).
EXTRA_ENV_VARS_JSON="\"SANDBOX_BACKEND\": \"apptainer_jupyter\",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 优先级:P1

请恢复 agent Python 路径的解析、校验和 runtime env 传递。原启动脚本会从 DEEPEYES_V2_APP_ENV_ROOT 解析该路径,这里没有对应设置,env.sh.example 也仅设置环境根目录。run_agent_app.sh 启用了 set -eu,第 22 行直接读取 DEEPEYES_V2_APP_PYTHON;隔离运行已复现 unbound variable,因此按默认准备流程启动时,每个 agent 都会在生成轨迹前退出。

可在构造环境 JSON 时恢复这项配置:

Suggested change
EXTRA_ENV_VARS_JSON="\"SANDBOX_BACKEND\": \"apptainer_jupyter\",
DEEPEYES_V2_APP_PYTHON="${DEEPEYES_V2_APP_PYTHON:-${DEEPEYES_V2_APP_ENV_ROOT:-/tmp/deepeyes-v2-app-env}/.venv/bin/python}"
if [ ! -x "${DEEPEYES_V2_APP_PYTHON}" ]; then
echo "ERROR: Run bash ${SCRIPT_DIR}/scripts/prepare_app_env.sh to prepare ${DEEPEYES_V2_APP_PYTHON}."
exit 1
fi
EXTRA_ENV_VARS_JSON="\"DEEPEYES_V2_APP_PYTHON\": \"${DEEPEYES_V2_APP_PYTHON}\",
\"SANDBOX_BACKEND\": \"apptainer_jupyter\",

--n-samples-per-eval-prompt 8
--eval-max-response-len 4096
--eval-top-p 0.7
--agentic-eval-prepare-pool-size 32

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 优先级:P2

请使用当前的评估并发参数。relax/utils/arguments.py 只注册了 --agentic-eval-concurrency,而 Megatron 解析入口设置了 ignore_unknown_args=True,这里的 prepare-pool 参数会被忽略。评估实际按训练会话容量推导并发;例如将 ROLLOUT_BATCH_SIZE 改为 64、保持每组 8 个样本时,评估会运行 64 组,而非此处配置的 32 组,增加 agent/sandbox 资源占用。

Suggested change
--agentic-eval-prepare-pool-size 32
--agentic-eval-concurrency 32

第 176 行的 --agentic-prepare-pool-size 0 同样未注册,应删除;当前训练组并发由 --agentic-concurrency 控制,且要求大于 0。

@rai-studio-bot rai-studio-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

描述已补充镜像及 200 步运行、raw_reward 对齐说明,上次的描述建议已部分落实。当前提交未变化,原行级评论中的启动和并发参数问题仍待修复;本次没有新增行级评论。CI 已全部通过。

  • P3 优先级:P3 非行级:新补充的验证命令指向 run_deepeyes_v2_agentic.sh,而本 PR 新增的是 run_deepeyes_v2_agentic_klx.sh,尚无法确认 200 步结果是否覆盖新增脚本。若实际验证的是昆仑芯脚本,请将命令更正为 bash examples/deepeyes_v2_agentic/run_deepeyes_v2_agentic_klx.sh,并注明运行所用提交和必要的环境覆盖;若运行的是原脚本,请明确该验证范围,并补充新增脚本的运行结果。
Powered by Nyanpasu with gpt-6-astra medium, please check the suggestions carefully.

@grootZ grootZ changed the title feat(klx): deepeyes v2 agentic run through and match feat(klx): deepeyes v2 agentic run through and raw_reward match Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants