Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 46 additions & 6 deletions metainfer/tasks/dcu_kernel_auto_opt/WORKFLOW.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,13 +88,13 @@ M/N/K 是否合法(M 范围、K%32==0、N%16==0、(K,N) 是否在 TP4/TP8 表
`w8a8_baselines.py::fixed_triton_graph_baseline(shape_id, shape)` 按 `(tp, M, N, K)` 查表,
**查不到就抛 ValueError**(baseline 阶段直接失败),所以新 shape 必须先补表。
表值是 Triton Graph 基线(µs),TP4 M=4096 条目是 2026-08-06 在 worker29 用
`baseline/int8_utils.py`(lmslim)实测的(graph replay median:wqkv_a 13247、wq_b 20590、
`tools/baseline/int8_utils.py`(lmslim)实测的(graph replay median:wqkv_a 13247、wq_b 20590、
wo_b 19882、gate_up 8790、down 5546;wq_b 与 indexer.wq_b 共用一条)。

要自己测 Triton baseline:用 `matmul_int8`(即 SGLang/lmslim 实际调用路径),M>1024 默认 config
是 `BM256/BN256/BK64/GROUP8/SPLIT_K1/warps8`,GPU event、预分配 out(排除分配)、
热缓存协议建议 `warmups=10, samples=20, launches_per_sample=5`;可参考
`zth_infer/baseline/bench_triton_tp4_m4096.py`。
`tools/baseline/bench_triton_tp4_m4096.py`(脚本在 task 目录内,`int8_utils.py` 与之同目录)

### 2.4 Parallel explore(worker 生命周期,w8a8_pipeline.py)

Expand Down Expand Up @@ -180,8 +180,8 @@ TP8 另有 wq_b/wo_b (1024,4096)、gate_up (4096,512)、down (256,4096)、indexe
`MODEL_TP8_EXTRA_OPTIMIZATION_M_VALUES=(4096,)`(**不进** `DEFAULT_OPTIMIZATION_SHAPES`,
默认 42 不变,串行验证 fallback 不受影响);前端三个模型的 TP8 topology 加
`mValues: [2,16,3072,4096]`;基线表 15 条 `(8,4096,…)` 已实测
(`baseline/bench_triton_tp8_m4096.py`,int8_utils.matmul_kernel + CUDA-graph replay,
结果存 `baseline/tp8_m4096_graph.json`)。DeepSeek TP8 不加 M=4096。
(`tools/baseline/bench_triton_tp8_m4096.py`,int8_utils.matmul_kernel + CUDA-graph replay,
结果存 `tools/baseline/tp8_m4096_graph.json`)。DeepSeek TP8 不加 M=4096。
- `MIN_M=1, MAX_M=4096`;`WORKSPACE_BUDGET_BYTES=16MB`。
- **M=4096 时大部分 (N,K) 的 split-K workspace 容量为 0** → 大 M kernel 必须走 2D M-tile 路径,
不能依赖 split-K workspace。
Expand Down Expand Up @@ -254,8 +254,11 @@ TP8 另有 wq_b/wo_b (1024,4096)、gate_up (4096,512)、down (256,4096)、indexe
`int8-w8a8-gemm-foundations`;旧名 `int8-w8a8-quantized-gemm-optimization` 保留为路由器。
其余:`dcu-kernel-tuning`、`hygon-dcu-kernel`、`hygon-gfx928-memory-isa`、
`sglang-custom-kernel-integration`;环境/SSH/容器细节参考 `remote-dcu-env`。
改动 skill 库(`~/.dsh/skills/`)后记得跑 `sync_skill_libraries()` 镜像到
`~/.claude/skills/`(skill_store 测试里有覆盖)。
**规范 skill 以 `metainfer/tasks/dcu_kernel_auto_opt/skills/` 内的副本为种子**(int8-w8a8 家族,
2026-08-27 起随插件一起维护):新机器上 `sync_skill_libraries()`(或 WebUI 同步按钮)会自动
把缺失的 skill 补种进 `~/.dsh/skills/`,再镜像到 `~/.claude/skills/`;已有的库 skill 不被覆盖。
baseline 测量工具在 `tools/baseline/`(`bench_triton_tp4/tp8_m4096.py` + `int8_utils.py`)。
kernel-repos 默认在 MetaInfer 同级(`METAINFER_KERNEL_REPOS` 可改),不随插件目录走。
5. 改动任何行为后:更新本文件相关段落 + 跑 tests + 用真实任务验证(优先在 zth_meta 里)。

## 7. 不确定性标注
Expand All @@ -264,3 +267,40 @@ TP8 另有 wq_b/wo_b (1024,4096)、gate_up (4096,512)、down (256,4096)、indexe
- “恢复流程”是手工驱动(复用 `_synthesize_final_candidate`),不是 UI 一键重试;UI 是否提供重试以
`server/routes.py` 实际实现为准。
- 本文档不替代 skill 里的性能调优细节(tile 选择、LDS、DUMMA API、hipprof 用法),那些看对应 skill。

## 8. AHE 接入准备(M1,2026-09-09,纯新增/默认零行为变化)

设计文档:`MetaInfer/docs/ahe_dkao_integration_plan.md`、`ahe_dkao_design.md`、
`dkao_harness_eval_protocol.md`;AHE 官方参考在 `/root/zth_agent/ahe-ref`。

新增组件(都在本插件内,运行管线默认不消费、不改变行为):

| 文件 | 作用 | wired |
|---|---|---|
| `harness_default/manifest.yaml` | 可演化 harness 组件清单 | false |
| `harness_default/gates.yaml` | gate 规范值(漂移守卫 tests/test_harness_io.py) | false |
| `harness_default/planner_catalog.yaml` | 方案目录(14 个 plan id) | false |
| `orchestrator/harness_io.py` | 定位/读取/播种 harness(`METAINFER_HARNESS_ROOT` 可覆盖) | partial |
| `orchestrator/planner.py` | 状态条件化方案选择器 v0(P0 修复→P1 预算/plateau→P2 瓶颈→P3 覆盖→P4 兜底)+ `render_plan()` 渲染成轮次指令文字 | false |
| `tools/planner_parity.py` | 离线 parity 只读分析(历史轮次状态回放 planner vs 菜单,报告 JSON) | tool |
| `orchestrator/predictions.py` | 内层决策钩子:结构化 prediction 解析/核对(hit/miss/na) | partial |
| w8a8_pipeline.py(改动) | 轮记录加 `prediction_checked` / `plan_id`(proposal 带结构化字段才写) | partial |
| gen_and_opt_pipeline.py(改动) | Generate staging 写 `harness_snapshot/` + scaffold_manifest.harness(revision+digests) | partial |

- 运行时菜单(prompts.py::w8a8_round_strategy)**默认不变**;设 `METAINFER_PLANNER=1` 时
轮次指令改由 planner 渲染(`_round_strategy_text`,w8a8_pipeline.py),用于 A/B 对比。
- worker prompt 模板现提示可选 `plan_id` / `prediction`(expected_us_range/direction),
agent 自主决定是否带;带时轮记录会写入 `prediction_checked` 与 `plan_id`(决策钩子活化)。
- 受控 parity(tests/test_plan_render.py):fresh lane/plateau/ISA/faster_wrong 状态下
planner 渲染文字与菜单关键方向词一致。离线 corpus parity(tools/planner_parity.py)
曾抓出 planner "空 history 误判 fix_build" bug(已修),其精确分歧率受关键字启发式
与历史缺 PMC 影响,仅作 sanity,不作准绳。
- 测试:`tests/test_harness_io.py`、`test_planner.py`、`test_predictions.py`、
`test_plan_render.py`、`test_prompt_schema.py`、`test_planner_wiring.py`;
全量 269 passed。

## 9. harness_evolve 外循环插件(同仓、平级 task,2026-09-09)

新增 `metainfer/tasks/harness_evolve/`(自动发现即可见,headless CLI 用法与边界见其
`README.md`)。它把本插件的 DKAO 任务实例当评测单元跑 AHE 外层闭环(dry-run 已通;
`dkao-cli` evaluator 与真实 Evolve Agent 需在 worker29 后续迭代验证)。
25 changes: 22 additions & 3 deletions metainfer/tasks/dcu_kernel_auto_opt/assets/w8a8_bench.py
Original file line number Diff line number Diff line change
Expand Up @@ -191,9 +191,28 @@ def main() -> int:
parser.add_argument("--m", type=int)
parser.add_argument("--n", type=int)
parser.add_argument("--k", type=int)
parser.add_argument("--warmups", type=int, default=100)
parser.add_argument("--samples", type=int, default=30)
parser.add_argument("--replays-per-sample", type=int, default=100)
# Defaults follow the harness budget env when set (see
# orchestrator/validation_budget.py): METAINFER_BENCH_WARMUPS / _SAMPLES /
# _REPLAYS. Explicit CLI flags still win, so callers keep control.
def _env_default(name: str, fallback: int) -> int:
try:
value = int(os.environ.get(name, ""))
except (TypeError, ValueError):
return fallback
return value if value > 0 else fallback

parser.add_argument(
"--warmups", type=int,
default=_env_default("METAINFER_BENCH_WARMUPS", 100),
)
parser.add_argument(
"--samples", type=int,
default=_env_default("METAINFER_BENCH_SAMPLES", 30),
)
parser.add_argument(
"--replays-per-sample", type=int,
default=_env_default("METAINFER_BENCH_REPLAYS", 100),
)
parser.add_argument("--reference-cache-dir", type=Path)
parser.add_argument("--probe", action="store_true")
parser.add_argument("--self-test", action="store_true")
Expand Down
244 changes: 244 additions & 0 deletions metainfer/tasks/dcu_kernel_auto_opt/bridge/dsh/dsh_agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -189,6 +189,228 @@ def emit_system(session_id: str, model: str) -> None:
})


def _resolve_dsh_bin() -> Optional[str]:
"""Locate the dsh CLI (npm global lives in /usr/local/bin)."""
import shutil

found = shutil.which("dsh")
if found:
return found
for cand in ("/usr/local/bin/dsh", "/usr/bin/dsh"):
if os.path.isfile(cand) and os.access(cand, os.X_OK):
return cand
return None


def _with_dsh_bin_on_path() -> Dict[str, str]:
"""Env for the headless child with the dsh bin dir prepended to PATH."""
env = dict(os.environ)
dsh_bin = _resolve_dsh_bin()
if dsh_bin:
bindir = os.path.dirname(dsh_bin)
path = env.get("PATH") or ""
entries = [p for p in path.split(":") if p and p != bindir]
env["PATH"] = ":".join([bindir] + entries + ["/usr/bin", "/bin"])
return env


#: dsh-base tool rows -> the agent-facing tool names each one provides.
#: Used to build a profile patch that disables every row outside a whitelist.
_TOOL_ROWS: Dict[str, set] = {
"tool-fs": {"read", "write", "edit", "multiedit", "notebookedit"},
"tool-fs-search": {"glob", "grep", "search"},
"tool-str-replace-editor": {"strreplace", "str_replace", "edit"},
"tool-bash": {"bash", "bashoutput", "killshell", "run"},
"tool-pwsh": {"powershell"},
"tool-skill": {"skill"},
"tool-web": {"webfetch", "websearch", "web"},
"tool-todo": {"todowrite", "todoread", "todo"},
"tool-jobs": {"joboutput", "jobkill", "joblist", "jobs"},
"tool-workflow": {"workflow"},
"tool-ralph": {"ralph"},
"tool-goal": {"goalcreate", "goalupdate", "getgoal", "goal"},
"tool-subagent": {"task", "subagent"},
"tool-subagent-control": {"sendmessage", "interruptagent"},
"tool-subagent-list-agents": {"listagents"},
"tool-subagent-fork": {"subagentfork"},
"tool-subagent-report": set(),
}


def _parse_tool_list(raw: Optional[str]) -> set:
if not raw:
return set()
return {
part.strip().lower().replace("-", "").replace("_", "")
for part in raw.replace(" ", ",").split(",")
if part.strip()
}


def _tool_patch_file(args: Any) -> Optional[str]:
"""Write a dsh profile patch disabling tool rows outside the whitelist.

Returns the patch path (caller deletes it) or None when no gating was
requested. ``--tools`` is a whitelist; ``--disallowedTools`` additionally
removes rows. Rows not listed in the dsh-base bundle are never targeted,
so the patch stays valid across profiles.
"""
allowed = _parse_tool_list(getattr(args, "tools", None))
denied = _parse_tool_list(getattr(args, "disallowed_tools", None))
if not allowed and not denied:
return None
lines = [
"# generated by MetaInfer dsh_agent headless backend "
"(tool gating from --tools/--disallowedTools)",
]
disabled_any = False
for row, tools in _TOOL_ROWS.items():
if allowed:
keep = bool(tools & allowed)
else:
keep = True
if tools & denied:
keep = False
if not keep:
lines.append(f"- id: {row}")
lines.append(" disabled: true")
disabled_any = True
if not disabled_any:
return None
import tempfile
fd, path = tempfile.mkstemp(suffix=".patch.yml", prefix="dsh-tools-")
with os.fdopen(fd, "w", encoding="utf-8") as fh:
fh.write("\n".join(lines) + "\n")
return path


def _headless_backend_main(args: Any, prompt: str) -> int:
"""Run one task through the local `dsh --profile headless` CLI.

The full prompt is written to a temp brief file because prompts here can
exceed the kernel argv limit (one DKAO brief is up to ~300 KB) and the
headless app only accepts its task on argv. We ask the agent to Read the
brief and follow it. A keepalive line is written to stdout while the
sub-agent is still working so SubAgentManager's no-output watchdog never
mistakes a long task for a stuck process; the final assistant text is
emitted as the last message, matching the ccb stream-json contract.
"""
import subprocess
import tempfile
import threading
import time

dsh_bin = _resolve_dsh_bin()
if not dsh_bin:
sys.stderr.write(
"dsh_agent: headless backend requires the `dsh` CLI on PATH\n"
)
return 1

model = map_model(args.model)
session_id = args.resume or args.session_id or f"session-{os.urandom(8).hex()}"

fd, brief = tempfile.mkstemp(suffix=".md", prefix="dsh-task-brief-")
tool_patch: Optional[str] = None
try:
with os.fdopen(fd, "w", encoding="utf-8") as fh:
fh.write(prompt)
task = (
f"You are a coding sub-agent inside a kernel-optimization pipeline. "
f"Read the complete task brief at {brief} with the Read tool and "
"follow it to completion, autonomously, without asking the user. "
"When the task is finished, output exactly the final result text "
"and stop."
)
tool_patch = _tool_patch_file(args)
cmd = [dsh_bin, "--profile", "headless"]
if tool_patch:
cmd += ["--patch", tool_patch]
cmd.append(task)
proc = subprocess.Popen(
cmd,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
env=_with_dsh_bin_on_path(),
)
except Exception as exc: # noqa: BLE001
if tool_patch:
try:
os.unlink(tool_patch)
except OSError:
pass
sys.stderr.write(f"dsh_agent: failed to start headless run: {exc!r}\n")
return 1

out_lines: List[str] = []
err_lines: List[str] = []

def pump(pipe, sink: List[str]) -> None:
assert pipe is not None
for line in pipe:
sink.append(line)

t_out = threading.Thread(target=pump, args=(proc.stdout, out_lines), daemon=True)
t_err = threading.Thread(target=pump, args=(proc.stderr, err_lines), daemon=True)
t_out.start()
t_err.start()

emitted_system = False
last_beat = time.time()
beat_secs = float(os.environ.get("DSH_AGENT_BEAT_SECONDS", "45"))
while proc.poll() is None:
if not emitted_system:
emit_system(session_id, model)
emitted_system = True
if time.time() - last_beat >= beat_secs:
sys.stdout.write("# dsh-agent keepalive\n")
sys.stdout.flush()
last_beat = time.time()
time.sleep(1)

t_out.join(timeout=10)
t_err.join(timeout=10)

if not emitted_system:
emit_system(session_id, model)

def _cleanup_temps() -> None:
for path in (brief, tool_patch):
if path:
try:
os.unlink(path)
except OSError:
pass

if proc.returncode != 0:
tail = "".join(err_lines)[-2000:]
sys.stderr.write(
f"dsh_agent: headless run failed (rc={proc.returncode}): {tail}\n"
)
_cleanup_temps()
return 1
final_text = "".join(out_lines).strip()
if not final_text:
sys.stderr.write("dsh_agent: headless run produced no output\n")
_cleanup_temps()
return 1
_cleanup_temps()
emit({
"type": "assistant",
"session_id": session_id,
"message": {"content": [{"type": "text", "text": final_text}]},
})
emit({
"type": "result",
"session_id": session_id,
"result": final_text,
"finish_reason": "completed",
"usage": {},
})
return 0


def extract_text_blocks(content: Any) -> List[str]:
"""Pull text blocks from an assistant message content array."""
if not isinstance(content, list):
Expand Down Expand Up @@ -219,6 +441,11 @@ def build_parser() -> argparse.ArgumentParser:
p.add_argument("--resume")
p.add_argument("--session-id")
p.add_argument("--max-turns")
# Tool gating (used by the HE evolve agent). The headless backend maps
# these ccb-style tool lists onto a dsh profile patch that disables the
# tool rows outside the whitelist.
p.add_argument("--tools")
p.add_argument("--disallowedTools", dest="disallowed_tools")
# Anything else (claude-specific) is ignored.
return p

Expand All @@ -231,6 +458,23 @@ def main(argv: Optional[List[str]] = None) -> int:
sys.stderr.write("dsh_agent: empty prompt on stdin\n")
return 1

# Backend selection. ``headless`` runs the local `dsh --profile headless`
# CLI (one autonomous agent task, prints the final result); ``sdk`` uses
# the deepseek-harness python SDK + node carrier below. ``auto`` prefers
# headless when the dsh CLI is on PATH (the SDK node carrier needs a
# repo-built runtime closure that may be absent on worker hosts).
backend = os.environ.get("DSH_AGENT_BACKEND", "auto").strip().lower()
if backend == "auto":
backend = "headless" if _resolve_dsh_bin() else "sdk"
if backend == "headless":
return _headless_backend_main(args, prompt)
if backend != "sdk":
sys.stderr.write(
f"dsh_agent: unknown DSH_AGENT_BACKEND {backend!r} "
"(use sdk | headless | auto)\n"
)
return 1

try:
from deepseek_harness import DeepSeekHarness, DeepSeekHarnessConfig
except ImportError as exc: # pragma: no cover - environment check
Expand Down
Loading
Loading