feat(e2e): native (.cu/.hip/.cpp/CK) apply-back — in-place incremental recompile, reversible - #323
Open
zihaoanllm wants to merge 5 commits into
Open
feat(e2e): native (.cu/.hip/.cpp/CK) apply-back — in-place incremental recompile, reversible #323zihaoanllm wants to merge 5 commits into
zihaoanllm wants to merge 5 commits into
Conversation
…y_setup.py [Phase 1] The e2e overlay is Python-only (sitecustomize/PYTHONPATH inject + attr rebind), so a compiled-source kernel rewrite (aiter/CK .cu, local HIP .hip, cpp_extension, ...) can be measured isolated but never deployed into the live server — it gets rejected no_rebind_seam/editable=false. This adds the framework-agnostic, reversible plumbing to apply such a patch IN PLACE and undo it byte-exact, while keeping ONE tool + ONE manifest + ONE revert shared with the existing Python overlay. This is the safety spine only; it never invents a build command — the caller (e2e_integrator agent, Phase 2) discovers the install's own incremental build the same way benchmark_engineer does and passes it via --build-cmd, so the design stays general and hardcode-free. overlay_setup.py — new manifest section "natives" + subcommands: - add-native: back up source + named artifacts + tar-snapshot the invalidated cache dir(s), save the manifest BEFORE mutating (crash-safe: every entry references an existing backup), then swap the source in place, delete the cache(s) to force an incremental rebuild, and run the caller-supplied build. Only the changed translation unit rebuilds (ninja/jit/cpp_itfs cache keyed by the change). - verify-native: confirm each tracked artifact actually changed/was created (catch a silent no-op build). - revert: unified — Python overlay drops off PYTHONPATH; native sources/artifacts/cache dirs restored byte-exact from backup, reverse order, idempotent (natives cleared only on full success). - gc-stale: scan a root for a crashed run's still-applied native overlay and revert it, so a fresh session never starts on a dirty install. test_overlay_native.py — stdlib unittest (no GPU/server): apply+rebuild, fresh-build verify (pass + silent-no-op fail), byte-exact revert (source/artifact/cache), artifact-created-then-removed, double-revert idempotency, revert-without-build, mixed Python-overlay + native round-trip, gc-stale. 9 tests green. Default manifest gains "natives": [] (backward-compatible; existing flows untouched). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ne [Phase 2]
Phase 1 added the framework-agnostic reversible plumbing (overlay_setup.py add-native/verify-native/
revert/gc-stale). Phase 2 wires the "brain" so a compiled-kernel win actually deploys end-to-end:
- roles/e2e_integrator.md: new NATIVE patch path in the winner_kind=patch case. Detect native from
apply_kind=="native" OR a code_patch touching a compiled suffix (.cu/.cuh/.hip/.cpp/.cc/.cxx/.c/.h/.hpp).
DISCOVER the install's own incremental build (config.yaml compile_command / task_runner / Makefile /
build.sh / ninja / CMake / torch cpp_extension load(name=) / aiter cpp_itfs MD_NAME) by walking up from
the source — never invent it — and hand it to add-native via --build-cmd/--invalidate-cache. verify-native
to catch silent no-op builds. A/B order is REF-first (clean install) -> add-native -> CAND, since native
mutates the install globally; ALWAYS revert in a finally (accept/reject/crash) + assert install clean;
serialize. Graph-safe + mem-footprint gates apply as for authored kernels. no_build_seam when no
rebuildable seam exists.
- roles/kernel_extractor.md: native (compiled-source) kernels in a rebuildable tree are now editable=true
with apply_kind:"native" (target_callable:""); editable=false reserved for opaque prebuilt libs / read-only
installs. .py/Triton stay apply_kind:"python" (unchanged).
- e2e_workflow.js: add apply_kind ('python'|'native', default python) to EXTRACT_OP_SCHEMA + EXTRACT_SCHEMA;
thread it into the serial-head and milestone KERNEL_RESULT inputs (the integrator also auto-detects native
from patch suffixes, so missing threading degrades safely).
- knowledge/learned/native-apply-back.md (+ INDEX): the method card.
Default behavior byte-identical for Python/Triton winners (apply_kind defaults to python; the native path
only triggers on a compiled-suffix patch or explicit apply_kind=native). node --check clean; Phase-1 unit
tests still 9/9.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
General, default-off passthrough so a standalone e2e_workflow run can steer the System Architect / optimize prompts (e.g. 'only optimize compiled-source/HIP kernels; skip Triton'). Omitted => byte-identical (no hint). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The python|native exclusive axis prevented a run from composing backends across languages. Drop it everywhere (schema fields + JS threading + role docs + knowledge card). A patch's deploy mechanism is now derived per-hunk from the file suffix: compiled-suffix hunks recompile in place via overlay_setup.py add-native; .py hunks deploy as a PYTHONPATH overlay. One winner's patch may carry both -> each hunk routes by its own suffix, so a run can bank, e.g., a Triton norm and a native HIP GEMM in the same pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… no hardcode) A head GEMM authored a real 1.92x isolated kernel on Qwen3-4B/vLLM that engaged 0 times e2e and was gate-rejected. Root cause was 100% seam resolution: the extractor returned a guessed launcher (aiter.tuned_gemm:gemm_a16w16) inferred from the trace kernel name, but the live dense GEMM dispatched to closed hipBLASLt (the Cijk_* Tensile kernels) -- a different module, a shape-gated branch the model never hit, a closed leaf. The authored kernel bound a symbol nothing calls. Make seam resolution evidence-driven and framework-agnostic: - New method card knowledge/learned/method-resolve-live-seam.md: classify the profiled GPU kernel's backend family from the trace + owning .so; walk the LIVE installed dispatch source under the server's env + captured shapes; CONFIRM with a temporary throwaway [seam-probe] passthrough before authoring; closed-library leaf (Tensile/hipBLASLt) -> editable=false -> Config Tuner, not author; bind the nearest common chokepoint when decode/prefill branches diverge; prove engagement by re-observation. - kernel_extractor.md: replace the hardcoded "dense GEMM seam is aiter.tuned_gemm:gemm_a16w16" guidance (both PHASE=extract and PHASE=extract_op) with the resolve-then-confirm procedure; a Cijk_*/ Tensile/hipBLASLt leaf is closed -> editable=false. - gemm_attention_backends.md: scope the aiter seam to an EXAMPLE of one resolved path (sglang/gfx942), not the default; require resolution first. - INDEX.md: register the new method card. No backend is privileged; no module name is hardcoded; the seam is whatever the live dispatch actually calls for the captured shapes under the live env. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
e2e overlay is Python-only (PYTHONPATH inject / attr rebind). It can shadow a
.pybut NOT a.so,so a win on a COMPILED kernel (aiter/CK
.cu, HIP.hip, cpp_extension) can be measured isolated butnever deployed into the live server — it gets rejected
no_rebind_seam/editable=false. The change onlytakes effect after the package is recompiled.
What this does
Adds a NATIVE apply-back path: deploy a compiled-kernel win by recompiling it in place — general
(any build system), incremental (only the changed unit), reversible, and with no hardcoded framework paths.
overlay_setup.py:add-native(back up → swap source → invalidate only the affected cache → run thecaller-supplied incremental build) /
verify-native(confirm the artifact really rebuilt) /revert(byte-exact) /
gc-stale. Never invents a build command.e2e_integrator.md: native patch path — DISCOVER the install's own incremental build (config.yaml /task_runner / ninja / CMake / torch cpp_extension / aiter cpp_itfs) by walking up from the source,
REF-first A/B, ALWAYS revert in a finally.
kernel_extractor.md: compiled source in a rebuildable tree→
editable=true, apply_kind:"native".e2e_workflow.js:apply_kindschema/threading.run_e2e.py:optional
tasksteering passthrough.Validation
.hip→ add-native real recompile →.sochanges →FRESH_BUILD_OK → GPU output 4.0→7.0 → revert restores byte-exact → 4.0.
Default behavior byte-identical for Python/Triton winners (native path only triggers on a compiled-suffix
patch or explicit
apply_kind=native).