feat(hip-kernel-provider): rocKE AOT producer + kpack packaging (AICK-1473) by BrianHarrisonAMD · Pull Request #9207 · ROCm/rocm-libraries

BrianHarrisonAMD · 2026-07-08T22:22:54Z

Summary

Re-lands PR #8866's rocKE AOT build + sidecar generator onto the rocke/library/
layout from #9159, and extends it through kpack packaging and install. Producer
and packaging only: the engine's AotCatalog stays empty, so it still declines
every graph at runtime. The only C++ touched is a path constant
(defaultArtifactRoot()) pointed at the final install layout.

JIRA ID : AICK-1473

Install layout

Per arch, the producer packs HSACO into rocke_client_<arch>.kpack with a
rocke_client_<arch>.json (hipdnn.rocke.bundle/v1) bundle manifest, installed
next to the engine plugin at
lib/hipdnn_plugins/engines/hip_kernel_provider/rocke/<arch>/ (the future home
for rocke's per-arch kernel-selection heuristics too). Arch-scoped paths union
cleanly, so TheRock's multi-arch merge needs no kpack fusing. AotCatalog's
defaultArtifactRoot() now names this plugin-relative root; it is resolved next
to the loaded plugin and the device arch appended. It remains a path constant
(nothing reads it in Phase 1); the dispatcher-wiring work consumes it.

Cross-repo sequence

rocm-systems feat(kpack): add hipkernelprovider per-arch split handler (AICK-1616) rocm-systems#8347: the hipkernelprovider kpack split handler.
TheRock feat(hipkernelprovider): rocKE AOT kpack packaging & arch split TheRock#6427: build deps, wheel packaging, per-arch split (topology + artifact TOML + device wheel filter).
rocm-libraries (this PR): the AOT producer that emits the per-arch bundles.

Requires TheRock #6427's build deps (ROCKE_KPACK_PYTHON_DIR, comgr) to build
under the superbuild.

Risk Assessment

Low behaviorally. Adds a build-time producer plus packaging subsystem (Python and
CMake) and its pip deps to the rocke-pyenv; no engine behavior change and the
catalog stays empty, so there is no regression surface for #9159's dispatcher
tests. Risk is confined to the build/packaging path.

Testing Summary

AOT Python suites under the rocke-pyenv (schema, sidecar, instance, SDPA
handler, build-tool CLI, kpack round-trip, manifest validation), 90% gate.
CMake configure check of the library/aot subtree: target graph, per-arch
kpack and install rules, fail-loud rocm_kpack gate, comgr resolution.
Producer-to-packer end-to-end (stubbed compiler, real rocm_kpack): packed
sha256 == sidecar hsaco_sha256, manifest schema-valid, selection/launch/
args_signature carried through unchanged.
Paired TheRock multi-arch dry-run (gfx94x + gfx950) to validate build, install,
and provider tests.

Testing Checklist

AOT Python suites + coverage gate (--cov-fail-under=90) - 96 passed, 98.80%
CMake configure check (library/aot targets, install, rocm_kpack gate)
Producer-to-packer end-to-end (real rocm_kpack round-trip + manifest)
TheRock paired build/test (gfx94x + gfx950 dry-run) - in progress
GPU numeric verifier (sdpa_aot_numeric), gfx942/gfx950 - not run locally
PR CI

Technical Changes

Ports the AOT tooling into rocke/library/aot/: rocke_client_aot,
tools/rocke_aot_build.py, and shared plus family JSON schemas; family handler
to kernels/common/fmha_mfma_aot.py; instance lists to
kernels/<arch>/fmha_fwd_mfma/aot_list.json.
rocke_aot.cmake: runs under ${ROCKE_PYENV_PYTHON}; locates comgr via
find_library, then forwards the amd-llvm toolchain directories comgr's
transitive deps (libclang-cpp, libLLVM, vendored librocm_sysdeps_*) live
in, derived straight from the compiler (clang --print-resource-dir -> the
lib//bin/ and rocm_sysdeps siblings), existence filtered. The superbuild
stages comgr apart from amd-llvm so comgr's own RUNPATH cannot reach them; the
AOT tool's dlopen(libamd_comgr) needs them on its load path. Forwarding is via
LD_LIBRARY_PATH on POSIX and ROCKE_COMGR_DEP_DIRS -> os.add_dll_directory
on Windows. Whole directories are forwarded (no per-dependency resolution), so a
missing dep fails the AOT build loudly rather than degrading silently.
tools/rocke_kpack_pack.py: packs each arch's HSACO into
rocke_client_<arch>.kpack under toc_key = rocke/<op>/<family>/<name>,
verifying packed sha256 against the sidecar; emits the per-arch bundle manifest
as the single source of truth (aot_list.json is a build input, not installed).
Installs the .kpack + manifest under
${HIPDNN_RELATIVE_INSTALL_PLUGIN_ENGINE_DIR}/hip_kernel_provider/rocke/<arch>/.
AotCatalog.cpp/.hpp: point defaultArtifactRoot() at the installed
plugin-relative bundle root (path constant only; catalog stays empty).

Dependencies

TheRock bump the rock version in rocm-libraries #6427 supplies ROCKE_KPACK_PYTHON_DIR and the comgr build dep.
Packaging capture (listing the produced files in TheRock's
artifact-hipkernelprovider.toml) is handled in bump the rock version in rocm-libraries #6427.

therock-pr-bot · 2026-07-09T02:49:24Z

✅ All Checks Passed — Ready for Review

Check	Status	Details
🌿 Branch Name	✅ Pass	—
📝 PR Title/Description	✅ Pass	—
⛔ Forbidden Files	✅ Pass	—
🧪 Unit Test	✅ Pass	—
🔎 pre-commit	✅ Pass	—
🚫 Draft PR	🔜 To Be Enabled	—
🚩 Feature Flag	🔜 To Be Enabled	—
📊 Code Coverage	🔜 To Be Enabled	—
🤖 therock-pr-bot	✅ Pass	—

🎉 All checks passed! This PR is ready for review.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

therock-pr-bot · 2026-07-09T14:16:11Z

Pre-commit check failed

⛔ pre-commit failed

Please run locally:

python -m pip install pre-commit
pre-commit install
pre-commit run --all-files --show-diff-on-failure

This repo uses .pre-commit-config.yaml.

therock-pr-bot · 2026-07-09T14:24:39Z

🎉 All checks passed! This PR is ready for review.

…AOT producer+kpack # Conflicts: # dnn-providers/hip-kernel-provider/rocke/CMakeLists.txt

…-1473) Re-land the #8866 AOT build + sidecar generator onto #9159's rocke/library/ layout and extend it through kpack packaging + install. Producer + packaging only; no C++ engine/dispatcher change (AotCatalog stays empty). Leg 1 (producer): port aot tooling to library/aot/; family handler to kernels/common/fmha_mfma_aot.py (imports rewired to the kernels package); instances to kernels/<arch>/fmha_fwd_mfma/aot_list.json; schemas to library/aot/schemas/ (+ fmha_fwd_mfma/ overlay). Build tool takes --handler/--schema-dir; parse_instance_list gains handler_path. rocke_aot.cmake driven by ROCKE_PYENV_PYTHON, depends on the pyenv stamp; AOT pip deps via library/aot/requirements.txt installed into rocke-pyenv. Leg 2 (kpack): rocke_kpack_pack.py packs each HSACO under toc_key rocke/<op>/<family>/<name> into rocke_client_<arch>.kpack (verifying packed sha256 == sidecar artifact.hsaco_sha256) and emits a hipdnn.rocke.bundle/v1 manifest (bundle.schema.json) carrying selection/launch/args_signature through from the sidecars. rocke_client_aot_kpack target; rocke_client_aot_artifacts depends on it. Install .kpack + manifest + aot_list.json under HIPDNN_RELATIVE_INSTALL_PLUGIN_ENGINE_DIR/hip_kernel_provider/rocke/<arch>/. rocm_kpack from ROCKE_KPACK_PYTHON_DIR with a configure-time import gate. Tests: ported instance/schema/sidecar/sdpa suites + kpack round-trip and manifest-validation tests; coverage gate --fail-under=90 held.

defaultArtifactRoot() was a placeholder (share/hipdnn_plugins/rocke-client/aot) that predated the kpack install layout. Point it at the real plugin-relative root, <plugin_dir>/hip_kernel_provider/rocke/<arch>/, where the kpack packaging installs each arch's rocke_client_<arch>.kpack + .json sidecar (and future kernel-selection heuristics). Still a path constant only -- loadDefault() remains empty (Phase 1); the dispatcher-wiring work consumes it. Comments updated to the confirmed install form.

…k-1471-rocke-aot-producer-kpack # Conflicts: # dnn-providers/hip-kernel-provider/rocke/CMakeLists.txt

The AOT build dlopen's libamd_comgr, whose dependency closure (libclang-cpp, libLLVM, vendored rocm_sysdeps libs like librocm_sysdeps_z) lives in the compiler toolchain, not next to comgr: at build time comgr is a BUILD_DEP staged alone, so its $ORIGIN rpath siblings are absent. Chasing individual deps was fragile and broke whenever comgr gained a dependency. Forward the whole toolchain lib set instead, derived once from the compiler resource dir, probing both layouts (POSIX lib/, Windows bin/) and keeping whichever exist: LLVM/clang libs, rocm_sysdeps, comgr-adjacent libs, toolchain bin. On POSIX these go on LD_LIBRARY_PATH; on Windows they are forwarded in ROCKE_COMGR_DEP_DIRS and the comgr loader registers each via os.add_dll_directory (PATH is not a reliable dependent-DLL search path under Python's secure DLL loading). New comgr deps land in these same dirs, so this stops breaking as comgr evolves.

Replace the hand-listed toolchain lib dirs with CMake's runtime-dependency resolver (file(GET_RUNTIME_DEPENDENCIES), the machinery behind install(RUNTIME_DEPENDENCY_SET)). It reads libamd_comgr's actual dependency closure and we forward exactly the directories it resolves, so the set tracks comgr's real deps instead of guessing, and anything unresolved is reported at configure rather than surfacing as a runtime OSError. The amd-llvm toolchain (comgr's declared RUNTIME_DEP), located via the compiler resource dir, is only the resolver's search hint: comgr and its deps are staged separately at build time, so there is no in-tree rpath to read. The resolver is guarded on CMAKE_OBJDUMP and falls back to forwarding the search roots if unavailable, so it can never harden into a configure failure. Forwarding seam is unchanged: LD_LIBRARY_PATH on POSIX, ROCKE_COMGR_DEP_DIRS -> os.add_dll_directory on Windows.

…RUNTIME_DEPENDENCIES file(GET_RUNTIME_DEPENDENCIES) is documented as an install-time command, not for project mode, so using it at configure required manually wiring CMAKE_GET_RUNTIME_DEPENDENCIES_PLATFORM/TOOL/COMMAND from CMAKE_OBJDUMP. Worse, it still had to be handed the amd-llvm toolchain dirs as DIRECTORIES search hints (comgr's RUNPATH cannot reach its split-staged deps), so it never removed the derivation it was meant to replace, and it carried a silent fallback that would forward the raw search dirs whenever the resolver was unavailable. Since the env helper forwards whole directories (not individual libraries), the resolver's per-file precision was unused at runtime. Derive the two directories comgr's deps actually live in straight from the compiler: the amd-llvm lib/bin dir (from clang --print-resource-dir) and its rocm_sysdeps sibling, existence filtered. No objdump, no platform vars, no fallback. If comgr ever needs a library outside these dirs the AOT build fails loudly at that point instead of degrading silently. Forwarding seam unchanged (LD_LIBRARY_PATH on POSIX, ROCKE_COMGR_DEP_DIRS -> os.add_dll_directory on Windows).

… forwarding comgr's own RUNPATH ($ORIGIN, $ORIGIN/llvm/lib, $ORIGIN/rocm_sysdeps/lib) already resolves its full dependency closure (libLLVM, libclang-cpp, the vendored rocm_sysdeps libs) when comgr is loaded from an assembled tree, as it is under /opt/rocm. The forwarding machinery only existed because we resolved comgr from its isolated superbuild stage, where those $ORIGIN-relative siblings are absent. With TheRock now declaring amd-comgr as a build dep of hipkernelprovider, comgr's assembled dist (comgr + amd-llvm + rocm_sysdeps) is on CMAKE_PREFIX_PATH, so we resolve it via find_package(amd_comgr) and let its RUNPATH do the rest. This deletes the toolchain dep-dir derivation, the env helper's LD_LIBRARY_PATH / ROCKE_COMGR_DEP_DIRS wiring, and comgr.py's _register_dep_dirs(). comgr's own directory is still added on Windows (co-located deps), and ROCKE_COMGR_LIB still overrides discovery.

The rocKE per-arch AOT bundles installed to <plugin_dir>/hip_kernel_provider/rocke/<arch>/ which created an engines/hip_kernel_provider/ *directory* next to the hip_kernel_provider plugin file. hipDNN's plugin loader (and the integration harness, which passes engines/hip_kernel_provider in ABSOLUTE mode) resolves that base path: with the directory present, is_directory() is true so it scans the dir for *.so/*.dll, finds none, and loads no plugin -> every graph returns GRAPH_NOT_SUPPORTED. Only reproduces with rocKE enabled. Install under a generic per-arch container instead: <plugin_dir>/arch_content/<engine>/<arch>/ so rocKE is arch_content/rocke/<arch>/ and future engines (aiter, asm, ...) get sibling subdirs with no split-handler changes. arch_content is not a plugin base name, so it never shadows a plugin file. AotCatalog::defaultArtifactRoot() points at arch_content/rocke. JIRA ID : AICK-1471

DarylHawkinsAMD · 2026-07-10T16:56:51Z

@@ -0,0 +1,84 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "hipdnn.rocke.bundle/v1",


Should this be rocke.aot.bundle to follow the format of the other schema ID's

DarylHawkinsAMD · 2026-07-10T17:08:07Z

Is this intended to be committed?

DarylHawkinsAMD · 2026-07-10T17:08:47Z

+from rocke_client_aot.instance_schema import AOT_LIST_FILENAME
+from rocke_client_aot.json_schema import load_json_schema, validate_json_schema
+
+BUNDLE_SCHEMA = "hipdnn.rocke.bundle/v1"


This will also need to change if we align it with the other schema ID's

…k-1471-rocke-aot-producer-kpack

github-actions Bot added the project: hip-kernel-provider label Jul 8, 2026

assistant-librarian Bot added the organization: ROCm label Jul 8, 2026

BrianHarrisonAMD force-pushed the users/bharriso/aick-1471-rocke-aot-producer-kpack branch 4 times, most recently from 3bb81b5 to 8ecbe46 Compare July 9, 2026 01:50

Base automatically changed from users/bharriso/aick-1471-rocke-dispatcher-wiring to develop July 9, 2026 02:45

BrianHarrisonAMD force-pushed the users/bharriso/aick-1471-rocke-aot-producer-kpack branch 3 times, most recently from 80f2b23 to 5d91b29 Compare July 9, 2026 14:16

BrianHarrisonAMD force-pushed the users/bharriso/aick-1471-rocke-aot-producer-kpack branch from 5d91b29 to 67749be Compare July 9, 2026 18:43

BrianHarrisonAMD added a commit that referenced this pull request Jul 9, 2026

dry-run: merge #9115 (enable rocKE build in provider mode) into #9207 …

e51fb83

…AOT producer+kpack # Conflicts: # dnn-providers/hip-kernel-provider/rocke/CMakeLists.txt

BrianHarrisonAMD mentioned this pull request Jul 9, 2026

feat(hipkernelprovider): rocKE AOT kpack packaging & arch split ROCm/TheRock#6427

Draft

5 tasks

BrianHarrisonAMD added 2 commits July 9, 2026 16:06

BrianHarrisonAMD force-pushed the users/bharriso/aick-1471-rocke-aot-producer-kpack branch from 893eaae to 9cb5beb Compare July 9, 2026 20:06

BrianHarrisonAMD changed the title ~~feat(hip-kernel-provider): rocKE AOT producer + kpack packaging~~ feat(hip-kernel-provider): rocKE AOT producer + kpack packaging (AICK-1473) Jul 9, 2026

BrianHarrisonAMD mentioned this pull request Jul 9, 2026

feat(kpack): add hipkernelprovider per-arch split handler (AICK-1616) ROCm/rocm-systems#8347

Open

BrianHarrisonAMD added 6 commits July 9, 2026 16:30

Merge remote-tracking branch 'origin/develop' into users/bharriso/aic…

2a7ded0

…k-1471-rocke-aot-producer-kpack # Conflicts: # dnn-providers/hip-kernel-provider/rocke/CMakeLists.txt

BrianHarrisonAMD force-pushed the users/bharriso/aick-1471-rocke-aot-producer-kpack branch from 472753a to 2d02fa6 Compare July 10, 2026 15:32

DarylHawkinsAMD reviewed Jul 10, 2026

View reviewed changes

Comment thread dnn-providers/hip-kernel-provider/rocke/.coverage

DarylHawkinsAMD Jul 10, 2026

Copy link
Copy Markdown

Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this intended to be committed?

DarylHawkinsAMD reviewed Jul 10, 2026

View reviewed changes

Merge remote-tracking branch 'origin/develop' into users/bharriso/aic…

669e18e

…k-1471-rocke-aot-producer-kpack

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(hip-kernel-provider): rocKE AOT producer + kpack packaging (AICK-1473)#9207

feat(hip-kernel-provider): rocKE AOT producer + kpack packaging (AICK-1473)#9207
BrianHarrisonAMD wants to merge 9 commits into
developfrom
users/bharriso/aick-1471-rocke-aot-producer-kpack

BrianHarrisonAMD commented Jul 8, 2026 •

edited

Loading

Uh oh!

therock-pr-bot Bot commented Jul 9, 2026 •

edited

Loading

Uh oh!

therock-pr-bot Bot commented Jul 9, 2026

Uh oh!

therock-pr-bot Bot commented Jul 9, 2026

Uh oh!

DarylHawkinsAMD Jul 10, 2026

Uh oh!

DarylHawkinsAMD Jul 10, 2026

Uh oh!

DarylHawkinsAMD Jul 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

BrianHarrisonAMD commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Install layout

Cross-repo sequence

Risk Assessment

Testing Summary

Testing Checklist

Technical Changes

Dependencies

Uh oh!

therock-pr-bot Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ All Checks Passed — Ready for Review

Uh oh!

therock-pr-bot Bot commented Jul 9, 2026

Pre-commit check failed

Uh oh!

therock-pr-bot Bot commented Jul 9, 2026

Uh oh!

DarylHawkinsAMD Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

DarylHawkinsAMD Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

DarylHawkinsAMD Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

BrianHarrisonAMD commented Jul 8, 2026 •

edited

Loading

therock-pr-bot Bot commented Jul 9, 2026 •

edited

Loading