[comgr][hotswap] Add <kernel>.stub symbols for entry trampolines#3280
Open
lmoriche wants to merge 2 commits into
Open
[comgr][hotswap] Add <kernel>.stub symbols for entry trampolines#3280lmoriche wants to merge 2 commits into
lmoriche wants to merge 2 commits into
Conversation
lmoriche
force-pushed
the
users/lmoriche/comgr-hotswap-entry-trampoline-stub-symbols
branch
2 times, most recently
from
9fb235c to
e7d306e
Compare
lamb-j
approved these changes
Jul 9, 2026
lamb-j
left a comment
lamb-j
left a comment
Collaborator
There was a problem hiding this comment.
LGTM, should wait for @chinmaydd and @xintin to take a look though
chinmaydd
approved these changes
Jul 9, 2026
chinmaydd
left a comment
chinmaydd
left a comment
There was a problem hiding this comment.
LGTM, but I'd also like an idempotency test since we claim that as a feature
Idempotent: a second pass installs no new stubs and adds no duplicate symbols
chinmaydd
added
hotswap
Author
Done |
scchan
reviewed
Jul 9, 2026
| // Build the appended string names and symbol entries. | ||
| SmallVector<uint8_t> StrBlob, SymBlob; | ||
| for (const KernelEntryTrampolineFixup &F : Fixups) { | ||
| std::string Name = F.KernelName + ".stub"; |
There was a problem hiding this comment.
Should we use a complex suffix to further minimize the chance of symbol name collision?
Author
There was a problem hiding this comment.
Compiler generated suffixes are part of the Itanium C++ ABI, and generally considered safe as '.' is not a legal character in a C++ mangled name. AFAIK, our compiler does not use ".stub", so Hotswap can use it.
|
This requires a fix due to updated API for |
lancesix
reviewed
Jul 9, 2026
lancesix
left a comment
lancesix
left a comment
There was a problem hiding this comment.
Just minimal comments. No concern from a debugger perspective.
With entry trampolines installed, a kernel descriptor's entry now points at the appended stub instead of the kernel body. Tools that resolve a dispatch's entry address to a name -- e.g. rocgdb `info dispatches` -- then print a bare address because the stub has no symbol. Add a `<kernel_name>.stub` STT_FUNC symbol (covering the 256-byte stub) to the code object's `.symtab` for each appended stub. rocgdb/amd-dbgapi reads the non-alloc `.symtab` and now reports the dispatch as e.g. `hello[stub]`. Only the trailing non-alloc `.symtab` / `.strtab` sections grow, so no virtual addresses, program headers, or relocations change, and `.dynsym` (used by the HSA loader) is untouched -- the rewritten object still loads and runs identically. The string blob is padded so the section header table stays 8-byte aligned. Symbol addition is best-effort: on any structural problem the rewrite keeps the (correct) symbol-less object rather than failing. Applies to every entry-trampoline rewrite. Idempotent: a second pass installs no new stubs and adds no duplicate symbols. Verified that `info dispatches` shows `hello[stub]` and the kernel runs correctly through the trampoline.
… symbols Add KernelEntryTrampoline.SecondPassAddsNoDuplicateStubSymbol (HotswapMCTest.cpp), guaranteeing the idempotency claimed by the stub-symbol change: a second rewrite pass over an already-trampolined code object installs no new stub and defines no duplicate `<kernel>.stub` symbol. The test runs the full first pass on a synthetic gfx1250 object (appendKernelEntryTrampolines -> growWithTrampolines -> rewriteKernelEntryDescriptorOffsets -> addKernelEntryTrampolineSymbols) and asserts exactly one "kernel.stub" symbol. It then re-parses that output and runs appendKernelEntryTrampolines again; since the descriptor already targets the appended stub, the second pass reports zero new stubs and empty fixups, so the symbol pass never runs. Feeding the empty fixups to addKernelEntryTrampolineSymbols returns nullptr (no new buffer), and "kernel.stub" remains defined exactly once. A helper, countSymtabSymbolsNamed, counts .symtab symbols by name.
lmoriche
force-pushed
the
users/lmoriche/comgr-hotswap-entry-trampoline-stub-symbols
branch
from
2f6d3a3 to
124417e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With entry trampolines installed, a kernel descriptor's entry now points at the appended stub instead of the kernel body. Tools that resolve a dispatch's entry address to a name -- e.g. rocgdb
info dispatches-- then print a bare address because the stub has no symbol.Add a
<kernel_name>.stubSTT_FUNC symbol (covering the 256-byte stub) to the code object's.symtabfor each appended stub. rocgdb/amd-dbgapi reads the non-alloc.symtaband now reports the dispatch as e.g.hello[stub].Only the trailing non-alloc
.symtab/.strtabsections grow, so no virtual addresses, program headers, or relocations change, and.dynsym(used by the HSA loader) is untouched -- the rewritten object still loads and runs identically. The string blob is padded so the section header table stays 8-byte aligned. Symbol addition is best-effort: on any structural problem the rewrite keeps the (correct) symbol-less object rather than failing.Applies to every entry-trampoline rewrite. Idempotent: a second pass installs no new stubs and adds no duplicate symbols.
Verified that
info dispatchesshowshello[stub]and the kernel runs correctly through the trampoline.