Skip to content

Conversation

@sunilmut
Copy link
Member

Added a new feature mem-profile-tracing (disabled by default) that provides for heap memory profiling using the DHAT crate.

Steps:

  • Build OpenHCL with the feature mem-profile-tracing. Debug info is necessary to get backtraces in the tracing
    cargo xflowey build-igvm x64 --override-openvmm-hcl-feature mem-profile-tracing --with-debuginfo --release
  • Increase the VTL2 memory to account for the extra debug info/size of the OpenHCL IGVM

To collect the trace:
underhill-vm process trace:
ohcldiag-dev.exe <vmname> memory-profile-trace -n vm > vm.json

init process trace:
ohcldiag-dev.exe <vmname> memory-profile-trace -n underhill > underhill.json

Viewing the trace:
Load the trace in https://nnethercote.github.io/dh_view/dh_view.html or use the instructions from here to view the trace: https://docs.rs/dhat/latest/dhat/#viewing

Additional resources:
https://docs.rs/dhat/latest/dhat/
https://valgrind.org/docs/manual/dh-manual.html

Screenshot of a sample trace:
image

@sunilmut sunilmut requested a review from a team as a code owner February 11, 2026 19:39
Copilot AI review requested due to automatic review settings February 11, 2026 19:39
@sunilmut sunilmut requested a review from a team as a code owner February 11, 2026 19:39
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an optional (mem-profile-tracing, default-off) heap profiling capability for OpenHCL using the dhat allocator/profiler, and exposes collection via the diagnostics RPC surface and ohcldiag-dev.

Changes:

  • Introduces a mem-profile-tracing feature that swaps the global allocator to dhat::Alloc and instantiates dhat::Profiler in relevant Underhill processes.
  • Adds a new diagnostics RPC to fetch the DHAT heap profile output and wires it through server/client plumbing.
  • Adds an ohcldiag-dev memory-profile-trace command to request and write the trace output.

Reviewed changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
openhcl/underhill_entry/src/lib.rs Switches global allocator to dhat::Alloc when mem-profile-tracing is enabled.
openhcl/underhill_entry/Cargo.toml Adds mem-profile-tracing feature and optional dhat dependency.
openhcl/underhill_core/src/worker.rs Initializes a per-worker dhat::Profiler when the feature is enabled.
openhcl/underhill_core/src/lib.rs Adds diag request handling to collect/reset heap profile output (and optionally forward to worker).
openhcl/underhill_core/src/dispatch/mod.rs Adds VM RPC variant + LoadedVm profiler storage and handling.
openhcl/underhill_core/Cargo.toml Adds mem-profile-tracing feature wiring and optional dhat dependency.
openhcl/openvmm_hcl/Cargo.toml Plumbs mem-profile-tracing feature through the top-level OpenHCL crate.
openhcl/ohcldiag-dev/src/main.rs Adds CLI command to request/write the memory profile trace.
openhcl/diag_server/src/diag_service.rs Adds RPC handler that requests the trace from Underhill (or returns an error if disabled).
openhcl/diag_server/Cargo.toml Adds mem-profile-tracing feature flag to gate server-side behavior.
openhcl/diag_proto/src/diag.proto Adds new proto RPC + request/response messages for memory profile trace.
openhcl/diag_client/src/lib.rs Adds client helper to call the new memory profile trace RPC.
Cargo.toml Adds workspace dependency on dhat.
Cargo.lock Locks dhat and its transitive dependencies.

Comment on lines +893 to +894
} else {
pid.unwrap()
Copy link

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pid.unwrap() will panic if the user forgets to pass either --pid or --name. CoreDump already uses a clap ArgGroup to make one of these required; MemoryProfileTrace should do the same (or handle the None case with a proper error).

Suggested change
} else {
pid.unwrap()
} else if let Some(pid) = pid {
pid
} else {
anyhow::bail!("either --pid or --name must be specified")

Copilot uses AI. Check for mistakes.
Comment on lines 19 to 33
// Older methods.
service UnderhillDiag {
rpc Exec(ExecRequest) returns (ExecResponse);
rpc Wait(WaitRequest) returns (WaitResponse);
rpc Start(StartRequest) returns (google.protobuf.Empty);
rpc Crash(CrashRequest) returns (google.protobuf.Empty);
rpc Kmsg(KmsgRequest) returns (google.protobuf.Empty);
rpc Restart(google.protobuf.Empty) returns (google.protobuf.Empty);
rpc Pause(google.protobuf.Empty) returns (google.protobuf.Empty);
rpc Resume(google.protobuf.Empty) returns (google.protobuf.Empty);
rpc ReadFile(FileRequest) returns (google.protobuf.Empty);
rpc DumpSavedState(google.protobuf.Empty) returns (DumpSavedStateResponse);
rpc PacketCapture(NetworkPacketCaptureRequest) returns (NetworkPacketCaptureResponse);
rpc MemoryProfileTrace(MemoryProfileTraceRequest) returns (MemoryProfileTraceResponse);
}
Copy link

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

diag.proto has a comment stating that new methods should be added to OpenhclDiag because the older diagnostics service didn't handle unknown methods correctly, but this adds MemoryProfileTrace to UnderhillDiag. To preserve compatibility with older servers/clients, consider moving this RPC to OpenhclDiag (and updating client/server dispatch accordingly).

Copilot uses AI. Check for mistakes.
#[cfg(not(feature = "mem-profile-tracing"))]
{
let _ = request;
anyhow::bail!("Profiler tracing feature disabled")
Copy link

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When mem-profile-tracing is disabled, this returns "Profiler tracing feature disabled". Consider mentioning the exact build feature name (mem-profile-tracing) and/or how to enable it so the error is actionable.

Suggested change
anyhow::bail!("Profiler tracing feature disabled")
anyhow::bail!(
"Profiler tracing feature disabled: rebuild with the `mem-profile-tracing` feature enabled"
)

Copilot uses AI. Check for mistakes.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant