RFC-0048: RFC-0048: Linux Unwinding Abstraction Layer for traced_perf and heapprofd

[github-actions[bot]]

📄 RFC Doc: 0048-backend-agnostic-unwind-interface.md

---

RFC-0048: Linux Unwinding Abstraction Layer for traced_perf and heapprofd

Authors: @safayat-google
Status: In Review
Discussion: https://github.com/google/perfetto/discussions/7283\
Proof of concept: patch

---

1. Problem Statement & Motivation

Currently, callstack unwinding in Perfetto (traced_perf and heapprofd) is tightly coupled to Android's libunwindstack library (buildtools/android-unwinding).

On Android libunwindstack is essential due to its support for ART Dex files, ART JIT debug frames (JitDebug), and Bionic integration. However, for standalone Linux builds, coupling to libunwindstack introduces several significant drawbacks:

1. Build Fragility & Out-of-Tree Maintenance: Upstream API changes in libunwindstack frequently cause build breakages in Perfetto. Maintaining a standalone copy of libunwindstack for Linux host/target builds is costly.
2. Inability to Leverage Linux-Native Tooling: Upstream Linux profiling tools (e.g. Linux perf) rely on libunwind or libdw, which are standard across Linux distributions, actively maintained by the Linux toolchain community, and well-integrated with distribution debug packages and remote unwinding.
3. Tight Architectural Coupling: Core profiling components like traced_perf directly include <unwindstack/*.h>.

Scope

• Android: Continues to use libunwindstack by default. Full support for ART Dex files and JIT frames is preserved without regression.
• Standalone Linux: Decoupled from libunwindstack. The abstraction allows Linux builds to use alternative unwinding backends without depending on Android-specific libraries.

---

2. Proposed Unwinding Libraries for Standalone Builds

For standalone Linux builds, we propose two complementary unwinding options:

2.1 Primary DWARF Unwinder: libunwind

• It is the standard unwinding engine used by Linux perf and GNU tools. It provides a stable C API (unw_create_addr_space, unw_init_remote, unw_step), has an MIT/LLVM-compatible license, and supports remote unwinding across all standard Linux architectures (x86_64, ARM64, ARM32, RISC-V).

2.2 Built-in Zero-Dependency Frame Pointer Unwinder: FramePointerUnwinder

• It is a native, self-contained implementation with zero external library dependencies. It traverses frame pointer chains directly on the stack buffer, providing ultralow-overhead unwinding without parsing DWARF.

---

3. Proposed Interface Types

We introduce a high-level, backend-agnostic unwinding interface that encapsulates the entire unwinding and process-context lifecycle.

┌────────────────────────────────────────────────────────────────────────┐
│                        Sampling Producer                               │
│        traced_perf (CPU sampling) / heapprofd (Memory profiling)       │
└───────────────────────────────────┬────────────────────────────────────┘
                                    │ UnwindInputSample (regs + stack)
                                    ▼
┌────────────────────────────────────────────────────────────────────────┐
│               Common Unwinding Interface (unwind::Unwinder)            │
│                                                                        │
│   - CreateProcessContext(pid, maps_fd, mem_fd) -> ProcessUnwindContext │
│   - Unwind(sample, proc_ctx, options)          -> UnwindResult         │
│   - SupportsLiveSymbolization()                -> bool                 │
└───────────────────┬────────────────────────────────┬───────────────────┘
                    │                                │
        ┌───────────┴───────────┐        ┌───────────┴───────────┐
        ▼                       ▼        ▼                       ▼
┌──────────────────┐  ┌──────────────────┐  ┌──────────────────┐
│ libunwindstack   │  │ libunwind        │  │ frame_pointer    │
│ (Android + ART)  │  │ (Linux DWARF)    │  │ (Pure C++, 0 dep)│
└──────────────────┘  └──────────────────┘  └──────────────────┘

3.1 CpuRegisters (Zero-Allocation Register Representation)

Previously, traced_perf allocated std::unique_ptr<unwindstack::Regs> on the heap for every sample in the queue. In cpu_registers.h, this is replaced by a flat, stack-allocated POD struct:

struct CpuRegisters {
  CpuArch arch = CpuArch::kUnknown;
  uint64_t pc = 0;
  uint64_t sp = 0;
  uint64_t regs[64] = {};  // Flat register array covering all architectures

  bool is_valid() const { return arch != CpuArch::kUnknown && pc != 0; }
  void Reset() { arch = CpuArch::kUnknown; pc = 0; sp = 0; }
};

• Carried by value in sample queues without dynamic heap allocations.
• Architecture-aware mappings for different platforms.

3.2 Unwinder & ProcessUnwindContext

// src/profiling/unwind/unwinder.h
// Encapsulates per-process resources (procfs fds, memory maps, DWARF tables).
class ProcessUnwindContext {
 public:
  virtual ~ProcessUnwindContext() = default;
  virtual pid_t pid() const = 0;
  virtual void ReparseMaps() = 0;
  virtual base::StringView GetBuildId(uint64_t pc) = 0;
  virtual std::optional<UnwindFrame> BuildFrameFromPc(uint64_t pc) = 0;
};

// Abstract unwinder engine.
class Unwinder {
 public:
  virtual ~Unwinder() = default;
  virtual const char* Name() const = 0;

  virtual std::unique_ptr<ProcessUnwindContext> CreateProcessContext(
      pid_t pid,
      base::ScopedFile maps_fd,
      base::ScopedFile mem_fd) = 0;

  virtual UnwindResult Unwind(
      const UnwindInputSample& sample,
      ProcessUnwindContext* context,
      const UnwindOptions& options = {}) = 0;

  virtual bool SupportsLiveSymbolization() const { return false; }

  // Session lifecycle & maintenance hooks: backends manage their own resources/state
  virtual void OnTracingSessionStarted() {}
  virtual void OnTracingSessionEnded() {}
  virtual void OnPeriodicCleanup() {}
};

3.3 Output Types (UnwindFrame & UnwindResult)

// src/profiling/unwind/unwind_types.h
enum class SymbolStatus : uint8_t {
  kNotAttempted = 0,  // Symbolization was not requested or not supported by backend
  kResolved = 1,      // function_name was resolved
  kFailed = 2,        // Symbolization was requested and attempted, but not found in ELF/symtab
};

struct UnwindFrame {
  uint64_t pc = 0;
  uint64_t rel_pc = 0;
  uint64_t sp = 0;

  // Module / Mapping information (zero-copy string views)
  base::StringView map_name;
  uint64_t map_start = 0;
  uint64_t map_end = 0;
  uint64_t map_offset = 0;
  uint64_t map_elf_start_offset = 0;
  uint64_t map_load_bias = 0;
  base::StringView build_id;

  // Symbol resolution
  base::StringView function_name;
  uint64_t function_offset = 0;
  SymbolStatus symbol_status = SymbolStatus::kNotAttempted;
};

struct UnwindResult {
  UnwindErrorCode error_code = UnwindErrorCode::kNone;
  uint64_t warnings = 0;
  std::vector<UnwindFrame> frames;

  bool success() const { return error_code == UnwindErrorCode::kNone; }
};

---

4. Other Considerations

4.1 Making "Live" Symbolization Optional

• traced_perf: Does not perform live symbolization during sampling. It emits raw PCs and mapping information (build_id, rel_pc), leaving symbolization to Trace Processor offline.
• heapprofd: Optionally performs live symbolization when using libunwindstack.

4.2 Handling Library Divergences (File Opening & DWARF Discovery)

Unwinding libraries differ in how they locate DWARF unwind tables and access process memory:

• libunwindstack opens files, parses /proc/<pid>/maps, and loads DWARF internally via its own Memory and Elf classes.
• libunwind uses callback hooks (access_mem, access_reg) and unw_create_addr_space operating over stack snapshots and /proc/<pid>/mem.
• frame_pointer requires zero ELF or DWARF discovery, operating purely on the stack buffer.

These differences are completely encapsulated within each backend's ProcessUnwindContext implementation. The caller only provides the sampled stack buffer and register state to unwind.

4.3 Dwarf table Caching on abstract API

• Some backend supports dwarf caching and other does not. So, instead of polluting the api, we decided to leave it to the backends to implement their own dwarf caching policy. However Tracing lifecycle and periodic clear callback are provided for the backend to manage their caches.

---

5. Proof of Concept & Other References

• Proof of concept patch (Unwind Abstraction & Backends):
  • [1] - Working implementation of the abstraction layer, libunwindstack migration, zero-dependency frame_pointer, and libunwind backend.
  • [2] – Standalone microbenchmark harnesses for traced_perf and heapprofd.
  • [3] - Add ons + other unwinder backend
  • [4] - Benchmark numbers (micro-benchmarks, live CPU profiling, memory stress tests), tool comparison matrix, and alternative backend evaluation.

---

💬 Discussion Guidelines:

• This discussion is automatically synced with the RFC document
• Please provide constructive feedback and suggestions

[rsavitski]

The overall proposal is sensible but the phrasing of the "problem" section could be clearer. Currently it reads as if we're trying to remove libunwindstack dependency on all platforms, which we're not.

We want to add an abstraction layer to:

• decouple standalone linux builds from libunwindstack, as the out-of-tree libunwindstack dependency is unmaintainable as mentioned already.
• allow the use of alternative unwinding libraries (such as libunwind), starting with the standalone linux builds, but maybe even on android in the future.
• (iirc a bonus request) potentially allow switching unwinding libraries at runtime. I'd argue not in scope for the initial RFC.

The mentions of memory optimisations are also irrelevant to this proposal imo. The scope is purely improving maintainability and allowing for a choice of the unwinder/symboliser.

The RFC proposes modeling all headers and classes after the existing use of libunwindstack, but it's unclear how well that would fit other unwinding libraries. It'd be helpful if the RFC included a sketch of how e.g. libunwind's remote unwinding API [1] could be abstracted over by the proposed classes. Figuring out the abstraction layer is the most important part of this RFC since if we over-specialise on libunwindstack, it might make adding future unwinders more difficult.

Misc notes on the existing usage section:

• JitDebug and DexFiles methods are specific to ART processes on android, we don't need any analogues.
• Afaik we make no use of CreateProcessMemoryCached & CreateProcessMemoryThreadCached

[1] https://www.nongnu.org/libunwind/man/unw_create_addr_space(3).html

[github-actions[bot]]

📝 RFC Document Updated

View changes: Commit History

[safayat-google]

Thanks for the feedback @rsavitski! I have revised the RFC :

1. Scope: Made it explicit that libunwindstack remains the default on Android (preserving full ART Dex/JIT support). The goal is decoupling standalone Linux builds and providing a clean plug-in point for Linux-native unwinding engines like libunwind.
2. Abstraction Layer: Rather than duplicating libunwindstack classes (Regs, Memory, Maps), we introduced a high-level Unwinder and ProcessUnwindContext interface, along with a flat, zero-alloc CpuRegisters representation (uint64_t regs[64]) that decouples the unwinding queue from backend-specific register classes.
3. libunwind Remote Unwinding Design: Included the design for libunwind's remote unwinding API using unw_create_addr_space and custom accessors (access_mem reading from sampled stack memory + /proc/<pid>/mem, and access_reg reading from CpuRegisters).
4. Benchmarks & Validation: Implemented and benchmarked the abstraction on Linux x86_64:
   • DWARF unwinding is 2.6%–6.3% faster than the baseline (due to stack-allocated stack overlay memory and in-place register caching). But frame pointer unwinding add 400ns, which still have scope to improve.
   • Peak RSS is reduced by ~16%.
     I have updated the RFC doc accordingly: 0048-unwinding-without-libunwindstack.md. Looking forward to any further thoughts!

I will get the numbers with the libunwind and some alternative library to make sure the design allows to migrate to other libraries without major changes on the abstraction layer.

[github-actions[bot]]

📝 RFC Document Updated

View changes: Commit History

[github-actions[bot]]

📝 RFC Document Updated

View changes: Commit History

[github-actions[bot]]

📝 RFC Document Updated

View changes: Commit History

[github-actions[bot]]

📝 RFC Document Updated

View changes: Commit History

[rsavitski]

Focusing on section 3, one small question, but otherwise the proposal
looks good enough to figure out the rest in code reviews.

• Since only libunwindstack will be doing "live" symbolisation, are other
  libraries going to leave function_name empty? How do we distinguish failed
  "live" symbolisation from no symbolisation?

PS. Personal opinion on the doc structure: it doesn't focus enough on the part
that matters (the proposed abstraction interfaces), and has too many unnecessary
details (e.g. hypothetical addon caches, benchmarks and tool comparisons), as
well as weird implementation-level fragments (e.g. going out of the way to
comment that some type/operation doesn't require heap or a syscall).

Structure I would've expected:

• problem statement.
• proposed unwinding library for standalone builds.
• proposed new interface types to support both libraries.
• other considerations (e.g. how to best make "live" symbolisation optional, or where libraries diverge significantly in what they offer (e.g. automatically opening the files and locating the relevant DWARF info).
• link to a proof of concept patch that implements minimal libunwindstack &
  <library_of_choice> support using said types, and no more (i.e. no extra
  caches or nice to haves).

In other words, sections 1, 2, 3 would've been enough. The rest feels like it
distracts from the primary focus of the RFC. But no need to restructure it now,
let's focus on the real implementation and hammer out the rest in reviews.

[github-actions[bot]]

📝 RFC Document Updated

View changes: Commit History

[safayat-google]

Thanks Ryan for the feedback.

• The doc is updated with minimal details to focus on the abstraction layer only
• links added for the proof of concept
• Add-ons (i.e. caching), benchmark and other tables are moved to references

How do we distinguish failed "live" symbolisation from no symbolisation?

Thanks for pointing this out. SymbolStatus (kNotAttempted / kFailed ) is introduced to distinguish between the two. Also the backend is responsible to report if it supports symbolisation or not. Finally client side will use UnwindOptions to specify if it wants the function names.

let's focus on the real implementation and hammer out the rest in reviews.

Thanks, I am preparing the pull requests