LD_PRELOAD causes WSL crash due to recursive dlopen in libd3d12.so

[instant0]

Windows build number:

10.0.26100.7628

Your Distribution version:

24.04.4 LTS

Your WSL versions:

WSL version: 2.6.3.0
Kernel version: 6.6.87.2-1
WSLg version: 1.0.71
MSRDC version: 1.2.6353
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.26100.1-240331-1435.ge-release
Windows version: 10.0.26100.7628

WSLInterop is enabled

Steps to reproduce:

Compile any shared library that intercepts D3D12 functions using LD_PRELOAD:

# Example minimal hook (just logs and passes through)
g++ -shared -fPIC -o myhook.so myhook.cpp -ldl
LD_PRELOAD=./myhook.so vulkaninfo

2. Observe WSL crash

Our Minimal Test Case

We created a transparent pass-through hook that only logs and forwards all calls unchanged:

// This transparent hook STILL crashes WSL
extern "C" HRESULT D3D12CreateDevice(...) {
    // Just log and call real function
    return real_D3D12CreateDevice(...);
}

Even with zero modification to the D3D12 calls, WSL crashes.

WSL logs:

No response

WSL dumps:

No response

Expected behavior:

LD_PRELOAD hooks should work (or at minimum, not crash WSL). Standard Linux debugging/profiling tools that use LD_PRELOAD should function.

Actual behavior:

WSL crashes with segfault inside libd3d12.so. The crash is caused by infinite recursion in D3D12GetInterface.

Stack Trace (from ASAN - similar pattern with LD_PRELOAD)

#0  D3D12GetInterface (/usr/lib/wsl/lib/libd3d12.so+0x467de)
#1  ...libd3d12.so (calls D3D12GetInterface again)
#2  ...libd3d12.so (recursive!)
#3  D3D12GetInterface (recursive!)
... repeats infinitely until stack overflow ...

Root Cause Analysis

The dzn driver (Mesa's Direct3D 12 driver for WSL) internally calls D3D12GetInterface which uses dlopen to load additional COM DLLs. When LD_PRELOAD is active:

1. The hook intercepts D3D12 functions
2. Even a transparent pass-through changes library loading order
3. This triggers recursive library loading inside the proprietary Windows driver
4. WSL crashes due to stack overflow

Key finding: The crash happens even with zero modification - just the presence of LD_PRELOAD changes symbol resolution, breaking the internal loading mechanism.

This is related to WSL issue #9244 (ASAN causes same crash), but LD_PRELOAD is a more general case that affects:

• Custom debugging hooks
• profiling tools (Valgrind, perf)
• Memory sanitizers
• Game mods/hooks
• Wine/DXVK when using certain configurations

Diagnostic Logs

Please run the following and capture output:

# Enable logging in our fix layer
CIV7_LAYER_LOG=1 VK_INSTANCE_LAYERS=VK_LAYER_CIV7_DUALSOURCE_BLEND_FIX vulkaninfo

# Check dmesg for errors
dmesg | tail -50

Suggested Fix Direction

While Vulkan Layers are the proper way to intercept Vulkan calls (and we have implemented a fix that way), the underlying issue is that LD_PRELOAD should not cause WSL to crash. Potential fixes:

1. Fix in libd3d12.so: Prevent recursive dlopen by caching already-loaded interfaces
2. Workaround in WSL: Detect LD_PRELOAD and warn/disable dzn gracefully
3. Documentation: Clearly document that LD_PRELOAD is incompatible with WSLg graphics stack

Our Workaround

We implemented a Vulkan Layer that intercepts vkCreateGraphicsPipelines and returns VK_ERROR_FEATURE_NOT_PRESENT for unsupported dual-source blend configurations:

• Repository: [Link to your implementation]
• Usage: VK_INSTANCE_LAYERS=VK_LAYER_CIV7_DUALSOURCE_BLEND_FIX vulkaninfo

This avoids the need for LD_PRELOAD but doesn't fix the underlying issue.

Vulkaninfo

==========
VULKANINFO

Vulkan Instance Version: 1.3.275

Instance Extensions: count = 12

    VK_EXT_debug_report                    : extension revision 10
    VK_EXT_debug_utils                     : extension revision 2
    VK_EXT_headless_surface                : extension revision 1
    VK_EXT_layer_settings                  : extension revision 2
    VK_KHR_device_group_creation           : extension revision 1
    VK_KHR_get_physical_device_properties2 : extension revision 2
    VK_KHR_get_surface_capabilities2       : extension revision 1
    VK_KHR_portability_enumeration         : extension revision 1
    VK_KHR_surface                         : extension revision 25
    VK_KHR_xcb_surface                     : extension revision 6
    VK_KHR_xlib_surface                    : extension revision 6
    VK_LUNARG_direct_driver_loading        : extension revision 1

Layers: count = 10

VK_LAYER_INTEL_nullhw (INTEL NULL HW) Vulkan version 1.1.73, layer version 1:
Layer Extensions: count = 0
Devices: count = 1
GPU id = 0 (Microsoft Direct3D12 (NVIDIA GeForce RTX 4080))
Layer-Device Extensions: count = 0

=== Graphics Driver ===
total 179004
drwxr-xr-x 1 root root 60 Mar 28 23:23 .
drwxr-xr-x 4 root root 4096 Mar 15 16:11 ..
-r-xr-xr-x 4 root root 171008 Jun 23 2025 libcuda.so
-r-xr-xr-x 4 root root 171008 Jun 23 2025 libcuda.so.1
-r-xr-xr-x 4 root root 171008 Jun 23 2025 libcuda.so.1.1
-r-xr-xr-x 2 root root 9742088 Jun 23 2025 libcudadebugger.so.1
-r-xr-xr-x 1 root root 801840 Oct 20 2023 libd3d12.so
-r-xr-xr-x 1 root root 6880344 Oct 20 2023 libd3d12core.so
-r-xr-xr-x 1 root root 942048 Apr 1 2024 libdxcore.so
-r-xr-xr-x 3 root root 19020552 Jun 23 2025 libnvcuvid.so
-r-xr-xr-x 3 root root 19020552 Jun 23 2025 libnvcuvid.so.1
-r-xr-xr-x 2 root root 153723328 Jun 23 2025 libnvdxdlkernels.so
-r-xr-xr-x 3 root root 272448 Jun 23 2025 libnvidia-encode.so
-r-xr-xr-x 3 root root 272448 Jun 23 2025 libnvidia-encode.so.1
-r-xr-xr-x 2 root root 260880 Jun 23 2025 libnvidia-ml.so.1
-r-xr-xr-x 2 root root 4578440 Jun 23 2025 libnvidia-ngx.so.1
-r-xr-xr-x 3 root root 67584 Jun 23 2025 libnvidia-opticalflow.so
-r-xr-xr-x 3 root root 67584 Jun 23 2025 libnvidia-opticalflow.so.1
-r-xr-xr-x 2 root root 72616 Jun 23 2025 libnvoptix.so.1
lrwxrwxrwx 1 root root 15 Mar 28 23:23 libnvoptix_loader.so.1 -> libnvoptix.so.1
-r-xr-xr-x 2 root root 99206488 Jun 23 2025 libnvwgf2umx.so
-r-xr-xr-x 2 root root 5047376 Jun 23 2025 nvidia-ngx-updater
-r-xr-xr-x 2 root root 823096 Jun 23 2025 nvidia-smi

[instant0]

Root Cause Analysis: LD_PRELOAD Crash in WSL

Summary

Using LD_PRELOAD with WSL's dzn (Direct3D 12) driver causes WSL to crash due to infinite recursion in D3D12GetInterface's internal dlopen calls.

Technical Details

The Crash Pattern

When any LD_PRELOAD library is active, even a transparent pass-through hook that makes zero modifications to D3D12 calls causes WSL to crash.

The crash happens inside libd3d12.so (Microsoft's D3D12 implementation for WSL):

#26 D3D12GetInterface (/usr/lib/wsl/lib/libd3d12.so+0x467de)
#27 ...libd3d12.so (calls D3D12GetInterface again)
#28 ...libd3d12.so
#29 D3D12GetInterface (RECURSIVE!)
#30 ...libd3d12.so (RECURSIVE!)
... repeats infinitely until stack overflow ...

Why This Happens

1. D3D12GetInterface uses dlopen internally: The dzn driver calls D3D12GetInterface to load additional COM DLLs that provide Direct3D functionality.

2. LD_PRELOAD changes symbol resolution: Even a transparent hook changes how the dynamic linker resolves symbols.

3. Recursive loading: The internal dlopen triggers another call to D3D12GetInterface, which calls dlopen again, creating infinite recursion.

4. Stack overflow: The recursion continues until the stack is exhausted, causing a crash.

Why ASAN Has The Same Issue

AddressSanitizer (ASAN) works by preloading its own library (libasan) via LD_PRELOAD. Even without a custom hook, ASAN's presence triggers the same recursive pattern.

This was reported in WSL issue #9244 and remains unfixed.

Impact

This affects any Linux tool that uses LD_PRELOAD:

• Custom game mods/hooks (like our Civilization 7 fix)
• Debugging tools (gdb, lldb)
• Profiling tools (Valgrind, perf)
• Memory sanitizers (ASAN, MSAN, UBSAN)
• Dynamic analysis tools (dynamic binary instrumentation)

Why It Can't Be Easily Fixed

The root cause is architectural:

1. dzn is a bridge: It's a Mesa driver that translates Vulkan/D3D12 calls to Windows DirectX via VGPU10
2. Windows COM is recursive: D3D12GetInterface internally loads COM components which may call back
3. Linux dynamic linker + Windows COM = recursion: The combination creates an unexpected loop

Fixing this would require significant changes to either:

• How dzn loads COM components
• How WSL handles LD_PRELOAD with graphics drivers

Comparison to Native Linux

On native Linux:

• Drivers don't typically use dlopen internally in a way that creates recursion
• LD_PRELOAD works as expected
• Debugging/profiling tools function normally

Workaround

Use Vulkan Layers instead of LD_PRELOAD:

• Layers are loaded by the official Vulkan loader
• No interference with internal library loading
• Proper chain building by the loader

See the main project for our Vulkan Layer implementation.

References

• WSL Issue #9244: ASAN causes similar crash
• WSLg Issue: MESA errors with dzn driver
• Mesa dzn driver: Direct3D 12 driver for WSL

test_ld_preload_crash.sh

README.md

[instant0]

Opencode Zen / Big Pickle assisted with this bug report.