Crash (0x6BA RPC_S_SERVER_UNAVAILABLE) in waveOutOpen when Windows Audio Service is unavailable — no SEH guard in Adobe AIR.dll

[bobaoapae]

Summary

Native crash (0x000006BA — RPC_S_SERVER_UNAVAILABLE) in winmmbase!waveOutOpen when the Windows Audio Service (Audiosrv) is stopped or unavailable. Adobe AIR.dll calls waveOutOpen without any SEH guard (__try/__except), so the RPC exception raised internally by waveOutOpen propagates unhandled and crashes the process.

This affects every session for users whose Windows Audio Service is disabled or broken — the crash is 100% reproducible on those machines.

Environment

• AIR SDK: 51.1.3.10 (likely affects all versions)
• Platform: Windows 11 Pro (10.0.26200), x86 (32-bit)
• Affected machines: Any Windows system where Audiosrv is not running (disabled, crashed, or broken by third-party software)

Crash Analysis

Exception Details (from 5 crash dumps, all identical)

Exception Code:    0x000006BA (RPC_S_SERVER_UNAVAILABLE)
Faulting Module:   winmmbase.dll
Function:          winmmGetNewPnpEvents → NdrClientCall2 (RPC)

Full Call Stack (faulting thread)

winmmbase!winmmGetNewPnpEvents+0x103
winmmbase!CDeviceChange::InitPnpNotifications+0x67
winmmbase!CWavePlayer::Open+0x8f
winmmbase!waveOutOpen+0x4a       ← called by Adobe AIR.dll
Adobe_AIR+0x4ac24b               ← no SEH around this call
Adobe_AIR+0x4ac1db
Adobe_AIR+0x4ac04b
Adobe_AIR+0x4ab65a
Adobe_AIR+0x4aaaa7
...

What happens inside waveOutOpen

When Audiosrv is unavailable, waveOutOpen internally calls winmmGetNewPnpEvents, which uses NdrClientCall2 to make an RPC call to the audio service endpoint. When the RPC server is unreachable, the RPC runtime raises exception 0x6BA as a native Windows exception (not a return code).

This is a known Windows behavior — waveOutOpen can throw native exceptions when the audio service is down.

Disassembly of the Call Site in Adobe AIR.dll

; Adobe AIR.dll + 0x4AC24B
call    dword ptr [0x74502A20]   ; waveOutOpen via IAT (winmm.dll)
test    eax, eax                 ; check return value
jne     0x4AC263                 ; if error → jump to error path (returns false)
; ... continues with audio init ...

; Error path at 0x4AC263:
; Returns false — AIR gracefully handles waveOutOpen failure codes

Key observation: AIR already handles waveOutOpen failure gracefully — if waveOutOpen returns an error code (like MMSYSERR_NODRIVER), AIR returns false and continues without audio. The problem is that when the audio service is down, waveOutOpen never returns — it throws a native exception before returning.

Root Cause

Adobe AIR.dll lacks an SEH guard (__try/__except) around its waveOutOpen call. Since AIR already handles error return codes properly, the only missing piece is catching the native exception that occurs when the audio service is unavailable.

Why SEH is difficult here

We attempted two SEH-based approaches in our workaround:

1. C++ __try/__except: MSVC's /EHsc flag combined with /O2 + LTCG can strip the SEH frame from C++ translation units.
2. Pure C __try/__except (separate .c file with #pragma optimize("", off)): The SEH frame was correctly emitted (verified via disassembly — _except_handler4, fs:[0] registration, cmp 6BAh all present), but the RPC runtime's own nested exception handlers inside NdrClientCall2 interfere with outer SEH handlers, preventing the exception from reaching our handler.

Suggested Fix

The simplest reliable fix is a pre-check before calling waveOutOpen:

// Check if Windows Audio Service is running before calling waveOutOpen
static BOOL IsAudioServiceRunning()
{
    SC_HANDLE scm = OpenSCManagerW(NULL, NULL, SC_MANAGER_CONNECT);
    if (!scm) return TRUE; // can't check → assume running

    SC_HANDLE svc = OpenServiceW(scm, L"Audiosrv", SERVICE_QUERY_STATUS);
    if (!svc) { CloseServiceHandle(scm); return TRUE; }

    SERVICE_STATUS_PROCESS ssp;
    DWORD needed = 0;
    BOOL running = FALSE;

    if (QueryServiceStatusEx(svc, SC_STATUS_PROCESS_INFO,
                             (LPBYTE)&ssp, sizeof(ssp), &needed))
        running = (ssp.dwCurrentState == SERVICE_RUNNING);
    else
        running = TRUE; // query failed → assume running

    CloseServiceHandle(svc);
    CloseServiceHandle(scm);
    return running;
}

// Before calling waveOutOpen:
if (!IsAudioServiceRunning())
    return MMSYSERR_NODRIVER; // AIR already handles this gracefully

This approach:

• Uses the Service Control Manager API, which does not touch any audio APIs (no RPC to Audiosrv)
• Returns MMSYSERR_NODRIVER, which AIR already handles correctly
• Avoids the exception entirely — no SEH complications

Workaround (shipping in production)

We implemented the pre-check as an IAT hook in an Adobe AIR Native Extension (ANE) DLL:

1. On DLL_PROCESS_ATTACH, find Adobe AIR.dll via GetModuleHandleA
2. Walk the PE import table to find the IAT entry for waveOutOpen (imported from winmm.dll)
3. Replace the IAT entry with a wrapper that checks Audiosrv status via QueryServiceStatusEx
4. If the service is not running, return MMSYSERR_NODRIVER immediately
5. Otherwise, call the original waveOutOpen

This workaround is confirmed working in production — the affected player can now play without crashes.

Full workaround source: AudioSafetyHook.cpp + AudioSafetyHook.h + AudioSafetyHookSEH.c

Expected Behavior

waveOutOpen failures due to an unavailable audio service should be caught and converted to an error return code (e.g., MMSYSERR_NODRIVER) rather than crashing the process.

Additional Notes

• The crash affects all audio-enabled sessions on the affected machine, not intermittently — it is 100% reproducible when Audiosrv is down
• Disabling sound/music in application settings does not help if the runtime calls waveOutOpen during its own initialization before the application has a chance to configure audio preferences
• 5 crash dumps were analyzed, all with identical stacks, confirming a single root cause

[ajwfrost]

That's quite useful info (although I should probably point out clause 8 of the AIR SDK license agreement!!) - we will look at adding the check via OpenSCManager etc - seems to all be supported across the necessary versions etc.

thanks