Fix silent dictation: copy tap buffers, serialize, surface permission errors#18
Merged
Merged
Conversation
… errors The tap callback was spawning unstructured Tasks to forward each AVAudioPCMBuffer to SlidingWindowAsrManager.streamAudio. Two bugs: the engine reuses the tap buffer's underlying memory once the callback returns, so by the time the Task runs the samples may be stale or zeroed; and Task ordering is non-deterministic, so even when buffers do survive they can arrive at the ASR out of order. Deep-copy the buffer synchronously in the tap, then funnel through an AsyncStream with a single consumer task — preserves ordering and lifetime. Also surface the two silent-fail modes upfront: throw microphoneDenied / accessibilityDenied from start() instead of spinning the HUD over an engine that never delivers buffers or an injector that drops every keystroke. The AX case routes to the existing axLostBanner so the user gets the Re-grant Access button. Adds buffer-count logging so future regressions where the mic stops delivering audio show up in Console.
AXIsProcessTrusted() can return stale 'denied' results right after a grant (especially under ad-hoc signing), so the upfront check was flagging users whose permission was actually granted. The existing startAXPolling() in AppModel already catches genuine AX revocation within 2 seconds; lean on that instead of failing the start path on a check we can't trust. Keep the upfront mic check — mic denial silently delivers zero buffers forever, with no polling to catch it.
Previously both AVAudioEngine instances (DictationManager, RecordingManager) used the system default input device with no way to override. Users with multiple mics (e.g. AirPods + studio mic + built-in) had to flip the macOS system default whenever they wanted to switch. Adds: - AudioInputDevices: CoreAudio enumeration of input-capable devices by UID (persistent across launches), plus an apply(uid:to:engine:) helper that sets kAudioOutputUnitProperty_CurrentDevice on the input audio unit BEFORE the engine's input format is queried. - AppSettings.selectedInputDeviceUID (nil = follow system default), persisted in UserDefaults. - Microphone section in Settings → General with a Picker. Lists all current input devices plus a "System Default (Device Name)" option. If the saved UID is currently unplugged the picker keeps it as an "Unavailable" entry so the preference isn't silently lost on next plug-in. - Propagation hooks: bootstrap, toggleDictation, onMeetingStarted, and startManualRecording all sync the manager's inputDeviceUID from settings before the engine starts.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The tap callback was spawning unstructured Tasks to forward each
AVAudioPCMBuffer to SlidingWindowAsrManager.streamAudio. Two bugs:
the engine reuses the tap buffer's underlying memory once the
callback returns, so by the time the Task runs the samples may be
stale or zeroed; and Task ordering is non-deterministic, so even
when buffers do survive they can arrive at the ASR out of order.
Deep-copy the buffer synchronously in the tap, then funnel through
an AsyncStream with a single consumer task — preserves ordering and
lifetime.
Also surface the two silent-fail modes upfront: throw
microphoneDenied / accessibilityDenied from start() instead of
spinning the HUD over an engine that never delivers buffers or an
injector that drops every keystroke. The AX case routes to the
existing axLostBanner so the user gets the Re-grant Access button.
Adds buffer-count logging so future regressions where the mic stops
delivering audio show up in Console.