Skip to content

Commit 104b581

Browse files
execsumoclaude
andcommitted
Robustness and efficiency hardening pass
- Gate DebugFileLog behind Developer Mode, rotate at 5 MB, and stop logging transcript content (lengths only) — previously every dictated utterance was written to an unbounded plaintext log in every install - Move meeting-start title/roster AX scraping and the 15 s roster poll off the main thread, with a 1 s AXUIElementSetMessagingTimeout so a busy Teams can't stall the app - Generalize app-audio process collection per meeting app (MeetingApp.isProcessFamilyMember) so Zoom/Webex helpers are tapped, not just Teams - skipNaming: use processingJob (not activeJob) so a stale failed job can't pin the phase at .processing - Remove dead partialTranscript plumbing and its 10 Hz polling loop - Cap the dictation buffer at 4 hours (matches recording max duration) - Free pipeline sample buffers after their last consuming stage (mic after transcription, app after diarization) to cut peak memory - Validate the cached System Audio TCC grant once per launch so a revoked permission no longer shows "Granted" forever - Track skipped review findings in ROADMAP.md Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 400ff45 commit 104b581

11 files changed

Lines changed: 214 additions & 83 deletions

ROADMAP.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,9 @@ To preserve the focus, performance, and simplicity of Heard as a lean, native, s
99
These represent the remaining areas of focus to clean up and stabilize:
1010

1111
### Active Tech Debt & Known Issues
12+
- **Watchdog abort can leak a zombie pipeline task.** `PipelineProcessor.abortAndFailCurrentJob` cancels the pipeline task and starts the next job, but FluidAudio may not honor cancellation promptly. A stuck transcribe/diarize call that eventually returns will write into the shared per-job state (`appTrack`, `appTranscription`, `appDiarization`, `transcriptionProgress`) while the next job is using it. Fix: tag each run with a generation/job ID and have stage write-backs verify they still own the current generation before mutating shared state.
13+
- **App-audio self-test rebuild invalidates `micDelaySeconds`.** `attemptAppAudioRebuild` truncates and reopens the app WAV ~2–4 s after the original start and updates `appStartTime`, but the session's `micDelaySeconds` was computed once in `startRecording` and never recalculated. After a rebuild, mic/app track alignment is off by the rebuild delay, which skews `SegmentDeduplicator.dropMicBleed` and segment interleaving. Fix: recompute the delay (or store a rebuild offset on the session) after a successful rebuild.
14+
- **PermissionCenter republishes identical state every 3 s.** `refresh()` builds a fresh `statuses` array on every poll tick, so every observing view re-renders 20×/minute for the app's lifetime even when nothing changed — plus a system-wide AX IPC call per tick while Accessibility is ungranted. Fix: make `PermissionStatus` equatable and only assign when the array actually changed. Related: the detector polls at 1 s while the docs (CLAUDE.md/handoff.md) say 3 s — reconcile the cadence or the docs.
1215
- **In-meeting note editing.** Today the user edits notes by opening the rendered `.md` directly. A future polish: a "Notes" disclosure on each completed job in the menu bar dropdown that lists captured notes and lets the user edit/delete before the transcript is finalized (or rewrite the `.md` if it's already been written).
1316
- **`Views.swift` size.** All SwiftUI UI components live in a single ~1.9 kLOC file after the Paper design system was implemented. Split this by tab or view area to improve build times and maintenance once early UI iteration is finished.
1417
- **Menu bar dropdown height clipping.** The menu bar dropdown uses `.window` style and has a fixed max height. The jobs list can clip when many jobs accumulate.

Sources/HeardCore/AppModel.swift

Lines changed: 8 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,6 @@ public final class AppModel: ObservableObject {
1616

1717
// Dictation state (separate from phase — can coexist with meeting recording)
1818
@Published public var isDictating = false
19-
@Published public var partialTranscript = ""
2019
@Published public var dictationError: String?
2120
/// Set when AX permission is revoked while dictation is active. Cleared on dismiss or next start.
2221
@Published public var dictationAXLost = false
@@ -50,11 +49,13 @@ public final class AppModel: ObservableObject {
5049
public static func bootstrap() -> AppModel {
5150
try? FileManager.default.ensureHeardDirectories()
5251

52+
let settingsStore = SettingsStore()
53+
DebugFileLog.isEnabled = settingsStore.settings.developerMode
54+
5355
let version = Bundle.main.infoDictionary?["CFBundleShortVersionString"] as? String ?? "?"
5456
DebugFileLog.log("=== bootstrap — Heard v\(version) starting ===")
5557
DebugFileLog.startMainThreadHeartbeat()
5658

57-
let settingsStore = SettingsStore()
5859
let speakerStore = SpeakerStore()
5960
let queueStore = PipelineQueueStore()
6061
let modelCatalog = ModelCatalog()
@@ -237,6 +238,7 @@ public final class AppModel: ObservableObject {
237238
try self.recordingManager.startRecording(
238239
title: snapshot.title,
239240
meetingPID: snapshot.meetingPID,
241+
source: snapshot.source,
240242
rosterNames: snapshot.rosterNames
241243
)
242244
self.phase = .recording
@@ -337,7 +339,6 @@ public var filteredSpeakers: [SpeakerProfile] {
337339
// can take several seconds, keeping the HUD up and blocking further
338340
// hotkey presses via dictationToggleInFlight the whole time).
339341
isDictating = false
340-
partialTranscript = ""
341342
dictationError = nil
342343
if settingsStore.settings.showDictationHUD { DictationHUD.shared.hide() }
343344
dictationManager.modelKeepAliveSeconds = TimeInterval(settingsStore.settings.modelKeepAlive * 60)
@@ -364,8 +365,7 @@ public var filteredSpeakers: [SpeakerProfile] {
364365
isDictating = true
365366
dictationError = nil
366367
if settingsStore.settings.showDictationHUD { DictationHUD.shared.show() }
367-
// Observe partial transcript and watch for AX revocation
368-
observeDictationPartials()
368+
// Watch for AX revocation while dictation is active
369369
startAXPolling()
370370
DebugFileLog.log("START branch completed — isDictating=true")
371371
} catch {
@@ -542,17 +542,6 @@ public var filteredSpeakers: [SpeakerProfile] {
542542
hotkeyManager.activate()
543543
}
544544

545-
private func observeDictationPartials() {
546-
Task { [weak self] in
547-
guard let self else { return }
548-
// Poll the dictation manager's partial transcript (lightweight since it's @Published)
549-
while self.isDictating {
550-
self.partialTranscript = self.dictationManager.partialTranscript
551-
try? await Task.sleep(for: .milliseconds(100))
552-
}
553-
}
554-
}
555-
556545
/// Polls AXIsProcessTrusted() while dictation is active. If access is revoked mid-session,
557546
/// stops dictation and raises dictationAXLost so the menu bar can show a re-grant banner.
558547
private func startAXPolling() {
@@ -581,7 +570,6 @@ public var filteredSpeakers: [SpeakerProfile] {
581570
private func stopDictationIfActive() {
582571
guard isDictating else { return }
583572
isDictating = false
584-
partialTranscript = ""
585573
dictationError = nil
586574
if settingsStore.settings.showDictationHUD { DictationHUD.shared.hide() }
587575
Task {
@@ -789,7 +777,9 @@ public var filteredSpeakers: [SpeakerProfile] {
789777
}
790778
namingCandidates.removeAll()
791779
showNamingPrompt = false
792-
phase = queueStore.activeJob == nil ? .dormant : .processing
780+
// processingJob (not activeJob) — a stale .failed job in the queue must
781+
// not pin the phase at .processing after the prompt closes.
782+
phase = queueStore.processingJob == nil ? .dormant : .processing
793783
}
794784

795785
public func renameSpeaker(id: UUID, to name: String) {

Sources/HeardCore/AudioProcessing.swift

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -63,15 +63,29 @@ public struct PreprocessedTrack {
6363
public let samples: [Float]
6464
public let sampleRate: Double // always 16000
6565
public let vadMap: VadSegmentMap
66+
/// Captured at init so it stays valid after `releasingSamples()`.
67+
public let duration: TimeInterval
6668

6769
public init(samples: [Float], sampleRate: Double, vadMap: VadSegmentMap) {
6870
self.samples = samples
6971
self.sampleRate = sampleRate
7072
self.vadMap = vadMap
73+
self.duration = Double(samples.count) / sampleRate
7174
}
7275

73-
public var duration: TimeInterval {
74-
Double(samples.count) / sampleRate
76+
private init(sampleRate: Double, vadMap: VadSegmentMap, duration: TimeInterval) {
77+
self.samples = []
78+
self.sampleRate = sampleRate
79+
self.vadMap = vadMap
80+
self.duration = duration
81+
}
82+
83+
/// Copy with the sample buffer freed. The 16 kHz buffer for a multi-hour
84+
/// meeting runs to hundreds of MB; later pipeline stages only need the
85+
/// vadMap and duration, so callers drop the samples once the last
86+
/// audio-consuming stage has finished.
87+
public func releasingSamples() -> PreprocessedTrack {
88+
PreprocessedTrack(sampleRate: sampleRate, vadMap: vadMap, duration: duration)
7589
}
7690
}
7791

Sources/HeardCore/DebugFileLog.swift

Lines changed: 22 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,20 @@ import Foundation
44
/// NSLog/unified-log diagnostics aren't reaching `log stream` reliably and
55
/// we need a guaranteed observation channel for live debugging.
66
///
7+
/// Disabled unless the user turns on Developer Mode (Settings → General).
8+
/// Never log transcript or note content here — only metadata such as lengths.
9+
///
710
/// Tail it with:
811
/// tail -F ~/Library/Application\ Support/Heard/dict-debug.log
912
public enum DebugFileLog {
13+
/// Mirrors `AppSettings.developerMode`. Set at bootstrap and whenever
14+
/// settings persist. Plain bool read from multiple threads; a torn read
15+
/// at worst drops or emits one extra line.
16+
nonisolated(unsafe) public static var isEnabled = false
17+
18+
/// Rotate (truncate) the log once it exceeds this size.
19+
private static let maxFileBytes: UInt64 = 5_000_000
20+
1021
private static let queue = DispatchQueue(label: "com.execsumo.heard.debuglog")
1122
private static let formatter: DateFormatter = {
1223
let f = DateFormatter()
@@ -25,7 +36,7 @@ public enum DebugFileLog {
2536
/// later, main was unresponsive for those 8 seconds.
2637
@MainActor
2738
public static func startMainThreadHeartbeat() {
28-
guard heartbeatTimer == nil else { return }
39+
guard isEnabled, heartbeatTimer == nil else { return }
2940
var tick = 0
3041
heartbeatTimer = Timer.scheduledTimer(withTimeInterval: 1.0, repeats: true) { _ in
3142
tick += 1
@@ -38,6 +49,7 @@ public enum DebugFileLog {
3849
}
3950

4051
public static func log(_ message: String) {
52+
guard isEnabled else { return }
4153
let ts = formatter.string(from: Date())
4254
let line = "[\(ts)] \(message)\n"
4355
NSLog("Heard: [DICT-DBG] \(message)")
@@ -46,9 +58,15 @@ public enum DebugFileLog {
4658
let data = Data(line.utf8)
4759
if FileManager.default.fileExists(atPath: url.path) {
4860
if let handle = try? FileHandle(forWritingTo: url) {
49-
defer { try? handle.close() }
50-
_ = try? handle.seekToEnd()
51-
try? handle.write(contentsOf: data)
61+
let size = (try? handle.seekToEnd()) ?? 0
62+
if size > maxFileBytes {
63+
try? handle.close()
64+
try? FileManager.default.removeItem(at: url)
65+
try? data.write(to: url)
66+
} else {
67+
try? handle.write(contentsOf: data)
68+
try? handle.close()
69+
}
5270
}
5371
} else {
5472
try? FileManager.default.createDirectory(

Sources/HeardCore/DictationManager.swift

Lines changed: 17 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -37,9 +37,6 @@ public enum DictationError: Error, LocalizedError {
3737
public final class DictationManager: ObservableObject {
3838

3939
@Published public private(set) var state: DictationState = .idle
40-
/// Unused by the batch engine — kept @Published so existing UI bindings
41-
/// continue to compile. Always empty.
42-
@Published public var partialTranscript: String = ""
4340

4441
private var asrManager: AsrManager?
4542
private var asrModels: AsrModels?
@@ -52,6 +49,13 @@ public final class DictationManager: ObservableObject {
5249
private var micBufferContinuation: AsyncStream<AVAudioPCMBuffer>.Continuation?
5350
private var micForwardTask: Task<Void, Never>?
5451
private var bufferCount = 0
52+
private var bufferCapWarned = false
53+
54+
/// Hard cap on buffered audio: 4 hours at 16 kHz (matches the recording
55+
/// manager's max duration). A dictation session left running accumulates
56+
/// ~230 MB/hour; past the cap further audio is dropped rather than letting
57+
/// memory grow without bound.
58+
private static let maxBufferSamples = 4 * 3600 * 16_000
5559

5660
/// Called when transcription completes for a session with the final text.
5761
public var onUtterance: ((String) -> Void)?
@@ -114,7 +118,7 @@ public final class DictationManager: ObservableObject {
114118
}
115119

116120
audioBuffer.removeAll(keepingCapacity: true)
117-
partialTranscript = ""
121+
bufferCapWarned = false
118122

119123
try startMicCapture()
120124
state = .listening
@@ -163,11 +167,11 @@ public final class DictationManager: ObservableObject {
163167
let result = try await mgr.transcribe(
164168
samples, decoderState: &decoderState, language: .english
165169
)
166-
DebugFileLog.log("DictationManager.stop transcribe returned — rawText=\"\(result.text)\" (len=\(result.text.count))")
170+
DebugFileLog.log("DictationManager.stop transcribe returned — rawLen=\(result.text.count)")
167171
let boosted = await rescoreWithVocabulary(result: result, samples: samples)
168172
let normalized = TextNormalizer.shared.normalize(result: boosted).text
169173
let trimmed = normalized.trimmingCharacters(in: .whitespacesAndNewlines)
170-
DebugFileLog.log("DictationManager.stop final text=\"\(trimmed)\" (len=\(trimmed.count))")
174+
DebugFileLog.log("DictationManager.stop final len=\(trimmed.count)")
171175
guard !trimmed.isEmpty else {
172176
DebugFileLog.log("DictationManager.stop empty after normalize — skipping inject")
173177
return
@@ -341,6 +345,13 @@ public final class DictationManager: ObservableObject {
341345
}
342346

343347
private func appendSamples(_ samples: [Float]) {
348+
guard audioBuffer.count < Self.maxBufferSamples else {
349+
if !bufferCapWarned {
350+
bufferCapWarned = true
351+
NSLog("Heard: Dictation buffer hit the 4-hour cap — dropping further audio until stop")
352+
}
353+
return
354+
}
344355
audioBuffer.append(contentsOf: samples)
345356
bufferCount += 1
346357
if bufferCount == 1 {

Sources/HeardCore/RosterReader.swift

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,11 @@ public enum RosterReader {
6262
// their own when genuinely denied, so the result is the same either way.
6363
guard let pid = teamsPID else { return parseWindowTitle() }
6464

65-
let app = AXUIElementNode(AXUIElementCreateApplication(pid))
65+
let element = AXUIElementCreateApplication(pid)
66+
// Bound each AX round-trip so a busy/hung Teams can't stall the caller
67+
// for the system default timeout on every node of the tree walk.
68+
AXUIElementSetMessagingTimeout(element, 1.0)
69+
let app = AXUIElementNode(element)
6670
let names = readRosterFromNode(app)
6771
return names.isEmpty ? parseWindowTitle() : names
6872
}

0 commit comments

Comments
 (0)