From 80d615a0f545da28ea6ee1b2058e74a254324fdf Mon Sep 17 00:00:00 2001 From: giraphant Date: Mon, 12 Oct 2026 16:46:40 +0100 Subject: [PATCH 1/6] Tune Qwen final-pass deadlines and preload kernels --- ARCHITECTURE.md | 55 ++++++++--- Sources/Apps/Dousha/AppDelegate.swift | 5 +- Sources/Engines/MLXASR/MLXModelStore.swift | 76 ++++++++++++---- Sources/Engines/MLXASR/MLXStreamingASR.swift | 49 +++++----- .../Tools/SmokeCLI/MLXSmokeTranscriber.swift | 25 +++-- Sources/Tools/SmokeCLI/SmokeCLI.swift | 3 +- Tests/DoushaTests/MLXFinalPassTests.swift | 57 +++++++++++- Tests/DoushaTests/MLXModelPrewarmTests.swift | 91 +++++++++++++++++++ 8 files changed, 298 insertions(+), 63 deletions(-) create mode 100644 Tests/DoushaTests/MLXModelPrewarmTests.swift diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index d396c3b..ef41886 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -109,7 +109,11 @@ fatal `.error` stream event. `MLXBackend` passes the snapshotted Dousha language identifier (`zh-CN`, `en-US`, `auto`) unchanged; `MLXStreamingASR` alone maps it to the Qwen3 -language name. Nil, auto and unknown identifiers mean auto-detect. Local +language name. Nil, auto and unknown identifiers mean auto-detect. Keep the +user's selection for both streaming and batch: in the 2026-10-12 code-switched +clip comparison, forced Chinese retained a Chinese lead-in that the full-audio +`auto` pass dropped. The earlier observation that forced Chinese garbled English +was streaming-only, so it is not evidence to force auto for the final pass. Local streaming text is a HUD preview; stop transcribes the entire retained recording with Qwen3's batch API, and that nonempty result becomes the final text. Empty, failed or overdue batch output falls back to the filtered streaming text. @@ -141,7 +145,7 @@ visibility. Timing constants live as statics on the controller | `SpeechBackend` | one session = one `AsyncStream`; `stop()` yields exactly one terminal `.final`, `cancel()` finishes with none | `MockSpeechBackend` (RecordingControllerTests) | | `PushCaptureEngine` (+ `PCMCaptureEngine` / `BufferCaptureEngine`) | engines receive pushed audio from the hub; two-phase `beginSession`/`openStream` | `MockBackend` (MultiEngineBackendTests, hub = nil) | | `MLXBackend` (`Engine.local`, `PCMCaptureEngine`) | adapts prepare / open / ingest / stop / cancel to `MLXStreamingASR`; snapshots the unmapped language identifier and glossary context | injected prepare closure + isolated Preferences (MLXEngineWiringTests) | -| `MLXModelStore` loader + process-wide session lease | loads/caches Qwen3 weights once; serializes local inference sessions | injected failing/suspended loader (MLXASRLifecycleTests); lease tests never allocate MLX arrays | +| `MLXModelStore` loader + process-wide session lease | loads/caches Qwen3 weights once; serializes prewarm and local inference sessions | injected loader / `MLXLoadedModel` fake (MLXASRLifecycleTests, MLXModelPrewarmTests); lease tests never allocate MLX arrays | | `MLXPreparedSession` stream + final-pass runner | exercises real stop/cancel/deadline/lease ownership without a model | fake event stream + suspended runner (MLXFinalPassTests) | | `RecordingEnvironment` | struct-of-closures: ALL side effects the controller performs (HUD, inject, refine, scheduling, clock) | closure fakes + virtual clock | | `SettingsActions` | struct-of-closures: system-touching settings effects (login item, dock icon, credentials) | closure fakes | @@ -218,19 +222,37 @@ type is used, re-derive the reason or the annotation becomes a lie. auto-language headers. Partials, errors and the one stop completion dispatch on main and recheck the generation there; cancel immediately quarantines input and callbacks. -- **`MLXModelStore` / `MLXLoadedModel` (`@unchecked Sendable`)** — the store +- **`MLXModelStore` / `QwenLoadedModel` (`@unchecked Sendable`)** — the store actor caches weights and coalesces loading/warmup. A single process-wide lease gate, shared even by separately constructed stores, covers model load, - replacement, the complete streaming session and the subsequent batch pass. + replacement, kernel prewarm, the complete streaming session and the subsequent + batch pass. App launch (after hotkey installation) and engine-routing changes + call fire-and-forget `warmup` when Qwen3 is active; menu/model selection also + uses this path. A preload-owned new load runs one 1s silent batch (8-token + limit) off main under the same lease, discarding output. Cached models and + recording-owned loads never prewarm, including a warmup joining or queued + behind such a load. Prewarm failure logs an error but leaves weights `.ready`; + release the lease only after prewarm actually returns. State `.ready` denotes + loaded weights, not completion of prewarm; recordings still wait on the lease. + `[MLXModelStore] prewarm repoID=… ms=…` records its cost. + A separate 1s streaming-session prewarm was tested on the two 2026-10-12 + clips with 1.7B-8bit: first preview was 145/143ms versus 147/153ms with + batch-only prewarm, while total prewarm grew from 89–92ms to 200ms. This + small single-run difference did not demonstrate a useful streaming benefit; + keep only the batch prewarm (the installed app's initial batch was 1225ms + versus 284–377ms subsequently). Streaming cleanup precedes batch generation; never run both at once. - The loaded model is never - mutated by the store after publication and is only used by the lease holder; + The store never mutates the published model; only the lease holder uses it; model construction/loading runs off main. Cancelled callers do not cancel a shared download, and setup retains the lease until that load settles. - **MLX final-pass ownership and deadline** — stop waits up to 30s for setup, then up to 10s for `.ended` (saving the latest display text on timeout), then - up to 3s for streaming cleanup and up to `finalPassTimeout` (default 30s) - for batch. The batch runs off main with temperature 0, repetition penalty + up to 3s for streaming cleanup and up to `max(baseSeconds, audioRatio × + audioSeconds)` for batch. The injectable `MLXFinalPassDeadline` defaults to + 5s base and 0.10 ratio: ≤50s audio gets 5s; a five-minute recording gets 30s. + Batch elapsed time starts when generation begins, not during stream cleanup. + The existing `finalPass` log and smoke metrics include the computed deadline. + The batch runs off main with temperature 0, repetition penalty 1.15 and upstream's 32-token repetition context: a modest discouragement of loops without aggressively suppressing ordinary repeated words. The full PCM is passed in one `generate` call (upstream only splits very long audio at its @@ -346,6 +368,8 @@ xcodebuild -scheme smoke-cli -configuration Release \ ~/Library/Caches/Dousha/last_recording.wav --language auto --final-pass off .build/xcode/Build/Products/Release/smoke-cli mlx-transcribe \ ~/Library/Caches/Dousha/last_recording.wav --language zh-CN --context 'Vocabulary: Dousha, Qwen3' +.build/xcode/Build/Products/Release/smoke-cli mlx-transcribe \ + ~/Library/Caches/Dousha/last_recording.wav --language zh-CN --realtime --prewarm ``` `mlx-transcribe` uses `WavReader` (16kHz mono s16le only) and the actual engine @@ -356,12 +380,17 @@ accepts a compatible Hugging Face repo. Model load time includes any download ingest, and stop latency ends at completion. The command prints partial count, final length, streaming preview and final transcript; keep private transcripts out of shared reports. `--final-pass on|off` defaults to on; `--context` supplies -only the final batch system message. `stop_to_final_ms` includes streaming drain, +only the final batch system message. `--prewarm` awaits the same preload path +used at launch before opening the recording; `preload_ms` includes load and +kernel prewarm, while `model_load_ms` then measures the cached open. Compare +with/without prewarm in separate fresh processes. `stop_to_final_ms` includes streaming drain, cleanup and batch; `batch_ms` measures batch only (elapsed time at fallback -delivery on a deadline), and `final_source` is `batch` or `stream`. -It services main-queue callbacks through async waits, with 600s model-load and -80s stop watchdogs. Inspect engine logs for `finalTimedOut` even when a nonempty -fallback makes the command succeed. +delivery on a deadline), `deadline_seconds` is the computed batch allowance, +and `final_source` is `batch` or `stream`. +It services main-queue callbacks through async waits, with a 600s model-load +watchdog after open and a stop watchdog of max(80s, 48s + batch deadline). +Inspect engine logs for `finalTimedOut` even when a nonempty fallback makes the +command succeed. `Package.resolved` is committed and the mlx-audio-swift revision is pinned in `Package.swift`. Upgrade mlx-audio-swift, mlx-swift, mlx-swift-lm and their diff --git a/Sources/Apps/Dousha/AppDelegate.swift b/Sources/Apps/Dousha/AppDelegate.swift index 23c1d79..8a8a576 100644 --- a/Sources/Apps/Dousha/AppDelegate.swift +++ b/Sources/Apps/Dousha/AppDelegate.swift @@ -8,6 +8,7 @@ import AVFoundation import ServiceManagement import DoubaoASR import SonioxASR +import MLXASR import ASRSupport import ConcurrencySupport @@ -197,6 +198,7 @@ final class AppDelegate: NSObject, NSApplicationDelegate { if prefs.activeEngines.contains(.doubao) { DoubaoCredentialStore.shared.warmup() } + if prefs.activeEngines.contains(.qwen3) { MLXModelStore.shared.warmup(prefs.qwen3Model) } } /// If the app is relaunched while already running (Finder/Spotlight), bring @@ -389,8 +391,9 @@ final class AppDelegate: NSObject, NSApplicationDelegate { @objc private func handleEngineRoutingChanged() { // Routing slots / primary language changed in Settings. Refresh the menu - // (engine summary, checkmarks, language) and warm Doubao if now active. + // (engine summary, checkmarks, language) and warm active engines. if prefs.activeEngines.contains(.doubao) { DoubaoCredentialStore.shared.warmup() } + if prefs.activeEngines.contains(.qwen3) { MLXModelStore.shared.warmup(prefs.qwen3Model) } menuBar.rebuildMenu() } diff --git a/Sources/Engines/MLXASR/MLXModelStore.swift b/Sources/Engines/MLXASR/MLXModelStore.swift index 8ab00b9..3e5b85f 100644 --- a/Sources/Engines/MLXASR/MLXModelStore.swift +++ b/Sources/Engines/MLXASR/MLXModelStore.swift @@ -1,15 +1,43 @@ import Foundation import MLXAudioSTT +import MLX import ConcurrencySupport -/// Safe to transfer after loading: the store never mutates the published model, -/// and only the holder of the process-wide lease creates a streaming session -/// or runs its subsequent full-audio batch pass using it. MLXAudioSTT owns -/// synchronization inside the streaming session. Do not expose -/// the model to callers that bypass the lease (ARCHITECTURE §4). -internal final class MLXLoadedModel: @unchecked Sendable { +/// Offline seam for successful loads and prewarm without allocating MLX arrays. +/// Both operations require the process-wide lease and run off main. +internal protocol MLXLoadedModel: Sendable { + func prepareSession(language: String?) -> MLXPreparedSession + func prewarm() async throws +} + +/// Safe to transfer after loading: only the process-wide lease holder accesses +/// the native model, including prewarm, streaming and the final full-audio pass. +/// MLXAudioSTT owns synchronization inside the streaming session (§4). +private final class QwenLoadedModel: MLXLoadedModel, @unchecked Sendable { let model: Qwen3ASRModel init(_ model: Qwen3ASRModel) { self.model = model } + + func prepareSession(language: String?) -> MLXPreparedSession { + MLXPreparedSession( + stream: StreamingInferenceSession(model: model, config: StreamingConfig(language: language)), + finalPass: { request in + let samples = MLXASRSupport.floatSamples(fromPCM16LE: request.pcm) + guard !samples.isEmpty else { return "" } + return self.model.generate(audio: MLXArray(samples), maxTokens: 8192, + temperature: 0, context: request.context, + language: request.language, chunkDuration: 1200, + minChunkDuration: 1, repetitionPenalty: 1.15, + repetitionContextSize: 32).text + }) + } + + func prewarm() async throws { + // One second of silence exercises encoder + decoder kernels. Bound the + // token count so a silent-input hallucination cannot prolong startup. + _ = model.generate(audio: MLXArray(Array(repeating: Float(0), count: 16_000)), + maxTokens: 8, temperature: 0, language: "Chinese", + repetitionPenalty: 1.15, repetitionContextSize: 32) + } } /// A token can be released repeatedly, including after a cancelled acquire: @@ -75,21 +103,21 @@ public actor MLXModelStore { } public private(set) var state: State = .idle - private var cached: (repoID: String, model: MLXLoadedModel)? - private var inFlight: (repoID: String, task: Task)? + private var cached: (repoID: String, model: any MLXLoadedModel)? + private var inFlight: (repoID: String, task: Task)? private var warming: Set = [] - private let loadModel: @Sendable (String) async throws -> MLXLoadedModel + private let loadModel: @Sendable (String) async throws -> any MLXLoadedModel public init() { loadModel = { repoID in - MLXLoadedModel(try await Qwen3ASRModel.fromPretrained(repoID)) + QwenLoadedModel(try await Qwen3ASRModel.fromPretrained(repoID)) } doushaLog("[MLXModelStore] state=idle") } - /// Offline failure/cancellation tests inject a loader that never touches - /// Hugging Face or Metal. The production API always uses fromPretrained. - internal init(loadModel: @escaping @Sendable (String) async throws -> MLXLoadedModel) { + /// Offline lifecycle/prewarm tests inject a loader and model operations + /// that never touch Hugging Face or Metal. Production uses fromPretrained. + internal init(loadModel: @escaping @Sendable (String) async throws -> any MLXLoadedModel) { self.loadModel = loadModel doushaLog("[MLXModelStore] state=idle") } @@ -103,7 +131,8 @@ public actor MLXModelStore { try await MLXSessionLeaseGate.shared.acquire() } - private func preload(_ descriptor: MLXModelDescriptor) async { + /// Awaitable warmup for the smoke harness; app call sites use fire-and-forget warmup. + public func preload(_ descriptor: MLXModelDescriptor) async { if cached?.repoID == descriptor.repoID { return } // Coalesce warmups even while they are waiting for another session's // lease, including a failed load (no queue of identical retry loads). @@ -118,7 +147,22 @@ public actor MLXModelStore { do { let lease = try await acquireSessionLease() do { - _ = try await load(descriptor, lease: lease) + // Recheck after waiting: a recording may have loaded this model + // first. Never add prewarm to a recording-triggered load. + if cached?.repoID != descriptor.repoID { + let model = try await load(descriptor, lease: lease) + await Task.detached(priority: .userInitiated) { + let started = ProcessInfo.processInfo.systemUptime + do { + try await model.prewarm() + } catch { + // Weights remain usable; prewarm is only an optimization. + doushaLog("[MLXModelStore] prewarmFailed repoID=\(descriptor.repoID) error=\(error.localizedDescription)") + } + let milliseconds = Int((ProcessInfo.processInfo.systemUptime - started) * 1_000) + doushaLog("[MLXModelStore] prewarm repoID=\(descriptor.repoID) ms=\(milliseconds)") + }.value + } } catch { // load() publishes and logs the error; warmup never emits an // engine error or changes an active recording's callbacks. @@ -133,7 +177,7 @@ public actor MLXModelStore { /// Keeping the lease throughout loading prevents a model switch while the /// old session is still draining or running its final batch pass. A cancelled caller does not cancel the /// shared download; its setup path releases the lease after the load settles. - internal func load(_ descriptor: MLXModelDescriptor, lease: MLXSessionLease) async throws -> MLXLoadedModel { + internal func load(_ descriptor: MLXModelDescriptor, lease: MLXSessionLease) async throws -> any MLXLoadedModel { if let cached, cached.repoID == descriptor.repoID { return cached.model } if let inFlight, inFlight.repoID == descriptor.repoID { return try await inFlight.task.value diff --git a/Sources/Engines/MLXASR/MLXStreamingASR.swift b/Sources/Engines/MLXASR/MLXStreamingASR.swift index 007fdda..7731dd1 100644 --- a/Sources/Engines/MLXASR/MLXStreamingASR.swift +++ b/Sources/Engines/MLXASR/MLXStreamingASR.swift @@ -2,7 +2,6 @@ import Foundation import ASRSupport import ConcurrencySupport import MLXAudioSTT -import MLX /// The tap only appends bytes under this lock. Swapping the buffer on drain /// keeps conversion, inference, actor hops, tasks and logging off the audio thread. @@ -145,11 +144,27 @@ internal struct MLXPreparedSession: Sendable { let finalPass: @Sendable (MLXFinalPassRequest) async throws -> String } +/// Batch-only deadline; streaming shutdown retains its separate bounded waits. +public struct MLXFinalPassDeadline: Sendable { + public let baseSeconds: TimeInterval + public let audioRatio: Double + + public init(baseSeconds: TimeInterval = 5, audioRatio: Double = 0.10) { + self.baseSeconds = max(0, baseSeconds) + self.audioRatio = max(0, audioRatio) + } + + public func seconds(forAudioSeconds audioSeconds: Double) -> TimeInterval { + max(baseSeconds, audioRatio * audioSeconds) + } +} + public struct MLXFinalPassMetrics: Sendable { public let source: String /// Elapsed batch time at delivery; on a deadline this is not its eventual runtime. public let batchMilliseconds: Double public let audioSeconds: Double + public let deadlineSeconds: TimeInterval public let streamLength: Int public let finalLength: Int } @@ -254,7 +269,7 @@ public actor MLXStreamingASR { private let model: MLXModelDescriptor private let store: MLXModelStore private let finalPassEnabled: Bool - private let finalPassTimeout: TimeInterval + private let finalPassDeadline: MLXFinalPassDeadline private let streamEndTimeout: TimeInterval private let streamCleanupTimeout: TimeInterval internal typealias SessionFactory = @Sendable (MLXModelDescriptor, MLXModelStore, MLXSessionLease, String?) async throws -> MLXPreparedSession @@ -303,23 +318,23 @@ public actor MLXStreamingASR { private var recording: Recording? public init(model: MLXModelDescriptor = MLXModelCatalog.defaultModel, store: MLXModelStore = .shared, - finalPass: Bool = true, finalPassTimeout: TimeInterval = 30) { + finalPass: Bool = true, finalPassDeadline: MLXFinalPassDeadline = MLXFinalPassDeadline()) { self.model = model self.store = store self.finalPassEnabled = finalPass - self.finalPassTimeout = max(0, finalPassTimeout) + self.finalPassDeadline = finalPassDeadline self.streamEndTimeout = 10 self.streamCleanupTimeout = 3 self.makeSession = Self.makeNativeSession } internal init(store: MLXModelStore = .shared, finalPass: Bool = true, - finalPassTimeout: TimeInterval = 30, streamEndTimeout: TimeInterval = 10, + finalPassDeadline: MLXFinalPassDeadline = MLXFinalPassDeadline(), streamEndTimeout: TimeInterval = 10, streamCleanupTimeout: TimeInterval = 3, makeSession: @escaping SessionFactory) { self.model = MLXModelCatalog.defaultModel self.store = store self.finalPassEnabled = finalPass - self.finalPassTimeout = max(0, finalPassTimeout) + self.finalPassDeadline = finalPassDeadline self.streamEndTimeout = streamEndTimeout self.streamCleanupTimeout = streamCleanupTimeout self.makeSession = makeSession @@ -331,17 +346,7 @@ public actor MLXStreamingASR { try Task.checkCancellation() // Construction and generation both allocate MLX arrays; run off main. return await Task.detached(priority: .userInitiated) { - MLXPreparedSession( - stream: StreamingInferenceSession(model: loaded.model, config: StreamingConfig(language: language)), - finalPass: { request in - let samples = MLXASRSupport.floatSamples(fromPCM16LE: request.pcm) - guard !samples.isEmpty else { return "" } - return loaded.model.generate(audio: MLXArray(samples), maxTokens: 8192, - temperature: 0, context: request.context, - language: request.language, chunkDuration: 1200, - minChunkDuration: 1, repetitionPenalty: 1.15, - repetitionContextSize: 32).text - }) + loaded.prepareSession(language: language) }.value } @@ -505,13 +510,14 @@ public actor MLXStreamingASR { } let pcm = input.fullAudio(generation) let audioSeconds = Double(pcm.count / 2) / 16_000 + let deadlineSeconds = finalPassDeadline.seconds(forAudioSeconds: audioSeconds) let work = startFinalPass(current, pcm: pcm) // Includes up to 3s for streaming cleanup. After a streaming timeout, // natural .ended is still required before batch inference can start. // A detached owner retains the lease if either stage exceeds this wait. let output: WaitOutcome if case .signaled = await work.ready.wait(timeout: streamCleanupTimeout) { - output = await work.result.wait(timeout: max(0, finalPassTimeout - work.elapsedMilliseconds / 1_000)) + output = await work.result.wait(timeout: max(0, deadlineSeconds - work.elapsedMilliseconds / 1_000)) } else { output = .timeout } @@ -519,7 +525,7 @@ public actor MLXStreamingASR { guard isCurrent(current) else { return } let batch: String let milliseconds: Double - if case .signaled(let value) = output, value.milliseconds <= finalPassTimeout * 1_000 { + if case .signaled(let value) = output, value.milliseconds <= deadlineSeconds * 1_000 { batch = value.text milliseconds = value.milliseconds } else { @@ -579,10 +585,11 @@ public actor MLXStreamingASR { private func recordFinalPass(_ current: Recording, text: String, fallback: String, batchMilliseconds: Double, audioSeconds: Double, source: String = "stream") { + let deadlineSeconds = finalPassDeadline.seconds(forAudioSeconds: audioSeconds) finalPassMetrics = MLXFinalPassMetrics(source: source, batchMilliseconds: batchMilliseconds, - audioSeconds: audioSeconds, streamLength: fallback.count, + audioSeconds: audioSeconds, deadlineSeconds: deadlineSeconds, streamLength: fallback.count, finalLength: text.count) - log(current, "finalPass source=\(source) batchMs=\(Int(batchMilliseconds)) audioSeconds=\(audioSeconds) streamLength=\(fallback.count) finalLength=\(text.count)") + log(current, "finalPass source=\(source) batchMs=\(Int(batchMilliseconds)) audioSeconds=\(audioSeconds) deadlineSeconds=\(deadlineSeconds) streamLength=\(fallback.count) finalLength=\(text.count)") } public nonisolated func cancel() { diff --git a/Sources/Tools/SmokeCLI/MLXSmokeTranscriber.swift b/Sources/Tools/SmokeCLI/MLXSmokeTranscriber.swift index 86774d0..2ae93e0 100644 --- a/Sources/Tools/SmokeCLI/MLXSmokeTranscriber.swift +++ b/Sources/Tools/SmokeCLI/MLXSmokeTranscriber.swift @@ -7,6 +7,7 @@ struct MLXSmokeOptions { var model = MLXModelCatalog.defaultModel.repoID var language = "auto" var realtime = false + var prewarm = false var finalPass = true var context = "" @@ -36,6 +37,8 @@ struct MLXSmokeOptions { } finalPass = arguments[index] == "on" } + case "--prewarm": + prewarm = true case "--realtime": realtime = true default: @@ -75,10 +78,15 @@ final class MLXSmokeTranscriber { return false } print("wav: \(pcm.count) bytes, \(String(format: "%.3f", Double(pcm.count) / 32_000))s @16kHz mono s16le") - print("mlx: model=\(options.model) language=\(options.language) realtime=\(options.realtime) final_pass=\(options.finalPass ? "on" : "off")") + print("mlx: model=\(options.model) language=\(options.language) realtime=\(options.realtime) prewarm=\(options.prewarm) final_pass=\(options.finalPass ? "on" : "off")") let model = MLXModelDescriptor(repoID: options.model, displayName: options.model) let store = MLXModelStore.shared + if options.prewarm { + let started = now + await store.preload(model) + print("preload_ms: \(milliseconds(now - started)) (load + kernel prewarm)") + } let engine = MLXStreamingASR(model: model, store: store, finalPass: options.finalPass) await engine.prepareSession(onPartial: { partial in MainActor.assumeIsolated { @@ -136,11 +144,11 @@ final class MLXSmokeTranscriber { print("audio: fed \(offset) bytes in \(milliseconds(now - audioStarted))ms") let stopStarted = now - let result = await finish(engine) + let result = await finish(engine, audioSeconds: Double(pcm.count) / 32_000) let stopSeconds = now - stopStarted printMetrics(load: loadSeconds, stop: stopSeconds, finalLength: result?.text.count ?? 0) if let metrics = await engine.finalPassMetrics { - print("final_pass: batch_ms=\(String(format: "%.1f", metrics.batchMilliseconds)) final_source=\(metrics.source)") + print("final_pass: batch_ms=\(String(format: "%.1f", metrics.batchMilliseconds)) final_source=\(metrics.source) deadline_seconds=\(metrics.deadlineSeconds)") } print("streaming_preview_text: \(previewText)") if let result { @@ -150,15 +158,16 @@ final class MLXSmokeTranscriber { return failure == nil && result?.text.isEmpty == false } - private func finish(_ engine: MLXStreamingASR) async -> TranscriptionResult? { + private func finish(_ engine: MLXStreamingASR, audioSeconds: Double) async -> TranscriptionResult? { await withCheckedContinuation { continuation in completion = continuation - // The engine allows 30s setup + 10s stream + 3s cleanup + 30s batch. A separate - // CLI watchdog makes a missing callback visible instead of hanging. + // Cover 30s setup + 10s stream + 3s cleanup + the scaled batch + // deadline, with margin, so long WAVs don't hit the watchdog early. + let timeout = max(80, 48 + MLXFinalPassDeadline().seconds(forAudioSeconds: audioSeconds)) watchdog = Task { - do { try await Task.sleep(for: .seconds(80)) } + do { try await Task.sleep(for: .seconds(timeout)) } catch { return } - failure = "stop completion timed out after 80s" + failure = "stop completion timed out after \(timeout)s" print("mlx: ERROR — \(failure!)") engine.cancel() resolve(nil) diff --git a/Sources/Tools/SmokeCLI/SmokeCLI.swift b/Sources/Tools/SmokeCLI/SmokeCLI.swift index 69bd707..2cb9136 100644 --- a/Sources/Tools/SmokeCLI/SmokeCLI.swift +++ b/Sources/Tools/SmokeCLI/SmokeCLI.swift @@ -42,11 +42,12 @@ struct SmokeCLI { --format defaults to speech_opus (needs an encoder, macOS only); pcm probes whether the server accepts raw audio. --profile defaults to official; also supports fast and minimal. - smoke-cli mlx-transcribe [--model ] [--language ] [--realtime] [--final-pass on|off] [--context ] + smoke-cli mlx-transcribe [--model ] [--language ] [--realtime] [--prewarm] [--final-pass on|off] [--context ] Local MLX preview + final batch. WAV must be 16kHz mono s16le. Model defaults to mlx-community/Qwen3-ASR-0.6B-8bit; language defaults to auto (also accepts zh-CN / en-US). --realtime feeds at audio rate; default feeds with 10ms pauses. + --prewarm loads and warms kernels before streaming. --final-pass defaults to on; context reaches only the final batch. Build with xcodebuild for the Metal shader bundle. """) diff --git a/Tests/DoushaTests/MLXFinalPassTests.swift b/Tests/DoushaTests/MLXFinalPassTests.swift index ab8cc3f..ebe70fc 100644 --- a/Tests/DoushaTests/MLXFinalPassTests.swift +++ b/Tests/DoushaTests/MLXFinalPassTests.swift @@ -5,7 +5,7 @@ import ConcurrencySupport import MLXAudioSTT @testable import MLXASR -private final class FakeMLXStream: MLXStreamingSession { +final class FakeMLXStream: MLXStreamingSession { let events: AsyncStream let continuation: AsyncStream.Continuation let stopped = OneShotChannel() @@ -54,6 +54,57 @@ final class MLXFinalPassTests: XCTestCase { return result } + func testShortAudioUsesBaseDeadline() { + XCTAssertEqual(MLXFinalPassDeadline().seconds(forAudioSeconds: 1), 5) + XCTAssertEqual(MLXFinalPassDeadline().seconds(forAudioSeconds: 0), 5) + } + + func testLongAudioUsesScaledDeadline() { + XCTAssertEqual(MLXFinalPassDeadline().seconds(forAudioSeconds: 300), 30) + } + + func testDeadlineBoundaryAndInjectedPolicy() { + let deadline = MLXFinalPassDeadline() + XCTAssertEqual(deadline.seconds(forAudioSeconds: 49), 5) + XCTAssertEqual(deadline.seconds(forAudioSeconds: 50), 5) + XCTAssertEqual(deadline.seconds(forAudioSeconds: 51), 5.1, accuracy: 0.0001) + XCTAssertEqual(MLXFinalPassDeadline(baseSeconds: 2, audioRatio: 0.5).seconds(forAudioSeconds: 10), 5) + } + + func testTimeoutUsesAudioScaledDeadlineRatherThanBase() async throws { + let started = OneShotChannel() + let releaseBatch = OneShotChannel() + let engine = MLXStreamingASR(finalPassDeadline: MLXFinalPassDeadline(baseSeconds: 0.01, audioRatio: 0.1)) { _, _, _, _ in + MLXPreparedSession(stream: FakeMLXStream()) { _ in + started.finish(()) + try await releaseBatch.wait() + return "too late" + } + } + await engine.prepareSession(onPartial: { _ in }, onError: { _ in XCTFail("Unexpected error") }, language: "auto") + engine.ingest(Data(repeating: 0, count: 64_000)) // Exactly two seconds → 0.2s deadline. + engine.openStream() + let result = stop(engine) + _ = try await started.wait() + if case .signaled = await result.wait(timeout: 0.05) { + XCTFail("The base deadline was used instead of the audio-scaled deadline") + } + let outcome = await result.wait(timeout: 1) + releaseBatch.finish(()) + guard case .signaled(let final) = outcome else { + XCTFail("Computed deadline did not deliver fallback") + engine.cancel() + return + } + XCTAssertEqual(final.text, "preview") + let metrics = await engine.finalPassMetrics + XCTAssertEqual(metrics?.deadlineSeconds, 0.2) + XCTAssertEqual(metrics?.source, "stream") + XCTAssertGreaterThanOrEqual(metrics?.batchMilliseconds ?? 0, 190) + let lease = try await MLXModelStore().acquireSessionLease() + await lease.release() + } + func testBatchWinsAfterStreamingCleanupAndReceivesFullPCMContextAndLanguage() async throws { let stream = FakeMLXStream() let request = Lock(nil) @@ -135,7 +186,7 @@ final class MLXFinalPassTests: XCTestCase { let stream = FakeMLXStream() let started = OneShotChannel() let releaseBatch = OneShotChannel() - let engine = MLXStreamingASR(finalPassTimeout: 0.03) { _, _, _, _ in + let engine = MLXStreamingASR(finalPassDeadline: MLXFinalPassDeadline(baseSeconds: 0.03, audioRatio: 0)) { _, _, _, _ in MLXPreparedSession(stream: stream) { _ in started.finish(()) try await releaseBatch.wait() @@ -196,7 +247,7 @@ final class MLXFinalPassTests: XCTestCase { func testStreamingTimeoutNeverStartsConcurrentBatchAndRetainsLeaseUntilNaturalEnd() async throws { let stream = FakeMLXStream(endOnStop: false) - let engine = MLXStreamingASR(finalPassTimeout: 0.03, streamEndTimeout: 0.03, streamCleanupTimeout: 0.03) { _, _, _, _ in + let engine = MLXStreamingASR(finalPassDeadline: MLXFinalPassDeadline(baseSeconds: 0.03, audioRatio: 0), streamEndTimeout: 0.03, streamCleanupTimeout: 0.03) { _, _, _, _ in MLXPreparedSession(stream: stream) { _ in XCTFail("An abandoned final pass must not start late") return "batch" diff --git a/Tests/DoushaTests/MLXModelPrewarmTests.swift b/Tests/DoushaTests/MLXModelPrewarmTests.swift new file mode 100644 index 0000000..17690d4 --- /dev/null +++ b/Tests/DoushaTests/MLXModelPrewarmTests.swift @@ -0,0 +1,91 @@ +import XCTest +import Foundation +import ConcurrencySupport +@testable import MLXASR + +private struct FakeLoadedMLXModel: MLXLoadedModel { + let runPrewarm: @Sendable () async throws -> Void + + func prewarm() async throws { try await runPrewarm() } + func prepareSession(language: String?) -> MLXPreparedSession { + MLXPreparedSession(stream: FakeMLXStream()) { _ in "batch" } + } +} + +@MainActor +final class MLXModelPrewarmTests: XCTestCase { + func testWarmupPrewarmsOffMainOnceAndKeepsCachedModel() async throws { + let loads = Lock(0) + let prewarms = Lock(0) + let store = MLXModelStore { _ in + loads.withLock { $0 += 1 } + return FakeLoadedMLXModel { + XCTAssertFalse(Thread.isMainThread) + prewarms.withLock { $0 += 1 } + } + } + await store.preload(MLXModelCatalog.defaultModel) + await store.preload(MLXModelCatalog.defaultModel) + let lease = try await store.acquireSessionLease() + _ = try await store.load(MLXModelCatalog.defaultModel, lease: lease) + await lease.release() + XCTAssertEqual(loads.value(), 1) + XCTAssertEqual(prewarms.value(), 1) + } + + func testRecordingLoadAndWarmupJoiningThatLoadNeverPrewarm() async throws { + let started = OneShotChannel() + let finishLoad = OneShotChannel() + let store = MLXModelStore { _ in + started.finish(()) + try await finishLoad.wait() + return FakeLoadedMLXModel { XCTFail("A recording-triggered load must never prewarm") } + } + let engine = MLXStreamingASR(store: store) + await engine.prepareSession(onPartial: { _ in }, onError: { _ in XCTFail("Unexpected error") }, language: "zh-CN") + engine.openStream() + _ = try await started.wait() + store.warmup(MLXModelCatalog.defaultModel) + finishLoad.finish(()) + let completed = expectation(description: "recording finishes") + engine.stop { _ in completed.fulfill() } + await fulfillment(of: [completed], timeout: 2) + await store.preload(MLXModelCatalog.defaultModel) + let state = await store.state + XCTAssertEqual(state, .ready(repoID: MLXModelCatalog.defaultModel.repoID)) + } + + func testPrewarmHoldsProcessLeaseUntilItReturns() async throws { + let started = OneShotChannel() + let finishPrewarm = OneShotChannel() + let store = MLXModelStore { _ in + FakeLoadedMLXModel { + started.finish(()) + try await finishPrewarm.wait() + } + } + store.warmup(MLXModelCatalog.defaultModel) + _ = try await started.wait() + let acquired = OneShotChannel(MLXSessionLease.self) + let waiter = Task { acquired.finish(try await MLXModelStore().acquireSessionLease()) } + if case .signaled = await acquired.wait(timeout: 0.03) { + XCTFail("Prewarm released the process-wide lease before returning") + } + finishPrewarm.finish(()) + let lease = try await acquired.wait() + await lease.release() + _ = await waiter.result + } + + func testPrewarmFailureLeavesReadyAndReleasesLease() async throws { + let store = MLXModelStore { _ in + FakeLoadedMLXModel { throw CancellationError() } + } + await store.preload(MLXModelCatalog.defaultModel) + let state = await store.state + XCTAssertEqual(state, .ready(repoID: MLXModelCatalog.defaultModel.repoID)) + let lease = try await store.acquireSessionLease() + _ = try await store.load(MLXModelCatalog.defaultModel, lease: lease) + await lease.release() + } +} From 21cf179acd8a2c3fc08cd2d292686f29b30b2088 Mon Sep 17 00:00:00 2001 From: giraphant Date: Mon, 12 Oct 2026 16:55:59 +0100 Subject: [PATCH 2/6] feat(settings): unify native forms and engine option sheets --- ARCHITECTURE.md | 4 +- Sources/Apps/Dousha/SettingsWindow.swift | 617 +++++++++++------------ 2 files changed, 309 insertions(+), 312 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index ef41886..50ab11c 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -309,7 +309,9 @@ without new evidence. - **`SettingsView` is one struct; panes are not child views.** API keys and test results live only in `@State` until 保存并测试; `NavigationSplitView` destroys child views on pane switch, so child-view panes would silently drop - unsaved edits. + unsaved edits. Sheets follow the same rule: present them from `SettingsView` + and keep their editable state there across dismissal; helpers accept only + values, bindings, and closures, without owning draft state. - **`HotkeyMonitor` and `CancelKeyMonitor` stay separate classes.** The shared part (tap lifecycle) is already extracted into `EventTap`; merging the rest couples two independently-fragile event paths. diff --git a/Sources/Apps/Dousha/SettingsWindow.swift b/Sources/Apps/Dousha/SettingsWindow.swift index 5d2adfd..f692203 100644 --- a/Sources/Apps/Dousha/SettingsWindow.swift +++ b/Sources/Apps/Dousha/SettingsWindow.swift @@ -62,6 +62,11 @@ private enum SettingsPane: String, CaseIterable, Identifiable, Hashable { } } +private struct SettingsEngineSheet: Identifiable { + let engine: Engine + var id: Engine { engine } +} + struct SettingsView: View { let actions: SettingsActions @@ -106,9 +111,10 @@ struct SettingsView: View { @State private var sonioxStatusIsError: Bool = false @State private var isTestingSoniox: Bool = false - // Card expansion and model-management state belong to this view, so switching - // panes or collapsing a card never discards unsaved credentials/test results. - @State private var expandedEngines: Set = [.qwen3] + // Sheets and panes share this owner: dismissal/navigation must not discard + // unsaved credentials, test results, or model-management state. + @State private var selectedEngineSheet: SettingsEngineSheet? + @State private var presentedHelp: String? @State private var activeEngines: [Engine] = Preferences.shared.activeEngines @State private var qwen3ModelRepoID: String = Preferences.shared.qwen3ModelRepoID @State private var modelInventory: [String: MLXModelInventory.Entry] = [:] @@ -171,6 +177,9 @@ struct SettingsView: View { detail .navigationTitle((selectedPane ?? .general).title) } + .sheet(item: $selectedEngineSheet) { selection in + engineSheet(selection.engine) + } } @ViewBuilder @@ -195,14 +204,11 @@ struct SettingsView: View { applyLaunchAtLogin(newValue) } if !launchAtLoginError.isEmpty { - Text(launchAtLoginError) - .font(.callout) - .foregroundStyle(.red) - .fixedSize(horizontal: false, vertical: true) + statusLabel(launchAtLoginError, isError: true) } } - Section("显示") { + Section { Toggle("显示程序坞图标", isOn: $showDockIcon) .onChange(of: showDockIcon) { _, newValue in actions.setDockIconVisible(newValue) @@ -211,19 +217,16 @@ struct SettingsView: View { .onChange(of: showMenuBarIcon) { _, newValue in actions.setMenuBarIconVisible(newValue) } + } header: { + Text("显示") + } footer: { Text("两者都隐藏后，可从 Finder 或聚焦搜索重新打开豆沙以再次显示本窗口。") - .font(.callout) - .foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) } - Section("热键") { + Section { LabeledContent("触发键") { HStack(spacing: 8) { Text(HotkeyMonitor.displayName(forKeyCode: hotkeyKeyCode)) - .padding(.horizontal, 8) - .padding(.vertical, 4) - .background(RoundedRectangle(cornerRadius: 6).fill(Color.gray.opacity(0.12))) Button(isRecordingHotkey ? "按下修饰键…" : "录制") { toggleRecording() } } } @@ -231,25 +234,21 @@ struct SettingsView: View { Text("长按说话").tag(HotkeyMode.pushToTalk) Text("切换").tag(HotkeyMode.toggle) } - .pickerStyle(.segmented) .onChange(of: hotkeyMode) { _, newMode in let cfg = HotkeyConfig(keyCode: Preferences.shared.hotkey.keyCode, mode: newMode) Preferences.shared.hotkey = cfg NotificationCenter.default.post(name: .doushaHotkeyConfigChanged, object: nil) } + } header: { + Text("热键") + } footer: { Text("长按说话：按住时录音；切换：每次按下开始/停止。") - .font(.callout) - .foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) } - Section("取消") { + Section { LabeledContent("取消键") { HStack(spacing: 8) { Text(cancelHotkey.displayName) - .padding(.horizontal, 8) - .padding(.vertical, 4) - .background(RoundedRectangle(cornerRadius: 6).fill(Color.gray.opacity(0.12))) Button(isRecordingCancelHotkey ? "按任意键…" : "录制") { toggleCancelHotkeyRecording() } Button("关闭") { cancelHotkey = .disabled @@ -259,10 +258,11 @@ struct SettingsView: View { .disabled(!cancelHotkey.isEnabled) } } - Text("取消会丢弃当前录音且不转写。仅在录音时生效，其余情况下按键正常透传。") - .font(.callout) - .foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) + } header: { + Text("取消") + } footer: { + sectionFooter("取消键仅在录音时丢弃当前录音且不转写。", + details: "取消会丢弃当前录音且不转写。仅在录音时生效，其余情况下按键正常透传。") } } .formStyle(.grouped) @@ -339,12 +339,11 @@ struct SettingsView: View { @ViewBuilder private func microphoneRow(_ device: AudioInputDevice) -> some View { HStack(spacing: 8) { - Circle() - .fill(device.isDefaultInput ? Color.green : Color.gray.opacity(0.3)) - .frame(width: 8, height: 8) + Image(systemName: "mic") + .foregroundStyle(.secondary) VStack(alignment: .leading, spacing: 2) { Text(device.name) - Text("\(device.transportDescription) · \(device.inputChannelCount) 通道") + Text("\(device.transportDescription) · \(device.inputChannelCount) 通道\(device.isDefaultInput ? " · 系统默认" : "")") .font(.caption) .foregroundStyle(.secondary) } @@ -354,17 +353,19 @@ struct SettingsView: View { } .buttonStyle(.borderless) .help("上移") + .accessibilityLabel("上移 \(device.name)") Button(action: { moveMicrophone(device, delta: 1) }) { Image(systemName: "chevron.down") } .buttonStyle(.borderless) .help("下移") + .accessibilityLabel("下移 \(device.name)") } } private var audioPane: some View { Form { - Section("麦克风") { + Section { Toggle("使用系统默认", isOn: $microphoneUseSystemDefault) .onChange(of: microphoneUseSystemDefault) { _, _ in persistMicrophoneSelection() } @@ -378,22 +379,18 @@ struct SettingsView: View { .disabled(microphoneUseSystemDefault) } - HStack { - Button("刷新设备") { refreshMicrophones() } - Spacer() - if let active = activeMicrophoneName { - Text("当前：\(active)") - .font(.callout) - .foregroundStyle(.secondary) - } + if let active = activeMicrophoneName { + LabeledContent("当前", value: active) } - Text("关闭「使用系统默认」后，豆沙会按上方顺序尝试麦克风；内建或外接麦克风会优先尝试。下一次录音起生效。") - .font(.callout) - .foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) + Button("刷新设备") { refreshMicrophones() } + } header: { + Text("麦克风") + } footer: { + sectionFooter("关闭系统默认后，按上方顺序选择麦克风，下一次录音生效。", + details: "关闭「使用系统默认」后，豆沙会按上方顺序尝试麦克风；内建或外接麦克风会优先尝试。下一次录音起生效。") } - Section("音频和反馈") { + Section { Toggle("录音时静音", isOn: $muteSystemAudioDuringRecording) .onChange(of: muteSystemAudioDuringRecording) { _, newValue in Preferences.shared.muteSystemAudioDuringRecording = newValue @@ -402,10 +399,11 @@ struct SettingsView: View { .onChange(of: pauseMediaDuringRecording) { _, newValue in Preferences.shared.pauseMediaDuringRecording = newValue } - Text("录音时静音会临时静音系统输出并在结束后恢复；暂停媒体会在系统 Now Playing 显示正在播放时暂停，并在录音结束后恢复，未播放时不会启动媒体。少数不发布 Now Playing 状态的应用或网页可能不会被暂停，可同时开启录音时静音作为兜底。系统语音隔离当前不作为可用选项暴露，避免再次切到系统播放/多通道输入。") - .font(.callout) - .foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) + } header: { + Text("音频和反馈") + } footer: { + sectionFooter("录音时静音或暂停正在播放的媒体，结束后恢复。", + details: "录音时静音会临时静音系统输出并在结束后恢复；暂停媒体会在系统 Now Playing 显示正在播放时暂停，并在录音结束后恢复，未播放时不会启动媒体。少数不发布 Now Playing 状态的应用或网页可能不会被暂停，可同时开启录音时静音作为兜底。系统语音隔离当前不作为可用选项暴露，避免再次切到系统播放/多通道输入。") } } .formStyle(.grouped) @@ -413,11 +411,36 @@ struct SettingsView: View { } private var modelPane: some View { - ScrollView { - modelPaneContent - .padding(20) + Form { + Section { + Picker("主要语言", selection: $primaryLanguage) { + Text("中文").tag(Language.zh_CN) + Text("英文").tag(Language.en_US) + } + .onChange(of: primaryLanguage) { _, newValue in + Preferences.shared.language = newValue.rawValue + NotificationCenter.default.post(name: .doushaEngineRoutingChanged, object: nil) + } + Picker("中文", selection: $chineseEngine) { engineOptions() } + .onChange(of: chineseEngine) { _, _ in persistRouting() } + Picker("英文", selection: $englishEngine) { engineOptions() } + .onChange(of: englishEngine) { _, _ in persistRouting() } + Picker("中英混合", selection: $mixedEngine) { engineOptions() } + .onChange(of: mixedEngine) { _, _ in persistRouting() } + } header: { + Text("引擎路由") + } footer: { + sectionFooter("用到的引擎会在录音时并行运行，停录后按语言挑选结果。", + details: "按语言把识别结果路由到对应引擎。用到的引擎会在录音时并行运行——例如「中文→豆包、英文→Soniox」就会同时跑两家，停录后按整段语言占比挑一家的结果。三个都选同一个引擎即单引擎、零额外开销。主要语言（菜单「语言」）对应的引擎为主引擎，其实时字幕显示在悬浮窗。") + } + Section("引擎") { + engineRow(.qwen3) + engineRow(.soniox) + engineRow(.doubao) + engineRow(.apple) + } } - .background(Color(nsColor: .windowBackgroundColor)) + .formStyle(.grouped) .task { refreshEnginePreferences() await observeModelInventory() @@ -427,133 +450,72 @@ struct SettingsView: View { } } - // Computed fragments only: all bindings and tasks remain owned by SettingsView. - private var modelPaneContent: some View { - VStack(alignment: .leading, spacing: 14) { - routingCard - engineCards + private func engineSummary(_ engine: Engine) -> String { + let summary: String + switch engine { + case .qwen3: + summary = "本地 · \(selectedQwen3Model.displayName)" + case .soniox: + summary = "云端 · \(sonioxMode == .realtime ? "实时" : "高精度（异步）") · \(Preferences.shared.sonioxAPIKey.isEmpty ? "未配置密钥" : "密钥已配置")" + case .doubao: + summary = "云端 · 凭据自动管理" + case .apple: + summary = "本地 · 系统内置 · 语言在菜单中选择" } - .frame(maxWidth: .infinity, alignment: .leading) + return summary + (activeEngines.contains(engine) ? " · 使用中" : "") } - private var engineCards: some View { - VStack(alignment: .leading, spacing: 14) { - engineCard(.qwen3, summary: qwen3Summary) { qwen3Settings } - engineCard(.soniox, summary: "\(sonioxMode == .realtime ? "实时" : "高精度（异步）") · \(Preferences.shared.sonioxAPIKey.isEmpty ? "未配置密钥" : "密钥已配置")") { - sonioxSettings - } - engineCard(.doubao, summary: "实时识别 · 凭据自动管理") { - VStack(alignment: .leading, spacing: 10) { - Button("重置豆包凭据…") { actions.resetDoubaoCredentials() } - Text("删除缓存的 device_id 与 token，下次听写时重新注册。遇到 exceedconcurrentquota 等错误时使用。") - .font(.callout) - .foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) - } - } - engineCard(.apple, summary: "系统内置 · 无需配置") { - Text("使用系统内置语音识别，无需额外配置。语言在菜单「语言」中选择。") - .font(.callout) + private func engineRow(_ engine: Engine) -> some View { + HStack(spacing: 10) { + Image(systemName: engine.symbol) + .foregroundStyle(.secondary) + .accessibilityHidden(true) + VStack(alignment: .leading, spacing: 2) { + Text(engine == .apple ? "Apple 听写" : engine.displayName) + Text(engineSummary(engine)) + .font(.caption) .foregroundStyle(.secondary) .fixedSize(horizontal: false, vertical: true) } - } - .frame(maxWidth: .infinity, alignment: .leading) - } - - private var routingCard: some View { - VStack(alignment: .leading, spacing: 12) { - Text("引擎路由").font(.headline) - Picker("主要语言", selection: $primaryLanguage) { - Text("中文").tag(Language.zh_CN) - Text("英文").tag(Language.en_US) - } - .pickerStyle(.segmented) - .onChange(of: primaryLanguage) { _, newValue in - Preferences.shared.language = newValue.rawValue - NotificationCenter.default.post(name: .doushaEngineRoutingChanged, object: nil) + Spacer() + if engine != .apple { + Button("选项…") { selectedEngineSheet = SettingsEngineSheet(engine: engine) } + .accessibilityLabel("\(engine.displayName)，选项") + .accessibilityValue(engineSummary(engine)) } - Picker("中文", selection: $chineseEngine) { engineOptions() } - .onChange(of: chineseEngine) { _, _ in persistRouting() } - Picker("英文", selection: $englishEngine) { engineOptions() } - .onChange(of: englishEngine) { _, _ in persistRouting() } - Picker("中英混合", selection: $mixedEngine) { engineOptions() } - .onChange(of: mixedEngine) { _, _ in persistRouting() } - Text("按语言把识别结果路由到对应引擎。用到的引擎会在录音时并行运行——例如「中文→豆包、英文→Soniox」就会同时跑两家，停录后按整段语言占比挑一家的结果。三个都选同一个引擎即单引擎、零额外开销。主要语言（菜单「语言」）对应的引擎为主引擎，其实时字幕显示在悬浮窗。") - .font(.callout) - .foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) } - .padding(16) - .background(Color(nsColor: .controlBackgroundColor), in: RoundedRectangle(cornerRadius: 12)) - .overlay(RoundedRectangle(cornerRadius: 12).strokeBorder(.quaternary, lineWidth: 1)) + .help(engine == .apple ? "使用系统内置语音识别，无需额外配置。语言在菜单「语言」中选择。" : engineSummary(engine)) } - private func engineCard(_ engine: Engine, summary: String, - @ViewBuilder content: () -> Content) -> some View { - let isExpanded = expandedEngines.contains(engine) - return VStack(alignment: .leading, spacing: 0) { - Button { - if isExpanded { expandedEngines.remove(engine) } - else { expandedEngines.insert(engine) } - } label: { - VStack(alignment: .leading, spacing: 10) { - HStack(spacing: 10) { - Image(systemName: engine.symbol) - .font(.system(size: 19, weight: .medium)) - .foregroundStyle(engine == .qwen3 ? Color.accentColor : .secondary) - .frame(width: 36, height: 36) - .background(.quaternary.opacity(0.5), in: RoundedRectangle(cornerRadius: 9)) - Text(engine.displayName).font(.headline) - capsule(engine.kind) - Spacer(minLength: 8) - Image(systemName: isExpanded ? "chevron.up" : "chevron.down") - .font(.caption.weight(.semibold)) - .foregroundStyle(.secondary) - } - Text(summary) - .font(.callout) - .foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) - HStack(spacing: 6) { - ForEach(engine.tags, id: \.self) { capsule($0) } - if activeEngines.contains(engine) { capsule("使用中", active: true) } - } + // Computed fragments only: sheet presentation and every editable binding + // belong to SettingsView, just like the six navigation panes. + private func engineSheet(_ engine: Engine) -> some View { + VStack(spacing: 0) { + Form { + switch engine { + case .qwen3: qwen3Settings + case .soniox: sonioxSettings + case .doubao: doubaoSettings + case .apple: EmptyView() } - .frame(maxWidth: .infinity, alignment: .leading) - .contentShape(Rectangle()) - .padding(16) } - .buttonStyle(.plain) - .accessibilityLabel("\(engine.displayName)，\(isExpanded ? "收起" : "展开")设置") - .accessibilityValue(summary) - if isExpanded { - Divider().padding(.horizontal, 16) - content() - .frame(maxWidth: .infinity, alignment: .leading) - .padding(16) + .formStyle(.grouped) + HStack { + Spacer() + Button("完成") { selectedEngineSheet = nil } + .keyboardShortcut(.defaultAction) } + .padding() } - .background(Color(nsColor: .controlBackgroundColor), in: RoundedRectangle(cornerRadius: 12)) - .overlay(RoundedRectangle(cornerRadius: 12).strokeBorder(.quaternary, lineWidth: 1)) - } - - private func capsule(_ title: String, active: Bool = false) -> some View { - Text(title) - .font(.caption) - .foregroundStyle(active ? Color.accentColor : .secondary) - .padding(.horizontal, 8) - .padding(.vertical, 3) - .background(active ? Color.accentColor.opacity(0.1) : Color.secondary.opacity(0.08), in: Capsule()) + .frame(width: 560, height: engine == .doubao ? 240 : 420) } private var sonioxSettings: some View { - VStack(alignment: .leading, spacing: 12) { + Section { Picker("模式", selection: $sonioxMode) { Text("实时").tag(SonioxMode.realtime) Text("高精度（异步）").tag(SonioxMode.async) } - .pickerStyle(.segmented) .onChange(of: sonioxMode) { _, newValue in Preferences.shared.sonioxMode = newValue } @@ -561,33 +523,41 @@ struct SettingsView: View { HStack(spacing: 6) { SecureField("", text: $sonioxAPIKey) .disableAutocorrection(true) - Button("清除") { sonioxAPIKey = "" } + .accessibilityLabel("Soniox API 密钥") + Button("清除", role: .destructive) { sonioxAPIKey = "" } .help("删除已保存的 Soniox API 密钥") } } HStack { Spacer() - testStatusLabel(isTesting: isTestingSoniox, - status: sonioxStatus, - isError: sonioxStatusIsError) Button("保存并测试") { saveAndTestSoniox() } .disabled(isTestingSoniox || sonioxAPIKey.isEmpty) } - Text("Soniox 实时语音转写引擎。自动检测语言，无需选择。在菜单「引擎」中切换到 Soniox 后生效。") - .font(.callout) - .foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) + if isTestingSoniox || !sonioxStatus.isEmpty { + statusLabel(isTestingSoniox ? "测试中…" : sonioxStatus, + isError: sonioxStatusIsError, isProgress: isTestingSoniox) + } + } header: { + Text("Soniox") + } footer: { + sectionFooter("Soniox 自动检测语言，无需手动选择。", + details: "Soniox 实时语音转写引擎。自动检测语言，无需选择。在菜单「引擎」中切换到 Soniox 后生效。") } - .textFieldStyle(.roundedBorder) } - private var selectedQwen3Model: MLXModelDescriptor { - MLXModelCatalog.all.first { $0.repoID == qwen3ModelRepoID } ?? MLXModelCatalog.defaultModel + private var doubaoSettings: some View { + Section { + Button("重置豆包凭据…", role: .destructive) { actions.resetDoubaoCredentials() } + } header: { + Text("豆包") + } footer: { + sectionFooter("重置后，下次听写时自动重新注册凭据。", + details: "删除缓存的 device_id 与 token，下次听写时重新注册。遇到 exceedconcurrentquota 等错误时使用。") + } } - private var qwen3Summary: String { - let installed = modelInventory.values.filter(\.isInstalled).count - return "模型 \(MLXModelCatalog.all.count) · 已安装 \(installed)/\(MLXModelCatalog.all.count) · 使用中 \(selectedQwen3Model.displayName)" + private var selectedQwen3Model: MLXModelDescriptor { + MLXModelCatalog.all.first { $0.repoID == qwen3ModelRepoID } ?? MLXModelCatalog.defaultModel } private var isModelOperationPending: Bool { @@ -596,25 +566,18 @@ struct SettingsView: View { } private var qwen3Settings: some View { - VStack(alignment: .leading, spacing: 12) { + Section { ForEach(MLXModelCatalog.all, id: \.repoID) { descriptor in - if descriptor != MLXModelCatalog.all.first { Divider() } qwen3ModelRow(descriptor) } - Text("安装会下载并加载模型，首次可能需要几分钟；下载完成后可离线听写。切换模型在下一次录音生效。") - .font(.callout) - .foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) - Text("已加载的模型暂不可卸载。请先使用另一个模型并等待加载完成，或重新打开豆沙后卸载。") - .font(.caption) - .foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) if !modelManagementError.isEmpty { - Text(modelManagementError) - .font(.callout) - .foregroundStyle(.red) - .fixedSize(horizontal: false, vertical: true) + statusLabel(modelManagementError, isError: true) } + } header: { + Text("Qwen3 模型") + } footer: { + sectionFooter("下载后可离线听写，切换模型在下一次录音生效。", + details: "安装会下载并加载模型，首次可能需要几分钟；下载完成后可离线听写。切换模型在下一次录音生效。\n\n已加载的模型暂不可卸载。请先使用另一个模型并等待加载完成，或重新打开豆沙后卸载。") } } @@ -624,43 +587,49 @@ struct SettingsView: View { let selected = qwen3ModelRepoID == descriptor.repoID let loading = modelStoreState == .loading(repoID: descriptor.repoID) let loaded = modelStoreState == .ready(repoID: descriptor.repoID) + let pending = loading || requestedModelRepoID == descriptor.repoID || uninstallingModelRepoID == descriptor.repoID return VStack(alignment: .leading, spacing: 8) { - HStack(spacing: 8) { - VStack(alignment: .leading, spacing: 5) { - Text(descriptor.displayName).font(.body.weight(.medium)) - Text(modelSizeLabel(descriptor, entry: entry)) - .font(.caption) - .foregroundStyle(.secondary) + HStack(spacing: 12) { + Button { selectModel(descriptor) } label: { + HStack(spacing: 8) { + Image(systemName: selected ? "checkmark.circle" : "circle") + .foregroundStyle(.secondary) + VStack(alignment: .leading, spacing: 2) { + Text(descriptor.displayName) + .foregroundStyle(.primary) + Text(modelSizeLabel(descriptor, entry: entry) + (selected ? " · 使用中" : "")) + .font(.caption) + .foregroundStyle(.secondary) + } + } } + .buttonStyle(.borderless) + .disabled(!installed || selected || isModelOperationPending) + .accessibilityLabel("使用 \(descriptor.displayName)") + .accessibilityValue(selected ? "使用中" : (installed ? "已安装" : "尚未安装")) Spacer(minLength: 4) - if selected { capsule("使用中", active: true) } + if pending { + statusLabel(uninstallingModelRepoID == descriptor.repoID ? "正在删除…" : (loading ? "下载并加载中…" : "等待模型空闲…"), + isProgress: true) + } else if !installed { + Button { installModel(descriptor) } label: { + Image(systemName: "arrow.down.circle") + } + .buttonStyle(.borderless) + .disabled(isModelOperationPending) + .help("下载 \(descriptor.displayName)") + .accessibilityLabel("下载 \(descriptor.displayName)") + } if installed { - Button("卸载") { uninstallModel(descriptor) } + Button("删除", role: .destructive) { uninstallModel(descriptor) } .disabled(isModelOperationPending || loaded) .help(loaded ? "模型仍已加载，请先切换模型或重新打开豆沙" : "删除此模型的本地缓存") - if !selected { - Button("使用") { selectModel(descriptor) } - .disabled(isModelOperationPending) - } - } else { - Button("安装") { installModel(descriptor) } - .disabled(isModelOperationPending) - } - } - if loading || requestedModelRepoID == descriptor.repoID || uninstallingModelRepoID == descriptor.repoID { - HStack(spacing: 8) { - ProgressView().controlSize(.small) - Text(uninstallingModelRepoID == descriptor.repoID ? "正在卸载…" : (loading ? "下载并加载中…" : "等待模型空闲…")) - .font(.caption) - .foregroundStyle(.secondary) + .accessibilityLabel("删除 \(descriptor.displayName)") } } if case .failed(let repoID, let message) = modelStoreState, repoID == descriptor.repoID { HStack(alignment: .top) { - Text("加载失败：\(message)") - .font(.caption) - .foregroundStyle(.red) - .fixedSize(horizontal: false, vertical: true) + statusLabel("加载失败：\(message)", isError: true) Spacer(minLength: 0) Button("重试") { installModel(descriptor) }.disabled(isModelOperationPending) } @@ -742,48 +711,24 @@ struct SettingsView: View { private var glossaryPane: some View { Form { - Section("个性词库") { + Section { Toggle("启用个性词库", isOn: $glossaryEnabled) .onChange(of: glossaryEnabled) { _, newValue in Preferences.shared.glossaryEnabled = newValue } - - // Free-text editor: paste / edit a batch, separated by 顿号 / 逗号 / - // 换行. Parsed into terms on change; we deliberately don't rewrite - // the text while editing, so the cursor never jumps. The editor is - // gated by the toggle; the toggle itself stays enabled. - VStack(alignment: .leading, spacing: 6) { - ZStack(alignment: .topLeading) { - TextEditor(text: $glossaryText) - .font(.body) - .frame(height: 200) - .scrollContentBackground(.hidden) - .padding(4) - .overlay( - RoundedRectangle(cornerRadius: 6) - .stroke(Color(nsColor: .separatorColor)) - ) - .onChange(of: glossaryText) { _, newValue in - Preferences.shared.glossaryTerms = GlossaryText.parse(newValue) - } - if glossaryText.isEmpty { - Text("输入关键词、专业术语…\n示例：布迪厄、哈贝马斯、福柯") - .foregroundStyle(.tertiary) - .padding(.top, 12) - .padding(.leading, 9) - .allowsHitTesting(false) - } + // Parse without rewriting the draft so the insertion point stays put. + settingsEditor("个性词库", text: $glossaryText, + placeholder: "输入关键词、专业术语…\n示例：布迪厄、哈贝马斯、福柯") + .onChange(of: glossaryText) { _, newValue in + Preferences.shared.glossaryTerms = GlossaryText.parse(newValue) } - Text("共 \(GlossaryText.parse(glossaryText).count) 个术语") - .font(.caption) - .foregroundStyle(.secondary) - } - .disabled(!glossaryEnabled) - - Text("把常用术语、专有名词加进词库，用顿号、逗号或换行分隔；识别时会优先往这些词上靠，提高准确率。对豆包和 Soniox 引擎都生效，下一次录音起作用。Soniox 用词表（terms）偏置，效果通常更明显。") - .font(.callout) - .foregroundColor(.secondary) - .fixedSize(horizontal: false, vertical: true) + .disabled(!glossaryEnabled) + LabeledContent("术语数量", value: "共 \(GlossaryText.parse(glossaryText).count) 个术语") + } header: { + Text("个性词库") + } footer: { + sectionFooter("用顿号、逗号或换行分隔术语，下一次录音生效。", + details: "把常用术语、专有名词加进词库，用顿号、逗号或换行分隔；识别时会优先往这些词上靠，提高准确率。对豆包和 Soniox 引擎都生效，下一次录音起作用。Soniox 用词表（terms）偏置，效果通常更明显。") } } .formStyle(.grouped) @@ -793,7 +738,7 @@ struct SettingsView: View { private var enhancePane: some View { Form { - Section("文本格式") { + Section { Picker("引号", selection: $quoteStyle) { Text("中文用「」").tag(TranscriptCorrector.QuoteStyle.corner) Text("中文用“”").tag(TranscriptCorrector.QuoteStyle.curly) @@ -803,13 +748,14 @@ struct SettingsView: View { .onChange(of: quoteStyle) { _, newValue in Preferences.shared.quoteStyle = newValue } - Text("把听写出来的直引号配对成规范引号。英文始终用 “ ”，撇号写成 don’t；上面这项只决定中文语境用哪套。") - .font(.callout) - .foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) + } header: { + Text("文本格式") + } footer: { + sectionFooter("将听写中的直引号配对成规范引号。", + details: "把听写出来的直引号配对成规范引号。英文始终用 “ ”，撇号写成 don’t；上面这项只决定中文语境用哪套。") } - Section("润色") { + Section { Toggle("启用润色", isOn: $llmEnabled) .onChange(of: llmEnabled) { _, newValue in Preferences.shared.llmEnabled = newValue @@ -822,7 +768,8 @@ struct SettingsView: View { HStack(spacing: 6) { SecureField("", text: $apiKey, prompt: Text("sk-…")) .disableAutocorrection(true) - Button("清除") { apiKey = "" } + .accessibilityLabel("润色 API 密钥") + Button("清除", role: .destructive) { apiKey = "" } .help("删除已保存的 API 密钥") } } @@ -839,41 +786,37 @@ struct SettingsView: View { } HStack { Spacer() - testStatusLabel(isTesting: isTesting, - status: status, - isError: statusIsError) Button("保存并测试") { saveAndTestLLM() } .disabled(isTesting || apiKey.isEmpty || baseURL.isEmpty || model.isEmpty) } - Text("使用任意兼容 OpenAI 的对话补全接口来清理听写结果。默认仅修正明显的识别错误；可在下方自定义润色规则。") - .font(.callout) - .foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) + if isTesting || !status.isEmpty { + statusLabel(isTesting ? "测试中…" : status, + isError: statusIsError, isProgress: isTesting) + } + } header: { + Text("润色") + } footer: { + sectionFooter("使用兼容 OpenAI 的接口，按润色规则清理听写结果。", + details: "使用任意兼容 OpenAI 的对话补全接口来清理听写结果。默认仅修正明显的识别错误；可在下方自定义润色规则。") } - Section("润色提示词") { - TextEditor(text: $llmSystemPrompt) - .font(.body) + Section { + settingsEditor("润色提示词", text: $llmSystemPrompt, + placeholder: "输入润色规则…\n留空则使用默认规则。") .disableAutocorrection(true) - .frame(height: 220) - .padding(6) - .background(.background, in: RoundedRectangle(cornerRadius: 6)) - .overlay(RoundedRectangle(cornerRadius: 6).stroke(.quaternary)) - .accessibilityLabel("润色提示词") + LabeledContent("保存状态", value: llmSystemPrompt == savedLLMSystemPrompt ? "已保存" : "未保存") HStack { - Button("恢复默认") { llmSystemPrompt = TextRefiner.defaultSystemPrompt } + Button("恢复默认", role: .destructive) { llmSystemPrompt = TextRefiner.defaultSystemPrompt } .disabled(llmSystemPrompt == TextRefiner.defaultSystemPrompt) Spacer() - Text(llmSystemPrompt == savedLLMSystemPrompt ? "已保存" : "未保存") - .font(.callout) - .foregroundStyle(.secondary) Button("保存提示词") { saveLLMPrompt() } .disabled(llmSystemPrompt == savedLLMSystemPrompt) } - Text("自定义提示词会完整替换默认规则，可允许清理重复、修复断句或轻度润色。建议要求模型只输出处理后的正文。修改后点击「保存提示词」，后续润色请求生效；留空则使用默认规则。") - .font(.callout) - .foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) + } header: { + Text("润色提示词") + } footer: { + sectionFooter("保存后用于后续润色，留空则使用默认规则。", + details: "自定义提示词会完整替换默认规则，可允许清理重复、修复断句或轻度润色。建议要求模型只输出处理后的正文。修改后点击「保存提示词」，后续润色请求生效；留空则使用默认规则。") } } .formStyle(.grouped) @@ -889,13 +832,14 @@ struct SettingsView: View { Preferences.shared.historyLimit = newValue history.prune(limit: newValue) } + } header: { + Text("存储") + } footer: { Text("每次听写的原始录音和转录文本保存在本机缓存目录，超出条数自动删除最旧的。") - .font(.caption) - .foregroundColor(.secondary) } Section("最近录音") { if historyEntries.isEmpty { - Text("暂无录音").foregroundColor(.secondary) + Text("暂无录音").foregroundStyle(.secondary) } ForEach(historyEntries) { entry in historyRow(entry) @@ -926,20 +870,17 @@ struct SettingsView: View { HStack(spacing: 8) { VStack(alignment: .leading, spacing: 2) { HStack(spacing: 6) { - Text(entry.date, format: .dateTime.month().day().hour().minute()) - Text(Self.durationLabel(entry.duration)).foregroundColor(.secondary) - if !entry.engine.isEmpty { - Text(entry.engine).foregroundColor(.secondary) - } - if let err = entry.error { - Text("转录失败：\(err)").foregroundColor(.red).lineLimit(1) + Text(entry.transcript.isEmpty ? "（无文本）" : entry.transcript) + .lineLimit(1) + .truncationMode(.tail) + .foregroundStyle(entry.transcript.isEmpty ? .secondary : .primary) + if let error = entry.error { + statusLabel("转录失败：\(error)", isError: true, symbolOnly: true) } } - .font(.caption) - Text(entry.transcript.isEmpty ? "（无文本）" : entry.transcript) - .lineLimit(1) - .truncationMode(.tail) - .foregroundColor(entry.transcript.isEmpty ? .secondary : .primary) + Text(historyMetadata(entry)) + .font(.caption) + .foregroundStyle(.secondary) } Spacer() Button("重新转录") { actions.retranscribe(entry.id) } @@ -951,17 +892,28 @@ struct SettingsView: View { } label: { Image(systemName: "doc.on.doc") } + .buttonStyle(.borderless) .help("复制文本") + .accessibilityLabel("复制文本") .disabled(entry.transcript.isEmpty) Button { NSWorkspace.shared.activateFileViewerSelecting([history.wavURL(id: entry.id)]) } label: { Image(systemName: "folder") } + .buttonStyle(.borderless) .help("在 Finder 中显示") + .accessibilityLabel("在 Finder 中显示") } } + private func historyMetadata(_ entry: RecordingHistoryEntry) -> String { + var components = [entry.date.formatted(.dateTime.month().day().hour().minute()), + Self.durationLabel(entry.duration)] + if !entry.engine.isEmpty { components.append(entry.engine) } + return components.joined(separator: " · ") + } + private static func durationLabel(_ d: TimeInterval) -> String { let s = Int(d.rounded()) return String(format: "%d:%02d", s / 60, s % 60) @@ -988,24 +940,67 @@ struct SettingsView: View { // MARK: - 润色 / Soniox 操作 - /// Persistent, single-line test result shown beside the 「保存并测试」 button: - /// a spinner while testing, then a coloured icon + short text. - @ViewBuilder - private func testStatusLabel(isTesting: Bool, status: String, isError: Bool) -> some View { - if isTesting { - HStack(spacing: 4) { + // MARK: - Shared native presentation + + /// All result/progress text stays secondary; only the status symbol is coloured. + private func statusLabel(_ message: String, isError: Bool = false, + isProgress: Bool = false, symbolOnly: Bool = false) -> some View { + HStack(alignment: .firstTextBaseline, spacing: 6) { + if isProgress { ProgressView().controlSize(.small) - Text("测试中…").foregroundStyle(.secondary) + } else { + Image(systemName: isError ? "exclamationmark.triangle.fill" : "checkmark.circle.fill") + .foregroundStyle(isError ? .orange : .green) } - .font(.callout) - } else if !status.isEmpty { - HStack(spacing: 4) { - Image(systemName: isError ? "xmark.circle.fill" : "checkmark.circle.fill") - Text(status).lineLimit(1) + if !symbolOnly { + Text(message) + .foregroundStyle(.secondary) + .fixedSize(horizontal: false, vertical: true) + } + } + .help(message) + .accessibilityElement(children: .ignore) + .accessibilityLabel(message) + } + + /// Form supplies the native editor surface; neither editor adds a custom border. + private func settingsEditor(_ title: String, text: Binding, placeholder: String) -> some View { + ZStack(alignment: .topLeading) { + TextEditor(text: text) + .font(.body) + .frame(height: 200) + .accessibilityLabel(title) + if text.wrappedValue.isEmpty { + Text(placeholder) + .foregroundStyle(.tertiary) + .padding(.top, 8) + .padding(.leading, 5) + .allowsHitTesting(false) + .accessibilityHidden(true) + } + } + } + + /// The popover's presentation lives here too, so helper fragments have no state. + private func sectionFooter(_ summary: String, details: String) -> some View { + HStack(alignment: .firstTextBaseline, spacing: 4) { + Text(summary) + Button { presentedHelp = details } label: { + Image(systemName: "info.circle") + } + .buttonStyle(.borderless) + .accessibilityLabel("更多说明：\(summary)") + .help(details) + .popover(isPresented: Binding( + get: { presentedHelp == details }, + set: { if !$0 { presentedHelp = nil } } + )) { + Text(details) + .font(.body) + .padding() + .frame(width: 340) + .fixedSize(horizontal: false, vertical: true) } - .font(.callout) - .foregroundStyle(isError ? .red : .green) - .help(status) } } From e57842f69cdb1ea4c9caab69de1baee14953c342 Mon Sep 17 00:00:00 2001 From: giraphant Date: Mon, 12 Oct 2026 17:02:25 +0100 Subject: [PATCH 3/6] test(mlx): make duplicate-stop completion assertion order-independent --- Tests/DoushaTests/MLXFinalPassTests.swift | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/Tests/DoushaTests/MLXFinalPassTests.swift b/Tests/DoushaTests/MLXFinalPassTests.swift index ebe70fc..7cfeaff 100644 --- a/Tests/DoushaTests/MLXFinalPassTests.swift +++ b/Tests/DoushaTests/MLXFinalPassTests.swift @@ -195,13 +195,17 @@ final class MLXFinalPassTests: XCTestCase { } await prepare(engine) let completed = expectation(description: "one fallback completion") + completed.expectedFulfillmentCount = 1 + completed.assertForOverFulfill = true let count = Lock(0) - engine.stop { result in + let completion: @Sendable (TranscriptionResult) -> Void = { result in XCTAssertEqual(result.text, "preview") count.withLock { $0 += 1 } completed.fulfill() } - engine.stop { _ in XCTFail("Duplicate stop emitted a completion") } + // Nonisolated stop calls schedule unordered tasks; either callback may win. + engine.stop(completion: completion) + engine.stop(completion: completion) _ = try await started.wait() await fulfillment(of: [completed], timeout: 1) let store = MLXModelStore() From 0476da4c6fffb8536c1260fabd066723b1a2904f Mon Sep 17 00:00:00 2001 From: giraphant Date: Mon, 12 Oct 2026 17:13:55 +0100 Subject: [PATCH 4/6] refactor(settings): move footer guidance into native rows --- Sources/Apps/Dousha/SettingsWindow.swift | 274 +++++++++++++---------- 1 file changed, 153 insertions(+), 121 deletions(-) diff --git a/Sources/Apps/Dousha/SettingsWindow.swift b/Sources/Apps/Dousha/SettingsWindow.swift index f692203..2feda09 100644 --- a/Sources/Apps/Dousha/SettingsWindow.swift +++ b/Sources/Apps/Dousha/SettingsWindow.swift @@ -198,7 +198,7 @@ struct SettingsView: View { private var generalPane: some View { Form { - Section("启动行为") { + Section("启动与显示") { Toggle("开机时启动豆沙", isOn: $launchAtLogin) .onChange(of: launchAtLogin) { _, newValue in applyLaunchAtLogin(newValue) @@ -206,24 +206,20 @@ struct SettingsView: View { if !launchAtLoginError.isEmpty { statusLabel(launchAtLoginError, isError: true) } - } - - Section { Toggle("显示程序坞图标", isOn: $showDockIcon) .onChange(of: showDockIcon) { _, newValue in actions.setDockIconVisible(newValue) } - Toggle("显示状态栏图标", isOn: $showMenuBarIcon) - .onChange(of: showMenuBarIcon) { _, newValue in - actions.setMenuBarIconVisible(newValue) - } - } header: { - Text("显示") - } footer: { - Text("两者都隐藏后，可从 Finder 或聚焦搜索重新打开豆沙以再次显示本窗口。") + Toggle(isOn: $showMenuBarIcon) { + Text("显示状态栏图标") + Text("均隐藏后，从 Finder 或聚焦搜索重开豆沙") + } + .onChange(of: showMenuBarIcon) { _, newValue in + actions.setMenuBarIconVisible(newValue) + } } - Section { + Section("热键") { LabeledContent("触发键") { HStack(spacing: 8) { Text(HotkeyMonitor.displayName(forKeyCode: hotkeyKeyCode)) @@ -231,22 +227,15 @@ struct SettingsView: View { } } Picker("模式", selection: $hotkeyMode) { - Text("长按说话").tag(HotkeyMode.pushToTalk) - Text("切换").tag(HotkeyMode.toggle) + Text("长按说话（按住录音）").tag(HotkeyMode.pushToTalk) + Text("切换（按一下开始／停止）").tag(HotkeyMode.toggle) } .onChange(of: hotkeyMode) { _, newMode in let cfg = HotkeyConfig(keyCode: Preferences.shared.hotkey.keyCode, mode: newMode) Preferences.shared.hotkey = cfg NotificationCenter.default.post(name: .doushaHotkeyConfigChanged, object: nil) } - } header: { - Text("热键") - } footer: { - Text("长按说话：按住时录音；切换：每次按下开始/停止。") - } - - Section { - LabeledContent("取消键") { + LabeledContent { HStack(spacing: 8) { Text(cancelHotkey.displayName) Button(isRecordingCancelHotkey ? "按任意键…" : "录制") { toggleCancelHotkeyRecording() } @@ -256,13 +245,13 @@ struct SettingsView: View { NotificationCenter.default.post(name: .doushaCancelHotkeyConfigChanged, object: nil) } .disabled(!cancelHotkey.isEnabled) + settingsHelp("取消键仅在录音时丢弃当前录音且不转写。", + details: "取消会丢弃当前录音且不转写。仅在录音时生效，其余情况下按键正常透传。") } + } label: { + Text("取消键") + Text("仅在录音时丢弃，不转写") } - } header: { - Text("取消") - } footer: { - sectionFooter("取消键仅在录音时丢弃当前录音且不转写。", - details: "取消会丢弃当前录音且不转写。仅在录音时生效，其余情况下按键正常透传。") } } .formStyle(.grouped) @@ -366,8 +355,11 @@ struct SettingsView: View { private var audioPane: some View { Form { Section { - Toggle("使用系统默认", isOn: $microphoneUseSystemDefault) - .onChange(of: microphoneUseSystemDefault) { _, _ in persistMicrophoneSelection() } + Toggle(isOn: $microphoneUseSystemDefault) { + Text("使用系统默认") + Text("关闭后按下方顺序选择，下次录音生效") + } + .onChange(of: microphoneUseSystemDefault) { _, _ in persistMicrophoneSelection() } if orderedMicrophones.isEmpty { Text("未找到可用麦克风。") @@ -384,26 +376,34 @@ struct SettingsView: View { } Button("刷新设备") { refreshMicrophones() } } header: { - Text("麦克风") - } footer: { - sectionFooter("关闭系统默认后，按上方顺序选择麦克风，下一次录音生效。", - details: "关闭「使用系统默认」后，豆沙会按上方顺序尝试麦克风；内建或外接麦克风会优先尝试。下一次录音起生效。") + HStack { + Text("麦克风") + settingsHelp("关闭系统默认后，按上方顺序选择麦克风，下一次录音生效。", + details: "关闭「使用系统默认」后，豆沙会按列表顺序尝试麦克风；内建或外接麦克风会优先尝试。下一次录音起生效。") + } } Section { - Toggle("录音时静音", isOn: $muteSystemAudioDuringRecording) - .onChange(of: muteSystemAudioDuringRecording) { _, newValue in - Preferences.shared.muteSystemAudioDuringRecording = newValue - } - Toggle("录音时暂停媒体", isOn: $pauseMediaDuringRecording) - .onChange(of: pauseMediaDuringRecording) { _, newValue in - Preferences.shared.pauseMediaDuringRecording = newValue - } + Toggle(isOn: $muteSystemAudioDuringRecording) { + Text("录音时静音") + Text("临时静音系统输出，结束后恢复") + } + .onChange(of: muteSystemAudioDuringRecording) { _, newValue in + Preferences.shared.muteSystemAudioDuringRecording = newValue + } + Toggle(isOn: $pauseMediaDuringRecording) { + Text("录音时暂停媒体") + Text("暂停正在播放的媒体，结束后恢复") + } + .onChange(of: pauseMediaDuringRecording) { _, newValue in + Preferences.shared.pauseMediaDuringRecording = newValue + } } header: { - Text("音频和反馈") - } footer: { - sectionFooter("录音时静音或暂停正在播放的媒体，结束后恢复。", - details: "录音时静音会临时静音系统输出并在结束后恢复；暂停媒体会在系统 Now Playing 显示正在播放时暂停，并在录音结束后恢复，未播放时不会启动媒体。少数不发布 Now Playing 状态的应用或网页可能不会被暂停，可同时开启录音时静音作为兜底。系统语音隔离当前不作为可用选项暴露，避免再次切到系统播放/多通道输入。") + HStack { + Text("音频和反馈") + settingsHelp("录音时静音或暂停正在播放的媒体，结束后恢复。", + details: "录音时静音会临时静音系统输出并在结束后恢复；暂停媒体会在系统 Now Playing 显示正在播放时暂停，并在录音结束后恢复，未播放时不会启动媒体。少数不发布 Now Playing 状态的应用或网页可能不会被暂停，可同时开启录音时静音作为兜底。系统语音隔离当前不作为可用选项暴露，避免再次切到系统播放/多通道输入。") + } } } .formStyle(.grouped) @@ -413,9 +413,12 @@ struct SettingsView: View { private var modelPane: some View { Form { Section { - Picker("主要语言", selection: $primaryLanguage) { + Picker(selection: $primaryLanguage) { Text("中文").tag(Language.zh_CN) Text("英文").tag(Language.en_US) + } label: { + Text("主要语言") + Text("对应引擎提供悬浮窗实时字幕") } .onChange(of: primaryLanguage) { _, newValue in Preferences.shared.language = newValue.rawValue @@ -428,10 +431,11 @@ struct SettingsView: View { Picker("中英混合", selection: $mixedEngine) { engineOptions() } .onChange(of: mixedEngine) { _, _ in persistRouting() } } header: { - Text("引擎路由") - } footer: { - sectionFooter("用到的引擎会在录音时并行运行，停录后按语言挑选结果。", - details: "按语言把识别结果路由到对应引擎。用到的引擎会在录音时并行运行——例如「中文→豆包、英文→Soniox」就会同时跑两家，停录后按整段语言占比挑一家的结果。三个都选同一个引擎即单引擎、零额外开销。主要语言（菜单「语言」）对应的引擎为主引擎，其实时字幕显示在悬浮窗。") + HStack { + Text("引擎路由") + settingsHelp("用到的引擎会在录音时并行运行，停录后按语言挑选结果。", + details: "按语言把识别结果路由到对应引擎。用到的引擎会在录音时并行运行——例如「中文→豆包、英文→Soniox」就会同时跑两家，停录后按整段语言占比挑一家的结果。三个都选同一个引擎即单引擎、零额外开销。主要语言（菜单「语言」）对应的引擎为主引擎，其实时字幕显示在悬浮窗。") + } } Section("引擎") { engineRow(.qwen3) @@ -512,9 +516,12 @@ struct SettingsView: View { private var sonioxSettings: some View { Section { - Picker("模式", selection: $sonioxMode) { + Picker(selection: $sonioxMode) { Text("实时").tag(SonioxMode.realtime) Text("高精度（异步）").tag(SonioxMode.async) + } label: { + Text("模式") + Text("自动检测语言，无需手动选择") } .onChange(of: sonioxMode) { _, newValue in Preferences.shared.sonioxMode = newValue @@ -538,21 +545,22 @@ struct SettingsView: View { isError: sonioxStatusIsError, isProgress: isTestingSoniox) } } header: { - Text("Soniox") - } footer: { - sectionFooter("Soniox 自动检测语言，无需手动选择。", - details: "Soniox 实时语音转写引擎。自动检测语言，无需选择。在菜单「引擎」中切换到 Soniox 后生效。") + HStack { + Text("Soniox") + settingsHelp("Soniox 自动检测语言，无需手动选择。", + details: "Soniox 实时语音转写引擎。自动检测语言，无需选择。在菜单「引擎」中切换到 Soniox 后生效。") + } } } private var doubaoSettings: some View { - Section { - Button("重置豆包凭据…", role: .destructive) { actions.resetDoubaoCredentials() } - } header: { - Text("豆包") - } footer: { - sectionFooter("重置后，下次听写时自动重新注册凭据。", - details: "删除缓存的 device_id 与 token，下次听写时重新注册。遇到 exceedconcurrentquota 等错误时使用。") + Section("豆包") { + HStack { + Button("重置豆包凭据…", role: .destructive) { actions.resetDoubaoCredentials() } + Spacer() + settingsHelp("重置后，下次听写时自动重新注册凭据。", + details: "删除缓存的 device_id 与 token，下次听写时自动重新注册。遇到 exceedconcurrentquota 等错误时使用。") + } } } @@ -574,10 +582,11 @@ struct SettingsView: View { statusLabel(modelManagementError, isError: true) } } header: { - Text("Qwen3 模型") - } footer: { - sectionFooter("下载后可离线听写，切换模型在下一次录音生效。", - details: "安装会下载并加载模型，首次可能需要几分钟；下载完成后可离线听写。切换模型在下一次录音生效。\n\n已加载的模型暂不可卸载。请先使用另一个模型并等待加载完成，或重新打开豆沙后卸载。") + HStack { + Text("Qwen3 模型") + settingsHelp("下载后可离线听写，切换模型在下一次录音生效。", + details: "安装会下载并加载模型，首次可能需要几分钟；下载完成后可离线听写。切换模型在下一次录音生效。\n\n已加载的模型暂不可卸载。请先使用另一个模型并等待加载完成，或重新打开豆沙后卸载。") + } } } @@ -712,10 +721,13 @@ struct SettingsView: View { private var glossaryPane: some View { Form { Section { - Toggle("启用个性词库", isOn: $glossaryEnabled) - .onChange(of: glossaryEnabled) { _, newValue in - Preferences.shared.glossaryEnabled = newValue - } + Toggle(isOn: $glossaryEnabled) { + Text("启用个性词库") + Text("顿号、逗号或换行分隔，下次录音生效") + } + .onChange(of: glossaryEnabled) { _, newValue in + Preferences.shared.glossaryEnabled = newValue + } // Parse without rewriting the draft so the insertion point stays put. settingsEditor("个性词库", text: $glossaryText, placeholder: "输入关键词、专业术语…\n示例：布迪厄、哈贝马斯、福柯") @@ -725,10 +737,11 @@ struct SettingsView: View { .disabled(!glossaryEnabled) LabeledContent("术语数量", value: "共 \(GlossaryText.parse(glossaryText).count) 个术语") } header: { - Text("个性词库") - } footer: { - sectionFooter("用顿号、逗号或换行分隔术语，下一次录音生效。", - details: "把常用术语、专有名词加进词库，用顿号、逗号或换行分隔；识别时会优先往这些词上靠，提高准确率。对豆包和 Soniox 引擎都生效，下一次录音起作用。Soniox 用词表（terms）偏置，效果通常更明显。") + HStack { + Text("个性词库") + settingsHelp("用顿号、逗号或换行分隔术语，下一次录音生效。", + details: "把常用术语、专有名词加进词库，用顿号、逗号或换行分隔；识别时会优先往这些词上靠，提高准确率。对豆包和 Soniox 引擎都生效，下一次录音起作用。Soniox 用词表（terms）偏置，效果通常更明显。") + } } } .formStyle(.grouped) @@ -739,28 +752,35 @@ struct SettingsView: View { private var enhancePane: some View { Form { Section { - Picker("引号", selection: $quoteStyle) { + Picker(selection: $quoteStyle) { Text("中文用「」").tag(TranscriptCorrector.QuoteStyle.corner) Text("中文用“”").tag(TranscriptCorrector.QuoteStyle.curly) Text("不处理").tag(TranscriptCorrector.QuoteStyle.off) + } label: { + Text("引号") + Text("将听写中的直引号配对成规范引号") } .pickerStyle(.menu) .onChange(of: quoteStyle) { _, newValue in Preferences.shared.quoteStyle = newValue } } header: { - Text("文本格式") - } footer: { - sectionFooter("将听写中的直引号配对成规范引号。", - details: "把听写出来的直引号配对成规范引号。英文始终用 “ ”，撇号写成 don’t；上面这项只决定中文语境用哪套。") + HStack { + Text("文本格式") + settingsHelp("将听写中的直引号配对成规范引号。", + details: "把听写出来的直引号配对成规范引号。英文始终用 “ ”，撇号写成 don’t；这项只决定中文语境用哪套。") + } } Section { - Toggle("启用润色", isOn: $llmEnabled) - .onChange(of: llmEnabled) { _, newValue in - Preferences.shared.llmEnabled = newValue - NotificationCenter.default.post(name: .doushaLLMEnabledChanged, object: nil) - } + Toggle(isOn: $llmEnabled) { + Text("启用润色") + Text("通过兼容 OpenAI 的接口清理听写结果") + } + .onChange(of: llmEnabled) { _, newValue in + Preferences.shared.llmEnabled = newValue + NotificationCenter.default.post(name: .doushaLLMEnabledChanged, object: nil) + } TextField("API 地址", text: $baseURL, prompt: Text("https://api.openai.com/v1")) .disableAutocorrection(true) @@ -794,17 +814,23 @@ struct SettingsView: View { isError: statusIsError, isProgress: isTesting) } } header: { - Text("润色") - } footer: { - sectionFooter("使用兼容 OpenAI 的接口，按润色规则清理听写结果。", - details: "使用任意兼容 OpenAI 的对话补全接口来清理听写结果。默认仅修正明显的识别错误；可在下方自定义润色规则。") + HStack { + Text("润色") + settingsHelp("使用兼容 OpenAI 的接口，按润色规则清理听写结果。", + details: "使用任意兼容 OpenAI 的对话补全接口来清理听写结果。默认仅修正明显的识别错误；可在下方自定义润色规则。") + } } Section { settingsEditor("润色提示词", text: $llmSystemPrompt, placeholder: "输入润色规则…\n留空则使用默认规则。") .disableAutocorrection(true) - LabeledContent("保存状态", value: llmSystemPrompt == savedLLMSystemPrompt ? "已保存" : "未保存") + LabeledContent { + Text(llmSystemPrompt == savedLLMSystemPrompt ? "已保存" : "未保存") + } label: { + Text("保存状态") + Text("保存后生效；留空使用默认规则") + } HStack { Button("恢复默认", role: .destructive) { llmSystemPrompt = TextRefiner.defaultSystemPrompt } .disabled(llmSystemPrompt == TextRefiner.defaultSystemPrompt) @@ -813,10 +839,11 @@ struct SettingsView: View { .disabled(llmSystemPrompt == savedLLMSystemPrompt) } } header: { - Text("润色提示词") - } footer: { - sectionFooter("保存后用于后续润色，留空则使用默认规则。", - details: "自定义提示词会完整替换默认规则，可允许清理重复、修复断句或轻度润色。建议要求模型只输出处理后的正文。修改后点击「保存提示词」，后续润色请求生效；留空则使用默认规则。") + HStack { + Text("润色提示词") + settingsHelp("保存后用于后续润色，留空则使用默认规则。", + details: "自定义提示词会完整替换默认规则，可允许清理重复、修复断句或轻度润色。建议要求模型只输出处理后的正文。修改后点击「保存提示词」，后续润色请求生效；留空则使用默认规则。") + } } } .formStyle(.grouped) @@ -827,15 +854,16 @@ struct SettingsView: View { private var historyPane: some View { Form { Section { - Stepper("保留最近 \(historyLimit) 条录音", value: $historyLimit, in: 1...50) - .onChange(of: historyLimit) { _, newValue in - Preferences.shared.historyLimit = newValue - history.prune(limit: newValue) - } + Stepper(value: $historyLimit, in: 1...50) { + Text("保留最近 \(historyLimit) 条录音") + Text("本机缓存原始录音与文本，超出后删最旧的") + } + .onChange(of: historyLimit) { _, newValue in + Preferences.shared.historyLimit = newValue + history.prune(limit: newValue) + } } header: { Text("存储") - } footer: { - Text("每次听写的原始录音和转录文本保存在本机缓存目录，超出条数自动删除最旧的。") } Section("最近录音") { if historyEntries.isEmpty { @@ -963,11 +991,12 @@ struct SettingsView: View { .accessibilityLabel(message) } - /// Form supplies the native editor surface; neither editor adds a custom border. + /// Both editors share the form row's surface and an appearance-aware hairline. private func settingsEditor(_ title: String, text: Binding, placeholder: String) -> some View { ZStack(alignment: .topLeading) { TextEditor(text: text) .font(.body) + .scrollContentBackground(.hidden) .frame(height: 200) .accessibilityLabel(title) if text.wrappedValue.isEmpty { @@ -979,28 +1008,31 @@ struct SettingsView: View { .accessibilityHidden(true) } } + .overlay { + RoundedRectangle(cornerRadius: 4) + .strokeBorder(Color(nsColor: .separatorColor), lineWidth: 0.5) + .allowsHitTesting(false) + .accessibilityHidden(true) + } } /// The popover's presentation lives here too, so helper fragments have no state. - private func sectionFooter(_ summary: String, details: String) -> some View { - HStack(alignment: .firstTextBaseline, spacing: 4) { - Text(summary) - Button { presentedHelp = details } label: { - Image(systemName: "info.circle") - } - .buttonStyle(.borderless) - .accessibilityLabel("更多说明：\(summary)") - .help(details) - .popover(isPresented: Binding( - get: { presentedHelp == details }, - set: { if !$0 { presentedHelp = nil } } - )) { - Text(details) - .font(.body) - .padding() - .frame(width: 340) - .fixedSize(horizontal: false, vertical: true) - } + private func settingsHelp(_ summary: String, details: String) -> some View { + Button { presentedHelp = details } label: { + Image(systemName: "info.circle") + } + .buttonStyle(.borderless) + .accessibilityLabel("更多说明：\(summary)") + .help(details) + .popover(isPresented: Binding( + get: { presentedHelp == details }, + set: { if !$0 { presentedHelp = nil } } + )) { + Text(details) + .font(.body) + .padding() + .frame(width: 340) + .fixedSize(horizontal: false, vertical: true) } } From 635d6191720c139de4cc8bce370bd8dafad0bbe8 Mon Sep 17 00:00:00 2001 From: giraphant Date: Mon, 12 Oct 2026 17:36:50 +0100 Subject: [PATCH 5/6] fix: isolate test logs and preserve clipboard ownership --- Sources/Apps/Dousha/AppDelegate.swift | 4 ++++ Sources/Apps/Dousha/TextInjector.swift | 2 +- Sources/Common/ConcurrencySupport/Logging.swift | 4 ++-- Sources/Engines/DoubaoASR/DoubaoASR.swift | 8 +++----- Sources/Engines/SonioxASR/SonioxASR.swift | 3 +-- 5 files changed, 11 insertions(+), 10 deletions(-) diff --git a/Sources/Apps/Dousha/AppDelegate.swift b/Sources/Apps/Dousha/AppDelegate.swift index 8a8a576..8345a09 100644 --- a/Sources/Apps/Dousha/AppDelegate.swift +++ b/Sources/Apps/Dousha/AppDelegate.swift @@ -125,6 +125,10 @@ final class AppDelegate: NSObject, NSApplicationDelegate { _ = refiner.refineLater(text, traceId: traceId) { refined in let pb = NSPasteboard.general let changedWhileRefining = pb.changeCount != originalChangeCount + guard !changedWhileRefining else { + doushaLog("[Dousha] deferred refine skipped reason=clipboard_changed traceId=\(traceId)") + return + } pb.clearContents() let setOK = pb.setString(refined, forType: .string) if setOK { diff --git a/Sources/Apps/Dousha/TextInjector.swift b/Sources/Apps/Dousha/TextInjector.swift index 5b2ef97..79fc451 100644 --- a/Sources/Apps/Dousha/TextInjector.swift +++ b/Sources/Apps/Dousha/TextInjector.swift @@ -30,7 +30,7 @@ final class TextInjector { doushaLog("[Dousha] TextInjector.inject: SKIPPED (empty text)") return } - doushaLog("[Dousha] TextInjector.inject: text=\"\(text.prefix(60))\" len=\(text.count)") + doushaLog("[Dousha] TextInjector.inject: len=\(text.count)") let pasteboard = NSPasteboard.general pasteboard.clearContents() diff --git a/Sources/Common/ConcurrencySupport/Logging.swift b/Sources/Common/ConcurrencySupport/Logging.swift index 765d79b..4c85317 100644 --- a/Sources/Common/ConcurrencySupport/Logging.swift +++ b/Sources/Common/ConcurrencySupport/Logging.swift @@ -17,14 +17,14 @@ private let _doushaLogger = Logger(subsystem: "com.dousha.app", category: "gener /// it never captured a failure either. This file is the durable record. public func doushaLog(_ message: String) { _doushaLogger.log("\(message, privacy: .public)") - DoushaFileLog.shared.append(message) + DoushaFileLog.shared?.append(message) } /// Append-only persistent log at `~/Library/Logs/Dousha/dousha.log`, mirroring /// every `doushaLog` line with a timestamp. Thread-safe via a private serial /// queue; rotates once at ~5 MB so it can't grow unbounded. final class DoushaFileLog: @unchecked Sendable { - static let shared = DoushaFileLog() + static let shared = NSClassFromString("XCTestCase") == nil ? DoushaFileLog() : nil private let queue = DispatchQueue(label: "com.dousha.app.filelog") private let fileURL: URL? diff --git a/Sources/Engines/DoubaoASR/DoubaoASR.swift b/Sources/Engines/DoubaoASR/DoubaoASR.swift index c65c159..b73590a 100644 --- a/Sources/Engines/DoubaoASR/DoubaoASR.swift +++ b/Sources/Engines/DoubaoASR/DoubaoASR.swift @@ -674,8 +674,7 @@ public actor DoubaoASR { if contextHint.isEmpty { doushaLog("[DoubaoASR] StartSession context: (none)") } else { - let preview = contextHint.count > 60 ? String(contextHint.prefix(60)) + "…" : contextHint - doushaLog("[DoubaoASR] StartSession context.len=\(contextHint.count) preview=\(preview)") + doushaLog("[DoubaoASR] StartSession context.len=\(contextHint.count)") } try await sendData(AsrMessageBuilder.startSession(requestId: requestId, token: token, configJSON: configJSON)) let resp2 = try await waitForResponse(timeout: 5.0) { @@ -827,7 +826,7 @@ public actor DoubaoASR { signalFinished() return case "TaskFailed", "SessionFailed": - doushaLog("[DoubaoASR] \(resp.messageType) statusCode=\(resp.statusCode) statusMessage=\(resp.statusMessage) resultJson=\(resp.resultJson)") + doushaLog("[DoubaoASR] \(resp.messageType) statusCode=\(resp.statusCode) statusMessage=\(resp.statusMessage) resultJson.len=\(resp.resultJson.count)") let msg = resp.statusMessage.isEmpty ? "ASR failed (\(resp.statusCode))" : "\(resp.statusMessage) (\(resp.statusCode))" deliverError(NSError(domain: "DoubaoASR", code: Int(resp.statusCode), userInfo: [NSLocalizedDescriptionKey: msg])) @@ -840,9 +839,8 @@ public actor DoubaoASR { guard let update = resultState.ingest(resultJson: resp.resultJson) else { return } self.lastTranscriptAt = Date() - let preview = update.text.prefix(40).replacingOccurrences(of: "\n", with: " ") let stopDelta = stopStartedAt.map { " stop+\(elapsedMs(since: $0))ms" } ?? "" - doushaLog("[DoubaoASR] traceId=\(requestId) t=\(traceElapsedMs())ms\(stopDelta) result isInterim=\(update.isInterim) vadFinished=\(update.vadFinished) nonstream=\(update.nonstreamResult) textLen=\(update.text.count) currentInterimLen=\(update.previousInterimLength) newUtterance=\(update.looksLikeNewUtterance) preview=\(preview)") + doushaLog("[DoubaoASR] traceId=\(requestId) t=\(traceElapsedMs())ms\(stopDelta) result isInterim=\(update.isInterim) vadFinished=\(update.vadFinished) nonstream=\(update.nonstreamResult) textLen=\(update.text.count) currentInterimLen=\(update.previousInterimLength) newUtterance=\(update.looksLikeNewUtterance)") switch update.commit { case .rescued(let rescued): diff --git a/Sources/Engines/SonioxASR/SonioxASR.swift b/Sources/Engines/SonioxASR/SonioxASR.swift index 80238c4..30da1a1 100644 --- a/Sources/Engines/SonioxASR/SonioxASR.swift +++ b/Sources/Engines/SonioxASR/SonioxASR.swift @@ -381,8 +381,7 @@ public actor SonioxASR { if contextTerms.isEmpty { doushaLog("[SonioxASR] traceId=\(requestId) config context: (none)") } else { - let preview = contextTerms.prefix(8).joined(separator: "、") - doushaLog("[SonioxASR] traceId=\(requestId) config context.terms=\(contextTerms.count) preview=\(preview)") + doushaLog("[SonioxASR] traceId=\(requestId) config context.terms=\(contextTerms.count)") } try await ws.send(.string(json)) } From 5dfc1d37938f05e036c1473bad06d4eee10b98cb Mon Sep 17 00:00:00 2001 From: giraphant Date: Mon, 12 Oct 2026 17:39:21 +0100 Subject: [PATCH 6/6] docs: refresh usage and consolidate MLX architecture --- .gitignore | 8 +- AGENTS.md | 93 ++++++------- ARCHITECTURE.md | 364 +++++++++++++++++++++--------------------------- README.md | 20 ++- 4 files changed, 218 insertions(+), 267 deletions(-) diff --git a/.gitignore b/.gitignore index 8fd416e..38c0bca 100644 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,7 @@ .build/ .swiftpm/ *.xcodeproj/ +*.profraw .DS_Store icon.iconset/ @@ -8,12 +9,5 @@ icon.iconset/ *.apk DoubaoIME-decompiled/ -# Third-party reference projects kept locally for inspiration -BorderBeam/ - # Per-developer signing config (Apple Developer ID, notary profile) Makefile.local - -# Internal docs (reverse-engineering notes, manual test logs) — not for public repo -docs/ -.playwright-mcp/ diff --git a/AGENTS.md b/AGENTS.md index dd9aba1..4dac6b2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,67 +1,66 @@ # Agent instructions — Dousha -## Architecture invariants (READ FIRST for pipeline/concurrency changes) +## Read first Before touching the recording pipeline, any `@unchecked Sendable` type, the ASR clients, or anything that looks like duplicated code worth abstracting, -read `ARCHITECTURE.md`. It records the load-bearing invariants (three-phase -start order, generation guards, flag-before-HUD ordering) and the list of -**deliberately rejected refactors** — do not re-propose those without new -evidence. +read [ARCHITECTURE.md](ARCHITECTURE.md). Its invariants and deliberately rejected +refactors are normative; do not re-propose rejected designs without new evidence. +`CLAUDE.md` is a symlink to this file; keep it that way. ## Testing workflow (REQUIRED) -This is a menu-bar / accessibility app on macOS. You cannot test changes by running `swift build` or `swift run` directly — TCC permissions (Microphone, Accessibility, Speech Recognition) are bound to the signed bundle at `/Applications/Dousha.app`, and global hotkeys only work for the installed bundle the user already granted permissions to. +TCC permissions (Microphone, Accessibility, Speech Recognition) belong to the +signed bundle at `/Applications/Dousha.app`. Test audio, hotkeys, HUD and +permissions through that installed app, never a `swift run` product. -`make build` / `make install` now use `xcodebuild` with full Xcode to compile MLX's Metal shaders and package its resource bundles. `DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer swift build` and `swift test` still check compilation and offline logic, but their products cannot run MLX inference without the metallib. For headless real-model checks, build the `smoke-cli` scheme with `xcodebuild` and run the products-dir `smoke-cli mlx-transcribe`; see `ARCHITECTURE.md` §6 for commands. Keep the committed `Package.resolved` and pinned mlx-audio-swift revision together when upgrading dependencies. +`make build` / `make install` use full Xcode's `xcodebuild` to compile MLX Metal +shaders and package resources. For offline logic checks, run: -Every time you want the user to test a change, you must: - -1. **Ask the user to quit the running Dousha** (menu bar → Quit, or `killall Dousha`). `make install` will not replace a running `.app` cleanly, and the user will end up testing the old build without noticing. -2. Run `make install` — this does an Xcode release build, codesigns with the configured Developer ID identity (see below), and copies to `/Applications/Dousha.app`. -3. Launch the new bundle (`open /Applications/Dousha.app`) or ask the user to launch it. -4. Then ask the user to reproduce / test. - -Do **not** report a feature as "ready to test" after only `swift build` or `swift test`. Unit tests are fine for logic, but anything touching audio capture, hotkeys, HUD, or permissions must go through `make install` + relaunch before the user can verify it. - -## Release workflow - -Public releases go to GitHub Releases as a signed + notarized DMG. The pipeline is fully wired in the `Makefile`: - -``` -make release VERSION=0.1.2 +```sh +DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer swift test ``` -This target asserts the version matches `Resources/Info.plist`, requires a clean working tree, builds with hardened runtime + Developer ID, packs a DMG with an `/Applications` symlink, signs the DMG, submits to Apple notary, staples, and `gh release create`s the DMG at `vX.Y.Z`. +`swift build` checks compilation only; neither SwiftPM product supplies the +metallib for inference. See ARCHITECTURE.md §2.1 for MLX smoke commands and +dependency pins, and §6 for pipeline verification. `make install` does not run tests. + +Every time you want the user to test a change, you must: -Per-release checklist: +1. **Ask the user to quit Dousha** (menu bar → Quit, or `killall Dousha`), so they + cannot unknowingly keep running the old build. +2. Run `make install` (release build, signing, copy to `/Applications/Dousha.app`). +3. Launch with `open /Applications/Dousha.app`, or ask the user to launch it. +4. Ask the user to reproduce / test. -1. Bump `Resources/Info.plist` — both `CFBundleShortVersionString` (semver) and `CFBundleVersion` (monotonic integer; notary rejects re-used build numbers for the same short version). -2. Commit + push the bump. -3. Run `make release VERSION=X.Y.Z`. Notarization usually takes 2–5 min; the target blocks until Accepted. +Do not call a feature "ready to test" after only `swift build` or `swift test`. -One-time setup (per dev machine): +## Release workflow -- Apple Developer ID cert must be in the login keychain with its private key. -- Notary credentials stored as a keychain profile: - ``` - xcrun notarytool store-credentials \ - --apple-id --team-id \ - --password - ``` -- Create a `Makefile.local` (gitignored) overriding the public Makefile placeholders: - ``` - DEVELOPER_ID_IDENTITY := Developer ID Application: () - NOTARY_PROFILE := - ``` +1. Bump both `CFBundleShortVersionString` (semver) and `CFBundleVersion` + (monotonic integer) in `Resources/Info.plist`. +2. Commit and push the bump; the release target requires a clean tree and matching version. +3. Run `make release VERSION=X.Y.Z`. It builds with hardened runtime, notarizes + and staples the app, packs and signs the DMG, notarizes and staples the DMG, + publishes GitHub release `vX.Y.Z`, then updates the Homebrew cask. -### Release gotchas — do not regress +## Signing setup -- **Entitlements are mandatory for the dist build.** Hardened runtime auto-denies mic/camera access unless the relevant entitlement is present. `dist` passes `--entitlements Resources/Dousha.entitlements` (currently grants `com.apple.security.device.audio-input`). Symptom of the regression: TCC shows `kTCCServiceMicrophone com.dousha.app auth_value=2 auth_reason=4` (system_set deny), no prompt ever appears to the user. v0.1.0 shipped without this and was broken; v0.1.1 fixed it. -- The DMG must also be codesigned (not just the .app) — `dist` does this. Notary will reject an unsigned DMG. +Reuse the configured Developer ID Application certificate and private key in the +login keychain. Store notary credentials with `xcrun notarytool store-credentials`. +Each checkout/worktree needs a gitignored `Makefile.local` overriding the placeholders: -## Other notes +```make +DEVELOPER_ID_IDENTITY := Developer ID Application: () +NOTARY_PROFILE := +``` -- Single signing identity for everything: `Developer ID Application: ()`. Both `make install` (local dev) and `make release` (DMG) sign with it. TCC grants persist across both because the csreq is anchored to the team ID, not the cdhash — so a fresh local build and the latest DMG release share Mic / Speech / Accessibility approvals. -- If the Developer ID cert is missing from the keychain (fresh clone / CI), the Makefile falls back to ad-hoc signing and TCC grants reset on every rebuild. Flag this if you see the ad-hoc fallback message. -- `make reset-perms` wipes Mic / SpeechRecognition / Accessibility grants for `com.dousha.app`. Don't run it without asking. +- Local installs and releases use the same identity; the team-anchored TCC + requirement preserves grants across rebuilds. Flag any ad-hoc fallback: + missing configuration/certificate means grants will not persist. +- Preserve `Resources/Dousha.entitlements` with + `com.apple.security.device.audio-input`; hardened runtime otherwise denies the + mic without prompting (the v0.1.0 regression). +- Sign the DMG too, and staple **both** app and DMG: a DMG ticket does not travel + with the extracted app for offline Gatekeeper checks. +- Never run `make reset-perms` without asking; it wipes all three TCC grants. diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 50ab11c..bb4a660 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -31,30 +31,23 @@ settings Doubao servers; mlx-transcribe with local MLX weights ``` -macOS-only. (A tier-2 Windows shell, `DoushaWin`, existed under QUA-209; it -was removed in 2026-07 when its one user stopped using it — its platform -gates, the hand-rolled `InsecureMD5`, and the `OpusEncoding` seam went with -it. Resurrect from git history if a port is ever needed again.) +macOS-only; retired Windows code is recoverable from git history. Rules: - Lower layers never import higher ones. The three ASR libraries are peers, and the two entry points never import each other. -- `MLXASR` depends on mlx-audio-swift's `MLXAudioSTT`, pinned to revision - `244458d7b61e54ff135073ddc9ed76782bb20758`. Local model support is restricted - to real-time streaming models by product decision; the current catalog is - Qwen3-ASR 0.6B / 1.7B (8-bit), with 0.6B as the default. Do not introduce a - batch-only local model behind the streaming engine contract. +- Local-engine layering, lifecycle and verification are defined in §2.1. - Platform APIs (AppKit) live only in the entry points under `Apps/` and `Tools/`. - Library targets never read `Preferences` (or any app singleton). The app snapshots config at recording start and passes it in - (`SonioxBackend`/`DoubaoBackend` snapshot glossary + language and `MLXBackend` - snapshots the language identifier and glossary in `beginSession`). This keeps mid-recording - Settings changes from mutating a live session, and keeps the libraries testable. + (backends snapshot language + glossary in `beginSession`). Mid-recording + Settings changes must not mutate a live session. - All logging goes through `doushaLog` (ConcurrencySupport). Log strings are part of the observable surface — field debugging greps for them — so don't reword - them casually. + them casually. Log lengths, flags and timings, never user content (transcripts, + prompts, glossary/context or credentials). File logging is disabled under XCTest. ## 2. Recording pipeline data flow @@ -107,21 +100,6 @@ engine is still alive (`ErrorGate`); a primary failure hands the HUD to the secondary immediately. Only all-engines-dead (or capture failure) becomes a fatal `.error` stream event. -`MLXBackend` passes the snapshotted Dousha language identifier (`zh-CN`, -`en-US`, `auto`) unchanged; `MLXStreamingASR` alone maps it to the Qwen3 -language name. Nil, auto and unknown identifiers mean auto-detect. Keep the -user's selection for both streaming and batch: in the 2026-10-12 code-switched -clip comparison, forced Chinese retained a Chinese lead-in that the full-audio -`auto` pass dropped. The earlier observation that forced Chinese garbled English -was streaming-only, so it is not evidence to force auto for the final pass. Local -streaming text is a HUD preview; stop transcribes the entire retained recording -with Qwen3's batch API, and that nonempty result becomes the final text. Empty, -failed or overdue batch output falls back to the filtered streaming text. -`MLXBackend` snapshots the enabled, normalized glossary as a short plain-text -vocabulary list for the batch system context. Streaming still cannot receive -context ([giraphant/dousha#53](https://github.com/giraphant/dousha/issues/53)); -downstream `TranscriptCorrector` also continues to apply local corrections. - Fatal ≠ session end during live capture (issue #46): a fatal `.error` arriving while the controller is still `.recording` is latched (`pendingErrorMessage`), NOT transitioned — the mic + WAV don't need the network, so capture runs until @@ -138,23 +116,138 @@ thread reads it), then status + HUD glow, then dispatcher reset, then HUD visibility. Timing constants live as statics on the controller (`cancelTeardownGuard` 0.25s, `injectGreenFlash` 0.25s, `errorAutoReset` 3.0s). +### 2.1 Local MLX engine + +**Layering and input.** `MLXBackend` (`Engine.qwen3`, `PCMCaptureEngine`) adapts +app sessions to `MLXStreamingASR` → `MLXModelStore` → `MLXAudioSTT` from +mlx-audio-swift. Only streaming-capable models belong in this contract: Qwen3-ASR +0.6B / 1.7B (8-bit), default 0.6B. The backend snapshots Dousha's language +identifier and enabled, normalized glossary in `beginSession`; only the library +maps identifiers to Qwen language names (nil/auto/unknown → auto-detect). +Keep the user-selected language for **both** passes: the 2026-10-12 mixed-language +comparison retained a Chinese lead-in with forced Chinese that batch auto lost; +earlier English garbling was streaming-only evidence. Glossary is a short +plain-text vocabulary list for the final batch context only; streaming context +is unavailable ([#53](https://github.com/giraphant/dousha/issues/53)). +`TranscriptCorrector` still applies downstream; `prepareSession` context defaults to empty. + +**Preview and final.** Streaming text previews the HUD. Stop transcribes all +retained audio in one batch `generate` call; nonempty batch text wins, while +empty/failed/overdue output falls back to filtered streaming text. Batch runs +off main, after streaming cleanup, with temperature 0, repetition penalty 1.15 +and 32-token repetition context; upstream splits only very long audio at 1200s +low-energy boundaries. `finalPass: false` is for A/B smoke checks. +`MLXTranscriptState` strips complete Qwen headers from partials/finals, hides +incomplete tails only in auto mode, and preserves a safe confirmed prefix. +Preview/fallback reduce five-or-more consecutive repeats of a 1–4 character +unit (optional punctuation) to four; ordinary 哈哈哈 / 很多很多 stays intact. +Never apply that repetition guard to batch text, but always apply +`QwenTranscriptFilter`, even though upstream already strips auto-language headers. + +**Audio and generations.** `MLXStreamingASR` lifecycle is actor-isolated; +nonisolated `ingest` only appends PCM under the `MLXSessionInput` lock: no +conversion, MLX calls, tasks, logging or actor hops on the tap thread. That lock +protects the drain buffer and append-only full-session copy; prepare/cancel/ +completion clear both. A frozen final-pass PCM copy may survive delivery/cancel +until the non-cancellable batch returns. The detached feeder wakes every 100ms; +conversion, `feedAudio`, final drain and native stop/cancel share a private +serial queue off audio/main threads, so teardown cannot overtake feed. Carry +odd bytes into the next feed; discard only a malformed final odd byte. +Each recording owns fresh setup/final/event-finished channels; stale setup/cleanup +retains captured objects, never a newer lease. `.ended` signals final before +the generation guard. Only cumulative `.displayUpdate` replaces text; never +append confirmed/provisional events again. Partials, errors and exactly one +stop completion dispatch on main and recheck generation; cancel immediately +quarantines input and callbacks. + +**Model ownership.** The store actor coalesces loading/warmup and caches weights; +`QwenLoadedModel` is `@unchecked Sendable` only because a process-wide lease +(shared across separately constructed stores) covers loading/replacement, +prewarm, the whole streaming session and batch. Never overlap streaming and +batch inference. The store never mutates a published model; only its lease +holder uses it. Construction/loading runs off main; caller cancellation does +not cancel shared downloads, and setup retains its lease until loading settles. + +**Preload/prewarm.** Launch preloads after hotkey installation when Qwen3 is +active; routing/menu/model changes use the same fire-and-forget `warmup` path. +Only a preload-owned new load runs one off-main, 1s silent batch (8-token limit) +under the lease and discards output. Cached models and recording-owned loads +never prewarm, even when warmup joins or queues behind them. Failure logs an +error but leaves weights `.ready`; this state means loaded, not prewarm finished. +Recordings wait on the lease, released only after prewarm returns. +`[MLXModelStore] prewarm repoID=… ms=…` records cost. Keep batch-only prewarm: +a separate streaming prewarm on two 2026-10-12 clips showed no useful preview +benefit and roughly doubled prewarm cost. + +**Stop deadlines and shutdown.** Stop allows 30s for setup, 10s for `.ended` +(saving latest display text on timeout), 3s for streaming cleanup, then +`MLXFinalPassDeadline` defaults to `max(5 s, 0.10 × audioSeconds)` for batch +(injectable). The `finalPass` log includes the deadline; batch timing starts at +generation, not cleanup. Deadline expiry delivers fallback exactly once; the detached owner +retains the lease until batch returns and discards late text. Cancel suppresses +completion without releasing the lease early. +Cleanup waits 3s for queued native cancel; a stuck synchronous feed retains +the lease until cancel returns, then event closure gets 3s before consumer +cancellation/release. Upstream cancel has **no native decoder completion +barrier**: event closure does not prove detached GPU work ended. The final-pass +owner must therefore wait for natural `.ended` before cancel/cleanup/batch, +even after the initial 10s timeout. If natural stop exceeds the cleanup +allowance, deliver fallback, abandon batch and retain the lease until natural +stop ends. A stuck decoder can block subsequent sessions; never substitute +cancel's event closure or weaken the lease to allow concurrent inference. + +**Build and smoke verification.** Full Xcode's `xcodebuild` compiles Metal and +places `mlx-swift_Cmlx.bundle` beside the executable; the Makefile copies +resources into the app. SwiftPM builds/tests check compilation/offline logic +but do not supply the inference metallib. Build and run the headless scheme: + +```sh +export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer +xcodebuild -scheme smoke-cli -configuration Release \ + -destination 'platform=macOS,arch=arm64' -derivedDataPath .build/xcode \ + CODE_SIGNING_ALLOWED=NO build +.build/xcode/Build/Products/Release/smoke-cli mlx-transcribe \ + ~/Library/Caches/Dousha/last_recording.wav --language zh-CN +.build/xcode/Build/Products/Release/smoke-cli mlx-transcribe \ + ~/Library/Caches/Dousha/last_recording.wav --language auto --realtime +``` + +`WavReader` accepts 16kHz mono s16le WAVs. The harness uses the real lifecycle, +100ms chunks and 10ms sleeps (audio-rate with `--realtime`), downloads the default +model on first use, and accepts compatible repos via `--model`. Options: +`--final-pass on|off` (default on), `--context` (batch only), `--prewarm` (awaits +launch preload). Compare prewarm in fresh processes. `preload_ms` includes +load/prewarm; `model_load_ms` samples open→ready every 20ms, including downloads; +first-partial latency starts at ingest. `stop_to_final_ms` includes drain, +cleanup and batch; `batch_ms` is generation time (elapsed at fallback delivery +on timeout), `deadline_seconds` the allowance, `final_source` batch/stream. +Partials/final text are printed: keep private transcripts out of shared reports. +Async waits service main callbacks; watchdogs are 600s for model load and +max(80s, 48s + batch deadline) for stop. Inspect `finalTimedOut` logs even when +fallback makes the command succeed. + +Keep `Package.resolved` tracked with mlx-audio-swift revision +`244458d7b61e54ff135073ddc9ed76782bb20758` in `Package.swift`. Upgrade it, +mlx-swift, mlx-swift-lm and resolved dependencies together; rerun offline tests +and both language smoke checks. + ## 3. Protocol seams & testing | Seam | Purpose | Faked by | |---|---|---| | `SpeechBackend` | one session = one `AsyncStream`; `stop()` yields exactly one terminal `.final`, `cancel()` finishes with none | `MockSpeechBackend` (RecordingControllerTests) | | `PushCaptureEngine` (+ `PCMCaptureEngine` / `BufferCaptureEngine`) | engines receive pushed audio from the hub; two-phase `beginSession`/`openStream` | `MockBackend` (MultiEngineBackendTests, hub = nil) | -| `MLXBackend` (`Engine.local`, `PCMCaptureEngine`) | adapts prepare / open / ingest / stop / cancel to `MLXStreamingASR`; snapshots the unmapped language identifier and glossary context | injected prepare closure + isolated Preferences (MLXEngineWiringTests) | -| `MLXModelStore` loader + process-wide session lease | loads/caches Qwen3 weights once; serializes prewarm and local inference sessions | injected loader / `MLXLoadedModel` fake (MLXASRLifecycleTests, MLXModelPrewarmTests); lease tests never allocate MLX arrays | -| `MLXPreparedSession` stream + final-pass runner | exercises real stop/cancel/deadline/lease ownership without a model | fake event stream + suspended runner (MLXFinalPassTests) | +| `MLXBackend` | app adapter (§2.1) | injected prepare closure + isolated Preferences (MLXEngineWiringTests) | +| `MLXModelStore` loader + lease | model ownership (§2.1) | injected loader / `MLXLoadedModel` fake (MLXASRLifecycleTests, MLXModelPrewarmTests); lease tests never allocate MLX arrays | +| `MLXPreparedSession` stream + final-pass runner | stop/cancel/deadline ownership (§2.1) | fake event stream + suspended runner (MLXFinalPassTests) | | `RecordingEnvironment` | struct-of-closures: ALL side effects the controller performs (HUD, inject, refine, scheduling, clock) | closure fakes + virtual clock | | `SettingsActions` | struct-of-closures: system-touching settings effects (login item, dock icon, credentials) | closure fakes | Rules: a new side effect enters through a seam, never through a singleton reached from inside the controller/backends. Tests use `@testable import`, so `internal` visibility is test-reachable — never widen to `public` for tests. -Run `swift test` (not just `swift build`) before committing; the test target is -not built by `make install`. +For code changes, run the full offline suite (§6) before committing; +`make install` does not build the test target. ## 4. Concurrency model — `@unchecked Sendable` invariants @@ -195,85 +288,8 @@ type is used, re-derive the reason or the annotation becomes a lie. - **`AppleSpeechBackend`** — `sessionGen` behind a `Lock`, because three threads touch it: main, the audio-frame path, and Speech framework callbacks. Codes 203/216/1110/301 are benign and intentionally dropped. -- **`MLXStreamingASR` / `MLXSessionInput`** — lifecycle state is actor-isolated; - `ingest` is a nonisolated, lock-only PCM append for the real-time audio path: - no conversion, MLX calls, tasks, logging or actor hops on the tap thread. - The same lock protects both the drain buffer and an append-only full-session - PCM copy; prepare, cancel and completion clear both. Only append happens on - the tap thread. A final-pass request may retain its frozen PCM after delivery - or cancellation until the non-cancellable batch call returns. - The detached feeder wakes every 100ms and schedules PCM conversion + - `feedAudio` on a private serial queue, off both audio and main threads. - Final drain, native `stop` and native `cancel` use that same queue, so they - cannot overtake an in-progress feed. Odd PCM bytes carry into the next feed; - only a malformed final odd byte is discarded. -- **MLX generations and callbacks** — every recording owns fresh setup, - final and event-finished channels; stale setup/cleanup retains its captured - objects, never a newer session's lease. `.ended` signals its final channel - before the generation guard. Only cumulative `.displayUpdate` events replace - the transcript; confirmed/provisional events must not be appended again. - `MLXTranscriptState` strips complete Qwen protocol headers from partials and - finals, hiding incomplete tails only in auto mode and keeping a safe confirmed prefix. - The preview and streaming fallback also shorten five-or-more consecutive - repeats of a 1–4 character unit (with optional punctuation) to four repeats; - ordinary reduplication such as 哈哈哈 / 很多很多 stays untouched. This guard - never transforms batch text. Batch output is still passed through - `QwenTranscriptFilter`, although upstream `parseGeneratedChunk` already strips - auto-language headers. - Partials, errors and the one stop completion dispatch on main and recheck - the generation there; cancel immediately quarantines input and callbacks. -- **`MLXModelStore` / `QwenLoadedModel` (`@unchecked Sendable`)** — the store - actor caches weights and coalesces loading/warmup. A single process-wide - lease gate, shared even by separately constructed stores, covers model load, - replacement, kernel prewarm, the complete streaming session and the subsequent - batch pass. App launch (after hotkey installation) and engine-routing changes - call fire-and-forget `warmup` when Qwen3 is active; menu/model selection also - uses this path. A preload-owned new load runs one 1s silent batch (8-token - limit) off main under the same lease, discarding output. Cached models and - recording-owned loads never prewarm, including a warmup joining or queued - behind such a load. Prewarm failure logs an error but leaves weights `.ready`; - release the lease only after prewarm actually returns. State `.ready` denotes - loaded weights, not completion of prewarm; recordings still wait on the lease. - `[MLXModelStore] prewarm repoID=… ms=…` records its cost. - A separate 1s streaming-session prewarm was tested on the two 2026-10-12 - clips with 1.7B-8bit: first preview was 145/143ms versus 147/153ms with - batch-only prewarm, while total prewarm grew from 89–92ms to 200ms. This - small single-run difference did not demonstrate a useful streaming benefit; - keep only the batch prewarm (the installed app's initial batch was 1225ms - versus 284–377ms subsequently). - Streaming cleanup precedes batch generation; never run both at once. - The store never mutates the published model; only the lease holder uses it; - model construction/loading runs off main. Cancelled callers do not cancel a - shared download, and setup retains the lease until that load settles. -- **MLX final-pass ownership and deadline** — stop waits up to 30s for setup, - then up to 10s for `.ended` (saving the latest display text on timeout), then - up to 3s for streaming cleanup and up to `max(baseSeconds, audioRatio × - audioSeconds)` for batch. The injectable `MLXFinalPassDeadline` defaults to - 5s base and 0.10 ratio: ≤50s audio gets 5s; a five-minute recording gets 30s. - Batch elapsed time starts when generation begins, not during stream cleanup. - The existing `finalPass` log and smoke metrics include the computed deadline. - The batch runs off main with temperature 0, repetition penalty - 1.15 and upstream's 32-token repetition context: a modest discouragement of - loops without aggressively suppressing ordinary repeated words. The full PCM - is passed in one `generate` call (upstream only splits very long audio at its - default 1200s low-energy boundaries). `finalPass: false` disables the pass for - A/B smoke checks; `prepareSession` context defaults to empty for compatibility. - Deadline expiry delivers streaming fallback exactly once; the detached owner - keeps the lease until batch returns, discarding its late text. Cancel during - batch suppresses completion and likewise does not release the lease early. - Final-pass metrics log lengths and timings, never transcript or context text. -- **MLX streaming shutdown** — cleanup waits 3s for queued native cancel; if a - synchronous feed is stuck, it retains the lease until cancel actually returns. - It then waits up to 3s for event-stream closure before cancelling the consumer - and releasing the lease. Upstream `cancel()` has **no native decoder completion - barrier**: event-stream closure is its strongest exposed signal, not proof - that all detached GPU work has completed. Therefore the final-pass owner - waits for natural `.ended` before invoking cancel/cleanup and batch, including - after the initial 10s wait timed out. If natural stop outlasts the cleanup - allowance, deliver fallback, abandon batch and retain the lease until natural - stop actually ends; a stuck upstream decoder can consequently block subsequent - sessions. Do not substitute cancel's event closure for this barrier or weaken - the lease to permit concurrent inference. +- **MLX types** — §2.1 defines actor/lock isolation, generation guards, serial + feed/teardown ordering and the process-wide lease protecting `QwenLoadedModel`. - **`AudioTapHub` (actor)** — the tap closure runs on the audio thread and captures only local snapshots; it never hops back into the actor (real-time safety). The `nonisolated(unsafe) var fed` is safe because the converter @@ -306,12 +322,15 @@ without new evidence. - **`TextInjector` does not switch input sources** (QUA-132, Chromium desync) **and does not restore the clipboard** (matches Superwhisper/Vistaflow UX). Both are decisions, not omissions. -- **`SettingsView` is one struct; panes are not child views.** API keys and - test results live only in `@State` until 保存并测试; `NavigationSplitView` +- **`SettingsView` is one struct; panes are not child views.** All state, + including sheets, lives here. API keys and test results stay in `@State` + until 保存并测试; `NavigationSplitView` destroys child views on pane switch, so child-view panes would silently drop unsaved edits. Sheets follow the same rule: present them from `SettingsView` and keep their editable state there across dismissal; helpers accept only - values, bindings, and closures, without owning draft state. + values, bindings, and closures, without owning draft state. Use native + `Form` with `.formStyle(.grouped)` and standard controls, an engine list with + option sheets, and row subtitles / ⓘ help instead of footers. No chips/cards. - **`HotkeyMonitor` and `CancelKeyMonitor` stay separate classes.** The shared part (tap lifecycle) is already extracted into `EventTap`; merging the rest couples two independently-fragile event paths. @@ -320,8 +339,10 @@ without new evidence. agents: locality beats file count, and splitting an actor across extension files separates state from the code that guards it. Don't split files for size alone. -- **`MultiEngineBackend`'s helpers stay nested** — they are private to its - orchestration and their doc comments reference its callback ordering. +- **Keep multi-engine routing** even when only one engine is selected; parallel + capture, language selection and partial/error fallback are deliberate features. + `MultiEngineBackend` helpers stay nested: they are private to its orchestration + and their comments reference its callback ordering. ## 6. Directory map & verification discipline @@ -329,82 +350,19 @@ Directories group targets by role: ``` Sources/ - Common/ bottom of the graph — libraries everything imports - ConcurrencySupport/ concurrency utils (no deps) - ASRSupport/ shared ASR domain types + TranscriptFormatter - Engines/ one streaming ASR client per provider - DoubaoASR/ Doubao WS client (protobuf, opus, reconnect, credentials) - SonioxASR/ Soniox WS + async-batch client - MLXASR/ local Qwen3 streaming preview + batch final, model store and session lease - Apps/ - Dousha/ macOS app: AppDelegate wiring, RecordingController, - MultiEngineBackend + AudioTapHub, backends (adapters), - HUD (FloatingHUD*), Settings, hotkey monitors, Preferences - Tools/ developer-facing executables, never shipped - SmokeCLI/ smoke harness — real Doubao servers or real local MLX - weights; never Tests/ (suite must stay offline/deterministic) - SmokeCLISupport/ Doubao file-transcription helper -Tests/DoushaTests/ all unit tests (one target, @testable imports everything; - macOS-only because it imports the app target) + Common/ ConcurrencySupport (no deps), ASRSupport + Engines/ DoubaoASR, SonioxASR, MLXASR + Apps/Dousha/ app wiring, capture, orchestration, UI, preferences + Tools/SmokeCLI/ real-server / real-model checks; never shipped + Tools/SmokeCLISupport/ Doubao file-transcription helper +Tests/DoushaTests/ one target, @testable imports everything; macOS-only + (imports app); must stay offline/deterministic ``` -Builds use Xcode (`xcodebuild`) so MLX's Metal shaders are compiled and the -`mlx-swift_Cmlx.bundle` resource is placed beside the executable (and copied -into the app by the Makefile). `swift build` / `swift test` remain useful for -compilation and offline logic tests, but the SwiftPM build path does not supply -the metallib needed to run MLX inference. Use full Xcode, not CommandLineTools: - -```sh -export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer -swift build -swift test -xcodebuild -list # smoke-cli is the headless executable scheme -xcodebuild -scheme smoke-cli -configuration Release \ - -destination 'platform=macOS,arch=arm64' -derivedDataPath .build/xcode \ - CODE_SIGNING_ALLOWED=NO build -.build/xcode/Build/Products/Release/smoke-cli mlx-transcribe \ - ~/Library/Caches/Dousha/last_recording.wav --language zh-CN -.build/xcode/Build/Products/Release/smoke-cli mlx-transcribe \ - ~/Library/Caches/Dousha/last_recording.wav --language auto --realtime -.build/xcode/Build/Products/Release/smoke-cli mlx-transcribe \ - ~/Library/Caches/Dousha/last_recording.wav --language auto --final-pass off -.build/xcode/Build/Products/Release/smoke-cli mlx-transcribe \ - ~/Library/Caches/Dousha/last_recording.wav --language zh-CN --context 'Vocabulary: Dousha, Qwen3' -.build/xcode/Build/Products/Release/smoke-cli mlx-transcribe \ - ~/Library/Caches/Dousha/last_recording.wav --language zh-CN --realtime --prewarm -``` - -`mlx-transcribe` uses `WavReader` (16kHz mono s16le only) and the actual engine -lifecycle, with 100ms PCM chunks, 10ms inter-chunk sleeps by default, or audio -rate with `--realtime`. It downloads the default model on first use; `--model` -accepts a compatible Hugging Face repo. Model load time includes any download -(open→store ready, sampled every 20ms); first-partial latency starts at first -ingest, and stop latency ends at completion. The command prints partial count, -final length, streaming preview and final transcript; keep private transcripts -out of shared reports. `--final-pass on|off` defaults to on; `--context` supplies -only the final batch system message. `--prewarm` awaits the same preload path -used at launch before opening the recording; `preload_ms` includes load and -kernel prewarm, while `model_load_ms` then measures the cached open. Compare -with/without prewarm in separate fresh processes. `stop_to_final_ms` includes streaming drain, -cleanup and batch; `batch_ms` measures batch only (elapsed time at fallback -delivery on a deadline), `deadline_seconds` is the computed batch allowance, -and `final_source` is `batch` or `stream`. -It services main-queue callbacks through async waits, with a 600s model-load -watchdog after open and a stop watchdog of max(80s, 48s + batch deadline). -Inspect engine logs for `finalTimedOut` even when a nonempty fallback makes the -command succeed. - -`Package.resolved` is committed and the mlx-audio-swift revision is pinned in -`Package.swift`. Upgrade mlx-audio-swift, mlx-swift, mlx-swift-lm and their -resolved dependencies as one compatibility set; rerun offline tests and both -language smoke checks after an upgrade. - Verification for any pipeline change: -1. `swift test` — full suite, green. -2. `make install` (quit the running app first) + relaunch — see CLAUDE.md for - the TCC/signing rules; a worktree needs `Makefile.local` or signing falls - back to ad-hoc and resets TCC grants. +1. `DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer swift test` — full suite. +2. Installed-bundle verification: follow AGENTS.md (quit → install → relaunch). 3. Real-device regression set for engine/network changes: - short dictation round-trip; - 3+ minute dictation with mid-sentence pauses (ping keepalive); @@ -412,26 +370,19 @@ Verification for any pipeline change: log for reconnect outcome); - rapid back-to-back recordings (detached close / concurrent-quota path). -For local-engine inference, also run the Xcode-built `mlx-transcribe` checks -above. They validate file-based inference without touching the installed app; -they do not replace installed-bundle testing of audio, hotkeys, HUD or TCC. +For local inference, also run §2.1’s Xcode-built smoke checks; they do not +replace installed-app testing of audio, hotkeys, HUD or TCC. ## 7. Removed design assets (QUA-263 / QUA-265) `ASRSegmentModel` (QUA-265) and `StreamingTextReconciler` (QUA-263) were -pure, fully-tested logic written ahead of features that never materialized. -They sat unwired with zero production evidence for their differentiators -(`nonstream_result=true`: **0 of 2935** streamed results across two log -files; `pauseBoundary`: no consumer) and were deleted in 2026-07. Resurrect -from git history — or rewrite against the notes below — only when a trigger -fires: - -- Segment model: Doubao `nonstream_result` late revisions observed in - production, a third engine needs a shared segmentation layer, or +unwired speculative logic, deleted in 2026-07. Restore from history only when: + +- Segment model: production Doubao `nonstream_result` late revisions appear + (previously 0/2935 results), a third engine needs shared segmentation, or pause-aware utterance boundaries become a product feature. -- Reconciler: text injection becomes incremental (per-key backspace + - retype). Today `TextInjector` is clipboard+⌘V by design (§5) and the HUD - reveal already keeps the stable prefix stable. +- Reconciler: injection becomes incremental (backspace/retype); today it is + clipboard+⌘V (§5), and HUD reveal already preserves the stable prefix. Decisions worth not re-deriving: @@ -451,6 +402,5 @@ Decisions worth not re-deriving: utterance's VAD final → revision; Soniox growing token tail → partial, utterance text at `` → final. -`TranscriptCorrector` (QUA-264) is unrelated to the above and IS wired — -`RecordingController.handleFinal` applies it once per dictation. The other -two are not, and should not be wired without the triggers above. +`TranscriptCorrector` (QUA-264) remains wired once per dictation in +`RecordingController.handleFinal`; the removed designs require the triggers above. diff --git a/README.md b/README.md index 591bfc5..57704f3 100644 --- a/README.md +++ b/README.md @@ -4,31 +4,39 @@

豆沙 (Dousha)

-A tiny macOS menu-bar app that gives you Doubao's voice dictation **without installing the Doubao IME**. Doubao (豆包) minus its wrapper = Dousha (豆沙) — just the filling, no bun. +A menu-bar dictation app for **macOS 14+ on Apple Silicon**. Speech becomes text wherever you're typing. -Hold a key, talk, release — the transcript pastes into whatever you're typing in. +- Choose **豆包 (Doubao)**, **Soniox**, **Apple Speech**, or **local Qwen3 via MLX**. Qwen3 runs offline after a one-time model download. +- Optionally run engines in parallel and route Chinese, English, and mixed-language results to your preferred engine. +- Refine transcripts with an OpenAI-compatible LLM and a custom prompt. +- Add names and terms to your glossary, and re-transcribe saved recordings from history. Spiritual fork of [SpeechMore](https://github.com/gfreezy/SpeechMore) + [DoubaoASR](https://github.com/gfreezy/DoubaoASR) by [@gfreezy](https://github.com/gfreezy). ## Install -Via [Homebrew](https://brew.sh) (easiest — `brew upgrade` keeps it current): +Via [Homebrew](https://brew.sh): ```sh brew install --cask giraphant/tap/dousha ``` -Or download the DMG from the [latest release](https://github.com/giraphant/dousha/releases/latest), drag to `/Applications`, and grant Microphone + Accessibility when asked. +Or download the DMG from the [latest release](https://github.com/giraphant/dousha/releases/latest) and drag Dousha to `/Applications`. -Or build it yourself (macOS 14+): +To build from source, install full Xcode, quit any running Dousha, then run: ```sh make install +open /Applications/Dousha.app ``` +See [AGENTS.md](AGENTS.md) for signing setup. Grant Microphone and Accessibility when prompted; Apple Speech also needs Speech Recognition permission. + ## Use -Hold **Right Shift**, speak, release. That's it. Open Settings from the menu bar to change the key, switch to toggle mode, or pick a language / engine. +Hold **Right Shift**, speak, release (push-to-talk). In Settings, you can change the key or use toggle mode: press once to start, again to stop. + +Open Settings from the menu bar to choose engines and language routing, download a local model, or configure Soniox and LLM credentials. For fully offline dictation, select only Qwen3 and leave LLM refinement off. Re-transcription results are copied to the clipboard. ## License