[Added] Azure OpenAI realtime provider - #2
Conversation
|
@isnine Thank you so much for this thoughtful contribution! 🙏 This is a very meaningful direction for AirTranslate, and I really appreciate the care you put into the provider abstraction, settings UI, localization updates, and tests. I reviewed the implementation more closely. The overall direction looks good, but I’d like to clarify a few Azure-specific details before merging.
Again, thank you very much for the thoughtful work here. I really like the direction, and I’d love to merge this once these points are confirmed. |
|
Sure, I’ll update the PR and let you know once it’s ready. |
|
@himomohi Can you take a look |
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c24a8e6ea5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| case .openAI: | ||
| return URL(string: "wss://\(host)/v1/realtime/translations?model=\(encodedModel)") | ||
| case .azure: | ||
| return URL(string: "wss://\(host)/openai/v1/realtime/translations?model=\(encodedModel)") |
There was a problem hiding this comment.
When the new Azure provider connects to this translations endpoint, Azure realtime translation streams text as response.text.delta/response.text.done with the payload in text; the receive loop below only recognizes session.output_transcript.* and only decodes delta/transcript. In that Azure translation configuration the WebSocket can be healthy while didTranslate is never called, leaving live captions blank, so the Azure event names/fields need to be mapped before enabling this path.
Useful? React with 👍 / 👎.
| transcription: OpenAIRealtimeTranscriptionConfig( | ||
| model: modelID |
There was a problem hiding this comment.
In Azure realtime translation mode, modelID here is the translation deployment used for the /translations?model=... connection, so the new customAzureTranscriptionModelName setting is ignored and the session asks Azure to use the translation deployment for audio.input.transcription. If input transcription is needed, pass the transcription override separately; otherwise omit this audio.input block so Azure translation sessions are not configured with the wrong deployment.
Useful? React with 👍 / 👎.
📝 WalkthroughWalkthroughThe PR adds Azure OpenAI configuration and realtime provider routing, persists provider-specific model settings, introduces reusable settings UI components, and makes floating caption alignment and presentation timing configurable. ChangesAzure OpenAI and realtime provider support
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant User
participant SettingsView
participant TranslationSessionStore
participant OpenAIRealtimeTranscriber
participant AzureOpenAIConfigStore
User->>SettingsView: Select Azure and enter endpoint/API key
SettingsView->>TranslationSessionStore: Save Azure configuration
TranslationSessionStore->>AzureOpenAIConfigStore: Persist normalized endpoint and key
TranslationSessionStore->>OpenAIRealtimeTranscriber: Start with Azure provider configuration
OpenAIRealtimeTranscriber->>OpenAIRealtimeTranscriber: Build Azure URL, headers, and session payload
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
⚔️ Resolve merge conflicts
Comment |
|
@coderabbitai review |
✅ Action performedReview finished.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
Sources/AirTranslate/Support/FloatingCaptionPresentationPolicy.swift (1)
52-58: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd boundary tests for the presentation policy.
Cover normalized-equal text, the 0.45-second revision boundary, prefix/non-prefix candidates, immediate display, and the 1.4/3.6-second dwell clamps.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@Sources/AirTranslate/Support/FloatingCaptionPresentationPolicy.swift` around lines 52 - 58, Add boundary-focused tests for FloatingCaptionPresentationPolicy, covering normalized-equal text, the 0.45-second revision threshold, prefix and non-prefix candidate handling, immediate display, and dwell durations clamped at 1.4 and 3.6 seconds. Exercise the public policy methods and assert both sides of each boundary without changing the policy implementation.Sources/AirTranslate/Views/SidebarView.swift (1)
1076-1184: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚖️ Poor tradeoffConsolidate the duplicated Azure/OpenAI credential rows.
GPTAzureConfigRowinSidebarView.swiftandSettingsAzureConfigRowinSettingsComponents.swiftare near-identical (as areGPTAPIKeyRow/SettingsAPIKeyRow). Maintaining two copies already caused the save-on-failure behavior to drift between the two screens. Extracting a single shared component removes that drift risk.
Sources/AirTranslate/Views/SidebarView.swift#L1076-L1184: replaceGPTAzureConfigRowusage with the sharedSettingsAzureConfigRowcomponent (and reuse the shared key row for the.openAIcase).Sources/AirTranslate/Views/SettingsComponents.swift#L308-L400: keepSettingsAzureConfigRowas the single source of truth reused by bothSettingsViewandConfigurationSheetView.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@Sources/AirTranslate/Views/SidebarView.swift` around lines 1076 - 1184, Consolidate the duplicated Azure credential row by removing or replacing GPTAzureConfigRow in Sources/AirTranslate/Views/SidebarView.swift lines 1076-1184 with the shared SettingsAzureConfigRow, and reuse the shared key-row component for the .openAI case. Keep SettingsAzureConfigRow in Sources/AirTranslate/Views/SettingsComponents.swift lines 308-400 as the single implementation used by SettingsView and ConfigurationSheetView; no direct change is required there unless needed to support both callers.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@Sources/AirTranslate/Views/SidebarView.swift`:
- Around line 505-524: Move the azureAPIKey reset inside the successful `ok`
branch of the save closure, alongside the successful configuration updates.
Preserve the entered key when `saveAzureOpenAIConfig` fails while keeping the
existing failure alert and focus behavior unchanged.
---
Nitpick comments:
In `@Sources/AirTranslate/Support/FloatingCaptionPresentationPolicy.swift`:
- Around line 52-58: Add boundary-focused tests for
FloatingCaptionPresentationPolicy, covering normalized-equal text, the
0.45-second revision threshold, prefix and non-prefix candidate handling,
immediate display, and dwell durations clamped at 1.4 and 3.6 seconds. Exercise
the public policy methods and assert both sides of each boundary without
changing the policy implementation.
In `@Sources/AirTranslate/Views/SidebarView.swift`:
- Around line 1076-1184: Consolidate the duplicated Azure credential row by
removing or replacing GPTAzureConfigRow in
Sources/AirTranslate/Views/SidebarView.swift lines 1076-1184 with the shared
SettingsAzureConfigRow, and reuse the shared key-row component for the .openAI
case. Keep SettingsAzureConfigRow in
Sources/AirTranslate/Views/SettingsComponents.swift lines 308-400 as the single
implementation used by SettingsView and ConfigurationSheetView; no direct change
is required there unless needed to support both callers.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 5654e59b-7fc5-4a76-b7d0-d2f817904ff5
📒 Files selected for processing (13)
Sources/AirTranslate/Models/AppText.swiftSources/AirTranslate/Models/FloatingCaptionTextAlignment.swiftSources/AirTranslate/Models/OpenAIProvider.swiftSources/AirTranslate/Services/AzureOpenAIConfigStore.swiftSources/AirTranslate/Services/OpenAIRealtimeTranscriber.swiftSources/AirTranslate/Services/OpenAITranslationService.swiftSources/AirTranslate/Services/TranslationSessionStore.swiftSources/AirTranslate/Support/FloatingCaptionPresentationPolicy.swiftSources/AirTranslate/Views/FloatingCaptionWindowView.swiftSources/AirTranslate/Views/OpenAIRealtimeModelPickers.swiftSources/AirTranslate/Views/SettingsComponents.swiftSources/AirTranslate/Views/SettingsView.swiftSources/AirTranslate/Views/SidebarView.swift
| save: { | ||
| let ok = session.saveAzureOpenAIConfig(endpoint: azureEndpoint, apiKey: azureAPIKey) | ||
| azureAPIKey = "" | ||
| if ok { | ||
| azureEndpoint = session.azureOpenAIEndpoint | ||
| configurationNotice = nil | ||
| shouldFocusOpenAIAPIKey = false | ||
| saveAlert = SaveAlert( | ||
| title: AppText.azureOpenAIConfigSavedTitle, | ||
| message: session.statusMessage | ||
| ) | ||
| } else { | ||
| configurationNotice = session.statusMessage | ||
| shouldFocusOpenAIAPIKey = true | ||
| saveAlert = SaveAlert( | ||
| title: AppText.azureOpenAIConfigSaveFailedTitle, | ||
| message: session.statusMessage | ||
| ) | ||
| } | ||
| }, |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Azure API key is wiped even when the save fails.
azureAPIKey = "" runs before the ok check, so on a recoverable failure (e.g. invalid endpoint) the user's typed secret is cleared and must be re-entered. SettingsView.saveAzureOpenAIConfig only clears it in the success branch; align this closure with that behavior.
🔧 Proposed fix
save: {
let ok = session.saveAzureOpenAIConfig(endpoint: azureEndpoint, apiKey: azureAPIKey)
- azureAPIKey = ""
if ok {
+ azureAPIKey = ""
azureEndpoint = session.azureOpenAIEndpoint
configurationNotice = nil
shouldFocusOpenAIAPIKey = false
saveAlert = SaveAlert(
title: AppText.azureOpenAIConfigSavedTitle,
message: session.statusMessage
)
} else {📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| save: { | |
| let ok = session.saveAzureOpenAIConfig(endpoint: azureEndpoint, apiKey: azureAPIKey) | |
| azureAPIKey = "" | |
| if ok { | |
| azureEndpoint = session.azureOpenAIEndpoint | |
| configurationNotice = nil | |
| shouldFocusOpenAIAPIKey = false | |
| saveAlert = SaveAlert( | |
| title: AppText.azureOpenAIConfigSavedTitle, | |
| message: session.statusMessage | |
| ) | |
| } else { | |
| configurationNotice = session.statusMessage | |
| shouldFocusOpenAIAPIKey = true | |
| saveAlert = SaveAlert( | |
| title: AppText.azureOpenAIConfigSaveFailedTitle, | |
| message: session.statusMessage | |
| ) | |
| } | |
| }, | |
| save: { | |
| let ok = session.saveAzureOpenAIConfig(endpoint: azureEndpoint, apiKey: azureAPIKey) | |
| if ok { | |
| azureAPIKey = "" | |
| azureEndpoint = session.azureOpenAIEndpoint | |
| configurationNotice = nil | |
| shouldFocusOpenAIAPIKey = false | |
| saveAlert = SaveAlert( | |
| title: AppText.azureOpenAIConfigSavedTitle, | |
| message: session.statusMessage | |
| ) | |
| } else { | |
| configurationNotice = session.statusMessage | |
| shouldFocusOpenAIAPIKey = true | |
| saveAlert = SaveAlert( | |
| title: AppText.azureOpenAIConfigSaveFailedTitle, | |
| message: session.statusMessage | |
| ) | |
| } | |
| }, |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@Sources/AirTranslate/Views/SidebarView.swift` around lines 505 - 524, Move
the azureAPIKey reset inside the successful `ok` branch of the save closure,
alongside the successful configuration updates. Preserve the entered key when
`saveAzureOpenAIConfig` fails while keeping the existing failure alert and focus
behavior unchanged.
himomohi
left a comment
himomohi
left a comment
There was a problem hiding this comment.
@isnine Thank you for the update. Before this can be merged, please address the current blocking items:
- Sync the branch with the latest
masterand resolve the merge conflicts. - Handle Azure translation text events (
response.text.delta/response.text.done) so translated captions are actually published. - Use the configured Azure transcription deployment separately, or omit input transcription from translation-only sessions.
- Preserve the entered Azure API key when validation or saving fails, as noted by CodeRabbit.
- After updating, please provide fresh results for
swift test,swift build -c release, and a real Azure Realtime translation smoke test.
Please also resolve the inline review conversations after addressing them. CodeRabbit will automatically review the new commits. Thank you.
2 participants
Azure OpenAI Realtime Provider
OpenAIProvider:openAI/azure) in Settings.AzureOpenAIConfigStoresecurely stores endpoint + API key (Keychain + UserDefaults), with endpoint normalization (AzureOpenAIEndpoint).OpenAIRealtimeTranscribernow takes anOpenAIRealtimeProviderConfigand routes requests per provider:Authorization: Bearer …, Azure usesapi-key: ….gpt-realtime-1.5).OpenAIRealtimeModelPickerslets users pick transcription / translation models or override the model ID per provider.API Preview
Floating Caption Controls
FloatingCaptionTextAlignment(leading/center) — caption window aligns text, frame, and stack accordingly.FloatingCaptionPresentationPolicy— governs immediate vs. dwell-based caption updates:SidebarViewexposes the alignment toggle and immediate-display switch alongside existing caption controls.Related Document
Summary by CodeRabbit