Skip to content

feat: separate ASR from translation and improve translation speed & reliability - #28

Open
aopstudio wants to merge 9 commits into
yyaadet:mainfrom
aopstudio:main
Open

aopstudio wants to merge 9 commits into
yyaadet:mainfrom
aopstudio:main

Conversation

@aopstudio

Copy link
Copy Markdown

Description

Summary

This PR implements the feature requested in issue #27: allow users to edit source subtitles before translation, instead of having translation happen immediately after ASR. It also fixes JSON parsing failures and optimizes translation throughput.


New Workflow

Before (old):

Generate Subtitles → ASR + translate in one step → no chance to edit source

After (new):

Generate Subtitles → ASR only → edit source text → [Translate] → bilingual ready

An Auto Translate toggle (default OFF) is provided next to the Generate button for users who prefer the original one‑step behavior.


What Changed

1. ASR/Translation Separation (fixes #27)

  • Generate Subtitles now does ASR only — translation is no longer automatic
  • New Translate button appears after ASR completes (disabled when target language is None)
  • Users can open the subtitle editor, fix ASR recognition errors, then translate

2. Editor Enhancements

  • Translate button in the editor toolbar — translates all source subtitles
  • Re-translate (N) button — re-translates only subtitles whose source was edited post‑translation
  • Auto Re-translate toggle — when enabled, editing source text triggers automatic re-translation on focus loss
  • Subtitle list auto-scrolls to match video playback position
  • Blue 🔄 indicator on items that need re-translation
  • Labels added to all toggles for clarity

3. Translation Reliability

  • Switched from JSON5 prompt format (data: without quotes) to strictly valid JSON ("data" with quotes), eliminating LLM response format confusion
  • extractJSONs now falls back through 3 strategies: markdown code blocks → full‑text parsing → brace‑region extraction
  • doTranslate throws on parse failure instead of silently returning an empty dictionary, allowing proper retry

4. Translation Speed

  • Batches are processed concurrently via withThrowingTaskGroup
  • Removed per‑subtitle doTranslateOne retry — failed items are collected into a single retry batch instead of making N individual API calls
  • Batch size is now configurable in Settings → LLM Service → Translation Batch Size (range 10–200, default 60)
  • Translation progress now shows real‑time speed (subtitles/s) and ETA

5. Misc

  • transcribe() now correctly skips translation when targetLanguage == .None (also addresses issue Target Language of "None" makes Chinese. #21)
  • SRT files are auto‑saved after translation/re‑translation
  • All new settings (autoTranslateAfterAsr, autoReTranslate, translationBatchSize) are persisted via UserDefaults / INI file

Files Changed

File Change
Models/Subtitle.swift +needsRetranslation, +isTranslated, sourceText.didSet auto‑mark
Models/Settings.swift +translationBatchSize property + serialize/deserialize
Services/WhisperService.swift +transcribeOnly(); transcribe() handles .None
Services/TranslationService.swift strict JSON prompts, parallel batches, removed doTranslateOne
Services/LLMService.swift multi‑strategy extractJSONs + tryParseJSON fallback
ViewModels/SubtitleViewModel.swift ASR‑only generate, +translateCurrentSubtitles, +reTranslateEditedSubtitles, auto toggles
Views/ContentView.swift +Translate button, +Auto Translate toggle
Views/SubtitleEditView.swift +Translate/Re‑translate buttons, +Auto Re‑translate toggle, auto‑scroll
Views/SettingsView.swift +Translation Batch Size slider

Testing Notes

  • Builds and runs on macOS 12.4+ (Xcode 26.3)
  • Uses ffmpegkit v6.0 xcframeworks from codewithtamim/ffmpeg-kit-spm (since the original upstream repo was archived)
  • ZIPFoundation built via carthage bootstrap

… Subtitle

- needsRetranslation tracks source-text edits made after translation existed
- isTranslated computed property checks for non-empty translatedText
- sourceText.didSet automatically sets needsRetranslation when isTranslated
- Add transcribeOnly() for ASR without translation
- transcribe() now skips translation when targetLanguage == .None
- Both methods preserve subtitle index mapping
TranslationService:
- Use strict JSON format in prompts (quoted keys) instead of JSON5
- Throw on parse failure instead of silent empty-dict return
- Support both {'data': {...}} and flat {...} response shapes
- Process all batches concurrently via withThrowingTaskGroup
- Eliminate per-subtitle doTranslateOne retry, retry missing items as batch instead

LLMService:
- extractJSONs falls back to full-text parsing when no code blocks found
- Adds tryParseJSON with 3 strategies: standard, JSON5, brace-region extraction
- Add translationBatchSize property to LLMService settings (default 60)
- Add INI serialize/deserialize for persistence
- Add slider in Settings UI (10-200, step 10)
Core flow:
- generateSubtitles() now does ASR-only, no translation
- add translateCurrentSubtitles() for post-ASR translation
- add reTranslateEditedSubtitles() for editing re-translation
- add canTranslate, hasNeedsRetranslation, hasUntranslatedSubtitles helpers

Auto toggles (persisted via UserDefaults):
- autoTranslateAfterAsr: when on, Generate does ASR+translate in one go
- autoReTranslate: when on, edits trigger automatic re-translation
- Auto-save SRT after translate/re-translate
- Add Translate button after ASR completes
- Button label switches to Re-translate Edited when needed
- Add Auto Translate toggle (one-step vs two-step mode)
- Tooltip guidance for disabled states
…ranslate

- Add Translate and Re-translate (N) buttons in bottom toolbar
- Add Auto Re-translate toggle with label
- Auto-scroll subtitle list to match video playback position
- Blue retranslation indicator on subtitle items needing re-translate
- Auto-trigger reTranslateEditedSubtitles on focus loss when enabled
@aopstudio aopstudio closed this Jul 13, 2026
@aopstudio aopstudio reopened this Jul 13, 2026
The editor was showing stale persisted data when editingSubtitles
was out of sync with subtitles. Now showSubtitleEditor() always
syncs editingSubtitles = subtitles before opening the window.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: Allow Translation After Editing Source Subtitles

1 participant