feat: add live transcript to pill mic OSD

[wojciechsacewicz]

TL;DR

This PR adds an opt-in animated live transcript above the pill microphone OSD while recording with ElevenLabs Scribe v2 Realtime.

The preview follows partial transcription updates as the user speaks, keeps the newest words visible, and clears itself at the end of the recording lifecycle. Final transcription, clipboard/paste behavior, audio streaming, and the existing pill visualization remain unchanged.

Showcase

What is added

• A live, word-based transcript rendered directly above the pill while recording.
• A rolling word window with a configurable limit from 1 to 12 words.
• Stable positioning for words shared between consecutive partial results.
• Provider correction handling: growth of the unfinished last word updates in place instead of restarting its animation.
• Latest-state rendering: rapid partial updates replace the current target rather than building an animation queue that falls behind speech.
• A configurable idle timeout while the visual timing and typography remain part of the pill's style.
• Safe fitting for narrow surfaces: text scales down when necessary and oversized unbroken tokens are ellipsized.
• Regression coverage for animation behavior, config/schema synchronization, provider gating, ElevenLabs reconnects, rendering layout, whitespace, and cleanup.

When the preview is active

The live transcript is deliberately scoped to the combination that supports and benefits from it:

• transcription_backend is realtime-ws;
• websocket_provider is elevenlabs;
• websocket_model starts with scribe_v2_realtime;
• realtime_mode is transcribe;
• the mic OSD is enabled in overlay mode;
• mic_osd_style is pill;
• mic_osd_pill_transcript_enabled is true.

Other provider/style combinations keep their previous behavior. In particular, OpenAI gpt-realtime-whisper partial previews remain available for the existing non-pill OSD styles, while the pill transcript stays ElevenLabs-only.

How it works

1. ElevenLabs assembles the current preview

ElevenLabsRealtimeClient keeps committed segments and the current partial segment as separate state. Whenever ElevenLabs emits a partial or committed transcript event, the client joins those parts into the current preview and invokes the registered callback.

The callback belongs to the client rather than a specific SDK connection instance, so it survives internal connection replacement and automatic reconnects. Preview state is copied while holding the transcript lock, but application code is called after releasing it to avoid lock inversion or blocking the audio/transcript event path.

2. The realtime backend gates and clears the stream

RealtimeWsBackend enables the callback only for supported provider, model, mode, and OSD combinations. It also clears the preview when starting a clean recording buffer, discarding a take, disabling the eligible combination, or cleaning up the realtime client, so text from a previous take cannot leak into the next one.

3. The pill animator tracks the latest partial result

PillTranscriptAnimator reduces every update to the configured number of recent words and compares it with the currently displayed window. Shared words keep their positions, new words animate in, removed words animate out, and layout changes interpolate between the old and new word positions.

Incoming updates replace the animation target immediately. This is important for realtime speech because provider partials can arrive faster than an animation completes; queueing them would make the UI increasingly stale.

4. Cairo renders independent animated words

The pill surface reserves vertical space for the transcript only when the feature can be active. Each word is drawn independently so it can have its own alpha, vertical offset, and transition progress.

Layout uses Cairo glyph advances instead of ink bounds. A rendered space has zero ink width, so using ink bounds would visually concatenate separately drawn words; glyph advances preserve whitespace and font side bearings. The renderer scales the line to the pill's maximum width and ellipsizes a single token that remains too wide at the minimum font size.

Configuration

Minimal example:

{
  "transcription_backend": "realtime-ws",
  "websocket_provider": "elevenlabs",
  "websocket_model": "scribe_v2_realtime",
  "realtime_mode": "transcribe",
  "mic_osd_enabled": true,
  "mic_osd_style": "pill",
  "mic_osd_pill_transcript_enabled": true,
  "mic_osd_pill_transcript_word_limit": 4
}

The feature is disabled by default. Its three settings are optional:

Setting	Default	Accepted range / values	Effect
mic_osd_pill_transcript_enabled	false	boolean	Enables the transcript when all provider/style requirements are met.
mic_osd_pill_transcript_word_limit	4	1–12	Sets how many recent words remain visible.
mic_osd_pill_transcript_idle_timeout_ms	1400	0–30000 ms	Hides text after no new partials; 0 keeps it visible until explicit cleanup.

Invalid numeric values fall back to defaults and supported values are clamped to the documented ranges.

Files changed

• lib/mic_osd/transcript_preview.py — the three validated user settings, word-window reconciliation, animation state, easing, idle timeout, and frame generation.
• lib/mic_osd/window.py — pill-only surface sizing, independent word rendering, glyph-advance layout, width fitting, ellipsizing, and preview lifecycle integration.
• lib/mic_osd/visualizations/pill.py — exposes pill geometry so transcript placement stays aligned with the existing visualization.
• lib/src/elevenlabs_realtime_client.py — assembles committed and partial transcript state, stores the reconnect-safe callback, and emits/clears previews.
• lib/src/backends/realtime_ws_backend.py — provider/model/style gating and cleanup integration without changing final transcript handling.
• lib/src/config_manager.py — opt-in default plus word-limit and idle-timeout defaults.
• share/config.schema.json — public schema entries, descriptions, types, defaults, and numeric bounds.
• docs/CONFIGURATION.md — requirements, example configuration, animation behavior, and the complete settings reference.
• docs/assets/pill-live-transcript.gif — the live UI showcase above.
• tests/test_pill_transcript_preview.py — animator behavior, rolling-window stability, rapid updates, corrections, timeout, and config validation.
• tests/test_elevenlabs_pill_preview.py — ElevenLabs assembly, reconnect persistence, provider/style gating, callback replacement, and cleanup.
• tests/test_mic_osd_runner.py — word spacing, oversized-token fitting, and existing preview transport behavior.
• tests/test_pill_visualization.py — pill geometry and transcript-capable surface expectations.

Compatibility and behavior safety

• Final transcript generation and paste behavior use the existing path and are not changed by the preview.
• The preview callback is optional and failures inside it are contained, so rendering cannot abort the ElevenLabs transcription event handler.
• GUI dependencies remain optional; tests use stubs and do not require GTK, Cairo, audio hardware, Wayland, or a network connection.
• The extra OSD height is reserved only for an eligible pill transcript configuration.
• Existing non-pill visualizations and OpenAI preview behavior are preserved.

Validation

Focused regression suite:

python -m unittest -v \
  tests.test_config_schema_sync \
  tests.test_pill_transcript_preview \
  tests.test_elevenlabs_pill_preview \
  tests.test_mic_osd_runner \
  tests.test_pill_visualization \
  tests.test_realtime_cancel_recovery \
  tests.test_realtime_reconnect_policy

Ran 68 tests in 4.023s — OK

Additional checks:

• git diff --check — passed.
• Live-tested on Linux with the pill OSD and ElevenLabs Scribe v2 Realtime: partial updates, rolling words, final paste, spacing, start/stop sounds, cleanup, and repeated recordings worked as expected.
• The full repository discovery command still encounters the existing src.paths import-order errors and maintenance-repair assertion. They reproduce on pristine upstream/main (301 tests: 1 failure, 13 errors). This branch reports 313 tests: 1 failure, 15 errors; the two additional discovery errors are the new mic OSD test modules hitting the same pre-existing import-order issue.

[goodroot]

Haha cool! Will review later today. Cheers 🍻

…

-------- Original Message --------

On Tuesday, 07/21/26 at 04:37 Wojciech Sacewicz ***@***.***> wrote: ***@***.***(https://github.com/wojciechsacewicz) requested your review on: [goodroot/hyprwhspr#217](#217) feat: add live transcript to pill mic OSD as a code owner. — Reply to this email directly, [view it on GitHub](#217?email_source=notifications&email_token=ACILTJMH4KBL45FDD2S4YYL5F5IVZA5CNFSNUABQM5UWIORPF5TWS5BNNB2WEL2JONZXKZKFOZSW45CON52GSZTJMNQXI2LPNYXTEOBSGY2TKMRVGA2TJJTSMVQXG33OWBZGK5TJMV3V64TFOF2WK43UMVSKKZLWMVXHJLDGN5XXIZLSL5RWY2LDNM#event-28265525054), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/ACILTJO5D2MOABQJD2IMOBD5F5IVZAVCNFSNUABGKJSXA33TNF2G64TZHMYTANBVHAYDCMBRHA5US43TOVSTWNBZGM4DENJRGUZDLILWAI). You are receiving this because your review was requested.Message ID: ***@***.***>

[goodroot]

@wojciechsacewicz Thanks a lot for this. It's really cool.

This should actually work just fine with the OpenAI real time endpoint as well. It does the same sort of chunking.

I've picked a few nits and made it general. Can you please ensure that it is working as intended? If so, let me know and we'll merge.

[wojciechsacewicz]

Thanks! I pulled your commit and tested the updated flow locally.

I ran into one small issue where the pill appeared but the live text didn't. The callback could be registered slightly too early, before the provider client was ready. I pushed a small follow-up that reapplies it after the realtime backend finishes starting.

ElevenLabs now works correctly in live testing. OpenAI follows the same path and its integration test passes, although I don't have access to the live OpenAI endpoint to verify it manually. I'm happy with it now.

Cheers 🍻