Repository navigation
feat: separate speakers in a meeting's microphone recording (D-007) - #2
Merged
Merged
Conversation
For meetings held in a room, where everyone reaches the microphone: a new "Separate speakers" action on the Audio tab works out who said what and labels each microphone line Speaker 1, Speaker 2 and so on. The labels reach copy, export, search, Ask and the notes model through combine(). Engine (src-tauri/src/diarise.rs), on Kōrero's own ort 2.0.0-rc.12 (one ONNX runtime, shared with transcribe-rs), CPU only, after the meeting: - pyannote segmentation-3.0 read for speaker changes (powerset), re-anchored every 10 s window; WeSpeaker ResNet34-LM embeddings; whole-meeting average-linkage clustering on segments >= 1.5 s, clusters under 20 s of speech folded in, the rest attached by voice. - Models download on first use (about 32 MB) and are checked by size and sha256 before use; a mismatch is discarded, never loaded. - Microphone lines take the speaker they overlap most. A line where the speaker changes mid-sentence takes whoever said most of it (word-timestamp splitting is a later step). Data: TranscriptSeg gains speaker (serde default, so every saved meeting loads unchanged); meetings gain speakersFound and speakerNames (naming UI is M3). One voice found: lines stay unlabelled. Editing a line is refused while that meeting's speakers are being separated, so the result cannot overwrite it. Evidence (legion, 16 Oct): - cargo test --lib: 262 passed (11 new), 5 ignored. bun test unit-tests: 82 passed. tsc: clean. eslint: 0 errors (3 pre-existing warnings elsewhere). clippy: no findings in the new code (64 pre-existing elsewhere; CI step is advisory). - diar_ami_der (ignored; local, 177 MB of AMI audio): mean DER 21.9% with a 0.25 s collar on four AMI far-field test meetings, under the 25% bar; the same output scores 22.0% in pyannote.metrics, which validates the check's own scorer. 92 minutes of audio separated in 65 s.
- D-008 (Accepted): the two speaker-separation models download on first use, pinned by size and sha256, instead of being bundled. - L-012 closes as check-added: diar_ami_der guards the shipped models. - L-013 (open): scripted edits normalised mixed line endings in src/bindings.ts and wrote garbled macrons from an ANSI-read script; both caught before any push. - Brief: B4's first slice recorded. product-check.py: 0 FAIL, 1 WARN (O3). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M9niYsTjphhTSYCLhCwg5n
An independent read-only review of the first commit found no Critical or High issues and no path that loses saved meetings. Fixed here: - M1, M2: meetings whose lines can't be matched by time to one recording are refused, in Rust (timing_problem) and in the UI (separationBlocker), with the reason shown: pre-1.26 meetings (no line timings: re-transcribe first) and combined meetings (marked mergedFrom from now on; older ones are caught because their times run backwards at the join). - M3, L6: model downloads have a 20 s connect and 10 min total timeout, so a stalled connection can't hold the one job slot; a failed download never leaves a .partial file behind. - M4: Activity, Today and the top bar name the job "Separating speakers" or "Downloading the speaker models" with a percentage, not "Re-transcribing, part 1048576 of 5983836". - L1: re-transcribing clears the stale speakersFound count. - L2: combine() groups by (source, speaker), so meetings without speakers give exactly the text they gave before (tested, including two channels renamed to the same label). - L3: turns join across the 61 ms gap between segmentation windows (tested). - L4: a non-finite embedding is treated as none. - L8: one separation at a time, enforced in Rust. - L10: finding no voice at all gets its own message. - L12: progress shows "Separating" once the download ends; download events are throttled to one per percent. Evidence (legion, 16 Oct): cargo test --lib 264 passed (2 new); bun test unit-tests 92 passed (10 new, unit-tests/speakers.test.ts); tsc clean; eslint: 0 warnings in the touched files. diar_ami_der: mean DER 21.5% (was 21.9%) with a 0.25 s collar, bar 25%. Not fixed here (follow-ups in the PR): L5 overlapping same-speaker turns, L7 hashing 32 MB on the async runtime per run, L9 the whole recording held in memory (about 230 MB per hour), L11 a line already open for editing when separation starts, L13 regenerate bindings.ts with tauri-specta, L14 CPU load if a recording starts during separation, L15 two licences to verify. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M9niYsTjphhTSYCLhCwg5n
nkeating-cmd
force-pushed
the
feat/diarise
branch
from
October 16, 2026 22:36
8d94503 to
5f0e7b7
Compare
This was referenced Oct 16, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Separate speakers for meetings held in a room, where everyone reaches the microphone. A new action on a meeting's Audio tab works out who said what from the microphone recording and labels each microphone line Speaker 1, Speaker 2 and so on. The labels reach the transcript view, copy, export, search, Ask and the notes model.
src-tauri/src/diarise.rs), D-007:ort 2.0.0-rc.12, so there is one ONNX runtime, shared withtranscribe-rs.TranscriptSeg.speakeris optional, with a serde default, so every saved meeting loads unchanged.speakersFound,speakerNames(for naming in M3) andmergedFrom.Stacked on #1 (the brief, decision log and lessons log). Merge that first; this diff then shrinks to the feature.
Why
Bet B4 in
docs/PRODUCT.md(in Now;product-check.py gate B4: PASS). Today a meeting's only "speakers" are the two capture channels, so everyone in a room is "You".Evidence (main laptop, 16 Oct)
cargo test --lib: 264 passed, 0 failed (13 new indiarise).bun test unit-tests: 92 passed (10 new inunit-tests/speakers.test.ts). These include checks that meetings without speakers give exactly the text they gave before.tsc: clean.eslint: 0 errors, and no warnings in the touched files.clippy: no findings in the new code (64 pre-existing elsewhere; the CI step is advisory).diar_ami_der(ignored; run locally, 177 MB of AMI audio):pyannote.metricsto within 0.2 points.cargo tree -i ort-sys: one ort.Not done yet
bindings.tswith tauri-specta. It was already stale; this PR adds one command by hand, matching the Rust exactly.Decision records: D-007, D-008 in
docs/DECISIONS.md. Lessons: L-012 closed (check-added), L-013 added.🤖 Generated with Claude Code
https://claude.ai/code/session_01M9niYsTjphhTSYCLhCwg5n