diff --git a/LESSONS.md b/LESSONS.md new file mode 100644 index 0000000000..d0c60ccb2f --- /dev/null +++ b/LESSONS.md @@ -0,0 +1,41 @@ +# Lessons learned + +What went wrong or nearly went wrong in this repo, and what now stops it happening again. Kyt standard rule S12: +no lesson stays `open` for more than 14 days. Geordi reads this file every Monday. + +**Add a row when:** an incident or near miss happens, a code-review finding scores 80 or more, a scanner finds +something real, a release step fails, or a Claude session hits a trap worth not hitting twice. Record it the same +day, as `open`, in plain words. + +**Close it by choosing one status** (within 14 days): + +| Status | Means | The last column must say | +|---|---|---| +| `check-added` | A test, CI step, hook or script now fails if it happens again | the file or job that fails | +| `template-changed` | The shared template in `kyt-dev-platform` changed, so every repo gets it | the template | +| `skill-changed` | A skill (for example `ship`) now does it differently | the skill and step | +| `advice-only` | No check is practical; the reason is written down | why not | +| `retired` | No longer applies | why | +| `open` | Recorded, not yet decided | (blank) | + +Portfolio-wide lessons live in `kyt-dev-platform/lessons/PORTFOLIO-LESSONS.md`; a repo lesson that applies to every +repo is copied there by Geordi's weekly review. + +L-001 to L-010 were backfilled on 16 Oct 2026 from the Kōrero project record (state file and build notes). Their +dates are when each was found. Statuses were checked against the code on 16 Oct; where no check could be found, +the row is `open`, so several start past the 14-day limit and need a decision. + +| ID | Recorded | What happened | Lesson | Status | Check or change | +|---|---|---|---|---|---| +| L-001 | 2026-09-25 | The New Zealand English pass had never run for anyone on the default settings: it needs `en-NZ` and the default was `en`. A separate `nz_english_pass_enabled` (default on) looked like the switch and did nothing on its own (F13). | Test the shipped defaults, not only the feature with the right settings. | open | Fixed in 1.41 (`ensure_nz_locale_default`, `settings.rs`). No test that the pass runs under default settings was found on 16 Oct. | +| L-002 | 2026-09-25 | Meetings were lost or reverted when the user left the Meetings tab: Stop built the meeting inside the page, and a returning page saved its older copy over background results (B1–B3). | Background work writes through the store, never through the page that started it. | open | Fixed in 1.41 (`fix/meetings-background`). Whether a test covers it was not checked. | +| L-003 | 2026-09-25 | A 34-minute meeting produced no transcript: the microphone peaked at RMS 0.0073, under the fixed 0.010 speech gate, and nothing said so until after Stop (T1). | A fixed loudness gate fails quiet or distant microphones; check the signal while recording. | open | 1.41 reports it at Stop. The in-room plan (M1) replaces the fixed gate. | +| L-004 | 2026-09-25 | Release step B stopped with "uncommitted changes" in 1.41 and again in 1.42: the signed build rewrites `src-tauri/Cargo.toml` line endings (content identical). Restored by hand each time. | A build step must not leave the tree dirty; fix the cause, not each release. | open | A `.gitattributes` rule for `Cargo.toml`, or the release script restoring it, would close this. | +| L-005 | 2026-10-01 | A local `vite build` from `C:\Dev\korero` (capital D) fails on entry pages with inline styles; the folder on disk is `C:\dev`. | Build from the exact-case path. | advice-only | Vite's behaviour, not Kōrero's. The release scripts use `C:\dev\korero`; CI is unaffected. | +| L-006 | 2026-10-01 | Critical, found in review: "Move existing recordings" could write the meetings store after a load error, replacing the user's meetings (R-02). | Never write a store that failed to load. | open | Fixed in 1.42 (`storeLoadError` guards). Whether a test covers it was not checked. | +| L-007 | 2026-10-02 | Cloud AI provider keys are never stored in Windows Credential Manager: `keyring` is built without a platform feature, so it uses a mock store, and the migration then blanks the plaintext key. Keys are lost on restart (F15). Ollama is unaffected. | A secure-storage dependency needs a test that a value survives a restart. | open | Still present on 16 Oct (`keyring = "3.6"`, no platform feature). | +| L-008 | 2026-10-08 | Notes for long meetings covered only the first 48,000 characters on 8 of 35 meetings, and lost every flagged moment, with no warning. | Never cut input silently: either handle all of it or say in the output what was left out. | open | Fixed on `fix/1.44` (`notes_parts.rs`, 13 tests). Becomes `check-added` when 1.44 merges. | +| L-009 | 2026-10-08 | Audio brief failed with "voice engine not found" on the main laptop from the first build with PRIV-1 until 1.44: removing a developer path removed the only lookup that found the engine. | When a lookup path is removed, give its replacement (a setting) and a health check in the same change. | open | Fixed on `fix/1.44` (engine folder setting). | +| L-010 | 2026-09-25 | On the main laptop, local `main` tracks `origin`, which is upstream Handy, not `korero`; a bare `git push` targets the wrong repo. | Always push with `git push korero `. | open | One-line fix: `git branch -u korero/main main` on each machine. | +| L-011 | 2026-10-16 | The first in-room plan listed `pyannote-rs` as a fallback engine. The M0 probe found it does not compile against Kōrero's `ort` (29 errors), ignores the model's speaker-change output and drifts about 60 ms per 10 s. | Compile a dependency and read its source before naming it in a plan, even as a fallback. | advice-only | `app-dev-pm` Phase 1.5 already requires probing claims; this was a planning slip, caught by M0 before any app code. | +| L-012 | 2026-10-16 | M0's first scores were worse than calling everyone one speaker. The cause was the CAM++ embedding model packaged with `pyannote-rs`: it ran without error and gave stable output, but told AMI speakers apart no better than chance, even on headset audio. The harness was checked first (reference timing, feature scale, window type), then models were compared on clean labelled turns; WeSpeaker ResNet34-LM separated them 82–100% of the time. | A model that runs is not a model that works: score it on labelled data, and check the harness before trusting a bad number. | open | Plan M2 adds a check that scores the shipped models on the AMI test set and fails above the agreed error. | diff --git a/docs/DECISIONS.md b/docs/DECISIONS.md new file mode 100644 index 0000000000..7c3458ce64 --- /dev/null +++ b/docs/DECISIONS.md @@ -0,0 +1,21 @@ +# Kōrero: decision log + +The one decision log for Kōrero: decisions that change how Kōrero is built, released or behaves, oldest first. +Started 16 Oct 2026. D-001 to D-006 are backfilled from the project record (`KORERO_STATE.md` and the build notes +in the Kōrero claude.ai project), where some carried older labels (shown in brackets); those labels are retired. + +- **Status:** `Proposed` (written down, not decided; binds nothing), `Accepted`, `Superseded by D-NNN` or + `Withdrawn`. The code follows `Accepted` rows only. +- **Adding a row:** it goes in the same PR as the change it records. Take the next free number on `korero/main` + when the PR is ready to merge, not when drafting. +- **Product order** (what to build next) lives in `docs/PRODUCT.md`, not here. + +| # | Date | Status | Decision | Why | Alternatives rejected | Revisit when | +|---|---|---|---|---|---|---| +| D-001 | 2026-09-25 | Accepted | **Kōrero is built and released from the fork** (`nkeating-cmd/korero`, branch `main`). The overlay-and-patches build in `korero-build-tooling` (`korero-build.ps1`, `korero-ship.ps1`) is not used for releases. (Was "D8".) | Measured 23 Sept: the overlay was at 1.38.0 and none of the 26 files v1.40 changed matched it; `korero-ship.ps1` would force-push `korero-fork` over `main`. 1.41.0 shipped from the fork. | Overlay plus patches on upstream Handy: keeps upstream close, but the landing debt was already larger than the features. | A rebase onto upstream Handy is attempted (191+ commits behind at the time). | +| D-002 | 2026-09-25 | Accepted | **New Zealand English is the default** (`default_selected_language = "en-NZ"`). A one-time step moves an inherited `en` to `en-NZ` and records that it ran, so a user who switches it off keeps their choice. (Was "D12"; finding F13.) | The NZ pass (macrons, NZ place names and spelling) only runs under `en-NZ`, and the default was `en`, so it had never run for anyone on the default, including both of Nic's machines. | Leave `en` and add a prompt: depends on users noticing a setting they did not know mattered. | A user reports the NZ pass changing text they did not want changed. | +| D-003 | 2026-10-02 | Accepted | **Calls to a local Ollama model use Ollama's own chat API with the context sized to the request**, not its OpenAI-compatible endpoint; thinking is off for Ollama (1.42). | Ollama's default context is 4k tokens on GPUs under 24 GB and the OpenAI endpoint cannot raise it, so long meetings got notes from part of the transcript, silently. gemma4:12b timed out with thinking on. | Tell users to raise the context in Ollama: invisible to most users and not checked by the app. | Ollama's OpenAI endpoint gains a context setting; another local runtime is added. | +| D-004 | 2026-10-02 | Accepted | **Before a local model reads a long transcript, the speech model is freed** (local model, at least 4,000 characters, nothing recording, unload not set to Never; never waits). Nic chose this ("option A") for 1.43. | Both models together did not fit the graphics card on the main laptop. Cost: Parakeet reloads in about 4 s, mostly hidden behind the next recording. | Keep both loaded: out-of-memory on 12 GB cards. Smaller notes model: worse notes. | A machine with ample video memory is the norm, or the reload cost becomes noticeable. | +| D-005 | 2026-10-08 | Accepted | **Notes for a meeting over 48,000 characters are written in parts and combined** (split at speaker turns, at most 8 parts); shorter meetings are unchanged. Nic: "go with reading in parts". | 8 of 35 meetings on the main laptop had notes from the start only, and lost every flagged moment. | A bigger context window: needs more video memory than the target machines have, and quality falls off on long inputs. | Ask needs whole-meeting answers (it still reads one part and says so). | +| D-006 | 2026-10-08 | Accepted | **`wasapi` was upgraded from 0.23 to 0.25 to clear a security advisory**, rather than recording an exemption. Nic's choice during the 1.44 publish gates. | An exemption leaves the advisory in place; the upgrade's side effect (Windows manifest embedding in test binaries) was fixable. | Exempt the advisory in `cargo deny`. | — | +| D-007 | 2026-10-16 | Accepted | **In-room speaker separation runs in Kōrero's own code on its existing ONNX runtime** (`ort 2.0.0-rc.12`), after Stop, on the CPU, never live: pyannote segmentation-3.0 (MIT) read for speaker changes, not only speech; WeSpeaker ResNet34-LM speaker embeddings (CC BY 4.0, attribution in the third-party notices); clustering of the whole meeting on segments of 1.5 s or more, clusters under 20 s of speech folded into the nearest one, short fragments attached by voice. Plan: `KORERO_1.45_IN-ROOM-SPEAKERS-PLAN_2026-10-16.md` (v2). (Approved by Nic 16 Oct 2026: "Execute D007".) | M0 probe, 16 Oct, on four AMI test meetings recorded by one far-field room microphone (92 minutes): mean error 23.1% with a 0.25 s collar, 28.1% strict, against 22.4% strict for pyannote's own pipeline on all 16 test meetings; right number of people in 2 of 4 meetings (one too many in one, two too few in the other). About 80 s of CPU time per hour of audio on 4 threads. It builds in the same binary as `transcribe-rs` with one `ort`. | `pyannote-rs` 0.3.4: does not compile against `ort` rc.12 (29 errors); reads the model only as a speech detector; its clock drifts 60.6 ms per 10 s; its packaged CAM++ embedding model told AMI speakers apart no better than chance, even on headset audio (57.5% error with the right model, 80.5% as published). `sherpa-onnx`: a second ONNX runtime beside `ort`. Telling the clusterer the number of people: worse (31.5%) than finding it. The notes model guessing speakers: invents attributions. | Results on Nic's own room recordings differ from AMI; `knf-rs` (C++ build, `ndarray` 0.16) is replaced by Rust feature code. | diff --git a/docs/PRODUCT.md b/docs/PRODUCT.md new file mode 100644 index 0000000000..a1a65659ef --- /dev/null +++ b/docs/PRODUCT.md @@ -0,0 +1,99 @@ +--- +type: product-brief +app: korero +reviewed: 2026-10-16 +owner: Nic (decides the order); dev-product skill (maintains this file) +source_of_truth: this file; detail in the Kōrero claude.ai project (KORERO_STATE.md, plans and build notes) and the backlog kept outside the repo +wip_limit: 3 +--- + +# Kōrero: product brief + +What this app is for, who it is for, and the order we build in. Checked by +`product-check.py` (kyt-dev-platform). A session starts feature work only on a bet in **Now** +(`product-check.py gate docs/PRODUCT.md `); fixes and security work skip the gate. + +## Customer + +- **Who buys:** nobody yet. Kōrero is free and open source (a fork of Handy), released on GitHub. +- **Who uses:** Nic, on two Windows laptops. Release downloads suggest one or two installs in + total (E3), so for now the customer is Nic. +- **Job to be done:** when Nic is in a meeting, on a call or in a room, he wants an accurate local record of who + said what, and notes he can act on, so he can stay in the conversation and follow up without a cloud service + hearing it. When dictating, he wants New Zealand English and te reo Māori written correctly. +- **What they use today:** Kōrero itself; before that, cloud meeting tools and typing notes by hand. +- **Biggest unknown:** whether anyone besides Nic needs Kōrero. Decided 20 Sept: external users are parked, so + the brief optimises for Nic's own use until that changes. + +## Outcomes + +| ID | Outcome | Measure | Target date | +|---|---|---|---| +| O1 | Nothing said in a recorded meeting is lost | Over a month of Nic's meetings: no meeting with audio but an empty transcript, and no notes that silently skip part of the meeting | 2026-11-30 | +| O2 | In-room meetings come out with who said what | Speaker error on the AMI far-field test set at or under the bar Nic sets from M0, and Nic rates 3 real room meetings as usable | 2026-12-18 | +| O3 | New Zealand English and te reo Māori come out right | Te reo term accuracy on Nic's 108-line recording (never measured yet) | 2027-02-26 | +| O4 | Releases are safe and update themselves | Each release signed and gated; the second laptop offered the update on its own | 2026-11-30 | + +## Evidence + +Basis, in the same words Kriya uses: **research** (a real customer said or did it, or usage data shows it), +**requirement** (a law, rule, contract or accreditation obliges it), **hypothesis** (our belief, untested), +**simulated** (a persona panel or other AI output; never counts as proof). Quote the source so anyone can check it. +Sources named `KORERO_*` are in the Kōrero claude.ai project. + +| ID | Basis | Source | Date | What it shows | +|---|---|---|---|---| +| E1 | research | Meetings store on the main laptop, counted for `KORERO_MEETINGS-RELIABILITY_2026-09-25.md` | 2026-09-25 | Of 60 meetings, 5 had audio but an empty transcript and 25 had a transcript but no notes; Nic reported failures and work stopping when he left the Meetings tab | +| E2 | research | Meetings store on the main laptop, `KORERO_1.44_FIX-PLAN_2026-10-08.md` | 2026-10-08 | Notes for 8 of 35 transcribed meetings covered only the first 48,000 characters and lost every flagged moment | +| E3 | research | GitHub release download counts, `KORERO_REVIEW-NEXT-ROUND_2026-10-01.md` | 2026-10-01 | v1.41.0: 1 installer download and 2 `latest.json` downloads in 6 days, so about 1–2 installs | +| E4 | research | Nic's request in the Kōrero project chat | 2026-10-16 | Nic wants meetings designed for in-person use, with speakers separated automatically | +| E5 | research | A 34-minute meeting on the main laptop (T1 in `KORERO_MEETINGS-RELIABILITY_2026-09-25.md`) | 2026-09-14 | The microphone peaked at RMS 0.0073, under the fixed 0.010 speech gate, so nothing was transcribed | +| E6 | research | Nic's report, diagnosed in `KORERO_1.44_FIX-PLAN_2026-10-08.md` | 2026-10-08 | Audio brief failed with "voice engine not found" | +| E7 | hypothesis | Kōrero project plan, 16 Oct | 2026-10-16 | In-room meetings will often be recorded on a phone and imported later | +| E8 | hypothesis | `KORERO_v1.40_CHARTER_2026-09-06.md` | 2026-09-06 | Te reo Māori terms are often transcribed wrongly, and a lexicon or model choice can fix it; never measured | +| E9 | hypothesis | `KORERO_STATE.md` ("still to prove") | 2026-10-02 | Installed copies update themselves; never observed on the second laptop | +| E10 | hypothesis | Kōrero project plan, 16 Oct | 2026-10-16 | Remembering voices across meetings (auto-labelling Nic) would be worth the privacy cost | +| E11 | requirement | Biometric Processing Privacy Code 2025 (Privacy Commissioner), in force 3 Nov 2025 | 2025-11-03 | Any feature that uses a stored voiceprint to recognise people must meet the code's rules; it constrains B7, it does not justify it | +| E12 | hypothesis | Kōrero project plan, 16 Oct | 2026-10-16 | Speakers can be told apart well enough from one laptop or room microphone to be useful | +| E13 | hypothesis | Kōrero project plan, 16 Oct | 2026-10-16 | Remote voices picked up by the room microphone from a loudspeaker can be removed in software well enough | + +## Bets + +Type: **build** (a production feature), **learn** (exists to produce evidence: an interview round, a demo, a +prototype, an experiment), **enable** (platform, security or compliance the product needs either way). +Value and Effort are 1 to 5. Score = Value x Confidence / Effort for build and enable (Confidence comes from the +strongest evidence: research or requirement 1, hypothesis 0.5, simulated 0.25); Value / Effort for learn. +A build or enable bet enters Now only on research or a requirement. Keep Now to `wip_limit` bets. + +| ID | Bet | Type | Outcome | Evidence | Value | Effort | Status | Since | Link | +|---|---|---|---|---|---|---|---|---|---| +| B1 | Release 1.44: long-meeting notes read in parts; audio brief finds its engine | build | O1 | E2, E6 | 4 | 1 | now | 2026-10-08 | branch `fix/1.44`; `KORERO_1.44_BUILD_2026-10-08.md` | +| B2 | M0 probe: can speakers be separated on Kōrero's own runtime, from a room mic? | learn | O2 | E4, E12 | 4 | 1 | done | 2026-10-16 | `KORERO_1.45_IN-ROOM-SPEAKERS-PLAN_2026-10-16.md` | +| B3 | "In the room" meeting mode: mic only, no false call-audio warning, a speech gate that hears the far end of the table (M1) | build | O2, O1 | E4, E5 | 4 | 2 | now | 2026-10-16 | plan M1 | +| B4 | Separate speakers after Stop, then name, merge and reassign them; names reach the notes (M2–M3) | build | O2 | E4 | 5 | 4 | now | 2026-10-16 | plan M2–M3; `docs/DECISIONS.md` D-007 | +| B5 | Imports return timed segments and get speakers separated (M4) | build | O2 | E7 | 3 | 2 | next | 2026-10-16 | plan M4 | +| B6 | Confirm the second laptop is offered the next release on its own | learn | O4 | E9 | 3 | 1 | next | 2026-10-16 | `KORERO_STATE.md` | +| B7 | Remember voices across meetings | build | O2 | E10 | 2 | 4 | later | 2026-10-16 | plan D2; must meet E11 if built | +| B8 | Hybrid meetings (room plus remote people): stop remote speech appearing twice (M5) | learn | O2 | E4, E13 | 2 | 2 | later | 2026-10-16 | plan M5 | +| B9 | Record and score the 108-line te reo take; decide on the lexicon work from the number | learn | O3 | E8 | 3 | 2 | later | 2026-09-25 | `KORERO_PLAN_2026-09-25.md` | + +## Decisions and why + +Dated one-liners for every reorder, drop or gate decision, newest first. Larger product decisions go to +`docs/DECISIONS.md`. + +- 2026-10-16: Nic: "Execute D007". B4 moves to Now beside B1 and B3 (3 of 3), and is built on its own branch from `main`, ahead of B3 (M1); D-007 is Accepted. +- 2026-10-16: First brief (dev-product Mode A). Proposed order, for Nic to confirm: B1 then B3 in Now; B6, B4 and B5 + in Next. Nic sets the order; this is the session's proposal. +- 2026-10-16: B2 done. M0 found in-room separation workable on the room microphone (mean error 23.1% on four AMI + far-field meetings, collar 0.25), but only with a different embedding model from the one `pyannote-rs` ships + (D-007, L-012). Nic accepted D-007 the same evening. +- 2026-10-16: The "How many people?" prompt was dropped from B3. Told the count, the clusterer did worse (31.5%) than + finding the count itself (23.1%). +- 2026-10-16: B5 rests on a hypothesis (E7: room meetings get recorded on a phone), so it ranks below B4. Asking Nic + how he records room meetings would settle it. +- 2026-10-16: **O3 has no bet in Now or Next (checker WARN).** B9's kill date (2 Oct, `KORERO_PLAN_2026-09-25.md`) + passed with no record of the te reo take. Nic to decide: drop B9 and park O3, or schedule the 25-minute take. +- 2026-10-16: Fixes are not ranked here and skip the gate: cloud provider keys lost on restart (L-007) and the + release line-ending stop (L-004) are the two open ones. +- 2026-09-20: External users parked (Nic): the brief optimises for Nic's own use until that changes.