feat(lumi-web): notas de voz end-to-end + deploy del demo

[R0SEWT]

Summary

Implementa soporte end-to-end de notas de voz en el demo web de Lumi (UI + API + puerto de transcripción con adapters), junto con documentación y artefactos de deploy; además incorpora el merge más reciente de main resolviendo el conflicto en Dockerfile.

Changes

• Añade flujo de notas de voz en el demo web (muestras predefinidas y grabación real).
• Introduce el puerto de transcripción y adapters (canned, Azure Whisper y faster-whisper local opcional).
• Extiende la API FastAPI para listar/reproducir notas de voz y subir audio con validaciones básicas.
• Agrega pruebas para adapters/transcripción y endpoints de voz, incluyendo límites/validaciones.
• Documenta arquitectura/configuración de voice notes y despliegue del demo en Azure App Service.
• Agrega artefactos de build/deploy del demo (Dockerfile, .dockerignore, extras opcionales de voz).
• Merge de origin/main con resolución quirúrgica del conflicto add/add en Dockerfile (manteniendo dependencias runtime necesarias del demo).

Testing

En esta sesión no se re-ejecutaron tests/lint porque el entorno no tenía herramientas instaladas (uv, pytest, ruff no disponibles).

Related

Actualización de mantenimiento del PR: sincronización con main y resolución de conflictos de merge.

[sourcery-ai]

Reviewer's Guide

Adds end-to-end voice note support to the Lumi web demo (UI, HTTP API, transcription port + adapters, and tests), wires Azure/OpenAI Whisper and optional faster-whisper as engines behind a new Transcriber port, and documents voice notes plus the Azure App Service demo deploy, including a minimal Docker image setup.

Sequence diagram for POST /api/voice/upload flow

sequenceDiagram
    actor Caregiver
    participant Browser
    participant FastAPIApp as FastAPI_app
    participant DemoRuntime as DemoRuntime_state
    participant Transcriber
    participant AzureOpenAI

    Caregiver->>Browser: click recBtn
    Browser->>Browser: toggleRecording()
    Browser->>Browser: uploadVoice(blob, duration_s)
    Browser->>FastAPIApp: POST /api/voice/upload
    FastAPIApp->>FastAPIApp: voice_upload(VoiceUploadIn)
    FastAPIApp->>FastAPIApp: base64.b64decode(audio_b64)
    FastAPIApp->>DemoRuntime: state.runtime.transcriber
    FastAPIApp->>Transcriber: transcribe(VoiceClip)
    Transcriber->>AzureOpenAI: audio.transcriptions.create(...)
    AzureOpenAI-->>Transcriber: text
    Transcriber-->>FastAPIApp: Transcript
    FastAPIApp->>DemoRuntime: _voice_turn(text, _fmt_duration)
    DemoRuntime-->>FastAPIApp: {transcript, reply, snapshot}
    FastAPIApp-->>Browser: JSON response
    Browser->>Browser: update voiceBubble()
    Browser->>Browser: renderPanel(snapshot)

Loading

File-Level Changes

Change	Details	Files
Implement voice note UX in the demo web UI, including sample chips and in-browser recording that call new voice APIs.	Extend chat HTML/CSS with voice-note specific chips, bubbles, waveform visuals, and recording button states. Add client-side logic to list sample voice notes, send scripted samples via /api/voice, and render transcript plus Lumi reply. Integrate MediaRecorder-based audio capture, base64 upload to /api/voice/upload, and appropriate fallbacks when recording is unsupported or transcription is not configured.	src/lumi/api/static/index.html
Expose server-side HTTP endpoints to serve sample voice notes, replay canned samples, and handle real audio uploads via a Transcriber abstraction.	Define new Pydantic models for voice note input and uploads, including size limits and base64 validation for audio payloads. Add helper utilities to format voice durations and to record a transcribed voice turn into the conversation transcript and route it through the normal check-in path. Implement /api/voice/samples, /api/voice, and /api/voice/upload routes using scripted fixtures and the runtime transcriber, including edge validation and empty-transcript behaviour. Wire a Transcriber instance into DemoRuntime and ensure it is available on the app state.	src/lumi/api/web.py src/lumi/api/bootstrap.py src/lumi/api/voice_samples.py
Introduce a speech-to-text port and concrete adapters for canned, Azure Whisper, and faster-whisper-based local transcription.	Define a Transcriber protocol plus VoiceClip and Transcript dataclasses as the speech-to-text port, keeping dependencies stdlib-only. Add a deterministic CannedTranscriber that resolves transcripts from an id->text map for demos and fast tests. Implement AzureWhisperTranscriber using AzureOpenAI on the classic deployment-scoped audio path, with environment-driven settings and lazy imports for openai/azure-identity. Implement FasterWhisperTranscriber using faster-whisper (CTranslate2), mapping MIME types to file suffixes and running offline transcription via a temp file. Provide a build_transcriber factory that prefers Azure, then optional local faster-whisper, then falls back to canned; gate construction on the use_ai flag so tests/offline demos avoid Azure clients.	src/lumi/ports/transcription.py src/lumi/adapters/media/__init__.py src/lumi/adapters/media/canned.py src/lumi/adapters/media/azure_whisper.py src/lumi/adapters/media/faster_whisper.py src/lumi/api/bootstrap.py
Add comprehensive tests around transcription port, adapters, voice APIs, and core boundary guarantees.	Test the canned transcriber and demo voice-note fixtures, including id uniqueness and duration formatting. Test AzureWhisperTranscriber behaviour with a fake client: argument wiring, MIME-to-filename mapping, env var parsing, endpoint normalisation, and non-storage of API keys. Test FasterWhisperTranscriber with a slow, optional integration that generates synthetic audio and asserts a well-formed Transcript when faster-whisper is installed. Test the new voice HTTP endpoints end-to-end using TestClient, including sample listing, scripted and uploaded flows, empty-engine behaviour, and base64/size validation errors. Extend the boundary test suite to assert that no heavy voice dependencies (faster_whisper/ctranslate2/torch) are imported at module level anywhere in the lumi package.	tests/lumi/media/test_transcription.py tests/lumi/media/test_azure_whisper.py tests/lumi/media/test_faster_whisper_slow.py tests/lumi/test_voice_api.py tests/lumi/test_boundaries.py
Document the voice note architecture and the live Azure App Service demo deploy, and add a minimal Docker image and optional voice extra.	Add a detailed VOICE_NOTES.md describing the Transcriber port, adapters, Azure Whisper path gotcha, configuration, and testing strategy. Extend README with sections for Voice notes and Live demo (deploy), including Azure CLI snippets to create the Whisper deployment, build/redeploy the container, and tear down resources, plus RBAC-driven design decisions. Introduce a Dockerfile for a slim web demo image that installs only Lumi runtime dependencies ([web]+[azure]) and runs the FastAPI app. Add a [voice] extra in pyproject.toml for faster-whisper and ensure the heavy dep stays out of the core install. Add a .dockerignore and update sample env as part of the deploy/docs setup.	README.md docs/VOICE_NOTES.md Dockerfile pyproject.toml .dockerignore .env.example

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've found 4 issues, and left some high level feedback:

• The MIME→file-suffix mapping is duplicated in both azure_whisper.py and faster_whisper.py; consider centralizing this mapping in a shared helper to keep behavior consistent and avoid drift.
• In the frontend voice-note handlers (sendVoice / uploadVoice), the response object is assumed to exist even if fetch fails; it would be safer to guard uses of r.transcript/r.reply behind a successful fetch check to avoid runtime errors.
• The Dockerfile hard-codes Python dependencies rather than installing from pyproject.toml, which can lead to version skew; consider using uv/pip with a lockfile or constraints derived from pyproject.toml instead.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- The MIME→file-suffix mapping is duplicated in both `azure_whisper.py` and `faster_whisper.py`; consider centralizing this mapping in a shared helper to keep behavior consistent and avoid drift.
- In the frontend voice-note handlers (`sendVoice` / `uploadVoice`), the response object is assumed to exist even if `fetch` fails; it would be safer to guard uses of `r.transcript`/`r.reply` behind a successful fetch check to avoid runtime errors.
- The Dockerfile hard-codes Python dependencies rather than installing from `pyproject.toml`, which can lead to version skew; consider using `uv`/`pip` with a lockfile or constraints derived from `pyproject.toml` instead.

## Individual Comments

### Comment 1
<location path="src/lumi/api/static/index.html" line_range="378-387" />
<code_context>
+    `<span class="meta">${fmtTime(when)} <span class="ticks">✓✓</span></span>`;
+  return b;
+}
+async function sendVoice(id, duration) {
+  const stream = $("stream");
+  const vb = voiceBubble(duration, new Date());
+  stream.appendChild(vb);
+  stream.scrollTop = stream.scrollHeight;
+  const typing = typingBubble();
+  stream.appendChild(typing);
+  stream.scrollTop = stream.scrollHeight;
+  let r;
+  try {
+    r = await fetch("/api/voice", {
+      method: "POST", headers: { "Content-Type": "application/json" },
+      body: JSON.stringify({ id }),
+    }).then((x) => x.json());
+  } finally {
+    typing.remove();
+  }
+  vb.querySelector(".vtrans").innerHTML =
+    `<b>📝 transcripción</b>${escapeHtml(r.transcript || "—")}`;
+  stream.appendChild(bubble({ role: "lumi", text: r.reply }, new Date()));
</code_context>
<issue_to_address>
**issue:** Guard against failed /api/voice requests before dereferencing the response object.

If `fetch("/api/voice")` fails, `r` stays `undefined` and the `vb.querySelector(".vtrans").innerHTML = ... r.transcript ...` line in `finally` will throw, breaking the UI after a failed request.

Consider aligning with `uploadVoice` by initializing `r` to `null`, handling errors in the `catch` (e.g. set an error message in `.vtrans` and return), and only dereferencing `r.transcript` after the `try/catch/finally`:

```js
let r = null;
try {
  r = await fetch(...).then((x) => x.json());
} catch (e) {
  vb.querySelector(".vtrans").innerHTML =
    `<b>📝</b> <i>no se pudo transcribir</i>`;
  return;
} finally {
  typing.remove();
}

const t = (r && r.transcript) || "";
// ...
```

This avoids throwing on failure and lets the bubble show a graceful error message.
</issue_to_address>

### Comment 2
<location path="src/lumi/adapters/media/azure_whisper.py" line_range="43-52" />
<code_context>
+    use_api_key: bool = False
</code_context>
<issue_to_address>
**nitpick:** Remove or use the unused `use_api_key` field to avoid configuration confusion.

`use_api_key` is never read; the logic only checks `AZURE_OPENAI_API_KEY`. This is misleading because toggling `use_api_key` has no effect. Please either remove the field or wire it into `AzureWhisperTranscriber.__init__` (e.g., to force Entra ID even when an API key is present) so the configuration accurately reflects behavior.
</issue_to_address>

### Comment 3
<location path="tests/lumi/test_voice_api.py" line_range="83-90" />
<code_context>
+    assert client.post("/api/voice/upload", json={"audio_b64": ""}).status_code == 422
+
+
+def test_upload_rejects_oversize_audio(client):
+    # An over-limit base64 string is rejected (413) BEFORE it is decoded, so a
+    # huge payload can't exhaust memory at the decode step.
+    from lumi.api.web import _MAX_AUDIO_B64_LEN
+
+    oversize = "A" * (_MAX_AUDIO_B64_LEN + 4)
+    r = client.post("/api/voice/upload", json={"audio_b64": oversize})
+    assert r.status_code == 413
</code_context>
<issue_to_address>
**suggestion (testing):** Add a test for the oversize-audio path *after* base64 decoding

There’s also a decoded-size guard in `/api/voice/upload` (`len(data) > _MAX_AUDIO_BYTES`) that we don’t currently exercise. Please add a test that sends a base64 string shorter than `_MAX_AUDIO_B64_LEN` but which decodes to more than `_MAX_AUDIO_BYTES`, and assert a 413 response to cover that branch.
</issue_to_address>

### Comment 4
<location path="README.md" line_range="95-98" />
<code_context>
+> ⚠️ Whisper on Azure is served on the **classic deployment-scoped path**
+> (`/openai/deployments/{deployment}/audio/transcriptions?api-version=…`), **not**
+> the `/openai/v1` surface the chat extractor uses — so its adapter drives the
+> `AzureOpenAI` client. Full details, config and the deploy gotcha in
+> [`docs/VOICE_NOTES.md`](docs/VOICE_NOTES.md).
+
</code_context>
<issue_to_address>
**nitpick (typo):** Sentence seems grammatically incomplete; consider adding a verb for clarity.

"Full details, config and the deploy gotcha in [`docs/VOICE_NOTES.md`]" reads incomplete without a verb. Consider wording like "Full details, config, and the deploy gotcha are in [`docs/VOICE_NOTES.md`]."

```suggestion
> (`/openai/deployments/{deployment}/audio/transcriptions?api-version=…`), **not**
> the `/openai/v1` surface the chat extractor uses — so its adapter drives the
> `AzureOpenAI` client. Full details, config, and the deploy gotcha are in
> [`docs/VOICE_NOTES.md`](docs/VOICE_NOTES.md).
```
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[sourcery-ai]

issue: Guard against failed /api/voice requests before dereferencing the response object.

If fetch("/api/voice") fails, r stays undefined and the vb.querySelector(".vtrans").innerHTML = ... r.transcript ... line in finally will throw, breaking the UI after a failed request.

Consider aligning with uploadVoice by initializing r to null, handling errors in the catch (e.g. set an error message in .vtrans and return), and only dereferencing r.transcript after the try/catch/finally:

let r = null;
try {
  r = await fetch(...).then((x) => x.json());
} catch (e) {
  vb.querySelector(".vtrans").innerHTML =
    `<b>📝</b> <i>no se pudo transcribir</i>`;
  return;
} finally {
  typing.remove();
}

const t = (r && r.transcript) || "";
// ...

This avoids throwing on failure and lets the bubble show a graceful error message.

[sourcery-ai]

nitpick: Remove or use the unused use_api_key field to avoid configuration confusion.

use_api_key is never read; the logic only checks AZURE_OPENAI_API_KEY. This is misleading because toggling use_api_key has no effect. Please either remove the field or wire it into AzureWhisperTranscriber.__init__ (e.g., to force Entra ID even when an API key is present) so the configuration accurately reflects behavior.

[sourcery-ai]

suggestion (testing): Add a test for the oversize-audio path after base64 decoding

There’s also a decoded-size guard in /api/voice/upload (len(data) > _MAX_AUDIO_BYTES) that we don’t currently exercise. Please add a test that sends a base64 string shorter than _MAX_AUDIO_B64_LEN but which decodes to more than _MAX_AUDIO_BYTES, and assert a 413 response to cover that branch.

[sourcery-ai]

nitpick (typo): Sentence seems grammatically incomplete; consider adding a verb for clarity.

"Full details, config and the deploy gotcha in [docs/VOICE_NOTES.md]" reads incomplete without a verb. Consider wording like "Full details, config, and the deploy gotcha are in [docs/VOICE_NOTES.md]."

Suggested change

	> (`/openai/deployments/{deployment}/audio/transcriptions?api-version=…`), **not**
	> the `/openai/v1` surface the chat extractor uses — so its adapter drives the
	> `AzureOpenAI` client. Full details, config and the deploy gotcha in
	> [`docs/VOICE_NOTES.md`](docs/VOICE_NOTES.md).
	> (`/openai/deployments/{deployment}/audio/transcriptions?api-version=…`), **not**
	> the `/openai/v1` surface the chat extractor uses — so its adapter drives the
	> `AzureOpenAI` client. Full details, config, and the deploy gotcha are in
	> [`docs/VOICE_NOTES.md`](docs/VOICE_NOTES.md).

[Copilot]

Pull request overview

This PR adds end-to-end voice note support to the Lumi demo (HTTP endpoints + web UI), introduces a speech-to-text port with multiple adapters (canned/demo, Azure Whisper, faster-whisper), and documents voice notes plus an Azure App Service deployment path.

Changes:

• Add a Transcriber port with adapters for Azure OpenAI Whisper, local faster-whisper, and a canned/demo transcriber.
• Extend the FastAPI demo with voice-note endpoints and update the static demo UI to send scripted and recorded voice notes.
• Add tests, docs (VOICE_NOTES.md), and deployment artifacts (Dockerfile, env example, .dockerignore).

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/lumi/test_voice_api.py	Adds API-level tests for voice sample listing, scripted voice routing, upload validation, and no-engine behavior.
tests/lumi/test_boundaries.py	Adds an AST-based boundary test to ensure heavy voice deps aren’t imported at module level.
tests/lumi/media/test_transcription.py	Tests the transcription port types and canned/demo transcript behavior.
tests/lumi/media/test_faster_whisper_slow.py	Adds a slow integration smoke test for the faster-whisper adapter.
tests/lumi/media/test_azure_whisper.py	Adds a fake-client unit test suite for the Azure Whisper adapter and env settings.
src/lumi/ports/transcription.py	Introduces VoiceClip, Transcript, and the Transcriber protocol as the STT port.
src/lumi/api/web.py	Adds /api/voice/samples, /api/voice, and /api/voice/upload plus payload bounds and decoding/validation.
src/lumi/api/voice_samples.py	Adds scripted demo voice-note fixtures and a transcript map.
src/lumi/api/static/index.html	Updates the demo UI to show voice-note chips and record/upload audio via MediaRecorder.
src/lumi/api/bootstrap.py	Wires a new build_transcriber() into the demo runtime with Azure→local→no-op selection.
src/lumi/adapters/media/faster_whisper.py	Adds a local faster-whisper adapter with deferred import and temp-file transcription.
src/lumi/adapters/media/canned.py	Adds a deterministic canned transcriber for demo/tests.
src/lumi/adapters/media/azure_whisper.py	Adds an Azure OpenAI Whisper adapter and settings loader with endpoint normalization.
src/lumi/adapters/media/init.py	Adds a package marker/docstring for media adapters.
README.md	Documents voice notes and the live demo deployment commands/decisions.
pyproject.toml	Adds the optional [voice] extra for faster-whisper.
docs/VOICE_NOTES.md	Adds detailed architecture/config docs for voice notes and Azure Whisper “classic path” gotcha.
Dockerfile	Adds a minimal container image for running the Lumi web demo with only web + azure deps.
.env.example	Adds env vars for enabling transcription deployments and language selection.
.dockerignore	Restricts Docker build context to keep the demo image build small.
.beads/issues.jsonl	Updates the project’s issue tracking data file with new entries.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 250b104678

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[R0SEWT]

@copilot resolve the merge conflicts in this pull request merge

[Copilot]

@copilot resolve the merge conflicts in this pull request merge

Conflictos resueltos: hice merge de origin/main en esta rama y resolví el conflicto en Dockerfile en el commit 9c4030c.