Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions sage/embodiment/ear.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,14 @@
the room: "only being directly addressed should wake it" -- run with SAGE_LISTEN=wake and SAGE_WAKE_NAME,
so words become heard lines only after the being's name (listening.py).

* TRANSCRIBE OUT OF PROCESS: SAGE_LISTEN_BACKEND=whispercpp. The capture is read by a Python thread, so
a transcriber that decodes in this process competes with it for the GIL. On McNugget (2026-10-24) the
MLX backend, with music in the room, was decoding in 20 of 20 py-spy samples and the reader in none:
samples were lost, "hey McNugget" reached whisper as fragments ("Okay." x28, dropped as silence), and
five days of tries never chimed. whisper-server (sage/embodiment/whispercpp.py) decodes in its own
process; the same audio replayed through it woke and heard correctly, and the first live try chimed.
MLX is fine for a quiet room or a test; a room with music needs the out-of-process backend.

Run as a service: python -m sage.embodiment.ear (SAGE_INSTANCE / SAGE_MEMBER as for presence.py)
"""
from __future__ import annotations
Expand Down
2 changes: 2 additions & 0 deletions sage/embodiment/listening.py
Original file line number Diff line number Diff line change
Expand Up @@ -294,6 +294,8 @@ def _load(self):
if BACKEND == "mlx":
# Apple silicon (McNugget, 2026-10-17): the same whisper model through MLX on the GPU. Measured there:
# base.en ~0.1 s per short phrase warm. Its segments carry no_speech_prob / avg_logprob like torch's.
# IT DECODES IN THIS PROCESS, holding the GIL the capture reader needs: in a room with music it starved
# the reader on McNugget (2026-10-24, see ear.py). For a live ear prefer whispercpp (its own process).
try:
import numpy as np
import mlx_whisper
Expand Down
Loading