Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion plugins/narrator/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ With a language set (see Settings) the switch shows it after its words: `⏵ nar

Where the footer is short the switch gives up `on` or `off`, then the language, then `narrator` (or `narrating`), and stays a `⏵`, `⏸︎`, `⏹︎` or `⏭︎` in its color, so the row never breaks in two. Beside [Sub Meter](../sub-meter), the meter shrinks first.

When a turn ends, Haiku (at low effort, so it is quick) reads your request, the tools Claude used and its final message, and writes one or two short spoken sentences in first person ("I added the narrator and the tests pass"), in the language you wrote in, or the one you set (see Settings). Haiku also names that language, and the mod picks the best voice installed for it; where there is none, the English one. A short answer already says it, so it is read out as it is, with no Haiku call: a path by its file's name (`hooks/register.tsx`: "register") and no commit hashes, in the language its words or your request tell, else the one the last narration was in (on Windows and Linux, Haiku narrates it while neither tells; macOS tells the language from the text). With a language set, a short answer is read as it is only when its words or your request tell it is in that language; any other, Haiku says in it.
When a turn ends, Haiku (at low effort, so it is quick) reads your request, the tools Claude used and its final message, and writes one or two short spoken sentences in first person ("I added the narrator and the tests pass"), in the language you wrote in, or the one you set (see Settings). Haiku also names that language, and the mod picks the best voice installed for it; where there is none, the English one. When the request's words tell its language, Haiku is told to speak it, and a narration in another language is not said: the answer itself is read instead. A short answer already says it, so it is read out as it is, with no Haiku call: a path by its file's name (`hooks/register.tsx`: "register") and no commit hashes, in the language its words or your request tell, else the one the last narration was in (on Windows and Linux, Haiku narrates it while neither tells; macOS tells the language from the text). With a language set, a short answer is read as it is only when its words or your request tell it is in that language; any other, Haiku says in it.

| System | Voice |
| --- | --- |
Expand Down
23 changes: 23 additions & 0 deletions plugins/narrator/hooks/register.test.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -442,6 +442,29 @@ test('the language setting shows beside the switch, tells the narrator and reach
expect((await ui.find({ key: 'narrator' }))?.text).toBe('⏹︎ narrating · ES')
})

/** A Spanish answer too long to be read as it is. */
const LARGO = `He añadido el **narrador** en \`plugins/narrator\`. ${'Habla después de cada turno, en tu idioma, con la mejor voz que tiene el sistema. '.repeat(3)}`

test('a request whose words tell its language: the narrator is told to speak it', async ($, on) => {
const w = world(on, { store: { enabled: true } })
await startSession($)
const speech = w.nextSpeech()
await turn($, { prompt: 'añade el narrador y que hable en el idioma de la petición', answer: LARGO })
await speech
expect(w.systems[0]).toContain('Always speak Spanish (es)')
expect(w.spoken[0]!.env.NARRATOR_TEXT).toBe('He añadido el narrador y los tests pasan.')
})

test('a narration in another language than the request is not said: the answer is read instead', async ($, on) => {
const w = world(on, { reply: 'es\nI added the narrator and the tests pass, so it is ready for you.', store: { enabled: true } })
await startSession($)
const speech = w.nextSpeech()
await turn($, { prompt: 'añade el narrador y que hable en el idioma de la petición', answer: LARGO })
await speech
expect(w.spoken[0]!.env.NARRATOR_TEXT).toStartWith('He añadido el narrador en narrator.')
expect(w.spoken[0]!.env.NARRATOR_LANG).toBe('es')
})

test('a region is dropped, and a value that is no known language means automatic', () => {
expect(languageOf('es-MX')).toBe('es')
expect(languageOf(' PT ')).toBe('pt')
Expand Down
15 changes: 12 additions & 3 deletions plugins/narrator/hooks/register.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -412,15 +412,24 @@ async function narrate ($: EngineInterface, answer: string) {
await speak($, narration, direct, directLanguage)
return
}
// The language to speak: the setting, else the request's when its words tell it. The model is told
// it in so many words: left to tell it from a request in one language and tool calls in another, it
// sometimes narrates in the wrong one.
const asked = settings.language || guessLanguage(request)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Do not treat the six-language guess as authoritative

In automatic mode, a normal unsupported-language request can be misclassified because guessLanguage's WORDS table in hooks/text.ts:6-13 only covers six languages and counts words shared with other languages. For example, Afegeix el narrador i fes que parli en la llengua de la petició is Catalan but returns es; this line therefore changes the system prompt to Always speak Spanish (es), and the later validation enforces the same mistake. Previously Haiku could infer Catalan from the request, so Catalan (and similarly Dutch) users now reliably hear a different language unless the setting is explicit; keep the automatic prompt when the detector cannot rule out an unlisted language.

Useful? React with 👍 / 👎.

const reply = await $.model.complete(
// A short summary: no deep thinking, so it is said right after the turn.
{ model: settings.model, system: NARRATOR(settings.language), prompt: brief(answer), maxTokens: 150, effort: 'low', timeoutMs: 30_000 },
{ model: settings.model, system: NARRATOR(asked), prompt: brief(answer), maxTokens: 150, effort: 'low', timeoutMs: 30_000 },
{ signal: narration.abort.signal }
).catch(() => undefined)
if (!narration.isActive) return
const told = reply?.isAnswered ? parseNarration(reply.text) : undefined
// No narration written: the answer itself, as far as it reads aloud.
const { text, language } = told?.text ? told : { text: cut(direct, 250), language: directLanguage }
// A narration in another language than the request's is not said, whatever code the model wrote:
// its words tell. The setting is left to the model, as the answer may be in neither.
const toldLanguage = told ? guessLanguage(told.text) || told.language : ''
const isOtherLanguage = !settings.language && asked !== '' && toldLanguage !== '' && toldLanguage !== asked
if (isOtherLanguage) $.ui.log(`narrator: the model narrated in ${toldLanguage}, not ${asked}: reading the answer instead`, { to: 'debug' })
// No narration written, or not in the request's language: the answer itself, as far as it reads aloud.
const { text, language } = told?.text && !isOtherLanguage ? told : { text: cut(direct, 250), language: directLanguage }
if (language) spokenLanguage = language
if (text) await speak($, narration, text, language)
} catch (error) {
Expand Down