Skip to content

Recognition runs in the user's language, the parser only speaks English #28

Description

@ostapondo

VoiceManager.recognizer() already prefers Locale.current when the on-device
model for it is on the Mac, and falls back to en-US only when it is not:

for locale in [Locale.current, Locale(identifier: "en-US")] {

So on a German Mac the transcript arrives in German. VoiceCommand.parse then
matches against placeVerbs, launchVerbs and numberWords, all English, so
nothing matches and every single utterance takes the long road to the agent —
including the dozen the app was supposed to answer itself, offline and
instantly. The feature is not broken, it is silently unavailable, which is
worse: it looks like the agent is just slow.

What to do

Not settled — the shape of this is worth arguing about on the issue before
anyone writes it. The two questions:

  • Where the phrases live. Today they are private static let arrays in
    VoiceCommands.swift. A second language could be a second set of arrays
    keyed by language code, or a table moved out to a resource. The first is a
    smaller change and reviewable; the second scales past two or three.
  • Which language a transcript is in. SFSpeechRecognizer was created with
    a known locale, so the answer is already on hand and does not need guessing
    from the text. Pass it to the parser rather than sniffing.

normalise is worth a look either way: it keeps anything isLetter says is a
letter, so it already survives accents and non-Latin scripts. That part does not
need changing.

The bar in the file stays exactly where it is, in every language. A false
positive silently does the wrong thing; a miss just takes the long road and
still works. The homophone problem gets worse, not better, across languages —
read the comment above numberWords about "to" once meaning zone two.

A good first contribution even without Swift

Adding a language is mostly not code. What is actually hard is knowing the four
or five ways people say "put this on the left" in a language, and which of them
are common enough in a sentence meant for the agent that matching them would
misfire. If you speak one natively, the list is the contribution — post it here
and someone else can wire it up.

Done when

At least one non-English language runs the local commands with the recogniser in
that locale, both the matches and the near misses are covered in
App/Tests/plonkTests/VoiceCommandTests.swift, and ./scripts/test.sh passes.
English behaviour is unchanged, tests included.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is needed

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions