Summary
Observed across 7/10 device test scenarios for STT normalisation (PR #1070). When the user asks about a Māori/NZ cultural term (wharepaku, chocka, taniwha, kumara), the model calls query_wikipedia instead of checking the nz_truth_memories corpus — even though those terms are already in core/inference/src/main/assets/nz_truth_memories.json:2333-2388.
Root cause
The problem is prompt/tool-selection, not missing data. The cultural memory corpus already contains entries for all four terms. The model chooses Wikipedia over memory retrieval, often with worse results (e.g. Wikipedia returns the Māori language article for wharepaku, which is semantically correct but not useful).
Observed examples
- "what is a fattybaku" → wharepaku → Wikipedia (Māori language article) instead of cultural memory
- "what is chocka" → Wikipedia instead of cultural memory (which has the NZ slang definition)
- "tell me about taniwha" → Wikipedia instead of cultural memory
Suggested fix
Either:
- System prompt guidance to search memory before Wikipedia for NZ/cultural domain terms
- Re-rank tool selection to prefer memory retrieval when the query contains recognised Māori/NZ cultural terms
- Lower the threshold for
searchMemory vs query_wikipedia in the tool-selection decision
Acceptance criteria
Summary
Observed across 7/10 device test scenarios for STT normalisation (PR #1070). When the user asks about a Māori/NZ cultural term (wharepaku, chocka, taniwha, kumara), the model calls
query_wikipediainstead of checking the nz_truth_memories corpus — even though those terms are already incore/inference/src/main/assets/nz_truth_memories.json:2333-2388.Root cause
The problem is prompt/tool-selection, not missing data. The cultural memory corpus already contains entries for all four terms. The model chooses Wikipedia over memory retrieval, often with worse results (e.g. Wikipedia returns the Māori language article for wharepaku, which is semantically correct but not useful).
Observed examples
Suggested fix
Either:
searchMemoryvsquery_wikipediain the tool-selection decisionAcceptance criteria