diff --git a/README.md b/README.md index ffab7e40..d88af38c 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,7 @@ Built with SwiftUI and Liquid Glass on iPhone, and Jetpack Compose on Android. L ## Android -A native Android client is available in [`apps/android/`](apps/android/README.md), with voice and written conversation, the same eight language modules, local learning records and iPhone-compatible JSON backups. Its interface is English, and Spanish on a phone set to Spanish. It runs on Android 8.0 or later and uses your own OpenAI API key stored with Android Keystore. The iPhone client remains available below. +A native Android client is available in [`apps/android/`](apps/android/README.md), with voice and written conversation, the same nine language modules, local learning records and iPhone-compatible JSON backups. Its interface is English, and Spanish on a phone set to Spanish. It runs on Android 8.0 or later and uses your own OpenAI API key stored with Android Keystore. The iPhone client remains available below. See the [Android installation/build guide](docs/run-on-android.md) and [Android verification record](verification/android-validation.md). Build a personal-install APK with Java 17 and Android SDK 36: @@ -83,7 +83,9 @@ A free Personal Team can run the app on your own phone; TestFlight and App Store - **A fresh start:** the Talk screen returns to its greeting 15 seconds after a conversation ends. Tap **New conversation** to reset immediately. Your saved conversations and learning remain. - **Local records:** export or import a JSON learning backup, delete a conversation, or delete all learning data from Settings. -The modules teach Norwegian Bokmål with an Eastern Norwegian voice target, Spanish from Spain, international English, French from France, German from Germany, Italian from Italy, Brazilian Portuguese and Standard Mandarin with Simplified Chinese. Each language has its own conversation themes, teaching guidance and progress. Valid regional alternatives are accepted. +The modules teach Norwegian Bokmål with an Eastern Norwegian voice target, Spanish from Spain, international English, French from France, German from Germany, Italian from Italy, Brazilian Portuguese, Standard Mandarin with Simplified Chinese, and Tagalog (Filipino) from the Philippines. Each language has its own conversation themes, teaching guidance and progress. Valid regional alternatives are accepted. + +Tagalog includes six teaching stages, Philippine conversation settings and vocabulary guidance that preserves verb focus. It uses the same meanings, word lookup and independent progress flows. [Teaching choices and references](docs/tagalog.md) explain the `tl` identity and the limits of automatic language detection. On iPhone, Mandarin includes optional pinyin in Talk, transcripts and word details. Chinese word lookup uses word boundaries, and the original characters remain available for copying from transcripts. Pinyin uses system dictionary readings; names, ambiguous words and tone changes in connected speech still need listening checks. Voice accent and teaching guidance are model instructions, and fluent-speaker review is still needed before making pronunciation or learning-effectiveness claims. @@ -115,12 +117,14 @@ xcodebuild -project apps/ios/Mural.xcodeproj -scheme Mural \ CODE_SIGNING_ALLOWED=NO ARCHS=arm64 ONLY_ACTIVE_ARCH=YES build ``` -For UI tests, create or select an iPhone 17 simulator in Xcode, then run **Product → Test**. The tests use in-memory fixtures and do not require an API key. More commands and preview options are in [the build guide](docs/build-and-test.md). +For UI tests, create a dedicated iPhone 17 simulator in Xcode, then run **Product → Test**. Most tests use in-memory fixtures; the relaunch test also writes language preferences to that simulator's local store. No API key is required. More commands and preview options are in [the build guide](docs/build-and-test.md). -The iPhone language release recorded on 13 September 2026 passed **70 core tests and 20 native UI tests**, including Mandarin pinyin, all four new onboarding choices and the largest accessibility text size. **79 backend tests** passed with an isolated PostgreSQL database and no skips. German, Italian, Brazilian Portuguese and Mandarin each passed a live iPhone check using synthetic typed replies and real voice output, meanings and word lookup. These checks do not establish human speech-recognition, pronunciation or correction quality. [Verification record](verification/validation.md) +On 13 September 2026, the nine-language build passed **90 core tests, 25 native UI tests and 80 backend tests**, with no failures or skipped backend tests. Debug simulator and unsigned Release iPhone builds passed. Tagalog checks include the largest accessibility text size, transcript retention, language isolation and normal relaunch persistence. Tagalog real-device and proficient-speaker review remain pending. Earlier German, Italian, Brazilian Portuguese and Mandarin live checks used synthetic typed replies and real voice output; those checks do not establish human speech-recognition, pronunciation or correction quality. [Verification record](verification/validation.md) The Android release branch is being prepared separately. See [release progress](verification/android-release-progress.md) for its checks and remaining gates. +After integration with the native-app monorepo on 14 September 2026, the Tagalog branch passed **93 Swift core tests, 25 iOS UI tests, 258 Android JVM tests, 304 API tests and 45 repository-tool tests**. Android lint and Debug APK assembly, cross-platform content checks, and the unsigned Release iPhone build passed. Tagalog is included in Android's generated catalog; physical-device and live-provider review remain pending. [Integration verification](verification/validation.md#14-september-2026--tagalog-integration-with-the-native-app-monorepo) + ## Code map | Directory | Contents | diff --git a/TAGALOG-PLAN.md b/TAGALOG-PLAN.md new file mode 100644 index 00000000..48ed57b7 --- /dev/null +++ b/TAGALOG-PLAN.md @@ -0,0 +1,87 @@ +# Tagalog (Filipino) — Philippines implementation record + +This completes the repository plan prepared on 13 September 2026 against `4aef373d85d5d64aee5d6adbf43092dae83c1f55`. Implementation and automated verification are the delivery scope. Real-device conversation and proficient-speaker review remain explicitly pending. + +## Feature contract + +| Field | Value | +| --- | --- | +| Module | `LanguageModule.tagalog` in `apps/ios/Core/Languages/Tagalog.swift` | +| Stable learning/storage ID | `tl` | +| Display name and native name | `Tagalog (Filipino)` | +| Variety / picker label | `Philippines` / `Tagalog (Filipino) · Philippines` | +| Locale | `tl-PH` | +| Greeting / lookup word | `Kumusta!` / `kumusta` | +| Writing system | Contemporary Latin-script Tagalog | + +Filipino is a learner-facing alias, with a single Tagalog progress namespace. The target uses the same onboarding, Settings, voice, typed reply, help, meaning, lookup, theme, topic, vocabulary and backup paths as existing languages. It does not add a second Filipino module or new subtitle/interface localization. The original eight IDs and Norwegian default stay unchanged. + +## Implementation map + +| Planned work | Implementation | +| --- | --- | +| Complete compiled language module | `apps/ios/Core/Languages/Tagalog.swift`: regional speech, writing, lemma rules, six challenge stages, topic placeholder and target-language lookup fallback. | +| Register the ninth target | `apps/ios/Core/Languages/LanguageModule.swift`: append `.tagalog`; both pickers derive their options and labels automatically. | +| Philippine themes | Override the existing `coffee`, `groceries`, `travel`, `cabin` and `traditions` IDs; inherit the other 19 themes. | +| All prompt paths | Existing `TeachingPolicy` functions interpolate the module, including its display alias; language-specific guidance keeps the target explicit. | +| Prevent false speech redirects | `TeachingPolicy.supportsSpeechLanguageDetection` declines automatic detection for `tl`; the coordinator avoids that unreliable check. Other targets retain their previous behavior. | +| Honest live-check diagnostics | `apps/ios/App/LanguageVerification.swift`: Tagalog reply/lookup fixtures, actual detector label/confidence, separate mechanical `flowPassed`, detection-dependent `passed` and pending language-quality review. | +| Native previews | `ConversationCoordinator.prepareConversationPreview`: matched Tagalog caption/meaning; Debug active-state fixture for the language-switch lock. | +| Experimental hosted locale | `services/api/src/live-provider.ts`: `tl-PH` regional target; negative aliases remain unsupported before any reservation/provider call. | +| Core regressions | `apps/ios/Tests/TagalogTests.swift`, expanded catalog fixtures, cross-language meaning and final-assessment cases. | +| Native regressions | `apps/ios/UITests/MuralUITests.swift`: onboarding, largest accessibility text, subtitle choice, Settings/themes/Words, transcript retention, switching lock and normal relaunch persistence. | +| Hosted regressions | Locale/provider request test plus PostgreSQL-backed Tagalog credit/minute reservation, settlement and invalid-alias coverage. | +| Android integration | Regenerated `apps/android/app/src/main/java/chat/mural/core/Languages.kt` from the Swift catalog; registry, English-support and archive-isolation regression tests. | +| Documentation | README, architecture, add-language/build guides, server contract, store metadata, [Tagalog references](docs/tagalog.md) and [verification record](verification/validation.md). | + +## Research and teaching decisions + +[Tagalog teaching choices and references](docs/tagalog.md) records the sources, including Tagalog.com dictionary entries and the University of Hawai‘i's aspect/focus grammar notes. The code contains original guidance, not a scraped dictionary or lesson corpus. + +- Use everyday Tagalog and context-appropriate polite address; accept valid regional usage and learner support languages. +- Teach aspect in context, with voice/focus and participant markers, rather than mechanically copying English tense categories. +- Group aspect variants under a consistent citation form while retaining meaningful affixes and distinct senses. For example, `kumain` and `kainin` stay separate. +- Preserve source spelling, diacritics, apostrophes, hyphens and exact quotations. Generated pronunciation annotations do not become learning evidence. +- Keep Philippine settings specific without treating all communities as culturally or linguistically identical. + +## Detector finding and chosen tradeoff + +The local `NLLanguageRecognizer` probe (Xcode 26.6 / Swift 6.3.3) returned the following results on both macOS and an iOS 26.5 simulator: + +| Input | Leading result | +| --- | --- | +| `Kumusta! Gusto kong matutong magsalita ng Tagalog. Maaari ba tayong mag-usap tungkol sa pagkain at paglalakbay sa Pilipinas?` | Indonesian (`id`), 0.99247 confidence | +| `Kung magbubukas ka ng kapihan, paano mo mapapanatiling abot-kaya ang mga presyo habang gumagamit ng mga lokal na sangkap?` | Indonesian (`id`), 0.99974 confidence | +| `Gusto ko ng kape.` | Croatian (`hr`), 0.38728 confidence | + +The long samples exceed the coordinator's 70-character threshold and the policy's 0.88-confidence threshold. A catalog-only addition would redirect valid speech. The fix disables this detector-based guard for Tagalog while retaining explicit target-language prompts. It does not alias Indonesian to Tagalog or force a positive detector result. Actual English drift also cannot be corrected through this detector, so live review must check for it. + +The verification helper records this limit: `flowPassed` can succeed independently, while `passed` requires reliable target-language detection. Neither outcome certifies pronunciation, recognition of a human learner, correction accuracy or teaching effectiveness. + +## Test coverage and acceptance + +- [x] One stable ID, correct locale and display alias; original language identities retained. +- [x] Every prompt path, all six stages and stable cultural-theme IDs covered. +- [x] Unaided recall, typed/subtitle-assisted practice, imitation, foreign/mixed/alias proposals and invalid provenance covered. +- [x] Aspect-family storage, separate focus/sense keys, hidden cognates, punctuation and exact archive text covered. +- [x] Archive v2 selection/topics/import and rejection of unsupported IDs tested; existing v1 migration retained. +- [x] Late meaning results/errors and final assessments cannot cross into another target. +- [x] Detector regressions cover high-confidence wrong labels, unsupported observations and unchanged behavior for previous targets. +- [x] Native onboarding, Settings, theme and vocabulary labels, large text, transcript retention, active switching lock and normal relaunch tests implemented. +- [x] Server accepts `tl-PH`, rejects unsupported aliases without credit reservation, and settles the fake-provider session correctly. + +See [the verification record](verification/validation.md) for executed checks, totals and build results. Simulator previews and the test database contain temporary synthetic data; they make no real provider calls. + +## Compatibility and limits + +No archive-version change, SwiftData migration, new package dependency or project regeneration is required. Older app versions without the module reject `tl` archives; upgrade before importing. Retain `tl` in future releases so saved records remain readable. The server addition does not enable hosted accounts/billing or require server deployment for the current native BYOK path. + +`LearningEngine.validate` checks proposal labels and transcript provenance, not the actual linguistic identity of the text. Tests using supplied lemmas validate storage and evidence behavior; they cannot establish that a model always produces correct lemmas or assessments. No inference of independent learning quality is made from synthetic typed fixtures. + +Remaining release review: + +- Real iPhone voice conversation with the saved API key, speaker/headphones, interruption, mute and closure. +- Human speech recognition, generated translations, sourced-topic behavior and correction/lemma quality in actual provider output. +- Proficient-speaker review of pronunciation, register, naturalness and cultural examples. + +Run the Debug helper with `--verify-audio --verify-language-flow --verify-language=tl` when real-device testing is available. It uses temporary learning records and incurs API usage. The full manual checklist is in [the build guide](docs/build-and-test.md) and [verification record](verification/validation.md). diff --git a/apps/android/README.md b/apps/android/README.md index 4626c2af..3bfbfe3e 100644 --- a/apps/android/README.md +++ b/apps/android/README.md @@ -1,8 +1,8 @@ # Mural for Android -Native Kotlin and Jetpack Compose client for Android 8.0 or later. It offers voice and written conversation, eight learning languages, 24 themes, meanings, vocabulary and local history, using the owner's OpenAI API key. Mandarin captions link each word and show optional pinyin on Android 10 or later; Android 8 and 9 keep word links without pinyin. +Native Kotlin and Jetpack Compose client for Android 8.0 or later. It offers voice and written conversation, nine learning languages including Tagalog (Filipino) from the Philippines, 24 themes, meanings, vocabulary and local history, using the owner's OpenAI API key. Mandarin captions link each word and show optional pinyin on Android 10 or later; Android 8 and 9 keep word links without pinyin. -Word taps in all eight languages open a contextual meaning sheet, matching the iOS flow. Mandarin uses a bundled offline phrase dictionary with an ICU fallback; unresolved common ambiguous readings retain their source characters. +Word taps in all nine languages open a contextual meaning sheet, matching the iOS flow. Mandarin uses a bundled offline phrase dictionary with an ICU fallback; unresolved common ambiguous readings retain their source characters. Interface copy lives in `res/values` (English) and `res/values-es` (Spanish); Android picks the translation from the phone's language. diff --git a/apps/android/app/src/main/java/chat/mural/core/Languages.kt b/apps/android/app/src/main/java/chat/mural/core/Languages.kt index 8a7292af..99496338 100644 --- a/apps/android/app/src/main/java/chat/mural/core/Languages.kt +++ b/apps/android/app/src/main/java/chat/mural/core/Languages.kt @@ -201,7 +201,27 @@ object LanguageRegistry { "cabin" to ConversationTheme("cabin", "周末出游", "A change of scene", "mountain.2", "Local life", "一起设想一个周末旅行，选择城市、海边或乡村，讨论实际安排和喜欢做的事情。", 2), "traditions" to ConversationTheme("traditions", "日常习俗", "Small customs, big stories", "flag", "Local life", "用普通话聊日常习俗和节日。比较学习者熟悉的地方，避免把任何一种习惯说成所有人的共同体验。", 2)) ) - val all = listOf(norwegian, spanish, english, french, german, italian, portuguese, mandarin) + private val tagalog = LanguageModule( + id = "tl", + name = "Tagalog (Filipino)", + nativeName = "Tagalog (Filipino)", + variety = "Philippines", + locale = "tl-PH", + greeting = "Kumusta!", + greetingWord = "kumusta", + speechGuidance = "Use clear, natural everyday Tagalog as spoken in the Philippines. Filipino is a learner-facing name for this teaching option; do not switch to another Philippine language. Use po and opo, and polite kayo, when the relationship and situation call for respect; do not force formal address between friends. Accept valid regional Tagalog and established loanwords without treating them as errors. Welcome English or mixed learner replies as support and bridge back to a useful Tagalog phrase, while keeping your own speech in Tagalog. Do not imitate a regional caricature or infer stress, glottal-stop or other pronunciation errors from a transcript alone.", + writingGuidance = "Use contemporary Latin-script Tagalog with natural conversational wording. Use ordinary spelling in replies; preserve any learner-supplied diacritics, apostrophes and hyphens in quotations. Keep meaningful distinctions such as ng and nang, linkers na and -ng, and reduplicated forms such as araw-araw. Accept common conversational contractions without automatically calling them wrong. Do not append English translations, pronunciation guides or Baybayin to ordinary replies.", + lemmaGuidance = "Use a consistent Tagalog dictionary citation form: nouns without ang, ng, sa or a plural marker, and verbs in their neutral/infinitive form with the voice/focus affix retained. For example, group kumakain and kakain under kumain, but kinain and kinakain under the separate lemma kainin; do not reduce both to the root kain. Keep useful chunks intact and preserve lexical hyphens. Use ordinary unaccented citation spelling consistently unless a diacritic is needed to distinguish a lexical meaning; keep distinct English senses separate. Preserve the observed form and exact quotation, including any diacritics. In mixed replies, include only identifiable Tagalog words or established loanwords used in Tagalog context; omit ambiguous or English-only evidence rather than labeling it tl.", + topicPlaceholder = "Food, travel, music, life in the Philippines…", + lookupUnavailableReply = "Hindi ko ito masuri ngayon. Kung gusto mo, puwede nating pag-usapan ang paksa sa pangkalahatan.", + teachingFocus = listOf("Greetings, introductions, thanks and useful requests such as kumusta, salamat and gusto ko ng, with one short exchange at a time.", "Everyday questions, pronouns including tayo versus kami, ang/ng/sa markers, basic linkers and polite po/opo in context.", "Connected accounts of activities and plans; common completed, ongoing and contemplated verb aspects with time expressions, rather than a direct copy of English tenses.", "Reasons and opinions, actor and object voice/focus, consistent participant markers, connected clauses and practical problem-solving.", "Hypotheticals, reported information, conversational particles such as na, pa, naman and daw/raw, nuanced requests and changes of register.", "Flexible extended discussion with precise, idiomatic Tagalog, natural aspect and voice choices, appropriate tone and respect for regional variation."), + themeOverrides = mapOf("coffee" to ConversationTheme("coffee", "Kape tayo?", "Something warm, please", "cup.and.saucer", "Everyday", "Meet at a neighbourhood café in the Philippines. Order a drink and chat about the learner's day. Use natural Tagalog and polite requests appropriate to the situation.", 0), + "groceries" to ConversationTheme("groceries", "Sa palengke", "A little of everything", "basket", "Everyday", "Shop at a local market in the Philippines. Practise quantities, prices and friendly requests. Respect regional food names and ask about preferences.", 2), + "travel" to ConversationTheme("travel", "Saan tayo pupunta?", "Find your way", "tram", "Everyday", "Plan a trip in the Philippines. Choose suitable transport together and practise directions, destinations and tickets without inventing current routes, fares or schedules.", 1), + "cabin" to ConversationTheme("cabin", "Isang maikling bakasyon", "A change of scene", "mountain.2", "Local life", "Plan an imagined weekend in the Philippines. Let the learner choose a city, coast or countryside, then discuss travel, food and activities.", 2), + "traditions" to ConversationTheme("traditions", "Kuwentuhan sa hapag", "Stay a little longer", "fork.knife", "Local life", "Talk over an imagined meal in the Philippines about routines, family and everyday customs. Compare individual experiences without assuming one religion, language or tradition represents everyone.", 2)) + ) + val all = listOf(norwegian, spanish, english, french, german, italian, portuguese, mandarin, tagalog) fun get(id: String) = all.firstOrNull { it.id == id } } diff --git a/apps/android/app/src/test/java/chat/mural/core/CoreTest.kt b/apps/android/app/src/test/java/chat/mural/core/CoreTest.kt index 4d60c6c7..507e6f43 100644 --- a/apps/android/app/src/test/java/chat/mural/core/CoreTest.kt +++ b/apps/android/app/src/test/java/chat/mural/core/CoreTest.kt @@ -49,7 +49,7 @@ class CoreTest { assertEquals(2,merged.sessions.size) } @Test fun languageRegistryAndThemesStayStable() { - assertEquals(listOf("nb","es","en","fr","de","it","pt","zh"),LanguageRegistry.all.map { it.id }) + assertEquals(listOf("nb","es","en","fr","de","it","pt","zh","tl"),LanguageRegistry.all.map { it.id }) assertEquals(24,Themes.shared.map { it.id }.toSet().size) assertEquals("Salut !",LanguageRegistry.get("fr")!!.greeting) for ((id, locale, greeting) in listOf(Triple("de","de-DE","Hallo!"),Triple("it","it-IT","Ciao!"),Triple("pt","pt-BR","Olá!"),Triple("zh","zh-CN","你好！"))) { @@ -60,4 +60,23 @@ class CoreTest { assertEquals("Norwegian",LanguageRegistry.get("nb")!!.name) assertEquals("Hei!",MeaningLanguages.greeting("Norwegian")) } + @Test fun tagalogContentAndArchiveUseOneNamespaceWithEnglishSupport() { + val language = LanguageRegistry.get("tl")!! + assertEquals("Tagalog (Filipino) · Philippines", language.settingsTitle) + assertEquals("tl-PH", language.locale) + assertEquals("Kumusta!", language.greeting) + assertEquals(6, language.teachingFocus.size) + assertEquals("Kape tayo?", language.themes.first { it.id == "coffee" }.title) + assertNull(LanguageRegistry.get("fil")) + assertTrue(TeachingPolicy.translation(language, "English").contains("into English")) + assertTrue(TeachingPolicy.assessment(language).contains("Use language tl for target-language evidence")) + val archive = Archive(sessions = mutableListOf(evidence(language = "tl"), evidence(language = "es")), + preferences = Preferences(learningLanguageID = "tl", meaningLanguage = "English")) + val restored = ArchiveCodec.decode(ArchiveCodec.encode(archive)) + assertEquals("tl", restored.preferences.learningLanguageID) + assertEquals("English", restored.preferences.meaningLanguage) + assertEquals("tl|radio|radio", LearningEngine.project(restored.sessions, languageID = "tl").words.single().id) + assertEquals("es|radio|radio", LearningEngine.project(restored.sessions, languageID = "es").words.single().id) + } + } diff --git a/apps/android/app/src/test/java/chat/mural/core/TagalogTest.kt b/apps/android/app/src/test/java/chat/mural/core/TagalogTest.kt new file mode 100644 index 00000000..22500895 --- /dev/null +++ b/apps/android/app/src/test/java/chat/mural/core/TagalogTest.kt @@ -0,0 +1,81 @@ +package chat.mural.core + +import org.junit.Assert.* +import org.junit.Test + +class TagalogTest { + private fun record(text: String, lemma: String, form: String, meaning: String, day: Int = 0, + languageID: String = "tl", typed: Boolean = false, supported: Boolean = false): SessionRecord { + val date = 810_000_000.0 + day * 86400 + val session = SessionRecord(languageID = languageID, startedAt = date, themeID = "coffee") + session.append(Fragment(speaker = Speaker.user, text = text, startMS = 100_000, endMS = 103_000, + receivedAt = date, meaningVisible = supported, typed = typed)) + val passage = session.passages.single() + session.assessments += Assessment(passageID = passage.id, revisionKey = passage.revisionKey, + outcome = Outcome.success, suggestedLevel = 2, nextGoal = "Magtanong tungkol sa presyo.", + capability = "Orders a drink", createdAt = date, + words = listOf(WordProposal(lemma, meaning, form, EvidenceKind.independent, .95, + passage.fragments.map { it.id }, text, languageID))) + return session + } + + @Test fun captionsPreserveTagalogWordLinksAndExactSourceText() { + for ((text, expected) in listOf( + " Mag-aaral ako araw-araw.\n" to listOf("Mag-aaral", "ako", "araw-araw"), + "Ako'y masaya. Siya’y narito!" to listOf("Ako'y", "masaya", "Siya’y", "narito"), + "Bukás ang pinto; búkas tayo aalis. ☕️" to listOf("Bukás", "ang", "pinto", "búkas", "tayo", "aalis"), + )) { + val segments = CaptionWords.segments(text, "tl", null) + assertEquals(text, segments.joinToString("") { it.text }) + assertEquals(expected, segments.mapNotNull { it.lookup }) + } + } + + @Test fun aspectVariantsShareCitationFormWhileVoiceFocusRemainsDistinctAfterExport() { + val sessions = mutableListOf( + record("Kumakain ako ngayon.", "kumain", "Kumakain", "eat"), + record("Kakain ako mamaya.", "kumain", "Kakain", "eat", day = 2), + record("Kinain ko ang saging.", "kainin", "Kinain", "eat", day = 2), + ) + val restored = ArchiveCodec.decode(ArchiveCodec.encode(Archive(sessions = sessions))) + val words = LearningEngine.project(restored.sessions, languageID = "tl", now = sessions[1].startedAt).words + assertEquals(setOf("tl|kumain|eat", "tl|kainin|eat"), words.map { it.id }.toSet()) + assertEquals(2, words.single { it.lemma == "kumain" }.independentCount) + assertEquals(1, words.single { it.lemma == "kainin" }.independentCount) + assertEquals(sessions.map { it.passages.single().text }, restored.sessions.map { it.passages.single().text }) + } + + @Test fun hidingOneHomographSenseDoesNotHideAnotherSenseOrLanguage() { + val sessions = listOf( + record("Bukás ang pinto.", "bukas", "Bukás", "open"), + record("Bukas ako aalis.", "bukas", "Bukas", "tomorrow"), + record("radio", "radio", "radio", "radio"), + record("radio", "radio", "radio", "radio", languageID = "es"), + ) + val hidden = listOf("tl|bukas|open", "tl|radio|radio") + assertEquals(listOf("tl|bukas|tomorrow"), LearningEngine.project(sessions, "tl", hidden).words.map { it.id }) + assertEquals(listOf("es|radio|radio"), LearningEngine.project(sessions, "es", hidden).words.map { it.id }) + } + + @Test fun foreignAndDisplayAliasEvidenceCannotEnterTagalogVocabulary() { + val session = record("Gusto ko ng kape.", "kape", "kape", "coffee") + val assessment = session.assessments.single() + for (language in listOf("en", "mixed", "fil", "tl-PH", "id")) { + val proposal = assessment.copy(words = assessment.words.map { it.copy(language = language) }) + assertTrue(language, LearningEngine.validate(proposal, session)!!.words.isEmpty()) + } + assertEquals("tl|kape|coffee", LearningEngine.validate(assessment, session)!!.words.single().key) + } + + @Test fun EnglishSubtitleSupportAndTypingCannotBecomeUnaidedRecall() { + for ((typed, supported) in listOf(true to false, false to true, true to true)) { + val session = record("Gusto ko ng kape.", "kape", "kape", "coffee", typed = typed, supported = supported) + val restored = ArchiveCodec.decode(ArchiveCodec.encode(Archive(sessions = mutableListOf(session), + preferences = Preferences(learningLanguageID = "tl", meaningLanguage = "English")))) + val evidence = LearningEngine.validate(restored.sessions.single().assessments.single(), restored.sessions.single())!! + assertEquals(EvidenceKind.assisted, evidence.words.single().kind) + assertEquals(0, LearningEngine.project(restored.sessions, languageID = "tl").words.single().independentCount) + assertEquals("English", restored.preferences.meaningLanguage) + } + } +} diff --git a/apps/ios/App/ConversationCoordinator.swift b/apps/ios/App/ConversationCoordinator.swift index c48e32f5..a3b27fa1 100644 --- a/apps/ios/App/ConversationCoordinator.swift +++ b/apps/ios/App/ConversationCoordinator.swift @@ -331,17 +331,19 @@ import MuralCore if state == .ended, let resetDeadline, Date() >= resetDeadline { resetConversation() } } #if DEBUG - func prepareEndedPreview() { + func prepareConversationPreview(active: Bool) { guard ProcessInfo.processInfo.arguments.contains("--preview") else { return } if let argument = ProcessInfo.processInfo.arguments.first(where: { $0.hasPrefix("--preview-language=") }) { selectLanguage(String(argument.dropFirst("--preview-language=".count))) } selectedTheme = language.themes.first { $0.id == "coffee" } var record = SessionRecord(languageID: language.id, themeID: selectedTheme?.id, title: selectedTheme?.title) - let sample = ["nb": "Jeg liker kaffe.", "de": "Ich mag Kaffee.", "it": "Mi piace il caffè.", "pt": "Eu gosto de café.", "zh": "我喜欢喝咖啡。"] + let sample = ["nb": "Jeg liker kaffe.", "de": "Ich mag Kaffee.", "it": "Mi piace il caffè.", "pt": "Eu gosto de café.", "zh": "我喜欢喝咖啡。", "tl": "Gusto ko ng kape."] record.append(Fragment(speaker: .assistant, text: sample[language.id] ?? language.greeting, startMS: 0, endMS: 1000)) record.translations[MeaningRequest.cacheKey(revisionKey: record.passages[0].revisionKey, language: "English")] = "I like coffee." - session = record; state = .closing; finish(final: true) + session = record + if active { state = .active; scheduleTranslation() } + else { state = .closing; finish(final: true) } } #endif #if DEBUG && targetEnvironment(simulator) @@ -396,7 +398,8 @@ import MuralCore } } private func checkLanguage() { - guard let p = assistantPassage, p.text.count > 70, p.id != lastLanguageCheck else { return } + guard TeachingPolicy.supportsSpeechLanguageDetection(language: language), + let p = assistantPassage, p.text.count > 70, p.id != lastLanguageCheck else { return } let recognizer = NLLanguageRecognizer(); recognizer.processString(p.text) if let detected = recognizer.languageHypotheses(withMaximum: 2).max(by: { $0.value < $1.value }), TeachingPolicy.shouldRedirectSpeech(language: language, detectedLanguageID: detected.key.rawValue, confidence: detected.value) { diff --git a/apps/ios/App/LanguageVerification.swift b/apps/ios/App/LanguageVerification.swift index 70062590..fd2bde40 100644 --- a/apps/ios/App/LanguageVerification.swift +++ b/apps/ios/App/LanguageVerification.swift @@ -15,6 +15,10 @@ extension AudioVerification { var connected = false var receivedGreeting = false var targetLanguageDetected = false + var languageDetectionReliable: Bool + var detectedLanguageID: String? + var detectedLanguageConfidence: Double? + var languageQualityReview = "pending" var typedReplies = 0 var translated = false var lookupReturned = false @@ -28,20 +32,21 @@ extension AudioVerification { var peakAudioLevel = 0.0 var outputPorts: Set = [] var failure: String? - var passed: Bool { - connected && receivedGreeting && targetLanguageDetected && typedReplies == 2 && translated && + var flowPassed: Bool { + connected && receivedGreeting && typedReplies == 2 && translated && lookupReturned && (languageID != "zh" || pinyinAvailable) && supportedEvidenceOnly && archiveRoundTrip && switchedAwayAndBack && cachedMeaningAfterEnd && closed && audioReleased && peakAudioLevel > 0.001 && outputPorts.contains(AVAudioSession.Port.builtInSpeaker.rawValue) && failure == nil } + var passed: Bool { flowPassed && languageDetectionReliable && targetLanguageDetected } } let id = coordinator.language.id - var report = Report(languageID: id) + var report = Report(languageID: id, languageDetectionReliable: TeachingPolicy.supportsSpeechLanguageDetection(language: coordinator.language)) let destination = URL.documentsDirectory.appendingPathComponent("language-verification-\(id).json") func write() { // Encode computed pass status explicitly alongside the report. - struct Output: Encodable { let passed: Bool; let report: Report } - if let data = try? JSONEncoder().encode(Output(passed: report.passed, report: report)) { + struct Output: Encodable { let passed: Bool; let flowPassed: Bool; let report: Report } + if let data = try? JSONEncoder().encode(Output(passed: report.passed, flowPassed: report.flowPassed, report: report)) { try? data.write(to: destination, options: .atomic) } } @@ -84,7 +89,8 @@ extension AudioVerification { "de": "Wenn du ein Café eröffnen würdest, wie würdest du regionale Zutaten und bezahlbare Preise miteinander vereinbaren?", "it": "Se aprissi un bar, come riusciresti a usare ingredienti locali mantenendo prezzi accessibili?", "pt": "Se você abrisse uma cafeteria, como conciliaria ingredientes locais com preços acessíveis?", - "zh": "如果你开一家咖啡馆，你会怎样在使用本地食材和保持价格合理之间取得平衡？" + "zh": "如果你开一家咖啡馆，你会怎样在使用本地食材和保持价格合理之间取得平衡？", + "tl": "Kung magbubukas ka ng kapihan, paano mo mapapanatiling abot-kaya ang mga presyo habang gumagamit ng mga lokal na sangkap?" ] for reply in ["I am learning. How can I politely order a coffee?", advanced[id] ?? "Tell me more."] { let before = coordinator.session?.fragments.filter { $0.speaker == .assistant }.count ?? 0 @@ -96,12 +102,14 @@ extension AudioVerification { } let recognizer = NLLanguageRecognizer() recognizer.processString(coordinator.caption) - if let detected = recognizer.dominantLanguage?.rawValue { - report.targetLanguageDetected = detected == id || detected.hasPrefix(id + "-") + if let detected = recognizer.languageHypotheses(withMaximum: 2).max(by: { $0.value < $1.value }) { + report.detectedLanguageID = detected.key.rawValue + report.detectedLanguageConfidence = detected.value + report.targetLanguageDetected = detected.key.rawValue == id || detected.key.rawValue.hasPrefix(id + "-") } report.pinyinAvailable = MandarinPinyin.reading(coordinator.caption) != nil report.translated = await waitFor(20) { !coordinator.meaning.isEmpty && !coordinator.translating } - let lookupWords = ["de": "Kaffee", "it": "caffè", "pt": "café", "zh": "咖啡"] + let lookupWords = ["de": "Kaffee", "it": "caffè", "pt": "café", "zh": "咖啡", "tl": "kape"] do { let result = try await coordinator.lookup(word: lookupWords[id] ?? coordinator.language.greetingWord, sentence: coordinator.caption) report.lookupReturned = !result.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty diff --git a/apps/ios/App/RootView.swift b/apps/ios/App/RootView.swift index 09fb2629..fc85eb8b 100644 --- a/apps/ios/App/RootView.swift +++ b/apps/ios/App/RootView.swift @@ -52,7 +52,8 @@ struct RootView: View { #if DEBUG .task { if AudioVerification.requested { await AudioVerification.run(coordinator) } - else if ProcessInfo.processInfo.arguments.contains("--ended-conversation") { coordinator.prepareEndedPreview() } + else if ProcessInfo.processInfo.arguments.contains("--ended-conversation") { coordinator.prepareConversationPreview(active: false) } + else if ProcessInfo.processInfo.arguments.contains("--active-conversation") { coordinator.prepareConversationPreview(active: true) } } #endif } diff --git a/apps/ios/Core/Languages/LanguageModule.swift b/apps/ios/Core/Languages/LanguageModule.swift index b3914a4f..2ea8ea9b 100644 --- a/apps/ios/Core/Languages/LanguageModule.swift +++ b/apps/ios/Core/Languages/LanguageModule.swift @@ -27,7 +27,7 @@ public struct LanguageModule: Identifiable, Sendable { public enum LanguageRegistry { public static let defaultID = "nb" - public static let all: [LanguageModule] = [.norwegian, .spanish, .english, .french, .german, .italian, .portuguese, .mandarin] + public static let all: [LanguageModule] = [.norwegian, .spanish, .english, .french, .german, .italian, .portuguese, .mandarin, .tagalog] public static func module(for id: String) -> LanguageModule? { all.first { $0.id == id } } } diff --git a/apps/ios/Core/Languages/Tagalog.swift b/apps/ios/Core/Languages/Tagalog.swift new file mode 100644 index 00000000..1bb7241d --- /dev/null +++ b/apps/ios/Core/Languages/Tagalog.swift @@ -0,0 +1,28 @@ +import Foundation + +extension LanguageModule { + public static let tagalog = LanguageModule( + id: "tl", name: "Tagalog (Filipino)", nativeName: "Tagalog (Filipino)", variety: "Philippines", locale: "tl-PH", + greeting: "Kumusta!", greetingWord: "kumusta", + speechGuidance: "Use clear, natural everyday Tagalog as spoken in the Philippines. Filipino is a learner-facing name for this teaching option; do not switch to another Philippine language. Use po and opo, and polite kayo, when the relationship and situation call for respect; do not force formal address between friends. Accept valid regional Tagalog and established loanwords without treating them as errors. Welcome English or mixed learner replies as support and bridge back to a useful Tagalog phrase, while keeping your own speech in Tagalog. Do not imitate a regional caricature or infer stress, glottal-stop or other pronunciation errors from a transcript alone.", + writingGuidance: "Use contemporary Latin-script Tagalog with natural conversational wording. Use ordinary spelling in replies; preserve any learner-supplied diacritics, apostrophes and hyphens in quotations. Keep meaningful distinctions such as ng and nang, linkers na and -ng, and reduplicated forms such as araw-araw. Accept common conversational contractions without automatically calling them wrong. Do not append English translations, pronunciation guides or Baybayin to ordinary replies.", + lemmaGuidance: "Use a consistent Tagalog dictionary citation form: nouns without ang, ng, sa or a plural marker, and verbs in their neutral/infinitive form with the voice/focus affix retained. For example, group kumakain and kakain under kumain, but kinain and kinakain under the separate lemma kainin; do not reduce both to the root kain. Keep useful chunks intact and preserve lexical hyphens. Use ordinary unaccented citation spelling consistently unless a diacritic is needed to distinguish a lexical meaning; keep distinct English senses separate. Preserve the observed form and exact quotation, including any diacritics. In mixed replies, include only identifiable Tagalog words or established loanwords used in Tagalog context; omit ambiguous or English-only evidence rather than labeling it tl.", + teachingFocus: [ + "Greetings, introductions, thanks and useful requests such as kumusta, salamat and gusto ko ng, with one short exchange at a time.", + "Everyday questions, pronouns including tayo versus kami, ang/ng/sa markers, basic linkers and polite po/opo in context.", + "Connected accounts of activities and plans; common completed, ongoing and contemplated verb aspects with time expressions, rather than a direct copy of English tenses.", + "Reasons and opinions, actor and object voice/focus, consistent participant markers, connected clauses and practical problem-solving.", + "Hypotheticals, reported information, conversational particles such as na, pa, naman and daw/raw, nuanced requests and changes of register.", + "Flexible extended discussion with precise, idiomatic Tagalog, natural aspect and voice choices, appropriate tone and respect for regional variation." + ], + topicPlaceholder: "Food, travel, music, life in the Philippines…", + lookupUnavailableReply: "Hindi ko ito masuri ngayon. Kung gusto mo, puwede nating pag-usapan ang paksa sa pangkalahatan.", + themeOverrides: [ + "coffee": .init("coffee", "Kape tayo?", "Something warm, please", "cup.and.saucer", "Everyday", "Meet at a neighbourhood café in the Philippines. Order a drink and chat about the learner's day. Use natural Tagalog and polite requests appropriate to the situation.", 0), + "groceries": .init("groceries", "Sa palengke", "A little of everything", "basket", "Everyday", "Shop at a local market in the Philippines. Practise quantities, prices and friendly requests. Respect regional food names and ask about preferences.", 2), + "travel": .init("travel", "Saan tayo pupunta?", "Find your way", "tram", "Everyday", "Plan a trip in the Philippines. Choose suitable transport together and practise directions, destinations and tickets without inventing current routes, fares or schedules.", 1), + "cabin": .init("cabin", "Isang maikling bakasyon", "A change of scene", "mountain.2", "Local life", "Plan an imagined weekend in the Philippines. Let the learner choose a city, coast or countryside, then discuss travel, food and activities.", 2), + "traditions": .init("traditions", "Kuwentuhan sa hapag", "Stay a little longer", "fork.knife", "Local life", "Talk over an imagined meal in the Philippines about routines, family and everyday customs. Compare individual experiences without assuming one religion, language or tradition represents everyone.", 2) + ] + ) +} diff --git a/apps/ios/Core/TeachingPolicy.swift b/apps/ios/Core/TeachingPolicy.swift index 7167042b..ee3c2fa7 100644 --- a/apps/ios/Core/TeachingPolicy.swift +++ b/apps/ios/Core/TeachingPolicy.swift @@ -37,7 +37,13 @@ public enum TeachingPolicy { public static func redirect(language: LanguageModule) -> String { "Return to \(language.name). Briefly restate the last idea in \(language.name) and continue ONLY in \(language.name). The learner may reply in any language; your speech must stay in \(language.name)." } + public static func supportsSpeechLanguageDetection(language: LanguageModule) -> Bool { + // Apple's recognizer can label valid Tagalog as Indonesian above 99% confidence. + // Keep explicit target-language prompts, but do not redirect on unreliable labels. + language.id != "tl" + } public static func shouldRedirectSpeech(language: LanguageModule, detectedLanguageID: String, confidence: Double) -> Bool { + guard supportsSpeechLanguageDetection(language: language) else { return false } let detected = detectedLanguageID.replacingOccurrences(of: "_", with: "-").lowercased() // NaturalLanguage reports Chinese script IDs (zh-Hans / zh-Hant). // These describe the transcript's script, not a different spoken language. diff --git a/apps/ios/Tests/AdditionalLanguageTests.swift b/apps/ios/Tests/AdditionalLanguageTests.swift index 348de85f..ee31f5f4 100644 --- a/apps/ios/Tests/AdditionalLanguageTests.swift +++ b/apps/ios/Tests/AdditionalLanguageTests.swift @@ -2,12 +2,13 @@ import XCTest @testable import MuralCore final class AdditionalLanguageTests: XCTestCase { - private let ids = ["de", "it", "pt", "zh"] + private let ids = ["de", "it", "pt", "zh", "tl"] private let samples = [ ("de", "Ich gehe über die Straße.", "die Straße", "Straße", "street"), ("it", "Vorrei un caffè.", "un caffè", "caffè", "coffee"), ("pt", "Eu gosto de pão e maçã.", "o pão", "pão", "bread"), - ("zh", "我想去银行。", "银行", "银行", "bank") + ("zh", "我想去银行。", "银行", "银行", "bank"), + ("tl", "Mag-aaral ako araw-araw.", "mag-aral", "Mag-aaral", "study") ] private func session(_ id: String, text: String = "radio", lemma: String = "radio", form: String = "radio", meaning: String = "radio", day: Int = 0, supported: Bool = false, typed: Bool = false) -> SessionRecord { @@ -24,8 +25,8 @@ final class AdditionalLanguageTests: XCTestCase { } func testRegistrationPreservesOldIDsAndSetsRequestedVarieties() { - XCTAssertEqual(LanguageRegistry.all.map(\.id), ["nb", "es", "en", "fr", "de", "it", "pt", "zh"]) - for (id, locale, greeting) in [("de", "de-DE", "Hallo!"), ("it", "it-IT", "Ciao!"), ("pt", "pt-BR", "Olá!"), ("zh", "zh-CN", "你好！")] { + XCTAssertEqual(LanguageRegistry.all.map(\.id), ["nb", "es", "en", "fr", "de", "it", "pt", "zh", "tl"]) + for (id, locale, greeting) in [("de", "de-DE", "Hallo!"), ("it", "it-IT", "Ciao!"), ("pt", "pt-BR", "Olá!"), ("zh", "zh-CN", "你好！"), ("tl", "tl-PH", "Kumusta!")] { XCTAssertEqual(LanguageRegistry.module(for: id)?.locale, locale) XCTAssertEqual(LanguageRegistry.module(for: id)?.greeting, greeting) } @@ -57,7 +58,7 @@ final class AdditionalLanguageTests: XCTestCase { } } - func testAllEightLanguagesRoundTripWithIsolatedProgressAndHiddenWords() throws { + func testAllRegisteredLanguagesRoundTripWithIsolatedProgressAndHiddenWords() throws { var archive = Archive() archive.sessions = LanguageRegistry.all.flatMap { [session($0.id), session($0.id, day: 2)] } archive.preferences.meaningLanguage = "Chinese (Simplified)" @@ -79,7 +80,7 @@ final class AdditionalLanguageTests: XCTestCase { let hidden = LearningEngine.project(restored.sessions, languageID: language.id, hiddenWords: restored.preferences.hiddenWords) XCTAssertEqual(hidden.words.count, language.id == "pt" ? 0 : 1) } - XCTAssertEqual(keys.count, 8) + XCTAssertEqual(keys.count, LanguageRegistry.all.count) } } diff --git a/apps/ios/Tests/FinalAssessmentTests.swift b/apps/ios/Tests/FinalAssessmentTests.swift index a36cf701..27c7aedf 100644 --- a/apps/ios/Tests/FinalAssessmentTests.swift +++ b/apps/ios/Tests/FinalAssessmentTests.swift @@ -52,6 +52,37 @@ import XCTest XCTAssertTrue(visible!.assessments.isEmpty) } + func testTagalogFinalAssessmentStaysWithItsOriginalSessionAfterSwitching() async { + for (oldID, newID) in [("tl", "es"), ("pt", "tl")] { + let old = ended(oldID), new = ended(newID), provider = Provider() + var records = [old.id: old, new.id: new] + let queue = FinalAssessmentQueue(assess: provider.assess) + queue.onResult = { result in + if let updated = result.applying(to: records[result.sessionID]) { records[updated.id] = updated } + } + XCTAssertTrue(queue.submit(old)) + await waitUntil { provider.pending.count == 1 } + provider.finish() + await waitUntil { !queue.isPending(old.id) } + XCTAssertEqual(records[old.id]?.assessments.first?.words.first?.language, oldID) + XCTAssertEqual(records[old.id]?.inputTokens, 100) + XCTAssertTrue(records[new.id]!.assessments.isEmpty) + XCTAssertTrue(LearningEngine.project([records[new.id]!], languageID: newID).words.isEmpty) + } + } + + func testTagalogResultCannotApplyToARecordWithAnotherLanguage() { + let original = ended("tl") + let passage = original.passages[0] + let assessment = Assessment(passageID: passage.id, revisionKey: passage.revisionKey, outcome: .success, + suggestedLevel: 1, nextGoal: "Magtanong pa.", capability: "Names an object", words: []) + let result = FinalAssessmentResult(sessionID: original.id, languageID: "fil", assessment: assessment) + XCTAssertNil(result.applying(to: original)) + let correct = FinalAssessmentResult(sessionID: original.id, languageID: "tl", assessment: assessment) + XCTAssertNotNil(correct.applying(to: original)) + XCTAssertNil(correct.applying(to: ended("es"))) + } + func testDeletedSessionIsNeverRecreatedByALateResult() async { let session = ended(), provider = Provider() var records = [session.id: session] diff --git a/apps/ios/Tests/LanguageTests.swift b/apps/ios/Tests/LanguageTests.swift index 6a733c32..9105fd6d 100644 --- a/apps/ios/Tests/LanguageTests.swift +++ b/apps/ios/Tests/LanguageTests.swift @@ -215,7 +215,8 @@ final class LanguageTests: XCTestCase { for language in LanguageRegistry.all { XCTAssertFalse(TeachingPolicy.shouldRedirectSpeech(language: language, detectedLanguageID: language.id, confidence: 0.99)) let otherID = language.id == "en" ? "fr" : "en" - XCTAssertTrue(TeachingPolicy.shouldRedirectSpeech(language: language, detectedLanguageID: otherID, confidence: 0.99)) + XCTAssertEqual(TeachingPolicy.shouldRedirectSpeech(language: language, detectedLanguageID: otherID, confidence: 0.99), + TeachingPolicy.supportsSpeechLanguageDetection(language: language)) for confidence in [0.0, 0.88, .nan, .infinity, 1.1] { XCTAssertFalse(TeachingPolicy.shouldRedirectSpeech(language: language, detectedLanguageID: otherID, confidence: confidence)) } diff --git a/apps/ios/Tests/MeaningTests.swift b/apps/ios/Tests/MeaningTests.swift index 7c0c9142..a18c1e9a 100644 --- a/apps/ios/Tests/MeaningTests.swift +++ b/apps/ios/Tests/MeaningTests.swift @@ -126,6 +126,37 @@ import XCTest XCTAssertEqual(controller.text, "Hi!") } + func testSwitchingBetweenTagalogAndSpanishDiscardsLateMeaningsAndErrors() async { + for (oldID, newID) in [("tl", "es"), ("es", "tl")] { + for fails in [false, true] { + let translator = Translator() + let controller = MeaningController(delay: .zero, translate: translator.translate) + let passage = Passage(id: "same-passage", speaker: .assistant, fragments: [ + Fragment(speaker: .assistant, text: "Kumusta!", startMS: 0, endMS: 1000) + ]) + func request(_ id: String) -> MeaningRequest { + MeaningRequest(sessionID: sessionID, passage: passage, learningLanguageID: id, meaningLanguage: "English") + } + var saved: [String] = [] + controller.onResult = { request, _ in saved.append(request.learningLanguageID) } + controller.update(request(oldID)) + await waitUntil { translator.pending.count == 1 } + controller.update(request(newID)) + await waitUntil { translator.pending.count == 2 } + if fails { translator.fail() } else { translator.succeed("Old meaning") } + try? await Task.sleep(for: .milliseconds(10)) + XCTAssertEqual(controller.text, "") + XCTAssertNil(controller.error) + XCTAssertTrue(controller.isLoading) + XCTAssertTrue(saved.isEmpty) + translator.succeed("New meaning") + await waitUntil { !controller.isLoading } + XCTAssertEqual(controller.text, "New meaning") + XCTAssertEqual(saved, [newID]) + } + } + } + func testChangingMeaningLanguageClearsOldTextAndUsesSeparateCacheKeys() async { let translator = Translator() let controller = MeaningController(delay: .zero, translate: translator.translate) diff --git a/apps/ios/Tests/TagalogTests.swift b/apps/ios/Tests/TagalogTests.swift new file mode 100644 index 00000000..3707b8ac --- /dev/null +++ b/apps/ios/Tests/TagalogTests.swift @@ -0,0 +1,261 @@ +import XCTest +@testable import MuralCore + +final class TagalogTests: XCTestCase { + private func language() throws -> LanguageModule { try XCTUnwrap(LanguageRegistry.module(for: "tl")) } + + private func record(_ text: String = "Gusto ko ng kape.", lemma: String = "kape", form: String = "kape", + meaning: String = "coffee", languageID: String = "tl", day: Int = 0, + typed: Bool = false, supported: Bool = false) -> SessionRecord { + let date = Date(timeIntervalSince1970: 1_780_000_000 + Double(day) * 86400) + var session = SessionRecord(languageID: languageID, themeID: "coffee") + session.startedAt = date + session.append(Fragment(speaker: .user, text: text, startMS: 100_000, endMS: 103_000, + receivedAt: date, meaningVisible: supported, typed: typed)) + let passage = session.passages[0] + session.assessments = [Assessment(passageID: passage.id, revisionKey: passage.revisionKey, outcome: .success, + suggestedLevel: 2, nextGoal: "Magtanong tungkol sa presyo.", capability: "Orders a drink", + words: [WordProposal(lemma: lemma, meaning: meaning, form: form, kind: .independent, confidence: 0.95, + sourceIDs: passage.fragments.map(\.id), quote: text, language: languageID)], createdAt: date)] + session.endedAt = date.addingTimeInterval(104) + return session + } + + func testIdentityUsesOneTagalogNamespaceWithFilipinoDisplayAlias() throws { + let module = try language() + XCTAssertEqual(module.name, "Tagalog (Filipino)") + XCTAssertEqual(module.nativeName, "Tagalog (Filipino)") + XCTAssertEqual(module.settingsTitle, "Tagalog (Filipino) · Philippines") + XCTAssertEqual(module.locale, "tl-PH") + XCTAssertEqual(module.greeting, "Kumusta!") + XCTAssertEqual(module.greetingWord, "kumusta") + XCTAssertEqual(LanguageRegistry.defaultID, "nb") + XCTAssertEqual(LanguageRegistry.all.filter { $0.id == "tl" }.count, 1) + for alias in ["fil", "fil-PH", "tgl", "tl-PH"] { XCTAssertNil(LanguageRegistry.module(for: alias)) } + } + + func testEveryPromptPathUsesTagalogWithoutAnotherTeachingTarget() throws { + let module = try language() + let theme = try XCTUnwrap(module.themes.first { $0.id == "coffee" }) + let voice = TeachingPolicy.voice(language: module, learner: LearningEngine.project([], languageID: "tl"), + theme: theme, interests: "music", meaningLanguage: "French") + let assessment = TeachingPolicy.assessment(language: module) + let prompts = [voice, assessment, TeachingPolicy.greeting(language: module), TeachingPolicy.help(language: module), + TeachingPolicy.redirect(language: module), TeachingPolicy.theme(theme, language: module), + TeachingPolicy.translation(language: module, meaningLanguage: "French"), TeachingPolicy.delegation(language: module), + TeachingPolicy.typedReply(language: module), TeachingPolicy.lookup(language: module, meaningLanguage: "French"), + TeachingPolicy.currentTopic(language: module)] + for prompt in prompts { + XCTAssertTrue(prompt.contains(module.name)) + for contaminant in ["Norwegian", "Bokmål", "Brazilian", "Spain", "Mandarin"] { + XCTAssertFalse(prompt.contains(contaminant), contaminant) + } + } + XCTAssertTrue(voice.contains(module.speechGuidance)) + XCTAssertTrue(voice.contains(module.writingGuidance)) + XCTAssertTrue(assessment.contains(module.lemmaGuidance)) + XCTAssertTrue(assessment.contains("Use language tl for target-language evidence")) + XCTAssertTrue(TeachingPolicy.translation(language: module, meaningLanguage: "French").contains("into French")) + } + + func testAllSixChallengesReceiveTheirTagalogFocusIncludingClampedBounds() throws { + let module = try language() + XCTAssertEqual(module.teachingFocus.count, 6) + XCTAssertEqual(Set(module.teachingFocus).count, 6) + for level in -1...6 { + var learner = LearningEngine.project([], languageID: "tl") + learner.challenge = level + let prompt = TeachingPolicy.voice(language: module, learner: learner, theme: nil, interests: "", meaningLanguage: "English") + XCTAssertTrue(prompt.contains(module.teachingFocus[min(5, max(0, level))])) + } + } + + func testPhilippineThemesOverrideStableIDsAndInheritTheRest() throws { + let module = try language() + XCTAssertEqual(Set(module.themeOverrides.keys), ["coffee", "groceries", "travel", "cabin", "traditions"]) + XCTAssertEqual(module.themes.map(\.id), ConversationTheme.shared.map(\.id)) + for theme in module.themes { + if let override = module.themeOverrides[theme.id] { + XCTAssertEqual(theme, override) + XCTAssertTrue(theme.situation.contains("Philippines")) + } else { XCTAssertEqual(theme, ConversationTheme.shared.first { $0.id == theme.id }) } + } + XCTAssertFalse(module.lookupUnavailableReply.isEmpty) + XCTAssertTrue(module.topicPlaceholder.contains("Philippines")) + } + + func testUnreliableDetectorCannotRedirectValidTagalogAtHighConfidence() throws { + let module = try language() + XCTAssertFalse(TeachingPolicy.supportsSpeechLanguageDetection(language: module)) + for id in ["id", "hr", "en", "tl", "tl-PH", "tl_PH", "fil", "fil-PH", "und", ""] { + for confidence in [0.0, 0.88, 0.99247, 0.99974, 1, .nan, .infinity] { + XCTAssertFalse(TeachingPolicy.shouldRedirectSpeech(language: module, detectedLanguageID: id, confidence: confidence)) + } + } + } + + func testDetectorPolicyStillProtectsEveryPreviouslySupportedTarget() { + for module in LanguageRegistry.all where module.id != "tl" { + XCTAssertTrue(TeachingPolicy.supportsSpeechLanguageDetection(language: module)) + let wrong = module.id == "en" ? "es" : "en" + XCTAssertTrue(TeachingPolicy.shouldRedirectSpeech(language: module, detectedLanguageID: wrong, confidence: 0.99), module.id) + for detected in [module.id, module.locale, module.locale.replacingOccurrences(of: "-", with: "_")] { + XCTAssertFalse(TeachingPolicy.shouldRedirectSpeech(language: module, detectedLanguageID: detected, confidence: 0.99)) + } + for confidence in [0, 0.88, 1.01, .nan, .infinity] { + XCTAssertFalse(TeachingPolicy.shouldRedirectSpeech(language: module, detectedLanguageID: wrong, confidence: confidence)) + } + } + } + + func testUnaidedTagalogRecallGrowsAcrossDaysAndStaysSeparate() throws { + _ = try language() + let first = record(), second = record(day: 2) + let sessions = [first, second, record("Eu gosto de café.", lemma: "o café", form: "café", languageID: "pt")] + let state = LearningEngine.project(sessions, languageID: "tl", now: second.startedAt) + XCTAssertEqual(state.observationCount, 2) + XCTAssertEqual(state.challenge, 1) + XCTAssertEqual(state.words.first?.id, "tl|kape|coffee") + XCTAssertEqual(state.words.first?.independentCount, 2) + XCTAssertEqual(state.words.first?.bars, 2) + XCTAssertTrue(LearningEngine.project(sessions, languageID: "nb").words.isEmpty) + XCTAssertEqual(LearningEngine.project(sessions, languageID: "pt").words.first?.id, "pt|o café|coffee") + } + + func testTypedVisibleMeaningAndImmediateImitationAreAssisted() throws { + _ = try language() + for (typed, supported) in [(true, false), (false, true), (true, true)] { + let session = record(typed: typed, supported: supported) + XCTAssertEqual(LearningEngine.validate(session.assessments[0], session: session)?.words.first?.kind, .assisted) + XCTAssertEqual(LearningEngine.project([session], languageID: "tl").words.first?.independentCount, 0) + } + var imitated = record() + imitated.append(Fragment(speaker: .assistant, text: "Gusto mo ba ng kape?", startMS: 90_000, endMS: 93_000)) + XCTAssertEqual(LearningEngine.validate(imitated.assessments[0], session: imitated)?.words.first?.kind, .assisted) + } + + func testForeignMixedAndAliasEvidenceCannotBecomeTagalogVocabulary() throws { + _ = try language() + for id in ["en", "es", "pt", "nb", "fil", "fil-PH", "tgl", "id", "mixed", "uncertain"] { + for kind in [EvidenceKind.independent, .assisted, .understanding, .exposure, .lapse] { + var session = record() + session.assessments[0].words[0].language = id + session.assessments[0].words[0].kind = kind + XCTAssertTrue(try XCTUnwrap(LearningEngine.validate(session.assessments[0], session: session)).words.isEmpty, "\(id) / \(kind)") + } + } + } + + func testMixedPassageRetainsOnlyItsExplicitTagalogProposal() throws { + _ = try language() + var session = record("I would like kape, please.") + var english = session.assessments[0].words[0] + english.lemma = "please"; english.form = "please"; english.meaning = "please"; english.language = "en" + session.assessments[0].words.append(english) + let words = try XCTUnwrap(LearningEngine.validate(session.assessments[0], session: session)).words + XCTAssertEqual(words.map(\.key), ["tl|kape|coffee"]) + } + + func testFabricatedOrStaleEvidenceIsRejected() throws { + _ = try language() + for mutation in 0..<4 { + var session = record() + switch mutation { + case 0: session.assessments[0].words[0].sourceIDs = ["invented"] + case 1: session.assessments[0].words[0].quote = "Wala akong kape." + case 2: session.assessments[0].words[0].form = "tsaa" + default: session.assessments[0].words[0].confidence = .nan + } + XCTAssertTrue(try XCTUnwrap(LearningEngine.validate(session.assessments[0], session: session)).words.isEmpty) + } + var revised = record() + revised.fragments[0].revision += 1 + XCTAssertNil(LearningEngine.validate(revised.assessments[0], session: revised)) + } + + func testAspectVariantsShareCitationFormButDifferentFocusStaysDistinct() throws { + _ = try language() + let sessions = [record("Kumakain ako ngayon.", lemma: "kumain", form: "Kumakain", meaning: "eat"), + record("Kakain ako mamaya.", lemma: "kumain", form: "Kakain", meaning: "eat", day: 2), + record("Kinain ko ang saging.", lemma: "kainin", form: "Kinain", meaning: "eat", day: 2)] + let words = LearningEngine.project(sessions, languageID: "tl", now: sessions[1].startedAt).words + XCTAssertEqual(Set(words.map(\.id)), ["tl|kumain|eat", "tl|kainin|eat"]) + XCTAssertEqual(words.first { $0.lemma == "kumain" }?.independentCount, 2) + XCTAssertEqual(words.first { $0.lemma == "kainin" }?.independentCount, 1) + } + + func testHomographsAndHiddenWordsKeepTheirSensesAndLanguage() throws { + _ = try language() + let sessions = [record("Bukás ang pinto.", lemma: "bukas", form: "Bukás", meaning: "open"), + record("Bukas ako aalis.", lemma: "bukas", form: "Bukas", meaning: "tomorrow"), + record("radio", lemma: "radio", form: "radio", meaning: "radio"), + record("radio", lemma: "radio", form: "radio", meaning: "radio", languageID: "es")] + let hidden = ["tl|bukas|open", "tl|radio|radio"] + XCTAssertEqual(Set(LearningEngine.project(sessions, languageID: "tl", hiddenWords: hidden).words.map(\.id)), ["tl|bukas|tomorrow"]) + XCTAssertEqual(LearningEngine.project(sessions, languageID: "es", hiddenWords: hidden).words.count, 1) + } + + func testWordLinksPreserveHyphensApostrophesDiacriticsAndWhitespace() { + for (text, expected) in [ + (" Mag-aaral ako araw-araw.\n", ["Mag-aaral", "ako", "araw-araw"]), + ("Ako'y masaya. Siya’y narito!", ["Ako'y", "masaya", "Siya’y", "narito"]), + ("Bukás ang pinto; búkas tayo aalis. ☕️", ["Bukás", "ang", "pinto", "búkas", "tayo", "aalis"]) + ] { + let segments = CaptionWords.segments(text, languageID: "tl") + XCTAssertEqual(segments.map(\.text).joined(), text) + XCTAssertEqual(segments.compactMap(\.lookup), expected) + } + } + + func testArchiveRetainsExactTextSelectionTopicsAndHiddenWords() throws { + _ = try language() + let text = "Ako'y mag-aaral araw-araw.\nBukás ang pinto." + var archive = Archive() + archive.preferences.learningLanguageID = "tl" + archive.preferences.meaningLanguage = "French" + archive.preferences.hiddenWords = ["tl|mag-aral|study"] + archive.sessions = [record(text, lemma: "mag-aral", form: "mag-aaral", meaning: "study"), record(languageID: "nb")] + archive.sessions[0].topics = [TopicBrief(languageID: "tl", query: "pagkain", text: "Pag-usapan natin ang pagkain.", sources: [])] + let restored = try Archive.decode(archive.encoded()) + XCTAssertEqual(restored.schemaVersion, 2) + XCTAssertEqual(restored.preferences.learningLanguageID, "tl") + XCTAssertEqual(restored.preferences.meaningLanguage, "French") + XCTAssertEqual(restored.preferences.hiddenWords, archive.preferences.hiddenWords) + XCTAssertEqual(restored.sessions.map(\.id), archive.sessions.map(\.id)) + XCTAssertEqual(restored.sessions[0].passages[0].text, text) + XCTAssertEqual(restored.sessions[0].assessments[0].words[0].quote, text) + XCTAssertEqual(restored.sessions[0].topics[0].languageID, "tl") + } + + func testImportMergesTagalogWithoutReplacingExistingLanguageOrHistory() throws { + _ = try language() + var existing = Archive(), incoming = Archive() + existing.preferences.learningLanguageID = "es" + existing.sessions = [record(languageID: "es")] + incoming.preferences.learningLanguageID = "tl" + incoming.sessions = [record()] + let merged = try existing.merging(Archive.decode(incoming.encoded())) + XCTAssertEqual(merged.preferences.learningLanguageID, "es") + XCTAssertEqual(merged.sessions.map(\.id), existing.sessions.map(\.id) + incoming.sessions.map(\.id)) + XCTAssertEqual(LearningEngine.project(merged.sessions, languageID: "tl").words.first?.id, "tl|kape|coffee") + } + + func testUnknownIDsAndMixedLanguageTopicsCannotReplaceAnArchive() throws { + _ = try language() + var original = Archive() + original.sessions = [record()] + let before = try original.encoded() + for id in ["fil", "tl-PH", "unknown"] { + var incoming = original + incoming.preferences.learningLanguageID = id + XCTAssertThrowsError(try Archive.decode(incoming.encoded())) + incoming.preferences.learningLanguageID = "tl" + incoming.sessions[0] = record(languageID: id) + XCTAssertThrowsError(try original.merging(incoming)) + } + var incoming = original + incoming.sessions[0].topics = [TopicBrief(languageID: "es", query: "food", text: "Comida", sources: [])] + XCTAssertThrowsError(try original.merging(incoming)) + XCTAssertEqual(try original.encoded(), before) + } +} diff --git a/apps/ios/UITests/MuralUITests.swift b/apps/ios/UITests/MuralUITests.swift index 01a1434c..0ac3e16b 100644 --- a/apps/ios/UITests/MuralUITests.swift +++ b/apps/ios/UITests/MuralUITests.swift @@ -39,6 +39,8 @@ final class MuralUITests: XCTestCase { } } + func testTagalogOnboarding() { checkNewOnboarding(id: "tl", greeting: "Kumusta!") } + func testGermanOnboarding() { checkNewOnboarding(id: "de", greeting: "Hallo!") } func testItalianOnboarding() { checkNewOnboarding(id: "it", greeting: "Ciao!") } func testBrazilianPortugueseOnboarding() { checkNewOnboarding(id: "pt", greeting: "Olá!") } @@ -73,7 +75,8 @@ final class MuralUITests: XCTestCase { ("German · Germany", "German", "Hallo!", "Ein Kaffee?"), ("Italian · Italy", "Italian", "Ciao!", "Un caffè?"), ("Portuguese · Brazil", "Portuguese", "Olá!", "Um cafezinho?"), - ("Mandarin Chinese · Mainland China", "Mandarin Chinese", "你好！", "喝杯咖啡？") + ("Mandarin Chinese · Mainland China", "Mandarin Chinese", "你好！", "喝杯咖啡？"), + ("Tagalog (Filipino) · Philippines", "Tagalog (Filipino)", "Kumusta!", "Kape tayo?") ] { app.buttons["Settings"].tap() app.buttons["learning-language-picker"].tap() @@ -128,6 +131,125 @@ final class MuralUITests: XCTestCase { XCTAssertTrue(app.buttons.matching(NSPredicate(format: "label CONTAINS %@", "喝杯咖啡？")).firstMatch.exists) } + func testTagalogOnboardingAtLargestAccessibilitySizePreservesSubtitleChoice() { + let app = XCUIApplication() + app.launchArguments = ["--preview", "--preview-onboarding", "-UIPreferredContentSizeCategoryName", "UICTContentSizeCategoryAccessibilityXXXL"] + app.launch() + let choice = app.buttons["onboarding-language-tl"] + XCTAssertTrue(choice.waitForExistence(timeout: 10)) + reveal(choice, in: app) + choice.tap() + XCTAssertTrue(choice.isSelected) + XCTAssertEqual(choice.label, "Tagalog (Filipino) · Philippines") + let viewport = app.scrollViews.firstMatch.frame + let continueButton = app.buttons["onboarding-continue"] + XCTAssertGreaterThan(choice.frame.height, 0) + XCTAssertTrue(viewport.contains(choice.frame), "The entire Tagalog card must fit inside the visible scroll area") + XCTAssertTrue(app.frame.contains(continueButton.frame), "Continue must fit inside the screen") + XCTAssertLessThan(choice.frame.maxY, continueButton.frame.minY) + XCTAssertTrue(continueButton.isHittable) + let screen = XCTAttachment(screenshot: app.screenshot()) + screen.name = "Tagalog onboarding - largest accessibility text"; screen.lifetime = .keepAlways; add(screen) + app.buttons["onboarding-continue"].tap() + app.buttons["onboarding-meaning-picker"].tap() + app.buttons["French"].tap() + app.buttons["onboarding-back"].tap() + reveal(choice, in: app) + XCTAssertTrue(choice.isSelected) + app.buttons["onboarding-continue"].tap() + XCTAssertEqual(app.staticTexts["onboarding-meaning-example"].label, "Salut !") + app.buttons["onboarding-continue"].tap() + XCTAssertTrue(app.staticTexts["target-caption"].waitForExistence(timeout: 5)) + XCTAssertEqual(app.staticTexts["target-caption"].label, "Kumusta!") + XCTAssertEqual(app.staticTexts["meaning-caption"].label, "Salut !") + XCTAssertFalse(app.buttons["pinyin-toggle"].exists) + } + + func testTagalogTranscriptAndMeaningSurviveResetAndLanguageSwitch() { + let app = XCUIApplication() + app.launchArguments = ["--preview", "--ended-conversation", "--preview-language=tl"] + app.launch() + XCTAssertTrue(app.buttons["new-conversation"].waitForExistence(timeout: 10)) + XCTAssertEqual(app.staticTexts["target-caption"].label, "Gusto ko ng kape.") + XCTAssertEqual(app.staticTexts["meaning-caption"].label, "I like coffee.") + XCTAssertFalse(app.buttons["pinyin-toggle"].exists) + app.buttons["Conversation transcript"].tap() + XCTAssertTrue(app.staticTexts["Gusto ko ng kape."].exists) + let screen = XCTAttachment(screenshot: app.screenshot()) + screen.name = "Tagalog transcript and English meaning"; screen.lifetime = .keepAlways; add(screen) + app.buttons["Done"].tap() + if app.buttons["new-conversation"].exists { app.buttons["new-conversation"].tap() } + XCTAssertEqual(app.staticTexts["target-caption"].label, "Kumusta!") + app.buttons["Settings"].tap() + app.buttons["learning-language-picker"].tap() + app.buttons["Spanish · Spain"].tap() + app.buttons["Done"].tap() + app.tabBars.buttons["Words"].tap() + app.buttons["Past conversations"].tap() + XCTAssertFalse(app.buttons.matching(NSPredicate(format: "label CONTAINS %@", "Kape tayo?")).firstMatch.exists) + app.navigationBars.buttons.element(boundBy: 0).tap() + app.tabBars.buttons["Talk"].tap() + app.buttons["Settings"].tap() + app.buttons["learning-language-picker"].tap() + app.buttons["Tagalog (Filipino) · Philippines"].tap() + app.buttons["Done"].tap() + app.tabBars.buttons["Words"].tap() + app.buttons["Past conversations"].tap() + let saved = app.buttons.matching(NSPredicate(format: "label CONTAINS %@", "Kape tayo?")).firstMatch + XCTAssertTrue(saved.exists) + saved.tap() + XCTAssertTrue(app.staticTexts["Gusto ko ng kape."].exists) + } + + func testTagalogLanguageSwitchIsDisabledDuringConversation() { + let app = XCUIApplication() + app.launchArguments = ["--preview", "--active-conversation", "--preview-language=tl"] + app.launch() + XCTAssertTrue(app.staticTexts["target-caption"].waitForExistence(timeout: 10)) + XCTAssertEqual(app.staticTexts["target-caption"].label, "Gusto ko ng kape.") + app.buttons["Settings"].tap() + XCTAssertFalse(app.buttons["learning-language-picker"].isEnabled) + XCTAssertTrue(app.staticTexts["End this conversation to switch languages. Each language keeps its own words and progress."].exists) + } + + func testTagalogSelectionPersistsAcrossNormalRelaunch() { + let app = XCUIApplication() + addTeardownBlock { + app.terminate() + app.launch() + XCTAssertTrue(app.buttons["Settings"].waitForExistence(timeout: 10)) + app.buttons["Settings"].tap() + app.buttons["learning-language-picker"].tap() + app.buttons["Norwegian · Bokmål"].tap() + app.buttons["Done"].tap() + app.terminate() + app.launch() + XCTAssertTrue(app.staticTexts["target-caption"].waitForExistence(timeout: 10)) + XCTAssertEqual(app.staticTexts["target-caption"].label, "Hei!") + app.terminate() + } + app.launch() + if app.buttons["onboarding-continue"].waitForExistence(timeout: 5) { + let choice = app.buttons["onboarding-language-tl"] + reveal(choice, in: app) + choice.tap() + app.buttons["onboarding-continue"].tap() + app.buttons["onboarding-continue"].tap() + } + XCTAssertTrue(app.buttons["Settings"].waitForExistence(timeout: 10)) + app.buttons["Settings"].tap() + app.buttons["learning-language-picker"].tap() + app.buttons["Tagalog (Filipino) · Philippines"].tap() + app.buttons["Done"].tap() + app.terminate() + app.launch() + XCTAssertTrue(app.staticTexts["target-caption"].waitForExistence(timeout: 10)) + XCTAssertEqual(app.staticTexts["target-caption"].label, "Kumusta!") + XCTAssertFalse(app.buttons["onboarding-continue"].exists) + app.buttons["Settings"].tap() + XCTAssertTrue(app.buttons["learning-language-picker"].label.contains("Tagalog (Filipino)")) + } + private func launch(ended: Bool = false) -> XCUIApplication { let app = XCUIApplication(); app.launchArguments = ["--preview"] + (ended ? ["--ended-conversation"] : []) app.launch(); return app diff --git a/docs/add-language.md b/docs/add-language.md index ded0c989..1c4b4dc4 100644 --- a/docs/add-language.md +++ b/docs/add-language.md @@ -4,9 +4,11 @@ Use this guide with the existing Swift package and Xcode project. [The architect 1. Add a file in `apps/ios/Core/Languages/`, following `Spanish.swift` or `Norwegian.swift`. Define a `LanguageModule` with a unique, stable language ID, display names, locale, greeting, speech and writing guidance, lemma rules, six teaching stages and a topic placeholder. Include `lookupUnavailableReply`, the spoken fallback when a web lookup is unavailable, in the target language. Include valid regional alternatives in the correction guidance. 2. Override culture-specific themes by their existing IDs. Keep shared IDs stable. New common themes belong in `apps/ios/Core/Themes.swift`. -3. Register the module in `LanguageRegistry.all`. The learning-language picker, shared teaching policy, evidence validation and progress projection use the registry automatically. Keep published IDs stable and retain modules needed to open existing learning records. Run `python3 scripts/export_android_content.py` so the Android client picks up the new module. +3. Register the module in `LanguageRegistry.all`. The learning-language picker, shared teaching policy, evidence validation and progress projection use the registry automatically. Keep published IDs stable and retain modules needed to open existing learning records. Run `python3 scripts/export_android_content.py` to regenerate the Android catalog and extend its registry tests. Run the Android and cross-platform checks in [the build guide](build-and-test.md). 4. Add language fixtures in `apps/ios/Tests/LanguageTests.swift`. Check supported versus independent recall, evidence from another language, archive round trips, vocabulary isolation and prompt contamination. Add a UI check for selecting the language and returning to an existing one. -5. Run `swift test --package-path Mural` from the workspace root, then build and run the native UI tests described in the [README](../README.md). Core Swift files are discovered by Swift Package Manager. If you add app files, also run `python3 Mural/scripts/generate_project.py`. -6. Validate a short live conversation on an iPhone with its saved API key: greeting, input in a support language, target-language output, corrections, subtitles, a cultural theme and a sourced topic. Confirm progress remains separate after switching and relaunching. Review pronunciation and correction quality with a competent speaker. +5. Check the actual `NLLanguageRecognizer` output on representative target-language text before relying on automatic redirects. If it is unreliable, disable detector-driven redirects for that module in `TeachingPolicy.supportsSpeechLanguageDetection` and report target-language review as pending instead of forcing a detector success. Extend `apps/ios/App/LanguageVerification.swift` with a target-language advanced reply and lookup word, and `ConversationCoordinator.prepareConversationPreview` with matching preview text. +6. Add the locale and regional prompt to `services/api/src/live-provider.ts` and extend `services/api/tests/languages.test.ts`; the experimental hosted catalog is independent of the Swift registry. Update the locale list in `services/api/README.md` and run the server checks with a dedicated test database for integration coverage. +7. Run `swift test --package-path apps/ios` from the repository root, then build and run the native UI tests described in [the build guide](build-and-test.md). Core Swift files are discovered by Swift Package Manager. If you add app files, also run `python3 scripts/generate_project.py`. +8. Validate a short live conversation on an iPhone with its saved API key: greeting, input in a support language, target-language output, corrections, subtitles, a cultural theme and a sourced topic. Confirm progress remains separate after switching and relaunching. Review pronunciation and correction quality with a competent speaker. For an explicitly authorized audio check in a Debug build, launch with `--verify-audio --verify-language=`. It makes two real API connections, uses temporary learning data, and writes non-content diagnostics to `Documents/audio-verification.json` in the app container. It is excluded from Release builds and ordinary tests. Network and provider failures still need investigation; a successful connection alone does not establish teaching quality. diff --git a/docs/build-and-test.md b/docs/build-and-test.md index f171705a..2aef38ad 100644 --- a/docs/build-and-test.md +++ b/docs/build-and-test.md @@ -12,7 +12,7 @@ xcodebuild -project apps/ios/Mural.xcodeproj -scheme Mural \ CODE_SIGNING_ALLOWED=NO ARCHS=arm64 ONLY_ACTIVE_ARCH=YES build ``` -Create an iPhone 17 simulator in Xcode’s **Devices and Simulators** window. If you name it `iPhone 17`, run UI tests with: +Create a dedicated iPhone 17 simulator in Xcode’s **Devices and Simulators** window. Most tests use temporary preview storage; the normal-relaunch test also writes language preferences to the simulator's persistent store. If you name it `iPhone 17`, run UI tests with: ```sh xcodebuild -project apps/ios/Mural.xcodeproj -scheme Mural \ @@ -22,7 +22,7 @@ xcodebuild -project apps/ios/Mural.xcodeproj -scheme Mural \ -parallel-testing-enabled NO test ``` -The core suite covers evidence validation, transcript revisions, language isolation, recall spacing, archive validation, translation cancellation, and managed-account configuration and security parsing. Native UI tests exercise the screens with in-memory data. Neither suite needs an API key. Configured provider sign-in and account deletion need the separate device checks in [managed accounts](managed-accounts.md). +The core suite covers evidence validation, transcript revisions, language isolation, recall spacing, archive validation, translation cancellation, and managed-account configuration and security parsing. Native UI tests exercise the screens with synthetic data and verify language selection across relaunch. Neither suite needs an API key. Configured provider sign-in and account deletion need the separate device checks in [managed accounts](managed-accounts.md). ## Check the Android port and the cross-platform contracts @@ -42,7 +42,7 @@ The last two catch generated language content and a Swift core change without it ## Preview without saving learning data -In **Product → Scheme → Edit Scheme → Run → Arguments**, add `--preview`. The app opens with temporary storage and skips onboarding. In a Debug build, add `--ended-conversation` to exercise the ended-conversation state. Preview fixtures make no API calls. +In **Product → Scheme → Edit Scheme → Run → Arguments**, add `--preview`. The app opens with temporary storage and skips onboarding. In a Debug build, add `--ended-conversation` to exercise the ended-conversation state. Preview fixtures make no API calls. Debug `--preview --active-conversation --preview-language=tl` supplies a temporary active conversation for checking the disabled language picker; it opens no microphone or network session. `--preview --ended-conversation --preview-language=tl` supplies the corresponding ended transcript and cached English meaning. Remove preview arguments before testing normal persistence. For actual speech, [install on an iPhone](run-on-iphone.md) and use the key saved through Settings. @@ -62,7 +62,7 @@ After changing audio, prompts or a language module, check a short conversation o Debug-only `--verify-audio --verify-language=` starts two real voice sessions using the phone’s saved key. `--verify-meaning` adds the translation/reset check. These flags incur API usage, use temporary learning data, and write content-free diagnostics in the app container. Run them only when live testing is intended; they are excluded from Release builds. -For German, Italian, Brazilian Portuguese or Mandarin, `--verify-audio --verify-language-flow --verify-language=` runs one live session with a support-language beginner request and a more complex target-language typed reply. It checks received audio, detected target language, meanings, word lookup, supported evidence, archive decoding and switching away and back. The microphone is muted once connected. The report is `Documents/language-verification-.json`; it contains no transcript, audio or credentials. These synthetic typed turns do not verify recognition of human speech or the quality of corrections and pronunciation. Reopen the app without verification flags to return to its persistent learning record. +For German, Italian, Brazilian Portuguese, Mandarin or Tagalog, `--verify-audio --verify-language-flow --verify-language=` runs one live session with a support-language beginner request and a more complex target-language typed reply. It checks received audio, detected target language, meanings, word lookup, supported evidence, archive decoding and switching away and back. The microphone is muted once connected. The report is `Documents/language-verification-.json`; it contains no transcript, audio or credentials. These synthetic typed turns do not verify recognition of human speech or the quality of corrections and pronunciation. The report exposes mechanical `flowPassed` separately from `passed`, which additionally requires reliable target-language detection. Tagalog detection is unreliable on the tested Apple recognizer, so `passed` remains false even when the mechanical flow succeeds; its actual detected label/confidence is recorded and `languageQualityReview` remains `pending`. Review Tagalog output with a proficient speaker rather than forcing a passing detector result. Reopen the app without verification flags to return to its persistent learning record. Record the build, checks and remaining limitations in `verification/validation.md`. Successful API transport does not establish pronunciation quality or teaching effectiveness. diff --git a/docs/language-architecture.md b/docs/language-architecture.md index f2a5a45c..9ec68db0 100644 --- a/docs/language-architecture.md +++ b/docs/language-architecture.md @@ -14,6 +14,9 @@ Language-specific content lives in `apps/ios/Core/Languages/`. Each module defin | `it` | Italian from Italy | `it-IT` | | `pt` | Brazilian Portuguese | `pt-BR` | | `zh` | Standard Mandarin, Simplified Chinese | `zh-CN` | +| `tl` | Tagalog (Filipino) from the Philippines | `tl-PH` | + +Tagalog uses Filipino as a display alias and keeps one `tl` storage namespace. Its iPhone automatic speech-language detector is disabled because valid Tagalog can be classified as Indonesian with high confidence; explicit target-language teaching instructions remain in use. See [Tagalog teaching choices and references](tagalog.md) for the evidence, vocabulary conventions and verification limits. These locales describe the initial teaching targets. Modules accept valid regional usage from learners. Regional pronunciation is a model instruction and still needs listening checks. Portuguese's stable `pt` storage ID currently belongs to the Brazilian module; a future independently selectable variety must not silently reinterpret existing progress. @@ -27,7 +30,7 @@ Mandarin builds on [richardguerre's contribution in #4](https://github.com/Chulo Pinyin appears separately below selectable Chinese text, with a Show/Hide control. Word links use Chinese word boundaries. Lemmas stay in characters, observed forms and quotations stay unchanged, and generated pinyin never becomes learning evidence. Script identifiers such as `zh-Hans` and `zh-Hant` are accepted by the spoken-language check, so Chinese text does not trigger a false language redirect. Simplified Chinese is also available for meaning subtitles. -These are compiled modules. Adding one ships with an app update; there is no remote module download or extra service. Every new language needs a proficient-speaker teaching and pronunciation review. The Android contribution is not integrated in this checkout, so there is no Android generated catalog to update here. +These are compiled modules. Adding one ships with an app update; there is no remote module download or extra service. Every new language needs a proficient-speaker teaching and pronunciation review. Regenerate the Android catalog when adding or changing a Swift language module. See [how to add a language](add-language.md) for the implementation steps. diff --git a/docs/tagalog.md b/docs/tagalog.md new file mode 100644 index 00000000..f5ddf3f5 --- /dev/null +++ b/docs/tagalog.md @@ -0,0 +1,51 @@ +# Tagalog (Filipino) from the Philippines + +Mural's Tagalog option uses the stable language ID `tl` and locale `tl-PH`. Onboarding and Settings show **Tagalog (Filipino) · Philippines** so learners recognize either name. Filipino is a display alias for this learning option, not a second module or progress record. The [IANA registry](https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry) registers Tagalog (`tl`) and Filipino (`fil`) separately; [ISO 639-1](https://www.loc.gov/standards/iso639-2/php/langcodes_name.php?code_ID=444) assigns `tl` to Tagalog. + +## Teaching choices + +The module teaches everyday Tagalog through conversation, using contemporary Latin-script writing. It accepts support-language replies and established loanwords, bridges back to Tagalog, and uses polite address when the situation calls for it. Other Philippine languages are not treated as incorrect Tagalog dialects. The six challenge stages move from short exchanges through connected stories, aspect and voice/focus, reasons, register and nuanced discussion. They are provisional teaching guidance, not proficiency certification. + +Vocabulary uses dictionary citation forms with meaningful affixes retained. Aspect variants such as `kumakain` and `kakain` can share `kumain`, while the object-focus family retains `kainin`. The module asks for consistent ordinary citation spelling and stable English senses; the learner's observed form and quotation stay intact. Storage tests verify the handling of these proposals, not the accuracy of a model's linguistic judgment. + +Meaning subtitles keep their existing independent choices. Adding this learning target does not add Filipino subtitles or translate the app interface. The five cultural overrides reuse the shared café, market, transport, weekend and customs theme IDs; the other themes remain available. + +## Research references + +These references informed original prompt guidance and test fixtures. No dictionary dataset, lesson text, audio or external runtime dependency is bundled. + +| Reference | How it informs the module | +| --- | --- | +| [Tagalog.com: kumusta](https://www.tagalog.com/dictionary/kumusta) | Greeting use and respectful address. The module uses ordinary `Kumusta!` spelling. | +| [Tagalog.com: kumain](https://www.tagalog.com/dictionary/kumain) | Actor-focus citation form and its aspect family. | +| [Tagalog.com: kainin](https://www.tagalog.com/dictionary/kainin) | Object-focus citation form; preserve its distinction from `kumain`. | +| [University of Hawai‘i: aspect](https://www.hawaii.edu/filipino/Grammar_Topics/Grammar_2-1.html) | Teach completion/progression and neutral forms in context rather than substituting English tense labels mechanically. | +| [University of Hawai‘i: focus](https://www.hawaii.edu/filipino/Grammar_Topics/Grammar_2-2.html) | Verb affixes indicate participant roles; do not strip every form down to a shared root. | + +Tagalog.com was reviewed in a normal browser after automated page retrieval returned HTTP 403. Dictionary conjugation tables and editorial entries informed the implementation; community-submitted sentences were not treated as reviewed teaching material. These sources support the design choices, not a claim that the complete generated conversation has been assessed by a teacher. + +## Speech-language detection + +Apple's `NLLanguageRecognizer` can confidently misclassify Tagalog. On macOS with Xcode 26.6 / Swift 6.3.3 and on the iOS 26.5 simulator, two long Tagalog passages produced Indonesian (`id`) hypotheses at 0.99247 and 0.99974 confidence. Both would have triggered the existing language-redirect guard. The reproduction text and baseline are in [the implementation record](../TAGALOG-PLAN.md). + +`TeachingPolicy.supportsSpeechLanguageDetection` therefore disables detector-driven redirection for `tl`. It does not treat Indonesian as Tagalog, force the recognizer to a desired answer, or change behavior for existing targets. Explicit Tagalog teaching instructions remain active. The tradeoff is that actual language drift also cannot be corrected through this detector; device and speaker review should check for English leakage. + +The Debug language-flow report separates `flowPassed` from `passed`, which additionally requires reliable target-language detection. A Tagalog run can complete the mechanical flow while `passed` remains false and `languageQualityReview` remains `pending`. It records the detector's actual label/confidence rather than claiming Tagalog was recognized. Even for existing languages, automated detection does not certify pronunciation or correction quality. + +## Data and service compatibility + +Tagalog uses archive version 2 with its own vocabulary, topics and progress. Existing IDs and the Norwegian default are unchanged. Older app versions that do not contain `tl` reject Tagalog archives; upgrade them before importing. Retain the module in subsequent versions so existing learning records remain readable. + +The BYOK iPhone flow uses the module's teaching instructions directly. Android's catalog is generated from the same Swift module; its regression tests cover the display alias, English support and isolated archive progress. The Apple detector exception applies only to iPhone; Android uses its own platform classifier and still needs live Tagalog review. The hosted service separately accepts `tl-PH` and rejects bare IDs and unconfigured aliases before reserving credit or conversation minutes. This addition does not enable hosted billing or require a deployment for native BYOK use. + +See [build and test instructions](build-and-test.md) and [verification results](../verification/validation.md). Real-device conversation and proficient-speaker review remain separate from automated test completion. + +## Lessons from other language contributions + +Reviewed the related contributions on 15 September 2026: + +- [Afrikaans, PR #21](https://github.com/Chuloo/mural/pull/21), adds language-specific apostrophe checks and updates Android store metadata. Applying those checks to Tagalog exposed a missing store-description entry, now corrected. Android now also tests the same contraction, hyphen, accent, aspect/focus, homograph and assisted-evidence cases as the existing Swift Tagalog suite. Its proposed Dutch detector alias is not adopted: a known wrong-language label cannot establish that Tagalog was spoken. +- [European Portuguese, PR #27](https://github.com/Chuloo/mural/pull/27), preserves Brazilian progress with a distinct regional ID and matches primary detector subtags. Tagalog already has one stable `tl` namespace with `fil` rejected as an evidence/storage alias. The regional matcher does not address Apple's Tagalog misclassification, so the existing safeguard stays in place. Translating the interface is separate from adding a learning target. +- [Mandarin, PR #4](https://github.com/Chuloo/mural/pull/4), its [four-language integration, PR #15](https://github.com/Chuloo/mural/pull/15), and [Android parity, PR #23](https://github.com/Chuloo/mural/pull/23), establish exact-text word links, independent meanings, shared content and archive tests. Their integrated implementation is already inherited through upstream; the new Android Tagalog regressions exercise those paths with Tagalog text. +- [Multilingual classrooms, PR #26](https://github.com/Chuloo/mural/pull/26), proposes a separate spoken explanation language and guided lessons. That remains a separate product change. This feature supports English meanings, contextual lookup and English/mixed learner replies while the tutor speaks Tagalog. +- [International English startup, PR #28](https://github.com/Chuloo/mural/pull/28), checks hosted admission against actual native registries, exposing the existing `en` versus `en-US` mismatch. Its registry check and English fix were subsequently merged upstream and are now inherited here, alongside Tagalog's `tl-PH` admission and credit/minute settlement tests. The provider test covers nine native locales plus the existing `en-US` compatibility alias. diff --git a/marketing/screenshots/language-support/README.md b/marketing/screenshots/language-support/README.md index 48796dea..5ca6bd5b 100644 --- a/marketing/screenshots/language-support/README.md +++ b/marketing/screenshots/language-support/README.md @@ -1,6 +1,6 @@ # Language support screenshots -Three unedited captures of Mural's native Talk screen on an iPhone 16 Pro simulator running iOS 26.4.1, taken on 13 September 2026 from the app at `7ea5a8168d35894323ce0717ec8b2caee0e26191`. +The original three unedited captures show Mural's native Talk screen on an iPhone 16 Pro simulator running iOS 26.4.1, taken on 13 September 2026 from the app at `7ea5a8168d35894323ce0717ec8b2caee0e26191`. | File | Language | Visible behavior | | --- | --- | --- | @@ -11,3 +11,17 @@ Three unedited captures of Mural's native Talk screen on an iPhone 16 Pro simula The captures use the Debug preview's sample conversation data, with the conversation ended and microphone off. They demonstrate the interface; separate live checks on the owner's iPhone are recorded in [validation.md](../../../verification/validation.md). To reproduce, install a Debug simulator build and launch it with `--preview --ended-conversation --preview-language= -UIPreferredContentSizeCategoryName UICTContentSizeCategoryL`. Capture within 15 seconds, before the ended conversation resets. These images were taken after three seconds with `xcrun simctl io screenshot ` and a 9:41 status bar override. + +## Tagalog + +Three unedited captures from the Tagalog feature on 13 September 2026, using a dedicated iPhone 17 simulator running iOS 26.5: + +| File | Visible behavior | +| --- | --- | +| `tagalog.png` | Philippine café theme, original Tagalog caption and cached English meaning, ended conversation and microphone off | +| `tagalog-onboarding.png` | Selected Tagalog (Filipino) option with Philippines variety | +| `tagalog-accessibility.png` | The full selection label and Continue button at the largest accessibility text size | + +The accessibility capture is scrolled to Tagalog; preceding rows may be partially outside the viewport. The Tagalog regression explicitly checks that the entire selected card fits within the visible scroll area, that Continue fits within the screen, and that the card does not overlap the button. A fresh capture from the 14 September review-fix run was also visually checked for readable, untruncated Tagalog text. + +The onboarding captures are attachments from the passing 25-test suite in `.build/Tagalog-UI.xcresult`. The Talk capture uses `--preview --ended-conversation --preview-language=tl -UIPreferredContentSizeCategoryName UICTContentSizeCategoryL`, captured after two seconds with `simctl io screenshot`. These are synthetic interface fixtures, with no live provider call or generated speech. Physical-device and proficient-speaker review remain pending. diff --git a/marketing/screenshots/language-support/tagalog-accessibility.png b/marketing/screenshots/language-support/tagalog-accessibility.png new file mode 100644 index 00000000..635cbb1c Binary files /dev/null and b/marketing/screenshots/language-support/tagalog-accessibility.png differ diff --git a/marketing/screenshots/language-support/tagalog-onboarding.png b/marketing/screenshots/language-support/tagalog-onboarding.png new file mode 100644 index 00000000..acbdd087 Binary files /dev/null and b/marketing/screenshots/language-support/tagalog-onboarding.png differ diff --git a/marketing/screenshots/language-support/tagalog.png b/marketing/screenshots/language-support/tagalog.png new file mode 100644 index 00000000..b1dbf02a Binary files /dev/null and b/marketing/screenshots/language-support/tagalog.png differ diff --git a/release/android/metadata/en-US/full-description.txt b/release/android/metadata/en-US/full-description.txt index d8cba5e1..27aa1c4a 100644 --- a/release/android/metadata/en-US/full-description.txt +++ b/release/android/metadata/en-US/full-description.txt @@ -10,7 +10,7 @@ Build vocabulary in context Revisit words from your conversations. Three recall bars reflect the evidence from your practice over time. They are learning guidance, not a test score or a promise of fluency. Choose your language -Practise Norwegian Bokmål, Spanish from Spain, international English, French from France, German, Italian, Brazilian Portuguese or Mandarin. Mandarin uses Simplified Chinese with optional pinyin. Each language keeps its own learning history. +Practise Norwegian Bokmål, Spanish from Spain, international English, French from France, German, Italian, Brazilian Portuguese, Mandarin or Tagalog (Filipino) from the Philippines. Mandarin uses Simplified Chinese with optional pinyin. Each language keeps its own learning history. Keep your learning with you Conversations, vocabulary and progress are saved on your device. Export a learning backup and import it on another Mural installation. Mural does not save raw audio. Your audio and selected text are sent to OpenAI while you practise, and provider retention policies apply. diff --git a/release/app-store-metadata.md b/release/app-store-metadata.md index cbefcfd9..ff138c35 100644 --- a/release/app-store-metadata.md +++ b/release/app-store-metadata.md @@ -43,7 +43,7 @@ Choose from 24 themes, from ordering coffee to discussing a film. For something Your vocabulary grows from words you use in conversation. Three recall bars reflect repeated practice over time. Each language keeps its own conversations and progress; the bars and ability observations are guidance, not a language qualification. -Practise Norwegian Bokmål, Spanish from Spain, international English, French from France, German, Italian, Brazilian Portuguese or Mandarin with Simplified Chinese and optional pinyin. Your conversations and learning records stay in the app on your device. You can export a backup and import it on another installation. +Practise Norwegian Bokmål, Spanish from Spain, international English, French from France, German, Italian, Brazilian Portuguese, Mandarin with Simplified Chinese and optional pinyin, or Tagalog (Filipino) from the Philippines. Your conversations and learning records stay in the app on your device. You can export a backup and import it on another installation. This version requires your own OpenAI API key, API billing and access to the supported models. API usage is billed by OpenAI. Mural has no account requirement or credit purchases in this version. Audio and selected text are sent to OpenAI while you practise; provider retention policies apply. Mural does not save raw audio. @@ -55,7 +55,7 @@ Four 6.9-inch upload PNGs are prepared in [screenshots/en-US](screenshots/en-US/ ## TestFlight beta description -Try Mural, a voice conversation app for language practice. The beta includes Norwegian, Spanish, English, French, German, Italian, Brazilian Portuguese and Mandarin, with meaning subtitles, everyday themes, vocabulary recall bars and local learning backups. Mandarin uses Simplified Chinese with optional pinyin. This build uses your own OpenAI project key and bills API usage to that project. An internet connection is required. +Try Mural, a voice conversation app for language practice. The beta includes Norwegian, Spanish, English, French, German, Italian, Brazilian Portuguese, Mandarin and Tagalog (Filipino) from the Philippines, with meaning subtitles, everyday themes, vocabulary recall bars and local learning backups. Mandarin uses Simplified Chinese with optional pinyin. This build uses your own OpenAI project key and bills API usage to that project. An internet connection is required. ## What to test diff --git a/services/api/README.md b/services/api/README.md index 1b05254e..97ea5e54 100644 --- a/services/api/README.md +++ b/services/api/README.md @@ -93,7 +93,7 @@ All monetary strings are integer **nanoUSD**: 1 USD = 1,000,000,000 nanoUSD. A d | `POST /v1/checkout` | Bearer token; `Idempotency-Key`; `product` = `ai-10-usd` or `ai-25-usd` | `checkoutURL`, `orderID`, `sandbox: true`, itemized `quote` | | `POST /v1/webhooks/stripe` | Raw signed Stripe JSON | Atomic payment/refund journal update and duplicate receipt | | `POST /v1/trial/eligibility` | Apple attestation proof | `503 trial_attestation_unavailable` until a verified adapter is configured | -| `POST /v1/live/sessions` | Bearer token; `Idempotency-Key`; `sdp`; `language` (`nb-NO`, `es-ES`, `en-US`, `fr-FR`, `de-DE`, `it-IT`, `pt-BR`, `zh-CN`) | Default `503`; allowlisted experiment returns `sessionID`, `providerSessionID`, `sdp`, `deadline`, `reservedNanoUSD`, `rateVersion`, `experimental` | +| `POST /v1/live/sessions` | Bearer token; `Idempotency-Key`; `sdp`; `language` (`nb-NO`, `es-ES`, `en-US`, `fr-FR`, `de-DE`, `it-IT`, `pt-BR`, `zh-CN`, `tl-PH`) | Default `503`; allowlisted experiment returns `sessionID`, `providerSessionID`, `sdp`, `deadline`, `reservedNanoUSD`, `rateVersion`, `experimental` | | `GET /v1/live/sessions/:id` | Bearer token belonging to the session owner | Minimal state, cumulative milliseconds, confirmed provider cost and customer debit | | `POST /v1/live/sessions/:id/close` | Bearer token belonging to the session owner | Requests server closure; never accepts client-reported usage | diff --git a/services/api/src/live-provider.ts b/services/api/src/live-provider.ts index e80054eb..fb4d272d 100644 --- a/services/api/src/live-provider.ts +++ b/services/api/src/live-provider.ts @@ -39,7 +39,8 @@ export interface LiveProvider { const languages: Record = { 'nb-NO': 'Norwegian Bokmål with an Eastern Norwegian pronunciation', 'es-ES': 'Spanish from Spain', 'en': 'English', 'en-US': 'English', 'fr-FR': 'French from France', 'de-DE': 'Standard German as spoken in Germany', 'it-IT': 'Italian as spoken in Italy', - 'pt-BR': 'Brazilian Portuguese', 'zh-CN': 'Standard Mandarin with Simplified Chinese writing' }; + 'pt-BR': 'Brazilian Portuguese', 'zh-CN': 'Standard Mandarin with Simplified Chinese writing', + 'tl-PH': 'Tagalog as spoken in the Philippines, using natural everyday wording and polite po/opo when appropriate; accept valid regional Tagalog and established loanwords' }; export const supportsLanguage = (language: string) => Object.hasOwn(languages, language); const sessionPath = (id: string) => { if (!id || id.length > 256 || /[\x00-\x20]/.test(id)) throw new ServiceError('invalid_provider_session', 502); diff --git a/services/api/tests/hosted.test.ts b/services/api/tests/hosted.test.ts index 34273e15..a7837a1a 100644 --- a/services/api/tests/hosted.test.ts +++ b/services/api/tests/hosted.test.ts @@ -599,6 +599,39 @@ integration('a late provider result cannot reopen a session closed by operator r } finally { await f.cleanup(); } }); +integration('Tagalog locale creates and settles voice while aliases cannot reserve credit or minutes', async () => { + for (const minuteAllowance of [undefined, 90_000]) { + const f = await fixture(2_000_000_000n, minuteAllowance); + try { + const beforeWallet = await f.wallet(), beforeMinutes = await f.minutes(); + for (const language of ['tl', 'fil', 'fil-PH', 'tgl-PH', 'tl_PH', '__proto__']) { + await assert.rejects(f.controller.create(f.account, `tagalog-invalid-${language}`, 'v=0', language), { code: 'invalid_live_offer' }); + } + assert.equal(f.creates, 0); + assert.deepEqual(await f.wallet(), beforeWallet); + assert.deepEqual(await f.minutes(), beforeMinutes); + for (const table of ['reservations', 'minute_reservations', 'hosted_sessions']) + assert.equal((await f.db.query(`SELECT count(*) FROM ${table}`)).rows[0].count, '0'); + const live = await f.controller.create(f.account, 'tagalog-valid-offer', 'v=0', 'tl-PH'); + assert.equal(f.creates, 1); + assert.match((f.payloads[0] as { session: { instructions: string } }).session.instructions, /Speak only Tagalog as spoken in the Philippines/); + if (minuteAllowance === undefined) assert.equal((await f.wallet()).reserved_nano, '500000000'); + else assert.deepEqual(await f.minutes(), { balance_ms: '90000', reserved_ms: '90000' }); + await f.controller.close(f.account, live.sessionID); + f.send(live.providerSessionID, { type: 'session.closed', usage: { seconds: 20 } }); + await until(async () => (await f.controller.status(f.account, live.sessionID)).state === 'closed'); + const status = await f.controller.status(f.account, live.sessionID); + if (minuteAllowance === undefined) { + assert.equal((await f.wallet()).reserved_nano, '0'); + assert.equal(status.chargedNanoUSD, '16666667'); + } else { + assert.deepEqual(await f.minutes(), { balance_ms: '70000', reserved_ms: '0' }); + assert.equal(status.chargedMilliseconds, 20_000); + } + } finally { await f.cleanup(); } + } +}); + integration('signed-in International English uses free minutes and settles the conversation', async () => { const f = await fixture(2_000_000_000n, 600_000, 50_000_000n, true); try { diff --git a/services/api/tests/languages.test.ts b/services/api/tests/languages.test.ts index d3bee6c9..c4755891 100644 --- a/services/api/tests/languages.test.ts +++ b/services/api/tests/languages.test.ts @@ -20,18 +20,19 @@ test('all native locales reach the provider with the intended regional speech ta for (const [locale, target] of [ ['nb-NO', 'Norwegian Bokmål'], ['es-ES', 'Spanish from Spain'], ['en', 'English'], ['en-US', 'English'], ['fr-FR', 'French from France'], ['de-DE', 'Standard German as spoken in Germany'], ['it-IT', 'Italian as spoken in Italy'], - ['pt-BR', 'Brazilian Portuguese'], ['zh-CN', 'Standard Mandarin with Simplified Chinese writing'] + ['pt-BR', 'Brazilian Portuguese'], ['zh-CN', 'Standard Mandarin with Simplified Chinese writing'], + ['tl-PH', 'Tagalog as spoken in the Philippines'] ]) { assert.equal(supportsLanguage(locale!), true, locale); await provider.create('v=0', locale!); assert.ok(requests.at(-1).session.instructions.includes(`Speak only ${target}`)); assert.equal(requests.at(-1).session.store, false); } - for (const unsupported of ['pt-PT', 'de', 'zh', 'zh-TW', '__proto__', 'constructor', '']) { + for (const unsupported of ['pt-PT', 'de', 'zh', 'zh-TW', 'tl', 'fil', 'fil-PH', 'tgl-PH', 'tl_PH', 'TL-ph', '__proto__', 'constructor', '']) { assert.equal(supportsLanguage(unsupported), false); await assert.rejects(provider.create('v=0', unsupported), { code: 'invalid_language' }); } - assert.equal(requests.length, 9); + assert.equal(requests.length, 10); } finally { await new Promise(resolve => server.close(() => resolve())); } diff --git a/verification/validation.md b/verification/validation.md index a49e42aa..c71d9416 100644 --- a/verification/validation.md +++ b/verification/validation.md @@ -177,3 +177,73 @@ Visual review after the full UI run found that the fixed consent footer crowded After the owner unlocked the phone, the final build launched normally at 16:37:51 CEST. Its running process was confirmed. This reopened the persistent learning store without verification arguments. The device results verify the application/provider paths with synthetic typed input and real voice output. They do not verify recognition of a human speaker, pronunciation, tones, correction quality, unscripted interruptions, headphones or cellular operation. The proficient-speaker checks requested in issues #10–#13 remain open. Pinyin uses dictionary tones and may need correction for names, ambiguous words and connected-speech tone changes. The Android contribution is not integrated in this checkout, so there is no generated Android language catalog to update here. + +## Tagalog (Filipino) from the Philippines + +13 September 2026. Added the ninth learning target with stable ID `tl`, locale `tl-PH` and display label `Tagalog (Filipino) · Philippines`. It uses the existing voice, text, meaning, vocabulary and archive paths. [Teaching choices and sources](../docs/tagalog.md) document the Tagalog.com and University of Hawai‘i references; [the implementation record](../TAGALOG-PLAN.md) maps the feature and tests. + +Automated checks on Xcode 26.6 / Swift 6.3.3: + +- **90 core tests passed, zero failures.** The 20 added cases cover module identity and prompt paths, six-stage selection, Philippine themes, unreliable detector labels, unaided and assisted evidence, foreign/alias proposals, mixed passages, exact provenance, aspect/focus vocabulary identities, homographs, hidden words, punctuation, archive/import boundaries and late results after switching languages. The pre-implementation run of the 17 focused Tagalog tests produced 15 failures; the final full suite passed after implementation and integration with the existing tests. +- **80 backend tests passed, zero failures and zero skips**, with Node 26.8.1 and an isolated PostgreSQL 17 database. TypeScript type checking and build passed. The locale test failed on `tl-PH` before the provider map was updated, then passed for all nine locales. The added integration test proves invalid aliases create neither reservations nor provider calls, and a valid Tagalog session reserves and settles credit through the local fake HTTP/WebSocket provider. +- **Debug simulator test build passed**, including the updated live-verification helper. **Unsigned Release iPhone build passed**. Existing transport/dependency warnings remain; no dependency version, signing configuration, deployment or billing configuration changed. +- **25 native UI tests passed, zero failures**, on a dedicated iPhone 17 simulator running iOS 26.5. The full suite includes five new Tagalog tests and the existing Spanish, Portuguese, Norwegian and Mandarin paths. Tagalog coverage verifies onboarding, the full display alias at maximum accessibility size, subtitle choice after Back, Settings/themes/Words, a disabled language picker during a conversation, saved transcripts after reset and switching, and persistence across a normal relaunch. Result: `.build/Tagalog-UI.xcresult`, completed at 23:03 WEST. Visual inspection confirmed readable selection, captions and English meanings; [unedited screenshots](../marketing/screenshots/language-support/README.md) are included. + +Xcode's initial WebRTC artifact download stalled. The same pinned release archive was downloaded separately, verified against the manifest SHA-256 `115cb9944248a3302c0c8af17462e2576a28ccc7adef9f6a1fe66ee75d9e1cc8`, and placed in SwiftPM's artifact cache. Xcode then built successfully. This was a local dependency-cache recovery, not a package/lockfile change. + +The speech guard intentionally skips Tagalog because the local Apple recognizer misclassified valid Tagalog as Indonesian above 99% confidence on both macOS and the iOS 26.5 simulator. Other languages retain their previous detector behavior. The Debug report now separates mechanical `flowPassed` from reliable target detection and records the actual detector label/confidence. It does not turn unavailable language validation into a passing result. + +No real-provider call or physical-device speech test was performed for this change. The owner chose automated verification with device review pending. Synthetic fixtures do not validate human speech recognition, generated pronunciation, translation accuracy, correction quality or model-generated lemmas. A proficient Tagalog speaker should review a real conversation before making those claims. Check a greeting, English support and mixed replies, café/market requests, polite register, a longer answer, meaningful corrections, English meanings, contextual lookup, a sourced topic and unavailable-lookup fallback; then interruption, mute, speaker/headphones and closure. Include unaided spoken evidence and switching/relaunch with the persistent store. + + +## 14 September 2026 — Tagalog integration with the native-app monorepo + +Integrated upstream `main` at `1c175af64b863335bc5155588968f45f8d29d839`. The Tagalog module and tests now live under `apps/ios`, and the hosted locale and regression tests live under `services/api`. The deleted legacy `server/tests/hosted.test.ts` was not restored. Its Tagalog case was ported into the current suite and now verifies both credit-funded and minute-funded reservation/settlement, including rejection of aliases before any provider call or reservation. + +Android's generated language catalog includes the same Tagalog module. A new Kotlin regression checks its label, locale, greeting, six stages, Philippine café theme, English support, single namespace and isolated archive progress. The Apple recognizer exception remains iPhone-specific; no Android classifier behavior was changed. + +Checks on the integrated tree: + +- **93 Swift core tests passed**, including the shared archive compatibility fixtures and all 17 focused Tagalog cases. +- **304 API tests passed with zero failures and zero skips**, using a dedicated PostgreSQL 17 test database. TypeScript check and build passed. The initial database launch used the default port while the test URL selected the dedicated port; correcting that local launch configuration allowed the full suite to run. The database was stopped afterward. +- **258 Android JVM tests passed with zero failures, errors or skips**. `:app:lintDebug` completed with zero errors (44 warnings and two hints), and `:app:assembleDebug` produced the APK. Java 17 and Android SDK 36 were used. +- **45 repository-tool tests passed**. Android generated-content, cross-platform contract and release-file checks passed. +- **Unsigned Release iPhone build passed** from `apps/ios/Mural.xcodeproj`. +- **25 native iOS UI tests passed with zero failures** on the iPhone 17 / iOS 26.5 simulator, including the complete Tagalog and existing-language flows. The Debug build and tests used the relocated project. Result: `.build/Tagalog-Merge-UI.xcresult`. + +No physical-device or live-provider check was performed. The original pending Tagalog pronunciation, recognition, correction and proficient-speaker review remains pending on both platforms. Android verification here covers JVM behavior, lint and compilation; it is not an Android emulator or device conversation check. + +### CodeRabbit review follow-up + +The normal-relaunch UI test now restores Norwegian in an XCTest teardown block and verifies that restoration across a further normal launch. The maximum-accessibility test explicitly checks containment of the complete Tagalog card in the scroll viewport, containment of Continue in the screen and separation between them. The main README's remaining eight-language reference was corrected to nine. + +Both affected UI tests ran twice: **four executions passed with zero failures** in `.build/Tagalog-Review-Fixes.xcresult` on the same iPhone 17 / iOS 26.5 simulator. Visual inspection of a fresh accessibility attachment confirmed the Tagalog text and Continue button are untruncated; the preceding list row is intentionally partly outside the viewport. Production app behavior was unchanged, so the earlier core, API, Android and full UI suite results remain recorded above. The generic 80% docstring-coverage warning was not treated as a repository requirement or a reason to add boilerplate documentation to self-describing test functions. + + +## 15 September 2026 — upstream Android parity and recovery integration + +Merged upstream `main` at `926fd95` into the Tagalog branch. The Android README retains the new Mandarin word-link and platform-specific pinyin guidance while reporting all nine languages. The hosted-session suite retains every new upstream rejection/recovery test and the existing Tagalog credit/minute reservation and settlement test. Android content was regenerated with the updated exporter, preserving the upstream Swift-derived defaults and meaning-language catalog. + +- **351 API tests passed with zero failures or skips**, using an isolated PostgreSQL 17 database, which was stopped afterward. TypeScript check and build passed. +- **315 Android JVM tests passed with zero failures, errors or skips**. Android Debug APK assembly and lint passed. +- **53 repository-tool tests passed**. Generated-content and cross-platform consistency checks passed, and the PR diff against upstream passed whitespace validation. +- **Swift core build passed** using the installed Command Line Tools. The Swift test rerun could not compile because those tools do not include XCTest; the selected Xcode installation now requires license acceptance. The earlier successful Swift/UI results remain historical, not a new test pass. No iOS application source changed in this merge; upstream added a shared redirect-fixture test. Native UI and physical-device/provider checks were not rerun. + +### Review of other language PRs + +Reviewed Afrikaans (#21), European Portuguese (#27), multilingual classrooms (#26), the Mandarin/four-language contributions (#4/#15), Android parity (#23) and the related English startup fix (#28). [Decisions and source links](../docs/tagalog.md#lessons-from-other-language-contributions) distinguish inherited improvements from separate product changes. + +Added five Android Tagalog regression tests covering exact caption text and lookup links, aspect versus voice/focus vocabulary through archive export/import, homographs and language-scoped hidden words, rejection of foreign/display-alias evidence, and subtitle/typed support that cannot count as unaided recall. Corrected the Android store description to include Tagalog. + +- **320 Android JVM tests passed, zero failures, errors or skips**, including all five added cases; Debug lint passed. +- **53 repository-tool tests passed**; generated-content and cross-platform checks passed. +- No production application or API code changed in this follow-up. Swift/UI/API suites were not rerun. These fixture tests validate storage and text handling, not generated linguistic judgments. Real-device/provider and proficient-speaker review remain pending. + +### Integration of the International English startup fix + +Merged upstream `3a12147` (PR #28). Resolved the append conflict in `services/api/tests/hosted.test.ts` by retaining both complete English and Tagalog integration cases. The provider test now expects ten requests: nine native locales plus the existing `en-US` compatibility alias. The upstream registry-to-API admission check is retained unchanged and includes Tagalog automatically. + +- **357 API tests passed with zero failures or skips**, including both funded language flows and actual native-registry admission. TypeScript check/build passed; the isolated PostgreSQL test database was stopped afterward. +- **324 Android JVM tests passed with zero failures, errors or skips**; Debug assembly and lint passed. +- **53 repository-tool tests passed**; generated-content and cross-platform checks passed. +- No iOS source changed. Swift/UI and physical-device/provider checks were not rerun; the previously documented linguistic review remains pending.