Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ jobs:
run: pwsh -File tools/android/Test-DualDexAndroidTools.ps1

- name: Run Kotlin tests
run: .\gradlew.bat :parser-core:test :parser-cli:test :companion-core:test :companion-simulator:test :companion-server:test --stacktrace
run: .\gradlew.bat verifySecureBuildDependencies :parser-core:test :parser-cli:test :companion-core:test :companion-simulator:test :companion-server:test --stacktrace

- name: Test web UI
working-directory: companion-web
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ jobs:
docs/reports/2026-08-13-map-first50-release-gate-raw.json >/dev/null
jq -e '.summary.identities == 50 and .summary.completeTables == 50 and .summary.malformedRows == 0 and .summary.deterministicHashes == 50 and .summary.sqliteReopenExact == 50 and .summary.originalCompleteTablesPreserved == .summary.originalCompleteTables and (.rows | length) == 50' \
docs/reports/2026-08-14-first50-evolution-completeness-raw.json >/dev/null
jq -e '.baseFirst50Selected == 50 and .mapFirst50Available == 26 and .evolutionFirst50Complete == 50 and .arm7Applicable == 46 and .arm7ProductionComplete == 38 and (has("debugApkSha256") | not)' \
jq -e '.baseFirst50Selected == 50 and .mapFirst50Available == 26 and .evolutionFirst50Complete == 50 and .secureBuildDependencyGate == true and .securityAdvisoriesCovered == 48 and .arm7Applicable == 46 and .arm7ProductionComplete == 38 and (has("debugApkSha256") | not)' \
release/v1-ready.json >/dev/null
if grep -Eiq '[A-Z]:\\|dualdex-expanded-corpus|ExtractedPath' \
reports/dualdex-parser-compatibility.json \
Expand Down Expand Up @@ -173,7 +173,7 @@ jobs:
- name: Test all modules, lint, and build unsigned release APK
shell: bash
run: >-
bash ./gradlew test :app:lintDebug :app:assembleRelease
bash ./gradlew verifySecureBuildDependencies test :app:lintDebug :app:assembleRelease
-PdualdexVersionName=${{ steps.metadata.outputs.version_name }}
-PdualdexVersionCode=${{ steps.metadata.outputs.version_code }}
--stacktrace
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ DualDex is a passive Pokédex companion for mainline-family Pokémon games runni
The game remains on the primary display. DualDex detects the active GB, GBC, or GBA content, parses the user's ROM into a local SQLite Pokédex, and refreshes seen/caught/team/area knowledge from checksum-valid SaveRAM. Validated live layouts can supersede stale disk state for current location, party, and battle context through RetroArch's read-only Network Commands. It does this without OCR, screenshots, cheats, memory writes, or per-hack profiles. A separately isolated issue-report tool can export read-only evidence for unsupported layouts, but its dumps never feed the production Pokédex.

> [!IMPORTANT]
> The pure-Kotlin ROM parser, materialized SQLite catalog, Gen I–III SaveRAM readers, validated live-WRAM paths, loopback web host, Thor-first UI, passive RetroArch activation, Docked/Overlay modes, and isolated read-only issue reports are implemented. Release candidate `v1.0.0-rc.28` adds broader first-50 move-table coverage and clearer loading/map behavior to RC27's complete evolution catalogs, consistent Organic identity artwork, normalized ROM-derived maps, semantic ARM7TDMI mechanics, and proven-wild Rarity behavior. Stable `v1.0.0` has not been released.
> The pure-Kotlin ROM parser, materialized SQLite catalog, Gen I–III SaveRAM readers, validated live-WRAM paths, loopback web host, Thor-first UI, passive RetroArch activation, Docked/Overlay modes, and isolated read-only issue reports are implemented. Release candidate `v1.0.0-rc.29` adds source-backed compiled Pokédex-description recovery and a Dependabot-audited build-toolchain gate to RC28's broader first-50 move-table coverage, complete evolution catalogs, consistent Organic identity artwork, normalized ROM-derived maps, semantic ARM7TDMI mechanics, and proven-wild Rarity behavior. Stable `v1.0.0` has not been released.

## Thor-first UI direction

Expand Down Expand Up @@ -195,7 +195,7 @@ These denominators are deliberately different. A selected base catalog is not co

Numeric ability mechanics are tracked separately from names and descriptions. The production resolver follows decoded calls and use-def relationships from parser-selected layouts into typed battle fields, predicates, arithmetic, and writeback. It never substitutes familiar series values, names, hashes, symbols, or fixed routine addresses for missing proof.

Read the player-facing [ROM Hacks Compatibility report](reports/dualdex-rom-hacks-compatibility.md), with its [machine-readable JSON](reports/dualdex-rom-hacks-compatibility.json), for the reviewed first 50 ROMs grouped by generation and engine family. The separate [Parser Compatibility report](reports/dualdex-parser-compatibility.md) and [schema-11 JSON evidence](reports/dualdex-parser-compatibility.json) retain the reviewed RC24 evidence contract; the independent [exact first-50 base release gate](docs/reports/2026-08-13-base-first50-release-gate.md) records RC25's 50/50 result without rewriting that historical report. Optional capability evidence is published independently in the [exact first-50 evolution gate](docs/reports/2026-08-14-first50-evolution-completeness.md), [world-map first-50 release gate](docs/reports/2026-08-13-map-first50-release-gate.md), and [ARM7TDMI first-50 survey](docs/reports/arm7-first50-compatibility-survey.md). The [full unique-ROM base audit](docs/reports/2026-08-13-base-full332-compatibility.md) keeps broader coverage visible without treating every optional feature as resolved. Reports contain structural evidence and hashes, but no decoded bulk tables, sprites, ROM bytes, saves, trainer data, or private paths.
Read the player-facing [ROM Hacks Compatibility report](reports/dualdex-rom-hacks-compatibility.md), with its [machine-readable JSON](reports/dualdex-rom-hacks-compatibility.json), for the reviewed first 50 ROMs grouped by generation and engine family. The separate [Parser Compatibility report](reports/dualdex-parser-compatibility.md) and [schema-11 JSON evidence](reports/dualdex-parser-compatibility.json) retain the reviewed RC24 evidence contract; the independent [exact first-50 base release gate](docs/reports/2026-08-13-base-first50-release-gate.md) records RC25's 50/50 result without rewriting that historical report. Optional capability evidence is published independently in the [exact first-50 evolution gate](docs/reports/2026-08-14-first50-evolution-completeness.md), [Celia Pokédex-description closure](docs/reports/2026-08-14-first50-celia-pokedex-descriptions.md), [world-map first-50 release gate](docs/reports/2026-08-13-map-first50-release-gate.md), and [ARM7TDMI first-50 survey](docs/reports/arm7-first50-compatibility-survey.md). The [full unique-ROM base audit](docs/reports/2026-08-13-base-full332-compatibility.md) keeps broader coverage visible without treating every optional feature as resolved. Reports contain structural evidence and hashes, but no decoded bulk tables, sprites, ROM bytes, saves, trainer data, or private paths.

SaveRAM evidence is reported separately for [Generations I/II](docs/reports/gen1-gen2-saveram-compatibility.md) and [Generation III](docs/reports/gen3-saveram-compatibility.md). These reports contain no ROM/save bytes, trainer data, or private filesystem paths.

Expand Down
95 changes: 93 additions & 2 deletions build.gradle.kts
Original file line number Diff line number Diff line change
@@ -1,6 +1,97 @@
// Top-level build file where you can add configuration options common to all sub-projects/modules.
buildscript {
configurations.configureEach {
resolutionStrategy.eachDependency {
val secureVersion = when (requested.group to requested.name) {
"org.bouncycastle" to "bcpkix-jdk18on",
"org.bouncycastle" to "bcprov-jdk18on",
"org.bouncycastle" to "bcutil-jdk18on" -> "1.84"
"org.bitbucket.b_c" to "jose4j" -> "0.9.6"
"org.jdom" to "jdom2" -> "2.0.6.1"
"org.apache.commons" to "commons-lang3" -> "3.18.0"
"org.apache.httpcomponents" to "httpclient" -> "4.5.14"
else -> null
}
if (secureVersion != null) {
useVersion(secureVersion)
because("Keep the Android build toolchain above published security floors")
}
}
}
}

plugins {
alias(libs.plugins.android.application) apply false
id("org.jetbrains.kotlin.android") version "2.3.10" apply false
id("org.jetbrains.kotlin.jvm") version "2.3.10" apply false
id("org.jetbrains.kotlin.android") version "2.4.20-Beta2" apply false
id("org.jetbrains.kotlin.jvm") version "2.4.20-Beta2" apply false
}

allprojects {
configurations.configureEach {
resolutionStrategy.eachDependency {
val secureVersion = when {
requested.group == "io.netty" -> "4.1.136.Final"
requested.group == "org.bouncycastle" && requested.name in setOf(
"bcpkix-jdk18on",
"bcprov-jdk18on",
"bcutil-jdk18on",
) -> "1.84"
requested.group == "org.bitbucket.b_c" && requested.name == "jose4j" -> "0.9.6"
requested.group == "org.jdom" && requested.name == "jdom2" -> "2.0.6.1"
requested.group == "org.apache.commons" && requested.name == "commons-lang3" -> "3.18.0"
requested.group == "org.apache.httpcomponents" && requested.name == "httpclient" -> "4.5.14"
else -> null
}
if (secureVersion != null) {
useVersion(secureVersion)
because("Keep build and test tooling above published security floors")
}
}
}
}

tasks.register("verifySecureBuildDependencies") {
group = "verification"
description = "Verifies the resolved build and Android test toolchains against the RC29 security pins."

doLast {
val expectedVersions = mapOf(
"org.jetbrains.kotlin:kotlin-gradle-plugin" to setOf("2.4.20-Beta2"),
"org.bouncycastle:bcpkix-jdk18on" to setOf("1.84"),
"org.bouncycastle:bcprov-jdk18on" to setOf("1.84"),
"org.bitbucket.b_c:jose4j" to setOf("0.9.6"),
"org.jdom:jdom2" to setOf("2.0.6.1"),
"org.apache.commons:commons-lang3" to setOf("3.18.0"),
"org.apache.httpcomponents:httpclient" to setOf("4.5.14"),
"com.google.protobuf:protobuf-java" to setOf("3.25.5", "4.28.3"),
"com.google.protobuf:protobuf-kotlin" to setOf("4.28.3"),
)
val resolved = mutableSetOf<Pair<String, String>>()
val configurationsToCheck = buildList {
addAll(rootProject.buildscript.configurations.filter { it.isCanBeResolved })
rootProject.allprojects.forEach { project ->
addAll(project.configurations.filter { it.isCanBeResolved })
}
}

configurationsToCheck.forEach { configuration ->
configuration.incoming.resolutionResult.allComponents.forEach { component ->
component.moduleVersion?.let { module ->
resolved += "${module.group}:${module.name}" to module.version
}
}
}

val violations = resolved.mapNotNull { (coordinate, version) ->
val expected = when {
coordinate.startsWith("io.netty:") -> setOf("4.1.136.Final")
else -> expectedVersions[coordinate]
} ?: return@mapNotNull null
if (version in expected) null else "$coordinate:$version (expected ${expected.joinToString(" or ")})"
}.sorted()

check(violations.isEmpty()) {
"Unsafe build-tool dependency versions resolved:\n${violations.joinToString("\n")}"
}
}
}
64 changes: 64 additions & 0 deletions docs/reports/2026-08-14-first50-celia-pokedex-descriptions.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
{
"schemaVersion": 1,
"date": "2026-08-14",
"scope": "exact-first-50",
"change": "compiled Gen III Pokedex-entry descriptions",
"implementationCommit": "b5809db",
"numericCompatibility": {
"first50AverageBefore": 95.53,
"first50AverageAfter": 95.63,
"celiaBefore": 90.40,
"celiaAfter": 95.14,
"celiaResolvedFeaturesBefore": 19,
"celiaResolvedFeaturesAfter": 20,
"celiaExpectedFeatures": 21
},
"capabilities": {
"pokedexDescriptionsAvailableBefore": 35,
"pokedexDescriptionsAvailableAfter": 35,
"pokedexDescriptionsPartialBefore": 11,
"pokedexDescriptionsPartialAfter": 12,
"pokedexDescriptionsNotFoundBefore": 4,
"pokedexDescriptionsNotFoundAfter": 3,
"evolutionsAvailable": 50
},
"celia": {
"sha256": "81ac9b9d4e7bdd3bf06ed53954d784118a743372906c6c6fc62b3cbc19587148",
"family": "FIRERED_LEAFGREEN",
"tableRows": 386,
"recordSize": 36,
"descriptionPointerOffset": 16,
"decodedPhysicalRows": 384,
"semanticCoveredRows": 382,
"semanticExpectedRows": 384,
"semanticSha256": "d62c951fc4ddbf35f3a251ea9b26c4e1664c1ff2b957601db800d73ecbfedad7",
"referenceErrors": 0,
"sqliteBytes": 1527808,
"catalogSections": 12,
"sourceOracleCommit": "8b31f2472810f75571d122159d164467e149d4a8"
},
"exact50": {
"rows": 50,
"selected": 50,
"ambiguous": 0,
"noFamilyMatch": 0,
"errors": 0,
"exactShaIdentities": 50,
"routingDeltas": 0,
"first33RoutingOrReferenceDeltas": 0,
"referenceErrorRows": 0,
"persistedAndReopened": 50,
"sqliteQuickCheckErrors": 0,
"sqliteForeignKeyErrors": 0,
"catalogSectionCounts": [12]
},
"verification": {
"nonThumbParserTests": 996,
"nonThumbParserFailures": 0,
"nonThumbParserErrors": 0,
"nonThumbParserSkipped": 106,
"focusedDescriptionGate": "BUILD_SUCCESSFUL",
"rawJsonSha256": "ac10886a7bc3afb2f48c49202eb9bfb3011db0bb045cc14f43ca0377a7813d02",
"rawMarkdownSha256": "639a99c9679be39eb906d045cad3894b0e95d399892387974a8f214ee85e9592"
}
}
52 changes: 52 additions & 0 deletions docs/reports/2026-08-14-first50-celia-pokedex-descriptions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# First-50 Celia Pokédex-description closure — 2026-08-14

## Outcome

Celia's Stupid Romhack now resolves its compiled Gen III Pokédex-entry table.

- First-50 numeric compatibility average: **95.53% → 95.63%**.
- Celia numeric compatibility: **90.40% → 95.14%** (**19/21 → 20/21** applicable features).
- First-50 `POKEDEX_DESCRIPTIONS`: **35 available / 11 partial / 4 not found → 35 available / 12 partial / 3 not found**.
- Celia publishes **382 decoded descriptions across 384 navigable Pokédex species**. The two unsupported active rows remain unavailable rather than receiving external or fabricated text.
- Evolutions remain **50/50 available**. THUMB ability mechanics are unchanged and intentionally deferred.

## Structural authority

Production selection uses ROM-derived structure only:

- the source-defined published Gen III header supplies a bounded Pokédex count only when its fixed species-name, move-name, and sprite pointer roles are valid;
- the published count must be within the independently decoded species-name domain;
- compiled ROM references nominate description-table candidates;
- the typed codec validates all 386 records under the selected 36-byte ABI and description pointer at `+16`;
- the typed candidate must agree with the existing structural description validator;
- the independently compiled species-to-Dex mapping defines the navigable semantic domain;
- partial rows and out-of-domain internal slots stay unavailable.

No ROM name, SHA-256, source symbol, or absolute table offset participates in production selection. Exact identity and addresses exist only in the live regression.

The comparative source oracle is Celia's Stupid Repository commit `8b31f2472810f75571d122159d164467e149d4a8`. Its `PokedexEntry` layout and `NATIONAL_DEX_COUNT` agree with the independently decoded ROM table. The exact live ROM SHA-256 is `81ac9b9d4e7bdd3bf06ed53954d784118a743372906c6c6fc62b3cbc19587148`.

## Fail-closed behavior

- Retail and previously supported hack layouts retain their existing typed selection path.
- A published count without the required fixed header pointer roles is ignored.
- A count outside the independently decoded species-name domain is ignored.
- Typed discovery is used only when the existing structural description path fails.
- Typed and legacy structural selection must agree on root, count, and record width.
- The two non-decodable active rows remain missing; a partial table cannot erase otherwise valid species identities.

## Verification

- Implementation commit: `b5809db` (`Resolve compiled Pokedex descriptions`).
- Full non-THUMB parser regression: **996 tests, 0 failures, 0 errors**; 106 opt-in controls skipped.
- Focused live description, resolver, published-header, and one-pass architecture gate: **BUILD SUCCESSFUL**.
- Exact first-50: **50/50 SELECTED**, 0 ambiguous, 0 no-family, 0 errors.
- Exact identity/order: **50/50**; routing deltas: **0/50**; first-33 routing/reference deltas: **0/33**.
- Reference errors: **0/50**.
- Persistence: **50/50** catalogs written and reopened; each contains 12 sections.
- SQLite validation: **50/50** `quick_check=ok`; **0** foreign-key findings.
- Celia SQLite: **1,527,808 bytes**, 12 sections, reopened successfully.
- Raw JSON SHA-256: `ac10886a7bc3afb2f48c49202eb9bfb3011db0bb045cc14f43ca0377a7813d02`.
- Raw Markdown SHA-256: `639a99c9679be39eb906d045cad3894b0e95d399892387974a8f214ee85e9592`.

Only one exact-50 corpus pass was run after the focused implementation and controls.
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,7 @@
9. Add a real-ROM RED requiring complete move names/details plus malformed and duplicate-root fail-closed mutations.
10. Implement compiled-nominated, dense-domain unified move selection and the typed 48-byte decoder; reuse the shared parser session and existing catalog materializers.
11. Verify Dreamstone through CatalogParser, SQLite, and reference closure; run focused controls and one exact first-50 regression matrix only.
12. Freeze Celia's source-defined 386-row `PokedexEntry` layout and exact compiled-reference evidence without building the source project.
13. Add a real-ROM regression for the typed 36-byte rows, description pointer at `+16`, partial physical rows, and semantic Pokédex coverage.
14. Integrate the bounded published Pokédex count into the shared description phase, requiring typed and legacy structural agreement and preserving independently compiled species identities.
15. Verify Celia through CatalogParser and SQLite, run the full non-THUMB parser regression, then one fresh exact-first-50 matrix and publish the numeric before/after evidence.
Loading
Loading