F6.16 — TCGdex multi-locale sync (gap-fill + force update)

[jbourdin]

Feature ID: F6.16
Domain: TCGdex sync (extends F6.13)

---

Context

TCGdex publishes new Pokemon TCG expansions in English first, then adds translations (French, others) over the following days/weeks. The current admin panel exposes two buttons (Sync new data / Sync & update metadata) that both pin the API to /v2/en and never fetch translations, so production cannot pick up French translations that arrive after the initial sync.

The admin surface will be simplified to one "Sync" button that fills any gap (missing series/sets/cards or missing locales on existing cards), plus a separate "Force update" form that takes a set picker and re-fetches every card in that set unconditionally. The two existing buttons will be removed.

TCGdex is working on adding a set-level updated_at field; once that lands, freshness diffing will move there. This issue captures the API's per-card updated timestamp into a new tcgdex_updated_at column now — so by the time set-level diffing is viable, every card row already has a populated baseline. The column is written on every API touch but is not yet used as a skip-decision input; the active freshness signal remains "is any configured locale missing?".

Key facts (verified against the live API)

• https://api.tcgdex.net/v2/fr/cards/{id} returns a fully French payload (verified against swsh3-136 → "Fouinar"). Same shape as /v2/en/....
• Per-card responses include "updated": "...". Set responses expose no timestamps anywhere — neither on the set top level nor on its cards[] stubs ({id, image, localId, name} only). Set-level diffing must wait until TCGdex adds the field.
• TcgdexCard already stores name, abilities, attacks, effect, evolveFrom as JSON maps keyed by locale. nameEn / nameFr are MySQL generated columns over name->>'$.en' / name->>'$.fr'. Schema is already multilingual-ready — only the fetcher is English-only.
• The archetype/deck forms already use a TcgdexSet EntityType dropdown built from TcgdexSetRepository::createExpandedSetsQueryBuilder() — reusable verbatim for the force-update form.

Recommended approach

UI changes (single screen, two surfaces)

In templates/admin/technical/dashboard.html.twig:220-233:

1. Remove both existing buttons (tcgdex_sync.insert_button and tcgdex_sync.update_button).
2. Add one new button: "Sync" (app.admin.technical.tcgdex_sync.sync_button) — POSTs to a new route that dispatches a top-level series sync in gap-fill mode.
3. Add a small form below it: a TcgdexSet select (reusing the archetype selector) + a submit button labelled "Force update" (app.admin.technical.tcgdex_sync.force_update_button). On submit, the controller dispatches a SyncTcgdexSetMessage for the chosen set in force-update mode, skipping series/serie discovery.

Sync modes (replace existing enum cases)

Drop the current SyncMode::Insert and SyncMode::Update (and any references — tests, console commands, message constructors). Replace with:

Case	Behavior
SyncMode::Sync	Cascade-walks the catalog. Inserts new series/sets/cards. For each existing card, if every configured locale is already populated, skip with no HTTP call. Otherwise probe /v2/en/..., then fetch each missing-locale endpoint and merge into the JSON columns.
SyncMode::ForceUpdate	Targeted at a single set. Re-fetches every card in the set across every configured locale, unconditionally, merging into the JSON columns. Also picks up any new card in the set that we didn't have.

Locale list

A new container parameter app.tcgdex.locales: ['en', 'fr'] injected into SyncTcgdexCardHandler and TcgdexCardHydrator. Future locales = one-line config change.

Critical files

File	Change
src/Entity/TcgdexCard.php	Add nullable tcgdexUpdatedAt datetime_immutable property + getter/setter. Persisted via the new migration; populated by the hydrator from data['updated'].
migrations/Version*.php	New Doctrine migration adding tcgdex_updated_at DATETIME NULL to tcgdex_card. No backfill — existing rows start NULL and acquire a value the next time they're touched.
src/Enum/SyncMode.php (or wherever it lives)	Remove Insert/Update cases, add Sync and ForceUpdate. Audit all callers via grep -r 'SyncMode::'.
src/MessageHandler/SyncTcgdexCardHandler.php	Replace private const BASE_URL with injected tcgdex.host + tcgdex.locales parameters. Build per-locale URLs. Implement the gap-fill skip-check and the locale fan-out described above.
src/MessageHandler/SyncTcgdexSerieHandler.php, SyncTcgdexSeriesHandler.php, SyncTcgdexSetHandler.php	Update the hard-coded BASE_URL to use the injected host (these handlers only need en for discovery — locale fan-out is per-card).
src/Service/Tcgdex/TcgdexCardHydrator.php	Add mergeLocaleFields(TcgdexCard \$card, string \$locale, array \$data): void — writes only the [locale] key of each multilingual field, preserving other locales. Reuse the existing wrapAbilitiesOrAttacks structure but as a locale-keyed merger. Also: in applyApiFields and the new merge method, parse data['updated'] (ISO-8601 string) into a DateTimeImmutable and call \$card->setTcgdexUpdatedAt(...).
src/Controller/AdminTechnicalController.php	Replace the two existing actions (lines 273-304) with one app_admin_technical_tcgdex_sync action (no params, dispatches SyncTcgdexSeriesMessage(SyncMode::Sync)) and one app_admin_technical_tcgdex_force_update action (POST, handles the set form, dispatches SyncTcgdexSetMessage(\$setId, SyncMode::ForceUpdate)).
src/Form/TcgdexForceUpdateFormType.php (new)	One latestSet-style EntityType<TcgdexSet> field using TcgdexSetRepository::createExpandedSetsQueryBuilder() and the same choice_label callback as ArchetypeVariantFormType:60-71.
templates/admin/technical/dashboard.html.twig	Replace the two-button block with the new Sync button + Force update form.
translations/messages.en.xlf + messages.fr.xlf	Remove tcgdex_sync.insert_button / tcgdex_sync.update_button. Add tcgdex_sync.sync_button, tcgdex_sync.force_update_button, tcgdex_sync.force_update_set_label, tcgdex_sync.flash.queued.
config/services.yaml	New parameter app.tcgdex.host: 'https://api.tcgdex.net/v2' + app.tcgdex.locales: ['en', 'fr']. Bind to the three handlers and the hydrator.
docs/technicalities/tcgdex_sync.md	Rewrite the UI section. Update the line that claims "French translations are not available from the API" (it's wrong). Note that set-level updated_at diffing is deferred pending TCGdex API changes.

Reused code (avoid duplication)

• Set selector: lift the field config from src/Form/ArchetypeVariantFormType.php:60-71 (EntityType, the same query_builder callback into createExpandedSetsQueryBuilder, the same choice_label showing [PTCG code] — [localized name]). The repository method is already public.
• Cascade: the existing SyncTcgdexSeriesMessage → SyncTcgdexSerieMessage → SyncTcgdexSetMessage → SyncTcgdexCardMessage chain stays as-is. Only the per-card handler's inner behavior branches on SyncMode. The force-update path enters the cascade at SyncTcgdexSetMessage.
• JSON merging shape: reuse wrapEnglish / wrapAbilitiesOrAttacks / wrapSingleAbilityOrAttack from the hydrator, just parameterized by locale instead of hard-coding 'en'.

Gap-fill skip rule (the cost-saver)

In SyncTcgdexCardHandler for SyncMode::Sync, before any HTTP call:

```php
if ($card !== null && $this->hasAllConfiguredLocales($card)) {
return; // skipped — no HTTP, no merge, no flush
}
```

hasAllConfiguredLocales checks \$card->getName() contains a non-empty string for every locale in app.tcgdex.locales. Cheap, derived from the JSON column (no extra schema), and trivially correct: if French is missing, we probe; if French is present, we trust it.

The same check is not applied in SyncMode::ForceUpdate — that mode exists precisely to override the skip and re-fetch everything.

Verification

1. Pre-state: in dev DB, find a card with name->>'\$.fr' IS NULL.
2. Hit the new Sync button. Monitor the messenger workers (make worker.sync). Confirm cards with all locales populated produce no HTTP calls; cards missing fr get one EN probe + one FR fetch, and the JSON column now has both keys after the run.
3. Re-run Sync immediately — every card should be skipped (idempotent). Verify tcgdex_updated_at was populated on the cards touched in step 2 and matches the API's updated timestamp.
4. Pick a set in the Force update form and submit. Confirm every card in that set is re-fetched in every configured locale even if it was already complete.
5. make test — including a new TcgdexCardHydratorTest::testMergeLocaleFieldsPreservesOtherLocales that verifies merging fr does not clobber a pre-existing en.
6. make lint-i18n, make twig-cs-fix, make cs-fix, make phpstan.
7. symfony console c:c after deploy.

Deferred (explicitly out of scope)

• Using tcgdex_updated_at as a skip-decision input. The column is written now so the data accrues, but the gap-fill skip rule stays purely locale-completeness-based until TCGdex exposes a set-level updated_at. At that point the cascade gains an early-exit at the set handler and the per-card column becomes the second-level filter beneath it.
• A console command for cron — the Sync button is the trigger for now.

Dependencies

• F6.13 (incremental TCGdex sync — extended by this issue)
• F6.2 (card validation via TCGdex — provides the existing hydrator and message handlers)