experiment: single-language meta descriptions for de/fr/nl#189
Conversation
A/B test the multilingual-first vs single-language-first homepage meta description hypothesis. Full design + measurement schedule in docs/seo-ctr-tracking-2026-04-08.md (gitignored, local only). ## Why PR #187 shipped multilingual-first descriptions for the top 10 non-English homepages, e.g.: /es: "Juega a Wordle en más de 80 idiomas — del árabe al yoruba…" The reasoning was that Google had rewritten our **brand query** (`wordle global`) meta to use multilingual framing. I extrapolated from one data point (Google likes multilingual for the brand query) to a recommendation for all homepages. That extrapolation was unjustified — the brand query and language-specific queries have different intents. A user typing `wordle español` has signaled they want Spanish wordle, not a polyglot tool. Every successful single-language wordle competitor I curl-checked leads with single-language framing; none lead with multilingual: - Spanish SERP: La Palabra del Día, wordleespanol.org, santiagodebus.com — all single-language framing - German: SZ.de, Spiegel — all "Wordle auf Deutsch", no multilingual - French: SUTOM, others — all single-language Strong qualitative signal that single-language framing wins for language-specific queries, but I have zero quantitative evidence. ## Experiment design | Group | Languages | Style | |-------|-----------|-------| | Control | es, it, pt | Keep multilingual (current) | | Treatment (this PR) | de, fr, nl | Switch to single-language | | Untouched | ar, hr, fi, tr | Saturated/low-traffic — no signal | 3 vs 3, similar combined impression volume, both groups span Romance + Germanic for fair comparison. Saturated markets (FI 84% CTR, AR 21% CTR, HR 17% CTR) excluded because their CTRs are pinned at the ceiling and won't move regardless of snippet. ## Treatment descriptions (LLM, native review pending) de: Wordle auf Deutsch — täglich ein neues 5-Buchstaben-Wort, kostenlos, ohne Anmeldung. Unbegrenzter Modus, 8 Spielmodi, Wortdefinitionen nach jedem Spiel. (152 chars) fr: Wordle en français — un nouveau mot de 5 lettres chaque jour, gratuit, sans inscription. Mode illimité, 8 modes de jeu, définitions après chaque partie. (152 chars) nl: Wordle in het Nederlands — elke dag een nieuw woord van 5 letters, gratis, geen account. Onbeperkte modus, 8 spelmodi, woorddefinities na elke ronde. (149 chars) All under Google's ~160-char display cap. Each leads with the language name, mentions daily/free/no-account upfront, then closes with the differentiators (unlimited mode, 8 modes, definitions). ## Measurement schedule Apr 16 — first GSC check, verify Google reindexed Apr 23 — early CTR signal (2 weeks) **May 7 — primary measurement**, compare control vs treatment delta May 21 — final assessment, roll out winner or revert Decision rule: - Treatment lifts ≥0.5pp more than control on average → roll out single-language framing to all non-English homepages - Treatment lifts ≤0pp more than control → revert to multilingual - Mixed → investigate per-language ## Test plan - [x] pytest tests/test_language_config.py — 1,417 pass - [x] All 3 descriptions validated as JSON, under 160 chars - [ ] Post-deploy: curl prod to confirm new descriptions live - [ ] May 7: GSC export and CTR comparison
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughThree language configuration files were updated with region-specific Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~2 minutes Possibly related PRs
Poem
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary
A/B test the multilingual-first vs single-language-first homepage meta description hypothesis. Tests whether the multilingual framing PR #187 shipped is actually right for non-English homepages.
Background
PR #187 shipped multilingual-first descriptions for the top 10 non-English homepages, e.g.:
```
/es: "Juega a Wordle en más de 80 idiomas — del árabe al yoruba…"
```
The reasoning was that Google had rewritten our brand query (`wordle global`) meta to use multilingual framing. I extrapolated from one data point (Google likes multilingual for the brand query) to a recommendation for all homepages. That extrapolation was unjustified — the brand query and language-specific queries have different intents.
A user typing `wordle español` has signaled they want Spanish wordle, not a polyglot tool. Every successful single-language wordle competitor I curl-checked leads with single-language framing; none lead with multilingual:
Strong qualitative signal that single-language framing wins for language-specific queries, but I have zero quantitative evidence. Hence the experiment.
Experiment design
3 vs 3, similar combined impression volume, both groups span Romance + Germanic for fair comparison. Saturated markets excluded because their CTRs are pinned at the ceiling.
Treatment descriptions (LLM, native review pending)
```
de: Wordle auf Deutsch — täglich ein neues 5-Buchstaben-Wort, kostenlos, ohne
Anmeldung. Unbegrenzter Modus, 8 Spielmodi, Wortdefinitionen nach jedem
Spiel. (152 chars)
fr: Wordle en français — un nouveau mot de 5 lettres chaque jour, gratuit, sans
inscription. Mode illimité, 8 modes de jeu, définitions après chaque
partie. (152 chars)
nl: Wordle in het Nederlands — elke dag een nieuw woord van 5 letters, gratis,
geen account. Onbeperkte modus, 8 spelmodi, woorddefinities na elke
ronde. (149 chars)
```
All under Google's ~160-char display cap. Each leads with the language name, mentions daily/free/no-account upfront, then closes with the differentiators (unlimited mode, 8 modes, definitions).
Measurement schedule
Decision rule
Test plan
Full design + measurement notes in `docs/seo-ctr-tracking-2026-04-08.md` (gitignored, local only).
Summary by CodeRabbit