Expand catalogue 140 → 5,162 + harden validator + Pages polish - #2
Merged
Merged
Conversation
nschmeller
force-pushed
the
claude/expand-th18-catalogue-acno3
branch
from
April 27, 2026 02:09
824d660 to
712cc9a
Compare
Per-entry errors (fatal)
E1-E4 URL shape, payload bytes, slot ∈ {1,2,3}, TH-vs-URL match.
E5 collection index < 1_000 (real ids observed: 0..93).
E6 HMAC-tag Shannon entropy ≥ 2.5 bits/byte (real HMAC output
is ~3.8-4.0; very low entropy = structured input).
E7 reject pathological tags (all-zero, all-one-byte, monotonic
ramp, ≥15/16 printable-ASCII bytes -- P ≈ 1e-6 from random).
E8 HMAC-tag uniqueness across the catalogue. A collision is
cryptographically impossible from real Supercell signatures.
E9 required fields present and well-typed.
E10 `id` and `link` unique across the catalogue.
E11 `type` ∈ {War, Trophy, Farm, Hybrid, Progress, Fun,
Home Village} -- 7-category whitelist.
E12 `image` required + YouTube/TikTok/Instagram thumbnail
blacklist (those host video frames or social-share cards,
not a base preview).
E13 `image` URL unique across the catalogue (was W4; promoted
because two cards on the same screenshot are misleading).
E14 normalised-link uniqueness (lang-prefix and trailing-slash
insensitive) -- catches the same blob committed twice with
`/en?` vs `/en/?`, which E10's exact-string check misses.
Per-entry warnings (--strict promotes to errors)
W1 lang prefix on observed Supercell whitelist.
W2 auto-generated id slug matches the upstream id encoded in
`builder` (cocbases layout id, basemelon design #).
W3 builder hostname matches the image-host association table.
Client-style liveness (--liveness, parallel)
L1 final HTTP status is 2xx.
L2 redirect chain preserves `TH<n>%3A<v>%3A<blob>` verbatim
(catches DNS hijacks that strip the query, bad rewriters).
L3 body contains both `clashofclans://` and `Clash of Clans` --
Supercell Next.js share-app fingerprint; a captive portal /
parked-domain page / DNS hijack will be missing one.
Auto-retries non-en URLs with `en/` to dodge Supercell's broken
Polish-locale stub.
Image probe (--check-images, parallel)
L4 Range-fetch first 1024 bytes; verify PNG/JPEG/WebP/GIF/SVG
magic bytes; extract dimensions; reject anything below 200x200
(placeholders, tracker pixels).
Performance + ergonomics
- Two-pool runner: liveness and image-check spawn as concurrent
threads with their own 16-worker host budgets, so a combined
pass takes max(t_live, t_img) instead of the sum.
- Progress logging every 100 entries / 2 seconds on stderr.
- --diff-against PATH: per-entry checks only run on entries new
in HEAD; catalogue-wide uniqueness still scans the full file.
Used by CI to keep PR latency at a few seconds even as the
catalogue grows.
- decode_link_payload(): fast tag-only decoder used for the
catalogue-wide E8 check on entries that are out of scope --
skips the per-entry validation work we'd discard anyway.
scripts/add-base.sh -- bound-check town_hall to 1..20 in-tool so
contributors get immediate feedback; the validator caught it later
but earlier is friendlier.
NB: none of L1-L4 prove a layout id is *cryptographically valid* --
that requires Supercell's private HMAC key, which only the in-app
handler verifies. Both link.clashofclans.com and
api.clashofclans.com/v1/ return identical responses for real and fake
ids (verified empirically). Structural E1-E14 + curation by upstream
sources is the strongest programmatic confidence we can offer.
Two workflows:
validate-bases.yml -- reusable, workflow_call only. Materialises
the previous bases.json from HEAD~1 (the synthetic merge commit's
target-branch parent on PRs; the previous main commit on push)
and runs the validator with --diff-against + --liveness +
--check-images + --strict-liveness. Per-entry checks (E1-E13,
W1-W3, L1-L4) only run on entries new in HEAD; catalogue-wide
uniqueness (E8/E10/E14) still scans the full file.
deploy.yml -- triggers on pull_request, push to main, and
workflow_dispatch. Three jobs:
validate-bases -- workflow_call into the reusable validator.
build -- depends on validate-bases. Trunk-builds
the WASM bundle and uploads the Pages
artifact.
deploy -- depends on build. Gated by
`if: github.ref == 'refs/heads/main'` so
a workflow_dispatch from a feature branch
doesn't accidentally publish half-finished
work.
Concurrency is per-ref with cancel-in-progress only on PRs, so
rapid PR commits supersede each other but main pushes never cancel
one another.
Python is pinned to 3.14.
Net effect: PRs run validate + build (so contributors see the same
checks main runs); only the main branch ever publishes to Pages.
A push that introduces a structurally-bad layout link, a duplicate
image, or a broken/non-base image fails fast before any Pages
artifact is built.
Typical PR run: ~2 seconds (only new entries get the L1-L4 probes).
Full audit on the 5_000-entry catalogue: ~6 minutes (Cloudfront
body downloads dominate).
22 with duplicate image URLs (E13)
cocbases occasionally serves the same generic preview for several
layout ids; blueprintcoc article-link entries share the article
hero image. We keep the first occurrence and drop the rest.
40 with broken images (L4)
- 34 cocbases entries pointing at media.oneclash.com URLs that
return HTTP 403 due to triple-encoded path segments
(e.g. `Clash%252520of%252520Clans_2023-09-05...`).
- 6 entries (chiraggamer7, gaming-with-akhil) using Flutter
relative-path assets like `Image/Townhall 16/b12.png` that
are not http(s) URLs and don't render in the browser.
Catalogue: 5_224 → 5_162. All remaining entries pass:
structural E1-E14, L1-L3 share-link liveness (with en/ retry
for the broken pl/ Supercell locale), L4 image probe (HTTP 2xx
+ valid magic bytes + ≥200x200 dimensions).
src/main.rs
- pagination (PAGE_SIZE=60 + "Show more"). The largest TH tab
has 535 cards; without this the default tab would render all
of them on first paint.
- bases wrapped in Rc so cloning the state handle each render
doesn't deep-copy a 5_000-entry vector.
- per-TH counts and the type list memoised with use_memo so
search keystrokes don't re-walk the catalogue.
- search.trim().to_lowercase() hoisted out of the per-base
filter closure.
- aria-label on search input + type select (placeholder isn't
a label). aria-live="polite" on the result count and copy
button so screen readers announce updates. Per-card
aria-label on Open / Copy actions.
- autocomplete="off" + spellcheck="false" on search (instant-
filter convention; builder names like "KLAWKLA" aren't
dictionary words and would get red squiggles).
favicon.svg + index.html
- Tiny inline SVG (orange-gradient crenellated wall). index.html
declares it via <link rel="icon" type="image/svg+xml"> and Trunk
copies it into dist/.
- Open Graph metadata for rich previews in Slack / Discord /
iMessage / WhatsApp / LinkedIn / Bluesky / Mastodon / GitHub
PR comments / etc.
- <meta name="theme-color"> matches the accent gradient.
- <noscript> fallback so JS-disabled visitors don't see a blank
page.
- <title> is no longer just "Clash Bases".
styles.css
- .load-more-wrap to host the pagination button.
- Footer link styling (blends with the existing palette).
- :focus-visible ring on tabs / buttons / footer link for
keyboard users.
- prefers-reduced-motion: drops hover-zoom and slide transforms
for users who've opted out at the OS level.
Cargo.toml
- rust-version bumped 1.85 → 1.95.
- Removed unused deps (serde_json, gloo-utils, js-sys) and unused
web-sys features (Location, UrlSearchParams).
- strip = "symbols" in [profile.release] to shave WASM bytes.
README.md
- Catalogue counts updated (5_617 → 5_162 after the prune;
per-TH histogram regenerated from current bases.json).
- Removed the per-source breakdown table -- it goes stale fast,
the validator's stdout is the source of truth.
- Field reference: `image` is now required (E12); `town_hall`
is 1..20; `type` is the 7-element whitelist (was a non-
exhaustive list including stale categories like "CWL" and
"Builder Hall").
- Rust prerequisite bumped 1.85 → 1.95 to match Cargo.toml.
.gitignore
- Replace the narrow `scripts/__pycache__/` with the standard
Python catchall `__pycache__/` + `*.py[co]`.
nschmeller
force-pushed
the
claude/expand-th18-catalogue-acno3
branch
from
April 27, 2026 02:32
e4aee96 to
6040e7d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Branch grew the catalogue from 140 → 5,162 entries across TH4–TH18, hardened the validator with cryptographic-shape checks + client-style probes, and added a few site-quality wins (pagination, favicon, repo footer link).
Catalogue
builderand links to a per-base preview image. 393 imageless entries removed and 62 fail-validation entries pruned (image-URL duplicates, broken oneclash CDN paths, Flutter relative-path assets).Validator (
scripts/validate-bases.py)Per-entry errors (fatal):
typewhitelist.imagerequired + YouTube/TikTok/Instagram thumbnail blacklist.imageURL unique across the catalogue.Catalogue-wide warnings (W1–W3) for lang-prefix whitelist, builder/id-slug agreement, image-host vs source agreement.
Client-style liveness (
--liveness):TH<n>%3A<v>%3A<blob>verbatim.clashofclans://andClash of Clans).enURLs withen/to dodge Supercell's broken Polish-locale stub.Image probe (
--check-images):Performance + ergonomics:
--diff-against PATH: only run per-entry checks on entries new in HEAD; catalogue-wide uniqueness still scans the full file.CI (
.github/workflows/validate-bases.yml)bases.jsonchanges (PRs and pushes tomain).git show <ref>:bases.json.--diff-against+--liveness+--check-images+--strict-liveness.The existing
deploy.ymlworkflow (push-to-main → GH Pages) was also tightened: dropped the--livenessflag from CI because the share endpoint serves byte-identical responses for real and fake ids in 2026, and the probe alone would take >1 hour serially on 5k entries.Site (Rust/Yew/WASM)
Rc<Vec<Base>>to stop deep-copying the catalogue on every state-handle clone.use_memoon derived collections (per-TH counts, type list).search.trim().to_lowercase()out of the per-base filter.serde_json,gloo-utils,js-sys, and unusedweb-sysfeatures.What we can't do
None of the L1-L4 probes prove a layout id is cryptographically valid — that requires Supercell's private HMAC key, which only the in-app handler verifies. Both
link.clashofclans.comandapi.clashofclans.com/v1/return identical responses for real and fake ids (verified empirically). Structural E1-E14 + curation by upstream sources is the strongest programmatic confidence the catalogue can offer; the validator's docstring says so.Test plan
python3 scripts/validate-bases.py bases.json→OK: 5162 base(s) validated structurally.python3 scripts/validate-bases.py bases.json --liveness --check-images→Liveness: 5162 alive / 0 stale.Images: 5162 ok / 0 bad.cargo check→ no warnings.validate-bases.ymlworkflow exercises on this PR.deploy.ymlworkflow rebuilds + deploys to Pages on merge.https://claude.ai/code/session_01HWFs7faLdxZP3sTpwbGeVz
Generated by Claude Code