Fix includeSubdomains dropping all subdomain URLs + broken cache-clear

[stretchcloud]

Bug 1 — includeSubdomains:true dropped every subdomain URL

POST /api/map on https://www.sonarsource.com with includeSubdomains:true returned 1,727 URLs, all on www. — even though discovery found 38k+ across docs./community./etc. (the sitemaps were pulled). The subdomains it discovered were then thrown away by the filter.

Cause: URLValidationUtils.matchesDomain compared each candidate against the full seed hostname. For a www.<domain> seed it kept only www.<domain> and *.www.<domain>, so docs.sonarsource.com failed both checks and was filtered out — discarding exactly what includeSubdomains exists to include.

Fix: match on the registrable domain. New URLValidationUtils.registrableDomain() reduces www.sonarsource.com and docs.sonarsource.com both to sonarsource.com (with a curated two-level-TLD set so docs.a.co.uk → a.co.uk, not co.uk — which would otherwise match every .co.uk site). The discovery service's baseDomainOf now delegates to the same helper, so the subdomains it finds and the subdomains the filter keeps can't disagree.

Bug 2 — cache/clear was a silent no-op

POST /api/map/cache/clear returned success:true but cleared nothing. Keys were url-discovery:<sha256> (URL folded into the hash), yet clearCache filtered keys with .includes(encodeURIComponent(url)) — which never appears in a hash. Fixed by embedding a normalized URL token in the key (url-discovery:<urlToken>:<optionsHash>) and clearing by that prefix; clearCache now returns the count and the controller reports keysCleared.

Verification (Docker, against the live sites)

	Before	After
www.sonarsource.com + includeSubdomains	1,727 (www only)	30,000 — community 19,921 / docs 8,350 / www 1,727 / events 2
cross-domain leakage	—	none (registrable boundary holds)
look-alikes (evilsonarsource.com, sonarsource.com.attacker.net)	—	correctly excluded
cache-clear round-trip	reported success, cleared 0	prime→fromCache:true→clear keysCleared:1→fromCache:false

tsc ✅ · eslint ✅ · 201 tests ✅ (186 + 15 new) · openapi:check ✅

🤖 Generated with Claude Code