Fix /api/map timing out to a 500 with an empty list

[stretchcloud]

Problem

POST /api/map on https://www.sonarsource.com (with the default includeSubdomains:true) returned:

500  {"success":false, "error":"Discovery timeout after 30000ms", "data":{"links":[],"total":0,...}}

…while docs.sonarsource.com returned thousands of URLs. A slow-but-successful crawl was indistinguishable from a hard failure. Two bugs combined:

1. map.controller.ts used Promise.race([discovery, rejectAfter(timeoutMs)]). The moment the deadline passed, the race rejected and discarded every URL already found → 500 with an empty list.
2. The slow phases were unbounded. In discoverUrls, the internal AbortController was dead code (its signal was never wired to any fetch), and Phase 1.5 (subdomain sitemap expansion) + Phase 2 (browser crawl) were plain awaits. For a www.<domain> seed with includeSubdomains, subdomain expansion pulls each sibling's sitemap — and docs.sonarsource.com's huge sitemap alone blows the whole 30s budget. (That's exactly why bare docs. worked but www didn't.)

Fix — treat timeoutMs as a soft deadline, always return what was found

• New src/services/discovery-deadline.ts (pure, unit-tested): withDeadline(p, ms, fallback) resolves to fallback instead of rejecting; makeDeadline(totalMs, now?) tracks the remaining budget with an injectable clock.
• discoverUrls: Phase 1 methods now collect URLs as each settles (a hung sitemap can't erase the others) and are awaited only up to the deadline. Subdomain expansion and browser crawl run only while budget remains and are each bounded by withDeadline. The method always resolves with the accumulated, filtered URLs and sets partial: true when truncated.
• Controller drops the reject-race and surfaces partial in the response metadata.
• Partial results are not cached — a truncated list can't poison the cache for the full TTL.
• Default timeoutMs 30s → 60s. The deadline is only a ceiling (fast sites still return in 1–3s); this lets legitimately large sites finish before going partial. Preserves the existing speed tiering (sitemap-first; browser crawl only when sparse).

Verification (Docker, against the live sites)

Case	Before	After
www.sonarsource.com + includeSubdomains	500, 0 URLs	200, 100 URLs (37s, complete)
docs.sonarsource.com	200, big list	200, 200 URLs (unchanged)
www.fuel-finder.uk	200	200 (unchanged)
forced timeoutMs:2500	would 500	200 + partial:true, returns what it found, not cached
cached replay	—	0.006s fromCache:true (complete runs only)

tsc ✅ · eslint ✅ · 186 tests ✅ (180 + 6 new deadline tests) · openapi:check ✅

Note: withDeadline stops awaiting slow work; it doesn't abort the underlying fetches (there's no cross-cutting abort signal wired through discovery yet). They finish/time out on their own via axios/browser timeouts. Wiring a real AbortSignal end-to-end is a good follow-up but out of scope here.

🤖 Generated with Claude Code