seo hygiene P0: cache headers + robots dedup + case redirects + og:image - #1473
Merged
Conversation
Four verified crawl / SEO issues confirmed on prod (2026-07-26):
1. Sitemap + /api/citable served no s-maxage, every crawl re-ran the
loader chain (x-vercel-cache: MISS observed). Add explicit
Cache-Control on both via next.config.ts headers().
2. /404 emitted two identical <meta name="robots" content="noindex">
tags because not-found.tsx set { index: false } and Next merged
instead of replacing the layout's robots shape. Give not-found the
full IS_STAGING-style shape so Next replaces.
3. Mixed-case URLs served 200 instead of 308-lowercase
(/products/Alchemy, /Benchmarks/foo). Add case-normalising 308 at
the top of middleware; broaden matcher to cover every HTML route.
4. Bench pages with chain variants (aggregator-head-lag, perp-fees)
emitted 4-5 <meta property="og:image"> tags via generateImageMetadata
fanning out per-chain entries. Social scrapers picked
nondeterministically. Chain-scoped OG cards were never rendered on
the actual /[chain] pages anyway (they inherit the site-wide OG),
so the fan-out gave no upside. Return only the default entry.
Flotapponnier
added a commit
that referenced
this pull request
Jul 26, 2026
…age (#1473) (#1474) Four verified crawl / SEO issues confirmed on prod (2026-07-26): 1. Sitemap + /api/citable served no s-maxage, every crawl re-ran the loader chain (x-vercel-cache: MISS observed). Add explicit Cache-Control on both via next.config.ts headers(). 2. /404 emitted two identical <meta name="robots" content="noindex"> tags because not-found.tsx set { index: false } and Next merged instead of replacing the layout's robots shape. Give not-found the full IS_STAGING-style shape so Next replaces. 3. Mixed-case URLs served 200 instead of 308-lowercase (/products/Alchemy, /Benchmarks/foo). Add case-normalising 308 at the top of middleware; broaden matcher to cover every HTML route. 4. Bench pages with chain variants (aggregator-head-lag, perp-fees) emitted 4-5 <meta property="og:image"> tags via generateImageMetadata fanning out per-chain entries. Social scrapers picked nondeterministically. Chain-scoped OG cards were never rendered on the actual /[chain] pages anyway (they inherit the site-wide OG), so the fan-out gave no upside. Return only the default entry. Co-authored-by: Florent Tapponnier <contact@mobula.io>
2 tasks
Flotapponnier
added a commit
that referenced
this pull request
Jul 26, 2026
Next Metadata routes ignore next.config.ts headers() — the previous sitemap.ts left Next in charge of the response headers and hard-coded Cache-Control: public, max-age=0, must-revalidate on force-dynamic sitemaps. Every crawler hit re-ran the full loader chain (x-vercel-cache: MISS observed on prod 2026-07-26 after PR #1473 landed the config-level attempt). Confirmed by curl: my next.config /sitemap.xml rule was silently dropped. Move the builder logic to src/lib/sitemap-builder.ts and add a Route Handler at src/app/sitemap.xml/route.ts that returns a raw Response with: Content-Type: application/xml Cache-Control: public, s-maxage=3600, stale-while-revalidate=86400 Same builder logic, same buildFullSitemap + buildStaticFallback, just serialized to XML in the route and shipped with a real edge cache header. Also drop the now-redundant /sitemap.xml rule from next.config headers(). Co-authored-by: Florent Tapponnier <contact@mobula.io>
Flotapponnier
added a commit
that referenced
this pull request
Jul 26, 2026
Next Metadata routes ignore next.config.ts headers() — the previous sitemap.ts left Next in charge of the response headers and hard-coded Cache-Control: public, max-age=0, must-revalidate on force-dynamic sitemaps. Every crawler hit re-ran the full loader chain (x-vercel-cache: MISS observed on prod 2026-07-26 after PR #1473 landed the config-level attempt). Confirmed by curl: my next.config /sitemap.xml rule was silently dropped. Move the builder logic to src/lib/sitemap-builder.ts and add a Route Handler at src/app/sitemap.xml/route.ts that returns a raw Response with: Content-Type: application/xml Cache-Control: public, s-maxage=3600, stale-while-revalidate=86400 Same builder logic, same buildFullSitemap + buildStaticFallback, just serialized to XML in the route and shipped with a real edge cache header. Also drop the now-redundant /sitemap.xml rule from next.config headers(). Co-authored-by: Florent Tapponnier <contact@mobula.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Four verified crawl / SEO issues confirmed live on prod (2026-07-26):
Sitemap + /api/citable have no s-maxage — every crawler hit re-runs the loader chain (
x-vercel-cache: MISSobserved). Add explicitCache-Control: public, s-maxage=3600, stale-while-revalidate=86400via next.config.ts headers()./404 emitted 2 identical
<meta name="robots" content="noindex">tags — not-found.tsx set{ index: false }and Next merged instead of replacing the layout's robots shape. Give not-found the full IS_STAGING-style shape so Next replaces.Mixed-case URLs served 200 instead of 308-lowercase — /products/Alchemy + /Benchmarks/foo both 200. Add case-normalising 308 at the top of middleware; broaden matcher to cover every HTML route.
Bench pages with chain variants emitted 4-5 og:image tags — aggregator-head-lag has 5, perp-fees has 4. Social scrapers pick nondeterministically. Chain-scoped OG cards were never actually rendered on the /[chain] pages anyway (they inherit site-wide OG), so the fan-out gave no upside. Return only the default entry from generateImageMetadata.
Test plan