From 8c00e7952fabc8ddc43fd3e1b3690c38c308fbe8 Mon Sep 17 00:00:00 2001 From: Lukce Date: Fri, 13 Nov 2026 07:15:56 -0500 Subject: [PATCH] feat(skills): AEO skills that own every audit rule Twenty new agent skills so an agent can fix each AEO finding in the stack it is working in, not just detect it. Each one detects, decides, implements (per-stack references), and verifies. - aeo: router that triages an audit and hands off to owning skills - access: crawler access, server rendering, llms.txt, indexability - identity: organization entity, article/author, FAQ/QA, validation, product, local business - extractability: answer-first content, site structure, metadata, freshness, E-E-A-T trust signals - off-site + measurement: citation earning, visibility tracking, AI referral analytics, Bing coverage skills/rule-map.json assigns all 58 scoring rules to exactly one skill; skills-map.test.ts fails CI on an unmapped rule, a missing skill, or drift in the router's copy of the map. --- packages/scoring/src/skills-map.test.ts | 41 ++++ skills/README.md | 119 ++++++--- skills/aeo-ai-crawler-access/SKILL.md | 199 +++++++++++++++ .../aeo-ai-crawler-access/references/astro.md | 43 ++++ .../references/nextjs.md | 31 +++ .../references/static-html.md | 12 + .../references/wordpress.md | 33 +++ skills/aeo-ai-referral-analytics/SKILL.md | 89 +++++++ .../references/ga4-setup.md | 39 +++ skills/aeo-answer-first-content/SKILL.md | 147 +++++++++++ .../references/examples.md | 73 ++++++ skills/aeo-article-author-schema/SKILL.md | 181 ++++++++++++++ .../references/astro.md | 27 ++ .../references/nextjs.md | 44 ++++ .../references/static-html.md | 11 + .../references/wordpress.md | 30 +++ skills/aeo-citation-earning/SKILL.md | 118 +++++++++ skills/aeo-content-freshness/SKILL.md | 159 ++++++++++++ skills/aeo-eeat-trust-signals/SKILL.md | 105 ++++++++ skills/aeo-faq-qa-schema/SKILL.md | 195 +++++++++++++++ skills/aeo-faq-qa-schema/references/astro.md | 20 ++ skills/aeo-faq-qa-schema/references/nextjs.md | 32 +++ .../references/static-html.md | 18 ++ .../aeo-faq-qa-schema/references/wordpress.md | 17 ++ skills/aeo-indexability/SKILL.md | 171 +++++++++++++ skills/aeo-indexability/references/astro.md | 37 +++ skills/aeo-indexability/references/nextjs.md | 51 ++++ .../references/static-html.md | 41 ++++ .../aeo-indexability/references/wordpress.md | 33 +++ skills/aeo-llms-txt/SKILL.md | 151 ++++++++++++ skills/aeo-llms-txt/references/astro.md | 37 +++ skills/aeo-llms-txt/references/nextjs.md | 42 ++++ skills/aeo-llms-txt/references/static-html.md | 41 ++++ skills/aeo-local-business-schema/SKILL.md | 130 ++++++++++ .../references/static-html.md | 35 +++ .../references/wordpress.md | 13 + skills/aeo-organization-entity/SKILL.md | 231 ++++++++++++++++++ .../references/astro.md | 28 +++ .../references/nextjs.md | 66 +++++ .../references/static-html.md | 16 ++ .../references/wordpress.md | 40 +++ skills/aeo-page-metadata/SKILL.md | 155 ++++++++++++ skills/aeo-page-metadata/references/astro.md | 55 +++++ skills/aeo-page-metadata/references/nextjs.md | 76 ++++++ .../references/static-html.md | 42 ++++ .../aeo-page-metadata/references/wordpress.md | 47 ++++ skills/aeo-product-schema/SKILL.md | 130 ++++++++++ .../aeo-product-schema/references/nextjs.md | 43 ++++ .../aeo-product-schema/references/shopify.md | 27 ++ .../references/woocommerce.md | 22 ++ skills/aeo-server-rendering/SKILL.md | 125 ++++++++++ .../aeo-server-rendering/references/astro.md | 34 +++ .../aeo-server-rendering/references/nextjs.md | 53 ++++ .../references/static-html.md | 34 +++ skills/aeo-site-structure/SKILL.md | 172 +++++++++++++ .../aeo-structured-data-validation/SKILL.md | 192 +++++++++++++++ .../references/nextjs.md | 15 ++ .../references/wordpress.md | 33 +++ skills/aeo-visibility-tracking/SKILL.md | 105 ++++++++ skills/aeo/SKILL.md | 120 +++++++++ skills/aeo/references/rules.md | 64 +++++ skills/rule-map.json | 63 +++++ skills/seo-bing-coverage/SKILL.md | 95 +++++++ 63 files changed, 4617 insertions(+), 31 deletions(-) create mode 100644 packages/scoring/src/skills-map.test.ts create mode 100644 skills/aeo-ai-crawler-access/SKILL.md create mode 100644 skills/aeo-ai-crawler-access/references/astro.md create mode 100644 skills/aeo-ai-crawler-access/references/nextjs.md create mode 100644 skills/aeo-ai-crawler-access/references/static-html.md create mode 100644 skills/aeo-ai-crawler-access/references/wordpress.md create mode 100644 skills/aeo-ai-referral-analytics/SKILL.md create mode 100644 skills/aeo-ai-referral-analytics/references/ga4-setup.md create mode 100644 skills/aeo-answer-first-content/SKILL.md create mode 100644 skills/aeo-answer-first-content/references/examples.md create mode 100644 skills/aeo-article-author-schema/SKILL.md create mode 100644 skills/aeo-article-author-schema/references/astro.md create mode 100644 skills/aeo-article-author-schema/references/nextjs.md create mode 100644 skills/aeo-article-author-schema/references/static-html.md create mode 100644 skills/aeo-article-author-schema/references/wordpress.md create mode 100644 skills/aeo-citation-earning/SKILL.md create mode 100644 skills/aeo-content-freshness/SKILL.md create mode 100644 skills/aeo-eeat-trust-signals/SKILL.md create mode 100644 skills/aeo-faq-qa-schema/SKILL.md create mode 100644 skills/aeo-faq-qa-schema/references/astro.md create mode 100644 skills/aeo-faq-qa-schema/references/nextjs.md create mode 100644 skills/aeo-faq-qa-schema/references/static-html.md create mode 100644 skills/aeo-faq-qa-schema/references/wordpress.md create mode 100644 skills/aeo-indexability/SKILL.md create mode 100644 skills/aeo-indexability/references/astro.md create mode 100644 skills/aeo-indexability/references/nextjs.md create mode 100644 skills/aeo-indexability/references/static-html.md create mode 100644 skills/aeo-indexability/references/wordpress.md create mode 100644 skills/aeo-llms-txt/SKILL.md create mode 100644 skills/aeo-llms-txt/references/astro.md create mode 100644 skills/aeo-llms-txt/references/nextjs.md create mode 100644 skills/aeo-llms-txt/references/static-html.md create mode 100644 skills/aeo-local-business-schema/SKILL.md create mode 100644 skills/aeo-local-business-schema/references/static-html.md create mode 100644 skills/aeo-local-business-schema/references/wordpress.md create mode 100644 skills/aeo-organization-entity/SKILL.md create mode 100644 skills/aeo-organization-entity/references/astro.md create mode 100644 skills/aeo-organization-entity/references/nextjs.md create mode 100644 skills/aeo-organization-entity/references/static-html.md create mode 100644 skills/aeo-organization-entity/references/wordpress.md create mode 100644 skills/aeo-page-metadata/SKILL.md create mode 100644 skills/aeo-page-metadata/references/astro.md create mode 100644 skills/aeo-page-metadata/references/nextjs.md create mode 100644 skills/aeo-page-metadata/references/static-html.md create mode 100644 skills/aeo-page-metadata/references/wordpress.md create mode 100644 skills/aeo-product-schema/SKILL.md create mode 100644 skills/aeo-product-schema/references/nextjs.md create mode 100644 skills/aeo-product-schema/references/shopify.md create mode 100644 skills/aeo-product-schema/references/woocommerce.md create mode 100644 skills/aeo-server-rendering/SKILL.md create mode 100644 skills/aeo-server-rendering/references/astro.md create mode 100644 skills/aeo-server-rendering/references/nextjs.md create mode 100644 skills/aeo-server-rendering/references/static-html.md create mode 100644 skills/aeo-site-structure/SKILL.md create mode 100644 skills/aeo-structured-data-validation/SKILL.md create mode 100644 skills/aeo-structured-data-validation/references/nextjs.md create mode 100644 skills/aeo-structured-data-validation/references/wordpress.md create mode 100644 skills/aeo-visibility-tracking/SKILL.md create mode 100644 skills/aeo/SKILL.md create mode 100644 skills/aeo/references/rules.md create mode 100644 skills/rule-map.json create mode 100644 skills/seo-bing-coverage/SKILL.md diff --git a/packages/scoring/src/skills-map.test.ts b/packages/scoring/src/skills-map.test.ts new file mode 100644 index 0000000..f49acd3 --- /dev/null +++ b/packages/scoring/src/skills-map.test.ts @@ -0,0 +1,41 @@ +/** + * Every rule the engine can fail must have exactly one skill that teaches an agent how to fix it. + * `skills/rule-map.json` is that mapping; this test keeps it honest in both directions, so adding a + * rule without a skill (or renaming a skill directory) fails CI instead of leaving a dead end. + */ +import { existsSync, readFileSync } from 'node:fs'; +import { fileURLToPath } from 'node:url'; +import { describe, expect, it } from 'vitest'; +import { technicalSeoRules } from './technical-seo-rules.js'; +import { aeoRules } from './aeo-rules.js'; +import { eeatSignalDefs } from './eeat-rules.js'; + +const SKILLS_DIR = fileURLToPath(new URL('../../../skills/', import.meta.url)); +const { rules: ruleMap } = JSON.parse(readFileSync(`${SKILLS_DIR}rule-map.json`, 'utf8')) as { + rules: Record; +}; +const ruleIds = [...technicalSeoRules, ...aeoRules, ...eeatSignalDefs].map((r) => r.id); + +describe('skills/rule-map.json', () => { + it('maps every rule id', () => { + expect(ruleIds.filter((id) => !(id in ruleMap))).toEqual([]); + }); + + it('maps no rule id that does not exist', () => { + expect(Object.keys(ruleMap).filter((id) => !ruleIds.includes(id))).toEqual([]); + }); + + it('points every rule at a skill that exists', () => { + const missing = [...new Set(Object.values(ruleMap))].filter( + (skill) => !existsSync(`${SKILLS_DIR}${skill}/SKILL.md`), + ); + expect(missing).toEqual([]); + }); + + // The router skill ships its own copy so it works when copied alone; it must not drift. + it('is mirrored exactly in the aeo router skill', () => { + const table = readFileSync(`${SKILLS_DIR}aeo/references/rules.md`, 'utf8'); + const rows = [...table.matchAll(/^\| `([^`]+)` \| \[`([^`]+)`\]/gm)].map(([, id, skill]) => [id, skill]); + expect(Object.fromEntries(rows)).toEqual(ruleMap); + }); +}); diff --git a/skills/README.md b/skills/README.md index 53f8b96..eed7e09 100644 --- a/skills/README.md +++ b/skills/README.md @@ -1,19 +1,62 @@ # AEO Toolkit skills -Claude Skills that turn the [`aeo-search` MCP server](https://aeo.advancelabs.dev/mcp) from a set -of tools into a set of **workflows**. +Claude Skills that give an agent the full method for Answer Engine Optimization: how to make a site +reachable, understandable and citable by ChatGPT, Claude, Perplexity, Gemini, Google AI Overviews +and Copilot, implemented in the stack the agent is actually working in. -The MCP server answers questions like "what are the numbers". These skills answer the -questions people actually ask: *why did traffic drop*, *which pages compete with each other*, -*what should I refresh*. The judgement in each skill — which readings are supportable, which -are guesses, when to say "I don't know" — is the part that does not fit in a tool description. +The MCP servers answer **"what is wrong, and how is it performing?"** These skills answer **"how +do I fix it here, and how do I know it worked?"** Each one follows the same loop: **detect** with +a rule or tool, **decide** whether it is really a problem, **implement** with per-stack recipes, +then **verify** with the same check. -| Skill | Answers | Tools used | +Start with [`aeo`](aeo/SKILL.md). It reads an audit, orders the work and hands each failing rule +to its owner. + +## Every audit rule has an owner + +The scoring engine has 58 rules. [`rule-map.json`](rule-map.json) assigns each one to exactly one +skill, and a test in `packages/scoring` fails CI if any of these drift: +- a rule exists without an owning skill +- a mapped skill does not exist +- the router's copy of the map diverges + +## The skills + +| Skill | Fixes | Uses | |---|---|---| -| [`seo-traffic-drop`](seo-traffic-drop/SKILL.md) | Traffic dropped. What caused it? | `gsc_traffic_drop`, `list_gsc_sites` | -| [`seo-cannibalization`](seo-cannibalization/SKILL.md) | Are my own pages competing? | `gsc_cannibalization`, `list_gsc_sites` | -| [`seo-content-decay`](seo-content-decay/SKILL.md) | What content needs refreshing? | `gsc_decay`, `gsc_traffic_drop`, `list_gsc_sites` | -| [`seo-high-intent-ctr`](seo-high-intent-ctr/SKILL.md) | Which buyer-intent searches do I rank for but lose the click on? | `gsc_search_analytics`, `gsc_ctr_gaps`, `list_gsc_sites` | +| [`aeo`](aeo/SKILL.md) | Router: audit → triage → hand off | `analyze_website_aeo` | +| **Access** | | | +| [`aeo-ai-crawler-access`](aeo-ai-crawler-access/SKILL.md) | AI bots blocked by robots.txt or a CDN; training vs search bots | `analyze_website_aeo` | +| [`aeo-server-rendering`](aeo-server-rendering/SKILL.md) | Content that only exists after JavaScript runs | `analyze_website_aeo` | +| [`aeo-llms-txt`](aeo-llms-txt/SKILL.md) | llms.txt, served without cannibalizing your pages | `analyze_website_aeo` | +| [`aeo-indexability`](aeo-indexability/SKILL.md) | Sitemaps, canonicals, noindex, redirects, hreflang, HTTPS | `gsc_url_inspect`, `engine_divergence` | +| **Identity** | | | +| [`aeo-organization-entity`](aeo-organization-entity/SKILL.md) | Organization JSON-LD, `@id` graph, consistent identity | `analyze_website_aeo` | +| [`aeo-article-author-schema`](aeo-article-author-schema/SKILL.md) | Article + Person author markup | `analyze_website_aeo` | +| [`aeo-faq-qa-schema`](aeo-faq-qa-schema/SKILL.md) | FAQPage / QAPage / HowTo, honestly scoped | `analyze_website_aeo` | +| [`aeo-structured-data-validation`](aeo-structured-data-validation/SKILL.md) | Invalid or conflicting structured data | `analyze_website_aeo` | +| [`aeo-product-schema`](aeo-product-schema/SKILL.md) | Product / Offer / Review for stores | `analyze_website_aeo` | +| [`aeo-local-business-schema`](aeo-local-business-schema/SKILL.md) | LocalBusiness, NAP consistency, Business Profile | `check_ai_visibility` | +| **Extractability** | | | +| [`aeo-answer-first-content`](aeo-answer-first-content/SKILL.md) | Question headings, answer-first paragraphs, depth | `discover_ranking_prompts` | +| [`aeo-site-structure`](aeo-site-structure/SKILL.md) | Breadcrumbs and internal linking | `analyze_website_aeo` | +| [`aeo-page-metadata`](aeo-page-metadata/SKILL.md) | Titles, descriptions, Open Graph, lang, viewport, alt text | `gsc_ctr_gaps` | +| [`aeo-content-freshness`](aeo-content-freshness/SKILL.md) | Honest dates across page, JSON-LD and sitemap | `gsc_decay` | +| [`aeo-eeat-trust-signals`](aeo-eeat-trust-signals/SKILL.md) | About, contact, policies, references | `analyze_website_aeo` | +| **Off-site** | | | +| [`aeo-citation-earning`](aeo-citation-earning/SKILL.md) | Where competitors get cited, mentions, outreach | backlink tools | +| **Measurement** | | | +| [`aeo-visibility-tracking`](aeo-visibility-tracking/SKILL.md) | Prompt sets, baselines, Perplexity citation share | visibility tools | +| [`aeo-ai-referral-analytics`](aeo-ai-referral-analytics/SKILL.md) | AI-assistant referral traffic in GA4 | `ga4_run_report` | +| [`seo-bing-coverage`](seo-bing-coverage/SKILL.md) | Google vs Bing gaps | `engine_divergence`, `bing_*` | +| [`seo-traffic-drop`](seo-traffic-drop/SKILL.md) | Traffic dropped. What caused it? | `gsc_traffic_drop` | +| [`seo-cannibalization`](seo-cannibalization/SKILL.md) | Are my own pages competing? | `gsc_cannibalization` | +| [`seo-content-decay`](seo-content-decay/SKILL.md) | What content needs refreshing? | `gsc_decay` | +| [`seo-high-intent-ctr`](seo-high-intent-ctr/SKILL.md) | Buyer-intent searches that rank but lose the click | `gsc_ctr_gaps` | + +Implementation recipes live in each skill's `references/` (Next.js, Astro, WordPress, static HTML, +and Shopify/WooCommerce where relevant). The agent only loads the one for the stack in front of +it. ## Install @@ -22,7 +65,7 @@ Skills are plain directories. Copy the ones you want into your skills folder: ```bash # Claude Code, for one project mkdir -p .claude/skills -cp -r path/to/aeo-toolkit/skills/seo-traffic-drop .claude/skills/ +cp -r path/to/aeo-toolkit/skills/aeo* .claude/skills/ # or for every project cp -r path/to/aeo-toolkit/skills/* ~/.claude/skills/ @@ -31,15 +74,20 @@ cp -r path/to/aeo-toolkit/skills/* ~/.claude/skills/ They activate on their own when a request matches the `description` in the frontmatter. You do not need to name the skill. -## They need the MCP server +## MCP servers: helpful, mostly optional -A skill is instructions, not data. Every one of these calls the `aeo-search` MCP server, which -holds the Google OAuth connection to your Search Console account. Without it the skill has -nothing to read. +The `aeo-*` skills work without any MCP server. Each **Detect** step has a no-MCP fallback, for +example `curl` or view-source. Connecting the servers makes detection and verification faster +and more complete. The `seo-*` and measurement skills read Search Console, GA4, Bing or +Perplexity data, and they need the matching server. -Connect it from [aeo.advancelabs.dev/mcp](https://aeo.advancelabs.dev/mcp). You will need -Search Console access for the property you want to analyse; the server handles the OAuth flow -and stores tokens encrypted. +| Server | Endpoint | Auth | +|---|---|---| +| `aeo-visibility` | `https://aeo.advancelabs.dev/api/mcp/ai-visibility/mcp` | none | +| `aeo-backlink` | `https://aeo.advancelabs.dev/api/mcp/backlink/mcp` | none | +| `aeo-search` | `https://aeo.advancelabs.dev/api/mcp/search/mcp` | Google sign-in; Bing key optional | + +Setup for each client is at [advancelabs.dev/mcp](https://advancelabs.dev/mcp). > **On "no API keys" claims.** Search Console requires OAuth. Anything promising GSC data with > no authentication is either reading an already-authenticated local session or not reading @@ -47,19 +95,28 @@ and stores tokens encrypted. ## What these will not do -- **They cannot see your competitors' data.** Search Console reports your property only. Any - competitor comparison needs a third-party rank-tracking source, which these do not use. -- **They cannot separate seasonality from decline.** GSC has no notion of why demand moved. - Each skill says so where it matters rather than implying more certainty than the data - supports. -- **They do not change anything.** Every tool is read-only. +- **They don't promise citations.** A fix removes a known blocker. Whether an engine then cites + you is measured, not assumed. +- **The visibility tools see one engine.** `check_ai_visibility` and its siblings query + Perplexity, so their readings are Perplexity citations, not "AI" in general. +- **They can't see competitors' Search Console data.** Competitor comparisons use public + signals (citations, links) only. +- **They don't change anything remotely.** Every MCP tool is read-only. Code changes happen in + your repo, where you can review them. ## Contributing a skill -Same bar as the rest of the repo: a skill should encode judgement a tool description cannot, -be explicit about what the data cannot answer, and never assert a cause it cannot evidence. +Same bar as the rest of the repo: +- encode judgement a tool description cannot hold +- be explicit about what the data cannot answer +- never assert a cause you can't evidence +- date every volatile fact (bot names, Google policies) in a `## Sources` section, against a + primary source + +The frontmatter `description` decides whether the skill activates, so write it as *when to use +this* (including the phrase "Use when"), not as a title. -The frontmatter `description` is what decides whether the skill activates, so write it as -*when to use this*, not as a title. A test in the private platform repo reads these skills, validates the -frontmatter and checks that every tool a skill mentions is registered on the server, so a -renamed tool fails CI there instead of silently breaking the skill. +Two tests keep the skills honest: +- `packages/scoring/src/skills-map.test.ts` here checks rule ownership. +- A test in the private platform repo checks frontmatter, and checks that every MCP tool a skill + names is registered on one of the three servers. diff --git a/skills/aeo-ai-crawler-access/SKILL.md b/skills/aeo-ai-crawler-access/SKILL.md new file mode 100644 index 0000000..1c0accd --- /dev/null +++ b/skills/aeo-ai-crawler-access/SKILL.md @@ -0,0 +1,199 @@ +--- +name: aeo-ai-crawler-access +description: Check and fix whether AI crawlers can reach a site, and decide which ones to allow. Use when someone asks "why isn't ChatGPT or Perplexity citing my site", "should I block GPTBot", "allow AI crawlers", "AI bots blocked", "robots.txt for AI", "Cloudflare is blocking AI bots", or when the audit fails aeo.ai-bots-allowed or tech.robots-present. Covers training bots vs search and user-fetch bots, robots.txt, and CDN/WAF blocking that overrides robots.txt. Uses the aeo-visibility MCP server if connected; works without it. +--- + +# AI crawler access + +Answer engines can only cite what their bots can fetch. This skill checks robots.txt and the +CDN in front of it, then helps choose which AI bots to allow. Blocking AI *training* while +allowing AI *search* is a legitimate choice, and the audit rules do not always know that. + +## When this matters / when it doesn't + +It matters when a site wants to appear in ChatGPT search, Claude, Perplexity, Gemini or +Google AI Overviews and has never looked at its robots.txt or its CDN bot settings. + +It does **not** matter much, and the finding may be a deliberate choice, when: + +- The owner has decided not to be used for model training. Blocking `GPTBot` or `ClaudeBot` + is a policy decision, not a defect. **Ask** before recommending they undo it. +- The site is private, paywalled or intentionally unlisted. +- Only the training bots are blocked and the search/retrieval bots are allowed. That site + can still be cited. The audit rule may still fail it (see below). Say so plainly. + +## Detect + +Rules owned: + +- `tech.robots-present`: passes when `/robots.txt` answers 2xx with a non-HTML content type. + A 404, or a catch-all that returns the app shell as `text/html`, fails. +- `aeo.ai-bots-allowed`: fails when robots.txt disallows `/` for any of **GPTBot, ClaudeBot, + PerplexityBot, OAI-SearchBot or Google-Extended**. No robots.txt means nothing is blocked + and the rule passes. Note the list mixes training bots (GPTBot, ClaudeBot, Google-Extended) + with search bots (OAI-SearchBot, PerplexityBot). A site that blocks only the training bots on + purpose will fail this rule. Report it as "intentional, rule is blunt", not as a bug. + +With the MCP server: call `analyze_website_aeo` on the site root. The result lists AI-bot +access per bot. + +Without it: + +```bash +curl -sI https://example.com/robots.txt | head -5 # 200 and not text/html +curl -s https://example.com/robots.txt +# Does the CDN treat bots differently from robots.txt? Compare status codes: +curl -s -o /dev/null -w "%{http_code}\n" -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot" https://example.com/ +curl -s -o /dev/null -w "%{http_code}\n" -A "Mozilla/5.0 (compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)" https://example.com/ +``` + +A 403, 429 or a challenge page for a bot while a normal browser gets 200 means something in +front of the site is blocking it, whatever robots.txt says. Caveat: a CDN that verifies bots +by IP range will let the real bot through and block your spoofed curl, so a `200` here proves +little and a `403` is a hint, not proof. Server logs and the CDN's bot analytics are the real +evidence. + +## Decide + +Each vendor documents separate user agents for separate jobs. Verified against their own +pages on 2026-11-13. + +| Operator | User agent | Job | Blocking it means | +|---|---|---|---| +| OpenAI | `OAI-SearchBot` | Surfaces sites in ChatGPT search | Not shown in ChatGPT search answers (can still appear as plain links) | +| OpenAI | `GPTBot` | Crawls content that may train foundation models | Opt-out of training | +| OpenAI | `ChatGPT-User` | Fetches a page when a user asks ChatGPT or a Custom GPT | OpenAI says robots.txt rules may not apply to it | +| OpenAI | `OAI-AdsBot` | Checks landing pages of ads submitted to ChatGPT | Only matters if you advertise there | +| Anthropic | `ClaudeBot` | Collects content that may contribute to training | Opt-out of training | +| Anthropic | `Claude-SearchBot` | Indexes content to improve search results | May reduce visibility in Claude search answers | +| Anthropic | `Claude-User` | Fetches when a user asks Claude a question | May reduce visibility for user-directed web search | +| Perplexity | `PerplexityBot` | Surfaces sites in Perplexity search; not used for foundation-model training | Not shown in Perplexity results | +| Perplexity | `Perplexity-User` | Fetches when a user asks | Perplexity says it generally ignores robots.txt | +| Google | `Googlebot` | Search, and therefore AI Overviews and AI Mode | Leaves Search entirely | +| Google | `Google-Extended` | Product token only, no separate fetch. Controls use for Gemini training and for grounding in Gemini Apps / Vertex | Does not affect Search inclusion or ranking | +| Apple | `Applebot` | Powers Spotlight, Siri, Safari search; data may also train Apple models | Leaves Apple search | +| Apple | `Applebot-Extended` | Opt-out token for training Apple foundation models | Opt-out of training; Applebot can still crawl | + +Things this table settles that people get wrong: + +- **There is no robots.txt switch for Google AI Overviews.** Google says `Googlebot` is the + control, and `nosnippet`, `data-nosnippet`, `max-snippet` or `noindex` limit what is shown. + `Google-Extended` does not remove you from AI Overviews. +- **User-fetch bots (`ChatGPT-User`, `Perplexity-User`, `Claude-User`) act for a person.** + OpenAI and Perplexity both say robots.txt may not govern them. Do not expect a `Disallow` + to stop them, and do not call a hit from one a violation. +- OpenAI and Perplexity both say robots.txt changes can take up to about 24 hours to take effect. Anthropic gives no number. + +Recommended defaults, offered as options, not orders: + +1. **Wants to be cited** (most sites): allow all the search and user-fetch bots. Training bots are + a separate choice for the owner. +2. **Wants no training but wants citations**: block `GPTBot`, `ClaudeBot`, `Google-Extended`, + `Applebot-Extended` (and optionally `CCBot`, Common Crawl's crawler; its docs were not + re-checked for this skill). Allow `OAI-SearchBot`, `Claude-SearchBot`, `PerplexityBot`, + `Googlebot`, `Bingbot`. Expect `aeo.ai-bots-allowed` to fail for this site and say that the + failure is the chosen trade-off. +3. **Wants out of AI entirely**: block everything above, and understand that this removes the + site from AI citations. Confirm that is the intent. + +Anthropic also says that since 2026-10-30 it honours a machine-readable "no AI training" +preference signal based on the IETF aipref draft, and that a robots.txt rule naming `ClaudeBot` +takes priority over the general signal. Google says it intends to support the same shared +signal in early 2027. The standard is still a draft, so treat it as extra, not a replacement +for naming the bots. + +## Implement + +Shape of the fix, any stack: + +1. Serve a real `/robots.txt` (text/plain, 200). Reference the sitemap. +2. Write explicit groups for the bots the decision above names. A group naming a bot replaces + the `User-agent: *` group for that bot, so repeat any `Disallow` paths you still want. +3. Check the layer in front of the origin (CDN, WAF, host "bot protection", plugins). + +``` +User-agent: * +Allow: / + +# Allowed: search and retrieval +User-agent: OAI-SearchBot +Allow: / +User-agent: Claude-SearchBot +Allow: / +User-agent: PerplexityBot +Allow: / + +# Opt out of training only (delete this block if the owner is happy to be trained on) +User-agent: GPTBot +Disallow: / +User-agent: ClaudeBot +Disallow: / +User-agent: Google-Extended +Disallow: / +User-agent: Applebot-Extended +Disallow: / + +Sitemap: https://example.com/sitemap.xml +``` + +**The CDN overrides robots.txt.** robots.txt is a request; a WAF rule is enforcement. If a bot +gets a 403 or a challenge at the edge it never reads robots.txt. Anthropic says as much: +blocking its IP ranges can stop it reading robots.txt at all, so an opt-out should be done in +robots.txt, not by IP. Common culprits: + +- **Cloudflare managed robots.txt / AI Crawl Control.** Setting the AI *Training* policy to + Disallow adds managed `Disallow` lines (Amazonbot, Applebot-Extended, Bytespider, CCBot, + ClaudeBot, Google-Extended, GPTBot, meta-externalagent) and, with Bot Preference Sync on, + enables blocking at the edge for AI training crawlers. Cloudflare says mixed-use crawlers are + excluded from the edge block, and the Search and Agent policies add their own directives when + set to Block. If the site has no robots.txt, Cloudflare serves one with those rules. Check + Security settings, Bot traffic, "AI Crawler and Agent access policies". +- A custom WAF rule, rate limit or "Bot Fight Mode" that challenges unknown user agents. +- Host-level firewall plugins (WordPress security plugins are the usual source). + +If you want a bot through a WAF, allow it by **user agent plus published IP range**, not user +agent alone, because user agents are trivially forged. OpenAI publishes ranges at +`openai.com/searchbot.json`, `gptbot.json`, `chatgpt-user.json`; Perplexity at +`perplexity.ai/perplexitybot.json` and `perplexity-user.json`; Anthropic links its list from +the crawler help page. Perplexity's guide walks through a Cloudflare Skip rule ordered above any +blocking rule. + +Per-stack file recipes: read `references/nextjs.md`, `references/astro.md`, +`references/wordpress.md` or `references/static-html.md`. + +## Verify + +1. `curl -s https://example.com/robots.txt` shows the groups you intended, as text/plain. +2. Re-run `analyze_website_aeo`. `tech.robots-present` should pass. `aeo.ai-bots-allowed` + should pass if you allowed all five key bots; if you chose option 2 or 3 it will fail + by design, and the report should say which bots are blocked on purpose. +3. Check the edge: look in the CDN's bot analytics or server logs for 403/429 on the search + bots' user agents over the last few days. No blocks there is the real pass. +4. Allow a day for the bots to pick up the change. + +## Pitfalls + +- Blocking `Google-Extended` and expecting to leave AI Overviews. It does not. +- Adding `User-agent: *` / `Disallow: /` "temporarily" on a staging config that ships to + production. Check the live file after every deploy. +- A catch-all route that returns the app shell for `/robots.txt`. The audit (and some + crawlers) treat an HTML response as "no robots.txt". +- Allowing a bot in robots.txt and leaving a Cloudflare AI policy on Block. Both layers must agree. +- Treating `ChatGPT-User` or `Perplexity-User` traffic as crawling. It is a person asking. +- `Disallow:` with an empty value allows everything; `Disallow: /` blocks everything. One + character, opposite meanings. +- Legacy names (`Claude-Web`, `anthropic-ai`) still appear in some lists and the toolkit still probes + them, but Anthropic's current help page lists only ClaudeBot, Claude-User and Claude-SearchBot. +- Naming a bot in a group with no rules does nothing useful. Give it `Allow: /` or `Disallow: /`. + +## Sources + +- OpenAI crawlers: https://platform.openai.com/docs/bots (verified 2026-11-13) +- Anthropic crawlers: https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler (verified 2026-11-13) +- Perplexity crawlers and WAF guide: https://docs.perplexity.ai/guides/bots (verified 2026-11-13) +- Google common crawlers (Google-Extended): https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers (verified 2026-11-13) +- Google AI features and your website: https://developers.google.com/search/docs/appearance/ai-features (verified 2026-11-13) +- Apple Applebot: https://support.apple.com/en-us/119829 (verified 2026-11-13) +- Cloudflare managed robots.txt: https://developers.cloudflare.com/bots/additional-configurations/managed-robots-txt/ (verified 2026-11-13) +- robots.txt introduction (Google): https://developers.google.com/search/docs/crawling-indexing/robots/intro (verified 2026-11-13) +- Common Crawl's CCBot page was not fetched; its role is stated from general knowledge and not verified. diff --git a/skills/aeo-ai-crawler-access/references/astro.md b/skills/aeo-ai-crawler-access/references/astro.md new file mode 100644 index 0000000..f5f1bf0 --- /dev/null +++ b/skills/aeo-ai-crawler-access/references/astro.md @@ -0,0 +1,43 @@ +# Astro 5 + +Static file: put `robots.txt` in `public/`. For a dynamic file (to build the sitemap URL from +`site`), use an endpoint. Astro calls `GET` at build time in static mode and writes the body. + +```ts +// src/pages/robots.txt.ts +import type { APIRoute } from 'astro' + +export const GET: APIRoute = ({ site }) => { + const sitemap = new URL('sitemap-index.xml', site) + const body = `User-agent: * +Allow: / + +User-agent: OAI-SearchBot +Allow: / +User-agent: Claude-SearchBot +Allow: / +User-agent: PerplexityBot +Allow: / + +# Training opt-out. Remove if the owner is happy to be trained on. +User-agent: GPTBot +Disallow: / +User-agent: ClaudeBot +Disallow: / +User-agent: Google-Extended +Disallow: / +User-agent: Applebot-Extended +Disallow: / + +Sitemap: ${sitemap.href} +` + return new Response(body, { headers: { 'Content-Type': 'text/plain; charset=utf-8' } }) +} +``` + +- `site` must be set in `astro.config.mjs`, otherwise `site` is undefined here and the + `@astrojs/sitemap` integration refuses to build. The integration writes `sitemap-index.xml` + and `sitemap-0.xml`. +- Do not keep both `public/robots.txt` and `src/pages/robots.txt.ts`. +- If the host is Cloudflare Pages/Workers, check the AI Crawl Control policy as described in + the main skill. It can add its own `Disallow` lines on top of yours. diff --git a/skills/aeo-ai-crawler-access/references/nextjs.md b/skills/aeo-ai-crawler-access/references/nextjs.md new file mode 100644 index 0000000..a9701cc --- /dev/null +++ b/skills/aeo-ai-crawler-access/references/nextjs.md @@ -0,0 +1,31 @@ +# Next.js (App Router) + +`app/robots.ts` returns a `MetadataRoute.Robots` object and Next serves it as `/robots.txt`. +Multiple bots go in the `rules` array; `userAgent` can be a string or an array. + +```ts +// app/robots.ts +import type { MetadataRoute } from 'next' + +export default function robots(): MetadataRoute.Robots { + return { + rules: [ + { userAgent: '*', allow: '/' }, + { userAgent: ['OAI-SearchBot', 'Claude-SearchBot', 'PerplexityBot'], allow: '/' }, + // Training opt-out. Remove this entry if the owner is happy to be trained on. + { userAgent: ['GPTBot', 'ClaudeBot', 'Google-Extended', 'Applebot-Extended'], disallow: '/' }, + ], + sitemap: 'https://example.com/sitemap.xml', + } +} +``` + +- A static `public/robots.txt` also works, but do not have both: pick one. +- Use the real production origin in `sitemap`. Do not build it from a preview URL env var, or + the deploy preview's host ends up in production's robots.txt. +- Check `middleware.ts` / `proxy.ts` matchers: a rule that rewrites or redirects unknown user + agents will also catch bots. Exclude `/robots.txt` and `/sitemap.xml` from auth middleware. +- On Vercel, also check Firewall rules and Attack Challenge Mode, which challenge bots at the + edge regardless of robots.txt. + +Verify: `curl -s https:///robots.txt`. diff --git a/skills/aeo-ai-crawler-access/references/static-html.md b/skills/aeo-ai-crawler-access/references/static-html.md new file mode 100644 index 0000000..13df975 --- /dev/null +++ b/skills/aeo-ai-crawler-access/references/static-html.md @@ -0,0 +1,12 @@ +# Static HTML / any host + +1. Create `robots.txt` in the directory the host serves as the site root (`public/`, `dist/`, + `docs/`, whatever deploys to `/`). Use the template in the main skill. +2. Make sure the host serves it as `text/plain` and does not rewrite unknown paths to + `index.html` for it. A single-page-app fallback rule that catches `/robots.txt` makes it + `text/html`, and the audit then says it is missing. +3. Per-subdomain: each host needs its own robots.txt (`www.example.com`, `blog.example.com`). + Anthropic's page says to do this for every subdomain you want covered. +4. If the host or CDN has an "AI bots" toggle (Cloudflare has one), check it matches the robots.txt you just wrote. + +Verify: `curl -sI https://example.com/robots.txt` shows `200` and `content-type: text/plain`. diff --git a/skills/aeo-ai-crawler-access/references/wordpress.md b/skills/aeo-ai-crawler-access/references/wordpress.md new file mode 100644 index 0000000..784b8f7 --- /dev/null +++ b/skills/aeo-ai-crawler-access/references/wordpress.md @@ -0,0 +1,33 @@ +# WordPress 6.x + +WordPress serves a **virtual** `/robots.txt` unless a physical `robots.txt` exists in the web +root. A physical file wins and ignores everything below. + +Add rules through the `robots_txt` filter (parameters: `$output`, `$public`): + +```php +// in a small mu-plugin or the theme's functions.php +add_filter( 'robots_txt', function ( $output, $public ) { + if ( ! $public ) { + return $output; // site set to "discourage search engines": leave it alone + } + $output .= "\nUser-agent: OAI-SearchBot\nAllow: /\n"; + $output .= "User-agent: Claude-SearchBot\nAllow: /\n"; + $output .= "User-agent: PerplexityBot\nAllow: /\n"; + // Training opt-out. Remove if the owner is happy to be trained on. + foreach ( array( 'GPTBot', 'ClaudeBot', 'Google-Extended', 'Applebot-Extended' ) as $bot ) { + $output .= "\nUser-agent: {$bot}\nDisallow: /\n"; + } + return $output; +}, 20, 2 ); +``` + +- Settings, Reading, "Discourage search engines from indexing this site" makes WordPress emit + `Disallow: /` for everyone. This is the most common reason a WordPress site is invisible. + It is often left on after a staging copy goes live. +- Yoast, Rank Math and AIOSEO each edit robots.txt from their own screen. Look there first; + two plugins writing it produces a file nobody chose. +- Security plugins and host firewalls (Wordfence, Sucuri, host-level bot protection) can block + AI user agents before WordPress runs. Check their logs, not robots.txt. +- Core generates a sitemap at `/wp-sitemap.xml`; SEO plugins replace it with their own. + Reference whichever one is live. diff --git a/skills/aeo-ai-referral-analytics/SKILL.md b/skills/aeo-ai-referral-analytics/SKILL.md new file mode 100644 index 0000000..a6b4392 --- /dev/null +++ b/skills/aeo-ai-referral-analytics/SKILL.md @@ -0,0 +1,89 @@ +--- +name: aeo-ai-referral-analytics +description: Measure traffic that AI assistants send to a site, using Google Analytics 4. Use when someone asks "how much traffic comes from ChatGPT/Perplexity/Claude", "are AI answers sending us visitors", "set up an AI traffic channel in GA4", "which pages do AI assistants send people to", or "is our AEO work driving visits". Covers the GA4 query, a reusable referrer regex, landing pages, and why the number is always an undercount. Requires the aeo-toolkit aeo-search MCP server with Google OAuth. +--- + +# AI referral analytics (GA4) + +AI assistants send some visitors with a referrer or a `utm_source` tag, and GA4 records +them. That subset is measurable. It is **a floor, not the total**: most AI influence never +produces a tagged click. Report it as "at least N sessions", never as the size of AI impact. + +## When this matters / when it doesn't + +- Matters: showing whether AEO work produces visits, and which pages earn them. +- Weak evidence on its own: small sites will have tens of sessions, where week-to-week + moves are noise. Say so instead of charting a trend. +- Not a visibility metric. Whether the site is *cited* is `aeo-visibility-tracking`. + +## Steps + +1. `list_ga4_properties`, confirm the property with the user. Don't guess the ID. +2. **Pull sources, then filter yourself.** `ga4_run_report` takes `propertyId`, `dateRanges`, + `dimensions`, `metrics` and `limit` only: **there is no filter argument.** So request + a broad grouping and filter the returned rows: + - dimensions: `sessionSource`, `sessionMedium` (add `landingPage` for step 4) + - metrics: `sessions`, `engagedSessions`, `keyEvents` (use only metric names GA4 accepts for + the property; if a call errors, drop the metric rather than guessing) + - `limit`: high (e.g. 10000) so the long tail of sources is not truncated. + Use a range of at least 28 days. Keep the same length when comparing periods. +3. **Match AI sources.** Apply the regex in `references/ga4-setup.md` to `sessionSource`. + Also list sources that *almost* match and any unfamiliar source containing `ai`, `gpt`, + `chat` or `claude`, and show them to the user: new assistants appear faster than any + list. Do not silently drop or add rows. +4. **Landing pages.** Re-run with `landingPage` and `sessionSource`, filter to the AI + sources, rank pages by sessions. These are the pages assistants actually send people to. + Compare against pages you expected; mismatches are the useful finding. +5. **Quality.** Compare engaged sessions per session and key events for AI sessions against + organic search. With under ~30 AI sessions, say the comparison is too thin to read. +6. **Trend.** Re-run `date` + `sessionSource` weekly if the volume supports it. + +## How the traffic is tagged + +- OpenAI's publisher FAQ states ChatGPT adds `utm_source=chatgpt.com` to links it sends, and + that this tracking depends on the site allowing OAI-SearchBot. Verified via an excerpt + of that FAQ; OpenAI's help pages block automated fetching, so re-read it before quoting. +- For other assistants (Perplexity, Claude, Gemini, Copilot) we did **not** verify referrer + hostnames or tagging against primary sources. The regex lists likely hostnames; step 3's + "unfamiliar source" review is how you confirm what really appears in this property. +- GA4 dimension names `sessionSource`, `sessionMedium`, `sessionDefaultChannelGroup`, + `landingPage` and `pageReferrer` exist in the Data API schema (verified 2026-11-13). + `pageReferrer` is the full previous URL, per page, not per session. + +## How to read the result + +- **Signal:** AI sessions that are consistently present across weeks, concentrated on + identifiable pages, with engagement similar to other referral traffic. +- **Noise:** week-to-week swings on a few dozen sessions; a one-day spike (often a single + mention or a crawler/bot). +- Always state the denominator: AI sessions as a share of all sessions, with both counts. + +## What it cannot tell you (undercounting, be explicit) + +- **No-click influence.** A user reads an answer that names the brand, then searches it + or types the URL later. That arrives as organic or **direct**. +- **Stripped or missing referrers.** Copied links, apps, in-app browsers and some privacy + settings drop the referrer, so visits land in "direct". One secondary source says this + happens for in-app browsers and copied links; not confirmed by OpenAI. +- **Assistants that don't tag.** Without a referrer or UTM, nothing in GA4 identifies them. +- **Bot and crawler traffic** is not a visitor; GA4 filters known bots, not all of them, so crawler + activity needs server logs, not GA4. +- Causality: a visit does not show that an AEO change caused it. + +Do not "correct" for undercounting with a multiplier. There is no verified one. + +## Pitfalls + +- Presenting the AI-referral total as total AI impact. +- Reading `direct` growth as AI traffic with no other evidence. +- A regex that is stale: review unmatched sources each time. +- Comparing periods of different length or weekday mix. +- Changing the channel definition mid-trend without noting the date. + +## Sources + +- GA4 Data API schema, https://developers.google.com/analytics/devguides/reporting/data/v1/api-schema (dimension names), verified 2026-11-13. +- OpenAI Publishers and Developers FAQ, help.openai.com article 12627856 (`utm_source=chatgpt.com`), verified 2026-11-13 via search excerpt only. +- OpenAI crawler docs, https://platform.openai.com/docs/bots, verified 2026-11-13. +- Tool behaviour from the aeo-search `ga4_run_report` definition, verified 2026-11-13. +- Hostnames other than chatgpt.com: NOT verified. diff --git a/skills/aeo-ai-referral-analytics/references/ga4-setup.md b/skills/aeo-ai-referral-analytics/references/ga4-setup.md new file mode 100644 index 0000000..b5cb1d5 --- /dev/null +++ b/skills/aeo-ai-referral-analytics/references/ga4-setup.md @@ -0,0 +1,39 @@ +# GA4 setup for AI referral traffic + +## Regex for AI sources (Session source) + +Starting point, case-insensitive match on `sessionSource`: + +``` +(?i)^(chatgpt\.com|chat\.openai\.com|perplexity(\.ai)?|claude\.ai|gemini\.google\.com|copilot\.microsoft\.com|copilot|edgeservices\.bing\.com)$ +``` + +Only `chatgpt.com` (via OpenAI's documented `utm_source`) is verified against a primary source. +The rest are commonly observed hostnames to confirm in your own data. Review unmatched +sources before trusting totals, and extend the regex when you see a new assistant. +Anchor with `^...$` so `gemini.google.com` does not catch ordinary `google`. + +GA4's "matches regex" is a full match, so the anchors are optional there; keep them for +filtering returned rows in code. + +## Custom channel group (GA4 UI) + +Whether a new channel group applies to past data is a GA4 behaviour we did not verify; test it on a past range before promising history. + +1. Admin > Data display > Channel groups > Create new channel group. +2. Add a channel named "AI assistants". +3. Condition: Source, "matches regex", paste the regex above. +4. Drag it **above** "Referral" (channels match top to bottom; the first match wins). +5. Save. Use the new group in Acquisition reports and as a comparison. + +## Exploration (no channel group needed) + +Explore > Free form. Dimensions: Session source, Landing page. Metrics: Sessions, Engaged +sessions, Key events. Filter: Session source matches regex (above). + +## Checks + +- Compare totals from the API (`sessionSource`, filtered in code) with the Explore view; + they should agree for the same range. +- Look at Session medium for the matched sources. `(none)` suggests missing referrer; + `referral` suggests a referrer was passed. The mix is informative, not a verdict. diff --git a/skills/aeo-answer-first-content/SKILL.md b/skills/aeo-answer-first-content/SKILL.md new file mode 100644 index 0000000..51e981c --- /dev/null +++ b/skills/aeo-answer-first-content/SKILL.md @@ -0,0 +1,147 @@ +--- +name: aeo-answer-first-content +description: Restructure existing pages so an answer engine can lift a clean answer out of them, without wrecking the author's voice. Use when someone asks why ChatGPT, Perplexity or Google AI Overviews don't quote their page, how to write for AI answers, how to structure a post for featured snippets, whether to add FAQ-style or question headings, why a page is flagged as thin, has no headings, has two H1s or skips heading levels, or when an audit fails aeo.question-headings, aeo.answerable-content, aeo.content-extractability, tech.content-not-thin, tech.single-h1 or tech.heading-hierarchy. Uses the aeo-visibility MCP server if connected; works without it. +--- + +# Answer-first content + +An answer engine can only quote what it can find and cut out cleanly. Pages that open each section +with the direct answer, then explain, are easier to quote than pages that circle the point for four +paragraphs. This skill covers how to restructure a page that way, and how to leave alone the pages +that do not need it. + +## When this matters, and when it doesn't + +It matters for pages that exist to answer a question: guides, comparisons, pricing explainers, +documentation, help articles, "what is X" pages. + +It mostly does not matter for: + +- **Homepages, product landing pages, contact and legal pages.** They are not answers. Do not + force question headings onto them to turn a check green. +- **Narrative or opinion writing.** An essay's value is the argument. Chopping it into Q&A + destroys it. Add a short summary at the top if you want something quotable. +- **Pages that are already well structured but get no citations.** Then the cause is probably + elsewhere: the page may not be fetchable (see `aeo-server-rendering`, `aeo-ai-crawler-access`), + or nothing off-site points to it. Do not rewrite a good page on a hunch. + +Be honest about what is known. Google's own guidance on AI features says there are no extra +requirements or special optimizations to appear in AI Overviews or AI Mode beyond normal SEO +fundamentals, and that no special schema is needed. There is no verified public recipe for how any +other engine picks a passage, so do not promise that a given rewrite will earn a citation. What you +can say is that clear, complete, well-structured text is easier to extract, and it is also better +for human readers. + +## Detect + +Rule IDs this skill owns, with what the audit actually checks (from `@advance-labs/scoring` and +`@advance-labs/html-parser`): + +| Rule | Passes when | Read it carefully | +|---|---|---| +| `aeo.question-headings` | At least one heading across **all audited pages** is question-like: it ends in `?`, or its first word is who, what, when, where, why, how, which, can, should, is, are, do, does or will | One heading anywhere is enough. "How to ..." and "Who we are" both count. A pass proves little. | +| `aeo.answerable-content` | Mean visible word count across pages is 300 or more **and** at least one `
    ` or `
      ` exists somewhere | Site-wide mean, not per page. | +| `aeo.content-extractability` | Mean `

      ` count per page is 3 or more **and** at least one list or table exists somewhere | Same caveat. | +| `aeo.speakable-structured-answers` | Any page has Speakable, HowTo or QAPage schema, a "how to" or "step-by-step" heading, or FAQ/QA schema | A "how to" heading alone passes it. Low weight; do not chase it. | +| `tech.content-not-thin` | Every page has 200 or more visible words | Word count is the whole `` text minus script, style and ``, so nav and footer text counts. A page with a long footer can pass while being thin. | +| `eeat.exp.content-depth` | Mean words across pages is 600 or more | A mean. Fix the pages that need depth, not every page. | +| `tech.headings-present` | Every page has at least one heading | | +| `tech.single-h1` | Every page has exactly one `

      ` | | +| `tech.heading-hierarchy` | No heading jumps down more than one level (h2 to h4 fails; h3 back up to h2 is fine) | The first heading on the page may be any level. | + +**With the MCP server.** Call `analyze_website_aeo` with the site URL. It crawls up to 10 pages and +returns `topFixes` (at most 8) with rule IDs. It only lists failed rules in the aeo, structured-data, +content and metadata categories, and it does not return per-page URLs. So you will see which rule +failed but not which page. Use the fallback below to find the pages. + +**Without it.** Fetch the page and read the outline: + +```bash +curl -sL https://example.com/guide | grep -oiE ']*>.*?' | sed -E 's/<[^>]+>//g' +``` + +Look at the heading list the way a reader would. Then read the first paragraph under each heading. + +To decide *which* pages deserve this work, use `discover_ranking_prompts` (needs a Perplexity key +passed to the tool). It returns the prompts people plausibly ask about a topic. Match each prompt +to the page that should answer it. A prompt with no page is a content gap, not a formatting problem. + +## Decide + +Work page by page. For each candidate page, answer these in order: + +| Question | If no | If yes | +|---|---|---| +| Does the page answer a real question someone types or asks? | Leave it. | Continue. | +| Does it already open each section with its point? | Leave it. | Continue. | +| Is the answer buried (a long preamble, a story first, the number in paragraph five)? | Leave it. | Restructure. | +| Is the page genuinely too short to answer the question? | Restructure only. | Add real content, see below. | + +**Question headings: use them where the heading really is the question.** "How much does X cost?" +is a good heading for a pricing section, because that is what readers ask. "Our Approach" does not +need to become "What is our approach?". Rewriting every heading as a question reads as spam and +adds nothing. A reasonable target is question headings on the sections that answer a discrete +question, and descriptive headings everywhere else. + +**Thin versus deliberately short.** A 120-word contact page is not a problem. A 150-word "guide" is. +Google states it has no preferred word count and warns against writing to one. Treat the 200 and +300 and 600 word thresholds as the audit's heuristics, not as targets. Never pad. If a page is +thin, ask what the reader still needs to know, and answer that. + +## Implement + +Stack-agnostic shape of the change. See `references/examples.md` for before and after rewrites. + +1. **One `

      `, then a clean outline.** The h1 is the page's topic. Sections are h2, their + subsections h3. Do not pick a heading level for its font size; fix the size in CSS. Layout + components that render a second h1 (a site-name logo, a hero banner, a card title) are the usual + cause of `tech.single-h1`. Change the element, not the look. +2. **Lead each section with the answer.** First one or two sentences (roughly the first 40 to 60 + words) state the direct answer: the number, the yes or no, the definition. Then give the + reasoning, caveats and examples. This helps human skimmers as much as any engine. The 40 to 60 + word figure is a writing guideline, not a measured threshold from any engine. +3. **Make the answer self-contained.** A passage lifted out of context should still make sense. + Name the subject instead of "it" or "this", and include the unit and the scope ("in Ontario", + "as of 2026"). +4. **Use a list or table when the content is a list or a comparison.** Steps are an ordered list. + Options with attributes are a table. Do not turn flowing reasoning into bullets. +5. **Keep the author's voice.** Edit structure, not style. Move the answer sentence up, split a + long paragraph, add a heading. Do not rewrite their prose into a template. If the voice is + distinctive, show the proposed change as a diff and let them accept it. +6. **Keep the content in the HTML the server sends.** If the page is rendered client-side, none of + this is visible to crawlers that do not run JavaScript. Check `aeo-server-rendering` first. + +Do not add FAQ markup in this skill; that belongs to `aeo-faq-qa-schema`. Do not mark up content +that is not visible on the page. + +## Verify + +1. Re-run `analyze_website_aeo` (or re-read the outline with the curl command) and confirm the + owned rules that failed now pass. Remember the audit caps at 10 pages and several rules are + site-wide, so a pass may come from other pages. +2. Check outline validity yourself: one h1, no level skipped on the way down. +3. For each rewritten section, read only its first two sentences. Do they answer the heading? +4. Optionally run `check_ai_visibility` for a target prompt before and after. Treat the result as + one noisy sample. Engines vary by run and over time, and a single check is not proof either + way. See `aeo-visibility-tracking` for how to measure properly. + +## Pitfalls + +- **Turning every heading into a question.** Over-optimization that hurts readers. +- **Padding to reach 300 or 600 words.** The thresholds are heuristics. Padding makes the page + worse and gets no credit from Google, which says it has no preferred word count. +- **Treating a green audit as the goal.** `aeo.question-headings` passes with one question heading + on the whole site. A pass says nothing about whether any page is quotable. +- **Hiding the answer in an accordion or tab that loads on click.** Content that is not in the + initial HTML may not be read. +- **Fixing `tech.content-not-thin` by adding boilerplate.** The check counts nav and footer text, + so the number can move without the page improving. Judge the main content. +- **A second h1 from a component library** (logo, hero, modal). Look at rendered HTML, not source. +- **Rewriting without asking.** If the page's purpose is unclear, ask what question it should + answer before changing it. + +## Sources + +- Google Search Central, Creating helpful, reliable, people-first content (no preferred word count; do not change dates to seem fresh): https://developers.google.com/search/docs/fundamentals/creating-helpful-content, verified 2026-11-13 +- Google Search Central, AI features and your website (no additional requirements or special optimization for AI Overviews and AI Mode; no special schema needed): https://developers.google.com/search/docs/appearance/ai-features, verified 2026-11-13 +- Scoring rule source: `packages/scoring/src/aeo-rules.ts`, `technical-seo-rules.ts`, `eeat-rules.ts`; heading and word-count logic in `packages/html-parser/src/content.ts` and `headings.ts`, read 2026-11-13 diff --git a/skills/aeo-answer-first-content/references/examples.md b/skills/aeo-answer-first-content/references/examples.md new file mode 100644 index 0000000..a109ebb --- /dev/null +++ b/skills/aeo-answer-first-content/references/examples.md @@ -0,0 +1,73 @@ +# Before and after: answer-first rewrites + +Each example changes structure only. The author's facts and tone stay. Invented numbers below are +placeholders; never ship an example figure as a real claim. + +## 1. Buried answer + +Before: + +```html +

      Pricing

      +

      When we started the studio we wanted a model that felt fair to small teams. Over several years +we tried hourly billing, retainers and fixed scopes. Each had trade-offs. Today most projects are +quoted as a fixed scope, and the figure depends on the pages and integrations involved.

      +

      A typical brochure site lands between [LOW] and [HIGH].

      +``` + +After: + +```html +

      How much does a website cost?

      +

      A typical brochure site costs between [LOW] and [HIGH], quoted as a fixed scope. The figure +depends on the number of pages and the integrations involved.

      +

      When we started the studio we wanted a model that felt fair to small teams. We tried hourly +billing and retainers before settling on fixed scopes, because ...

      +``` + +What changed: the heading is the real reader question, the number moved to the first sentence, the +backstory stayed, just later. + +## 2. Comparison written as prose + +Before: three paragraphs each describing a plan. + +After: one sentence answer ("Pro suits most teams; Basic is for solo use."), then a table. + +```html + + + + + + +
      PlanSeatsBest for
      Basic1Solo use
      ProUp to 10Small teams
      +``` + +Use a table only when the content really has rows and columns. + +## 3. Heading that should stay as it is + +`

      Our approach

      ` followed by a description of the firm's method. Leave it. It is not a +question a reader asks, and "What is our approach?" is filler. Question headings belong where the +section answers a discrete question. + +## 4. Second h1 from a component + +Symptom: `tech.single-h1` fails on every page. The cause is often a layout that renders the logo +or hero title as `

      ` while each page also has its own. + +Fix: change the layout element to a `
      ` or `

      ` and keep the visual style with a class. Keep +exactly one h1 per page, the page topic. + +## 5. Skipped heading level + +Before: `h1`, `h2`, `h4` (the h4 chosen because it looked right). + +After: change the `h4` to `h3` and restyle with CSS. Going back up (h3 then h2) is always fine. + +## 6. A thin page + +A 140-word "guide to X". Do not pad. Ask what a reader still needs: the steps, the edge case, the +cost, the common mistake. Write those. If there is nothing more to say, the page may be better +merged into a stronger page than kept thin (check with `seo-cannibalization` before merging). diff --git a/skills/aeo-article-author-schema/SKILL.md b/skills/aeo-article-author-schema/SKILL.md new file mode 100644 index 0000000..d5c1e50 --- /dev/null +++ b/skills/aeo-article-author-schema/SKILL.md @@ -0,0 +1,181 @@ +--- +name: aeo-article-author-schema +description: Mark up articles and blog posts with Article, BlogPosting or TechArticle JSON-LD whose author is a real Person linked to an author page, with datePublished, dateModified and a publisher that points at the organization. Use when someone asks to add article schema, author markup, byline or author bio schema, fix "article and author schema missing", "author not identified" or "Person schema" audit findings, or improve E-E-A-T attribution for blog and docs content. Covers aeo.article-author-schema, eeat.expertise.person-schema and eeat.expertise.article-markup. Uses the aeo-visibility MCP server if connected; works without it. +--- + +# Articles with a named, linkable author + +Article markup says "this page is a piece of writing"; the author node says who wrote it and +where that person can be verified. Together they give an engine something to attribute a quote +to. The markup must describe what is visibly on the page: a real byline, real dates. + +## When this matters / when it doesn't + +**It matters** for blog posts, guides, docs articles, news and research: pages where a named +person or the organization stands behind the content. + +**It does not apply to** homepages, product pages, pricing, landing pages or tool pages. +Marking those up as `Article` to satisfy an audit is misleading markup. The audit knows this: +`aeo.article-author-schema` is skipped when a single-page audit targets the site root. If the +site has no editorial content at all, the honest answer is "this does not apply", not "add a +fake blog post". + +**An author is not required for the markup to be valid.** Google's Article documentation lists +no required properties. `author` is recommended, and it can be an `Organization` when the +content is genuinely published by the company with no individual byline. Do not invent a +person to fill the field, and do not credit a real person who did not write the piece. + +**AI-generated or ghostwritten content** needs a human decision about who is credited. Ask the +user; do not choose a name. + +## Detect + +| Rule | Passes when | +|---|---| +| `aeo.article-author-schema` | Across the crawled pages, at least one `Article`-family node exists **and** at least one `Person` node exists. | +| `eeat.expertise.article-markup` | At least one Article-family node exists. | +| `eeat.expertise.person-schema` | At least one `Person` node exists. | + +What that really means: + +- "Article-family" is `Article`, `NewsArticle`, `BlogPosting`, `TechArticle`, `ScholarlyArticle`, + `Report`, `APIReference` and the other schema.org subtypes of Article. `TechArticle` counts + for presence. +- Detection is **at any nesting depth**, so `author: { "@type": "Person" }` inside the Article + satisfies the Person check. +- The rules are **site-wide ORs, not per-page**. A Person anywhere (a founder on the About + page) plus an Article anywhere passes `aeo.article-author-schema`, even if no article is + attributed to anyone. The audit passing does not prove the author is linked to the article. + Check that yourself. +- The toolkit's validator requires `headline` on `Article`, `NewsArticle` and `BlogPosting`, and + `name` on `Person`. A missing one marks the item invalid (`tech.structured-data-valid`). + `TechArticle` has no required-property check in the validator, so a headline-less TechArticle + is not flagged. Include `headline` anyway. + +With the MCP server: `analyze_website_aeo` on a sample article URL (not only the homepage), then +read the three rules and the structured-data detail. + +Without it: view-source an article, find the `ld+json` blocks, and answer these in order: is +there an Article-family node; does it have an `author`; is the author a `Person` with `name` +and a `url` or `sameAs`; does that URL resolve to a real author page; are both dates present +and ISO 8601; does `publisher` reference the Organization `@id`. + +## Decide + +| Situation | Do this | +|---|---| +| Posts have a visible byline linking to an author page | Person node with `url` to that page and `sameAs` to external profiles. | +| Byline exists, no author page | Create a short author page (name, role, bio, links) before marking it up; or link the best existing profile via `sameAs`. Ask the user which. | +| Posts are published as the company, no byline | `author` is the Organization (`{"@id": ".../#organization"}`). | +| Multiple authors | One `author` array entry per person. Never one entry with "A, B". | +| Guest post | The guest is the `author`; the site is the `publisher`. | +| Docs or API reference page | `TechArticle` (or `Article`). Same author rules. | +| Page is a changelog or release note | `Article` is fine; do not use `NewsArticle` unless it is news. | +| Dates are unknown | Omit them rather than guessing. Do not set `dateModified` to today on every build. | +| Homepage or landing page | Do nothing. Not an article. | + +## Implement + +Shape, independent of stack: + +1. One `Person` node per author, defined once on that author's page with a stable `@id` + (`https://example.com/authors/jane-doe#person`). Give it `name`, `url` (the author page), + `sameAs` (LinkedIn, GitHub, personal site, ORCID if relevant), and `jobTitle` and + `worksFor` if true and shown on the page. +2. Each article references that person by `@id` in `author`, and the organization by `@id` in + `publisher`. Also set `mainEntityOfPage` to the page and `isPartOf` to the `WebSite`. +3. `datePublished` and `dateModified` in ISO 8601 with a timezone offset + (`2026-11-13T09:00:00-05:00`). Google recommends the offset; without it the timezone of + Googlebot is assumed. The dates must match the visible ones. +4. `image`: relevant article images, crawlable URLs, ideally in 16:9, 4:3 and 1:1. +5. Generate all of it from the same post data that renders the byline, so they cannot diverge. + +```json +{ + "@context": "https://schema.org", + "@graph": [ + { + "@type": "BlogPosting", + "@id": "https://example.com/blog/answer-engines#article", + "headline": "How answer engines choose what to cite", + "description": "A one-sentence summary that matches the meta description.", + "mainEntityOfPage": { "@id": "https://example.com/blog/answer-engines" }, + "url": "https://example.com/blog/answer-engines", + "datePublished": "2026-11-13T09:00:00-05:00", + "dateModified": "2026-11-20T14:30:00-05:00", + "image": ["https://example.com/img/answer-engines-16x9.jpg"], + "author": { "@id": "https://example.com/authors/jane-doe#person" }, + "publisher": { "@id": "https://example.com/#organization" }, + "isPartOf": { "@id": "https://example.com/#website" } + }, + { + "@type": "Person", + "@id": "https://example.com/authors/jane-doe#person", + "name": "Jane Doe", + "url": "https://example.com/authors/jane-doe", + "jobTitle": "Head of Research", + "worksFor": { "@id": "https://example.com/#organization" }, + "sameAs": [ + "https://www.linkedin.com/in/janedoe", + "https://github.com/janedoe" + ] + } + ] +} +``` + +The Organization node (`/#organization`) and `WebSite` node are defined on the homepage; see +`aeo-organization-entity`. If a referenced `@id` is not defined on the same page, a validator +cannot see what it points to, and some engines will not follow it. Where that matters, include +the Organization node in the same graph on the article page too, with identical values. + +Google's author guidance, which the markup must follow: + +- `author.name` is **only the name**. No job title, no "Posted by", no honorifics, no publisher + name; those have their own properties (`jobTitle`, `honorificPrefix`, `publisher`). +- Use `Person` for people and `Organization` for organizations. Do not use the wrong one or `Thing`. +- List every author shown on the page, each as its own entry. +- `author.url` should link to a page that uniquely identifies the author (author page, social + profile). `sameAs` is accepted as an alternative and Google understands both when + disambiguating. For an author page on your own site, Google suggests marking it up with + `ProfilePage` structured data; that is optional and has its own documentation. + +Author page: a real, crawlable page at the `url`, with the person's name, a short bio, a +photo and links to the posts. The `Person` node is defined there. It should be linked from +the bylines, in the sitemap and not `noindex`. + +Stack recipes: `references/nextjs.md`, `references/astro.md`, `references/wordpress.md`, +`references/static-html.md`. + +## Verify + +1. Re-run `analyze_website_aeo` on an article URL (or the validator on its HTML). +2. The three rules above should be `pass`. If only the Person rule fails, the author is a bare + string (`"author": "Jane Doe"`); convert it to a `Person`. +3. Rich Results Test and Schema Markup Validator on the article URL: no errors; warnings for + missing recommended properties are acceptable. +4. Open the page and compare, field by field, to the markup: headline, byline, dates, image. + Any mismatch is a defect even if the validators are happy. +5. Click the author `url`; it must return 200, be indexable and show the same name. + +## Pitfalls + +- **`"author": "Jane Doe"` as a string.** Valid for some consumers, but it is not a Person node, + so the Person rules fail and there is nothing to link. +- **Author pages that 404, are `noindex` or are blocked in robots.txt.** +- **`dateModified` bumped on every deploy.** Engines and users read it as a freshness claim. + Change it only when the content changed. See `aeo-content-freshness`. +- **Dates without a timezone offset** shift by hours in other zones. +- **Two Article blocks on one page** from a theme and an SEO plugin, with different authors or + dates. See `aeo-structured-data-validation`. +- **A whole site marked `NewsArticle`.** Use it for news. `Article` or `BlogPosting` otherwise. +- **Fake authors, stock photos, invented credentials.** This is the opposite of E-E-A-T and + violates Google's guideline against misrepresenting who is behind the content. +- **Marking up the homepage as an Article** to pass the rule. + +## Sources + +- Google, Article structured data (no required properties; author, dates, best practices): https://developers.google.com/search/docs/appearance/structured-data/article (verified 2026-11-13) +- Google, general structured data guidelines: https://developers.google.com/search/docs/appearance/structured-data/sd-policies (verified 2026-11-13) +- schema.org Article, BlogPosting, TechArticle, Person: https://schema.org/Article , https://schema.org/BlogPosting , https://schema.org/TechArticle , https://schema.org/Person (verified 2026-11-13) +- Toolkit logic: `packages/scoring/src/aeo-rules.ts`, `packages/scoring/src/eeat-rules.ts`, `packages/schema-validator/src/analyze.ts`, `validation.ts` diff --git a/skills/aeo-article-author-schema/references/astro.md b/skills/aeo-article-author-schema/references/astro.md new file mode 100644 index 0000000..f26119b --- /dev/null +++ b/skills/aeo-article-author-schema/references/astro.md @@ -0,0 +1,27 @@ +# Article and author schema in Astro + +In the blog layout, compute the graph from frontmatter or the content collection entry and emit +it with `set:html`. + +```astro +--- +import { SITE, ORG_ID } from '../lib/entity'; +const { post, author } = Astro.props; // from getEntry / getCollection +const url = new URL(`/blog/${post.id}`, Astro.site).href; +const personId = `${SITE}/authors/${author.id}#person`; +const graph = { '@context': 'https://schema.org', '@graph': [ + { '@type': 'BlogPosting', '@id': `${url}#article`, headline: post.data.title, url, + mainEntityOfPage: { '@id': url }, + datePublished: post.data.pubDate.toISOString(), + dateModified: (post.data.updatedDate ?? post.data.pubDate).toISOString(), + author: { '@id': personId }, publisher: { '@id': ORG_ID } }, + { '@type': 'Person', '@id': personId, name: author.data.name, + url: `${SITE}/authors/${author.id}`, sameAs: author.data.links }, +] }; +--- + +``` + +Keep the wording identical in both places. With more than a few questions, generate both from +one source file at build time; hand-edited copies drift. Escape double quotes in JSON strings +and write apostrophes and ampersands as plain characters, not HTML entities, inside the script. diff --git a/skills/aeo-faq-qa-schema/references/wordpress.md b/skills/aeo-faq-qa-schema/references/wordpress.md new file mode 100644 index 0000000..90b874b --- /dev/null +++ b/skills/aeo-faq-qa-schema/references/wordpress.md @@ -0,0 +1,17 @@ +# FAQ markup in WordPress + +- **Yoast SEO and Rank Math** both ship FAQ blocks for the block editor that render a visible Q&A + section and emit matching FAQPage markup. Prefer these over a hand-written script: the visible + text and the markup come from the same fields. +- **Extend vs replace:** use the plugin block for normal FAQs. Hand-write JSON-LD only for + structures the block cannot express, and then disable the block on that page so the page + does not carry two FAQPage nodes. +- **Elementor/Divi/accordion plugins** often render a visible accordion with no markup, or + markup the SEO plugin also generates. Pick one source, and look at view-source. +- A FAQ plugin that loads answers through AJAX on click leaves the answers out of the HTML. + Choose one that outputs them in the markup. +- For community Q&A (QAPage), use a Q&A or forum plugin that supports it (and lets visitors submit + answers) rather than a FAQ block. + +Since FAQ rich results no longer exist in Google, there is no reason to buy a plugin or add +markup to every post to "get FAQ stars". diff --git a/skills/aeo-indexability/SKILL.md b/skills/aeo-indexability/SKILL.md new file mode 100644 index 0000000..c4e748e --- /dev/null +++ b/skills/aeo-indexability/SKILL.md @@ -0,0 +1,171 @@ +--- +name: aeo-indexability +description: Find and fix reasons search and answer engines cannot index or trust a site's pages. Use when someone asks "why isn't my page indexed", "sitemap missing pages", "noindex by mistake", "canonical points to a 404", "redirect loop", "redirect chain", "broken pages", "hreflang errors", "submit to Google and Bing", "IndexNow", or when the audit fails tech.sitemap-present, tech.sitemap-covers-pages, tech.indexable, tech.canonical-present, tech.canonical-resolves, tech.no-broken-pages, tech.no-redirect-loops, tech.short-redirect-chains, tech.hreflang-valid or tech.https. Covers per-engine submission (Google, Bing, IndexNow). Uses the aeo-search MCP server if connected; works without it. +--- + +# Indexability + +A page cannot be cited from an index it is not in. This skill covers the plumbing: sitemaps, +noindex, canonicals, redirects, broken pages, hreflang and HTTPS, plus how to tell each search +engine about changes. Several of these checks are blunt, so decide with the user before +"fixing" something that may be deliberate. + +## When this matters / when it doesn't + +It matters when pages the owner wants found are missing from Google or Bing, or the audit +reports crawl and index failures. + +Do not "fix" these without asking: + +- **`noindex` on purpose**: login, checkout, thank-you pages, internal search, staging, tag + archives. The audit already treats a noindex page that the sitemap does *not* list as + deliberate. It flags noindex on the home page, on pages the sitemap *does* list, or on any + page when there is no sitemap to show intent. +- **A cross-domain canonical.** Pointing at another host is a legitimate way to consolidate + (for example a proxied path whose canonical is the original). The audit does not flag it; + neither should you. +- **No hreflang.** A single-language site has nothing to validate and the rule passes. +- **A tiny site with no sitemap.** Not required, but cheap and worth adding. + +## Detect + +Rules owned, with the real thresholds from `packages/scoring/src/technical-seo-rules.ts`: + +| Rule | Fails when | +|---|---| +| `tech.sitemap-present` | no `/sitemap.xml` file and no sitemap entries found (a sitemap named in robots.txt counts if the crawler reads entries from it) | +| `tech.sitemap-covers-pages` | a sitemap exists, the crawl found more than one page, and **under 90%** of crawled indexable HTML pages are listed (a page counts as covered if its canonical is listed) | +| `tech.indexable` | `noindex` (meta or `X-Robots-Tag`) on the home page, on a page the sitemap lists, or on anything when there is no sitemap | +| `tech.canonical-present` | an indexable page has no `` | +| `tech.canonical-resolves` | a canonical is relative or malformed, or points at a URL the crawl saw return 4xx/5xx | +| `tech.no-broken-pages` | any crawled URL returns 4xx/5xx | +| `tech.no-redirect-loops` | a URL's redirect chain answers twice from the same URL (a cycle) | +| `tech.short-redirect-chains` | a chain has **more than 2** hops | +| `tech.hreflang-valid` | an hreflang value is not a valid language[-script][-region] tag or `x-default`, or a same-host target was not reached by the crawl | +| `tech.https` | the site was not reached over HTTPS | + +`tech.sitemap-lastmod-trustworthy` is related but belongs to `aeo-content-freshness`. + +With the MCP servers: + +1. `analyze_website_aeo` for the rule results and affected URLs. +2. `list_gsc_sites`, then `gsc_url_inspect` for **definitive Google status** (up to 25 URLs per + call, about 2,000 inspections per property per day). It returns verdict, coverageState, + lastCrawlTime, googleCanonical and userCanonical. It is read-only: Google offers no API to + request indexing. Compare `googleCanonical` with `userCanonical`; if Google chose a + different canonical than you declared, that is the finding. +3. For Bing, `bing_index_health` (crawl stats, issues, URL-submission quota, single-URL status + with `url`). **Known caveat:** its `isPageIndexed` field is always false because Bing's + response omits it. Judge by `DocumentSize > 0` instead. +4. `compare_engines` and `engine_divergence` to see where Google and Bing disagree. A page that + gets impressions on Google and none on Bing is a Bing coverage problem, not necessarily a + content problem. + +Without MCP: + +```bash +curl -sI https://example.com/page | grep -iE "^HTTP|location|x-robots-tag" # status, redirect, header noindex +curl -s https://example.com/page | grep -iE "rel=\"canonical\"|name=\"robots\"|hreflang" +curl -sIL -o /dev/null -w "%{num_redirects} hops, final %{url_effective}\n" http://example.com/page +curl -s https://example.com/sitemap.xml | grep -c "" +``` + +Search Console's Page indexing report and URL Inspection are the manual equivalents. + +## Decide + +Work from the worst to the least: + +1. **HTTPS and redirect loops first.** A loop makes a page unreachable. HTTP not redirecting + to HTTPS splits signals. +2. **Accidental noindex.** Ask: "Is this page meant to be in search?" A framework default or a + staging setting that shipped is the usual cause. +3. **Broken pages and bad canonicals.** Decide per URL: restore it, 301 it to the closest + replacement, or remove links to it. Never 301 everything to the home page; engines treat + irrelevant redirects like soft 404s. +4. **Sitemap coverage.** Missing pages usually mean the sitemap is hand-written or generated + from a stale list. Fix the generator, not the file. +5. **Redirect chains.** Make each old URL go to the final URL in one hop, and update internal + links to the final URL. +6. **hreflang.** Only if the site is actually multilingual. Each page needs reciprocal tags and a + self reference; values must be valid codes. + +## Implement + +Stack-agnostic shape: + +- **Sitemap:** generate it from the same content list that builds the site. List only canonical, + indexable, 200 URLs. Reference it from robots.txt (`Sitemap: https://example.com/sitemap.xml`). + One file holds up to 50,000 URLs or 50MB uncompressed; beyond that use a sitemap index. Google + ignores `` and `` and uses `` only if it is consistently + accurate, so set `lastmod` from real content edits, not build time. +- **Canonical:** one absolute `https://` URL per page, self-referencing on the preferred + version. Redirects and canonicals both signal the preferred URL; do not contradict them. +- **noindex:** remove from pages meant to be found. For non-HTML files use the + `X-Robots-Tag` header. A page blocked in robots.txt cannot show its noindex, so do not combine + `Disallow` with the intent to deindex. +- **Redirects:** 301/308 for permanent moves, one hop, to the final URL. +- **HTTPS:** redirect all HTTP to HTTPS at the host or CDN and set HSTS once you are sure. + +Per-stack recipes: `references/nextjs.md`, `references/astro.md`, `references/wordpress.md`, +`references/static-html.md`. + +### Telling each engine + +Channels differ. Pick the right one, and do not overpromise. + +| Engine | What to use | What not to expect | +|---|---|---| +| Google | Submit the sitemap in Search Console (Sitemaps report) and keep it in robots.txt. `gsc_url_inspect` to check status. "Request indexing" in the Search Console UI for single URLs. | Google says sitemap submission is **only a hint**, with no guarantee it will be downloaded or used. Google **does not take IndexNow**: it is not among the participating engines. The sitemap "ping" endpoint was announced as going away (June 2023); do not use it. The Indexing API is **only for pages with JobPosting or BroadcastEvent (in a VideoObject) markup**; using it for ordinary pages is outside its stated purpose. | +| Bing | Verify the site in Bing Webmaster Tools, submit the sitemap, and use IndexNow for changes. | IndexNow tells Bing a URL changed; it does not guarantee indexing. | +| Yandex, Naver, Seznam, Yep, Amazon | IndexNow | Participation is per the IndexNow FAQ; a single submission is shared to all participating engines. | + +**IndexNow** works by hosting a key file you control and sending the changed URL(s) to one +endpoint (`https://api.indexnow.org/indexnow` or Bing's own). Only submit URLs that actually +changed, and submit deletions too. + +**Why Bing is worth checking.** Microsoft Copilot and Bing's AI answers draw on Bing's index, +and Bing Webmaster Tools has an AI Performance report for Copilot and Bing citations. Many +write-ups also claim ChatGPT search uses Bing's index. **That claim could not be confirmed from +OpenAI's own documentation**, which describes its own search crawler, `OAI-SearchBot` (see +`aeo-ai-crawler-access`). Say "ChatGPT may draw on Bing" at most, and do not build a plan on it. +Bing coverage is worth fixing for Copilot and Bing alone. + +## Verify + +1. Re-run `analyze_website_aeo`. The owned rules pass, or the remaining failures are the ones the + user chose to keep, and you can name them. +2. `gsc_url_inspect` on the fixed URLs after Google has had time to recrawl (days to weeks, not + minutes). Expect `verdict: PASS`-style indexed status, `googleCanonical` equal to `userCanonical`. +3. `bing_index_health` with `url`, then `DocumentSize > 0`. Do not read `isPageIndexed`. +4. `engine_divergence` or `compare_engines` a few weeks later. The Google-vs-Bing gap for the fixed + pages should narrow. +5. Spot-check redirects: `curl -sIL` shows one hop and a 200 at the end. + +## Pitfalls + +- Reading "Discovered, currently not indexed" or "Crawled, currently not indexed" as a technical + fault. Those can be quality or priority decisions. Do not "fix" them by resubmitting. +- Submitting the same URL to every channel repeatedly. It does nothing extra and can hit quotas. +- A sitemap that lists redirected, noindex or 404 URLs. It tells engines two things at once. +- Build-time `lastmod` on every entry. It teaches crawlers to ignore the field. +- `Disallow` plus `noindex`: the crawler never fetches the page, so never sees the noindex. +- Changing the domain or moving to HTTPS without redirecting every URL and updating canonicals. +- Treating Search Console data as instant. It lags, and "URL is on Google" can differ from + "page was recrawled after your fix". +- A staging `X-Robots-Tag: noindex` set in a shared CDN rule that also covers production. +- Using `bing_index_health.isPageIndexed`. It is always false. + +## Sources + +- Google, build and submit a sitemap (size limits, lastmod, submission is a hint): https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap (verified 2026-11-13) +- Google, "Sitemaps ping endpoint is going away" (June 2023): https://developers.google.com/search/blog/2023/06/sitemaps-lastmod-ping (verified 2026-11-13, title only; body not read) +- Google Indexing API (JobPosting and BroadcastEvent only): https://developers.google.com/search/apis/indexing-api/v3/quickstart (verified 2026-11-13) +- Google, canonicalization methods: https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls (verified 2026-11-13) +- Google, noindex: https://developers.google.com/search/docs/crawling-indexing/block-indexing (verified 2026-11-13) +- Google, localized versions (hreflang): https://developers.google.com/search/docs/specialty/international/localized-versions (verified 2026-11-13) +- Google Search Console API limits (URL Inspection 2,000 QPD and 600 QPM per site): https://developers.google.com/webmaster-tools/limits (verified 2026-11-13) +- IndexNow documentation and FAQ (participating endpoints: Amazon, Bing, Naver, Seznam.cz, Yandex, Yep; Google not listed): https://www.indexnow.org/documentation and https://www.indexnow.org/faq (verified 2026-11-13) +- Bing AI Performance in Webmaster Tools: https://blogs.bing.com/webmaster/February-2026/Introducing-AI-Performance-in-Bing-Webmaster-Tools-Public-Preview (found in search 2026-11-13; page not fetched) +- "ChatGPT search uses Bing": not verified against any OpenAI primary source. +- Toolkit rule implementation: `packages/scoring/src/technical-seo-rules.ts`. diff --git a/skills/aeo-indexability/references/astro.md b/skills/aeo-indexability/references/astro.md new file mode 100644 index 0000000..af9ed27 --- /dev/null +++ b/skills/aeo-indexability/references/astro.md @@ -0,0 +1,37 @@ +# Astro 5 + +**Sitemap:** `npx astro add sitemap`, and set `site` in `astro.config.mjs`. The integration +needs `site` to build. + +```js +// astro.config.mjs +import { defineConfig } from 'astro/config' +import sitemap from '@astrojs/sitemap' + +export default defineConfig({ + site: 'https://example.com', + integrations: [ + sitemap({ + filter: (page) => !page.includes('/thank-you/'), + // lastmod: only set if you have a real per-page date; a single build-time value is the + // antipattern the audit flags. Check the integration docs for per-page serialize(). + }), + ], +}) +``` + +It writes `sitemap-index.xml` and `sitemap-0.xml`. Reference `sitemap-index.xml` in robots.txt. + +**Canonical:** in your layout, ``. +Use `Astro.site`, not `Astro.url.origin`, so preview hosts never leak into production canonicals. + +**Redirects:** static redirects in config; they output meta-refresh HTML pages on static +builds, which is weaker than a server 301. On a host with its own redirect file (`_redirects`, +`vercel.json`), prefer the host-level 301. + +```js +redirects: { '/old-page': '/new-page' } +``` + +**noindex:** `` in the layout, passed in by a prop. +**hreflang:** `` per language, reciprocal. diff --git a/skills/aeo-indexability/references/nextjs.md b/skills/aeo-indexability/references/nextjs.md new file mode 100644 index 0000000..8dae5c2 --- /dev/null +++ b/skills/aeo-indexability/references/nextjs.md @@ -0,0 +1,51 @@ +# Next.js (App Router) + +**Sitemap** from your content list, with real `lastModified`: + +```ts +// app/sitemap.ts +import type { MetadataRoute } from 'next' +import { getAllPages } from '@/lib/content' // { path, updatedAt: Date }[] + +export default async function sitemap(): Promise { + const pages = await getAllPages() + return pages.map((p) => ({ + url: `https://example.com${p.path}`, + lastModified: p.updatedAt, // the content's edit date, never new Date() + })) +} +``` + +Over 50,000 URLs: use `generateSitemaps()` to split. Reference it from `app/robots.ts` +(`sitemap: 'https://example.com/sitemap.xml'`). + +**Canonical** per page through the Metadata API (set `metadataBase` once in the root layout): + +```ts +export const metadata = { alternates: { canonical: '/pricing' } } +// or per dynamic route: +export async function generateMetadata({ params }) { + const { slug } = await params + return { alternates: { canonical: `/blog/${slug}` } } +} +``` + +With `metadataBase: new URL('https://example.com')` this renders an absolute canonical. + +**noindex:** `export const metadata = { robots: { index: false, follow: true } }` on the page. +Search the codebase for `index: false` and for `X-Robots-Tag` in `next.config` and middleware +when something is unexpectedly noindex. Preview deployments often set noindex by design; make +sure that logic keys off the environment, not a flag that reaches production. + +**Redirects** in `next.config.ts`; use `permanent: true` (308) and point at the final URL: + +```ts +async redirects() { + return [{ source: '/old-blog/:slug', destination: '/blog/:slug', permanent: true }] +} +``` + +Check for a loop between a trailing-slash setting (`trailingSlash`) and a redirect rule that +adds or removes the slash. `curl -sIL` on the URL shows it. + +**hreflang:** `alternates: { languages: { 'en-CA': '/en-ca', 'fr-CA': '/fr-ca' } }` in metadata. diff --git a/skills/aeo-indexability/references/static-html.md b/skills/aeo-indexability/references/static-html.md new file mode 100644 index 0000000..c4ffdfa --- /dev/null +++ b/skills/aeo-indexability/references/static-html.md @@ -0,0 +1,41 @@ +# Plain HTML and any host + +**Sitemap:** a `sitemap.xml` at the root. Generate it in the build; a hand-edited file goes +stale. + +```xml + + + https://example.com/pricing2026-10-02 + +``` + +`lastmod` is the date the page's content last meaningfully changed, not today's date. Only list +canonical, indexable, 200 URLs. + +**Canonical:** in every page's ``: ``. +Absolute `https://` URL. Relative values fail `tech.canonical-resolves`. + +**noindex:** `` in the head, or the `X-Robots-Tag` header +for non-HTML files. + +**Redirects (one hop, to the final URL):** + +``` +# Netlify / Cloudflare Pages: _redirects +/old-page /new-page 301 +``` +```nginx +# nginx +location = /old-page { return 301 https://example.com/new-page; } +server { listen 80; server_name example.com www.example.com; return 301 https://example.com$request_uri; } +``` +```apache +# .htaccess +Redirect 301 /old-page https://example.com/new-page +``` + +**IndexNow:** put a key file at the root (`https://example.com/.txt` containing the key), +then `curl "https://api.indexnow.org/indexnow?url=https://example.com/new-page&key="` +after each deploy for changed URLs. Check the current format on indexnow.org before scripting a +batch POST. diff --git a/skills/aeo-indexability/references/wordpress.md b/skills/aeo-indexability/references/wordpress.md new file mode 100644 index 0000000..51d7b5c --- /dev/null +++ b/skills/aeo-indexability/references/wordpress.md @@ -0,0 +1,33 @@ +# WordPress 6.x + +**Sitemap:** core serves `/wp-sitemap.xml`. An SEO plugin (Yoast, Rank Math, AIOSEO) disables it +and serves its own (`/sitemap_index.xml` for Yoast). Reference whichever is live in robots.txt, +and do not run two. Core can be turned off with the `wp_sitemaps_enabled` filter, which +returns a 404 for the core sitemap. + +**Accidental noindex.** Check, in this order: + +1. Settings, Reading, "Discourage search engines from indexing this site". If ticked, WordPress + adds a noindex robots meta tag to every page. Staging-to-live copies often keep it. +2. The SEO plugin's per-post "Allow search engines to show this page" / robots setting, and its + site-wide defaults for post types and taxonomies. +3. A security or caching plugin adding `X-Robots-Tag`. + +To set a robots directive in code, use the `wp_robots` filter: + +```php +add_filter( 'wp_robots', function ( $robots ) { + if ( is_page( 'thank-you' ) ) { + $robots['noindex'] = true; + $robots['follow'] = true; + } + return $robots; +} ); +``` + +**Canonical:** core and the SEO plugins output it. Problems come from plugin conflicts (two +canonicals on one page) and from `home`/`siteurl` set to HTTP or the wrong host. +**Redirects:** use the SEO plugin's redirect manager or a redirect plugin; keep it to one hop. +When changing permalink structure, redirect old paths to new ones in the same step. +**HTTPS:** set both WordPress Address and Site Address to `https://`, then force the redirect at +the host or in `.htaccess`. diff --git a/skills/aeo-llms-txt/SKILL.md b/skills/aeo-llms-txt/SKILL.md new file mode 100644 index 0000000..f51ea71 --- /dev/null +++ b/skills/aeo-llms-txt/SKILL.md @@ -0,0 +1,151 @@ +--- +name: aeo-llms-txt +description: Create, fix or decide whether to publish an llms.txt file (and llms-full.txt) for a site. Use when someone asks "add llms.txt", "do I need llms.txt", "does ChatGPT read llms.txt", "generate llms.txt", "llms.txt showing up in Google", or when the audit fails tech.llms-txt-present or aeo.llms-txt-present. Covers the llmstxt.org format, honest adoption status, serving it with noindex, and generating it from source instead of by hand. Uses the aeo-visibility MCP server if connected; works without it. +--- + +# llms.txt + +`llms.txt` is a Markdown file at your site root that gives language models a short, curated map +of your content. It is cheap to publish. **No major answer engine has confirmed that it reads +it for ranking or citation**, so treat it as low-cost groundwork, not a lever. Say that to the +user before they expect traffic from it. + +## When this matters / when it doesn't + +Worth doing when: + +- The site is developer docs or an API reference. This is where the file is actually used: + coding agents fetch it to find the right pages, and the spec's own author says documentation + is its heaviest use. +- You already generate content from structured sources, so producing the file is nearly free. +- A tool or agent your users run is known to read it (some documentation platforms and agent + tools do). Ask which. + +Not worth much effort when: + +- The goal is "show up in ChatGPT / Google AI Overviews". Google's John Mueller has said + publicly (June 2025, repeated since) that no AI system currently uses llms.txt, and Google's + AI features guidance says there are no extra requirements to appear in AI Overviews beyond + normal SEO. Do not promise a result. Time spent on server rendering, bot access and answer-first + content pays back far more (`aeo-server-rendering`, `aeo-ai-crawler-access`, `aeo-answer-first-content`). +- The site is five pages. A sitemap and good headings already say the same thing. + +The audit still scores the file (see Detect). Passing the rule and the file being used are +different things; be straight about that when reporting. + +## Detect + +Rules owned: `tech.llms-txt-present` and `aeo.llms-txt-present`. Both pass when `/llms.txt` +answers 2xx with a non-HTML content type. The crawler also probes `/llms-full.txt` and records +it, but no rule scores it. An SPA that answers `/llms.txt` with its HTML shell fails, which is +correct: that is not the file. + +With the MCP server: `analyze_website_aeo` returns the crawl-hint files it found. + +Without it: + +```bash +curl -sI https://example.com/llms.txt | head -8 # 200, content-type text/plain or text/markdown +curl -s https://example.com/llms.txt | head -20 # starts with "# Site name" +curl -sI https://example.com/llms.txt | grep -i x-robots-tag +``` + +## Decide + +1. **Should we publish one?** If the answer is "to get cited", explain the evidence above and let + the owner decide. Publishing is harmless, just not a priority. +2. **Site-wide or per-section?** The spec allows `/llms.txt` or a file at any subpath (for + example `/docs/llms.txt` covering only `/docs/`). When several apply, agents should use the + most specific one. A big site with one documentation area can start there. +3. **Is `llms-full.txt` needed?** It is a common convention (one file holding the full text of + the key pages) used by documentation platforms, but the current llmstxt.org spec page does not + define it. Only add it for docs sites, and generate it, never hand-write it. A multi-megabyte + file defeats the point of a short map. + +## Implement + +**Format** (llmstxt.org, current version). A Markdown file with these parts, in order: + +1. An H1 with the site or project name. **The only required part.** +2. A blockquote with a short summary. +3. Optional paragraphs or lists of any kind except headings, with more detail. +4. Zero or more H2 sections, each a list of `- [Link title](url): optional notes`. + +An H2 named `Optional` is, by convention, secondary material an agent can skip when it needs a +shorter context. The proposal also suggests serving a clean Markdown version of pages at the +same URL plus `.md`, and linking it with `rel="alternate" type="text/markdown"`. + +```markdown +# Example Co + +> Example Co makes invoicing software for small agencies in Canada. + +Pricing is per seat, billed monthly in CAD. Docs are versioned; prefer the latest. + +## Docs +- [Quickstart](https://example.com/docs/quickstart): Create an account and send a first invoice +- [API reference](https://example.com/docs/api): REST endpoints and auth + +## Policies +- [Pricing](https://example.com/pricing): Plans and what each includes + +## Optional +- [Changelog](https://example.com/changelog) +``` + +Rules of thumb that follow from the format: only list URLs that return 200 and are canonical; +write link notes a person could act on; keep it short enough to fit comfortably in a context +window; never put marketing copy in the summary. + +**Generate it from source. Do not hand-maintain it.** A hand-written file goes stale the first +time a page moves, and the audit will not tell you. Build it in the same step that builds your +sitemap, from the same content list, so the two cannot disagree. Where a manifest of pages +already exists (CMS, MDX frontmatter, route list), loop over it. Per-stack code: +`references/nextjs.md`, `references/astro.md`, `references/static-html.md` (also WordPress). + +The Advance Labs `create-llms-txt` npm package (`npm create llms-txt`) writes a starter file by +reading a project's routes, supports Next, Astro, SvelteKit, Nuxt, WordPress themes and plain +HTML, and never overwrites an existing file. It is separate from this toolkit's packages and its +output is a starting point: review the summary and prune it. Its output includes extra sections +beyond the minimal format above. + +**Serve it with `X-Robots-Tag: noindex`.** Otherwise Google can index the file, and because it +repeats your page titles and descriptions it can compete with the real pages in results. We saw +this on advancelabs.dev: our own `/llms.txt` was showing up for queries that our llms.txt +guide pages should have won. John Mueller has said that noindex for it "could make sense", +because sites may link to it and it could otherwise be indexed, which would be odd for users. +That is advice, not a rule, and it is a trade-off, not a requirement: noindex only affects search +indexes. it does not stop an AI agent fetching and reading the file, and the file's purpose is not search. + +- Use the HTTP header, since a `.txt` file has no `` for a meta tag. +- Do **not** also `Disallow: /llms.txt` in robots.txt. Blocking it hides the file from the agents + you wrote it for, and a crawler that cannot fetch the file cannot see the noindex either. +- Content type: `text/plain; charset=utf-8` or `text/markdown`. Not `text/html`. + +## Verify + +1. `curl -sI https://example.com/llms.txt` shows `200`, a non-HTML content type, and + `x-robots-tag: noindex`. +2. Every link in the file resolves: `grep -o 'https\?://[^) ]*' llms.txt | sort -u | xargs -n1 curl -s -o /dev/null -w "%{http_code} %{url}\n"` shows no 3xx/4xx. +3. Re-run `analyze_website_aeo`. Both llms.txt rules pass. +4. After a content change and rebuild, the file changed with it. If it did not, generation is not wired in. +5. In Search Console, `gsc_url_inspect` on `https://example.com/llms.txt` after Google recrawls should show + it is excluded by noindex. It can take weeks. + +## Pitfalls + +- A catch-all SPA route returning the app shell for `/llms.txt`. +- Hand-maintained files that rot. Generate. +- Listing noindex, redirected or 404 URLs. The file should point only at pages you want read. +- Treating it as a robots.txt substitute. It permits nothing and blocks nothing. +- Putting private or paywalled URLs in the file. +- Making `llms-full.txt` enormous. Include the pages that answer real questions, not the site. +- Over-claiming. "We added llms.txt, so AI will find us" is not supported by evidence. + +## Sources + +- llmstxt.org specification, v2, modified August 10, 2026 (format, subpath files, "Optional" section, `.md` page variants; the page does not define llms-full.txt): https://llmstxt.org/ (verified 2026-11-13) +- John Mueller on llms.txt and noindex, as reported by Search Engine Roundtable (primary post is on Bluesky and could not be fetched here): https://www.seroundtable.com/google-noindex-llms-txt-39798.html and https://www.seroundtable.com/google-ai-llms-txt-39607.html (verified 2026-11-13 via search results and one page fetch; secondary reporting) +- Google, AI features and your website (no additional requirements for AI Overviews): https://developers.google.com/search/docs/appearance/ai-features (verified 2026-11-13) +- npm package `create-llms-txt` 1.0.0 (published 2026-09-24 by Advance Labs Inc.): https://www.npmjs.com/package/create-llms-txt (verified 2026-11-13 via the npm registry) +- Adoption claims in the llmstxt.org page (thousands of sites, OpenAI/Anthropic/Gemini publishing their own) are the spec author's, not independently checked. diff --git a/skills/aeo-llms-txt/references/astro.md b/skills/aeo-llms-txt/references/astro.md new file mode 100644 index 0000000..7c63459 --- /dev/null +++ b/skills/aeo-llms-txt/references/astro.md @@ -0,0 +1,37 @@ +# Astro 5 + +```ts +// src/pages/llms.txt.ts -> /llms.txt, generated at build time +import { getCollection } from 'astro:content' +import type { APIRoute } from 'astro' + +export const GET: APIRoute = async ({ site }) => { + const posts = await getCollection('docs') + const body = [ + '# Example Co', + '', + '> Example Co makes invoicing software for small agencies in Canada.', + '', + '## Docs', + ...posts.map((p) => `- [${p.data.title}](${new URL(`/docs/${p.id}/`, site).href}): ${p.data.description ?? ''}`), + '', + ].join('\n') + return new Response(body, { headers: { 'Content-Type': 'text/plain; charset=utf-8' } }) +} +``` + +Static output cannot set response headers at build time, so `X-Robots-Tag` has to come from the +host. Put it in the host's headers file or config: + +- Netlify and Cloudflare Pages: a `public/_headers` file containing + +``` +/llms.txt + X-Robots-Tag: noindex +``` + +- Vercel: `vercel.json` `headers` entry for `/llms.txt`. +- Self-hosted nginx/Apache: add the header on that location. + +(The `_headers` file format is Netlify's and Cloudflare Pages'; check your host's docs if it +is something else.) diff --git a/skills/aeo-llms-txt/references/nextjs.md b/skills/aeo-llms-txt/references/nextjs.md new file mode 100644 index 0000000..131e403 --- /dev/null +++ b/skills/aeo-llms-txt/references/nextjs.md @@ -0,0 +1,42 @@ +# Next.js (App Router) + +A route handler at `app/llms.txt/route.ts` builds the file from the same source as the sitemap. + +```ts +// app/llms.txt/route.ts +import { getAllPages } from '@/lib/content' // your own manifest: { title, path, summary }[] + +export const dynamic = 'force-static' // build once; use revalidate for ISR + +export async function GET() { + const pages = await getAllPages() + const body = [ + '# Example Co', + '', + '> Example Co makes invoicing software for small agencies in Canada.', + '', + '## Docs', + ...pages.map((p) => `- [${p.title}](https://example.com${p.path}): ${p.summary}`), + '', + ].join('\n') + + return new Response(body, { + headers: { + 'Content-Type': 'text/plain; charset=utf-8', + 'X-Robots-Tag': 'noindex', + }, + }) +} +``` + +Share `getAllPages()` with `app/sitemap.ts` so they cannot drift. + +If you prefer a static file in `public/llms.txt`, add the header in `next.config.ts`: + +```ts +async headers() { + return [{ source: '/llms.txt', headers: [{ key: 'X-Robots-Tag', value: 'noindex' }] }] +} +``` + +Check auth/redirect middleware does not catch `/llms.txt`. diff --git a/skills/aeo-llms-txt/references/static-html.md b/skills/aeo-llms-txt/references/static-html.md new file mode 100644 index 0000000..2fd9722 --- /dev/null +++ b/skills/aeo-llms-txt/references/static-html.md @@ -0,0 +1,41 @@ +# Plain HTML, WordPress and other hosts + +**Plain HTML / any build tool.** Write `llms.txt` at the web root, or better, generate it in the +build step from the same page list as `sitemap.xml`. A 20-line script that walks your content +folder and writes `dist/llms.txt` is enough. + +**Header.** Add `X-Robots-Tag: noindex` for `/llms.txt` at the host: + +```nginx +location = /llms.txt { add_header X-Robots-Tag "noindex"; } +``` + +```apache + + Header set X-Robots-Tag "noindex" + +``` + +Netlify / Cloudflare Pages: `_headers` file, see `astro.md`. Vercel: `vercel.json` headers. + +**WordPress.** A physical `llms.txt` in the web root is served as-is; add the header at the +server as above. To generate dynamically, serve it from a rewrite and emit the header in PHP: + +```php +add_action( 'template_redirect', function () { + if ( untrailingslashit( parse_url( $_SERVER['REQUEST_URI'], PHP_URL_PATH ) ) !== '/llms.txt' ) { + return; + } + status_header( 200 ); + header( 'Content-Type: text/plain; charset=utf-8' ); + header( 'X-Robots-Tag: noindex' ); + echo "# " . get_bloginfo( 'name' ) . "\n\n> " . get_bloginfo( 'description' ) . "\n\n## Pages\n"; + foreach ( get_pages() as $p ) { + echo '- [' . $p->post_title . '](' . get_permalink( $p ) . ")\n"; + } + exit; +} ); +``` + +A physical file is simpler and more predictable than the dynamic route. Some SEO plugins may +generate this file themselves (not verified here); if one does, use it and add the header. diff --git a/skills/aeo-local-business-schema/SKILL.md b/skills/aeo-local-business-schema/SKILL.md new file mode 100644 index 0000000..2de3bed --- /dev/null +++ b/skills/aeo-local-business-schema/SKILL.md @@ -0,0 +1,130 @@ +--- +name: aeo-local-business-schema +description: Add or fix LocalBusiness structured data and keep name, address and phone (NAP) consistent across a site, Google Business Profile and directories, so local searchers and AI answer engines get the right facts. Use when someone asks about "local SEO schema", "LocalBusiness JSON-LD", "opening hours markup", "NAP consistency", "service area", "show up for plumber near me in ChatGPT", or wrong hours or address appearing in answers. Works for WordPress and static HTML. Uses the aeo-visibility MCP server (analyze_website_aeo, check_ai_visibility) if connected. +--- + +# LocalBusiness schema and NAP consistency + +For a business with a physical location or service area, the facts that matter are name, +address, phone, hours, and where it serves. This skill puts those facts in markup and makes them +agree everywhere they appear. + +**Note:** the scoring engine has no local-business rule yet. `analyze_website_aeo` will not fail +a site for missing LocalBusiness markup. Detection uses its structured-data findings plus manual +inspection. + +## When this matters / when it doesn't + +Matters for businesses customers visit or that travel to customers (trades, clinics, restaurants, +shops). + +Does not apply, or needs care, when: +- The business is online-only with no location or service area. Use Organization markup + (`aeo-organization-entity`), not LocalBusiness. +- The owner does not want the address public (home-based service business). Google Business + Profile guidance is that service-area businesses "should hide your business address". Do not put + a home address in markup either. Ask. +- Reviews are the goal. Self-reviews do not earn stars (see Pitfalls). + +Markup is not what makes a business appear in local results. Google says local results rely +mainly on relevance, distance and prominence, and that "there's no way to request or pay for a +better local ranking". Do the work because accurate facts help users and engines, not as a +ranking lever. + +## Detect + +1. `analyze_website_aeo` on the homepage and contact page; read structured-data findings. +2. View source for `ld+json`. Look for a LocalBusiness (or subtype) node, `address`, + `telephone`, `openingHoursSpecification`. +3. Collect the business's name, address and phone from: the site footer, the contact page, the + schema, Google Business Profile, and two or three directories. Put them in a table and + compare character by character (suite numbers, "St" vs "Street", phone format, business name + variants). +4. Ask the owner which version is true. Never decide this yourself. + +## Decide + +- **Subtype:** Google says use "the most specific LocalBusiness sub-type possible". schema.org + has many (Store, Dentist, LegalService, FoodEstablishment, HomeAndConstructionBusiness, + MedicalBusiness and others). For several types, use an array in `@type`; Google notes + `additionalType` isn't supported. +- **Storefront or service area?** Storefront: show the address. Service-area business: hide the + address in Business Profile, and describe the area in text and with `areaServed`. Note that + Google's LocalBusiness property tables do not list `areaServed`; it is valid schema.org, so it + is harmless and helps other consumers, but do not expect a Google feature from it. +- **Multiple locations:** one node per location, each with its own page, phone and hours. +- **Departments:** nest as `department`, named `{store name} {department name}` unless the + department has its own brand. + +Google's properties: + +| | Properties | +|---|---| +| Required | `name`, `address` (PostalAddress, as complete as possible) | +| Recommended | `telephone` (with country and area code, e.g. +12122459600), `url`, `geo` (latitude and longitude with at least 5 decimal places), `openingHoursSpecification`, `priceRange` (under 100 characters), `department`; `aggregateRating`/`review` only for sites that capture reviews of *other* local businesses | + +## Implement + +1. One LocalBusiness JSON-LD block on the homepage or contact page (and each location page). + Use the same string for `name` as on the storefront and the Business Profile. +2. Hours, per Google's format: + ```json + "openingHoursSpecification": [ + { "@type": "OpeningHoursSpecification", + "dayOfWeek": ["Monday","Tuesday","Wednesday","Thursday","Friday"], + "opens": "09:00", "closes": "17:00" } + ] + ``` + Omit `validFrom` and `validThrough` for year-round hours. Closing past midnight: one spec, for + example `opens: "18:00"`, `closes: "03:00"`. Closed all day: `00:00` and `00:00`. Seasonal + closure: `00:00`/`00:00` with `validFrom` and `validThrough` in YYYY-MM-DD. +3. Put the same NAP in visible HTML (footer or contact page). Markup must describe visible + content. +4. **Google Business Profile.** Claim and verify it; this is separate from your site and drives + the map listing. Rules from Google's guidelines: the name "should reflect your business's + real-world name, as used consistently on your storefront, website, stationery"; no keywords, + taglines, or phone numbers in the name; a precise address, no P.O. boxes or virtual offices; + service-area businesses may have one profile with a service area (not much beyond about two + hours' drive) and should hide a home address; use a local phone number and the + location-specific website URL; choose few, specific categories. Keep hours, including holiday + hours, current. Respond to reviews. +5. **Directories.** Update the main ones to the confirmed NAP. Google lists prominence factors + as including links and reviews; it does not publish a "citations" rule, so do not claim that + directory listings are a ranking factor. Consistent real-world facts are the goal. + +Stack specifics: `references/wordpress.md`, `references/static-html.md`. + +## Verify + +- Rich Results Test and validator.schema.org: no errors; warnings reviewed. +- Rendered hours and phone in the markup equal the visible page and the Business Profile. +- Run `check_ai_visibility` with **location-specific prompts** written like a customer, for + example "emergency plumber in Guelph Ontario" or "best dentist near Westboro Ottawa". The + tool has no location parameter; the place name must be inside the prompt. It queries + Perplexity Sonar only and needs a Perplexity key. Each run is one sample, so repeat prompts, + record which sources are cited, and compare before and after rather than claiming a cause. + Use `compare_competitor_visibility` with the business and two or three local rivals to see who + is cited. +- Search a few prompts yourself and read whether the engine states the right hours and address. + Wrong facts there usually trace to an outdated directory, not the site. + +## Pitfalls + +- Name variants ("Smith Plumbing", "Smith Plumbing Ltd.", "Smith Plumbing Guelph") across + sources. +- Stuffing keywords or the city into the business name; it can get the Business Profile suspended. +- Markup for a location the business does not staff. +- Hours in markup that ignore holidays or differ from the profile. +- Self-serving reviews: if the business controls the reviews about itself, its + `LocalBusiness`/`Organization` pages are ineligible for star results, including via an + embedded third-party widget. Do not add `aggregateRating` to your own business node. +- A home address published against the owner's wishes. +- Several LocalBusiness nodes on one page that conflict. + +## Sources + +- Local business structured data: https://developers.google.com/search/docs/appearance/structured-data/local-business (verified 2026-11-13) +- Review snippet (self-serving reviews): https://developers.google.com/search/docs/appearance/structured-data/review-snippet (verified 2026-11-13) +- Business Profile guidelines: https://support.google.com/business/answer/3038177 (verified 2026-11-13) +- Tips to improve local ranking: https://support.google.com/business/answer/7091 (verified 2026-11-13) +- schema.org LocalBusiness: https://schema.org/LocalBusiness ; areaServed: https://schema.org/areaServed ; OpeningHoursSpecification: https://schema.org/OpeningHoursSpecification (verified 2026-11-13) diff --git a/skills/aeo-local-business-schema/references/static-html.md b/skills/aeo-local-business-schema/references/static-html.md new file mode 100644 index 0000000..2951f2c --- /dev/null +++ b/skills/aeo-local-business-schema/references/static-html.md @@ -0,0 +1,35 @@ +# Static HTML + +Add to the `` (or end of ``) of the homepage and each location page: + +```html + +``` + +All values are placeholders: replace with the confirmed details. Service-area business with a +hidden address: omit `streetAddress` only if the owner agrees and keep `addressLocality` and +`addressRegion`; Google requires `address`, so give as much as the owner is willing to publish. +Remember to update the block whenever hours change. diff --git a/skills/aeo-local-business-schema/references/wordpress.md b/skills/aeo-local-business-schema/references/wordpress.md new file mode 100644 index 0000000..37b9e44 --- /dev/null +++ b/skills/aeo-local-business-schema/references/wordpress.md @@ -0,0 +1,13 @@ +# WordPress + +I could not verify specific plugin documentation on 2026-11-13, so check each plugin's current +docs before following a recipe. General approach: + +1. View source on the homepage and contact page; list every `ld+json` block. SEO plugins and + themes often each emit an Organization or LocalBusiness node. +2. Keep one source. If an SEO plugin has a local-business setting, fill it with the confirmed + NAP and hours and disable the theme's duplicate. If you prefer manual control, disable the + plugin's local schema and paste one block (see static-html.md) via a header snippet or a + per-page custom field. +3. Make sure the footer or contact page shows the same NAP as visible text. +4. After changes, clear page cache and re-test the live URL, not the editor preview. diff --git a/skills/aeo-organization-entity/SKILL.md b/skills/aeo-organization-entity/SKILL.md new file mode 100644 index 0000000..41f995b --- /dev/null +++ b/skills/aeo-organization-entity/SKILL.md @@ -0,0 +1,231 @@ +--- +name: aeo-organization-entity +description: Publish one consistent Organization (or Person) entity in JSON-LD, with a stable @id, logo and sameAs links, so answer engines can tell who is behind a site. Use when someone asks to add Organization schema, fix "Organization schema missing" or "organization identity inconsistent" audit findings, set up sameAs or a Wikidata/Knowledge Graph presence, make ChatGPT or Perplexity recognise their brand, or stop two subdomains describing the same company differently. Covers aeo.organization-schema, aeo.entity-identity-consistent and eeat.auth.organization-schema. Uses the aeo-visibility MCP server if connected; works without it. +--- + +# One organization, described the same way everywhere + +Organization markup tells engines which real-world entity owns a site: its name, its URL, its +logo, and the other places it exists online (`sameAs`). The audit passes when an Organization is +present, and fails a separate rule when two first-party nodes contradict each other. The +second failure is the more damaging one and the easier one to cause. + +## When this matters / when it doesn't + +**It matters** for any site that represents a company, product, publication or institution, +especially one with several subdomains or several apps that each ship their own markup. + +**It matters less than you might think for rich results.** Google's Organization documentation +lists no required properties, and says the markup does not need to be on every page; the +homepage or a single About page is enough. Do not promise a visible search feature from it. +What it buys is disambiguation: a consistent entity with corroborating `sameAs` links is +easier for a model or knowledge graph to resolve and attribute. + +**It is not the right entity when the site is a person.** A personal portfolio, a freelancer or +a solo author blog is about a `Person`. Marking a person up as `Organization` is wrong, and +Google's author guidance says not to use the wrong type. Use a `Person` node with the same +`@id` and `sameAs` discipline, and drop the Organization. + +**Do not invent facts to fill the schema.** No `sameAs` link to a profile that does not exist, +no `foundingDate` you are guessing at, no logo the company does not use. + +## Detect + +Rules this skill owns, and what they actually check: + +| Rule | Passes when | +|---|---| +| `aeo.organization-schema` | Any crawled page has a node whose type is `Organization` or `LocalBusiness` (found at any nesting depth, JSON-LD, Microdata or RDFa). | +| `eeat.auth.organization-schema` | Same detection, in the E-E-A-T pillar. | +| `aeo.entity-identity-consistent` | Fewer than two first-party `Organization` nodes exist, or all of them share one `@id` and one `url`. | + +Details that change how you read a result: + +- **Subtypes are not all detected.** `Corporation`, `NGO`, `NewsMediaOrganization`, + `OnlineBusiness` and the other schema.org subtypes are valid, but the audit only counts the + exact short type `Organization` plus `LocalBusiness`. A site marked up as `@type: "Corporation"` + alone can fail the presence rule while being correct schema.org. Declaring a type array such as + `["Organization", "Corporation"]` satisfies presence, but the consistency rule compares only + nodes whose type is exactly `Organization`, so such a node is skipped there. +- **"First-party" means the host matches.** Nodes whose `@id` or `url` is on another domain + (a client on a case-study page) are ignored by the consistency rule. Subdomains of the + audited host count as the same site, which is the point: `app.example.com` and `example.com` + must agree. +- **`@id` comparison keeps the fragment.** `/#organization` and `/#org` are different ids and + the rule reports that as a conflict. A trailing-slash or `www` difference in `url` is + normalised, so those alone are not a failure. +- A missing `@id` is not reported by the consistency rule. Add one anyway. + +With the MCP server: call `analyze_website_aeo` on the homepage and a second subdomain if there +is one, and read the three rule results. The `detail` string on the consistency rule says +whether the disagreement is in `@id` or `url`. + +Without it, fetch each page and list the Organization nodes: + +```bash +curl -s https://example.com/ | grep -o ' +``` + +Not verified: whether your theme already emits this (many do, but check the rendered page), and +which Shopify fields map to GTIN and brand. Inspect the output for one product and compare it to +the product's admin fields before relying on it. + +Steps: +1. View source on a product page, search `ld+json`, count Product or ProductGroup nodes. +2. If a theme and an app (reviews, SEO) both output one, disable one. Reviews apps often add + `aggregateRating`; confirm those ratings are real and visible on the page. +3. If missing, add the snippet above to the product template (`main-product` section or + `product.liquid`, depending on theme). +4. Add GTIN and brand data in the product admin, then check they appear in the output. If + they do not, extend the JSON-LD in Liquid, and keep values from the product fields, not + hard-coded. diff --git a/skills/aeo-product-schema/references/woocommerce.md b/skills/aeo-product-schema/references/woocommerce.md new file mode 100644 index 0000000..b2e1d4a --- /dev/null +++ b/skills/aeo-product-schema/references/woocommerce.md @@ -0,0 +1,22 @@ +# WooCommerce + +Verified 2026-11-13 from the WooCommerce code reference +(https://woocommerce.github.io/code-reference/classes/WC-Structured-Data.html): `WC_Structured_Data` +generates Product data (`generate_product_data`, hooked to `woocommerce_single_product_summary`) +and outputs it in the footer. Default types include product, review, breadcrumblist, website and +order. It validates GTINs (8, 12, 13 or 14 digits), and on variable products it uses the selected +variation only when the request identifies exactly one purchasable variation; otherwise it falls +back to the parent product's `AggregateOffer`. + +Not verified: the exact filter names for customising the output. Search the plugin source for +`apply_filters` in `class-wc-structured-data.php` rather than trusting a remembered name. Also not +verified: how Yoast, Rank Math or other SEO plugins interact; test it. + +Steps: +1. Check the rendered product page for duplicate Product nodes (WooCommerce plus an SEO plugin). + Keep one source. +2. Confirm price and stock status in the JSON-LD match the page, including on sale and + out-of-stock products. +3. Fill the GTIN/identifier field if your WooCommerce version has one; otherwise add it through + a documented extension point. Never fill with made-up values. +4. If a reviews plugin adds ratings, confirm they come from real customers and show on the page. diff --git a/skills/aeo-server-rendering/SKILL.md b/skills/aeo-server-rendering/SKILL.md new file mode 100644 index 0000000..7b8b19f --- /dev/null +++ b/skills/aeo-server-rendering/SKILL.md @@ -0,0 +1,125 @@ +--- +name: aeo-server-rendering +description: Make sure a page's real content is in the HTML the server sends, not built by JavaScript afterward. Use when someone asks "why can't ChatGPT or Perplexity read my site", "AI crawlers see an empty page", "do I need SSR for AEO", "my React or Vite SPA isn't being cited", "prerender my site", "view source is empty", or when the audit fails aeo.content-server-rendered. Covers testing with curl, SSR/SSG/prerender per framework, and content that only appears after hydration (tabs, accordions, client-fetched text). Uses the aeo-visibility MCP server if connected; works without it. +--- + +# Content must be in the initial HTML + +Many AI crawlers read the HTML they are given and do not run JavaScript. If a page is an empty +`

      ` until a script fills it, those crawlers see nothing to cite. The fix is to send +the content in the first response: server-render it, generate it at build time, or prerender it. + +## When this matters / when it doesn't + +It matters for any page you want cited: articles, docs, product and pricing pages, about pages. + +It matters less, or not at all, for: + +- Logged-in app screens, dashboards, checkout. Nobody expects those to be cited. +- Interactive widgets on an otherwise server-rendered page (a calculator, a chat bubble). Their + absence from the HTML is fine. The surrounding text is what counts. +- **Googlebot specifically.** Google says it renders JavaScript with an evergreen Chromium, after + a queue. A client-rendered page can rank on Google and still be invisible to other crawlers. + Do not tell someone their site is "not indexed" because of this finding: the claim is about + AI crawlers and other engines, not Google. + +How sure are we that AI crawlers skip JavaScript? The main public evidence is a Vercel and MERJ +analysis from December 2024, which found no JavaScript execution from OpenAI's crawlers +(including OAI-SearchBot and ChatGPT-User), ClaudeBot, Meta's crawler or Bytespider. It does not +show PerplexityBot's behaviour. Crawlers change, and none of the vendors guarantee either +behaviour. Server-rendering is cheap insurance, not a proven requirement for every engine. + +## Detect + +Rule owned: `aeo.content-server-rendered`. It fails when a page has **both**: + +1. An empty app shell: a known mount point (`#root`, `#app`, `#__next`, `#__nuxt`, `#nuxt`, `#___gatsby`, `#q-app`, `#svelte`, `[data-reactroot]`, `[data-server-rendered]`) with no visible text + (3 words or fewer), **and at least one `` in a string cannot break out of the tag. +3. **Missing required property.** Output the property from real data, or omit the whole node + when the data does not exist. Do not emit empty strings. +4. **Duplicate and conflicting blocks.** Identify every source (view-source, search for + `ld+json`; note which are inside the theme, which carry plugin signatures such as + `yoast-schema-graph`). Keep one emitter per entity type. Disable the others rather than + hand-merging. If the nodes are identical, duplication is harmless but wasteful; if they + differ, it is the defect. +5. **`@id` collisions.** `@id` identifies a node, and Google's guidelines use `@id` to link + related items. Two different things sharing an `@id` (for example the `WebPage` and the + Organization both using `https://example.com/`) are ambiguous. Use distinct fragments: + `/#organization`, `/#website`, `/page#webpage`, `/page#article`. A given `@id` should mean the + same entity wherever it appears, and different entities should never share one. +6. **Wrong value types.** Dates as ISO 8601, `url`/`image` as absolute URLs, numbers as + numbers when a number is expected, `sameAs` as an array of URL strings. +7. **Prefer JSON-LD.** Google supports JSON-LD, Microdata and RDFa equally if valid, and + recommends JSON-LD where the setup allows. Do not convert working Microdata for its own sake. + Mixing JSON-LD and Microdata for the same entity is where duplicates and conflicts start. +8. **Keep it connected.** One `@graph` per page with stable `@id`s beats a pile of + disconnected blobs. See `aeo-organization-entity` and `aeo-article-author-schema` for the + node shapes. + +Stack notes: `references/nextjs.md`, `references/wordpress.md`. Plain HTML and Astro have +no stack-specific causes beyond the above; see the recipes in the other schema skills. + +## Verify + +1. Re-run `analyze_website_aeo` or the validator script on each page type (home, article, + product, FAQ). `tech.structured-data-present` and `tech.structured-data-valid` should + pass, and no item should be `(unparsed)`. +2. Count the blocks: `curl -s URL | grep -c 'application/ld+json'`. It should equal the number + you intend. +3. Run the Rich Results Test and the Schema Markup Validator on one URL per template. Errors + must be zero; read warnings and decide. +4. Spot-check that markup matches the visible page. +5. After deployment, check Search Console for the same URL if available. + +## Pitfalls + +- **Validating the template, not the output.** Test the rendered HTML, not the source file. +- **Testing a pasted snippet instead of the live URL.** The live page may carry extra blocks. +- **Caching.** A CDN or page cache can serve old markup after the fix. +- **Staging identities.** `@id` and `url` generated from the request host publish preview + domains. +- **Trusting the green tick.** Both the toolkit and the external tools check syntax and + declared requirements, not truth. +- **Fixing warnings by adding invented data.** +- **Assuming a deprecated feature is a markup error.** See `aeo-faq-qa-schema`. + +## Sources + +- Google, intro to structured data (formats; Rich Results Test; JSON-LD recommended): https://developers.google.com/search/docs/appearance/structured-data/intro-structured-data (verified 2026-11-13) +- Google, structured data general guidelines: https://developers.google.com/search/docs/appearance/structured-data/sd-policies (verified 2026-11-13) +- Google, Rich Results Test: https://search.google.com/test/rich-results (verified 2026-11-13) +- Schema Markup Validator: https://validator.schema.org/ (verified 2026-11-13, page loads; no description on the page) +- schema.org data model (`@id` in JSON-LD): https://schema.org/docs/datamodel.html (verified 2026-11-13) +- Toolkit: `packages/schema-validator/src/{validation,json-ld,analyze}.ts`, `packages/scoring/src/technical-seo-rules.ts` (read 2026-11-13) diff --git a/skills/aeo-structured-data-validation/references/nextjs.md b/skills/aeo-structured-data-validation/references/nextjs.md new file mode 100644 index 0000000..c77899f --- /dev/null +++ b/skills/aeo-structured-data-validation/references/nextjs.md @@ -0,0 +1,15 @@ +# Structured data validation in Next.js + +- Emit JSON-LD as a native `