Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,7 @@ The framework exposes machine-readable docs in Next.js, with sitemap routes avai
- `/.well-known/mcp`
- `/docs/<slug>.md`
- `/docs/<slug>` with `Accept: text/markdown`
- `/docs/<slug>` with `Signature-Agent`

The canonical API routes remain available under `/api/docs`, including `/api/docs?format=skill`,
`/api/docs/mcp`, and `/api/docs/agent/spec`.
Expand Down
30 changes: 30 additions & 0 deletions examples/next/app/docs/getting-started/agent-ready-docs/agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,9 @@ Keep this contract true:

- `/docs/getting-started/agent-ready-docs` renders the human HTML page
- `/docs/getting-started/agent-ready-docs.md` returns this file
- `/docs/getting-started/agent-ready-docs` with `Signature-Agent` returns this file
- pages without `agent.md` should still work through `/docs/<slug>.md`
- pages without `agent.md` should also work through `/docs/<slug>` with `Signature-Agent`
- the human page remains readable and does not need to be rewritten for agents

## Primary Files
Expand All @@ -32,13 +34,16 @@ packages/fumadocs/src/docs-api.ts
```ts
"/docs.md" -> "/api/docs?format=markdown"
"/docs/:slug*.md" -> "/api/docs?format=markdown&path=:slug*"
"/docs" + Signature-Agent -> "/api/docs/markdown"
"/docs/:slug*" + Signature-Agent -> "/api/docs/markdown/:slug*"
```

For this page:

```txt
/docs/getting-started/agent-ready-docs
/docs/getting-started/agent-ready-docs.md
/docs/getting-started/agent-ready-docs with Signature-Agent
```

## Markdown Route Shape
Expand All @@ -55,11 +60,30 @@ export async function GET(request, { params }) {
}
```

## Signature-Agent Behavior

`Signature-Agent` lets agents request the canonical docs URL and still receive markdown when they do
not send `Accept: text/markdown`.

```bash
curl http://localhost:3000/docs/getting-started/agent-ready-docs \
-H "Signature-Agent: https://chatgpt.com"
```

Implementation contract:

- detect any non-empty `Signature-Agent` header
- only apply the markdown response under the configured docs entry route
- preserve request headers when forwarding through the generated markdown bridge route
- derive `path` from the canonical docs slug and call `/api/docs` with `format=markdown`
- keep normal browser requests on the HTML page

## Implementation Notes

- `contentDir` must fall back to `app/${entry}` in the Next example
- `withDocs()` should auto-generate the markdown bridge route and rewrites
- the existing `/api/docs` GET handler should own markdown mode
- `Signature-Agent` rewrites should target the generated markdown bridge route, not a `.md` URL
- normalize slug paths before matching page URLs
- use the shared docs page source for fallback markdown, so standard pages do not need extra setup
- fall back to normal page markdown when a page does not have `agent.md`
Expand Down Expand Up @@ -92,14 +116,20 @@ surface for both cases:
```bash
curl http://localhost:3000/docs/getting-started/agent-ready-docs
curl http://localhost:3000/docs/getting-started/agent-ready-docs.md
curl http://localhost:3000/docs/getting-started/agent-ready-docs \
-H "Signature-Agent: https://chatgpt.com"
curl http://localhost:3000/docs/getting-started/quickstart.md
curl http://localhost:3000/docs/getting-started/quickstart \
-H "Signature-Agent: https://chatgpt.com"
curl "http://localhost:3000/api/docs?format=markdown&path=getting-started/quickstart"
```

## Accept When

- `/docs/getting-started/agent-ready-docs.md` returns this file verbatim
- `/docs/getting-started/agent-ready-docs` with `Signature-Agent` returns this file verbatim
- `/docs/getting-started/quickstart.md` returns normal page markdown
- `/docs/getting-started/quickstart` with `Signature-Agent` returns normal page markdown
- `/docs/getting-started/agent-ready-docs` still renders the browser page
- MCP `read_page("/docs/getting-started/quickstart")` still returns normal docs content
- the implementation still works for pages that do not have `agent.md`
66 changes: 57 additions & 9 deletions examples/next/app/docs/getting-started/agent-ready-docs/page.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,19 @@ single docs topic in two different forms:

- `/docs/getting-started/agent-ready-docs` for human HTML
- `/docs/getting-started/agent-ready-docs.md` for the page's direct `agent.md`
- `/docs/getting-started/agent-ready-docs` with `Signature-Agent` for automatic markdown

Because this folder includes a sibling `agent.md`, the `.md` route returns that file. For pages
without `agent.md`, the same `.md` route falls back to the normal page markdown.
without `agent.md`, the same `.md` route and header-based markdown response fall back to the normal
page markdown.

## What To Open

1. Open `/docs/getting-started/agent-ready-docs` in the browser to see the full narrative.
2. Open `/docs/getting-started/agent-ready-docs.md` to see the focused agent payload.
3. Use your own prompt cases to compare the human page output and the `.md` agent payload.
4. Compare how much broader explanation humans get versus how much narrower implementation detail an agent gets.
3. Request `/docs/getting-started/agent-ready-docs` with `Signature-Agent` to get the same focused payload from the canonical URL.
4. Use your own prompt cases to compare the human page output and the markdown agent payload.
5. Compare how much broader explanation humans get versus how much narrower implementation detail an agent gets.

## Why It Matters

Expand All @@ -38,9 +41,11 @@ The example app exposes two ways to read the same docs topic:

- `/docs/<slug>` for human HTML pages
- `/docs/<slug>.md` for machine-readable markdown
- `/docs/<slug>` with `Signature-Agent` for agents that read canonical URLs

When a page folder has an `agent.md`, the `.md` route should return that file. When it does not,
the same route should fall back to the normal page markdown.
When a page folder has an `agent.md`, the `.md` route and `Signature-Agent` response should return
that file. When it does not, both markdown entry points should fall back to the normal page
markdown.

That fallback comes from the same docs source lookup used by the shared page reader, so pages
without `agent.md` do not need any extra configuration.
Expand All @@ -55,6 +60,7 @@ examples/next/app/api/docs/mcp/route.ts
examples/next/app/docs/getting-started/agent-ready-docs/page.mdx
examples/next/app/docs/getting-started/agent-ready-docs/agent.md
examples/next/docs.config.tsx
packages/docs/src/agent.ts
packages/next/src/config.ts
packages/next/src/api.ts
```
Expand All @@ -76,8 +82,8 @@ export default withDocs({
## Built-In Markdown Route

`withDocs()` wires the public `.md` URLs into the existing `/api/docs` handler. The public rewrite
can land on `/api/docs` with the original docs pathname preserved, so the shared docs API accepts
both direct markdown queries and rewritten page URLs like `/docs/getting-started/agent-ready-docs.md`:
lands on `/api/docs` with `format=markdown`, so the shared docs API accepts both direct markdown
queries and rewritten page URLs like `/docs/getting-started/agent-ready-docs.md`:

```ts title="packages/next/src/config.ts"
function buildDocsMarkdownRewrites(entry: string) {
Expand Down Expand Up @@ -109,6 +115,38 @@ if (markdownRequest) {
}
```

## Signature-Agent Auto-Rewrite

Some agents fetch the canonical page URL and identify themselves with `Signature-Agent` instead of
sending `Accept: text/markdown`. `withDocs()` handles that case automatically: the normal HTML URL
continues to work for browsers, while requests with the header are rewritten to a generated markdown
bridge route.

```bash title="terminal"
curl http://localhost:3000/docs/getting-started/agent-ready-docs \
-H "Signature-Agent: https://chatgpt.com"
```

```ts title="packages/next/src/config.ts"
{
source: `/${entry}/:slug*`,
has: [{ type: "header", key: "signature-agent" }],
destination: "/api/docs/markdown/:slug*",
}
```

The bridge route converts the slug back into the shared markdown API shape:

```ts title="app/api/docs/markdown/[[...slug]]/route.ts"
url.searchParams.set("format", "markdown");
if (slug) url.searchParams.set("path", slug);

return docsApi.GET(new Request(url.toString(), request));
```

No docs config flag is required. The header only changes docs page responses under the configured
entry route, so unrelated pages are not hijacked.

## MCP Route

The example app also exposes the standard MCP surface:
Expand Down Expand Up @@ -161,8 +199,14 @@ That is the whole pattern.

When a request comes in for `/docs/getting-started/agent-ready-docs.md`, `withDocs()` handles the
rewrite automatically into the existing `/api/docs` handler. The shared docs API accepts either the
explicit `format=markdown&path=...` query or the preserved rewritten `.md` pathname, then resolves
the requested slug and checks whether the page has `agentRawContent`.
explicit `format=markdown&path=...` query produced by the rewrite or a direct `.md` pathname in
adapters that pass the public URL through, then resolves the requested slug and checks whether the
page has `agentRawContent`.

When a request comes in for `/docs/getting-started/agent-ready-docs` with `Signature-Agent`,
`withDocs()` routes it through the generated markdown bridge route. That route preserves the request
headers, sets `format=markdown`, derives `path` from the current docs slug, and calls the same
`/api/docs` markdown handler.

Because this page has a sibling `agent.md`, the response becomes the raw contents of that file. If
another page does not have `agent.md`, the same route falls back to the normal page markdown.
Expand All @@ -172,6 +216,8 @@ another page does not have `agent.md`, the same route falls back to the normal p
```bash title="terminal"
curl http://localhost:3000/docs/getting-started/agent-ready-docs
curl http://localhost:3000/docs/getting-started/agent-ready-docs.md
curl http://localhost:3000/docs/getting-started/agent-ready-docs \
-H "Signature-Agent: https://chatgpt.com"
curl http://localhost:3000/docs/getting-started/quickstart.md
```

Expand All @@ -186,6 +232,8 @@ curl http://localhost:3000/api/docs?format=markdown\&path=getting-started/quicks
- In the Next example, `contentDir` should default to `app/${entry}` when the config does not set
it explicitly.
- The `.md` route is automatic in Next when you use `withDocs()`.
- `Signature-Agent` is automatic in Next when you use `withDocs()` and does not require a
`docs.config` option.
- `agent.md` should stay hidden from normal docs discovery and sidebar generation.
- When `agent.md` exists, it overrides the normal markdown for that slug.

Expand Down
6 changes: 3 additions & 3 deletions examples/next/app/docs/page.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@ Before implementing from this docs site, fetch `/api/docs/agent/spec` from the s
returned markdown route pattern, search endpoint, MCP endpoint, `llms.txt` routes, skills metadata,
locale config, and feedback endpoints instead of hard-coding defaults.

Prefer markdown reads over HTML scraping. For Next.js, `/docs.md`, `/docs/<slug>.md`, and
`Accept: text/markdown` on `/docs/<slug>` are the page-level machine-readable entry points when
reported by the spec.
Prefer markdown reads over HTML scraping. For Next.js, `/docs.md`, `/docs/<slug>.md`,
`Accept: text/markdown` on `/docs/<slug>`, and `Signature-Agent` on `/docs/<slug>` are the
page-level machine-readable entry points when reported by the spec.
</Agent>

## Features
Expand Down
37 changes: 37 additions & 0 deletions packages/docs/src/agent.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { describe, expect, it } from "vitest";
import {
buildDocsAgentDiscoverySpec,
findDocsMarkdownPage,
hasDocsMarkdownSignatureAgent,
isDocsAgentDiscoveryRequest,
isDocsMcpRequest,
isDocsPublicGetRequest,
Expand Down Expand Up @@ -83,6 +84,15 @@ describe("agent route helpers", () => {
{ sitemap: { routePrefix: "/docs" } },
),
).toBe(true);
expect(
isDocsPublicGetRequest(
"docs",
new URL("https://example.com/docs/install"),
new Request("https://example.com/docs/install", {
headers: { "Signature-Agent": "https://chatgpt.com" },
}),
),
).toBe(true);
expect(
isDocsPublicGetRequest(
"docs",
Expand All @@ -109,6 +119,23 @@ describe("agent route helpers", () => {
);
expect(acceptRoute).toEqual({ requestedPath: "install" });

const signatureAgentRoute = resolveDocsMarkdownRequest(
"docs",
new URL("https://example.com/docs/install"),
new Request("https://example.com/docs/install", {
headers: { "Signature-Agent": "https://chatgpt.com" },
}),
);
expect(signatureAgentRoute).toEqual({ requestedPath: "install" });

expect(
hasDocsMarkdownSignatureAgent(
new Request("https://example.com/docs/install", {
headers: { "Signature-Agent": "https://chatgpt.com" },
}),
),
).toBe(true);

const apiFormatRoute = resolveDocsMarkdownRequest(
"docs",
new URL("https://example.com/api/docs?format=markdown&path=install"),
Expand All @@ -122,6 +149,15 @@ describe("agent route helpers", () => {
new Request("https://example.com/blog?format=markdown&path=install"),
);
expect(hijackRoute).toBeNull();

const signatureAgentHijackRoute = resolveDocsMarkdownRequest(
"docs",
new URL("https://example.com/blog/install"),
new Request("https://example.com/blog/install", {
headers: { "Signature-Agent": "https://chatgpt.com" },
}),
);
expect(signatureAgentHijackRoute).toBeNull();
});

it("builds per-page markdown alternate URLs", () => {
Expand Down Expand Up @@ -222,6 +258,7 @@ describe("agent route helpers", () => {

expect(spec.api.agentSpecDefault).toBe("/.well-known/agent.json");
expect(spec.markdown.rootPage).toBe("/docs.md");
expect(spec.markdown.signatureAgentHeader).toBe("Signature-Agent");
expect(spec.llms.publicTxt).toBe("/llms.txt");
expect(spec.skills.file).toBe("skill.md");
expect(spec.skills.route).toBe("/skill.md");
Expand Down
19 changes: 18 additions & 1 deletion packages/docs/src/agent.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ export const DEFAULT_LLMS_FULL_TXT_WELL_KNOWN_ROUTE = "/.well-known/llms-full.tx
export const DEFAULT_SKILL_MD_ROUTE = "/skill.md";
export const DEFAULT_SKILL_MD_WELL_KNOWN_ROUTE = "/.well-known/skill.md";
export const DEFAULT_AGENT_FEEDBACK_ROUTE = "/api/docs/agent/feedback";
export const DOCS_MARKDOWN_SIGNATURE_AGENT_HEADER = "Signature-Agent";

export interface DocsAgentFeedbackDiscoveryConfig {
enabled?: boolean;
Expand All @@ -50,6 +51,7 @@ export interface DocsAgentDiscoverySpecOptions {
sitemap?: boolean | DocsSitemapConfig;
markdown?: {
acceptHeader?: boolean;
signatureAgentHeader?: boolean;
};
}

Expand All @@ -63,6 +65,7 @@ export interface DocsSkillDocumentOptions {
sitemap?: boolean | DocsSitemapConfig;
markdown?: {
acceptHeader?: boolean;
signatureAgentHeader?: boolean;
};
}

Expand Down Expand Up @@ -200,7 +203,7 @@ export function resolveDocsMarkdownRequest(
};
}

if (acceptsMarkdown(request)) {
if (acceptsMarkdown(request) || hasDocsMarkdownSignatureAgent(request)) {
if (pathname === normalizedEntry) {
return { requestedPath: "" };
}
Expand All @@ -215,6 +218,10 @@ export function resolveDocsMarkdownRequest(
return null;
}

export function hasDocsMarkdownSignatureAgent(request: Request): boolean {
return Boolean(request.headers.get(DOCS_MARKDOWN_SIGNATURE_AGENT_HEADER)?.trim());
}

export function findDocsMarkdownPage<T extends DocsMarkdownPage>(
entry: string,
pages: T[],
Expand Down Expand Up @@ -276,6 +283,8 @@ export function renderDocsSkillDocument({
`Use ${siteTitle} through markdown routes, llms.txt, agent discovery, search, and MCP when available.`,
);
const markdownAcceptHeader = markdown?.acceptHeader === false ? null : "text/markdown";
const markdownSignatureAgentHeader =
markdown?.signatureAgentHeader === false ? null : DOCS_MARKDOWN_SIGNATURE_AGENT_HEADER;
const lines = [
"---",
"name: docs",
Expand Down Expand Up @@ -306,6 +315,12 @@ export function renderDocsSkillDocument({
lines.push(`- You can also request ${markdownAcceptHeader} from normal page URLs.`);
}

if (markdownSignatureAgentHeader) {
lines.push(
`- Requests with ${markdownSignatureAgentHeader} on normal page URLs receive markdown automatically.`,
);
}

if (searchEnabled) {
lines.push(
`- Search with ${DEFAULT_DOCS_API_ROUTE}?query={query} when you do not know the page.`,
Expand Down Expand Up @@ -524,6 +539,8 @@ export function buildDocsAgentDiscoverySpec({
markdown: {
enabled: true,
acceptHeader: markdown?.acceptHeader === false ? null : "text/markdown",
signatureAgentHeader:
markdown?.signatureAgentHeader === false ? null : DOCS_MARKDOWN_SIGNATURE_AGENT_HEADER,
pagePattern: `/${normalizedEntry}/{slug}.md`,
rootPage: `/${normalizedEntry}.md`,
apiPattern: `${DEFAULT_DOCS_API_ROUTE}?format=markdown&path={slug}`,
Expand Down
2 changes: 2 additions & 0 deletions packages/docs/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,10 @@ export {
DEFAULT_MCP_WELL_KNOWN_ROUTE,
DEFAULT_SKILL_MD_ROUTE,
DEFAULT_SKILL_MD_WELL_KNOWN_ROUTE,
DOCS_MARKDOWN_SIGNATURE_AGENT_HEADER,
buildDocsAgentDiscoverySpec,
findDocsMarkdownPage,
hasDocsMarkdownSignatureAgent,
isDocsAgentDiscoveryRequest,
isDocsMcpRequest,
isDocsPublicGetRequest,
Expand Down
Loading
Loading