Document stopOnContent crawl option

[devin-ai-integration]

Summary

Documents the new opt-in stopOnContent crawler option from firecrawl/firecrawl#4740 (merge docs after that PR ships).

• api-reference/v2-openapi.json: adds stopOnContent (string[], 1-20 items, each 1-500 chars) to the POST /crawl request; monitor crawl target crawlOptions description now notes it accepts the same crawler options as /crawl, including stopOnContent.
• features/crawl.mdx, advanced-scraping-guide.mdx: scope bullet + config table rows.

Semantics documented (matching the API implementation): case-insensitive, whitespace-normalized match against visible content (script/style/template/noscript ignored); the matching page is still returned and billed, only its outgoing links are not followed. Not accepted on /map.

Example use case — soft-200 pagination that never 404s:

{ "url": "https://example.com/release-notes", "stopOnContent": ["No release notes found"] }

Localized files untouched per repo guidelines.

Link to Devin session: https://app.devin.ai/sessions/ff67a236ee6c4b289ce3c730bed121d7
Open in Devin Desktop: https://app.devin.ai/desktop/session/ff67a236ee6c4b289ce3c730bed121d7?variant=devin
Requested by: @Chadha93

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated
firecrawl	🟢 Ready	View Preview	Sep 23, 2026, 10:18 AM

💡 Tip: Enable Automations to automatically generate PRs for you.