Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 16 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -419,6 +419,7 @@ Scrape content from a single URL with advanced options.

**Branding format:** Extracts comprehensive brand identity (colors, fonts, typography, spacing, logo, UI components) for design analysis or style replication.
**Privacy:** Set `redactPII: true` to return content with personally identifiable information redacted.
**Hosted server:** On the hosted server (`CLOUD_SERVICE=true`) scrape is read-only. It takes no browser `actions` and cannot accept provider terms. A named `profile` loads saved browser state without saving changes to it. An organization admin accepts terms in the dashboard.

**Returns:**

Expand Down Expand Up @@ -747,17 +748,17 @@ The agent performs web searches, follows links, reads pages, and gathers data au
- `enabled`, `toolkits` (up to 5 provider slugs), `maxCalls` (1 to 30), `requireApproval` (paid calls end the turn with a `pendingApproval`; needs `mode: "chat"` on the same call, even on a follow-up)
- `onTermsRequired`: what to do when an Alexandria provider the agent would use needs data terms your team has not accepted. Gated providers are never called in any mode. Omitted on a follow-up keeps the previous turn's value.
- `"skip"` (default): answer with accepted providers only. `exchange.skippedProviders` on the status result lists the gated providers that would have helped.
- `"ask"`: the same, plus a terms `pendingApproval` and `exchange.requiresAction` with the exact `terms/show` and `terms/accept` calls for each provider. Each provider's `digest` is always present and is `string | null`; when it is `null`, `terms/show` returns the current digest to send.
- `"ask"`: the same, plus a terms `pendingApproval` and `exchange.requiresAction` with the approval ID and provider requirements. Read terms with `terms/show`; an organization admin accepts them in the Firecrawl dashboard.
- `approve`: `{ approvalId, callIds?, always? }` answers yes to the `pendingApproval` the previous turn ended on. `callIds` and `always` apply to paid-call approvals only.
- `decline`: `{ approvalId }` answers no. A declined terms offer keeps those providers out of the rest of the thread.
- `approve` and `decline` need `threadId`, and only one of them can be sent.

**Provider terms (ask mode):** there is no auto-accept mode. When a turn ends on a terms offer, the status result carries `pendingApproval` (`kind: "terms"`) and `exchange.requiresAction` with the `approvalId` and the exact `terms/show` and `terms/accept` calls. To use the provider:
**Provider terms (ask mode):** there is no auto-accept mode. When a turn ends on a terms offer, the status result carries `pendingApproval` (`kind: "terms"`) and `exchange.requiresAction` with the `approvalId` and provider requirements. Any `terms/accept` descriptor in that API payload is unavailable through MCP. To use the provider:

1. Show the user the terms (`terms/show` through `firecrawl_scrape` with `alexandria`).
2. Get the user's explicit consent to that provider's terms. A data request is not consent.
3. Run the `terms/accept` call through `firecrawl_scrape`.
4. Continue the same thread: call `firecrawl_agent` with the same `threadId` and `exchange.approve: { "approvalId": "..." }`.
2. Direct an organization admin to accept the terms at the provider's URL, or [data sources settings](https://www.firecrawl.dev/app/settings?tab=data-sources). A data request is not consent.
3. Wait for the admin to confirm acceptance in the dashboard.
4. Continue the same thread: call `firecrawl_agent` with the same `threadId` and `exchange.approve: { "approvalId": "..." }`. This resumes research and does not accept terms.

If the user says no, call `firecrawl_agent` with the same `threadId` and `exchange.decline: { "approvalId": "..." }` instead.

Expand Down Expand Up @@ -818,13 +819,13 @@ Then poll with `firecrawl_agent_status` using the returned job ID.
}
```

**Usage Example (continue the thread after the user accepted a provider's terms):**
**Usage Example (continue the thread after an admin confirmed dashboard acceptance):**

```json
{
"name": "firecrawl_agent",
"arguments": {
"prompt": "I accepted the Apollo terms. Continue.",
"prompt": "The admin confirmed acceptance of the Apollo terms in the dashboard. Continue.",
"threadId": "0199a1b2-0000-7000-8000-000000000031",
"exchange": { "approve": { "approvalId": "0199a1b2-0000-7000-8000-000000000033" } }
}
Expand Down Expand Up @@ -1113,16 +1114,14 @@ HTTP 403 and this body:
The tool result relays it as an error with `structuredContent` carrying `code`,
`status: 403`, `requestId`, the `requiresAction` object unchanged, and
`next_actions` (`human_action_required` then `retry_same_request`). Accepting
terms is a legal act. Use the returned `nextTool` call to read the agreement through `firecrawl_scrape`
with `alexandria: [{provider: "firecrawl", capability: "terms/show", options: {provider: "<provider>"}}]`.
Present it to the user and obtain explicit authorization to bind their organization
before calling `firecrawl_scrape` with capability `terms/accept` under provider `firecrawl`.
Its options are `provider`, the exact reviewed `version`
and 64-character lowercase hexadecimal `digest`, and `confirmed: true`. A request
for data is not consent. Authority or eligibility errors may require an organization
admin to use `requiresAction.url`. No automatic acceptance or uncertain retries occur.
Send terms calls separately from execution. These are nested capabilities, not top-level MCP tools.
No credits are charged for the blocked retrieval. After confirmed acceptance, call the same
terms is a legal act, so an organization admin accepts them in the Firecrawl dashboard, not
through MCP. Use the returned `nextTool` call to read the agreement through `firecrawl_scrape`
with `alexandria: [{provider: "firecrawl", capability: "terms/show", options: {provider: "<provider>"}}]`,
sent separately from provider execution, and present it to the user. An organization admin then
accepts it at `requiresAction.url`, or at https://www.firecrawl.dev/app/settings?tab=data-sources.
`firecrawl_scrape` refuses every other `terms/*` capability, so it makes no account changes. A request
for data is not consent, and no automatic acceptance or uncertain retries occur.
No credits are charged for the blocked retrieval. After the admin confirms acceptance, call the same
tool again with the identical payload and `requestId`.

### 16. Credit Usage Tool
Expand Down
7 changes: 7 additions & 0 deletions docs/search-profile.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,13 @@ names only tools this surface exposes. `firecrawl_find_tools` is registered the
same way. Alexandria results on this surface carry no `feedbackTool` pointer,
since `firecrawl_feedback` is not registered here.

`firecrawl_scrape` is read-only here (`readOnlyHint: true`): the surface runs in
hosted safe mode, so it takes no browser `actions`. Provider terms can be read
with the nested `terms/show` capability. As on the full surface, an organization
admin accepts them in the dashboard: `firecrawl_scrape` refuses every other
`terms/*` capability, and terms errors link to `requiresAction.url` or
https://www.firecrawl.dev/app/settings?tab=data-sources.

## Alexandria source

`sources` entries are source names (`web`, `news`, `images`, `alexandria`) or
Expand Down
39 changes: 32 additions & 7 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1082,6 +1082,26 @@ type TermsRequiredAction = {

type ExchangeErrorContext = { tool: string; requestId?: string; providers?: string[] };

const DATA_SOURCES_SETTINGS_URL =
'https://www.firecrawl.dev/app/settings?tab=data-sources';

// An organization admin accepts provider terms in the dashboard. Through MCP,
// agents only read them with terms/show, so firecrawl_scrape changes no
// account state.
function isTermsWrite(call: { provider: string; capability: string }): boolean {
const capability = call.capability.trim().toLowerCase();
return (
call.provider.trim().toLowerCase() === 'firecrawl' &&
capability.startsWith('terms/') &&
capability !== 'terms/show'
);
}

function termsWriteError(): UserError {
const message = `Provider terms are accepted in the Firecrawl dashboard, not through this connection. Read them with terms/show, then ask an organization admin to accept them at ${DATA_SOURCES_SETTINGS_URL}.`;
return new UserError(message, { code: 'invalid_option', status: 400, message });
}

function termsRequiredAction(body: unknown): TermsRequiredAction | undefined {
const data = body as
| { code?: unknown; requiresAction?: unknown }
Expand Down Expand Up @@ -1116,7 +1136,7 @@ function termsRequiredError(
const retryIdentity = requestId ? ` and requestId ${requestId}` : '';
const message = `Alexandria provider terms required. An organization admin must accept the ${action.terms} provider's terms (version ${action.version}) before this request can run.`;
return new UserError(
`${message}\n\n1. Read the agreement with firecrawl_scrape using alexandria: {provider: "firecrawl", capability: "terms/show", options: {provider: "${action.terms}"}} and present it to the user. Only after explicit authorization to accept that exact version and digest, use firecrawl_scrape with alexandria: {provider: "firecrawl", capability: "terms/accept", options: {provider: "${action.terms}", version: "<reviewed-version>", digest: "<reviewed-digest>", confirmed: true}}. Send terms calls separately from provider execution. If authority or eligibility requires a dashboard action, ask an organization admin to visit ${action.url} or https://www.firecrawl.dev/app/settings?tab=data-sources if that page is unavailable. Never infer acceptance from a data request.\n2. After they confirm, call ${context.tool} again with the identical payload${retryIdentity}. If the same terms error persists, stop and ask an organization admin to check access at https://www.firecrawl.dev/app/settings?tab=data-sources.\n\nDo not retry until acceptance is confirmed.`,
`${message}\n\n1. Read the agreement with firecrawl_scrape using alexandria: {provider: "firecrawl", capability: "terms/show", options: {provider: "${action.terms}"}} and present it to the user. Send terms/show separately from provider execution. Terms are accepted in the Firecrawl dashboard, not through this connection: ask an organization admin to accept them at ${action.url}, or ${DATA_SOURCES_SETTINGS_URL} if that page is unavailable. Never infer acceptance from a data request.\n2. After the admin confirms, call ${context.tool} again with the identical payload${retryIdentity}. If the same terms error persists, stop and ask an organization admin to check access at ${DATA_SOURCES_SETTINGS_URL}.\n\nDo not retry until acceptance is confirmed.`,
{
code: TERMS_REQUIRED_CODE,
status: 403,
Expand Down Expand Up @@ -1203,7 +1223,7 @@ async function relayExchangeError(
: undefined;
if (disabledProvider) {
throw new UserError(
`${message} Read the provider terms and status using nextTool and present them to the user. Never infer acceptance from a data request. Only after explicit authorization for the reviewed version and digest may you call firecrawl_scrape with alexandria:{provider:"firecrawl",capability:"terms/accept",options:{provider:"${disabledProvider}",version:"<reviewed-version>",digest:"<reviewed-digest>",confirmed:true}}. Send terms calls separately. Acceptance may not restore disabled access; an organization admin may need to review https://www.firecrawl.dev/app/settings?tab=data-sources. Retry the original request only after access is restored.`,
`${message} Read the provider terms and status using nextTool and present them to the user. Never infer acceptance from a data request. Terms are accepted in the Firecrawl dashboard, not through this connection; acceptance may not restore disabled access, and an organization admin can review access at ${DATA_SOURCES_SETTINGS_URL}. Retry the original request only after access is restored.`,
{
code: typeof data?.code === 'string' ? data.code : 'exchange_error',
status: 403,
Expand Down Expand Up @@ -2698,7 +2718,9 @@ const scrapeTool: RegisteredTool = {
name: 'firecrawl_scrape',
annotations: {
title: 'Firecrawl scrape',
readOnlyHint: false, // Alexandria capabilities can record provider agreement acceptance.
// Hosted scrape omits browser actions and refuses provider terms writes
// before execution.
readOnlyHint: SAFE_MODE,
openWorldHint: true, // Accepts any user-supplied URL on the public web.
destructiveHint: false, // Does not modify, delete, or write to external websites.
},
Expand Down Expand Up @@ -2727,6 +2749,9 @@ Alexandria mode, on an authenticated session with Alexandria access: \`alexandri
} & Record<string, unknown>;
if (alexandria) {
assertExchangeCredential(session);
if ((Array.isArray(alexandria) ? alexandria : [alexandria]).some(isTermsWrite)) {
throw termsWriteError();
}
log.info('Executing Alexandria capabilities', {
count: Array.isArray(alexandria) ? alexandria.length : 1,
});
Expand Down Expand Up @@ -3008,7 +3033,7 @@ server.addTool(findToolsTool);
// (no crawl, map, interact, monitor, parse or feedback references). Registered
// on the search surface in place of the module-level tools above.
const SEARCH_SURFACE_SCRAPE_DESCRIPTION = `
Scrape one URL and return its content, or execute catalogued Alexandria capabilities. URL mode returns markdown by default, or HTML, links, screenshots, branding data, a targeted answer, or JSON matching a supplied schema, plus page metadata. Firecrawl may serve recently indexed content; set \`maxAge: 0\` for a live fetch. A successful response does not by itself confirm the page is still current. Browser actions can change the live page when interactive actions are enabled, and a named browser profile can load saved session data and overwrite its stored state.
Scrape one URL and return its content, or execute catalogued Alexandria capabilities. URL mode returns markdown by default, or HTML, links, screenshots, branding data, a targeted answer, or JSON matching a supplied schema, plus page metadata. Firecrawl may serve recently indexed content; set \`maxAge: 0\` for a live fetch. A successful response does not by itself confirm the page is still current. A named browser profile loads saved session data.

\`firecrawl_search\` with \`sources\` unset and \`firecrawl_find_tools\` can discover providers for the same fields across several pages; a matching Alexandria provider returns typed records in one call.

Expand Down Expand Up @@ -3838,7 +3863,7 @@ const agentExchangeSchema = z
})
.optional()
.describe(
'Answer yes to the pendingApproval the previous turn of this thread ended on (its id, also exchange.requiresAction.approvalId). Needs threadId. For a terms offer, send it only after the user explicitly agreed and terms/accept succeeded; callIds and always are ignored on terms offers. For paid calls, callIds picks a subset (default all) and always stops asking for the rest of the thread.'
'Answer yes to the pendingApproval the previous turn of this thread ended on (its id, also exchange.requiresAction.approvalId). Needs threadId. For a terms offer, send it only after an organization admin confirms acceptance in the Firecrawl dashboard; this does not accept terms. callIds and always are ignored on terms offers. For paid calls, callIds picks a subset (default all) and always stops asking for the rest of the thread.'
),
decline: z
.strictObject({ approvalId: z.string().uuid() })
Expand All @@ -3850,7 +3875,7 @@ const agentExchangeSchema = z
.enum(['skip', 'ask'])
.optional()
.describe(
'What to do when a provider the agent would use needs data terms the team has not accepted. Gated providers are never called. "skip" (default): answer with accepted providers and list the rest in exchange.skippedProviders. "ask": the same, plus a terms pendingApproval and exchange.requiresAction with the terms/show and terms/accept calls. Each provider digest is string | null and always present; when null, terms/show returns it. There is no auto-accept. Omitted on a follow-up keeps the previous turn\'s value.'
'What to do when a provider the agent would use needs data terms the team has not accepted. Gated providers are never called. "skip" (default): answer with accepted providers and list the rest in exchange.skippedProviders. "ask": the same, plus a terms pendingApproval and exchange.requiresAction. Read terms with terms/show; an organization admin accepts them in the Firecrawl dashboard. There is no auto-accept. Omitted on a follow-up keeps the previous turn\'s value.'
),
})
.describe(
Expand All @@ -3872,7 +3897,7 @@ This call returns only a job ID, not the research result. Read the job with \`fi

The job also returns a \`threadId\`. To continue that thread, pass it with a follow-up \`prompt\`; omitted \`mode\`, \`urls\`, \`schema\` and exchange settings carry over from the previous turn.

The agent only calls Alexandria providers whose data terms the team has accepted. The status result's \`exchange.skippedProviders\` lists gated providers that would have helped. With \`exchange.onTermsRequired\` "ask", a terms offer ends the turn: \`pendingApproval\` (kind "terms") and \`exchange.requiresAction\` carry the \`approvalId\` and the exact terms/show and terms/accept calls. Show the user the terms, get their EXPLICIT consent, run terms/accept through \`firecrawl_scrape\`, then call \`firecrawl_agent\` with the same \`threadId\` and \`exchange.approve: {approvalId}\`. If they decline, send \`exchange.decline: {approvalId}\` instead. Never call terms/accept without that consent; a data request is not consent.
The agent only calls Alexandria providers whose terms the team has accepted. \`exchange.skippedProviders\` lists gated providers. With \`exchange.onTermsRequired\` "ask", a terms \`pendingApproval\` and \`exchange.requiresAction\` carry the \`approvalId\` and provider requirements. Read terms/show through \`firecrawl_scrape\`. An organization admin must accept terms in the Firecrawl dashboard at the provider URL or ${DATA_SOURCES_SETTINGS_URL}. Ignore any terms/accept call in the API response. Only after the admin confirms acceptance, resume with the same \`threadId\` and \`exchange.approve: {approvalId}\`; this does not accept terms. To decline, use \`exchange.decline: {approvalId}\`. Never infer acceptance from a data request.
`,
outputSchema: agentOutputSchema,
parameters: z.object({
Expand Down
4 changes: 2 additions & 2 deletions src/tool-output.ts
Original file line number Diff line number Diff line change
Expand Up @@ -248,10 +248,10 @@ export const agentStatusOutputSchema = z
message: unknown('The agent\'s reply; in chat mode, the short answer to a follow-up.'),
suggestions: unknown('Follow-ups the agent offers; send one as the prompt of the next turn with this threadId.'),
exchange: unknown(
'What the job did with Alexandria providers: onTermsRequired, paidCalls, creditsUsed, skippedProviders (gated providers that would have helped), and requiresAction (approvalId plus the terms/show and terms/accept calls, each provider digest string | null and always present; call accept only with the user\'s explicit consent, then continue the thread with exchange.approve: {approvalId}).'
'What the job did with Alexandria providers: onTermsRequired, paidCalls, creditsUsed, skippedProviders (gated providers that would have helped), and requiresAction (approvalId and provider requirements). Read terms with terms/show. Ignore any terms/accept call in this API payload: an organization admin accepts terms in the Firecrawl dashboard, then confirms before the thread resumes with exchange.approve: {approvalId}.'
),
pendingApproval: unknown(
'Set when the job ended waiting on the caller. Answer it by calling firecrawl_agent with this threadId and exchange.approve or exchange.decline carrying its id. kind "terms" lists providers whose data terms need accepting; otherwise calls lists paid calls waiting for approval.'
'Set when the job ended waiting on the caller. Answer it by calling firecrawl_agent with this threadId and exchange.approve or exchange.decline carrying its id. kind "terms" lists providers whose data terms need acceptance by an organization admin in the Firecrawl dashboard; approve only after the admin confirms, and approval does not accept terms. Otherwise calls lists paid calls waiting for approval.'
),
})
.describe('Progress or final result of a research agent job.');
Expand Down
Loading
Loading