feat(api): link keyless prompts to opaque per-identity /k links by rakshith48 · Pull Request #4856 · firecrawl/firecrawl

rakshith48 · 2026-09-30T06:22:50Z

Keyless prompts now link to the caller's own signup link, https://firecrawl.dev/k/<token>, instead of /signin?utm_source=keyless&utm_medium=api. The token is stateless: it is the prompt itself, encrypted, so nothing is stored per identity.

Token (src/lib/keyless-signup-link.ts, one shared layout comment, mirrored in firecrawl-web)

12 lowercase Crockford base32 characters (0123456789abcdefghjkmnpqrstvwxyz), no query string.
Plaintext is 60 bits, most significant first: keyless IPv4 (32 bits), surface (2: api, mcp, cli), prompt reason (3: limit, account_only_tool, unsupported_endpoint, suspicious_ip; the rest reserved), check (23, all zero). No time in the token.
Encrypted with FF1 (NIST SP 800-38G) over AES-128, radix 32, 12 numerals, tweak fc-keyless-v1, built on Node's AES with no new dependency. Tested against the NIST FF1 AES-128 samples 1, 2 (radix 10) and 3 (radix 36).
A token is valid only if the 23 check bits decrypt to zero, about 1 in 8.4M forgery odds per key. Reserved surface or reason codes decode to null.
The link reveals nothing about the caller: no IP, surface or UTM in the clear. The same identity, surface and reason always get the same token.

Where tokens are used

Every keyless prompt for a valid IPv4 identity (existing eligibility check and ::ffff: normalization) gets a token:
- limit: request and credit caps, credit reservation, browser budget
- unsupported_endpoint: the 401, with the client IP from keylessClientIp(req)
- suspicious_ip: the 403
- eligibility refusals: limit for requests/credits, suspicious_ip for suspicious
- account_only_tool: /v2/keyless/eligibility?signup_link=1, used by MCP account-only tools, whatever the eligibility result
Anything else gets the regular link, keylessFallbackSignupUrl(surface) = https://www.firecrawl.dev/signin?utm_source=keyless&utm_medium=<surface>: no key configured, a non-IPv4 identity, or an eligibility refusal because the tier is off or the limiter is down.
Surface classification is unchanged (keylessSignupSurface: cli when origin or integration is cli, mcp when origin starts with mcp or the hosted MCP relayed with the proxy secret, else api; case-insensitive; a v1-prefaulted body origin api defers to x-origin).
Responses carry the link as signup_url; the eligibility endpoint returns signupUrl. The keyless_exhausted and suspicious-IP logs carry signupRef, now the token.
Token generation is pure CPU with no I/O and never throws.

Removed: the keyless_signup_links Drizzle schema, the upsert, the Redis cache, the 300ms timeout and 30s backoff, existingKeylessSignupUrl and resetKeylessSignupLinkStateForTests.

Env: KEYLESS_SIGNUP_LINK_KEYS (added to the config schema and .env.example)

Comma-separated base64 16-byte AES keys, e.g. from openssl rand -base64 16.
The first key encrypts. Decryption tries each key in turn and accepts the first whose check bits are zero.
To rotate: prepend a new key and keep the old one listed while its links are still around.
A malformed value fails config validation at startup. Unset: every prompt sends the regular link.
Must be the same value in firecrawl-web.

Tests: unit tests for the token (NIST vectors, shared cross-repo vector AAECAwQFBgcICQoLDA0ODw== + {203.0.113.8, mcp, limit} → hrxch5c20tcs, round trip for every surface and reason, every one-character tamper rejected, rotated key, no key, non-IPv4, no IP in the clear), plus prompt, eligibility, browser and auth tests. The v2/keyless snip drops the DB-row checks and decodes each token's surface and reason when the test env sets KEYLESS_SIGNUP_LINK_KEYS (skipped otherwise).

Summary by cubic

Keyless prompts now link to the caller's own opaque https://firecrawl.dev/k/<token> URL instead of the tagged /signin link, so the link reveals nothing about the caller or the surface.

The 12-character lowercase Crockford base32 token is stateless: it carries the keyless IPv4, surface, and prompt reason, encrypted with FF1 (NIST SP 800-38G) over AES-128. Nothing is stored per identity; the web app decrypts the token at signup.
Keys come from KEYLESS_SIGNUP_LINK_KEYS (comma-separated base64 16-byte; the first encrypts, all try to decrypt). With no key, or for a non-IPv4 identity, prompts keep the regular keyless signin URL.
Surface follows the warehouse usage_source signals — cli, mcp, otherwise api — matched case-insensitively; the x-origin header counts when a v1 schema prefaults the body origin to api.
Every keyless prompt gets a token, including the unsupported-endpoint 401, suspicious-IP 403, and the eligibility check; ?signup_link=1 is the account-only-tool reason.
Responses carry the link as signup_url; /v2/keyless/eligibility returns signupUrl on refusals; keyless_exhausted logs add signupRef; a shared test pins the cross-repo keylessTeamId → UUID vector used to join signups.
Token generation is pure CPU with no I/O, so it never fails a request; without a token, prompts fall back to the surface-tagged signin URL.

Migration

Needs the /k route live and KEYLESS_SIGNUP_LINK_KEYS set in both the API and firecrawl-web first; unset keys just send the regular signup link.

^{Written for commit ad37e43. Summary will update on new commits.}

Rollout order (ship in this order)

firecrawl/firecrawl-db#310: migrations
firecrawl/firecrawl-web#3849: /k/<token> route and signup decrypt (must be live before the API ships, or new links 404)
feat(api): link keyless prompts to opaque per-identity /k links #4856: API issues the links
feat: relay the caller's own keyless signup link from the API firecrawl-mcp-server#467: MCP relays them
feat: print the API's keyless signup link as-is and send X-Origin: cli cli#291: CLI release

Web and API must share KEYLESS_SIGNUP_LINK_KEYS before the API ships. Set the same value in the web env first, then in the API env. If the API encrypts with a key the web doesn't have, signups keep the raw token in keyless_ref but get no decoded columns.

🤖 Generated with Claude Code

Keyless prompts (daily limit, unsupported endpoint, suspicious IP, browser budget, credit reservation) now link to https://firecrawl.dev/k/<id> instead of /signin?utm_source=keyless&utm_medium=api. The id is 8 random lowercase Crockford base32 characters, issued once per (keyless identity, surface) into keyless_signup_links and cached in Redis, so repeated prompts on a surface show the same link. The row holds only the id, the keyless team UUID (the same value as keyless_credit_usage.team_id, so the warehouse joins a signup to the keyless ledger without a secret), the surface and when it was first issued. Nothing about the IP or the surface is in the link. The surface comes from the signals the warehouse uses for usage_source: cli when origin or integration is cli, mcp when origin starts with mcp or the hosted MCP relayed the request with the proxy secret, api otherwise. Issuance never fails a request: it is capped at 300ms, backs off for 30s after a database error, and falls back to the bare /k link. Responses also carry the link as signup_url (and signupUrl on /v2/keyless/eligibility, which accepts ?signup_link=1) so the MCP server and CLI can relay it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

cubic-dev-ai

All reported issues were addressed across 19 files

_{Re-trigger cubic}

…the warehouse - Treat a timed-out issuance like a failed one and engage the 30s backoff, so a slow database is not queried once per blocked request. - Classify origin and integration case-insensitively, and let x-origin count when a v1 schema has prefaulted the body origin to "api". - Keep durable link mappings in the keyless snip instead of deleting every surface's row for the shared test identity. - Assert response statuses in the eligibility and browser error tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

superagent-security

Superagent found 1 security concern(s).

cubic-dev-ai

0 issues found across 4 files (changes from recent commits).

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Re-trigger cubic}

The unsupported-endpoint (401) and suspicious-IP (403) prompts run before any quota check, so issuing there let anonymous or rotating traffic write a keyless_signup_links row per source IP. Those prompts, and suspicious eligibility refusals, now use the bare /k link. The unsupported-endpoint check moves back ahead of IP resolution, as on main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

cubic-dev-ai

All reported issues were addressed across 5 files (changes from recent commits).

_{Tip: Review your code locally with the cubic CLI to iterate faster.

Fix all with cubic | Re-trigger cubic}

Dismissed because Cubic found issues in a newer review.

Count keyless_signup_links before and after the 401 request, so the snip catches issuance that writes a row while still returning the bare link. Skipped when the database has no links table. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

cubic-dev-ai

0 issues found across 1 file (changes from recent commits).

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Re-trigger cubic}

… the regular signup link - The unsupported-endpoint (401) and suspicious-IP (403) prompts, and suspicious eligibility refusals, now reuse a link the identity was already given, read from the cache only. They still never issue or write a row, but a signup from an identity that has hit a quota prompt now joins. - Whenever no id can be given (no identity, a slow or failing database, the 30s backoff, or a new identity on a pre-quota prompt), the prompt links to the regular signup URL, signin?utm_source=keyless&utm_medium=<surface>, instead of the bare /k. The surface attribution survives; only the per-identity join is lost. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

superagent-security

Superagent found 1 security concern(s).

cubic-dev-ai

All reported issues were addressed across 8 files (changes from recent commits).

_{Tip: Review your code locally with the cubic CLI to iterate faster.

Fix all with cubic | Re-trigger cubic}

Dismissed because Cubic found issues in a newer review.

Replace the database-backed short id with a 12-character token that carries the prompt itself: the keyless IPv4, the surface (api/mcp/cli) and the prompt reason (limit, account_only_tool, unsupported_endpoint, suspicious_ip), plus 23 zero check bits, encrypted with FF1 (NIST SP 800-38G) over AES-128 in radix 32 and spelled in lowercase Crockford base32. Nothing is stored per identity: the web app decrypts the token at signup. Keys come from KEYLESS_SIGNUP_LINK_KEYS (comma-separated base64 16-byte keys; the first encrypts, all are tried to decrypt). With no key, or for a non-IPv4 identity, prompts keep the regular keyless signin link. Every keyless prompt now gets a token, including the unsupported-endpoint 401, the suspicious-IP 403 and the eligibility check (signup_link=1 is the account-only tool reason). Token generation is pure CPU with no I/O, so the upsert, Redis cache, timeout, backoff and the keyless_signup_links Drizzle schema are gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

cubic-dev-ai

All reported issues were addressed across 12 files (changes from recent commits).

_{Tip: Review your code locally with the cubic CLI to iterate faster.

Fix all with cubic | Re-trigger cubic}

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

cubic-dev-ai

0 issues found across 1 file (changes from recent commits).

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Re-trigger cubic}

rakshith48 requested review from mogery and nickscamara as code owners September 30, 2026 06:22

This was referenced Sep 30, 2026

feat: relay the caller's own keyless signup link from the API firecrawl/firecrawl-mcp-server#467

Merged

feat: print the API's keyless signup link as-is and send X-Origin: cli firecrawl/cli#291

Merged

cubic-dev-ai Bot reviewed Sep 30, 2026

View reviewed changes

superagent-security Bot reviewed Sep 30, 2026

View reviewed changes

Comment thread apps/api/src/controllers/auth.ts Outdated

cubic-dev-ai Bot previously approved these changes Sep 30, 2026

View reviewed changes

cubic-dev-ai Bot reviewed Sep 30, 2026

View reviewed changes

Comment thread apps/api/src/__tests__/snips/v2/keyless.test.ts Outdated

cubic-dev-ai Bot previously approved these changes Sep 30, 2026

View reviewed changes

superagent-security Bot reviewed Sep 30, 2026

View reviewed changes

Comment thread apps/api/src/lib/keyless-signup-link.ts Outdated

cubic-dev-ai Bot reviewed Sep 30, 2026

View reviewed changes

Comment thread apps/api/src/lib/keyless.signup-prompt.test.ts Outdated

Comment thread apps/api/src/lib/keyless-signup-link.test.ts Outdated

cubic-dev-ai Bot reviewed Sep 30, 2026

View reviewed changes

Comment thread apps/api/src/lib/keyless-signup-link.ts

test(keyless): pin the cross-repo keyless team UUID vector

ad37e43

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

cubic-dev-ai Bot approved these changes Sep 30, 2026

View reviewed changes

micahstairs approved these changes Sep 30, 2026

View reviewed changes

rakshith48 merged commit 1b0904c into main Sep 30, 2026
11 of 12 checks passed

Conversation

rakshith48 commented Sep 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by cubic

Rollout order (ship in this order)

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

superagent-security Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

superagent-security Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

rakshith48 commented Sep 30, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading