Skip to content

Send objective with Search feedback #474

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
KrisOei wants to merge 1 commit into from
Closed

Send objective with Search feedback #474

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion src/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3370,7 +3370,7 @@ if (!SEARCH_FEEDBACK_DISABLED && !isLocalKeylessStartup()) {
destructiveHint: false, // Additive only; records feedback and may refund credits, does not delete data.
},
description: `
Records schema-validated quality feedback for a prior \`firecrawl_search\` UUID \`searchId\`. A \`good\` rating requires a valuable source, \`partial\` a valuable source or at least one \`missingContent\` entry, and \`bad\` at least one \`missingContent\` entry or a query suggestion; caps are 50 \`valuableSources\` and 20 \`missingContent\` entries.
Records schema-validated quality feedback for a prior \`firecrawl_search\` UUID \`searchId\`. A \`good\` rating requires a valuable source, \`partial\` a valuable source or at least one \`missingContent\` entry, and \`bad\` at least one \`missingContent\` entry or a query suggestion; caps are 50 \`valuableSources\` and 20 \`missingContent\` entries. Include \`objective\`: the underlying goal behind the search, not only what the query looked for.

Eligibility is limited to successful searches within the feedback age window. The record is idempotent per search ID. Eligible first feedback for a search can refund 1 credit; refunds are subject to the team's daily cap. The response reports whether a refund was applied, along with submission and daily-cap status.
`,
Expand Down Expand Up @@ -3407,6 +3407,16 @@ Eligibility is limited to successful searches within the feedback age window. Th
'longer `description`.'
),
querySuggestions: z.string().max(2000).optional(),
objective: z

@cubic-dev-ai cubic-dev-ai Bot Oct 2, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: This objective caps at 2000 characters, but firecrawl_search's own objective (src/index.ts line 999, z.string().trim().min(1).max(5000)) allows up to 5000. An agent that reuses the search's objective value when submitting feedback hits a schema validation error for objectives between 2001 and 5000 characters. Align the caps via a shared constant or document why the feedback limit is tighter.

Prompt for AI agents 
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At src/index.ts, line 3410:

<comment>This `objective` caps at 2000 characters, but `firecrawl_search`'s own `objective` (src/index.ts line 999, `z.string().trim().min(1).max(5000)`) allows up to 5000. An agent that reuses the search's objective value when submitting feedback hits a schema validation error for objectives between 2001 and 5000 characters. Align the caps via a shared constant or document why the feedback limit is tighter.</comment>

<file context>
@@ -3407,6 +3407,16 @@ Eligibility is limited to successful searches within the feedback age window. Th
             'longer `description`.'
         ),
       querySuggestions: z.string().max(2000).optional(),
+      objective: z
+        .string()
+        .trim()
</file context>
Fix with cubic

.string()
.trim()
.min(1)
.max(2000)
.optional()
.describe(
'The underlying goal behind the search: what you or your user were ultimately trying to accomplish ' +
'(for example, "shortlist federal IT contracts to bid on this quarter"), not only what this query looked for.'
),
}),
execute: async (
args: unknown,
Expand All @@ -3419,12 +3429,14 @@ Eligibility is limited to successful searches within the feedback age window. Th
valuableSources,
missingContent,
querySuggestions,
objective,
} = args as {
searchId: string;
rating: 'good' | 'bad' | 'partial';
valuableSources?: { url: string; reason?: string }[];
missingContent?: { topic: string; description?: string }[];
querySuggestions?: string;
objective?: string;
};

const apiBase = resolveApiBaseUrl();
Expand All @@ -3443,6 +3455,7 @@ Eligibility is limited to successful searches within the feedback age window. Th
body.missingContent = missingContent;
}
if (querySuggestions) body.querySuggestions = querySuggestions;
if (objective) body.objective = objective;

const headers: Record<string, string> = {
...AGENT_HINTS_HEADERS,
Expand Down
5 changes: 5 additions & 0 deletions tests/mcp-smoke.test.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1763,6 +1763,7 @@ test('stdio transport calls Firecrawl API through a tool end to end', async (t)

const searchFeedbackResult = await client.request('tools/call', {
arguments: {
objective: 'Shortlist federal IT contracts to bid on this quarter',
querySuggestions: 'Use a narrower query',
rating: 'bad',
searchId: toolPayload.id,
Expand Down Expand Up @@ -1793,6 +1794,10 @@ test('stdio transport calls Firecrawl API through a tool end to end', async (t)
'/v2/search/00000000-0000-4000-8000-000000000000/feedback'
);
assert.equal(searchFeedbackRequest.body.rating, 'bad');
assert.equal(
searchFeedbackRequest.body.objective,
'Shortlist federal IT contracts to bid on this quarter'
);
const endpointFeedbackRequests = fakeApi.requests.filter(
(request) => request.url === '/v2/feedback'
);
Expand Down
Loading