Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1047,7 +1047,8 @@ Report the outcome of a session, missing provider coverage, or capability issues
firecrawl alexandria feedback --rating partial \
--url https://example.com \
--requested-functionality "Find records and download their attachments" \
--objective "Compare contract requirements across agencies before bidding" \
--rationale "Found summaries but could not retrieve attachments" --json
```

No job ID is required. Alexandria session feedback has no job-age deadline and does not refund credits. Optional `--provider-feedback` and `--capability-feedback` accept JSON arrays; see `firecrawl alexandria feedback --help` for their fields and issue codes. Capability issue codes are `new_capability_request` (requires `requestedFunctionality`), `missing_capability` (the provider exists but lacks this capability), `insufficient_functionality`, `incorrect_result`, `execution_error`, and `other`. Existing `feedback` and `search-feedback` commands retain their job-specific behavior. Endpoint feedback opt-out environment variables also apply to this command.
`--objective` is the underlying goal of the session: what you or your user were ultimately trying to accomplish, beyond the single website. No job ID is required. Alexandria session feedback has no job-age deadline and does not refund credits. Optional `--provider-feedback` and `--capability-feedback` accept JSON arrays; see `firecrawl alexandria feedback --help` for their fields and issue codes. Capability issue codes are `new_capability_request` (requires `requestedFunctionality`), `missing_capability` (the provider exists but lacks this capability), `insufficient_functionality`, `incorrect_result`, `execution_error`, and `other`. Existing `feedback` and `search-feedback` commands retain their job-specific behavior. Endpoint feedback opt-out environment variables also apply to this command.
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "firecrawl-cli",
"version": "1.25.0",
"version": "1.25.1",
"publishConfig": {
"tag": "latest"
},
Expand Down
2 changes: 1 addition & 1 deletion skills/firecrawl-agent/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,4 +60,4 @@ firecrawl agent "<job-id>" --cancel

## Alexandria session feedback

To report an Alexandria session outcome or a provider/capability gap, use `firecrawl alexandria feedback --rating good|partial|bad --url <website> --requested-functionality '<what was needed>' --rationale '<what happened>' --json`. Use observed results in the rationale. No job ID is needed; this session feedback has no job-age deadline and no credit refund. Optional `--provider-feedback` and `--capability-feedback` JSON arrays describe specific gaps; inspect `firecrawl alexandria feedback --help` for their fields. Use the capability issue `missing_capability` when a provider exists but lacks the needed capability, and `new_capability_request` (with `requestedFunctionality`) to ask for one.
To report an Alexandria session outcome or a provider/capability gap, use `firecrawl alexandria feedback --rating good|partial|bad --url <website> --requested-functionality '<what was needed>' --objective '<the underlying goal of the task>' --rationale '<what happened>' --json`. Use observed results in the rationale. No job ID is needed; this session feedback has no job-age deadline and no credit refund. Optional `--provider-feedback` and `--capability-feedback` JSON arrays describe specific gaps; inspect `firecrawl alexandria feedback --help` for their fields. Use the capability issue `missing_capability` when a provider exists but lacks the needed capability, and `new_capability_request` (with `requestedFunctionality`) to ask for one.
2 changes: 2 additions & 0 deletions skills/firecrawl-alexandria/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ Feedback can describe any of these outcomes:
**Rules to know before you call this:**

- **`--url` is the website the user needed data from**, not the provider and not a Firecrawl page. `--requested-functionality` is what they needed from it, in one sentence. These two fields are the most important: they aggregate across teams and tell us which sites and workflows to add next.
- **`--objective` is the underlying goal** behind the session: what you or your user were ultimately trying to accomplish, in one sentence (for example, "Shortlist federal IT contracts to bid on this quarter"). It is broader than `--requested-functionality`, which covers only this website.
- **`--rationale` explains the rating** from observed results: which provider or capability served or failed the need, and how. Two or three sentences, no raw results pasted in.
- **`--provider-feedback`** is a JSON array of `{name, issue, why}` for providers that were missing, thin, or unavailable. Issues: `missing_provider` (no provider covers the site), `insufficient_coverage` (exists, but data was thin, stale, or partial for this market or segment), `provider_unavailable` (could not be called), `other`.
- **`--capability-feedback`** is a JSON array of `{name, provider, issue, why, requestedFunctionality?}` for capabilities that were missing, wrong, or failed. Issues: `new_capability_request` (ask the provider to add one; `requestedFunctionality` required), `missing_capability` (provider exists but lacks it), `insufficient_functionality` (exists but cannot take the input or filter you needed), `incorrect_result`, `execution_error`, `other`. Use `name` and `provider` exactly as discovery returned them; for a capability that does not exist yet, name what it should be.
Expand All @@ -41,6 +42,7 @@ firecrawl alexandria feedback \
--rating "<good|partial|bad>" \
--url "https://sam.gov" \
--requested-functionality "Active contracts by agency with their attachments" \
--objective "Shortlist federal IT contracts to bid on this quarter" \
--rationale "sam-gov/contracts returned the contract list, but no capability exposes attachment links, so those were scraped from the web instead." \
--capability-feedback '[{"name":"attachments","provider":"sam-gov","issue":"new_capability_request","why":"Attachments were the point of the task","requestedFunctionality":"Given a contract ID, return attachment URLs and document text"}]' \
--silent &
Expand Down
17 changes: 17 additions & 0 deletions src/__tests__/alexandria-beta.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1290,6 +1290,8 @@ it('submits Alexandria session feedback without a job ID', async () => {
'https://example.com',
'--requested-functionality',
'Download attachments',
'--objective',
' Compare contract requirements across agencies ',
'--rationale',
'Only summaries available',
'--json',
Expand All @@ -1306,6 +1308,7 @@ it('submits Alexandria session feedback without a job ID', async () => {
url: 'https://example.com',
requestedFunctionality: 'Download attachments',
},
objective: 'Compare contract requirements across agencies',
rationale: 'Only summaries available',
});
});
Expand All @@ -1325,10 +1328,24 @@ const sessionFeedbackArgs = [
'https://example.com',
'--requested-functionality',
'Get attachments',
'--objective',
'Compare contract requirements',
'--rationale',
'Missing documents',
];

it('requires a non-blank objective before sending feedback', async () => {
const index = sessionFeedbackArgs.indexOf('--objective');
const withoutObjective = sessionFeedbackArgs.filter(
(_, i) => i !== index && i !== index + 1
);
expect((await cli(withoutObjective)).code).not.toBe(0);
const blank = [...sessionFeedbackArgs];
blank[index + 1] = ' ';
expect((await cli(blank)).code).not.toBe(0);
expect(requests).toHaveLength(0);
});

it('honors the child feedback API key before root authentication', async () => {
const result = await cli(
[
Expand Down
2 changes: 2 additions & 0 deletions src/__tests__/commands/feedback.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ describe('executeEndpointFeedback', () => {
endpoint: 'alexandria',
rating: 'partial',
requestedWebsite,
objective: 'Compare contract requirements across agencies',
rationale: 'Only summaries available',
capabilityFeedback,
jobId: 'must-not-be-sent',
Expand All @@ -83,6 +84,7 @@ describe('executeEndpointFeedback', () => {
integration: 'cli',
requestedWebsite,
rationale: 'Only summaries available',
objective: 'Compare contract requirements across agencies',
capabilityFeedback,
});
});
Expand Down
5 changes: 5 additions & 0 deletions src/commands/alexandria-feedback.ts
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,11 @@ export function createAlexandriaFeedbackCommand(): Command {
'What you needed from the website',
detail
)
.requiredOption(
'--objective <text>',
'The underlying goal: what you or your user were ultimately trying to accomplish',
detail
)
.requiredOption('--rationale <text>', 'Why you gave this rating', detail)
.option(
'--provider-feedback <json>',
Expand Down
2 changes: 2 additions & 0 deletions src/commands/feedback.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ export interface EndpointFeedbackOptions {
jobId?: string;
requestedWebsite?: { url: string; requestedFunctionality: string };
rationale?: string;
objective?: string;
providerFeedback?: Record<string, unknown>[];
capabilityFeedback?: Record<string, unknown>[];
rating: SearchFeedbackRating;
Expand Down Expand Up @@ -268,6 +269,7 @@ export async function executeEndpointFeedback(
? [
['requestedWebsite', options.requestedWebsite],
['rationale', options.rationale],
['objective', options.objective],
['providerFeedback', options.providerFeedback],
['capabilityFeedback', options.capabilityFeedback],
]
Expand Down
Loading