Skip to content

Add an Exa search-to-classification example - #41

Closed
RogueTex wants to merge 1 commit into
jkudish:mainfrom
RogueTex:codex/exa-classification-example
Closed

RogueTex wants to merge 1 commit into
jkudish:mainfrom
RogueTex:codex/exa-classification-example

Conversation

@RogueTex

Copy link
Copy Markdown

Exa retrieval results need a small adapter before they can be passed to jev_classify and joined back to their source URLs. This adds a runnable source-checkout example that accepts a saved Exa response or performs a direct search, deduplicates URLs, bounds excerpts, and classifies in batches of 64.

The application preserves the full Jev judgment and explicitly routes empty excerpts, invalid responses, and the manual_review class to review. It includes a synthetic offline fixture, four adapter tests, and instructions for evaluating a fixed labeled sample. No new MCP tools or dependencies are added.

Validation:

  • 243 offline tests pass; TypeScript checking and git diff --check pass.
  • All 19 live TypeSafe integration tests pass.
  • Three live runs on ten saved Exa excerpts completed in 267–292 ms per classification batch, with 8–9/10 agreement against agent-assigned reference labels. Every disagreement was routed to review. This is a smoke test, not a validated accuracy benchmark; jev-latest was used.
  • Synthetic live controls exercised library, tutorial, and empty-excerpt review routing.
  • Exa retrieval was performed through the connected Exa app; the standalone REST search branch has not been live-tested with an Exa API key.

@RogueTex
RogueTex marked this pull request as ready for review September 27, 2026 23:30

jkudish commented Sep 29, 2026

Copy link
Copy Markdown
Owner

Thank you for this — a clean search → classify pipeline: URL-preserving, fragment-deduped, batched to the tool's limits, review-routed, with offline adapter tests. Merged with one small amendment: the example pointer is folded into the classify bullets.

@jkudish jkudish closed this Sep 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants