docs: add eve evals reporter integration (LAM-1855)#186
Open
laminar-coding-agent[bot] wants to merge 2 commits into
Open
docs: add eve evals reporter integration (LAM-1855)#186laminar-coding-agent[bot] wants to merge 2 commits into
laminar-coding-agent[bot] wants to merge 2 commits into
Conversation
Document the LaminarReporter for eve.dev evals: how each eval maps to a Laminar datapoint, gate vs. soft score encoding, registering the reporter in evals.config.ts, and reading/querying the results. Includes list and detail screenshots from a verified support-triage run, a changelog entry, and the docs.json nav entry. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Contributor
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
Contributor
Author
|
Re-verified this page against eve 0.22.6 (it was written against 0.16.2) and a real end-to-end run of the companion SDK PR (lmnr-ai/lmnr-ts#266) against a local Laminar stack. No changes needed:
|
Resolve changelog conflict: eve reporter entry moves to July 2026 since that is when the feature ships. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Contributor
Author
|
Re-verified this page against eve 0.22.6 (latest) and the implementation in lmnr-ai/lmnr-ts#266:
No changes needed on this branch. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
evaluations/evepage documenting theLaminarReporterfor eve evals: how each eval maps to a Laminar datapoint, gate vs. soft score encoding, registering the reporter inevals.config.ts, and reading/querying results.support-triagerun against the local prod stack, including a genuine gate failure.docs.jsonnav entry (Evaluations group).Notes
LaminarReporter(lmnr-tsfeat/lam-1855-eve-laminar-reporter, commit1f6c6e6): the datapoint input is the eval id, and metadata carries verdict, status, session id, model id, tool-call names, and failed assertions.npx mintlify broken-links(clean); no em dashes or banned marketing words.Test plan
<Frame>Note
Low Risk
Documentation-only changes (MDX, nav, changelog); no application or SDK code in this diff.
Overview
Adds documentation for shipping eve eval runs to Laminar via
LaminarReporter: mapping (eval → datapoint, gates asgate:<name>scores, soft assertions, executor output), setup inevals/evals.config.ts, runningeve eval, and reading/querying results in the UI, SQL, CLI, and MCP. Includes list and detail screenshots and links to comparing runs and related eval docs.Navigation: new Evaluations → eve reporter page (
evaluations/eve) indocs.json.Changelog: July 2026 entry for the eve reporter feature.
Reviewed by Cursor Bugbot for commit 32f99e3. Bugbot is set up for automated code reviews on this repo. Configure here.