Skip to content

Seeking design partners for Mendmark 0.4 agent-eval pilots #1

Description

@danielgaskins

Mendmark 0.4 is ready for a small design-partner pilot with teams shipping tool-using agents.

The useful outcome is concrete: run Mendmark against an existing eval suite, inspect any surviving controlled faults, strengthen an evaluator if warranted, and measure whether the audit belongs in CI.

Good pilot fits:

  • At least one passing eval case for a tool-using agent
  • An engineer who can update the evaluator or release policy
  • DeepEval or the ability to exchange local JSON with an evaluator command
  • Permission to share aggregate setup time, runtime, mutation counts, and survivor counts

Mendmark runs locally. Do not post prompts, traces, arguments, outputs, credentials, or customer data here.

Read the pilot guide and apply through the repository's Mendmark pilot request issue form. The first cohort will be limited to 5–10 teams so onboarding can remain hands-on.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    pilotDesign-partner pilot

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions