Skip to content

MCP upgrade 1/6: Isolated contract evaluator - #4

Draft
Lauritz-Timm wants to merge 21 commits into
developfrom
agent/mcp-upgrade-01-evaluation
Draft

MCP upgrade 1/6: Isolated contract evaluator#4
Lauritz-Timm wants to merge 21 commits into
developfrom
agent/mcp-upgrade-01-evaluation

Conversation

@Lauritz-Timm

Copy link
Copy Markdown
Owner

Part of the six-phase full MCP service upgrade.

Why

The upgrade needs a repeatable acceptance surface that runs without network access, provider installations, user configuration, or machine-specific paths.

What

  • Adds an offline evaluator with generated mock homes, stores, provider files, and proxy state.
  • Pins the protocol, output, provider, proxy, and normalization contracts.
  • Exercises 294 scenarios in roots containing spaces and Unicode.
  • Records the corrected frozen baseline and compares candidates by normalized behavior.

Dependency

Standalone from develop; no production phase depends on merging this evaluator.

Verification

  • Evaluator tests passed 44/44; formatting, strict Clippy, design verification, and diff checks passed.
  • The approved frozen baseline receipt was reused without rerunning the baseline.
  • The final integrated candidate passed 294/294 scenarios in each root with normalized equality.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant