Skip to content

build(deps): bump @mastra/evals from 1.6.0 to 1.7.0 - #995

Merged
owk-owk130 merged 1 commit into
developfrom
dependabot/npm_and_yarn/mastra/evals-1.7.0
Aug 21, 2026
Merged

build(deps): bump @mastra/evals from 1.6.0 to 1.7.0#995
owk-owk130 merged 1 commit into
developfrom
dependabot/npm_and_yarn/mastra/evals-1.7.0

Conversation

@dependabot

@dependabot dependabot Bot commented on behalf of github Aug 10, 2026

Copy link
Copy Markdown
Contributor

Bumps @mastra/evals from 1.6.0 to 1.7.0.

Release notes

Sourced from @​mastra/evals's releases.

February 24, 2026

Highlights

Background Process Management in Workspaces & Sandboxes

Workspaces now support spawning and managing long-running background processes (via SandboxProcessManager / ProcessHandle), with new tools like execute_command (background: true), get_process_output, and kill_process plus improved streaming terminal-style UI.

Runtime Tool Configuration Updates via Workspace.setToolsConfig()

You can dynamically enable/disable tools at runtime on an existing workspace instance (including re-enabling all tools by passing undefined), enabling safer modes like plan/read-only without recreating the workspace.

Observational Memory Reliability & Introspection Improvements

Core adds Harness.getObservationalMemoryRecord() for public access to the full OM record for the current thread, while @mastra/memory fixes major OM stability issues (shared tokenizer to prevent OOM/memory leaks, plus PostgreSQL deadlock fixes and clearer errors when threadId is missing).

Changelog

@​mastra/core@1.7.0

Minor Changes

  • Added getObservationalMemoryRecord() method to the Harness class. Fixes #13392. (#13395)

    This provides public access to the full ObservationalMemoryRecord for the current thread, including activeObservations, generationCount, and observationTokenCount. Previously, accessing raw observation text required bypassing the Harness abstraction by reaching into private storage internals.

    const record = await harness.getObservationalMemoryRecord();
    if (record) {
      console.log(record.activeObservations);
    }
  • Added Workspace.setToolsConfig() method for dynamically updating per-tool configuration at runtime without recreating the workspace instance. Passing undefined re-enables all tools. (#13439)

    const workspace = new Workspace({ filesystem, sandbox });
    // Disable write tools (e.g., in plan/read-only mode)
    workspace.setToolsConfig({
    mastra_workspace_write_file: { enabled: false },
    mastra_workspace_edit_file: { enabled: false },
    });
    // Re-enable all tools
    workspace.setToolsConfig(undefined);

  • Added HarnessDisplayState so any UI can read a single state snapshot instead of handling 35+ individual events. (#13427)

    Why: Previously, every UI (TUI, web, desktop) had to subscribe to dozens of granular Harness events and independently reconstruct what to display. This led to duplicated state tracking and inconsistencies across UI implementations. Now the Harness maintains a single canonical display state that any UI can read.

    Before: UIs subscribed to raw events and built up display state locally:

... (truncated)

Changelog

Sourced from @​mastra/evals's changelog.

1.7.0

Minor Changes

  • Added a summarization scorer to @mastra/evals. It grades a summary on two axes and returns the lower score, so a summary cannot pass by being faithful but empty, or thorough but wrong. (#20293)

    Alignment checks that every claim in the summary is supported by the source text. Coverage draws closed-ended questions from the source and answers them using the summary alone, in a separate call that never receives the source, so a missing fact cannot be answered from the source instead. The final score is min(alignment, coverage) × scale, and the reason names the axis that produced it.

    The source text defaults to the user message of the run input. Pass source or sourceExtractor when the text being summarized comes from somewhere else, such as a tool result. maxQuestions bounds the coverage questions so cost does not grow with document length.

    import { createSummarizationScorer } from '@mastra/evals/scorers/prebuilt';
    const scorer = createSummarizationScorer({
    model: 'openai/gpt-5.5',
    options: { maxQuestions: 10 },
    });
    const result = await scorer.run(run);
    result.score;

    This restores the summarization metric that was removed with the legacy evals system, rebuilt on the scorers pipeline.

Patch Changes

1.7.0-alpha.0

Minor Changes

  • Added a summarization scorer to @mastra/evals. It grades a summary on two axes and returns the lower score, so a summary cannot pass by being faithful but empty, or thorough but wrong. (#20293)

    Alignment checks that every claim in the summary is supported by the source text. Coverage draws closed-ended questions from the source and answers them using the summary alone, in a separate call that never receives the source, so a missing fact cannot be answered from the source instead. The final score is min(alignment, coverage) × scale, and the reason names the axis that produced it.

    The source text defaults to the user message of the run input. Pass source or sourceExtractor when the text being summarized comes from somewhere else, such as a tool result. maxQuestions bounds the coverage questions so cost does not grow with document length.

    import { createSummarizationScorer } from '@mastra/evals/scorers/prebuilt';
    const scorer = createSummarizationScorer({
    model: 'openai/gpt-5.5',
    options: { maxQuestions: 10 },
    });
    const result = await scorer.run(run);
    result.score;

... (truncated)

Commits

@dependabot dependabot Bot added dependencies Pull requests that update a dependency file javascript Pull requests that update javascript code labels Aug 10, 2026
@github-actions

github-actions Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Coverage Report for server

Status Category Percentage Covered / Total
🟢 Lines 98.38% (🎯 97%) 3173 / 3225
🟢 Statements 97.7% (🎯 96%) 3322 / 3400
🟢 Functions 98.74% (🎯 97%) 551 / 558
🟢 Branches 89.1% (🎯 87%) 1300 / 1459
File CoverageNo changed files found.
Generated in workflow #1928 for commit 684df28 by the Vitest Coverage Report Action

@github-actions

github-actions Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Coverage Report for web

Status Category Percentage Covered / Total
🟢 Lines 97.32% (🎯 96%) 1526 / 1568
🟢 Statements 96.24% (🎯 95%) 1613 / 1676
🟢 Functions 95.82% (🎯 95%) 643 / 671
🟢 Branches 90.77% (🎯 88%) 1013 / 1116
File CoverageNo changed files found.
Generated in workflow #1928 for commit 684df28 by the Vitest Coverage Report Action

@github-actions

github-actions Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Coverage Report for shared

Status Category Percentage Covered / Total
🟢 Lines 99.12% (🎯 98%) 226 / 228
🟢 Statements 81.2% (🎯 80%) 242 / 298
🟢 Functions 100% (🎯 98%) 76 / 76
🟢 Branches 63.52% (🎯 62%) 101 / 159
File CoverageNo changed files found.
Generated in workflow #1928 for commit 684df28 by the Vitest Coverage Report Action

@github-actions

github-actions Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Coverage Report for widget

Status Category Percentage Covered / Total
🟢 Lines 99.56% (🎯 98%) 229 / 230
🟢 Statements 99.19% (🎯 96%) 246 / 248
🟢 Functions 100% (🎯 98%) 45 / 45
🟢 Branches 95.96% (🎯 92%) 119 / 124
File CoverageNo changed files found.
Generated in workflow #1928 for commit 684df28 by the Vitest Coverage Report Action

@dependabot
dependabot Bot force-pushed the dependabot/npm_and_yarn/mastra/evals-1.7.0 branch from 01cf2f3 to 98f99c7 Compare August 12, 2026 06:51
@dependabot
dependabot Bot force-pushed the dependabot/npm_and_yarn/mastra/evals-1.7.0 branch from 98f99c7 to 81d4744 Compare August 21, 2026 01:43
Bumps [@mastra/evals](https://github.com/mastra-ai/mastra/tree/HEAD/packages/evals) from 1.6.0 to 1.7.0.
- [Release notes](https://github.com/mastra-ai/mastra/releases)
- [Changelog](https://github.com/mastra-ai/mastra/blob/main/packages/evals/CHANGELOG.md)
- [Commits](https://github.com/mastra-ai/mastra/commits/@mastra/evals@1.7.0/packages/evals)

---
updated-dependencies:
- dependency-name: "@mastra/evals"
  dependency-version: 1.7.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
@dependabot
dependabot Bot force-pushed the dependabot/npm_and_yarn/mastra/evals-1.7.0 branch from 81d4744 to 684df28 Compare August 21, 2026 02:36
@owk-owk130

Copy link
Copy Markdown
Contributor

core 1.52.1 据え置きのまま上げられることを確認済み(バンドル・lint・テストとも緑)。同じ衛星グループの #1029 に検証内容をまとめてあります。同グループの @mastra/memory の workerd 実機確認が済んだらまとめてマージします。

@owk-owk130
owk-owk130 merged commit 63d802e into develop Aug 21, 2026
2 checks passed
@owk-owk130
owk-owk130 deleted the dependabot/npm_and_yarn/mastra/evals-1.7.0 branch August 21, 2026 04:33
@github-actions github-actions Bot mentioned this pull request Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file javascript Pull requests that update javascript code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant