Skip to content

test(cli): snapshot the --json envelopes of map, doctor, config and logs - #1124

Merged
HugoRCD merged 1 commit into
v3from
test/cli-json-snapshots
Nov 17, 2026
Merged

HugoRCD merged 1 commit into
v3from
test/cli-json-snapshots

Conversation

@HugoRCD

@HugoRCD HugoRCD commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Written by an AI agent on behalf of @HugoRCD. @HugoRCD has reviewed it.

Part of the CLI 1.0 audit (EVL-574). This PR freezes the --json output of map, doctor, config and logs so 1.x can't change it by accident. evloghq/action reads map --json.

test/json-contract.test.ts runs each command through run(), the CLI's real entry, against one Hono fixture. The fixture has an evlog.config.ts, an installed evlog and two fs-drain events. It then snapshots the exit code and the parsed envelope. It masks only what differs between machines and releases: the temp dir (<cwd>), the CLI version, the Node version and map.generatedAt. It sets EVLOG_CLI_ENV=test, so environment is also stable. The tests pass with and without CI=true GITHUB_ACTIONS=true.

Snapshot excerpt (map):

"map": {
  "cliVersion": "<cli-version>",
  "framework": "hono",
  "projectName": "hono-basic-fixture",
  "ruleSetVersion": 3,
  "score": 82,
  "routes": [{ "id": "ec65818b4193", "method": "POST", "path": "/checkout", "score": 100,
    "event": { "audit": ["checkout.completed"], "fields": ["user.id"] }, ... }]
},
"summary": { "dark": 1, "exempt": 0, "instrumented": 1, "partial": 0, "suppressedChecks": 0 }

Test-only, so no changeset.

map --json against published CLIs

Same fixture, evlog map --no-write --no-header --json, all three exit 0. generatedAt, cliVersion and environment are normalised, keys sorted with jq -S:

=== 0.6.3 -> 0.10.0
>   "grade": "good",
>   "ignored": 0,
=== 0.10.0 -> v3 (1.0.0-beta.2)
>         "event": {
>           "audit": [
>             "checkout.completed"
>           ],
>           "fields": [
>             "user.id"
>           ]
>         },
<     "ruleSetVersion": 1,
---
>     "ruleSetVersion": 3,

For the CLI 1.0 release notes and evloghq/action:

Checks

  • pnpm --filter @evlog/cli exec vitest run: 931 passed
  • pnpm --filter @evlog/cli run typecheck, eslint on the new file: clean

ELI5

Some tools read the CLI's machine-readable output. These tests record exactly what that output looks like today, so any change to it shows up in review instead of breaking those tools.

@vercel

vercel Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

5 Skipped Deployments
Project Deployment Actions Updated
evi Skipped Skipped Nov 17, 2026 8:12pm UTC
evlog-docs Skipped Skipped v0 Nov 17, 2026 8:12pm UTC
evlog-render-lab Skipped Skipped Nov 17, 2026 8:12pm UTC
evlog-telemetry Skipped Skipped Nov 17, 2026 8:12pm UTC
just-use-evlog Skipped Skipped Nov 17, 2026 8:12pm UTC

Request Review

@coderabbitai

coderabbitai Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 90533efd-5dd8-4f17-90c0-8307fc1cb1ba

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@github-actions

Copy link
Copy Markdown
Contributor

Thank you for following the naming conventions! 🙏

@github-actions

Copy link
Copy Markdown
Contributor

Risk score: Low (1/4)

Assessed at ba9198f from the changed files and the title, by scripts/risk-score. A new push replaces this score.

  • On merge: nothing ships or deploys
  • 2 files changed (329 lines), 2 of them tests
Area Level Why
@evlog/cli Low Tests only: snapshots/json-contract.test.ts.snap, test/json-contract.test.ts

@pkg-pr-new

pkg-pr-new Bot commented Nov 17, 2026

Copy link
Copy Markdown
@evlog/cli

npm i https://pkg.pr.new/@evlog/cli@1124

evlog

npm i https://pkg.pr.new/evlog@1124

@evlog/nuxthub

npm i https://pkg.pr.new/@evlog/nuxthub@1124

@evlog/signals

npm i https://pkg.pr.new/@evlog/signals@1124

@evlog/telemetry

npm i https://pkg.pr.new/@evlog/telemetry@1124

commit: ba9198f

@HugoRCD
HugoRCD merged commit 75c7848 into v3 Nov 17, 2026
31 checks passed
@HugoRCD
HugoRCD deleted the test/cli-json-snapshots branch November 17, 2026 20:30

This branch was previously deployed

5 inactive deployments
Preview – just-use-evlog — ba9198f9 Deployed Nov 17, 2026 by vercel[bot]
Preview – evlog-docs — ba9198f9 Deployed Nov 17, 2026 by vercel[bot]
Preview – evlog-telemetry — ba9198f9 Deployed Nov 17, 2026 by vercel[bot]
Preview – evi — ba9198f9 Deployed Nov 17, 2026 by vercel[bot]
Preview – evlog-render-lab — ba9198f9 Deployed Nov 17, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant