Skip to content

[Bug]: Grok timeout golden test fails under coverage #86

Open

Description

@PerfectPan

Summary

tests/golden/news-tool.test.ts > golden news tool > falls back to DeepSeek when Grok times out failed once under pnpm test:coverage: the snapshot expected one Grok invocation and got none. An unchanged rerun passed.

Steps to Reproduce

  1. Run pnpm test:coverage while other heavy work runs on the machine (2026-11-18, on a branch that changed only package.json's verify script, docs, and two tests that pin that string).
  2. Rerun the same command: it passed (854/854); main also passed (854/854).

Expected Behavior

The snapshot matches on every run: Grok is invoked once, times out, and the tool falls back to DeepSeek.

Actual Behavior

FAIL  tests/golden/news-tool.test.ts > golden news tool > falls back to DeepSeek when Grok times out
Error: Snapshot `golden news tool > falls back to DeepSeek when Grok times out 1` mismatched
-   "grokInvocations": 1,
+   "grokInvocations": 0,
❯ Module.matchSnapshot tests/golden/harness.ts:312:3

Evidence

Environment

  • OS: macOS
  • Runtime: Node.js 26
  • Version: main at ef8788c

Impact

  • User-visible impact: none; the test's timing (the Grok timeout racing the coverage-instrumented run) makes it fail without a code change.
  • Data, security, or compatibility risk: none.
  • Release or rollback urgency: low.

Additional Context

grokInvocations 0 suggests the timeout fired before the Grok call was counted; the test likely depends on real time rather than a controlled clock.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions