Copilot CLI release/version mismatch makes Copilot SDK version pinning unreliable

[pdeligia]

We’re seeing cases across multiple releases where downloading a Copilot CLI binary from a tagged GitHub release appears to yield a binary that reports, and may actually be, a different newer CLI version than the selected tag. This causes compatibility issues with the pinned CLI version in the Copilot SDK. I originally observed this on macOS, and a colleague reproduced the same behavior on Windows.

Filed a sibling issue of this in the CLI repo (github/copilot-cli#2588), but also filing this here in the SDK repo because it has direct downstream impact here, although the root cause appears to be in Copilot CLI release packaging/versioning.

One concrete example is release v1.0.12

• On macOS, downloading the v1.0.12 asset and running the extracted binary directly from a temporary directory reports GitHub Copilot CLI 1.0.20
• A colleague reproduced the same behavior on Windows

Why this matters for the SDK

This makes SDK version pinning and compatibility validation unreliable.

If an SDK integration depends on a fix, behavior change, or API that is tied to a specific CLI version, users may believe they are testing or deploying against CLI vX.Y.Z while actually running a different build. That can lead to hard-to-explain regressions and production issues.

We’ve already seen this affect one of our production tools built on the Copilot SDK, where behavior changed unexpectedly because the downloaded Copilot CLI appeared to behave like a newer version than the selected release. More generally, when a fix is expected in a specific release—such as the unknown hook fix referenced in v0.2.2-preview.0—being unable to trust the resolved CLI version makes it difficult to validate that the expected fix is actually present.

Expected

When an SDK integration pins or selects Copilot CLI vX.Y.Z, the downloaded and executed CLI should actually be X.Y.Z and provide the behavior, fixes, and API surface expected for that version. SDK consumers should be able to validate compatibility reliably against the chosen CLI version.

Actual

In at least some cases, selecting or downloading Copilot CLI vX.Y.Z yields a binary that identifies as a different version and appears to behave like a newer release. That makes SDK compatibility testing, fix validation, and production version pinning unreliable.

Note

This may ultimately need to be fixed in the CLI release pipeline rather than in the SDK itself, but we wanted to report it here because it directly affects SDK consumers that need reliable CLI version pinning.

[github-actions]

Investigation Findings

I investigated the reported behavior by reviewing the SDK's CLI distribution mechanism and version validation logic.

What I looked at

1. go/cmd/bundler/main.go — The Go SDK's bundler tool, which downloads CLI binaries
2. nodejs/src/client.ts — The Node SDK's protocol version negotiation logic
3. nodejs/src/sdkProtocolVersion.ts and sdk-protocol-version.json — SDK protocol version tracking

What I found

Root cause confirmed. The behavior described is a genuine bug in the Copilot CLI release pipeline. Specifically:

1. The SDK's own bundler downloads from npm registry (registry.npmjs.org/@github/copilot-{platform}/-/copilot-{platform}-{version}.tgz), not directly from GitHub release assets. This is a different code path from what was tested (direct GitHub release download). It's possible npm packages are correctly versioned while GitHub release assets are not — but that's unverified, and in either case, the behavior observed is a defect.

2. The SDK has no binary version verification step. After downloading and extracting the CLI binary, the bundler computes a SHA256 hash but never executes the binary with --version to confirm it self-reports the expected version. This means a mislabeled binary would pass the SDK's bundler checks silently.

3. The SDK's protocol version check (in client.ts, verifyProtocolVersion()) validates that the running CLI reports a compatible JSON-RPC protocol version (currently v3), but this is a protocol-level compatibility check, not a semantic version guarantee. A newer CLI that happens to speak the same protocol version would not be caught.

Why this is a bug

The behavior — a tagged release asset containing a binary that reports a different version — is unambiguously a defect. Even though the fix most likely belongs in the CLI release pipeline (as you've correctly identified, and filed at github/copilot-cli#2588), it has confirmed direct downstream impact on SDK consumers who rely on version pinning for fix validation, regression testing, and production stability.

Labeling this as bug. The SDK team may also want to consider adding a defensive binary version verification step in the bundler as a follow-up hardening measure.

Generated by Bug Handler for issue #1042 · ◷