Skip to content

feat: add PostHog event tracking for CLI installs and upgrades (ENG-2277)#293

Open
devin-ai-integration[bot] wants to merge 5 commits into
mainfrom
thomas-devin/track-cli-sdk-installs-posthog
Open

feat: add PostHog event tracking for CLI installs and upgrades (ENG-2277)#293
devin-ai-integration[bot] wants to merge 5 commits into
mainfrom
thomas-devin/track-cli-sdk-installs-posthog

Conversation

@devin-ai-integration
Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot commented May 5, 2026

Summary

Adds lightweight PostHog event tracking to the CLI with no new dependencies — uses only net/http and crypto/rand from stdlib. Mirrors the existing Sentry integration pattern (build-time key injection, opt-in consent via IsTrackingEnabled()).

Two events are tracked:

  • "Installed CLI" — fires once per new CLI version on startup, deduplicated via ~/.blaxel/telemetry.json
  • "Upgraded CLI" — fires after a successful bl upgrade with old_version and new_version properties

Events are fire-and-forget (async goroutine, 5s timeout). The PostHog API key is injected at build time via ldflags, same as sentryDSN.

Files changed:

  • cli/core/posthog.go (new) — PostHog client, telemetry state persistence, UUID generation, deduplication logic
  • main.go — adds posthogKey ldflag var, initializes PostHog when tracking is enabled
  • cli/core/root.go — calls TrackCLIInstalled(version) in Execute()
  • cli/upgrade.go — captures old version, detects new version post-upgrade, fires TrackCLIUpgraded
  • .goreleaser.yaml / Makefile / .github/workflows/release.yaml — build-time key injection

Review & Testing Checklist for Human

  • POSTHOG_KEY GitHub secret must be created before the next release. Verify that GoReleaser doesn't fail when POSTHOG_KEY env var is unset/empty (it may need to be set to "" like SENTRY_DSN).
  • detectInstalledVersion() output parsing — the version command uses core.Print/core.PrintInfo which may emit ANSI color codes. Verify that parsing "Version: X.Y.Z" works correctly when the new binary's stdout is not a TTY (piped through exec.Command(...).Output()).
  • FlushPosthog() uses time.Sleep(500ms) instead of a sync.WaitGroup — events could be silently dropped if the HTTP POST takes longer. Acceptable trade-off? Consider if a WaitGroup approach would be better.
  • loadTelemetryState uses sync.Once — telemetry state is loaded once per process. This is fine for normal CLI usage but means the state won't be re-read within a single process lifetime (relevant for upgrade flow where TrackCLIUpgraded then updates state).
  • Test the full flow locally: build with POSTHOG_KEY=phc_test make build-dev, run bl version twice, verify ~/.blaxel/telemetry.json is created with correct structure, and that only one "Installed CLI" event would fire.

Notes

  • No unit tests added for posthog.go — the code is intentionally minimal and mirrors the untested Sentry client pattern in this repo.
  • The getDistinctID() comment mentions "workspace email" but the implementation only uses a persisted anonymous UUID. The comment should be updated or the email lookup added if that's desired.
  • The SDKs field in telemetryState is forward-looking — it's unused by the CLI but will be used by the SDK repos (sharing the same ~/.blaxel/telemetry.json file).

ENG-2277

Link to Devin session: https://app.devin.ai/sessions/43a6073c1fe54dbfb87b42e3dc01db56
Requested by: @Grotoma


Open in Devin Review

Note

Adds lightweight PostHog event tracking to the CLI using only stdlib (net/http, crypto/rand). Tracks "Installed CLI" once per new version (deduplicated via ~/.blaxel/telemetry.json) and "Upgraded CLI" after a successful bl upgrade. Mirrors the existing Sentry integration pattern with build-time key injection and opt-in via IsTrackingEnabled().

Written by Mendral for commit 9e1da84.

- Add lightweight PostHog client (cli/core/posthog.go) using raw net/http
- Track 'Installed CLI' event on first run of each new version
- Track 'Upgraded CLI' event with old/new versions after successful upgrade
- Deduplicate events via ~/.blaxel/telemetry.json
- Inject PostHog API key at build time via ldflags
- Fire-and-forget async HTTP POST, non-blocking

ENG-2277

Co-Authored-By: tcrochet <tcrochet@blaxel.ai>
@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR that start with 'DevinAI' or '@devin'.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

mendral-app[bot]

This comment was marked as outdated.

…detection, comment fix

- Use conditional template in .goreleaser.yaml so POSTHOG_KEY env var is optional
- Clarify detectInstalledVersion re-resolves symlinks for brew upgrades
- Fix misleading comment on getDistinctID (UUID only, no email)

Co-Authored-By: tcrochet <tcrochet@blaxel.ai>
mendral-app[bot]

This comment was marked as outdated.

On macOS, os.Executable() returns the already-resolved cellar path,
so EvalSymlinks cannot follow the updated symlink after brew upgrade.
Using exec.LookPath finds the binary by name in PATH, which resolves
through the updated /usr/local/bin symlink to the new cellar entry.

Co-Authored-By: tcrochet <tcrochet@blaxel.ai>
mendral-app[bot]

This comment was marked as outdated.

devin-ai-integration[bot]

This comment was marked as resolved.

…erve unknown telemetry fields

Co-Authored-By: tcrochet <tcrochet@blaxel.ai>
mendral-app[bot]

This comment was marked as outdated.

…ss on os.Exit

Co-Authored-By: tcrochet <tcrochet@blaxel.ai>
Copy link
Copy Markdown
Contributor

@mendral-app mendral-app Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

All previous comments are resolved. The final commit correctly adds FlushPosthog() to ExitWithError, ExitWithMessage, and Exit — since os.Exit skips deferred calls, these explicit flushes are necessary and correct. Ready to merge.

Tag @mendral-app with feedback or questions. View session

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant