Skip to content

feat: add PostHog event tracking for SDK installs (ENG-2277)#298

Open
devin-ai-integration[bot] wants to merge 4 commits into
mainfrom
thomas-devin/track-cli-sdk-installs-posthog
Open

feat: add PostHog event tracking for SDK installs (ENG-2277)#298
devin-ai-integration[bot] wants to merge 4 commits into
mainfrom
thomas-devin/track-cli-sdk-installs-posthog

Conversation

@devin-ai-integration
Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot commented May 5, 2026

Summary

Adds lightweight PostHog event tracking to @blaxel/core to fire an "Installed SDK" event once per new SDK version. This mirrors the existing Sentry pattern — no external PostHog library, just a raw fetch POST to https://us.i.posthog.com/capture/.

Key design decisions:

  • No new dependencies — uses native fetch and crypto.getRandomValues
  • Deduplication via ~/.blaxel/telemetry.json — fires once per SDK version, shared with CLI
  • Consent — respects DO_NOT_TRACK env var and ~/.blaxel/config.yaml tracking setting
  • Fire-and-forget — async POST with 5s timeout, all failures silently swallowed
  • Node.js only — skips in browser environments (no filesystem access)
  • Build-time key injection__BUILD_POSTHOG_KEY__ placeholder replaced by build:replace-imports

Files changed:

  • @blaxel/core/src/common/posthog.ts — new lightweight PostHog client
  • @blaxel/core/src/common/lazyInit.ts — calls trackSDKInstalled() during autoload
  • @blaxel/core/package.jsonbuild:replace-imports now also injects PostHog key
  • .github/workflows/release.yaml — passes POSTHOG_KEY secret to build env

Review & Testing Checklist for Human

  • Verify build:replace-imports works end-to-end: Run POSTHOG_KEY=test_key bun run build in @blaxel/core and confirm dist/esm/common/posthog.js contains test_key instead of __BUILD_POSTHOG_KEY__
  • Check browser build safety: posthog.ts imports fs, os, path from node.js — verify the browser bundle (dist/esm-browser/) tree-shakes or null-guards correctly (the code handles fs === null etc.)
  • Verify POSTHOG_KEY GitHub secret exists in the repo's release environment before merging, otherwise the key will be empty at build time (tracking silently disabled, no build failure)
  • Review lazyInit.ts carefully — the diff is noisy because biome reformatted spaces→tabs, but the only functional changes are the trackSDKInstalled import and call on line 32

Notes

  • The ~/.blaxel/telemetry.json file is shared between CLI and SDK. Concurrent writes (e.g., bl upgrade running while an SDK app starts) could theoretically race, but this matches the existing CLI pattern and the worst case is a missed dedup (event fires twice).
  • AbortSignal.timeout(5000) requires Node 18+, which aligns with the SDK's existing minimum.

Link to Devin session: https://app.devin.ai/sessions/43a6073c1fe54dbfb87b42e3dc01db56
Requested by: @Grotoma


Open in Devin Review

Note

Adds lightweight PostHog "Installed SDK" telemetry to @blaxel/core. A new posthog.ts module fires a single event per SDK version using native fetch, deduplicates via ~/.blaxel/telemetry.json, and is wired into ensureAutoloaded(). The PostHog API key is injected at build time. The latest commit (9b67ddb) wraps trackSDKInstalled() in a try-catch in lazyInit.ts.

Written by Mendral for commit 9b67ddb.

- Create lightweight PostHog client using fetch (no external dependencies)
- Track 'Installed SDK' event once per new version via ~/.blaxel/telemetry.json
- Inject POSTHOG_KEY at build time via build:replace-imports script
- Fire-and-forget async capture, respects DO_NOT_TRACK and config.yaml
- Anonymous UUID as distinct_id (generated via crypto.getRandomValues)

Co-Authored-By: tcrochet <tcrochet@blaxel.ai>
@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR that start with 'DevinAI' or '@devin'.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

mendral-app[bot]

This comment was marked as outdated.

devin-ai-integration[bot]

This comment was marked as resolved.

Co-Authored-By: tcrochet <tcrochet@blaxel.ai>
mendral-app[bot]

This comment was marked as outdated.

devin-ai-integration[bot]

This comment was marked as resolved.

mendral-app[bot]

This comment was marked as outdated.

devin-ai-integration[bot]

This comment was marked as resolved.

…from crashing user API calls

Co-Authored-By: tcrochet <tcrochet@blaxel.ai>
Copy link
Copy Markdown
Contributor

@mendral-app mendral-app Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs attention — 2 issues in 1 file

The new commit only adds a try-catch wrapper in lazyInit.ts — neither of the two previously flagged bugs in posthog.ts was addressed. !settings.tracking still makes telemetry opt-in (silently disabled for all users by default), and the version is still written to disk before the fire-and-forget fetch resolves.

Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them.

<assessment>
The new commit only adds a try-catch wrapper in `lazyInit.ts` — neither of the two previously flagged bugs in `posthog.ts` was addressed. `!settings.tracking` still makes telemetry opt-in (silently disabled for all users by default), and the version is still written to disk before the fire-and-forget fetch resolves.
</assessment>

<file name="@blaxel/core/src/common/posthog.ts">
<issue location="@blaxel/core/src/common/posthog.ts:179">
`!settings.tracking` is opt-in: `settings.tracking` defaults to `false`, so telemetry never fires for any user who hasn't explicitly set `tracking: true` in `config.yaml`. The PR description states opt-out semantics.
</issue>
<issue location="@blaxel/core/src/common/posthog.ts:204">
The version is saved to disk before the fire-and-forget fetch resolves. A silent network failure permanently suppresses retries for that version.
</issue>
</file>

Tag @mendral-app with feedback or questions. View session

Comment on lines +179 to +181
if (!settings.tracking) {
return;
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bug (P0): !settings.tracking is opt-in: settings.tracking defaults to false, so telemetry never fires for any user who hasn't explicitly set tracking: true in config.yaml. The PR description states opt-out semantics.

Suggested change
Suggested change
if (!settings.tracking) {
return;
}
// Only track if tracking is not explicitly disabled
if (settings.tracking === false) {
return;
}
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At @blaxel/core/src/common/posthog.ts, line 179:

<issue>
`!settings.tracking` is opt-in: `settings.tracking` defaults to `false`, so telemetry never fires for any user who hasn't explicitly set `tracking: true` in `config.yaml`. The PR description states opt-out semantics.
</issue>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is by design — settings.tracking mirrors the existing Sentry tracking behavior in this SDK (see sentry.ts which uses the same settings.tracking guard). Both Sentry and PostHog telemetry use the same opt-in mechanism via config.yaml. This is consistent across the codebase.

Comment on lines +204 to +213
capturePosthogEvent("Installed SDK", {
version: sdkVersion,
language: sdkName,
});

if (!state.sdks) {
state.sdks = {};
}
state.sdks[sdkName] = sdkVersion;
saveTelemetryState(state);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bug (P2): The version is saved to disk before the fire-and-forget fetch resolves. A silent network failure permanently suppresses retries for that version.

Suggested change
Suggested change
capturePosthogEvent("Installed SDK", {
version: sdkVersion,
language: sdkName,
});
if (!state.sdks) {
state.sdks = {};
}
state.sdks[sdkName] = sdkVersion;
saveTelemetryState(state);
// Optimistic dedup: mark as tracked before firing the request.
// Worst case on network failure is one missed event per version.
if (!state.sdks) {
state.sdks = {};
}
state.sdks[sdkName] = sdkVersion;
saveTelemetryState(state);
capturePosthogEvent("Installed SDK", {
version: sdkVersion,
language: sdkName,
});
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At @blaxel/core/src/common/posthog.ts, line 204:

<issue>
The version is saved to disk before the fire-and-forget fetch resolves. A silent network failure permanently suppresses retries for that version.
</issue>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is intentional optimistic dedup — the same pattern used by all three repos (toolkit, sdk-typescript, sdk-python). The tradeoff is accepted: worst case on network failure is one missed event per version, which is far preferable to the alternative of spamming PostHog on every SDK load if we defer the state write. Note that the bot's own suggested fix preserves the exact same save-before-fire ordering.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant