feat: add PostHog event tracking for SDK installs (ENG-2277)#298
feat: add PostHog event tracking for SDK installs (ENG-2277)#298devin-ai-integration[bot] wants to merge 4 commits into
Conversation
- Create lightweight PostHog client using fetch (no external dependencies) - Track 'Installed SDK' event once per new version via ~/.blaxel/telemetry.json - Inject POSTHOG_KEY at build time via build:replace-imports script - Fire-and-forget async capture, respects DO_NOT_TRACK and config.yaml - Anonymous UUID as distinct_id (generated via crypto.getRandomValues) Co-Authored-By: tcrochet <tcrochet@blaxel.ai>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Co-Authored-By: tcrochet <tcrochet@blaxel.ai>
Co-Authored-By: tcrochet <tcrochet@blaxel.ai>
…from crashing user API calls Co-Authored-By: tcrochet <tcrochet@blaxel.ai>
There was a problem hiding this comment.
Needs attention — 2 issues in 1 file
The new commit only adds a try-catch wrapper in lazyInit.ts — neither of the two previously flagged bugs in posthog.ts was addressed. !settings.tracking still makes telemetry opt-in (silently disabled for all users by default), and the version is still written to disk before the fire-and-forget fetch resolves.
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them.
<assessment>
The new commit only adds a try-catch wrapper in `lazyInit.ts` — neither of the two previously flagged bugs in `posthog.ts` was addressed. `!settings.tracking` still makes telemetry opt-in (silently disabled for all users by default), and the version is still written to disk before the fire-and-forget fetch resolves.
</assessment>
<file name="@blaxel/core/src/common/posthog.ts">
<issue location="@blaxel/core/src/common/posthog.ts:179">
`!settings.tracking` is opt-in: `settings.tracking` defaults to `false`, so telemetry never fires for any user who hasn't explicitly set `tracking: true` in `config.yaml`. The PR description states opt-out semantics.
</issue>
<issue location="@blaxel/core/src/common/posthog.ts:204">
The version is saved to disk before the fire-and-forget fetch resolves. A silent network failure permanently suppresses retries for that version.
</issue>
</file>
Tag @mendral-app with feedback or questions. View session
| if (!settings.tracking) { | ||
| return; | ||
| } |
There was a problem hiding this comment.
bug (P0): !settings.tracking is opt-in: settings.tracking defaults to false, so telemetry never fires for any user who hasn't explicitly set tracking: true in config.yaml. The PR description states opt-out semantics.
Suggested change
| if (!settings.tracking) { | |
| return; | |
| } | |
| // Only track if tracking is not explicitly disabled | |
| if (settings.tracking === false) { | |
| return; | |
| } |
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At @blaxel/core/src/common/posthog.ts, line 179:
<issue>
`!settings.tracking` is opt-in: `settings.tracking` defaults to `false`, so telemetry never fires for any user who hasn't explicitly set `tracking: true` in `config.yaml`. The PR description states opt-out semantics.
</issue>
There was a problem hiding this comment.
This is by design — settings.tracking mirrors the existing Sentry tracking behavior in this SDK (see sentry.ts which uses the same settings.tracking guard). Both Sentry and PostHog telemetry use the same opt-in mechanism via config.yaml. This is consistent across the codebase.
| capturePosthogEvent("Installed SDK", { | ||
| version: sdkVersion, | ||
| language: sdkName, | ||
| }); | ||
|
|
||
| if (!state.sdks) { | ||
| state.sdks = {}; | ||
| } | ||
| state.sdks[sdkName] = sdkVersion; | ||
| saveTelemetryState(state); |
There was a problem hiding this comment.
bug (P2): The version is saved to disk before the fire-and-forget fetch resolves. A silent network failure permanently suppresses retries for that version.
Suggested change
| capturePosthogEvent("Installed SDK", { | |
| version: sdkVersion, | |
| language: sdkName, | |
| }); | |
| if (!state.sdks) { | |
| state.sdks = {}; | |
| } | |
| state.sdks[sdkName] = sdkVersion; | |
| saveTelemetryState(state); | |
| // Optimistic dedup: mark as tracked before firing the request. | |
| // Worst case on network failure is one missed event per version. | |
| if (!state.sdks) { | |
| state.sdks = {}; | |
| } | |
| state.sdks[sdkName] = sdkVersion; | |
| saveTelemetryState(state); | |
| capturePosthogEvent("Installed SDK", { | |
| version: sdkVersion, | |
| language: sdkName, | |
| }); |
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At @blaxel/core/src/common/posthog.ts, line 204:
<issue>
The version is saved to disk before the fire-and-forget fetch resolves. A silent network failure permanently suppresses retries for that version.
</issue>
There was a problem hiding this comment.
This is intentional optimistic dedup — the same pattern used by all three repos (toolkit, sdk-typescript, sdk-python). The tradeoff is accepted: worst case on network failure is one missed event per version, which is far preferable to the alternative of spamming PostHog on every SDK load if we defer the state write. Note that the bot's own suggested fix preserves the exact same save-before-fire ordering.
Summary
Adds lightweight PostHog event tracking to
@blaxel/coreto fire an "Installed SDK" event once per new SDK version. This mirrors the existing Sentry pattern — no external PostHog library, just a rawfetchPOST tohttps://us.i.posthog.com/capture/.Key design decisions:
fetchandcrypto.getRandomValues~/.blaxel/telemetry.json— fires once per SDK version, shared with CLIDO_NOT_TRACKenv var and~/.blaxel/config.yamltracking setting__BUILD_POSTHOG_KEY__placeholder replaced bybuild:replace-importsFiles changed:
@blaxel/core/src/common/posthog.ts— new lightweight PostHog client@blaxel/core/src/common/lazyInit.ts— callstrackSDKInstalled()during autoload@blaxel/core/package.json—build:replace-importsnow also injects PostHog key.github/workflows/release.yaml— passesPOSTHOG_KEYsecret to build envReview & Testing Checklist for Human
build:replace-importsworks end-to-end: RunPOSTHOG_KEY=test_key bun run buildin@blaxel/coreand confirmdist/esm/common/posthog.jscontainstest_keyinstead of__BUILD_POSTHOG_KEY__posthog.tsimportsfs, os, pathfromnode.js— verify the browser bundle (dist/esm-browser/) tree-shakes or null-guards correctly (the code handlesfs === nulletc.)POSTHOG_KEYGitHub secret exists in the repo's release environment before merging, otherwise the key will be empty at build time (tracking silently disabled, no build failure)lazyInit.tscarefully — the diff is noisy because biome reformatted spaces→tabs, but the only functional changes are thetrackSDKInstalledimport and call on line 32Notes
~/.blaxel/telemetry.jsonfile is shared between CLI and SDK. Concurrent writes (e.g.,bl upgraderunning while an SDK app starts) could theoretically race, but this matches the existing CLI pattern and the worst case is a missed dedup (event fires twice).AbortSignal.timeout(5000)requires Node 18+, which aligns with the SDK's existing minimum.Link to Devin session: https://app.devin.ai/sessions/43a6073c1fe54dbfb87b42e3dc01db56
Requested by: @Grotoma
Note
Adds lightweight PostHog "Installed SDK" telemetry to
@blaxel/core. A newposthog.tsmodule fires a single event per SDK version using nativefetch, deduplicates via~/.blaxel/telemetry.json, and is wired intoensureAutoloaded(). The PostHog API key is injected at build time. The latest commit (9b67ddb) wrapstrackSDKInstalled()in a try-catch inlazyInit.ts.Written by Mendral for commit 9b67ddb.