tools(sentry): measure gate uptake from the population that emits the errors - #176
Merged
Conversation
… errors
The AGE-105 grader multiplies the baseline by `1 - gated_share * efficacy`, so
`gated_share` decides the verdict. Its only source was Play's version share,
which cannot see this app's primary install channels: the self-hosted F-Droid
repo and the direct-APK release, both of which are what the marketing pages
actually point at.
Sentry release health can. Sessions are stored under a separate quota from
errors, so they keep arriving while the org is over its error quota and every
error is rate-limited away — which is exactly when this number is needed. It is
also the right population by construction: a device that sends a session is a
device that can send an error.
First live reading (7d to 2026-08-14, org vibetechnologies):
0.4.10 801 users 78.5% ungated
0.4.12 161 users 15.8% ungated
0.4.14 7 users 0.7% gated
gated share 0.7% (users) / 0.2% (sessions)
v0.4.10 has held ~78% for four weeks and v0.4.12 sat flat at ~16% for three,
so this install base does not auto-converge on the newest build. Until that
cohort moves, the ceiling on any client-side volume reduction is ~21.5%.
Tests pin the parts that fail silently: 0.4.9 vs 0.4.10 ordering, unparseable
releases staying in the denominator (dropping them inflates the share), an
empty window returning null rather than 0, user-vs-session divergence, and a
projection that refuses horizons and flat series instead of inventing uptake.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The number that decides AGE-105 was coming from a population that cannot see most of our users
noise-gate-report.mjsgrades the noise gate withgated_sharecame from Play's version share. This app's marketed install paths are the self-hosted F-Droid repo and the direct APK (docs/marketing/*), and Play production had been serving versionCode 136 (2026-06-22) for eight weeks when the gate shipped. Play's denominator cannot contain those users; Sentry's can, because it is built from the devices that actually send events.scripts/release-uptake.mjsreads gated share from Sentry release health. Sessions are billed against a different quota than errors, so they keep arriving while the org is over its error quota and every error is rate-limited away — precisely when this measurement is needed. It also needs no Play service account, so it runs on a laptop instead of only in CI.First live reading — 7d to 2026-08-14
v0.4.10 has held ~78% for four weeks; v0.4.12 sat flat at ~16% for three. This install base does not auto-converge on the newest build, which is the assumption written into the Play-based path ("Play auto-updates, so its active base converges on the newest build fast"). Until that cohort moves, the ceiling on any client-side reduction is ~21.5%, whatever the gate does on-device.
What the tests pin
Every one of these fails silently in production if it regresses:
0.4.9vs0.4.10ordering — lexical compare puts them on the wrong side of the gate.share: null, never0, so "no data" cannot grade as "nobody upgraded".node --test scripts/release-uptake.test.mjs— 10 pass. Already covered bynpm test(scripts/*.test.mjs).Deliberately not wired into
noise-gate-report.mjshereThat file is modified by both #175 and #174. This PR is new files only, so it cannot conflict with or re-orphan that stack. Wiring the Sentry source in as the preferred
gated_share(Play as cross-check) follows once they land.Refs AGE-105.