Measure lead time per commit, as DORA defines it - #69
Merged
Merged
Conversation
Lead time was the newest commit in each release, which answers "how stale was the freshest change" rather than "how long from commit to running". On the production export the difference is sevenfold: 26.7 h as a proxy against a real 193.6 h median, and the proxy is the flattering one. Three views do it properly. commit_sightings expands the commits array that push payloads already carry - id, author timestamp, parents, author type - into one row per commit on master. deploy_commit_ranges says which commit range a deploy shipped, today from the release-note bump links; that is the one view image_ref will replace, with "between the previous and current deploy of this app". lead_time_changes maps each commit to the first deploy per environment that carried it, so a release rolling out to four Apps counts each change once. The SQL lives in riptide_collector.views so the migration and the tests run the same text - an untestable metric definition drifts. Merge commits and service-account commits are classified, not dropped, and the documented default excludes both: they are artifacts of shipping, not changes that shipped. Bots get their own line rather than being blended, since they authored 87 percent of commits reaching production. Measured against the export: prod human p50 193.6 h / p90 505.6 h over 984 changes, intg p50 4.0 h. The gap between the two is the finding - delivery is fast, the wait sits in front of production.
- Join the App repo by exact slug instead of LIKE, and widen the slug pattern to allow '_' and '.', which are legal and would otherwise yield no ranges at all - with LIKE, '_' would also have matched any character. - Declare the new index on the model too, so autogenerate does not propose dropping it. - Doc query: match bots on login and display name, the same rule the rest of riptide uses, and with NOT EXISTS so an unresolved author is kept rather than dropped by NULL propagation. - Coverage query counted bumps, not deploys, which understated the unresolved share. Counts distinct deploys now: 365 of 1007 unresolved on the export. - Write down the two boundary assumptions rather than leaving them implicit: boundary SHAs and the GitOps release commit are push tips, and a missing boundary costs a whole release window, not one change, because the next range starts where this one ended. - Test a two-commit push, which nothing covered: both commits are attributed to the same release and each keeps its own commit timestamp. - README no longer suggests stratifying by change_type, which these commits do not carry.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lead time was measured as "the newest commit in each release" — it answers how stale the freshest change was, not DORA's "how long from commit to running in production". On the production export the difference is sevenfold, and the proxy is the flattering one:
The gap between intg and prod is the actual finding: delivery is fast, the wait sits entirely in front of production.
How
Three views, layered so the fragile part is swappable:
commit_sightings— expands thecommits[]array that push payloads already carry (id,authorTimestamp,parents, author + type) into one row per commit on master. Measured: the array is capped at 5 per push, which bites 0.6 % of master pushes.deploy_commit_ranges— which commit range a deploy shipped, read today from the release-note bump links. This is the only viewimage_refreplaces, with "between the previous and the current successful deploy of this app" — no release notes, both release paths.lead_time_changes— each commit against the first deploy per environment that carried it, so a release rolling out to four Apps counts each change once.The SQL lives in
riptide_collector/views.py, imported by the migration and by the tests, so the definition that ships is the definition under test.What counts as a change
Merge commits and service-account commits are classified, not dropped; the documented default excludes both — they are artifacts of shipping rather than changes that shipped. Bots get their own line instead of being blended: Renovate authored 87 % of commits reaching production here, so a blended median (209.3 h) describes the bot's cadence, not the team's.
Verification
Docs:
docs/dora-lead-time.md(definition, filters, limits), README lead-time row rewritten, one AGENTS.md line so the proxy does not come back.