docs(roadmap-planner): metrics audit (2026-05-19) + follow-up plan#188
Draft
danielfbm wants to merge 4 commits into
Draft
docs(roadmap-planner): metrics audit (2026-05-19) + follow-up plan#188danielfbm wants to merge 4 commits into
danielfbm wants to merge 4 commits into
Conversation
REPORT.md captures the state of the team / member metrics pipeline on 2026-05-19: 13 findings ranked by severity, with cross-checked numbers against GitHub Search, GitLab REST, Jira REST, and a snapshot of the prod SQLite. The biggest correctness issue is GitLab over-ingestion (`with_shared=true` default), which contaminates every downstream member metric — zhwang's reported 41 merged MRs are 100 % out-of-scope. PLAN.md breaks the findings into 10 numbered workstreams (W1..W10) sized for incremental PRs, with sequencing, risks, migrations, and 12 open questions for the maintainer. Nothing here changes runtime behaviour — it is the design document the upcoming PRs will execute.
Per Discord feedback on PR #188: - W1 redesigned: keep ingest scope WIDE (members contribute everywhere) + add per-member /merge_requests sweep + allowlist+denylist gate at calculation time; with_shared=false stays as the sub-fix. - W2 redesigned: instead of hiding bot reviews, reassign them to the existing synthetic "bot" member so volume stays visible without polluting humans (per Q10). - W4 status lists pulled live from Jira DEVOPS project — mapped Chinese + English statuses across all 17 issue types into todo / in_progress / done. - W5 elaborated: why epic_key is a misnomer and what breaks; hard-rename is safe (no API consumer). - W6 narrowed: changes_requested deferred per Q5; HydrateDiff flipped default to true. - W7 quarter source flagged: "2026Q1/Q2/Q4" labels are not in Jira fixVersions/sprints/JQL; 3 candidate sources listed for follow-up. - W8 confirmed (review latency p50 implementation, after W2). - W9 confirmed option (i): drop members.pillar_id, redefine cross_pillar_review_pct from PR-repo + Jira-component attribution. - W3 folded into W1 (same denylist). - W10 folded into W7. 4 follow-up questions remain: Q2-A (Cancelled bucket), Q4 (hard rename confirmation), Q7-A/B (quarter label source), Q10-B (bot seed list).
Last clarifications from maintainer (2026-05-19): - Q2-A: Cancelled / 已取消 stays under `done`. - Q4: hard-rename epic_key → jira_key everywhere (DB + Go + JSON tag). - Q7: quarter labels live as `YYYYQn` prefixes in Milestone Jira issue summaries (with FULL-WIDTH Chinese colon `：`, not ASCII `:`); Epics are linked to Milestones via blocking links. Verified live on jira.alauda.cn — sample milestones: DEVOPS-43598 "2026Q2：…", DEVOPS-42014 "2026Q1：…", DEVOPS-43568 "2026Q4：…". W7 redesigned to follow the Issue → Epic → Milestone → quarter chain, with dangling items falling back to calendar quarter of resolution_date / merged_at. New `quarter_assignments` table populated by a Milestone pass in the Jira sync (migration 0008). - Q10-B: explicit bot list only, no `*[bot]` / `*-bot` suffix magic. All 12 questions + 4 follow-ups resolved. Plan is fully specified.
…bucket) Maintainer clarified Q2-A: Cancelled / 已取消 gets its own counter, not merged into Done. Sprint card now reports todo / in_progress / done / cancelled separately so charts can include or exclude cancelled work without losing the count.
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Submits the metrics-pipeline audit and a structured follow-up plan as a docs-only PR. Nothing here changes runtime behaviour — these two markdown files exist so the next round of metric-fix PRs has a single source for what is broken and how each fix is sequenced.
docs/team-analytics/audit-2026-05-19/REPORT.md— 13 findings, cross-checked against GitHub Search, GitLab REST, Jira REST, and a snapshot of the prod SQLite atdevops-road.alaudatech.net. Headline:with_shared=trueis the GitLab API default, so the/groups/devops/projectscall pulls inalauda/artifacts,container-platform/*, andops/*— 59 % of GitLab rows in prod are outside the configured scope, andzhwang's reported 41 merged MRs are 100 % out-of-scope-noise.docs/team-analytics/audit-2026-05-19/PLAN.md— 10 workstreams (W1…W10) sized for incremental PRs, with migrations, rollback paths, tests, sequencing, and 12 open questions for the maintainer.Why a draft PR
The plan is gated on 12 design questions — until those are answered, the workstreams are scopable but not actionable. Once the maintainer responds, I'll convert each W block into a Jira task and start executing in order.
Background
Asked by maintainer in Discord on 2026-05-19 to audit team + individual metrics on the live deployment and produce evidence-backed findings. Evidence bundle (prod SQLite copy + raw API JSON + Go-based ad-hoc SQL runner) lives in
/tmp/audit/on the audit workstation and is referenced by file at the bottom ofREPORT.md.Test plan
git diff main…HEAD— docs-only changes🤖 Generated with Claude Code