Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/branch-lines.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

- 每个 run 归入它的工作所针对的分支:PR 运行归入 PR 的目标分支,push、手动、定时运行归入运行所在分支。
- 目标分支不是任何已配置分支线的 run(版本 tag、其他分支的 push、无法匹配 PR 的运行)归入默认分支线,因此每个 run 只计一次。Nightly 按运行所在分支归线,手动在 release 分支执行也归 release 线。
- 每条分支线都展示窗口内归入该线的全部已采集 run:PR、分支门禁、Daily、版本、工作流调用、其他工作流、未知触发七层;`workflow_call` 子 run 按各自 run ID 单独显示,不被父 run 替代。
- 每条分支线按 PR、分支门禁、合并队列、Daily、版本五层展示;`merge_group` 优先归入合并队列。工作流调用、其他工作流与无事件且无关联 PR 的门禁 run 不画点,也不通过脚注或最近 run 表另列;采集仍保存全部 run。

## 使用

Expand All @@ -33,7 +33,7 @@

| 页面 | 内容 |
| --- | --- |
| CI | 该分支 push / 手动门禁和目标为它的 PR 的成功率、包含全部运行类别的分层历史、Workflow 与 Job 健康、最近 run;门禁层显示完整分支名,不标成“主干” |
| CI | 该分支 push / 手动门禁和目标为它的 PR 的成功率、上述五层历史、Workflow 与 Job 健康、最近 run;门禁层显示完整分支名,不标成“主干” |
| 测试 | 最近用例数,以及该分支最新 push / 手动运行冻结的[标签化测试](tagged-tests.md)目录和规则;没有运行过的组合不显示 |
| Coverage | 只用该分支线运行上传的覆盖率摘要;完整基线来自该分支完整通过的 push 或手动门禁;PR 列表只含目标为它的 PR |

Expand Down Expand Up @@ -63,4 +63,4 @@ PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tests -p 'test_lines.p
node .e2e/node_modules/playwright/cli.js test --config test/playwright.config.cjs -g "CI, tests and coverage"
```

单元测试覆盖配置校验、同仓 / fork / 手动 / tag 运行的归线、release 手动 Nightly 与 legacy Daily 可见、工作流调用和其他工作流不被排除、旧分层快照重建、各线成功率、覆盖率产物归线、标签化目录分线与迁移。浏览器用例在桌面和窄屏验证切换、跨页与刷新后保持、Daily/调用/其他运行链接、各线数据不混合,以及总览列出其他分支线的失败。
单元测试覆盖配置校验、同仓 / fork / 手动 / tag 运行的归线、release 手动 Nightly 与 legacy Daily 可见、合并队列优先归类、工作流调用和其他工作流不展示但保留记录、旧分层快照重建、各线成功率、覆盖率产物归线、标签化目录分线与迁移。浏览器用例在桌面和窄屏验证切换、跨页与刷新后保持、Daily 与合并队列链接、空合并队列、各线数据不混合,以及总览列出其他分支线的失败。
14 changes: 6 additions & 8 deletions docs/ci-history-lanes.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,21 +2,19 @@

## 功能

CI 页的「CI 分层历史」按触发方式展示该分支线窗口内的全部已采集 Actions run,七层共用一条日期横轴:
CI 页的「CI 分层历史」按触发方式展示该分支线窗口内的 PR、分支门禁、合并队列、Daily 与版本运行,五层共用一条日期横轴:

| 层 | 归入的 run |
| --- | --- |
| PR | 门禁工作流(默认名称 `^CI$`)的 `pull_request` / `pull_request_target` |
| 主干 / 完整分支名 | 门禁工作流在该分支线上的 `push`,以及它自己的 `workflow_dispatch`(提示中标“手动”) |
| 合并队列 | 所有 `event=merge_group` 的 run,优先于工作流名称规则 |
| Daily | 名称匹配 daily 规则(Nightly)的 run,包括 `schedule` 和手动 `workflow_dispatch` |
| 版本 | 名称匹配 release 规则(Release)的 run,包括版本 tag 推送或手动运行 |
| 工作流调用 | `event=workflow_call` 的 run,以及门禁工作流带有其他调用事件(如 `schedule`)的 run |
| 其他工作流 | 不匹配上述名称规则的工作流,以及归入该线但不匹配其门禁层的分支事件 |
| 未知触发 | 缺少触发事件、且无法通过关联 PR 归类的门禁 run |

[分支线](branch-lines.md)的每条线都有上述完整分层。非默认分支的 push / 手动门禁层使用完整分支名;例如在 `releases/v0.3.0.beta` 手动运行 Nightly,会出现在该分支线的 Daily 层。

名称规则取 `board-config.json` 的 `workflows.gate` / `daily` / `release`,缺省时使用相同默认值。分层不会排除其他类别:若 GitHub 返回独立的 `workflow_call` 子 run,就按其 run ID 在工作流调用层单独画点。仅属于父 run 的 job 不会被虚构成新 run;同一 run 的重试仍只展示最后一次尝试。其他工作流和无法判断触发方式的旧记录也各有显示层,不再只留脚注计数。
`merge_group` 优先归入合并队列;其余名称规则取 `board-config.json` 的 `workflows.gate` / `daily` / `release`,缺省时使用相同默认值。`workflow_call` 子 run、其他工作流、缺少事件且没有关联 PR 的门禁 run,以及不属于该线门禁层的分支事件,不在分层图或最近 run 表中展示,也不追加脚注计数。采集仍保存所有 run,不过滤 Actions 查询;同一 run 的重试只展示最后一次尝试。没有事件但有关联 PR 的门禁 run 仍归 PR。

## 使用

Expand All @@ -34,8 +32,8 @@ CI 页的「CI 分层历史」按触发方式展示该分支线窗口内的全
- `gsb/ci_lanes.py`:`lane_of` 分层,`pr_number` 取 PR 号,`duration` 取每次 run 的耗时(记录中的 `duration_s`,否则为开始到最后更新的时间),`build_lanes` 生成 `sections.ci.data.lanes`,每格带 `duration_s`,每层摘要带 `median_duration_s`。`days` 是共同日期轴,每层 `days[i]` 与之对齐,格内已按时间排序。前端 `static/app.js` 的 `ciLanes` 只负责渲染。
- PR 号优先使用 run 的 `pull_requests`。fork PR 的该字段为空,改为按 head 分支匹配当时开放的 PR,再用 head 仓库、SHA、标题排除歧义;仍不唯一时不显示号码。推断出的号码在提示中标“按分支推断”。
- 增量模式(`incremental_project.build_snapshot`)只为窗口内的 run 读取完整记录,并只读取窗口内开放过的 PR。兼容模式(`project.build_project`)使用本次读取的 run 页。
- run 与 PR 记录新增 `head_repo`,历史搜索索引新增 `event`。旧索引行缺少 `event` 时,快照从对应记录重建该行一次,不改记录、不删除历史;新字段随增量采集与每周对账补齐。既无触发事件又无关联 PR 的门禁 run 显示在“未知触发”。
- 已发布快照没有 `lanes`,或任一分支线缺少完整分层时,下一次构建从已有记录重建快照;不改变 Actions 查询,也不重新下载产物。
- run 与 PR 记录新增 `head_repo`,历史搜索索引新增 `event`。旧索引行缺少 `event` 时,快照从对应记录重建该行一次,不改记录、不删除历史;新字段随增量采集与每周对账补齐。既无触发事件又无关联 PR 的门禁 run 不画点。
- 已发布快照没有 `lanes`,或任一分支线不是上述五层顺序时,下一次构建从已有记录重建快照;不改变 Actions 查询,也不重新下载产物。

## 边界

Expand All @@ -52,4 +50,4 @@ PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tests -p 'test_ci_lane
node .e2e/node_modules/playwright/cli.js test --config test/playwright.config.cjs -g "CI trends"
```

单元测试覆盖完整分层(PR、分支 push、Nightly 定时与手动、Release tag、独立 `workflow_call`、其他工作流、未知触发)、run ID 去重、同一天多次 run 的纵向顺序、各层日期对齐、PR 号推断以及旧索引缺字段降级。浏览器用例检查七层与 30 列对齐、同日纵向顺序、提示和链接、空层、窄屏滚动与标签固定。
单元测试覆盖五层分类与隐藏类别(合并队列优先、PR、分支 push、Nightly 定时与手动、Release tag,以及 `workflow_call`、其他工作流、无事件无关联 PR 不画点)、run ID 去重、同一天多次 run 的纵向顺序、各层日期对齐、PR 号推断以及旧索引缺字段降级。浏览器用例检查五层与 30 列对齐、同日纵向顺序、合并队列提示和链接、空队列、窄屏滚动与标签固定。
2 changes: 1 addition & 1 deletion docs/dashboard-pages.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@

CI、PR、Issue 数据不可读取时显示“状态未知”,不会显示为正常。较低优先级的提示(标签化用例从未覆盖、Issue 零回复或陈旧、大量未发布提交、默认分支未保护)折叠在“其他提示”中。

**流水线**(默认分支线):PR 门禁、主干、每日构建、版本构建四张卡片,给出最近一次运行的结论、时间、工作流、耗时和近 30 天成功率。下方是近 14 天的分层图,与 CI 页同一数据,[CI 分层历史](ci-history-lanes.md)有详细说明。
**流水线**(默认分支线):PR 门禁、主干、每日构建、版本构建四张卡片,给出最近一次运行的结论、时间、工作流、耗时和近 30 天成功率。下方是近 14 天的 PR、主干、合并队列、Daily、版本五层图,与 CI 页同一数据,[CI 分层历史](ci-history-lanes.md)有详细说明。

**交付与质量**:

Expand Down
29 changes: 14 additions & 15 deletions gsb/ci_lanes.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""All collected CI runs grouped by trigger on one calendar.
"""Selected CI trigger lanes on one calendar.

Each lane follows how the source repository triggers its Actions. All lanes
share the same day axis in one fixed display zone; runs of one day stay in
Expand All @@ -14,8 +14,7 @@
WINDOW_DAYS = 30
# The team reads the board in Asia/Shanghai, which has no daylight saving.
DISPLAY_ZONE = timezone(timedelta(hours=8), "UTC+8")
LANES = (("pr", "PR"), ("main", "主干"), ("daily", "Daily"), ("release", "版本"),
("called", "工作流调用"), ("other", "其他工作流"), ("unknown", "未知触发"))
LANES = (("pr", "PR"), ("main", "主干"), ("merge_queue", "合并队列"), ("daily", "Daily"), ("release", "版本"))
# Mirrors board-config.json so a site without that section still classifies runs.
DEFAULT_RULES = {"gate": ["^CI$"], "daily": ["nightly", "daily", "每日"], "release": ["release", "version", "版本"]}
PR_EVENTS = ("pull_request", "pull_request_target")
Expand Down Expand Up @@ -69,30 +68,32 @@ def duration(run):


def lane_of(run, rules=None, default_branch="main"):
"""Return a lane key, "called" for reusable-workflow children, "unknown" or "other"."""
"""Return a displayed lane key, or None for a run outside these five lanes."""
rules = {**DEFAULT_RULES, **{kind: patterns for kind, patterns in (rules or {}).items() if patterns}}
name, event = run.get("name") or "", run.get("event")
matches = lambda kind: any(re.search(pattern, name, re.I) for pattern in rules[kind])
# Queue checks belong together even when their workflow name matches Nightly or Release.
if event == "merge_group":
return "merge_queue"
if event == "workflow_call":
return "called"
return None
# Workflow names determine the category independently of the branch line.
if matches("release"):
return "release"
if matches("daily"):
return "daily"
if not matches("gate"):
return "other"
return None
if event in PR_EVENTS:
return "pr"
if event == "workflow_dispatch":
return "main"
if event == "push":
return "main" if run.get("branch") == default_branch else "other"
if event is None:
return "main" if run.get("branch") == default_branch else None
if not event:
# Degraded record without its trigger: only a linked PR is certain.
return "pr" if run.get("pull_requests") else "unknown"
# CI itself runs on PRs, pushes and dispatch; anything else came from a caller.
return "called"
return "pr" if run.get("pull_requests") else None
return None


def pr_number(run, prs=()):
Expand Down Expand Up @@ -126,7 +127,7 @@ def build_lanes(runs, *, default_branch, now, rules=None, prs=(), days=WINDOW_DA

``default_branch`` is the branch whose pushes form the "main" lane; a branch
line passes its own branch and labels its push / manual lane with that ref.
Called and unclassified runs remain visible as distinct run IDs."""
Runs outside the displayed lanes remain in history, without a fallback row."""
first = now.astimezone(zone).date() - timedelta(days=days - 1)
axis = [first + timedelta(days=offset) for offset in range(days)]
position = {day: index for index, day in enumerate(axis)}
Expand All @@ -138,7 +139,6 @@ def build_lanes(runs, *, default_branch, now, rules=None, prs=(), days=WINDOW_DA
if previous is None or (run.get("attempt") or 1) > (previous.get("attempt") or 1):
latest[run.get("id")] = run
cells = {key: [[] for _ in axis] for key, _ in lanes}
excluded = Counter()
for run in latest.values():
created = parse(run.get("created_at"))
if not created:
Expand All @@ -151,7 +151,6 @@ def build_lanes(runs, *, default_branch, now, rules=None, prs=(), days=WINDOW_DA
name = workflow_name(run, names)
key = lane_of({**run, "name": name}, rules, default_branch)
if key not in cells:
excluded[key] += 1
continue
cells[key][day].append((created, run.get("id") or 0, {
"id": run.get("id"), "attempt": run.get("attempt") or 1, "url": run.get("url"), "workflow": name,
Expand All @@ -178,4 +177,4 @@ def build_lanes(runs, *, default_branch, now, rules=None, prs=(), days=WINDOW_DA
since = since.astimezone(zone).date() if since else None
return {"days": [day.isoformat() for day in axis], "window_days": days, "zone": zone.tzname(None),
"order": "earliest_first", "collected_since": since.isoformat() if since and since > first else None,
"branch": default_branch, "lanes": rows, "excluded": dict(excluded)}
"branch": default_branch, "lanes": rows}
6 changes: 3 additions & 3 deletions gsb/incremental_project.py
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,7 @@ def build_snapshot(sync):
for line in lines:
sections = old.get("sections", {}) if line["default"] else old.get("line_sections", {}).get(line["key"], {})
view = ((sections.get("ci") or {}).get("data") or {}).get("lanes") or {}
lanes_upgrade = lanes_upgrade or {lane["key"] for lane in view.get("lanes", [])} != {key for key, _ in LANES}
lanes_upgrade = lanes_upgrade or [lane["key"] for lane in view.get("lanes", [])] != [key for key, _ in LANES]
# Reprojecting stored run categories must not invalidate test-artifact caches.
if (not history.changed and not sync.tagged.changed and not cache_upgrade and not lanes_upgrade and not trend_upgrade and old.get("sync") == progress
and old.get("generated_at", "")[:10] == stamp(midnight)[:10]):
Expand Down Expand Up @@ -282,8 +282,8 @@ def coverage_owner(artifact):
line_index = {ident: project_run(row) for ident, row in index.items() if run_line[ident] == key}
ci = collect_ci(Context(_stored_ci(history, line_index), cfg, midnight, line_meta))
# Lanes read complete records (event, linked PRs, head repository) for the
# displayed window only, plus the PRs that were open during it. Every
# line keeps all run categories, including manually dispatched Nightly.
# displayed window only, plus the PRs that were open during it. Each
# line keeps the same five lanes, including manually dispatched Nightly.
lane_runs = [project_run(history.get("runs", history_key(row)) or index[row["id"]]) for row in line_index.values() if (row.get("created_at") or "") >= start]
ci["lanes"] = build_lanes(lane_runs, default_branch=line["ref"], now=sync.now,
rules=sync.settings.get("workflows"), prs=[pr for pr in lane_prs if pr],
Expand Down
Loading
Loading