Skip to content

Commit f6e0883

Browse files
authored
Merge branch 'main' into feature/failure-attributes
2 parents 034ab33 + 37f4ca2 commit f6e0883

44 files changed

Lines changed: 2883 additions & 284 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,3 +26,4 @@ __pycache__/
2626

2727
# bug: This file is created in repo root on test discovery.
2828
/consumer_test.log
29+
.clwb

.pre-commit-config.yaml

Lines changed: 17 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -35,22 +35,26 @@ repos:
3535
language: system
3636
pass_filenames: false
3737
files: '(^|/)(MODULE\.bazel|.*\.bzl|BUILD(\.bazel)?)$'
38-
39-
- repo: https://github.com/rhysd/actionlint
40-
rev: v1.7.11
41-
hooks:
4238
- id: actionlint
43-
args:
44-
# Disable shellcheck and pyflakes for now, to enforce consistent behavior.
45-
- -shellcheck=
46-
- -pyflakes=
47-
48-
- repo: https://github.com/astral-sh/ruff-pre-commit
49-
rev: v0.15.9
50-
hooks:
39+
name: actionlint
40+
# Disable shellcheck and pyflakes for now, to enforce consistent behavior.
41+
entry: tools/run_tool.sh actionlint -shellcheck= -pyflakes=
42+
language: system
43+
require_serial: true
44+
types: [yaml]
45+
files: ^\.github/workflows/
5146
- id: ruff-check
52-
args: [ --fix ]
47+
name: ruff-check
48+
entry: tools/run_tool.sh ruff check --fix
49+
language: system
50+
require_serial: true
51+
types: [python]
5352
- id: ruff-format
53+
name: ruff-format
54+
entry: tools/run_tool.sh ruff format
55+
language: system
56+
require_serial: true
57+
types: [python]
5458

5559
# Note: this is super slow, therefore it is last
5660
- repo: local

BUILD

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,3 +26,15 @@ docs(
2626
],
2727
source_dir = "docs",
2828
)
29+
30+
# bazel run //:shellcheck
31+
alias(
32+
name = "shellcheck",
33+
actual = "@score_devcontainer//tools:shellcheck",
34+
)
35+
36+
# bazel run //:actionlint
37+
alias(
38+
name = "actionlint",
39+
actual = "@score_devcontainer//tools:actionlint",
40+
)

MODULE.bazel

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,3 +70,6 @@ http_file(
7070
)
7171

7272
bazel_dep(name = "score_process", version = "1.5.4")
73+
74+
# Provide the tools from the devcontainer to Bazel
75+
bazel_dep(name = "score_devcontainer", version = "1.5.0")

MODULE.bazel.lock

Lines changed: 15 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

docs.bzl

Lines changed: 26 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -125,7 +125,7 @@ def _missing_requirements(deps):
125125
fail(msg)
126126
fail("This case should be unreachable?!")
127127

128-
def docs(source_dir = "docs", data = [], deps = [], scan_code = [], known_good = None):
128+
def docs(source_dir = "docs", data = [], deps = [], scan_code = [], known_good = None, metamodel = None):
129129
"""Creates all targets related to documentation.
130130
131131
By using this function, you'll get any and all updates for documentation targets in one place.
@@ -135,13 +135,24 @@ def docs(source_dir = "docs", data = [], deps = [], scan_code = [], known_good =
135135
data: Additional data files to include in the documentation build.
136136
deps: Additional dependencies for the documentation build.
137137
scan_code: List of code targets to scan for source code links.
138+
known_good: Optional label to a "known good" JSON file for source links.
139+
metamodel: Optional label to a metamodel.yaml file. When set, the extension loads this
140+
file instead of the default metamodel shipped with score_metamodel.
138141
"""
139142

140143
call_path = native.package_name()
141144

142145
if call_path != "":
143146
fail("docs() must be called from the root package. Current package: " + call_path)
144147

148+
metamodel_data = []
149+
metamodel_env = {}
150+
metamodel_opts = []
151+
if metamodel != None:
152+
metamodel_data = [metamodel]
153+
metamodel_env = {"SCORE_METAMODEL_YAML": "$(location " + str(metamodel) + ")"}
154+
metamodel_opts = ["--define=score_metamodel_yaml=$(location " + str(metamodel) + ")"]
155+
145156
module_deps = deps
146157
deps = deps + _missing_requirements(deps)
147158
deps = deps + [
@@ -152,7 +163,7 @@ def docs(source_dir = "docs", data = [], deps = [], scan_code = [], known_good =
152163
sphinx_build_binary(
153164
name = "sphinx_build",
154165
visibility = ["//visibility:private"],
155-
data = data,
166+
data = data + metamodel_data,
156167
deps = deps,
157168
)
158169

@@ -187,19 +198,19 @@ def docs(source_dir = "docs", data = [], deps = [], scan_code = [], known_good =
187198
data_with_docs_sources = _rewrite_needs_json_to_docs_sources(data)
188199
additional_combo_sourcelinks = _rewrite_needs_json_to_sourcelinks(data)
189200
_merge_sourcelinks(name = "merged_sourcelinks", sourcelinks = [":sourcelinks_json"] + additional_combo_sourcelinks, known_good = known_good)
190-
docs_data = data + [":sourcelinks_json"]
191-
combo_data = data_with_docs_sources + [":merged_sourcelinks"]
201+
docs_data = data + metamodel_data + [":sourcelinks_json"]
202+
combo_data = data_with_docs_sources + metamodel_data + [":merged_sourcelinks"]
192203

193204
docs_env = {
194205
"SOURCE_DIRECTORY": source_dir,
195206
"DATA": str(data),
196207
"SCORE_SOURCELINKS": "$(location :sourcelinks_json)",
197-
}
208+
} | metamodel_env
198209
docs_sources_env = {
199210
"SOURCE_DIRECTORY": source_dir,
200211
"DATA": str(data_with_docs_sources),
201212
"SCORE_SOURCELINKS": "$(location :merged_sourcelinks)",
202-
}
213+
} | metamodel_env
203214
if known_good:
204215
docs_env["KNOWN_GOOD_JSON"] = "$(location "+ known_good + ")"
205216
docs_sources_env["KNOWN_GOOD_JSON"] = "$(location "+ known_good + ")"
@@ -293,16 +304,24 @@ def docs(source_dir = "docs", data = [], deps = [], scan_code = [], known_good =
293304
"--jobs",
294305
"auto",
295306
"--define=external_needs_source=" + str(data),
307+
"--define=score_sourcelinks_json=$(location :sourcelinks_json)",
308+
"--define=score_source_code_linker_plain_links=1",
296309
],
297310
formats = ["needs"],
298311
sphinx = ":sphinx_build",
299-
tools = data,
312+
tools = data + [":sourcelinks_json"],
300313
visibility = ["//visibility:public"],
301314
# Persistent workers cause stale symlinks after dependency version
302315
# changes, corrupting the Bazel cache.
303316
allow_persistent_workers = False,
304317
)
305318

319+
native.alias(
320+
name = "traceability_gate",
321+
actual = "@score_docs_as_code//scripts_bazel:traceability_gate",
322+
tags = ["cli_help=Enforce traceability coverage thresholds:\nbazel run //:traceability_gate -- --metrics-json bazel-bin/needs_json/_build/needs/metrics.json"],
323+
)
324+
306325
def _sourcelinks_json(name, srcs):
307326
"""
308327
Creates a target that generates a JSON file with source code links.
Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
..
2+
# *******************************************************************************
3+
# Copyright (c) 2026 Contributors to the Eclipse Foundation
4+
#
5+
# See the NOTICE file(s) distributed with this work for additional
6+
# information regarding copyright ownership.
7+
#
8+
# This program and the accompanying materials are made available under the
9+
# terms of the Apache License Version 2.0 which is available at
10+
# https://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# SPDX-License-Identifier: Apache-2.0
13+
# *******************************************************************************
14+
15+
Build Dashboards and Quality Gates
16+
==================================
17+
18+
Use this guide in repositories that consume docs-as-code as a Bazel
19+
dependency.
20+
21+
Goals:
22+
23+
1. Publish traceability dashboards from repository needs.
24+
2. Export machine-readable metrics.
25+
3. Enforce CI thresholds with ``traceability_gate``.
26+
27+
What You Get
28+
------------
29+
30+
With the ``docs(...)`` macro and ``score_metamodel`` extension enabled, your
31+
repository can:
32+
33+
- build an HTML dashboard from its own Sphinx needs,
34+
- include external needs from other repositories when desired,
35+
- export ``needs.json`` and ``metrics.json`` for machine-readable reporting,
36+
- gate CI on traceability thresholds via ``traceability_gate``.
37+
38+
Typical Setup
39+
-------------
40+
41+
For details, see :ref:`setup`.
42+
43+
Minimal Configuration Example
44+
-----------------------------
45+
46+
In ``docs/conf.py``:
47+
48+
.. code-block:: python
49+
50+
score_metamodel_requirement_types = "feat_req,comp_req,aou_req"
51+
score_metamodel_include_external_needs = False
52+
53+
Use ``score_metamodel_include_external_needs = True`` only in repositories that
54+
intentionally aggregate requirements across module dependencies, such as
55+
integration repositories. Use ``False`` for module repositories to gate only on
56+
local traceability.
57+
58+
Building the Dashboard
59+
----------------------
60+
61+
After building/running any docs command (i.e. ``bazel build //:needs_json`` or ``bazel run //:docs_check`` are the fastest):
62+
63+
The documentation build writes ``metrics.json`` via ``score_metamodel``, and the ``needs_json`` artifact contains:
64+
65+
- ``bazel-bin/needs_json/_build/needs/needs.json``
66+
- ``bazel-bin/needs_json/_build/needs/metrics.json``
67+
68+
The dashboard charts and the CI gate both use the same computed metrics.
69+
70+
Inputs for Linkage Metrics
71+
--------------------------
72+
73+
To get meaningful dashboard and gate values, consumer repositories typically
74+
need three inputs:
75+
76+
1. Requirement and architecture needs in the documentation itself.
77+
2. Source code references via :doc:`source_to_doc_links`.
78+
3. Test metadata via :doc:`test_to_doc_links`.
79+
80+
If one of those inputs is missing, the related chart or gate metric will remain
81+
empty or low.
82+
83+
Choosing Local vs Aggregated Views
84+
----------------------------------
85+
86+
There are two common modes:
87+
88+
**Module repository**
89+
90+
- Set ``score_metamodel_include_external_needs = False``.
91+
- Gate only on the needs owned by the repository itself.
92+
- Use this for per-module implementation progress and traceability.
93+
94+
**Integration repository**
95+
96+
- Set ``score_metamodel_include_external_needs = True``.
97+
- Aggregate requirements across module dependencies when that is the intended
98+
repository purpose.
99+
- Use this for system or integration-level dashboards.
100+
101+
CI Quality Gate
102+
---------------
103+
104+
Any docs build (``bazel run //:docs``, ``bazel run //:docs_check``, etc.)
105+
writes ``metrics.json`` alongside the build output. Run the gate on the
106+
exported metrics:
107+
108+
.. code-block:: bash
109+
110+
bazel run //:docs && \
111+
bazel run //:traceability_gate -- \
112+
--metrics-json bazel-bin/needs_json/_build/needs/metrics.json \
113+
--min-req-code 70 \
114+
--min-req-test 70 \
115+
--min-req-fully-linked 60 \
116+
--min-tests-linked 70
117+
118+
In CI, wire targets through Bazel dependencies so test execution and
119+
docs generation happen before the gate target.
120+
121+
In larger repositories, define a dedicated wrapper target for your standard
122+
gate thresholds so CI calls a single Bazel target.
123+
124+
Useful flags:
125+
126+
- ``--require-all-links`` for strict 100 percent gating
127+
128+
Recommended Rollout
129+
-------------------
130+
131+
For a new consumer repository:
132+
133+
1. Start with local-only metrics.
134+
2. Enable ``scan_code`` and verify ``source_code_link`` coverage first.
135+
3. Add test metadata and verify ``testlink`` coverage.
136+
4. Introduce modest thresholds in CI.
137+
5. Raise thresholds over time as the repository matures.
138+
139+
Related Guides
140+
--------------
141+
142+
- :ref:`setup`
143+
- :doc:`other_modules`
144+
- :doc:`source_to_doc_links`
145+
- :doc:`test_to_doc_links`

0 commit comments

Comments
 (0)