|
12 | 12 | # SPDX-License-Identifier: Apache-2.0 |
13 | 13 | # ******************************************************************************* |
14 | 14 |
|
| 15 | + # ╓ ╖ |
| 16 | + # ║ Some portions generated by Github Copilot ║ |
| 17 | + # ╙ ╜ |
| 18 | + |
15 | 19 | Reference Docs in Tests |
16 | 20 | ======================= |
17 | 21 |
|
@@ -58,38 +62,56 @@ Limitations |
58 | 62 | CI/CD Gate for Linkage Percentage |
59 | 63 | --------------------------------- |
60 | 64 |
|
61 | | -To enforce traceability in CI: |
| 65 | +The traceability checker can be used as a low-level CI gate over exported |
| 66 | +``needs.json`` data. |
| 67 | + |
| 68 | +Current workflow: |
62 | 69 |
|
63 | 70 | 1. Run tests. |
64 | 71 | 2. Generate ``needs.json``. |
65 | 72 | 3. Execute the traceability checker. |
66 | 73 |
|
| 74 | +In repository CI, the preferred setup is to wire the coverage check target |
| 75 | +to depend on the test-report and ``//:needs_json`` targets, so Bazel handles |
| 76 | +the build order automatically. |
| 77 | + |
| 78 | +You can run the checker as a standalone command, and you can also run it as |
| 79 | +part of documentation creation if your repository wiring does so. |
| 80 | + |
67 | 81 | .. code-block:: bash |
68 | 82 |
|
69 | 83 | bazel test //... |
70 | 84 | bazel build //:needs_json |
71 | 85 | bazel run //scripts_bazel:traceability_coverage -- \ |
72 | | - --needs-json bazel-bin/needs_json/_build/needs/needs.json \ |
73 | 86 | --min-req-code 100 \ |
74 | 87 | --min-req-test 100 \ |
75 | 88 | --min-req-fully-linked 100 \ |
76 | 89 | --min-tests-linked 100 \ |
77 | 90 | --fail-on-broken-test-refs |
78 | 91 |
|
| 92 | +If ``//:needs_json`` was built beforehand, the checker locates the default |
| 93 | +``needs.json`` output automatically. Use ``--needs-json`` only when you want |
| 94 | +to point to a non-standard location. |
| 95 | + |
79 | 96 | The checker reports: |
80 | 97 |
|
81 | 98 | - Percentage of implemented requirements with ``source_code_link`` |
82 | 99 | - Percentage of implemented requirements with ``testlink`` |
83 | 100 | - Percentage of implemented requirements with both links (fully linked) |
84 | | -- Percentage of test cases linked to at least one requirement |
85 | | -- Broken testcase references to unknown requirement IDs |
| 101 | + |
| 102 | +.. note:: |
| 103 | + |
| 104 | + Testcase-based metrics depend on testcase needs being present in the |
| 105 | + exported ``needs.json``. Testcases are currently generated as external |
| 106 | + needs, so values such as testcase linkage percentage or broken testcase |
| 107 | + references are only meaningful if those external testcase needs are also |
| 108 | + included in the exported dataset. |
86 | 109 |
|
87 | 110 | To check only unit tests, filter testcase types: |
88 | 111 |
|
89 | 112 | .. code-block:: bash |
90 | 113 |
|
91 | 114 | bazel run //scripts_bazel:traceability_coverage -- \ |
92 | | - --needs-json bazel-bin/needs_json/_build/needs/needs.json \ |
93 | 115 | --test-types unit-test |
94 | 116 |
|
95 | 117 | Use lower thresholds during rollout and tighten towards 100% over time. |
0 commit comments