ci(docs): check that in-repo GitHub links in docs/public resolve

[doublewhy]

Summary

docs/public/conf.py keeps every link into this repository out of the Sphinx link check (linkcheck_ignore). Its comment said those links "are covered by local policy and schema checks", but no check covered them. tools/check_public_docs.py accepts any target that contains ://, so a public page could link a blob or tree URL to a path that does not exist, and both nox -s docs-local and nox -s docs-links still passed (#1494).

This PR adds that check. tools/public_docs_links.py finds the links that use the blob or tree route on main, dev or HEAD under the prefixes linkcheck_ignore skips: https://github.com/OpenRAE/rae/ and the former owner RAESystem. It looks each path up in the files git ls-files -z lists. A blob link must name a tracked file, and a tree link must name a directory that holds one. tools/check_public_docs.py runs it with its other source checks, so it runs at the docs lane's existing "docs / public source boundary" stage. No job, session or workflow changes. All 33 such links in docs/public resolve at dev a4357f9a, so no page changes.

Requirement UIDs

• None. This is a docs tooling fix with no requirement scope: the branch carries no UID, and Public docs checks do not verify in-repo GitHub links #1494 has no Requirements section.

Related Issues

Closes #1494

ADR Impact

• No ADR required.

Changes

• tools/public_docs_links.py (new, 102 lines):
  • Scope: URLs that start with https://github.com/OpenRAE/rae/ or https://github.com/RAESystem/rae/ followed by blob/ or tree/. The check reads every line of each .md and .rst page under docs/public, code included, so it errs toward false positives, as the directive scan in tools/public_docs_markup.py does. A URL ends at whitespace or at the markup around it, and trailing sentence punctuation is dropped.
  • Refs: only main, dev and HEAD are checked. Those branches carry this tree: dev takes it when the change merges, and main (HEAD, the default branch) at the next release. Commit SHAs, tags and other branches can name content that the checkout lacks, so the check skips them.
  • Resolution: the path is percent-decoded and loses its query string, fragment and trailing slash, then looked up in git ls-files -z. Git's list matches case exactly, as GitHub does (blob/main/readme.md returns 404), and it leaves out untracked files such as the ignored /research/ corpus. A link whose route names the other kind of path fails, and the message names the route to use. GitHub serves such a link only through a 301 redirect, for example blob/main/specs to tree/main/specs. As Public docs checks do not verify in-repo GitHub links #1494's Chosen Architecture specifies, fragments are not checked against headings or line numbers.
  • Each failure renders as [public-docs-repository-link] docs/public/<page>:<line>: <url> .... The check fails closed when git cannot list the tracked files.
  • _link_problem returns None from one condition for both links without a problem: a link on a ref the check skips and a link that resolves. That keeps it at three returns, the most SonarCloud's python:S1142 allows. The refactor(docs) commit made this change after the first head failed sonar (see CI below), and it changes no behavior. The function now reads the path before it tests the ref, and that read cannot raise, because REPOSITORY_LINK only matches the blob and tree routes that ROUTE_KINDS maps.
• tools/check_public_docs.py (+5 lines, now 491): main() reads the public pages with the existing _source_paths and _bounded_text helpers and adds the link failures to the source failures it always reports.
• docs/public/conf.py: the comment above linkcheck_ignore now names the local check, and says that links to issues, pull requests and other repository pages stay unchecked, the scope Public docs checks do not verify in-repo GitHub links #1494 sets. linkcheck_ignore itself is unchanged, and test_public_docs_linkcheck_is_bounded_serialized_and_skips_own_repository still pins it.
• implementations/python/tests/test_issue_1494_public_docs_github_links.py (new):
  • Links that resolve: a bare URL before a period, Markdown, an autolink, reStructuredText, HTML, a trailing slash, the repository root, a fragment, a query string, a percent-encoded path, and the former owner.
  • Links that fail with the URL in the message: a missing file on main, a missing directory on dev, a wrong-case path on HEAD, a missing path under the former owner, a percent-encoded missing path, a URL in a code span, a URL that appears twice on a line (reported once), and both wrong-route cases.
  • Links outside the scope: a tag, a commit, another branch, another repository, an issue, an issue template, http://, another host and a relative link.
  • Integration-marked tests that run git: an untracked file on disk fails; a directory that git cannot list fails closed; check_public_docs.main() exits 1 and reports a dead link at docs/public/quickstart.md:4 with its URL, and exits 0 when the page has only a live link.
  • A default-marker test: a missing docs/public is reported once, and the link check does not run.

Test Plan

• Unit tests pass
• Integration tests pass if applicable
• Full completion suite required in CI before merge
• No coverage regression

uv run --project implementations/python --frozen --all-extras python -m pytest implementations/python/tests/test_issue_1494_public_docs_github_links.py -q passed (30 passed, 4 deselected). With -m integration, 4 passed. -m fuzz selects none of its tests. Every result in this section is from head d7e8db23, rebased onto dev a4357f9a. These checks show that the tests catch the defect and its parts:

• With tools/check_public_docs.py restored to origin/dev and the new module kept, the dead-link case of test_checker_reports_dead_links_by_page_line_and_url fails, because main() returns 0 instead of 1.
• With the is_dir() guard removed from the new main() lines, the missing-root test fails, because the link check adds a cannot list failure.
• With HEAD removed from the checked refs, or RAESystem removed from the pattern, the matching failing-link case fails.
• With the ref test dropped from _link_problem's combined condition, the tag, commit and branch cases of test_links_outside_the_checked_scope_are_ignored fail.

Under these tests, tools/public_docs_links.py has 100% line and branch coverage, and the new main() lines are covered.

test_public_docs_policy.py, test_issue_954_docs_publication_boundary.py and test_verification_plan.py passed: 90 tests by default and 32 with -m integration.

On the repository at this head, python tools/check_public_docs.py exits 0 after it scans 24 pages and finds 33 links to check. With docs/public/backends.md:19 edited to tree/main/contracts/schemas/backend-manifests (since reverted), it exits 1 and prints:

[public-docs-repository-link] docs/public/backends.md:19: https://github.com/OpenRAE/rae/tree/main/contracts/schemas/backend-manifests does not resolve: this checkout tracks no directory at contracts/schemas/backend-manifests

Nine other open PRs touch docs/public: #1431, #1436, #1437, #1444, #1449, #1463, #1465, #1470 and #1492. The check finds no failing blob or tree link in any of their heads, or in each head merged with this branch (git merge-tree --write-tree), so it breaks none of them today.

nox -s verify-fast-feedback -- --base-rev origin/dev (every stage that ran passed; the YAML and JSON syntax stages had no files to check, and requirement governance was skipped), nox -s lint, make policy and nox -s docs-local passed. In docs-local, every stage passed: public source boundary, Vale, executable quickstart, Sphinx HTML and public output inventory. tools/check_specification_coverage.py and tools/check_formal_semantic_validation.py passed. None of the changed files is in the research-evidence implementation digest, among the 11 tools/ files pinned by specs/formal/participant-semantics/participant-opacity-proof-evidence.json, or under tools/specification_coverage/.

Ground Control Checks

• Repository policy command passes
• Regression tests fail on the pre-fix checker and pass after

make policy passed. Requirement governance is skipped because the branch carries no requirement UID and no scope file. The dead-link main() case fails with tools/check_public_docs.py at origin/dev, as described in the Test Plan.

Traceability

• IMPLEMENTS: tools/public_docs_links.py, tools/check_public_docs.py, docs/public/conf.py
• TESTS: implementations/python/tests/test_issue_1494_public_docs_github_links.py
• Claim corrected: the comment above linkcheck_ignore in docs/public/conf.py (lines 56-59 at dev a4357f9a)

Checklist

• FM: not applicable. There is no semantic change (docs tooling and tests), and no executable or runtime adoption is claimed.
• No gate weakened: linkcheck_ignore and the docs lane's stages are unchanged, and the check only adds failures.
• The PR title is a Conventional Commit. ci does not cut a release.
• tools/check_public_docs.py stays under SonarCloud's 500-line file limit (python:S104) at 491 lines.
• SonarCloud reports no new issues: _link_problem has three returns (python:S1142), and the sonar job of CI run 38184864922 passed.

Documentation

• Updated: the link-check comment in docs/public/conf.py.
• Verified unchanged: every page under docs/public, because all 33 blob and tree links resolve.

CI

• Head d7e8db23 (rebased onto dev a4357f9a): all 32 reported checks pass, including the 14 required ones. Docs / deploy is skipped because it runs only on main. Runs: CI 38184864922, whose sonar job passed with the SonarCloud gate at OK and no new issues; Docs 38184864673; Bootstrap qualification 38184864683; CodeQL 38184862787; PR Body Policy 38184864681; PR Title Lint 38184864663.
• The previous head e8014138 failed only sonar (run 38164587590), on one new issue: python:S1142 at tools/public_docs_links.py:91, where _link_problem had four returns and three are allowed. The refactor(docs) commit fixes the code itself, with no suppression, as Changes describes.