From dc62ce03c9a4477a95ad5be1a7d7c484d08061bf Mon Sep 17 00:00:00 2001 From: Matt McKay Date: Mon, 3 Aug 2026 16:39:56 +1000 Subject: [PATCH 1/2] Comment only when a link we publish has actually rotted The action was posting a link-check comment on every PR to this repo, reporting this-host-cannot-resolve.invalid from broken-links.html. That host is a fixture built to be unresolvable, so the comment said the same thing on every run and carried no information -- the same signal-to-noise problem #2 was about, pointed at our own PRs. The fixture scan now runs with fail-on-broken false, which stops the comment without losing the real-network smoke coverage. The comment path moves somewhere it can mean something. A new check-own-docs job renders this repository's Markdown to HTML and checks the links in it. A finding there is a link we actually publish that has rotted, and the comment names it. When the docs are clean there are no findings, broken-links-found stays false, and no comment is posted at all. Rendering does the filtering for free: the placeholder URLs in the usage examples live inside fenced code blocks and never become anchors, so 4 genuine links are checked rather than the 15 a grep over the raw Markdown would find. It found a real bug on its first run. The README's Marketplace badge linked to github.com/marketplace/actions/ai-link-checker, which returns 404 -- no release was ever published there, as the 1.1.0 notes already recorded. Badge removed. With it gone the job reports 0 broken and 0 redirects, so it posts nothing; adding a rotted link back makes it report 1 broken and post again. Both directions verified locally. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/ci.yml | 70 ++++++++++++++++++++++++++++++++++++---- CHANGELOG.md | 7 ++++ README.md | 1 - tests/README.md | 8 +++++ 4 files changed, 78 insertions(+), 8 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index ad5b429..15cf95f 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -75,17 +75,18 @@ jobs: fail-on-broken: 'true' # Informational only, and expected to report findings: the fixture holds - # a deliberately unreachable host and three deliberate redirects. It runs - # with fail-on-broken so that the PR-comment path is exercised, and with - # continue-on-error so that neither those findings nor an outage at one of - # the redirect targets can gate CI. Do not read its result as an - # assertion -- the assertions are the deterministic steps below. + # a deliberately unreachable host and three deliberate redirects. + # + # fail-on-broken is false so that no PR comment is posted. Its findings + # are true by construction, so a comment about them says the same thing + # on every run and therefore says nothing. The check-own-docs job below + # exercises the comment path against links that can actually rot. - name: Live smoke scan (informational, does not gate) continue-on-error: true uses: ./ with: html-path: 'tests/broken-links.html' - fail-on-broken: 'true' + fail-on-broken: 'false' timeout: 30 # A real 200 over real HTTP was the one thing only a public site could @@ -327,4 +328,59 @@ jobs: - name: Test action on empty directory uses: ./ with: - html-path: 'empty-html' \ No newline at end of file + html-path: 'empty-html' + + # Dogfooding, and the only place a PR comment is worth posting. The links + # here are real ones in our own documentation, so a finding means something + # actually rotted and the comment names it. When the docs are clean there + # are no findings, broken-links-found stays false and no comment is posted + # at all -- which is the point. A comment that appears on every run carries + # no information. + # + # This action reads HTML, so the Markdown is rendered first. Rendering also + # does the filtering for free: the placeholder URLs in the usage examples + # live inside fenced code blocks and never become anchors, so only genuine + # links are checked. + check-own-docs: + runs-on: ubuntu-latest + name: Check links in our own docs + + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Render the Markdown docs to HTML + run: | + python3 -m pip install --quiet markdown + mkdir -p docs-html + python3 - <<'PY' + import markdown, pathlib + + out = pathlib.Path('docs-html') + sources = sorted(pathlib.Path('.').glob('*.md')) + [pathlib.Path('tests/README.md')] + for src in sources: + html = markdown.markdown(src.read_text(encoding='utf-8'), + extensions=['fenced_code', 'tables']) + dest = out / (src.as_posix().replace('/', '__') + '.html') + dest.write_text('' + html + '', encoding='utf-8') + print('rendered', src) + PY + + # continue-on-error because an outage at one of the linked sites is not a + # reason to fail this repository's build. The PR comment is the signal. + - name: Check the links in the rendered docs + id: docs + continue-on-error: true + uses: ./ + with: + html-path: 'docs-html' + fail-on-broken: 'true' + timeout: 20 + + - name: Report + run: | + echo "broken: ${{ steps.docs.outputs.broken-link-count }}" + echo "redirects: ${{ steps.docs.outputs.redirect-count }}" + if [ "${{ steps.docs.outputs.broken-links-found }}" = "true" ]; then + echo "::warning::Our own documentation has link findings -- see the PR comment" + fi \ No newline at end of file diff --git a/CHANGELOG.md b/CHANGELOG.md index 42872f3..90574d4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,8 +7,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added +- A `check-own-docs` CI job that renders this repository's Markdown and checks the links in it. It is the only job that posts a link-check comment on a pull request, and it posts one only when a link we publish has actually rotted + ### Changed - CI serves its own fixtures from `127.0.0.1`, so no gating step depends on the public internet any more. A clean page, a followed redirect, a reported error status and that status silenced by `silent-codes` are all asserted end to end against responses under our control. The scans of `good-links.html` and `broken-links.html` remain, as informational smoke that cannot turn CI red +- The fixture scans no longer post a link-check comment on pull requests. Their findings are true by construction, so the comment reported the same fixture on every run and carried no information + +### Fixed +- The README linked a GitHub Marketplace badge to `https://github.com/marketplace/actions/ai-link-checker`, which returns 404 — the action was never published there. Found by the new `check-own-docs` job on its first run ## [1.1.0] - 2026-08-03 diff --git a/README.md b/README.md index 72bde66..1a446c3 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,6 @@ # AI-Powered Link Checker Action [![CI](https://github.com/QuantEcon/action-link-checker/actions/workflows/ci.yml/badge.svg)](https://github.com/QuantEcon/action-link-checker/actions/workflows/ci.yml) -[![GitHub Marketplace](https://img.shields.io/badge/Marketplace-AI%20Link%20Checker-blue.svg?colorA=24292e&colorB=0366d6&style=flat&longCache=true&logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAA4AAAAOCAYAAAAfSC3RAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAM6wAADOsB5dZE0gAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAAAERSURBVCiRhZG/SsMxFEafKoEMFhyrdsFt6FYHNycunTo0Q4LgEhcnW4PgYgchGjoYiQ6ON2ARpK9nCxjUjuIFP+B+h3O/xmE2yVxPOJGkC3RgJ8qA3bQn7SiTKCQdC4J8HDW0v85CZaUHNzxhQcHdJvjZwM4mXaKJ4BdDMKxIsYoim1Smk2X6HPUdCnU5gO5D9POqvayBzY8nwoJJ+G9h9vGB0U8h8dNPgGLKlv1n6cJgAjjfY9lv1CVKq5f3oUAe5dJz9n3RkBhGA1ouJ/hT5a4c8yQQYSdF8vhN5gT1igMgZ9nJgzUqm9E1V+8rbYQhptmEURKA=)](https://github.com/marketplace/actions/ai-link-checker) A sophisticated GitHub Action that validates web links in HTML files with AI-powered suggestions for improvements. Goes beyond traditional link checkers by providing intelligent recommendations and handling modern web challenges. diff --git a/tests/README.md b/tests/README.md index 53ac1ae..ad62583 100644 --- a/tests/README.md +++ b/tests/README.md @@ -60,6 +60,14 @@ The HTML fixtures are scanned by the `test-action` job, and both Python test fil The `ignore-patterns` assertions in CI use a fixture generated by the workflow rather than one kept here, so that the expected counts stay exact as fixtures are added to this directory. +## The one place a PR comment is posted + +None of the fixtures here post a link-check comment on a pull request, and that is deliberate. Their findings are true by construction — `broken-links.html` contains a host that can never resolve, so a comment about it says the same thing on every run and therefore says nothing. + +The `check-own-docs` job is the exception. It renders this repository's own Markdown to HTML and checks the links in it, so a finding means a link we actually publish has rotted, and the comment names it. When the docs are clean there are no findings, `broken-links-found` stays `false`, and no comment is posted at all. + +Rendering does the filtering for free: the placeholder URLs in the usage examples sit inside fenced code blocks and never become anchors, so only genuine links are checked. It found its first real bug immediately — a README badge pointing at a GitHub Marketplace listing that was never published. + ## Running Tests Locally `action.yml` is a composite action, not an executable — run the checker directly instead. It takes one HTML file and prints a JSON report: From e2ae392ed63dda7090b6ca5ae22f93d3d7a6d47b Mon Sep 17 00:00:00 2001 From: Matt McKay Date: Mon, 3 Aug 2026 19:30:45 +1000 Subject: [PATCH 2/2] Silence status 0 in the docs check, and stop naming rot as the cause Copilot's suppressed comment was the substantive one of its four, and it holds. keepachangelog.com and semver.org are not on the built-in legitimate_domains allowance, so a transient timeout at either reported status 0 as broken and posted a comment about a link that is fine -- the same false positive this action shipped silent-codes '0' to prevent, unapplied to our own repo. The docs job now runs with silent-codes '0,403,503'. Verified both directions: a real 404 still posts a comment, so the class of finding that turned up the dead Marketplace badge is untouched, and a timeout at either bare domain is now quiet. The cost, recorded in both the workflow comment and tests/README.md, is that a domain which disappears outright is not reported here. Its three visible comments were all the same wording point, and also correct: a finding says a link came back broken, not that it rotted. The checker cannot distinguish the cause, so the four places that claimed it could now say what is actually observed. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/ci.yml | 21 +++++++++++++++------ CHANGELOG.md | 2 +- tests/README.md | 4 +++- 3 files changed, 19 insertions(+), 8 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 15cf95f..2cb769f 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -331,11 +331,11 @@ jobs: html-path: 'empty-html' # Dogfooding, and the only place a PR comment is worth posting. The links - # here are real ones in our own documentation, so a finding means something - # actually rotted and the comment names it. When the docs are clean there - # are no findings, broken-links-found stays false and no comment is posted - # at all -- which is the point. A comment that appears on every run carries - # no information. + # here are real ones in our own documentation, so a finding names a link we + # publish that came back broken. When the docs are clean there are no + # findings, broken-links-found stays false and no comment is posted at all + # -- which is the point. A comment that appears on every run carries no + # information. # # This action reads HTML, so the Markdown is rendered first. Rendering also # does the filtering for free: the placeholder URLs in the usage examples @@ -368,6 +368,14 @@ jobs: # continue-on-error because an outage at one of the linked sites is not a # reason to fail this repository's build. The PR comment is the signal. + # + # Status 0 is silenced. From a runner a transport failure cannot be told + # apart from a host that is simply down or throttling this IP range, and + # neither keepachangelog.com nor semver.org is on the legitimate_domains + # allowance, so a blip at either would otherwise post a comment about a + # link that is fine. A real 404 -- the class that found the dead + # Marketplace badge -- is unaffected. The cost is that a domain which + # disappears outright goes unreported here. - name: Check the links in the rendered docs id: docs continue-on-error: true @@ -375,6 +383,7 @@ jobs: with: html-path: 'docs-html' fail-on-broken: 'true' + silent-codes: '0,403,503' timeout: 20 - name: Report @@ -382,5 +391,5 @@ jobs: echo "broken: ${{ steps.docs.outputs.broken-link-count }}" echo "redirects: ${{ steps.docs.outputs.redirect-count }}" if [ "${{ steps.docs.outputs.broken-links-found }}" = "true" ]; then - echo "::warning::Our own documentation has link findings -- see the PR comment" + echo "::warning::A link in our own documentation came back broken -- see the PR comment" fi \ No newline at end of file diff --git a/CHANGELOG.md b/CHANGELOG.md index 90574d4..de4cbaf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,7 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Added -- A `check-own-docs` CI job that renders this repository's Markdown and checks the links in it. It is the only job that posts a link-check comment on a pull request, and it posts one only when a link we publish has actually rotted +- A `check-own-docs` CI job that renders this repository's Markdown and checks the links in it. It is the only job that posts a link-check comment on a pull request, and it posts one only when a link we publish is reported broken. It runs with `silent-codes: '0,403,503'`, so a transient timeout or DNS failure stays quiet and the comment is reserved for a real status such as a 404 ### Changed - CI serves its own fixtures from `127.0.0.1`, so no gating step depends on the public internet any more. A clean page, a followed redirect, a reported error status and that status silenced by `silent-codes` are all asserted end to end against responses under our control. The scans of `good-links.html` and `broken-links.html` remain, as informational smoke that cannot turn CI red diff --git a/tests/README.md b/tests/README.md index ad62583..af2cf26 100644 --- a/tests/README.md +++ b/tests/README.md @@ -64,7 +64,9 @@ The `ignore-patterns` assertions in CI use a fixture generated by the workflow r None of the fixtures here post a link-check comment on a pull request, and that is deliberate. Their findings are true by construction — `broken-links.html` contains a host that can never resolve, so a comment about it says the same thing on every run and therefore says nothing. -The `check-own-docs` job is the exception. It renders this repository's own Markdown to HTML and checks the links in it, so a finding means a link we actually publish has rotted, and the comment names it. When the docs are clean there are no findings, `broken-links-found` stays `false`, and no comment is posted at all. +The `check-own-docs` job is the exception. It renders this repository's own Markdown to HTML and checks the links in it, so a finding names a link we publish that came back broken. When the docs are clean there are no findings, `broken-links-found` stays `false`, and no comment is posted at all. + +It runs with `silent-codes: '0,403,503'`. From a runner a status 0 cannot be told apart from a host that is momentarily down or throttling the IP range, and neither `keepachangelog.com` nor `semver.org` is on the built-in `legitimate_domains` allowance, so without that setting a blip at either would post a comment about a link that is fine. A real status such as a 404 is unaffected. The trade is that a domain which disappears outright is not reported here. Rendering does the filtering for free: the placeholder URLs in the usage examples sit inside fenced code blocks and never become anchors, so only genuine links are checked. It found its first real bug immediately — a README badge pointing at a GitHub Marketplace listing that was never published.