docs: switch to preprocessor pattern for interface docs - #637
Open
james-garner-canonical wants to merge 1 commit into
Conversation
dwilding
approved these changes
Aug 26, 2026
dwilding
left a comment
dwilding
left a comment
Contributor
There was a problem hiding this comment.
Seems a reasonable approach to me. Thanks!
| # llms.txt is only generated in the final combined pass (the `html` recipe). | ||
| # Leaving it enabled here triggers a nested base-mode build that regenerates the | ||
| # interface reference docs, deleting the placeholder this package pass depends on. | ||
| # Leaving it enabled here triggers a nested base-mode build that the |
Contributor
There was a problem hiding this comment.
Very minor suggestion. To reinforce that the comment is talking about what we're not doing.
Suggested change
| # Leaving it enabled here triggers a nested base-mode build that the | |
| # Enabling it here would trigger a nested base-mode build that the |
Comment on lines
+30
to
+38
| This is a standalone preprocessor script rather than a Sphinx extension, | ||
| following the same pattern as ``diataxis_preprocessor.py``. Unlike the | ||
| package reference docs, interface docs don't use autodoc, so nothing here | ||
| needs to run inside a Sphinx build. Generating the pages once up front — | ||
| instead of in a ``builder-inited`` hook on every Sphinx pass — keeps the | ||
| per-package intermediate passes cheaper and the extension machinery simpler. | ||
| The companion ``interface_docs`` extension is now only a fallback: it writes | ||
| a placeholder page when this script hasn't run, so the glob toctree in | ||
| ``reference/interfaces.md`` still matches at least one document. |
Contributor
There was a problem hiding this comment.
I might cut this down bit. How about:
Suggested change
| This is a standalone preprocessor script rather than a Sphinx extension, | |
| following the same pattern as ``diataxis_preprocessor.py``. Unlike the | |
| package reference docs, interface docs don't use autodoc, so nothing here | |
| needs to run inside a Sphinx build. Generating the pages once up front — | |
| instead of in a ``builder-inited`` hook on every Sphinx pass — keeps the | |
| per-package intermediate passes cheaper and the extension machinery simpler. | |
| The companion ``interface_docs`` extension is now only a fallback: it writes | |
| a placeholder page when this script hasn't run, so the glob toctree in | |
| ``reference/interfaces.md`` still matches at least one document. | |
| This is a standalone preprocessor script and nothing here needs to be run | |
| inside a Sphinx build. Generating the pages once up front — instead of in a | |
| ``builder-inited`` hook on every Sphinx pass — keeps the per-package | |
| intermediate passes cheap and the extension machinery simple. The companion | |
| ``interface_docs`` extension is a fallback: it writes a placeholder page when | |
| this script hasn't run, so the glob toctree in ``reference/interfaces.md`` | |
| still matches at least one document. |
| """Rewrite relative markdown links to absolute GitHub URLs under ``base_url``.""" | ||
| return re.sub( | ||
| # match all non-http(s) markdown links and prepend base_url to matching links | ||
| r'\[(.+)\]\((?!https?://)([^)]+)\)', |
Contributor
There was a problem hiding this comment.
The (.+) will be too greedy if there's more than one link on a line. My agent says we should change this to (.+?), similarly to _rewrite_links in diataxis_preprocessor.py. I guess we should also add a test for multiple links on a line.
| import sphinx.application | ||
|
|
||
|
|
||
| def _app(confdir: pathlib.Path) -> sphinx.application.Sphinx: |
Contributor
There was a problem hiding this comment.
Matching _fake_ls?
Suggested change
| def _app(confdir: pathlib.Path) -> sphinx.application.Sphinx: | |
| def _fake_app(confdir: pathlib.Path) -> sphinx.application.Sphinx: |
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| # ruff: noqa: D103 (function docstrings) |
Contributor
There was a problem hiding this comment.
I think we don't need this in this particular test file. But perhaps you'd prefer to keep it for consistency across test files?
| index = _INDEX_TEMPLATE.format(label=label, interface_name=interface_name) | ||
| _write_if_needed(path=ref_dir / f'{interface_name}.md', content=index) | ||
| for v in (interface_dir / 'interface').glob('v[0-9]*'): | ||
| readme_raw = (v / 'README.md').read_text() |
Contributor
There was a problem hiding this comment.
If the interface version doesn't have a README.md, should we fail earlier with a helpful message?
james-garner-canonical
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR switches the docs build process to build the interface docs as a pre-processor pass rather than as a Sphinx extension. The reason it wasn't a pre-processor originally is purely historical -- we started with the package dos extension, added interface docs as an extension, and then added the pre-processor approach for the library diataxis docs.
We retain a thin extension so that building the docs without the pre-processor works (it writes a placeholder file so the TOC glob doesn't fail).
The positive delta in this PR is adding tests for the pre-processor and the thin extension (the previous fat extension didn't have dedicated tests).
Context: I'm trying to speed up the docs build. This doesn't really help, but it does simplify it, so it's a good first step. I have a couple more things to try that will follow up from here.