Build the docs with great-docs - #760
Conversation
|
Important Review skippedToo many files! This PR contains 112 files, which is 12 over the limit of 100. To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch. Upgrade to a paid plan to raise the limit. This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (4)
📒 Files selected for processing (112)
You can disable this status message by setting the 📝 WalkthroughWalkthroughThe PR migrates DASCore documentation to Great Docs, adding CI build automation, curated API configuration, alias and formatting fixups, new landing and tutorial content, revised page metadata and headings, and updated documentation cross-references. ChangesGreat Docs documentation migration
Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
⚔️ Resolve merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
coderabbitai
Bot
added
CI
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 7
🧹 Nitpick comments (3)
scripts/greatdocs_alias_inventory.py (1)
150-160: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winBlind
except Exception: continuehides alias-resolution failures.Both attribute-traversal loops swallow every exception without any trace. Since this script runs unattended in CI and its whole job is to keep cross-references resolving, a silent skip here could mask legitimate build regressions (e.g. a class attribute that now raises on class-level access) until a docs link quietly breaks.
♻️ Proposed diff
try: target = by_id.get(id(_unwrap(getattr(cls, attr)))) - except Exception: + except Exception as exc: + print(f"greatdocs_alias_inventory: skipping {cls.__name__}.{attr}: {exc}") continueAlso applies to: 166-176
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/greatdocs_alias_inventory.py` around lines 150 - 160, Replace the broad silent exception handling in both attribute-traversal loops around cls attribute resolution with explicit failure reporting: catch the expected attribute-access or unwrapping exceptions only, and emit a clear warning/error containing the class and attribute before continuing. Ensure unexpected exceptions propagate so CI exposes alias-resolution regressions, while preserving normal alias collection for successfully resolved attributes.Source: Linters/SAST tools
.github/workflows/build_docs_greatdocs.yaml (1)
33-37: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winPin
great-docsin this workflow. Installing it unpinned leaves the docs build exposed to upstream breaking changes; lock it to a fixed release or constraints file so CI stays reproducible.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/build_docs_greatdocs.yaml around lines 33 - 37, Pin the great-docs dependency in the workflow’s pip install command to a fixed, known-good release or reference a constraints file, while preserving the existing editable installation extras for docs, test, and extras.tutorial/configuration.qmd (1)
1-3: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueUse YAML front matter for the page title to maintain consistency.
To ensure consistent page metadata processing across the documentation generation, consider using YAML front matter for the title instead of a top-level Markdown heading.
♻️ Proposed refactor
-# Runtime Configuration - +--- +title: Runtime Configuration +--- + DASCore exposes a small runtime configuration surface through `dascore.config`.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tutorial/configuration.qmd` around lines 1 - 3, Replace the top-level “Runtime Configuration” Markdown heading in the page with YAML front matter defining the page title, while preserving the existing introductory content and title text.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/build_docs_greatdocs.yaml:
- Around line 39-41: Update the “Prepare test data” workflow step to remove the
unconditional `|| true` from the scripts/modify_h5_file.py invocation. Allow
genuine script failures to fail the job, while handling only the expected
“already modified” condition explicitly if the script requires that behavior.
- Around line 19-22: Harden the actions/checkout@v4 step by disabling persisted
Git credentials with the checkout action’s credential-persistence option, and
add a restrictive workflow/job permissions block granting only the access
required by the documentation build (read-only contents if needed). Keep the
existing fetch-tags and fetch-depth settings unchanged.
In `@dascore/proc/filter.py`:
- Line 541: Correct the typo in the FK recipe reference by changing “addtional”
to “additional” in the surrounding documentation text.
In `@recipes/contributing_to_documentation.qmd`:
- Line 3: Update the contributor build instructions in the documentation page to
use the Great Docs workflow, documenting `great-docs build` as the canonical
command. Replace or clearly label the legacy `scripts/build_api_docs.py` and
`quarto preview docs` commands so contributors do not follow them as the current
validation process.
In `@skills/dascore/SKILL.md`:
- Line 66: Update the range terminology in the documentation around the ellipsis
explanation to use the grammatically correct hyphenated form “open-ended,”
without changing the documented meaning or example.
In `@tutorial/patch.qmd`:
- Line 638: Change the “Coords in patch initialization” heading from level four
to level three so it is a sibling of “Dropping coordinates” under “Modifying
patches,” while leaving its content unchanged.
- Line 845: Change the “Numpy Functions” heading in tutorial/patch.qmd from
level four to level three so it is a sibling of “Patch operations with other
patches” under “Patch operations,” while leaving the surrounding content
unchanged.
---
Nitpick comments:
In @.github/workflows/build_docs_greatdocs.yaml:
- Around line 33-37: Pin the great-docs dependency in the workflow’s pip install
command to a fixed, known-good release or reference a constraints file, while
preserving the existing editable installation extras for docs, test, and extras.
In `@scripts/greatdocs_alias_inventory.py`:
- Around line 150-160: Replace the broad silent exception handling in both
attribute-traversal loops around cls attribute resolution with explicit failure
reporting: catch the expected attribute-access or unwrapping exceptions only,
and emit a clear warning/error containing the class and attribute before
continuing. Ensure unexpected exceptions propagate so CI exposes
alias-resolution regressions, while preserving normal alias collection for
successfully resolved attributes.
In `@tutorial/configuration.qmd`:
- Around line 1-3: Replace the top-level “Runtime Configuration” Markdown
heading in the page with YAML front matter defining the page title, while
preserving the existing introductory content and title text.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: b81397a0-b912-4f3d-a929-ef7368379b6b
⛔ Files ignored due to path filters (4)
assets/diataxis.pngis excluded by!**/*.pngassets/logo.pngis excluded by!**/*.pngassets/logo.svgis excluded by!**/*.svgassets/patch_n_spool.pngis excluded by!**/*.png
📒 Files selected for processing (68)
.github/workflows/build_docs_greatdocs.yaml.gitignoreabout/acknowledgements.qmdabout/contributors.qmdabout/supported_formats.qmdcontributing/adding_test_data.qmdcontributing/code_of_conduct.qmdcontributing/contributing.qmdcontributing/dev_install.qmdcontributing/documentation.qmdcontributing/extending_dascore.qmdcontributing/general_guidelines.qmdcontributing/new_format.qmdcontributing/profiling_benchmarks.qmdcontributing/publish_a_new_release.qmdcontributing/style_and_linting.qmdcontributing/testing.qmddascore/core/coordmanager.pydascore/examples.pydascore/io/core.pydascore/proc/correlate.pydascore/proc/filter.pydascore/proc/hampel.pydascore/proc/resample.pydascore/proc/whiten.pydascore/transform/fourier.pydascore/transform/integrate.pydascore/transform/strain.pydascore/utils/patch.pygreat-docs.ymlindex.qmdnotes/coordinate_internals.qmdnotes/dft_notes.qmdnotes/doc_strategy.qmdnotes/notes.qmdnotes/patch_attrs.qmdnotes/spool_chunking.qmdnotes/spool_index.qmdnotes/spool_selection.qmdnotes/velocity_to_strain_rate.qmdrecipes/add_spatial_coordinates_to_patch.qmdrecipes/contributing_to_documentation.qmdrecipes/correlate.qmdrecipes/despiking.qmdrecipes/docker_basic.qmdrecipes/edge_effects.qmdrecipes/external_conversion.qmdrecipes/fk.qmdrecipes/how_to_contribute.qmdrecipes/low_freq_proc.qmdrecipes/overview.qmdrecipes/parallelization.qmdrecipes/plotting_channel_number.qmdrecipes/real_time_proc.qmdrecipes/smoothing.qmdscripts/greatdocs_alias_inventory.pyscripts/greatdocs_build_fixups.pyskills/dascore/SKILL.mdtutorial/concepts.qmdtutorial/configuration.qmdtutorial/coords.qmdtutorial/file_io.qmdtutorial/patch.qmdtutorial/processing.qmdtutorial/remote_patches.qmdtutorial/spool.qmdtutorial/transformations.qmdtutorial/visualization.qmd
💤 Files with no reviewable changes (1)
- dascore/io/core.py
| - uses: actions/checkout@v4 | ||
| with: | ||
| fetch-tags: "true" | ||
| fetch-depth: "0" |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
Harden checkout: disable credential persistence and scope token permissions.
Static analysis flags credential persistence through the checkout action. Even though this workflow is workflow_dispatch-only, the runner's persisted git credentials remain available to any code that installs/executes during the build (e.g. pip install ... great-docs), which is an unnecessary supply-chain exposure. No permissions: block is set either, so the job gets the repo's default (potentially broad) GITHUB_TOKEN scope.
🔒 Proposed fix
+permissions:
+ contents: read
+
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-tags: "true"
fetch-depth: "0"
+ persist-credentials: false🧰 Tools
🪛 zizmor (1.26.1)
[warning] 19-22: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false
(artipacked)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/build_docs_greatdocs.yaml around lines 19 - 22, Harden the
actions/checkout@v4 step by disabling persisted Git credentials with the
checkout action’s credential-persistence option, and add a restrictive
workflow/job permissions block granting only the access required by the
documentation build (read-only contents if needed). Keep the existing fetch-tags
and fetch-depth settings unchanged.
Source: Linters/SAST tools
| - name: Prepare test data | ||
| run: | | ||
| python scripts/modify_h5_file.py || true |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
Suppressing all failures with || true hides real problems.
If modify_h5_file.py fails for a reason other than "already modified" (missing test data, syntax error, moved script), the build silently proceeds without prepared test data, and any resulting great-docs build failure becomes confusing to diagnose.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/build_docs_greatdocs.yaml around lines 39 - 41, Update the
“Prepare test data” workflow step to remove the unconditional `|| true` from the
scripts/modify_h5_file.py invocation. Allow genuine script failures to fail the
job, while handling only the expected “already modified” condition explicitly if
the script requires that behavior.
| >>> patch_filtered = patch.slope_filter(filt=filt) | ||
|
|
||
| The [FK recipe](`docs/recipes/fk.qmd`) provides addtional examples. | ||
| The [FK recipe](/recipes/fk.qmd) provides addtional examples. |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Fix typographical error.
addtional should be additional.
📝 Proposed fix
- The [FK recipe](/recipes/fk.qmd) provides addtional examples.
+ The [FK recipe](/recipes/fk.qmd) provides additional examples.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| The [FK recipe](/recipes/fk.qmd) provides addtional examples. | |
| The [FK recipe](/recipes/fk.qmd) provides additional examples. |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@dascore/proc/filter.py` at line 541, Correct the typo in the FK recipe
reference by changing “addtional” to “additional” in the surrounding
documentation text.
| ``` | ||
|
|
||
| ### Numpy Functions | ||
| #### Numpy Functions |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Wrong heading depth: "Numpy Functions" is nested under "Patch operations with other patches".
This #### heading sits under ### Patch operations with other patches (line 805), but the content (numpy ufuncs, PatchUFunc, apply_array_ufunc) is a distinct topic from patch-to-patch operations. It should be a sibling ### heading under ## Patch operations.
✏️ Proposed fix
-#### Numpy Functions
+### Numpy Functions📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| #### Numpy Functions | |
| ### Numpy Functions |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tutorial/patch.qmd` at line 845, Change the “Numpy Functions” heading in
tutorial/patch.qmd from level four to level three so it is a sibling of “Patch
operations with other patches” under “Patch operations,” while leaving the
surrounding content unchanged.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6636e4f0cc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| on: | ||
| workflow_dispatch: |
There was a problem hiding this comment.
Because this workflow is limited to manual dispatch, the active push/release workflows remain unchanged: test_doc_build.yml, build_deploy_master_docs.yaml, and build_deploy_stable_docs.yaml still invoke the composite actions that run quarto render docs using scripts/_templates/_quarto.yml. That template references docs/tutorial/*, docs/recipes/*, docs/_static/*, and other files deleted by this commit, so normal documentation CI and deployments will fail rather than exercise the prototype. Preserve the old tree until migration or update those active workflows in this change.
Useful? React with 👍 / 👎.
| # matching the old site layout, so existing relative links keep working. | ||
| sections: | ||
| - title: Tutorial | ||
| dir: tutorial |
There was a problem hiding this comment.
Moving the tutorial to this top-level directory removes its executable examples from the normal test suite: scripts/generate_doc_code_tests.py still hard-codes DOCS_PATH = REPO_ROOT / "docs", and .github/workflows/runtests.yml continues to invoke that generator. In this tree it discovers only three remaining QMD files under docs/, while the relocated tutorial, recipes, notes, contributing, and about directories contain many executable pages, so regressions in those examples will now pass CI silently.
Useful? React with 👍 / 👎.
| | Read data files | `dc.spool(path)[0]` or iterate | `dc.read` (low-level) | | ||
| | Discover file metadata cheaply | `dc.scan(path)` / `dc.scan_to_df(path)` | reading whole files | | ||
| | Subset by time/distance values | `patch.select(time=(t1, t2))` | index math | | ||
| | Subset by sample index | `patch.select(time=(0, 100), samples=True)` | `iselect` shorthand also exists | |
There was a problem hiding this comment.
iselect shorthand
When an agent follows this decision table, it may recommend or emit patch.iselect(...), but iselect has been removed from the current Patch API—repo-wide search finds no implementation, and this migration explicitly omits it from the curated reference. The skill should only show patch.select(..., samples=True) so generated guidance does not produce an AttributeError.
Useful? React with 👍 / 👎.
| @@ -1,5 +1,6 @@ | |||
| --- | |||
| title: Coordinate Internals | |||
There was a problem hiding this comment.
Relocating this page makes several generated API links point to a nonexistent file: dascore/core/coords.py, dascore/core/summary.py, and dascore/io/dasdae/utils.py still reference docs/notes/coordinate_internals.qmd. Since the alias-inventory hook only adds Python-object and module aliases, it cannot redirect this file path; these references need to target /notes/coordinate_internals.qmd or the new relative location.
Useful? React with 👍 / 👎.
| - title: About | ||
| dir: about | ||
| index: true |
There was a problem hiding this comment.
The new section list never includes docs/supported_plugins.qmd, and repo-wide search finds no relocated copy or other reference to it in great-docs.yml. Because great-docs only discovers narrative pages from the configured section directories, the existing Supported Third-Party Plugins page—and its dynamically generated registry table—disappears from the new site even though plugin support remains part of DASCore. Move it into about/ or otherwise add it to the build.
Useful? React with 👍 / 👎.
| out = ( | ||
| patch.detrend("time") # most proc funcs are methods | ||
| .pass_filter(time=(1, 100)) # units in Hz for time dim | ||
| .velocity_to_strain_rate() |
There was a problem hiding this comment.
For an arbitrary file loaded by the preceding lines, this call commonly raises PatchAttributeError: velocity_to_strain_rate is decorated with required_attrs={"data_type": "velocity"} and also requires a distance dimension. Presenting it as part of the general core workflow causes agents to generate failing code for strain, strain-rate, unlabeled, or non-DAS inputs; use a universally applicable operation here or explicitly construct and validate a velocity patch first.
Useful? React with 👍 / 👎.
| - Supported formats and their read/scan/write capabilities are listed at | ||
| https://dascore.org/supported_formats.html — writing is only supported |
There was a problem hiding this comment.
The new site places this page under the configured about directory—index.qmd already links to about/supported_formats.qmd—but the newly added skill still sends users to the old root URL. No redirect or root-level copy is configured, so after deploying the great-docs build this resource link resolves to a 404; update it to /about/supported_formats.html or preserve the legacy route with a redirect.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
🧹 Nitpick comments (3)
assets/dascore.scss (3)
32-35: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueFix empty comment warning.
Stylelint reports an unexpected empty comment on line 34. Consider replacing it with a blank line.
✨ Proposed fix
// great-docs shrinks h2/h3 to 1.5rem/1.3rem and drops the h2 underline Quarto // uses to separate major sections; the old site had Quarto's defaults for both. -// + // These are rules rather than `$h2-font-size` overrides because great-docs🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@assets/dascore.scss` around lines 32 - 35, Remove the empty comment reported by Stylelint in the explanatory comment block near the great-docs heading rules, replacing it with a blank line while preserving the surrounding documentation and styles.Source: Linters/SAST tools
68-71: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueFix empty comment warning.
Stylelint reports an unexpected empty comment on line 69. Consider replacing it with a blank line.
✨ Proposed fix
// Content tables (ported from docs/styles.css) -// + // Scoped to prose tables so great-docs' own components -- API summary tables // and Great Tables output -- keep their styling.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@assets/dascore.scss` around lines 68 - 71, Remove the empty comment line in the content-tables section of the SCSS, leaving a blank line between the existing comments so Stylelint no longer reports an unexpected empty comment.Source: Linters/SAST tools
43-46: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueAdd empty line before comment.
Stylelint expects an empty line before comments. Adding a blank line here improves readability and resolves the warning.
✨ Proposed fix
font-size: calc(1.29rem + 0.48vw); + // `--bs-border-color` is `#dee2e6` in light mode, which is exactly what the // old site drew, and Bootstrap darkens it under `data-bs-theme="dark"`. border-bottom: 1px solid var(--bs-border-color);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@assets/dascore.scss` around lines 43 - 46, Insert a blank line between the font-size declaration and the comment preceding the border-bottom declaration in the affected stylesheet rule, preserving all existing styles and comments.Source: Linters/SAST tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@assets/dascore.scss`:
- Around line 32-35: Remove the empty comment reported by Stylelint in the
explanatory comment block near the great-docs heading rules, replacing it with a
blank line while preserving the surrounding documentation and styles.
- Around line 68-71: Remove the empty comment line in the content-tables section
of the SCSS, leaving a blank line between the existing comments so Stylelint no
longer reports an unexpected empty comment.
- Around line 43-46: Insert a blank line between the font-size declaration and
the comment preceding the border-bottom declaration in the affected stylesheet
rule, preserving all existing styles and comments.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 6b46a9ed-e729-4f9e-8ef3-5cdee294bebc
📒 Files selected for processing (2)
assets/dascore.scssgreat-docs.yml
0362f83 to
d8ee8c6
Compare
d-chambers
changed the title
d8ee8c6 to
4374cf4
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## dev #760 +/- ##
==========================================
Coverage 100.00% 100.00%
==========================================
Files 182 182
Lines 21627 21773 +146
==========================================
+ Hits 21627 21773 +146
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
✅ Documentation built: |
Replace the custom doc-build pipeline (scripts/build_api_docs.py and friends) with a great-docs configuration: - Add great-docs.yml: curated, task-grouped API reference, sections, excludes, and bibliography wiring - Move narrative docs to top-level section dirs (tutorial, recipes, notes, contributing, about) so site paths are unchanged; move docs/_static to assets/ - Port the landing page to a root index.qmd - Add a curated AI agent skill (skills/dascore/SKILL.md) published at /.well-known/agent-skills by the build - Rewrite alias-path and module cross-links to canonical documented paths in narrative docs and docstrings - Fix docstring links that used docs/ path prefixes - Update the contributing documentation build instructions - Add a proposed CI workflow sketch The old build system is intentionally left in place for side-by-side comparison.
Add scripts/greatdocs_alias_inventory.py, run as a quarto pre-render hook (great-docs.yml pre_render key). After great-docs generates objects.json, the script imports dascore and appends inventory entries for: - every public access path to a documented object (Patch.pass_filter and the other dynamically attached methods, re-export paths like dascore.core.Patch, and canonical module.qualname paths) - module references (dascore.proc, dascore.viz, ...), which link to the matching section of the API index With this, docstrings and narrative pages can link to any public alias path or documentation page and the built site has zero unresolved cross references. Also document coords_from_df, get_summary_df, H5Reader and H5Writer, drop a stale see-also link to the removed iter_fs_contents, avoid an interlink in a docstring summary line, and update the contributing docs to describe the linking rules.
- Navy navbar with gold keyline and logo-derived accent colors (light/dark variants), APCA-picked navbar text - Lucide icons on top navigation entries - Navy headings, bordered h2s, softened code blocks via a small include_in_header stylesheet - Rewrite the landing page: hero carries the branding; drop the readme badge wall; installation first, then executable examples, citation, and support links - Hide the duplicate landing-page title and stop the navbar wordmark from truncating
Update the great-docs prototype to build cleanly against dev's current API and address several rendering issues surfaced by rebasing onto dev. great-docs.yml (API reference): - Drop members/objects removed on dev: Patch.assign_coords/iresample/ iselect/tran, PatchAttrs.coords_from_dims/dim_tuple/parse_coord_ attributes/rename_dimension, CoordManager.update_from_attrs, proc.BaseCoord.get_attrs_dict, transform.rfft/spectrogram, io.PyTables*. - Replace clients.dirspool.DirectorySpool with the unified Spool. - Restrict io.H5Reader/H5Writer to get_handle so griffe's dynamic introspection no longer recurses into the h5py.File constructor and fails; this keeps the whole API reference in dynamic mode, which resolves module-shadowing functions (proc.correlate, viz.waterfall, transform.hilbert, ...) to the function rather than framing them as modules. Build hooks and styling: - Add scripts/greatdocs_build_fixups.py (quarto pre_render) to undo the heading bump great-docs applies to code-cell comments in index.qmd, and to dedent/de-duplicate GitHub release bodies in changelog.qmd so their headings render instead of appearing as literal markdown. - Fix the homepage directory-spool example, which errored under dev's stricter spool selection, by making it an illustrative (non-executed) snippet. - CSS: wrap long sidebar entries at word boundaries and restore the "Notes" docstring callout box on API pages (shift-heading-level-by promotes it to h1, which great-docs' h2/h3 rule missed). CI: build_docs_greatdocs.yaml installs the docs/test/extras deps that the example pages import (pytest, findiff, xarray, obspy, ...).
Fix heading levels and enrich section-index cards across the tutorial, recipes, notes, contributing, and about sections. - Demote headings by one level in pages that used `#` for their top sections. Under great-docs' `shift-heading-level-by: -1` those `#` headings rendered as plain paragraphs and were missing from the page TOC; they now render as real headings. Code-cell comments are left untouched. Pages that already used `##` sections are unchanged. - Give the few pages that relied on a bare `#` title (or an empty frontmatter block) an explicit frontmatter `title:` so the title renders correctly. - Add a `description:` to every card-index page so the recipes/notes/ contributing/about landing pages show informative cards instead of bare titles.
The prototype styled the site with a <style> blob in great-docs.yml -- navy
headings and a navy/gold navbar painted over great-docs' default flatly theme.
That fought the framework and still did not look like the site it replaces.
The old build's appearance was not custom: it was Quarto's yeti Bootswatch
theme plus docs/styles.css. great-docs forwards site.theme straight into
_quarto.yml, and listing great-docs.scss explicitly (it is otherwise appended)
lets a third file layer on top, so this becomes ordinary Quarto theming:
site:
theme: [yeti, great-docs.scss, assets/dascore.scss]
assets/dascore.scss replaces the inline CSS. It ports docs/styles.css and
restores the handful of places great-docs diverges from the old site.
Verified against the live dascore.org stylesheet: font (Open Sans), primary
(#008cba), body color (#222), body size (18px), h2/h3 sizes and code-block
background now match, as do the responsive heading breakpoints.
Notes for future edits:
- great-docs assigns $h2-font-size/$h3-font-size without !default, so no
scss:defaults block can override them regardless of theme order; those two
had to be rules.
- Heading sizes are the fluid calc() pair the old site used, not flat rem.
h1 stays fluid, so a fixed h2 overtakes it on narrow screens.
- Colors ported from docs/styles.css only ever ran in light mode; table
borders/stripes go through local custom properties with dark counterparts,
and the h2 rule uses --bs-border-color, which Bootstrap darkens itself.
- great-docs hard-codes the active sidebar color for light mode but reads the
--gd-active-link token for dark, so both are handled.
Rebasing onto dev picked up API changes that break or skew the doc build: - BaseSpool.viz and io.PatchFileSummary were removed by dev; leaving them in the reference config fails the API-reference step outright. - Spool.stack moved up to BaseSpool, so document it there. - Drop exclude entries for names that no longer exist. - Point the new coordinate-internals docstring links at the site path used by the rest of the package.
great-docs 0.16 renders class signatures with their parameters and points See Also links at the pages the reference actually generates (0.14 emitted 404s for every one of them), so require it in the doc workflow. Document the API dev added since the prototype was written, which was turning up as unresolved cross references: scan_payloads/ScanPayload, PatchSummary, the segmented-coordinate types, ChunkPlan, clear_remote_file_cache and the new exceptions. Also document the aggregate functions (patch.mean, patch.std, ...), which the reference had never covered. Reference entries have to be reachable by attribute traversal from the dascore package: adding io.utils.get_exact_coord, where io.utils is only a submodule, makes griffe's dynamic introspection fail and silently drops the whole reference to static analysis, leaving compose_docstring templates unrendered. Pin the prototype workflow's actions to hashes so it passes the zizmor policy dev now enforces.
great-docs now renders the whole site, so the custom pipeline it replaced comes out: the api-doc scripts and their templates, the link-filling pandoc filter, the old landing page and stylesheet, and the prep_doc_build action. The three doc workflows share one build-docs action that installs great-docs and runs it; master deploys great-docs/_site to netlify with the netlify CLI, since `quarto publish` has no project to publish now that the site id in docs/_publish.yml is gone. Two pages from docs/ survive the move: the hand-maintained list of unreleased API changes (the generated changelog only covers published releases) and the third-party plugin table, both now in about/. generate_doc_code_tests.py walks the section directories rather than docs/, which is otherwise nearly empty; without this it would silently stop mirroring every narrative page into the test suite. Drop the section landing pages that only listed their own section's pages: great-docs generates a card index for each, so they showed up as a card inside themselves. contributing.qmd has content of its own and keeps its url, so only its title changes. The generated cards are laid out one per row with banner-sized padding; style them as a compact responsive grid.
The curated API reference covers the patch, spool and processing surface, which left the 169 public helpers in dascore.utils undocumented: the old build walked the package and rendered them all, a page each. A page per helper triples the site for objects that are supporting cast, so render them onto a single page instead. render_package_api walks the package and renders each module as a section and each object as an entry: signature and summary in the open, and the parameter table, returns and examples inside a collapsed callout, so the page reads as an index but carries the full docs. Parameter types come from the annotations, since DASCore documents them there rather than in the docstring. The page table of contents lists the 28 modules while every object keeps an explicit anchor. Those anchors are what cross references resolve to: the inventory hook now registers each helper at its anchor on the page, so existing links like `dascore.utils.chunk_plan.ChunkPlan` keep working and the five utils entries the reference used to carry are no longer needed.
Two gaps showed up once the whole of dascore.utils was rendered onto one page. Sections griffe parses into models rather than strings (Attributes, Other Parameters) fell to the catch-all branch and printed an object repr into the page; they now render as a table like Parameters does. And decorated helpers such as functools.cache wrappers are callables rather than functions, so the public-object walk skipped them, dropping both their docs and the anchor cross references resolve to.
great-docs records a member's uri with its fully qualified name as the fragment (Patch.html#dascore.Patch.data) while quarto gives that member's section the short id (#data), so every member cross reference landed at the top of the page instead of the member. Nothing warns: the file exists, only the fragment is absent. Repoint the inventory uris before the aliases are derived from them, and drop the fragment entirely for an object which owns its whole page.
The API reference is curated, which reads better than a generated dump but means new public API is invisible until someone lists it, and nothing about adding a function makes that omission noticeable: dev had grown PatchSummary, the config API, CoordSegmented and eleven aggregate functions with no signal. Compare the public surface against everything the docs cover and fail when an object appears nowhere, counting the reference, the utilities page, the supported formats table, and an explicit list of deliberate omissions. The same test checks the reference does not rot in the other direction: that entries still resolve by attribute access, which is what keeps one stale entry from silently dropping the whole reference to static analysis, that listed members exist, and that a documented class has no undocumented public members. Doing so found the config and inventory APIs, twenty-one properties and a handful of functions missing, along with an exclude entry for a package removed in #751; all are now covered.
The coverage test caught what it exists to catch: dev has since grown Patch.enrich, Spool.unselect, the inventory attach/remove/enrich methods, an inventory example pair, make_scan_payload and UnresolvedPatchError, none of which the curated reference listed. The inventory resolution helpers stay undocumented and are recorded as such; they are plumbing behind enrich rather than API, and are not re-exported.
The test pinned the contents of docs/changelog.qmd and keyed its skip on docs/index.qmd, so on this branch it silently skipped rather than failed: the docs no longer live under docs/, and great-docs generates the changelog page from the GitHub releases, which is what preserves the published URL. Assert the policy that survives the move -- that no changelog source exists and the config still names the repository the releases come from.
A docstring may hold a code block of its own, and compose_docstring's does.
Wrapping it in a fence of the same length ended the block at the example's
opening fence: the example body rendered as prose and the rest of the page,
including the next entry's anchor, was swallowed by the next fence pandoc
met. Pass an already-fenced example through, demoting an executable
`{python}` cell to a plain block so the page shows it rather than running
it, and size the wrapper fences around whatever the content contains.
The anchor repointing learns the same lesson from the other direction: the
id pandoc derives from a heading is lowercased, so `Patch.T` lands at `#t`,
and a member the page does not render (a dunder) has no anchor at all.
Each of these let a real omission pass. A helper defined in dascore/utils/__init__.py counted as covered while the utilities page, which walked only the submodules, rendered neither it nor its anchor; the page and the test now walk the package the same way. A class listed with an explicit empty `members` list was skipped rather than checked, so emptying one would silently undocument every member. An ImportError naming an installed package -- a misspelled `from numpy import ...` -- was swallowed as a missing optional dependency, dropping every object in that module from the comparison. The changelog scan hard-coded its directories, missing both the utilities section and anything nested, and accepted `repo:` with no value even though an empty one takes the generated changelog page with it. A reader was counted as listed on the formats page when it merely shared a registered reader's name and version, rather than being the registered one.
Rebasing picked up Patch.unselect, Spool.conform_to_inventory and Spool.split_by, none of which the curated reference listed. The placement helpers behind them, and the directory loader dc.inventory() reaches, stay undocumented and are recorded as such.
griffe parses the docstrings the utilities page is built from, and it arrives with the doc build rather than with dascore, so the minimal dependency, free-threaded and WASM jobs do not have it. Two things went wrong there: the render tests errored instead of skipping, and render_package_api swallowed the ImportError as a module with a missing optional dependency, so the whole page came back empty without a word. Import griffe through optional_import, guard the module import alone, and skip the render tests where griffe is absent.
The renderer carried three guards no docstring in the package reaches: a fallback for a signature inspect cannot read, which every function and class it documents has, and a text branch and a string branch for section shapes griffe's numpy parser does not produce. Fold the two text cases into one, let an unreadable signature fail the build rather than half-render, and drop unhandled sections outright, which is what the model-valued ones already do. The one guard that does earn its place, skipping a module whose optional dependency is missing, now has a test.
da5aedd to
7218e92
Compare
#891 moved the pydantic base classes and their annotated types out of dascore.utils.models into a dascore.models package, which took them off the utilities page that had been documenting them. Give them a reference section of their own, along with the two registry calls a package registering its own model would use; the rest of the tagging machinery stays internal. The package docstring links to DascoreBaseModel, so the link now resolves too.
1 participant
Description
Replaces DASCore's hand-written documentation build with great-docs 0.16, Posit's quartodoc-based static site builder.
The old build was ~2000 lines of local machinery —
scripts/build_api_docs.py, a qmd builder, an API indexer, a link validator, jinja templates and a generated_quarto.yml— all of which is deleted here. What replaces it is onegreat-docs.yml, two small pre-render hooks, and a sharedbuild-docsaction the three doc workflows call. Doc builds go from 44–54 minutes in CI to roughly 5, and the site gains agent-facing outputs it did not have:llms.txt, a markdown twin of every page, and agent skills under.well-known.Site structure. The narrative pages move out of
docs/into top-level section directories (tutorial/,recipes/,notes/,contributing/,about/), withindex.qmdas the landing page; published paths are unchanged.docs/keeps only the license and the bibliography. The changelog page is now generated from the GitHub releases rather than written by hand, which is the same policy #864 established for the repository.Theme. The site keeps the old one:
great-docs.ymlbuilds on theyetiBootswatch theme the previous build used, andassets/dascore.scssports the old custom CSS on top of it. Compiled variables were diffed against the live dascore.org stylesheet — font, primary color, body color and size, heading sizes and code-block background all match. Two structural differences remain, both belonging to the framework's layout rather than the CSS: the sidebar is a boxed panel, and the logo sits in the navbar rather than the left sidebar.Cross references. DASCore reaches most of its functionality through aliases — processing functions attached to
Patchas methods, objects re-exported at several paths, module links in prose — and great-docs indexes only each object's canonical location.scripts/greatdocs_alias_inventory.pyruns as a quarto pre-render hook and adds an inventory entry for every public path that reaches a documented object, so all the existing link spellings keep working. It also repoints member uris at the anchors the rendered pages actually emit: great-docs recordsPatch.html#dascore.Patch.datawhile quarto gives that section the id#data, lowercased. Every fragment in the built inventory was checked against the built HTML; none are dead.API reference. The reference is curated in
great-docs.ymland grouped by task rather than mirroring the package layout. Curation reads better than a generated dump but means new public API is invisible until someone lists it, sotests/test_doc_coverage.pycompares DASCore's public surface against everything the docs cover and fails when an object appears nowhere. An object counts as covered by a reference entry, by the utilities page, by the supported-formats table, or by an explicit list of deliberate omissions. The same test pins the reference against rotting in the other direction: entries must resolve by attribute access — one that does not silently degrades the entire reference to static analysis, which renders every templated docstring as its literal placeholder — members named must exist, and a documented class must have no uncovered public members.The 169 public helpers in
dascore.utilsare rendered onto a single page rather than given a page each, which would have tripled the size of the site for objects that are supporting cast. Each keeps its own anchor, so links to them resolve as before.Changelog
.qmdfiles move fromdocs/into top-level section directories, and site structure, styling and the API reference are configured ingreat-docs.yml. The site now also publishesllms.txt, a markdown twin of each page, and agent skills under.well-known.dascore.utils.docs.render_package_apiandrender_module_apirender a package's public API as markdown, which is how the whole ofdascore.utilsis documented on one page.Checklist
I have:
contributing/general_guidelines.qmd).I have (if applicable):