docs(storage): generate connector docs from runtime descriptor catalog - #507
Merged
Conversation
- Add `tests/storage_connector_docs.rs` integration test that reads authenticated built-in connector descriptor and localization APIs, then projects facts into committed artifacts - Add `docs/generated/storage-connectors.json` manifest with full capability, credential mode, and upload workflow data for 8 connectors - Replace hand-maintained backend tables in storage-backends index, storage-policies catalog, and storage-matrix with marker-bounded generated blocks (`storage-connectors:*:start/end`) - Drive `docs/astro.config.mts` storage-backend sidebar from manifest instead of a duplicate static list - Add `make storage-docs` (regenerate) and `make storage-docs-check` (drift check) targets; wire both into CI via `docs-check.yml` - Fix object-storage connector `capacity: false` — S3, OSS, COS, and Azure Blob data-plane APIs expose no portable remaining-capacity contract; add unit test asserting capacity claims per connector - Add `built_in_connector_capacity_claims_match_runtime_probe_support` unit test in `src/storage/connectors/tests.rs` - Expand `storage_credential_secret_key` docs to cover all static-secret connectors now encrypted at rest, not only OneDrive - Update contributor docs (en + zh-CN) with new authoring workflow and non-exhaustive-example rule for contextual provider mentions
|
Warning Review limit reached
Next review available in: 51 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
📝 WalkthroughWalkthrough该变更以运行时连接器描述符和本地化数据生成存储文档产物。新增 manifest、能力矩阵、策略目录和动态侧边栏。新增 Make targets、CI 校验及中英文维护规则。 Changes存储连接器文档事实投影
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant storage_connector_docs_test
participant AdminStorageApi
participant GeneratedDocs
participant AstroSidebar
participant storage_docs_check
storage_connector_docs_test->>AdminStorageApi: 获取连接器描述符和本地化数据
AdminStorageApi-->>storage_connector_docs_test: 返回连接器身份与能力事实
storage_connector_docs_test->>GeneratedDocs: 生成 manifest、索引、矩阵和策略目录
AstroSidebar->>GeneratedDocs: 读取 storage-connectors.json
GeneratedDocs-->>AstroSidebar: 返回动态侧边栏条目
storage_docs_check->>GeneratedDocs: 校验提交产物不存在漂移
Possibly related PRs
Suggested labels: Poem
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (3 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (3)
src/storage/connectors/tests.rs (1)
626-650: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win建议让容量断言覆盖全部注册 connector。
当前两个列表是手写的。新增内置 connector 时，测试不会失败，容量声明也不会被检查。可以先从
registry().descriptors()取全集，再断言两个列表的并集等于全集。这样以后加 connector 才不会偷偷漏掉。
♻️ 建议的补充断言
fn built_in_connector_capacity_claims_match_runtime_probe_support() { - for connector_id in [ + let with_capacity = [ LocalConnector::ID, OneDriveConnector::ID, RemoteConnector::ID, - ] { + ]; + let without_capacity = [ + S3Connector::ID, + AlibabaOssConnector::ID, + AzureBlobConnector::ID, + TencentCosConnector::ID, + SftpConnector::ID, + ]; + assert_eq!( + registry().descriptors().count(), + with_capacity.len() + without_capacity.len(), + "new built-in connector must declare its capacity support in this test" + ); + for connector_id in with_capacity {🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/storage/connectors/tests.rs` around lines 626 - 650, Update built_in_connector_capacity_claims_match_runtime_probe_support to derive the registered connector IDs from registry().descriptors() and verify the positive and negative capacity lists form a complete, non-overlapping partition of that registry. Keep the existing expected capacity assertions while adding coverage that fails when a newly registered connector is omitted.tests/storage_connector_docs.rs (2)
544-572: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win重复标记检查只覆盖了结束标记之后的区域。
assert!从end + end_marker.len()开始搜索start_marker。如果同一页在生成块内部又出现一个start_marker，检查不会触发，内容会被静默吞掉。建议在整篇文档里统计标记出现次数。顺手补一下，成本只有两行。
♻️ 建议改为全文计数
- assert!( - current[end + end_marker.len()..] - .find(start_marker) - .is_none(), - "{} contains duplicate marker {start_marker}", - path.display() - ); + assert_eq!( + current.matches(start_marker).count(), + 1, + "{} must contain exactly one {start_marker}", + path.display() + ); + assert_eq!( + current.matches(end_marker).count(), + 1, + "{} must contain exactly one {end_marker}", + path.display() + );🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/storage_connector_docs.rs` around lines 544 - 572, 更新 replace_generated_block 中的重复 start_marker 校验，改为统计整篇 current 文档中的 start_marker 出现次数并要求仅出现一次，覆盖生成块内部及其后的重复标记；保留现有缺失标记和替换逻辑不变。
69-118: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚖️ Poor tradeoff教程 slug 与“适合场景”硬编码在测试文件里。
PRESENTATIONS是 provider-owned 文档元数据，却和生成器逻辑放在同一个测试文件中。文档贡献者要改一句“适合场景”，就必须改 Rust 测试。可以考虑把这份表拆成一个受审查的数据文件（例如docs/generated/同级的 TOML/JSON 输入），生成器只负责读取。不着急，等 connector 再多几个的时候一定会痒。
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/storage_connector_docs.rs` around lines 69 - 118, 将 ConnectorDocumentationPresentation 表从测试文件中的 PRESENTATIONS 常量移出，放入受审查的 TOML/JSON 文档数据文件中；更新文档生成逻辑以读取该数据源，并保留现有 connector_id、tutorial_slug 及中英文 best_for 内容和生成结果不变。
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/docs-check.yml:
- Line 34: 在 docs-check 工作流的 actions/checkout 步骤中设置 persist-credentials:
false，关闭 GITHUB_TOKEN 的凭据持久化；保留现有只读检出行为不变。
In `@docs/src/content/docs/en/reference/storage-matrix.md`:
- Line 27: Update the English authentication documentation section for
[auth].storage_credential_secret_key to state that losing the key makes OneDrive
old refresh tokens unrecoverable and requires reauthorization. The anchor
docs/src/content/docs/en/reference/storage-matrix.md:27 and sibling
docs/src/content/docs/reference/storage-matrix.md:27 require no direct changes;
they already describe the encryption and key-preservation behavior.
In `@Makefile`:
- Around line 143-145: Update the storage-docs-check target to explicitly unset
ASTER_UPDATE_STORAGE_CONNECTOR_DOCS when invoking the
generated_storage_connector_docs_are_current test, ensuring inherited
environment variables cannot trigger assert_or_update to rewrite manifests or
documentation. Preserve the target’s existing exact test behavior.
---
Nitpick comments:
In `@src/storage/connectors/tests.rs`:
- Around line 626-650: Update
built_in_connector_capacity_claims_match_runtime_probe_support to derive the
registered connector IDs from registry().descriptors() and verify the positive
and negative capacity lists form a complete, non-overlapping partition of that
registry. Keep the existing expected capacity assertions while adding coverage
that fails when a newly registered connector is omitted.
In `@tests/storage_connector_docs.rs`:
- Around line 544-572: 更新 replace_generated_block 中的重复 start_marker 校验，改为统计整篇
current 文档中的 start_marker 出现次数并要求仅出现一次，覆盖生成块内部及其后的重复标记；保留现有缺失标记和替换逻辑不变。
- Around line 69-118: 将 ConnectorDocumentationPresentation 表从测试文件中的
PRESENTATIONS 常量移出，放入受审查的 TOML/JSON 文档数据文件中；更新文档生成逻辑以读取该数据源，并保留现有
connector_id、tutorial_slug 及中英文 best_for 内容和生成结果不变。
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 0accf27b-651d-4c3a-97a9-fae3dc4dcd37
⛔ Files ignored due to path filters (1)
docs/generated/storage-connectors.jsonis excluded by!**/generated/**
📒 Files selected for processing (21)
.github/workflows/docs-check.ymlMakefilecrates/aster_drive_storage/src/connector_descriptor.rsdeveloper-docs/en/contributing/documentation.mddeveloper-docs/en/design/storage-descriptor-normalization-contract.mddeveloper-docs/zh-CN/contributing/documentation.mddeveloper-docs/zh-CN/contributing/task-routing.mddeveloper-docs/zh-CN/design/storage-descriptor-normalization-contract.mddocs/astro.config.mtsdocs/src/content/docs/admin/storage-backends/index.mddocs/src/content/docs/admin/storage-policies.mddocs/src/content/docs/en/admin/storage-backends/index.mddocs/src/content/docs/en/admin/storage-policies.mddocs/src/content/docs/en/reference/config/auth.mddocs/src/content/docs/en/reference/index.mddocs/src/content/docs/en/reference/storage-matrix.mddocs/src/content/docs/reference/config/auth.mddocs/src/content/docs/reference/index.mddocs/src/content/docs/reference/storage-matrix.mdsrc/storage/connectors/tests.rstests/storage_connector_docs.rs
- Add `persist-credentials: false` to the docs-check workflow checkout step - Unset `ASTER_UPDATE_STORAGE_CONNECTOR_DOCS` in `storage-docs-check` to prevent accidental doc regeneration during verification - Refactor capacity claims test to assert all built-in connectors are covered exactly once, catching any newly added connector that is missing from either expectation list - Replace duplicate-marker detection with exact-count assertions for both start and end markers in `replace_generated_block`
Member
Author
|
CodeRabbit review-summary nitpicks triaged against head
Focused validation passed for the capacity test, marker-backed docs drift check, Rust formatting, and diff whitespace. |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
StorageConnectordescriptor and localization catalog as the canonical connector factstests/storage_connector_docs.rsas an integration-test projection tool, keeping documentation generation out of production modulesmake storage-docs,make storage-docs-check, and a docs CI drift checkBoundary
The generator calls the same authenticated admin descriptor/localization APIs used by the frontend. It does not expose the private registry, add a production documentation module, or define a parallel runtime capability type. Only tutorial slugs and short provider-owned “best for” summaries remain curated in the test projection.
Validation
ASTER_UPDATE_STORAGE_CONNECTOR_DOCS=1 cargo test --test storage_connector_docs generated_storage_connector_docs_are_current -- --exact --nocapturemake storage-docs-checkcargo test --lib storage::connectors(38 passed)cargo fmt --all -- --checkcd docs && bun run docs:build(163 pages, links valid)cd docs && bun run developer-docs:build(414 pages, links valid)git diff --checkCloses #473
Summary by CodeRabbit
新功能
文档