Skip to content

feat(spec,service-datasource): datasource.config 按驱动契约校验 (#4410) - #4465

Merged
os-zhuang merged 4 commits into
mainfrom
claude/datasource-config-validation-i7wdmo
Aug 1, 2026
Merged

feat(spec,service-datasource): datasource.config 按驱动契约校验 (#4410)#4465
os-zhuang merged 4 commits into
mainfrom
claude/datasource-config-validation-i7wdmo

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Closes #4410

config 是数据源上唯一一个作者会写、却完全没有校验的槽。schema 自己的注释说 "the driver's
own configSchema is what validates it" —— 没有任何东西这么做:两个 bundled driver spec 都
configSchema 设成 {},没有代码读这个字段,per-driver 的 zod schema 甚至没从包里导出
data/driver/ 只有它自己的测试能碰到)。于是 config: { hostname: 'db.internal' }(正确
的键是 host)被静默接受,数据源连到 localhost,而 parse、save、连接探测全部报告成功。

先回答 issue 的前置问题:前端确实在读驱动的 configSchema

不是第三处虚假声明,是一处重复声明。objectui 的 DatasourceResourcePage.tsx
GET /api/v1/datasources/drivers,用 configSchema.properties 渲染连接表单(title /
description / default / format:'password' / enumrequired 决定必填)。那句 JSDoc
是真的——但它读的是 service-datasource 里的 DRIVER_CATALOG,另一套手写 JSON-Schema
字面量,和 spec 的 zod schema 各写各的、互不校验、两边都不做验证。一活一死,中间没有任何
东西对得上。

改动

packages/spec/src/data/driver/ 成为唯一契约,三个消费者读它:

  • DatasourceSchema 解析 config 以及每个 readReplicas 条目;
  • DriverDefinitionSchema.configSchema 发布它的 JSON-Schema 投影;
  • Studio 连接表单渲染同一份投影,所以表单提供的字段恰好就是校验器接受的字段。

mysqlsqlite / sqlite-wasm 此前根本没有 config 形状,尽管表单在提供、工厂能构建。

Setup 向导是另一道作者入口且不经过 DatasourceSchemacreateDatasource
metadata.register,那里的校验是 name/label 结构检查,不是 zod parse),所以
DatasourceAdminService 的 create/update/test 也查同一个 registry ——testConnection 在探测
之前校验,否则向导会对着 localhost 报一个绿色的"连接成功"。

平台不提供契约的驱动(插件的 com.vendor.snowflake)保持 config 不校验:那是诚实的边界,
不是遗留的洞——拿一个我们没有的形状去否定作者的键,比它替换掉的沉默更糟。

装门禁反过来逼出的一批 inert 声明

config 一旦有门禁,里面每个键就都在声称自己被读,于是逐个对着读它的代码核了一遍。
接上的(此前声明了、但被丢在地上):

  • datasource.pool —— 声明过、strict、还被带进了 connection spec,然后被工厂硬编码的
    { min: 0, max: 5 } 覆盖掉;现在每个 SQL 驱动都honour它,并映射到 Mongo 的
    minPoolSize/maxPoolSize。未指定时的默认值保持不变。
  • datasource.schemaMode —— 在 record → spec 之间被丢掉,工厂只好去两个永远不可能有它的
    地方找。结果是 schemaMode: 'external' 的库被当作 managed 构建,驱动层的 DDL 闸门是开的。
  • datasource.ssl —— 同样停在 record,带 CA 证书的 TLS 块什么也没配上;这正是它自己
    schema 注释里警告的失效模式。
  • postgres schema(knex searchPath)、applicationNamestatementTimeout
  • mongo passwordauthSourceoptions。带 config.password 的 mongo 数据源此前是用
    空密码拼 URL 的。

没有可接的地方就移除:memory 的 indexes / maxRecordsPerObject —— InMemoryDriverConfig
没有这两个字段,驱动不建索引(每次读都是线性 Mingo 扫描)也不淘汰,按 ADR-0049 移除,拒绝
信息里带上原因。

一处中途改回的设计

config.ssl 最初建模成 boolean | { ca, cert, key },对客户端接受的形状是诚实的。读
objectui 的 renderField 后发现:不是 boolean/enum/number 的属性一律渲染成文本输入框——
那样向导会写出一个字符串,而新门禁恰好拒绝它,等于造出一个存不进去的表单。所以 config.ssl
收敛成开关简写,证书走数据源级 ssl 块(也就是这次刚接上的那个)。

顺带修的:docs 生成器只扫一层

它把新 schema 归到 misc 页,而那页的 "Source" 行指向 data/misc.zod.ts——一个不存在的
文件。这和严格性账本自己的覆盖 gate 曾经的 bug 是同一个形状。改成递归后,
data/driver/integration/connector/kernel/events/ 都拿到了指向真实文件的单文件页面;
逐个比对过,没有丢任何已文档化的 schema,另有 6 个此前未覆盖的 schema 现在有了页面。这也是
docs 的 diff 比本改动本身该有的大的原因。

迁移

一个此前被静默忽略的 config 现在会带着修正报错。重命名:userusername
connectionString/dsn/uriurlfile/path/databasefilename(sqlite)、
hostnamehostsearchPathschema。搬家:pool 尺寸 → 数据源的 pool 块、
schemaMode → 数据源顶层、readOnlycapabilities.readOnlyssl 的证书形式 → 数据源的
ssl 块。postgres/mysql/mongo 现在必须写明连接目标(database,或带上它的 url)——空
config 的含义本来就是"客户端自己的 localhost 默认值",那是这个缺陷最完整的形态。

完整 FROM → TO 表在 changeset 里,会随 npm 包的 CHANGELOG.md 发给消费者。

验证

已合并 main(7 个新提交,其中数个动了 spec)后重跑:

合并 main 后的 check:docs 抓到一处真实的陈旧:我修过的模块注释没有重新生成到 reference
页面。已修(9ad76d5)——那个 gate 就是干这个的。

留给后续

工厂里还有 4 处未声明的 ?? 容错(file/uri/user/connectionString)没有顺手删:删了
会静默改变已持久化记录的数据位置(存量 config: { file: … } 会落到 :memory:),正确路径是
ADR-0087 转换条目,已开 #4456 跟踪。


🤖 Generated with Claude Code

https://claude.ai/code/session_01WsgTqRF58HsQYKLsrZ5pQY


Generated by Claude Code

claude added 4 commits August 1, 2026 08:06
…s driver's contract (#4410)

`config` was the one authorable slot on a datasource with no gate at all. The
module comment justified the hole by saying "the driver's own `configSchema` is
what validates it". Nothing did: both bundled driver specs set
`configSchema: {}`, no code read the field, and the per-driver zod schemas were
not exported from the package — `data/driver/` was reachable only from its own
tests. So `config: { hostname: 'db.internal' }` (the key is `host`) was accepted
in silence and the datasource connected to localhost while the parse, the save
and the connection probe all reported success. That is #4001's original bug
verbatim, one level down, and #4001's own fix pointed authors straight into it.

The frontend question the issue raised has an answer, and it is not a third
false claim: objectui's DatasourceResourcePage really does render the connection
form from a driver `configSchema` (`GET /api/v1/datasources/drivers`, reading
properties/required/title/format). It reads DRIVER_CATALOG — a SECOND set of
hand-written JSON-Schema literals in service-datasource, never checked against
the spec's zod schemas and never validating anything. One live copy, one dead
copy, no gate between them.

So: `packages/spec/src/data/driver/` becomes the one contract, and three
consumers read it. `DatasourceSchema` parses `config` — and each `readReplicas`
entry — against the schema for the declared driver; `DriverDefinitionSchema
.configSchema` publishes its JSON-Schema projection; the catalog serves that
same projection, so the form offers exactly the fields the validator accepts.
`mysql` and `sqlite` / `sqlite-wasm` had no config shape anywhere, though both
were offered by the form and buildable by the factory.

The wizard is the other authoring door and does not reach DatasourceSchema:
createDatasource writes through `metadata.register`, whose validation is a
structural name/label check. DatasourceAdminService create/update/test now
consults the same registry — testConnection BEFORE probing, or a green
"connection successful" gets reported against localhost.

Enforcing the contract forced honouring it. A gate over `config` means every key
inside it claims to be read, so each was audited against the code that reads it:

  - `datasource.pool` reaches every SQL driver. It was declared, strict,
    carried into the connection spec — then overwritten with a hardcoded
    { min: 0, max: 5 }. Maps onto minPoolSize/maxPoolSize for mongo.
  - `datasource.schemaMode` reaches the driver. It was dropped between the
    record and the spec, so the factory looked for it in two places that could
    never hold it and an `external` database — one ObjectStack must never run
    DDL against — was constructed as `managed`.
  - `datasource.ssl` reaches the SQL clients, certificates and all. It stopped
    at the record, so a TLS block configured nothing: the failure its own schema
    comment warns about.
  - postgres `schema` (knex searchPath), `applicationName`, `statementTimeout`.
  - mongo `password`, `authSource`, `options`. A mongo datasource carrying a
    `config.password` composed its URL with an EMPTY password.

Two memory keys had nothing to wire to — `InMemoryDriverConfig` has no field for
`indexes` or `maxRecordsPerObject`, the driver keeps no indexes and evicts
nothing — so they are removed under ADR-0049 with the rejection carrying why.

`config.ssl` is the boolean shorthand only, deliberately. A `boolean | object`
union is honest about what the client accepts, but the form turns anything that
is not boolean/enum/number into a TEXT INPUT: the wizard would have produced a
string the new gate rejects. Certificates go in the datasource-level block,
which this change makes live.

One table for driver ids, in the spec. The factory kept its own copy, which
meant the id selecting a DRIVER and the id selecting that driver's CONFIG
CONTRACT could disagree — the same silent acceptance, reintroduced as a lookup
miss.

Also fixes the docs generator's one-level-deep source walk, which filed the new
schemas onto a `misc` page whose "Source" line named a file that does not exist
(the identical bug the strictness ledger's own coverage gate had). The recursive
walk gives `data/driver/`, `integration/connector/` and `kernel/events/` real
per-file pages; no documented schema was lost, six more are now covered.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsgTqRF58HsQYKLsrZ5pQY
`config.ssl` narrowed to the on/off shorthand when the connection form turned
out to render a non-boolean/enum/number prop as a TEXT INPUT — a
`boolean | object` union there would have produced a wizard whose every `ssl`
value the new gate rejects. These fixtures still passed the object form, so they
asserted a shape the contract no longer has.

They now assert the prescription instead: the certificate-bearing form is
rejected and named toward the datasource-level `ssl` block, which #4410 wired
through to the client.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsgTqRF58HsQYKLsrZ5pQY
The generated page carried the pre-fix wording of the module comment — the
`check:docs` gate caught it on the post-merge re-run, which is what that gate is
for.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WsgTqRF58HsQYKLsrZ5pQY
@vercel

vercel Bot commented Aug 1, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 1, 2026 8:41am

Request Review

@github-actions github-actions Bot added documentation Improvements or additions to documentation protocol:data tests tooling labels Aug 1, 2026
@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 2 package(s): @objectstack/service-datasource, @objectstack/spec.

107 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/ai/agents.mdx (via @objectstack/spec)
  • content/docs/ai/skills-reference.mdx (via @objectstack/spec)
  • content/docs/ai/skills.mdx (via @objectstack/spec)
  • content/docs/api/client-sdk.mdx (via @objectstack/spec)
  • content/docs/api/environment-routing.mdx (via @objectstack/spec)
  • content/docs/api/error-catalog.mdx (via @objectstack/spec)
  • content/docs/api/error-handling-client.mdx (via @objectstack/spec)
  • content/docs/api/error-handling-server.mdx (via @objectstack/spec)
  • content/docs/api/index.mdx (via @objectstack/spec)
  • content/docs/automation/approvals.mdx (via @objectstack/spec)
  • content/docs/automation/connectors.mdx (via @objectstack/spec)
  • content/docs/automation/flows.mdx (via @objectstack/spec)
  • content/docs/automation/hook-bodies.mdx (via packages/spec)
  • content/docs/automation/hooks.mdx (via @objectstack/spec)
  • content/docs/automation/index.mdx (via @objectstack/spec)
  • content/docs/automation/webhooks.mdx (via @objectstack/spec)
  • content/docs/automation/workflows.mdx (via @objectstack/spec)
  • content/docs/concepts/architecture.mdx (via @objectstack/spec)
  • content/docs/concepts/design-principles.mdx (via packages/spec)
  • content/docs/concepts/index.mdx (via @objectstack/spec)
  • content/docs/concepts/metadata-driven.mdx (via @objectstack/spec)
  • content/docs/concepts/metadata-lifecycle.mdx (via packages/spec)
  • content/docs/concepts/north-star.mdx (via packages/spec)
  • content/docs/data-modeling/analytics.mdx (via @objectstack/spec)
  • content/docs/data-modeling/drivers.mdx (via @objectstack/spec)
  • content/docs/data-modeling/external-datasources.mdx (via @objectstack/spec)
  • content/docs/data-modeling/field-types.mdx (via @objectstack/spec)
  • content/docs/data-modeling/fields.mdx (via @objectstack/spec)
  • content/docs/data-modeling/formulas.mdx (via @objectstack/spec)
  • content/docs/data-modeling/index.mdx (via @objectstack/spec)
  • content/docs/data-modeling/objects.mdx (via @objectstack/spec)
  • content/docs/data-modeling/queries.mdx (via @objectstack/spec)
  • content/docs/data-modeling/schema-design.mdx (via @objectstack/spec)
  • content/docs/data-modeling/seed-data.mdx (via @objectstack/spec)
  • content/docs/data-modeling/validation-rules.mdx (via @objectstack/spec)
  • content/docs/data-modeling/validation.mdx (via @objectstack/spec)
  • content/docs/deployment/cli.mdx (via @objectstack/spec)
  • content/docs/deployment/troubleshooting.mdx (via @objectstack/spec)
  • content/docs/deployment/validating-metadata.mdx (via @objectstack/spec)
  • content/docs/getting-started/build-with-claude-code.mdx (via @objectstack/spec)
  • content/docs/getting-started/common-patterns.mdx (via @objectstack/spec)
  • content/docs/getting-started/examples.mdx (via @objectstack/spec)
  • content/docs/getting-started/quick-reference.mdx (via @objectstack/spec)
  • content/docs/getting-started/quick-start.mdx (via @objectstack/spec)
  • content/docs/getting-started/your-first-project.mdx (via @objectstack/spec)
  • content/docs/kernel/cluster.mdx (via @objectstack/spec)
  • content/docs/kernel/contracts/auth-service.mdx (via packages/spec)
  • content/docs/kernel/contracts/cache-service.mdx (via packages/spec)
  • content/docs/kernel/contracts/data-engine.mdx (via @objectstack/spec)
  • content/docs/kernel/contracts/index.mdx (via @objectstack/spec)
  • content/docs/kernel/contracts/metadata-service.mdx (via packages/spec)
  • content/docs/kernel/contracts/storage-service.mdx (via packages/spec)
  • content/docs/kernel/index.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/email-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/index.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/queue-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/sharing-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/sms-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/storage-service.mdx (via packages/spec)
  • content/docs/kernel/services-checklist.mdx (via @objectstack/spec)
  • content/docs/kernel/services.mdx (via @objectstack/spec)
  • content/docs/permissions/authorization.mdx (via @objectstack/spec)
  • content/docs/permissions/permission-sets.mdx (via @objectstack/spec)
  • content/docs/permissions/permissions-matrix.mdx (via @objectstack/spec)
  • content/docs/permissions/positions.mdx (via @objectstack/spec)
  • content/docs/permissions/rls.mdx (via @objectstack/spec)
  • content/docs/permissions/sharing-rules.mdx (via @objectstack/spec)
  • content/docs/plugins/adding-a-metadata-type.mdx (via @objectstack/spec)
  • content/docs/plugins/development.mdx (via @objectstack/spec)
  • content/docs/plugins/index.mdx (via @objectstack/spec)
  • content/docs/plugins/packages.mdx (via @objectstack/spec)
  • content/docs/protocol/backward-compatibility.mdx (via @objectstack/spec)
  • content/docs/protocol/diagram.mdx (via packages/spec)
  • content/docs/protocol/kernel/config-resolution.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/i18n-standard.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/index.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/lifecycle.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/plugin-spec.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/runtime-capabilities.mdx (via @objectstack/spec)
  • content/docs/protocol/knowledge.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/index.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/query-syntax.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/schema.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/security.mdx (via packages/spec)
  • content/docs/protocol/objectql/state-machine.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/actions.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/concept.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/index.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/layout-dsl.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/record-alert.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/widget-contract.mdx (via @objectstack/spec)
  • content/docs/releases/implementation-status.mdx (via @objectstack/spec)
  • content/docs/releases/index.mdx (via @objectstack/spec)
  • content/docs/releases/v12.mdx (via @objectstack/spec)
  • content/docs/releases/v13.mdx (via @objectstack/spec)
  • content/docs/releases/v16.mdx (via @objectstack/spec)
  • content/docs/releases/v17.mdx (via @objectstack/spec)
  • content/docs/releases/v9.mdx (via @objectstack/spec)
  • content/docs/ui/actions.mdx (via @objectstack/spec)
  • content/docs/ui/create-vs-edit-form.mdx (via @objectstack/spec)
  • content/docs/ui/dashboards.mdx (via @objectstack/spec)
  • content/docs/ui/forms.mdx (via @objectstack/spec)
  • content/docs/ui/index.mdx (via @objectstack/spec)
  • content/docs/ui/public-data-collection.mdx (via @objectstack/spec)
  • content/docs/ui/setup-app.mdx (via @objectstack/spec)
  • content/docs/ui/translations.mdx (via @objectstack/spec)
  • content/docs/ui/views.mdx (via @objectstack/spec)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@os-zhuang
os-zhuang merged commit cdf4d9a into main Aug 1, 2026
18 checks passed
@os-zhuang
os-zhuang deleted the claude/datasource-config-validation-i7wdmo branch August 1, 2026 08:54
os-zhuang added a commit that referenced this pull request Aug 1, 2026
…ype (#4487) (#4489)

`GOVERNED` was a hand-maintained list and nothing compared it against the
registry it claims to cover. It governed 15 of 25 registered metadata types
while reporting itself complete: a type in the other ten was authorable —
served by /api/v1/meta/types/:type, editable in Studio — and was never asked
who reads its properties, so an inert key on it was invisible to CI and its
silence read as success.

`datasource` was in that state for its entire life. #4410, #4465 and #4481
found six inert keys on it by hand, two security-shaped: `schemaMode` was
dropped between record and connection spec, so a database ObjectStack must
never run DDL against was constructed as `managed`; `ssl` stopped at the
record, so a TLS block with a CA certificate configured nothing while looking
identical to one that worked.

The gate is now answerable to the registry. Every registered type must be in
GOVERNED or in PENDING_GOVERNANCE with a reason and an issue; registering a
type and forgetting the ledger fails CI with the entry to write. The reverse
rots too and also fails — a PENDING_GOVERNANCE row for a type since governed
claims a debt that no longer exists.

`datasource` is now governed: 43 properties classified with evidence, and the
result is the highest dead ratio of any governed type — 20 of 43 have no
runtime consumer. `capabilities.*` (11): the engine gates pushdown on the
runtime driver's own `supports.*` object, a non-overlapping vocabulary.
`healthCheck.*` (3): nothing schedules a datasource probe. `retryPolicy.*`
(4): no connect or query path retries. Plus `external.label` and
`external.requirePermission`.

One correction ships with this, and it is why the audit was worth doing.
`capabilities.readOnly` reads as a safety switch and gates nothing — and two
shipped prescriptions pointed authors at it: the externalSettingsUnknownKeyError
guidance in datasource.zod.ts and the #4465 changeset's relocation table. Both
now name `external.allowWrites: false`, the write gate the engine checks. The
v17 release notes carried a matching false claim about `capabilities` gating
pushdown; corrected here too.

The CLI advisory lint picks the ledger up, so `os compile` warns an author who
sets any of the 20. That needed `datasource` in TYPE_COLLECTIONS: coverage grows
by marking entries authorWarn only WITHIN a type the lint already walks, and a
governed type whose collection is unregistered has a correct ledger that warns
nobody.

Nine types remain ungoverned and are now enumerated rather than implied (#4488).


Claude-Session: https://claude.ai/code/session_01WsgTqRF58HsQYKLsrZ5pQY

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation protocol:data size/xl tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

datasource.config 至今无人校验:驱动 configSchema 是声明但完全惰性的(ADR-0049 enforce-or-remove,#4001 收尾发现)

2 participants