Skip to content

feat(objectql): engine option bags are a closed contract — unknown keys throw instead of silently doing nothing - #4400

Merged
os-zhuang merged 2 commits into
mainfrom
claude/engine-unknown-key-enforcement-b44637
Jul 31, 2026
Merged

feat(objectql): engine option bags are a closed contract — unknown keys throw instead of silently doing nothing#4400
os-zhuang merged 2 commits into
mainfrom
claude/engine-unknown-key-enforcement-b44637

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Closes #4371 — option (2), the complete answer (option (1) landed in #4387). Scoped by a full caller survey of every engine call site in this repo + cloud, plus every key the engine itself reads.

What changes

All six engine methods reject non-null option keys they do not execute, naming the legal set. Retired keys (cursor/distinct) quote their #4286 tombstone. null stays a withdrawal, same as the fold's rule.

Per-method legal keys = the method's Engine*OptionsSchema keys plus the documented extras the survey proved live:

A drift pin asserts each set against its schema's .shape (minus tombstones, plus the extras), so a key added to the spec cannot be silently rejected.

What the survey turned up and this PR also closes

  1. Bag object overrode the resolved object on the AST{ object, ...query } spread order let query.object win, splitting ast.object from the table actually queried (drivers use the first argument; middleware and hooks reading ast.object were lied to). AST now keeps the resolved name; a direct call passing object is rejected; the protocol refuses a POST-body object contradicting the route (400 QUERY_OBJECT_MISMATCH) instead of picking a winner.
  2. findData leaked protocol vocabulary onto the engine bag (object, count, joins, windowFunctions, cursor, distinct, non-aggregate having) — now stripped at the boundary; wire behaviour unchanged.
  3. Nested expand ASTs (expand: { rel: { sort } }) reject the four wire-only spellings exactly like the top-level bag did in fix(objectql): reject wire-only alias spellings (sort/select/skip/populate) on direct engine calls #4387.
  4. $search/$searchFields engine reads removed — the protocol normalizes to bare keys; a direct call passing the OData spelling now throws instead of half-working on find only.
  5. DbQueueAdapter.purge/purgeFailed were broken on the real engine: both passed { id }, a key the engine never read — purge deleted nothing (every delete threw "Delete requires an ID" into a warn-level catch) and purgeFailed always threw. The test fake's delete implemented the { id } signature the real engine rejects, which is exactly how this stayed green. Both fixed to { where: { id } }; the fake now matches the real contract.
  6. 19 test bags carrying the dead filters key cleaned up — two were intent-carrying (a locale multi-update and an oidc dogfood assert that both silently scanned unfiltered).
  7. driver.find JSDoc example showed wire spellings a driver can never receive — now canonical.

Tests

  • 21 pins in engine-unknown-option.test.ts (per-method rejection, tombstones, null withdrawal, extras legal where they work and rejected where they don't, object rejection, nested-expand rejection + canonical expand works, drift pin).
  • 3 pins in protocol-data.test.ts (protocol strips its keys; matching body object tolerated; contradicting body object 400s before the engine).
  • Full local runs: objectql 1486, runtime 1001, rest 539, security 677, auth 579, automation 538, cli 629, settings 196, client 204, datasource 180, approvals 330, sharing 226, metadata-protocol 136, queue 20 — all green; spec generated-artifact gates green (docs regenerated); typecheck green.

Wire/HTTP callers are unaffected. Direct engine callers passing junk keys get an error that names the fix.

🤖 Generated with Claude Code

os-zhuang and others added 2 commits July 31, 2026 18:44
…re spellings

The @example taught filters/sort/top — keys a driver can never receive:
the engine folds/rejects wire spellings before this layer (#4346, #4371),
and QuerySchema declares where/orderBy/limit. Also dedupes the doubled
@returns tag. Comment-only; no generated artifact reads @example (all 8
gates verified up to date).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…gs are a closed contract — unknown keys throw (#4371 option 2)

All six engine methods now reject non-null option keys they do not execute,
naming the legal set per method; cursor/distinct quote their #4286 tombstone;
null stays a withdrawal. Legal sets = schema keys + documented extras
(searchFields — now DECLARED on EngineQueryOptionsSchema, it was read and
sent all along; onFieldsDropped on update; driver pass-through keys on the
methods whose bag reaches driver options — count/aggregate forward nothing,
so they reject those too). A drift pin holds the sets equal to the schemas.

Same sweep also closes what the option-2 caller survey turned up:
- bag object no longer overrides the resolved AST object ({...query, object}
  order fix); protocol refuses a contradicting POST-body object (400
  QUERY_OBJECT_MISMATCH) and strips its own vocabulary (object/count/
  tombstones/non-aggregate having) off the engine bag
- nested expand ASTs reject the four wire-only spellings like the top level
- $search/$searchFields engine reads removed (protocol normalizes to bare)
- DbQueueAdapter purge/purgeFailed passed {id} — purge deleted NOTHING
  (throw swallowed into warn), purgeFailed always threw; both now pass
  {where: {id}}, and the test fake stops accepting the signature the real
  engine rejects
- 19 test bags carrying the dead 'filters' key cleaned up (two were
  intent-carrying: a locale multi-update and an oidc dogfood assert that
  both silently scanned unfiltered)
- driver.find JSDoc example shows canonical QueryAST keys

Closes #4371 (option 1 landed in #4387).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@vercel

vercel Bot commented Jul 31, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Jul 31, 2026 11:19am

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 4 package(s): @objectstack/metadata-protocol, @objectstack/objectql, @objectstack/service-queue, @objectstack/spec.

110 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/ai/agents.mdx (via @objectstack/spec)
  • content/docs/ai/skills-reference.mdx (via @objectstack/spec)
  • content/docs/ai/skills.mdx (via @objectstack/spec)
  • content/docs/api/client-sdk.mdx (via @objectstack/spec)
  • content/docs/api/environment-routing.mdx (via @objectstack/spec)
  • content/docs/api/error-catalog.mdx (via @objectstack/spec)
  • content/docs/api/error-handling-client.mdx (via @objectstack/spec)
  • content/docs/api/error-handling-server.mdx (via @objectstack/spec)
  • content/docs/api/index.mdx (via @objectstack/spec)
  • content/docs/automation/approvals.mdx (via @objectstack/spec)
  • content/docs/automation/connectors.mdx (via @objectstack/spec)
  • content/docs/automation/flows.mdx (via @objectstack/spec)
  • content/docs/automation/hook-bodies.mdx (via packages/spec)
  • content/docs/automation/hooks.mdx (via @objectstack/spec)
  • content/docs/automation/index.mdx (via @objectstack/spec)
  • content/docs/automation/webhooks.mdx (via @objectstack/spec)
  • content/docs/automation/workflows.mdx (via @objectstack/spec)
  • content/docs/concepts/architecture.mdx (via @objectstack/spec)
  • content/docs/concepts/design-principles.mdx (via packages/spec)
  • content/docs/concepts/index.mdx (via @objectstack/spec)
  • content/docs/concepts/metadata-driven.mdx (via @objectstack/spec)
  • content/docs/concepts/metadata-lifecycle.mdx (via @objectstack/metadata-protocol, @objectstack/objectql, packages/spec)
  • content/docs/concepts/north-star.mdx (via packages/spec)
  • content/docs/data-modeling/analytics.mdx (via @objectstack/spec)
  • content/docs/data-modeling/drivers.mdx (via @objectstack/spec)
  • content/docs/data-modeling/external-datasources.mdx (via @objectstack/spec)
  • content/docs/data-modeling/field-types.mdx (via @objectstack/spec)
  • content/docs/data-modeling/fields.mdx (via @objectstack/spec)
  • content/docs/data-modeling/formulas.mdx (via packages/objectql, @objectstack/spec)
  • content/docs/data-modeling/index.mdx (via @objectstack/spec)
  • content/docs/data-modeling/objects.mdx (via @objectstack/spec)
  • content/docs/data-modeling/queries.mdx (via @objectstack/spec)
  • content/docs/data-modeling/schema-design.mdx (via @objectstack/spec)
  • content/docs/data-modeling/seed-data.mdx (via @objectstack/spec)
  • content/docs/data-modeling/validation-rules.mdx (via @objectstack/spec)
  • content/docs/data-modeling/validation.mdx (via @objectstack/spec)
  • content/docs/deployment/cli.mdx (via @objectstack/spec)
  • content/docs/deployment/migration-from-objectql.mdx (via @objectstack/objectql)
  • content/docs/deployment/troubleshooting.mdx (via @objectstack/spec)
  • content/docs/deployment/validating-metadata.mdx (via @objectstack/spec)
  • content/docs/deployment/vercel.mdx (via @objectstack/objectql)
  • content/docs/getting-started/build-with-claude-code.mdx (via @objectstack/spec)
  • content/docs/getting-started/common-patterns.mdx (via @objectstack/spec)
  • content/docs/getting-started/examples.mdx (via @objectstack/spec)
  • content/docs/getting-started/quick-reference.mdx (via @objectstack/spec)
  • content/docs/getting-started/quick-start.mdx (via @objectstack/spec)
  • content/docs/getting-started/your-first-project.mdx (via @objectstack/spec)
  • content/docs/kernel/cluster.mdx (via @objectstack/spec)
  • content/docs/kernel/contracts/auth-service.mdx (via packages/spec)
  • content/docs/kernel/contracts/cache-service.mdx (via packages/spec)
  • content/docs/kernel/contracts/data-engine.mdx (via @objectstack/spec)
  • content/docs/kernel/contracts/index.mdx (via @objectstack/spec)
  • content/docs/kernel/contracts/metadata-service.mdx (via packages/spec)
  • content/docs/kernel/contracts/storage-service.mdx (via packages/spec)
  • content/docs/kernel/index.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/email-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/index.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/queue-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/sharing-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/sms-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/storage-service.mdx (via packages/spec)
  • content/docs/kernel/services-checklist.mdx (via @objectstack/metadata-protocol, @objectstack/objectql, @objectstack/service-queue, @objectstack/spec)
  • content/docs/kernel/services.mdx (via @objectstack/objectql, @objectstack/spec)
  • content/docs/permissions/authentication.mdx (via @objectstack/objectql)
  • content/docs/permissions/authorization.mdx (via @objectstack/spec)
  • content/docs/permissions/permission-sets.mdx (via @objectstack/spec)
  • content/docs/permissions/permissions-matrix.mdx (via @objectstack/spec)
  • content/docs/permissions/positions.mdx (via @objectstack/spec)
  • content/docs/permissions/rls.mdx (via @objectstack/spec)
  • content/docs/permissions/sharing-rules.mdx (via @objectstack/spec)
  • content/docs/plugins/adding-a-metadata-type.mdx (via @objectstack/spec)
  • content/docs/plugins/development.mdx (via @objectstack/spec)
  • content/docs/plugins/index.mdx (via @objectstack/objectql, @objectstack/spec)
  • content/docs/plugins/packages.mdx (via @objectstack/objectql, @objectstack/service-queue, @objectstack/spec)
  • content/docs/protocol/backward-compatibility.mdx (via @objectstack/spec)
  • content/docs/protocol/diagram.mdx (via packages/spec)
  • content/docs/protocol/kernel/config-resolution.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/i18n-standard.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/index.mdx (via @objectstack/objectql, @objectstack/spec)
  • content/docs/protocol/kernel/lifecycle.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/plugin-spec.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/runtime-capabilities.mdx (via @objectstack/spec)
  • content/docs/protocol/knowledge.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/index.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/query-syntax.mdx (via packages/objectql, @objectstack/spec)
  • content/docs/protocol/objectql/schema.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/security.mdx (via packages/spec)
  • content/docs/protocol/objectql/state-machine.mdx (via @objectstack/objectql, @objectstack/spec)
  • content/docs/protocol/objectui/actions.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/concept.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/index.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/layout-dsl.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/record-alert.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/widget-contract.mdx (via @objectstack/spec)
  • content/docs/releases/implementation-status.mdx (via @objectstack/objectql, @objectstack/service-queue, @objectstack/spec)
  • content/docs/releases/index.mdx (via @objectstack/spec)
  • content/docs/releases/v12.mdx (via @objectstack/spec)
  • content/docs/releases/v13.mdx (via @objectstack/spec)
  • content/docs/releases/v16.mdx (via @objectstack/spec)
  • content/docs/releases/v17.mdx (via @objectstack/spec)
  • content/docs/releases/v9.mdx (via @objectstack/metadata-protocol, @objectstack/spec)
  • content/docs/ui/actions.mdx (via @objectstack/spec)
  • content/docs/ui/create-vs-edit-form.mdx (via @objectstack/spec)
  • content/docs/ui/dashboards.mdx (via @objectstack/spec)
  • content/docs/ui/forms.mdx (via @objectstack/spec)
  • content/docs/ui/index.mdx (via @objectstack/spec)
  • content/docs/ui/public-data-collection.mdx (via @objectstack/spec)
  • content/docs/ui/setup-app.mdx (via @objectstack/spec)
  • content/docs/ui/translations.mdx (via @objectstack/spec)
  • content/docs/ui/views.mdx (via @objectstack/spec)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@github-actions github-actions Bot added documentation Improvements or additions to documentation protocol:data tests tooling size/l labels Jul 31, 2026
@os-zhuang
os-zhuang merged commit 58a03d2 into main Jul 31, 2026
18 checks passed
@os-zhuang
os-zhuang deleted the claude/engine-unknown-key-enforcement-b44637 branch July 31, 2026 11:31
os-zhuang added a commit that referenced this pull request Aug 1, 2026
…, and executes every option it declares (#4419) (#4459)

findOne reads a single row, which makes its predicate the only thing between
the caller and an arbitrary record. When the predicate is missing the result is
not `null` — it is the object's FIRST ROW: a real, plausible-looking record
with nothing to do with the request, which the `if (!row)` check every call
site already has cannot catch, and which then propagates into whatever is
computed next.

#4419 reported this against the `filter` key, which #4346 (fold on every entry
point) and #4400 (unknown keys throw) already closed. This is what those left
standing.

BREAKING: findOne refuses a query that selects nothing in particular.

  findOne(o) / findOne(o, {}) / findOne(o, { where: {} })
    -> findOne(o, { where: … })                     the record matching this
    -> findOne(o, { search: 'Acme' })                the record this search finds
    -> findOne(o, { orderBy: [{ field, order }] })   the FIRST in this order
    -> find(o, { limit: 1 })                         any row will do, said aloud

The error names all four. `find` and `count` are unchanged — returning or
counting every row is an honest answer. The guard reads the CALLER's predicate,
before RLS/sharing middleware injects its own.

Two silent drops that produced the same wrong record are fixed with it:

- findOne({ search }) now applies the search. The ADR-0061 expansion lived in
  find() alone while both methods are checked against the SAME legal-key set,
  so `search` passed the gate, reached a driver that does not read it, and the
  read ran unpredicated.
- MongoDBDriver.findOne now applies orderBy, fields and offset. It translated
  `where` and dropped the rest, so "the newest record" returned whichever
  document the scan reached first. No ordering is imposed when the caller
  supplies none (#4363) — unchanged on both drivers.

And a drift pin walks ENGINE_OPTION_KEY_SETS.findOne requiring each declared
key to have an observable effect, so the next declared-but-unexecuted key fails
CI instead of shipping. The Mongo cases live in one shared table read by both a
server-free options suite and the real-mongod suite, so neither half can drift.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation protocol:data size/l tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P2] A direct engine call silently drops sort/select/skip/populate — declared query contract, zero enforcement

1 participant