Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .changeset/protection-envelope-invariant-was-hollow.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
'@objectstack/spec': patch
---

The protection-envelope invariant test was hollow — it silently skipped 24 of 25 registered types. Fixed, and it immediately found 8 undeclared envelopes instead of 1.

The check shipped in the previous change asserted two things about every registered metadata type: that it does not *reject* the ADR-0010 envelope its loader stamps (the hard-422 case), and that it does not *strip* it (the silent-loss case). The reject half worked — it found `hook` and `datasource` on its first run.

The strip half did not. It probed each schema with one generic body and asked whether `_packageId` survived; a type whose required fields that body did not satisfy failed for unrelated reasons and the assertion returned early. **24 of the 25 types took that early return.** Only `field` was ever actually checked, and the suite reported green.

That is the campaign's own subject matter — a success signal covering an omission — reproduced inside the instrument built to detect it, one change after the ledger recorded the same lesson about the strictness gate's non-recursive directory walk. A check that skips is indistinguishable from a check that passes.

**The declaration side is now structural.** It walks the schema — unwrapping `lazy` / `pipe` / `optional` / `default` and expanding unions — and asks whether any resolved object shape declares the key. That answer does not require constructing a valid instance, so it cannot skip. Two guards keep it honest: a type whose shape the walker cannot resolve is a hard failure (the walker going quiet is exactly when this test would otherwise stop covering something), and the debt list carries a reverse pin that fails when an entry is fixed, so the list cannot outlive the debt it tracks.

**What it found:** 8 registered types do not declare the envelope, not 1 — `action`, `book`, `field`, `job`, `mapping`, `page`, `translation`, `validation`. `job` and `book` are closed here, leaving 6 on the list. Each is protection metadata lost on every round-trip today, and a hard 422 the day its schema is closed.
7 changes: 7 additions & 0 deletions content/docs/references/system/book.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,13 @@ const result = Book.parse(data);
| **order** | `number` | optional | Orders books within the portal |
| **audience** | `'org' \| 'public' \| { permissionSet: string }` | optional | Access audience; defaults to 'org' (inherits package grant) |
| **groups** | `{ key: string; label: string; translations?: Record<string, { label: string }>; order?: number; … }[]` | ✅ | The spine: ordered sections. Two levels total. |
| **_lock** | `Enum<'none' \| 'no-overlay' \| 'no-delete' \| 'full'>` | optional | Item-level lock — controls overlay & delete (ADR-0010). |
| **_lockReason** | `string` | optional | Human-readable reason shown when a write is refused by _lock. |
| **_lockSource** | `Enum<'artifact' \| 'package' \| 'env-forced'>` | optional | Layer that set _lock (artifact \| package \| env-forced). |
| **_provenance** | `Enum<'package' \| 'org' \| 'env-forced'>` | optional | Origin of the item (package \| org \| env-forced). |
| **_packageId** | `string` | optional | Owning package machine id. |
| **_packageVersion** | `string` | optional | Owning package version. |
| **_lockDocsUrl** | `string` | optional | Optional documentation link surfaced next to _lockReason. |


---
Expand Down
7 changes: 7 additions & 0 deletions content/docs/references/system/job.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,13 @@ const result = CronSchedule.parse(data);
| **retryPolicy** | `{ maxRetries?: integer; backoffMs?: integer; backoffMultiplier?: number }` | optional | Retry policy: failed runs (including timeouts) are retried with exponential backoff (delay = backoffMs * backoffMultiplier^(retry-1)) up to maxRetries retries after the initial attempt (#3494). Omit for the legacy single-attempt behavior. |
| **timeout** | `integer` | optional | Per-attempt time limit in milliseconds; an over-limit run is recorded with execution status "timeout" (#3494). The in-flight handler is abandoned, not forcibly cancelled. Omit for no time limit. |
| **enabled** | `boolean` | optional | Whether the job is enabled |
| **_lock** | `Enum<'none' \| 'no-overlay' \| 'no-delete' \| 'full'>` | optional | Item-level lock — controls overlay & delete (ADR-0010). |
| **_lockReason** | `string` | optional | Human-readable reason shown when a write is refused by _lock. |
| **_lockSource** | `Enum<'artifact' \| 'package' \| 'env-forced'>` | optional | Layer that set _lock (artifact \| package \| env-forced). |
| **_provenance** | `Enum<'package' \| 'org' \| 'env-forced'>` | optional | Origin of the item (package \| org \| env-forced). |
| **_packageId** | `string` | optional | Owning package machine id. |
| **_packageVersion** | `string` | optional | Owning package version. |
| **_lockDocsUrl** | `string` | optional | Optional documentation link surfaced next to _lockReason. |


---
Expand Down
31 changes: 28 additions & 3 deletions docs/audits/2026-07-unknown-key-strictness-ledger.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,9 +169,34 @@ dropped at parse, and nothing failed.

The check separates the two severities, because they are not the same bug:
*rejecting* the envelope is live breakage and is asserted unconditionally with
no exemption list; *stripping* it silently loses protection metadata on
round-trip and is tracked with a debt list (`field` only) — and each entry
there becomes a rejection the day its schema is closed.
no exemption list; *not declaring* it silently loses protection metadata on
round-trip and is tracked with a debt list — and each entry there becomes a
rejection the day its schema is closed.
9. **And then that check turned out to be hollow — one change after this file
recorded the same lesson about the gate above.** Its declaration half probed
each schema with one generic body and asked whether `_packageId` survived. A
type whose required fields that body did not satisfy failed for unrelated
reasons and the assertion returned early, so **24 of 25 registered types took
that early return**. Only `field` was ever really checked, and the suite
reported green.

Rewritten to walk the schema *structurally* — unwrapping `lazy` / `pipe` /
`optional` / `default`, expanding unions — which needs no valid instance and
therefore cannot skip. Two guards keep it honest: a type the walker cannot
resolve is a hard failure (the walker going quiet is precisely when the test
would otherwise stop covering something), and the debt list carries a reverse
pin that fails when an entry is fixed, so the list cannot outlive its debt.

It then found **8** undeclared envelopes rather than 1 — `action`, `book`,
`field`, `job`, `mapping`, `page`, `translation`, `validation`. `job` and
`book` were closed immediately; 6 remain.

Three occurrences now of one pattern, in three different instruments: the
ledger gate's non-recursive directory walk, the strip probe's early return,
and (from the other direction) `strictObject(` not matching the site count.
Each was a measuring tool reporting completeness it did not have. **The rule
this file keeps re-deriving: before trusting a green check, make it go red on
something you know is there.**

This is the empirical argument for the ratchet: the inference "no metadata in
the repo carries unknown keys" was **false three times over**, and only the
Expand Down
14 changes: 14 additions & 0 deletions packages/spec/authorable-surface.json
Original file line number Diff line number Diff line change
Expand Up @@ -5682,6 +5682,13 @@
"system/BatchProgress:status",
"system/BatchProgress:succeeded",
"system/BatchProgress:total",
"system/Book:_lock",
"system/Book:_lockDocsUrl",
"system/Book:_lockReason",
"system/Book:_lockSource",
"system/Book:_packageId",
"system/Book:_packageVersion",
"system/Book:_provenance",
"system/Book:audience",
"system/Book:description",
"system/Book:groups",
Expand Down Expand Up @@ -6136,6 +6143,13 @@
"system/IncidentResponsePolicy:triageDeadlineHours",
"system/IntervalSchedule:intervalMs",
"system/IntervalSchedule:type",
"system/Job:_lock",
"system/Job:_lockDocsUrl",
"system/Job:_lockReason",
"system/Job:_lockSource",
"system/Job:_packageId",
"system/Job:_packageVersion",
"system/Job:_provenance",
"system/Job:description",
"system/Job:enabled",
"system/Job:handler",
Expand Down
182 changes: 129 additions & 53 deletions packages/spec/src/kernel/metadata-type-schemas.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,30 +9,40 @@
* type, and `getMetaItemLayered` → `saveMetaItem` round-trips a body carrying
* the stamped `_packageId` / `_provenance`. A type whose schema does not declare
* {@link MetadataProtectionFields} therefore mishandles it in one of two ways,
* and the severities are different enough to assert separately:
* and the severities differ enough to assert separately:
*
* - **Rejects it** (the schema is `.strict()`): a hard 422 on the overlay path.
* Live breakage. Asserted unconditionally below — no debt list.
* - **Strips it** (the schema is strip-mode): the envelope is silently dropped
* on every parse, so protection metadata is lost on round-trip. Quieter, and
* it becomes the first case the day that schema is closed.
* Live breakage. Asserted unconditionally — no exemption list.
* - **Does not declare it** (strip mode): the envelope is silently dropped on
* every parse, so protection metadata is lost on round-trip. Quieter, and it
* becomes the first case the day that schema is closed.
*
* ## Why this file exists at all
* ## Why this file exists
*
* The same defect was found four separate times, by four different routes,
* before anyone wrote a check for it:
* before anyone wrote a check for it: `permission` (#4001 Tier-A, as a hard 422
* caught by the dogfood gate), `position` (step 2, by reading), `seed` + `doc`
* (the registered-types batch, while converting), and then `hook` +
* `datasource` — which THIS test found on its first run, both already strict on
* `main` and therefore both in the 422 class.
*
* 1. `permission` (#4001 Tier-A) — surfaced as a hard 422 on the ADR-0094
* overlay path, caught by the dogfood gate.
* 2. `position` (#4001 step 2) — found by reading, as the "known sibling gap".
* 3. `seed` + `doc` (#4001 registered-types batch) — found while converting.
* 4. `hook` + `datasource` — found by THIS test, on its first run. Both had
* gone `.strict()` in the #4001 data step without declaring the envelope,
* so both were in the hard-422 class on `main` at the time.
* ## Why the declaration check is structural, not a parse probe
*
* Finding one defect four times by hand is the signal that the check is
* missing, not that the search worked. Case 4 is the argument in miniature: two
* live bugs that three prior hand-searches had walked past.
* The first version of this file probed with one generic body and asked whether
* `_packageId` survived. It reported green. It was hollow: a type whose required
* fields the generic body did not satisfy failed for unrelated reasons, and the
* assertion returned early — **so 24 of 25 types were silently skipped and only
* `field` was ever really checked.** A check that skips is indistinguishable
* from a check that passes, which is the exact defect this whole campaign is
* about, reproduced in the instrument built to detect it.
*
* So the declaration side now walks the schema structurally — unwrapping
* `lazy` / `pipe` / `optional` / `default` and expanding unions — and asks
* whether any resolved object shape declares the key. That answer does not
* depend on constructing a valid instance, so it cannot skip. And a type whose
* shape cannot be resolved at all is a hard FAILURE rather than a pass: the
* walker not understanding a schema is exactly when this test would otherwise
* go quiet.
*/

import { describe, expect, it } from 'vitest';
Expand All @@ -42,12 +52,7 @@ import { listMetadataTypeSchemaTypes, getMetadataTypeSchema } from './metadata-t
/** The ADR-0010 stamp the loader puts on every registered item. */
const STAMP = { _packageId: 'pkg_probe', _provenance: 'package' as const };

/**
* A body that satisfies the required fields of the registered types generously
* enough to reach the unknown-key check. Types it does not fully satisfy fail
* on other grounds, which the assertions below distinguish and ignore — this
* test is only about how the `_`-prefixed envelope is treated.
*/
/** A body generous enough to reach the unknown-key check on most types. */
const PROBE: Record<string, unknown> = {
name: 'probe_item',
label: 'Probe',
Expand All @@ -58,6 +63,66 @@ const PROBE: Record<string, unknown> = {
...STAMP,
};

/**
* Registered types that parse the envelope but do not declare it, so it is
* dropped on every round-trip. Every entry is a bug awaiting a
* `...MetadataProtectionFields` spread — not a permanent exemption — and each
* becomes a hard 422 the day its schema is closed. Empty this list; never grow
* it. A NEW registered type belongs in neither list.
*
* The structural walk found 8 of these; the probe it replaced had been hiding 7.
* `job` and `book` were closed in the same pass, leaving 6.
*/
const UNDECLARED_ENVELOPE = new Set<string>([
'action', 'field', 'mapping', 'page', 'translation', 'validation',
]);

/**
* Every object shape reachable from `schema`, unwrapping the wrappers the
* registered types actually use and expanding unions. Returns `[]` only when
* the walker does not understand the schema — which the caller treats as a
* failure, never as a pass.
*/
function objectShapes(schema: unknown, depth = 0): Record<string, unknown>[] {
if (!schema || depth > 12) return [];
const s = schema as { shape?: Record<string, unknown>; _zod?: { def?: Def }; def?: Def };
const def = s._zod?.def ?? s.def;
switch (def?.type) {
case 'object':
return [s.shape ?? def.shape ?? {}];
case 'lazy':
try {
return objectShapes(def.getter?.(), depth + 1);
} catch {
return [];
}
case 'pipe':
return [...objectShapes(def.in, depth + 1), ...objectShapes(def.out, depth + 1)];
case 'union':
return (def.options ?? []).flatMap((o) => objectShapes(o, depth + 1));
case 'optional':
case 'nullable':
case 'default':
case 'prefault':
case 'readonly':
case 'nonoptional':
case 'catch':
return objectShapes(def.innerType, depth + 1);
default:
return [];
}
}

interface Def {
type?: string;
shape?: Record<string, unknown>;
getter?: () => unknown;
in?: unknown;
out?: unknown;
options?: unknown[];
innerType?: unknown;
}

/** `_`-prefixed keys the schema reported as unrecognized, if any. */
function rejectedEnvelopeKeys(type: string): string[] {
const result = getMetadataTypeSchema(type)!.safeParse(PROBE);
Expand All @@ -68,27 +133,33 @@ function rejectedEnvelopeKeys(type: string): string[] {
.filter((k) => k.startsWith('_'));
}

/**
* Types that parse the envelope but drop it. Every entry is a bug awaiting a
* `...MetadataProtectionFields` spread, not a permanent exemption — and each
* becomes a hard 422 the day its schema is closed. Empty this list; never grow
* it. A new registered type belongs in neither list.
*/
const STRIPS_ENVELOPE = new Set<string>([
// Registered so a single field can be addressed as a metadata item, but
// authored inside `object.fields`, where the object's own envelope covers the
// package. Closing this one means auditing that nesting, so it is tracked
// rather than bundled into the batch that found it.
'field',
]);

describe('registered metadata types', () => {
const types = listMetadataTypeSchemaTypes();

it('is a non-empty set — guards the derivation returning nothing', () => {
expect(types.length).toBeGreaterThan(15);
});

it('every registered type resolves to a schema', () => {
for (const type of types) {
expect(getMetadataTypeSchema(type), `no schema registered for '${type}'`).toBeDefined();
}
});

/**
* The no-silent-skip guard. If the walker stops understanding a schema shape,
* the declaration assertions below would quietly stop covering that type —
* so that condition fails here first, loudly, with the type named.
*/
it.each(types)('%s resolves to at least one object shape the walker understands', (type) => {
expect(
objectShapes(getMetadataTypeSchema(type)).length,
`the structural walker cannot resolve '${type}' to an object shape, so the `
+ 'envelope assertions below would silently skip it. Teach `objectShapes` the '
+ 'wrapper this schema uses.',
).toBeGreaterThan(0);
});

it.each(types)('%s does not REJECT the protection envelope its loader stamps', (type) => {
expect(
rejectedEnvelopeKeys(type),
Expand All @@ -98,25 +169,30 @@ describe('registered metadata types', () => {
).toEqual([]);
});

it.each(types.filter((t) => !STRIPS_ENVELOPE.has(t)))(
'%s does not STRIP the protection envelope',
it.each(types.filter((t) => !UNDECLARED_ENVELOPE.has(t)))(
'%s DECLARES the protection envelope',
(type) => {
const result = getMetadataTypeSchema(type)!.safeParse(PROBE);
// A probe that fails for unrelated reasons (a required field this generic
// body does not supply) tells us nothing about stripping — and the reject
// case is already covered unconditionally above.
if (!result.success) return;
const shapes = objectShapes(getMetadataTypeSchema(type));
expect(
(result.data as Record<string, unknown>)._packageId,
`'${type}' silently drops \`_packageId\` — protection metadata is lost on `
+ 'every round-trip. Add `...MetadataProtectionFields` to its schema.',
).toBe(STAMP._packageId);
shapes.some((shape) => '_packageId' in shape),
`'${type}' does not declare \`_packageId\`, so the envelope its loader stamps is `
+ 'dropped on every parse. Add `...MetadataProtectionFields` to its schema.',
).toBe(true);
},
);

it('every registered type resolves to a schema', () => {
for (const type of types) {
expect(getMetadataTypeSchema(type), `no schema registered for '${type}'`).toBeDefined();
}
});
it.each([...UNDECLARED_ENVELOPE])(
'%s is still on the undeclared-envelope debt list (remove it once fixed)',
(type) => {
// A reverse pin: when someone fixes one of these, this fails and forces the
// list to shrink. Without it the debt list would outlive the debt and start
// exempting types that no longer need exempting.
expect(types).toContain(type);
const shapes = objectShapes(getMetadataTypeSchema(type));
expect(
shapes.some((shape) => '_packageId' in shape),
`'${type}' now declares the envelope — remove it from UNDECLARED_ENVELOPE.`,
).toBe(false);
},
);
});
8 changes: 8 additions & 0 deletions packages/spec/src/system/book.zod.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

import { z } from 'zod';
import { lazySchema } from '../shared/lazy-schema';
import { MetadataProtectionFields } from '../kernel/metadata-protection.zod';

/**
* Package Documentation Navigation — the `book` element (ADR-0046 §6).
Expand Down Expand Up @@ -115,6 +116,13 @@ export const BookSchema = lazySchema(() =>
order: z.number().optional().describe('Orders books within the portal'),
audience: BookAudienceSchema.optional().describe("Access audience; defaults to 'org' (inherits package grant)"),
groups: z.array(BookGroupSchema).describe('The spine: ordered sections. Two levels total.'),

// ADR-0010 — runtime protection envelope (internal — set by the loader).
// `book` is a registered metadata type, so the artifact loader stamps
// `_packageId` / `_provenance` on it like every sibling. Undeclared, they
// were dropped on every parse — protection metadata lost on round-trip, and
// a hard 422 waiting for the day this shape is closed.
...MetadataProtectionFields,
}),
);

Expand Down
Loading
Loading