Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions .changeset/rate-limit-config-dual-source-c9.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
---
"@objectstack/spec": major
---

BREAKING(spec): `@objectstack/spec/integration` renames `RateLimitConfig` →
`ConnectorRateLimitConfig` (#4684, C9)

Two entry points exported `RateLimitConfig` for **two different declarations**,
so which one you got depended only on the import path — the #4411 trap. They are
not variants of one concept; they describe opposite directions of traffic:

| | `@objectstack/spec/shared` (unchanged) | `@objectstack/spec/integration` (renamed) |
|:--|:--|:--|
| what it limits | **inbound** — calls others make to our API | **outbound** — calls we make to an external system |
| written at | `apis[].rateLimit`, `httpServer.security.rateLimit` | `connectors[].rateLimitConfig` |
| window | `windowMs` (ms), defaults to 60000 | `windowSeconds` (s), **required**, min 1 |
| quota | `maxRequests`, defaults to 100 | `maxRequests`, **required**, min 1 |
| extras | `enabled` (default `false`) | `strategy`, `burstCapacity`, `respectUpstreamLimits`, `rateLimitHeaders` |

Neither schema is `.strict()`, so a snippet copied from one side to the other
parsed **clean** with its foreign keys silently stripped — `RateLimitConfigSchema
.parse({ windowSeconds: 60, strategy: 'token_bucket' })` returned
`{ enabled: false, windowMs: 60000, maxRequests: 100 }` and nothing said a word.
Per ADR-0112 D9(a) — the same ruling that produced `ConnectorErrorCategory` and
`ConnectorRetryStrategy` in the same file — the **connector side is renamed** so
one name means one thing.

## FROM → TO

```ts
// before
import { RateLimitConfigSchema, type RateLimitConfig } from '@objectstack/spec/integration';

// after
import {
ConnectorRateLimitConfigSchema,
type ConnectorRateLimitConfig,
} from '@objectstack/spec/integration';
```

No deprecated alias is kept: re-exporting the old name would be a third
declaration of it and would re-open the trap this change closes.

**Importing from `@objectstack/spec/shared` (or `/api`, `/system`)? Nothing
changes** — that `RateLimitConfig` keeps its name, its keys and its defaults.

## Authored metadata needs no migration

This renames a TypeScript export and an internal JSON Schema `$def`, not an
authorable key. Every one of the six keys an author can write under
`connectors[].rateLimitConfig` — `strategy`, `maxRequests`, `windowSeconds`,
`burstCapacity`, `respectUpstreamLimits`, `rateLimitHeaders` — parses exactly as
before. Existing stack metadata, stored `sys_metadata` rows and published apps
are byte-for-byte unaffected, which is why this change ships with **no ADR-0087
conversion and no tombstone**: nothing was retired.

The only edit an upgrade needs is the import above, in TypeScript that named the
type. The published JSON Schema `$id` moves with it:
`…/integration/RateLimitConfig.json` → `…/integration/ConnectorRateLimitConfig.json`.

## Gate change riding along

`scripts/build-schemas.ts` learns a declarative `RENAMED_DEFS` table
(`scripts/lib/renamed-defs.ts`). Its two ratchets measure in `$def` units, so a
def rename previously read as six authorable keys vanishing at once. The table
carries the old snapshot forward under the new name and enforces the rule a
rename must obey: **every key under the old def must exist under the new one, or
the build fails** — plus the target must be emitted and the source must not (a
def that is still published is a copy, not a rename). This is stricter than the
hand-edited baseline it replaces, which could drop any line without a trace.
20 changes: 18 additions & 2 deletions content/docs/references/integration/connector.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -132,8 +132,8 @@ with simple `auth` — or by `[automation/sync.zod.ts](/docs/references/automati
## TypeScript Usage

```typescript
import { CircuitBreakerConfigSchema, ConnectorSchema, ConnectorActionSchema, ConnectorErrorCategorySchema, ConnectorHealthSchema, ConnectorRetryStrategySchema, ConnectorStatusSchema, ConnectorTriggerSchema, ConnectorTypeSchema, DataSyncConfigSchema, DeclarativeConnectorEntrySchema, ErrorMappingConfigSchema, ErrorMappingRuleSchema, HealthCheckConfigSchema, RateLimitStrategySchema, RetryConfigSchema, SyncStrategySchema, WebhookConfigSchema, WebhookEventSchema, WebhookSignatureAlgorithmSchema } from '@objectstack/spec/integration';
import type { CircuitBreakerConfig, Connector, ConnectorErrorCategory, ConnectorHealth, ConnectorRetryStrategy, ConnectorStatus, ConnectorType, DataSyncConfig, DeclarativeConnectorEntry, ErrorMappingConfig, ErrorMappingRule, HealthCheckConfig, RateLimitStrategy, RetryConfig, SyncStrategy, WebhookConfig, WebhookEvent, WebhookSignatureAlgorithm } from '@objectstack/spec/integration';
import { CircuitBreakerConfigSchema, ConnectorSchema, ConnectorActionSchema, ConnectorErrorCategorySchema, ConnectorHealthSchema, ConnectorRateLimitConfigSchema, ConnectorRetryStrategySchema, ConnectorStatusSchema, ConnectorTriggerSchema, ConnectorTypeSchema, DataSyncConfigSchema, DeclarativeConnectorEntrySchema, ErrorMappingConfigSchema, ErrorMappingRuleSchema, HealthCheckConfigSchema, RateLimitStrategySchema, RetryConfigSchema, SyncStrategySchema, WebhookConfigSchema, WebhookEventSchema, WebhookSignatureAlgorithmSchema } from '@objectstack/spec/integration';
import type { CircuitBreakerConfig, Connector, ConnectorErrorCategory, ConnectorHealth, ConnectorRateLimitConfig, ConnectorRetryStrategy, ConnectorStatus, ConnectorType, DataSyncConfig, DeclarativeConnectorEntry, ErrorMappingConfig, ErrorMappingRule, HealthCheckConfig, RateLimitStrategy, RetryConfig, SyncStrategy, WebhookConfig, WebhookEvent, WebhookSignatureAlgorithm } from '@objectstack/spec/integration';

// Validate data
const result = CircuitBreakerConfigSchema.parse(data);
Expand Down Expand Up @@ -237,6 +237,22 @@ Connector health configuration
| **circuitBreaker** | `{ enabled: boolean; failureThreshold: number; resetTimeoutMs: number; halfOpenMaxRequests: number; … }` | optional | Circuit breaker configuration |


---

## ConnectorRateLimitConfig

### Properties

| Property | Type | Required | Description |
| :--- | :--- | :--- | :--- |
| **strategy** | `Enum<'fixed_window' \| 'sliding_window' \| 'token_bucket' \| 'leaky_bucket'>` | ✅ | Rate limiting strategy |
| **maxRequests** | `number` | ✅ | Maximum requests per window |
| **windowSeconds** | `number` | ✅ | Time window in seconds |
| **burstCapacity** | `number` | optional | Burst capacity |
| **respectUpstreamLimits** | `boolean` | ✅ | Respect external rate limit headers |
| **rateLimitHeaders** | `{ remaining: string; limit: string; reset: string }` | optional | Custom rate limit headers |


---

## ConnectorRetryStrategy
Expand Down
35 changes: 0 additions & 35 deletions content/docs/references/integration/http.mdx

This file was deleted.

1 change: 0 additions & 1 deletion content/docs/references/integration/meta.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@
"connector-auth",
"mapping",
"---Transport & Storage---",
"http",
"offline"
]
}
4 changes: 2 additions & 2 deletions packages/spec/api-surface.json
Original file line number Diff line number Diff line change
Expand Up @@ -3939,6 +3939,8 @@
"ConnectorOrigin (type)",
"ConnectorProviderContext (interface)",
"ConnectorProviderFactory (type)",
"ConnectorRateLimitConfig (type)",
"ConnectorRateLimitConfigSchema (const)",
"ConnectorRetryStrategy (type)",
"ConnectorRetryStrategySchema (const)",
"ConnectorSchema (const)",
Expand All @@ -3961,8 +3963,6 @@
"FieldMappingSchema (const)",
"HealthCheckConfig (type)",
"HealthCheckConfigSchema (const)",
"RateLimitConfig (type)",
"RateLimitConfigSchema (const)",
"RateLimitStrategy (type)",
"RateLimitStrategySchema (const)",
"ResolvedConnectorAuth (type)",
Expand Down
12 changes: 6 additions & 6 deletions packages/spec/authorable-surface.json
Original file line number Diff line number Diff line change
Expand Up @@ -4156,6 +4156,12 @@
"integration/ConnectorInstanceBearerAuth:credentialRef",
"integration/ConnectorInstanceBearerAuth:type",
"integration/ConnectorInstanceNoAuth:type",
"integration/ConnectorRateLimitConfig:burstCapacity",
"integration/ConnectorRateLimitConfig:maxRequests",
"integration/ConnectorRateLimitConfig:rateLimitHeaders",
"integration/ConnectorRateLimitConfig:respectUpstreamLimits",
"integration/ConnectorRateLimitConfig:strategy",
"integration/ConnectorRateLimitConfig:windowSeconds",
"integration/ConnectorTrigger:description",
"integration/ConnectorTrigger:interval",
"integration/ConnectorTrigger:key",
Expand Down Expand Up @@ -4219,12 +4225,6 @@
"integration/HealthCheckConfig:method",
"integration/HealthCheckConfig:timeoutMs",
"integration/HealthCheckConfig:unhealthyThreshold",
"integration/RateLimitConfig:burstCapacity",
"integration/RateLimitConfig:maxRequests",
"integration/RateLimitConfig:rateLimitHeaders",
"integration/RateLimitConfig:respectUpstreamLimits",
"integration/RateLimitConfig:strategy",
"integration/RateLimitConfig:windowSeconds",
"integration/RetryConfig:backoffMultiplier",
"integration/RetryConfig:initialDelayMs",
"integration/RetryConfig:jitter",
Expand Down
2 changes: 0 additions & 2 deletions packages/spec/dual-source-exports.baseline.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,6 @@
"HttpMethod — [./api, ./shared (type)] ≠ [./ui (type)]",
"PackageDependency — [./cloud (type)] ≠ [./kernel (type)]",
"PackageDependencySchema — [./cloud (const)] ≠ [./kernel (const)]",
"RateLimitConfig — [./integration (type)] ≠ [./shared (type)]",
"RateLimitConfigSchema — [./integration (const)] ≠ [./shared (const)]",
"TenantPlan — [./cloud (type)] ≠ [./system (type)]",
"TenantPlanSchema — [./cloud (const)] ≠ [./system (const)]"
]
Expand Down
2 changes: 1 addition & 1 deletion packages/spec/json-schema.manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -879,6 +879,7 @@
"integration/ConnectorInstanceBasicAuth",
"integration/ConnectorInstanceBearerAuth",
"integration/ConnectorInstanceNoAuth",
"integration/ConnectorRateLimitConfig",
"integration/ConnectorRetryStrategy",
"integration/ConnectorStatus",
"integration/ConnectorTrigger",
Expand All @@ -889,7 +890,6 @@
"integration/ErrorMappingRule",
"integration/FieldMapping",
"integration/HealthCheckConfig",
"integration/RateLimitConfig",
"integration/RateLimitStrategy",
"integration/RetryConfig",
"integration/SyncStrategy",
Expand Down
65 changes: 62 additions & 3 deletions packages/spec/scripts/build-schemas.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import fs from 'fs';
import path from 'path';
import { z } from 'zod';
import { schemaNameFromExportKey } from './lib/schema-name';
import { RENAMED_DEFS, carryAuthorableKey, checkRenameTable } from './lib/renamed-defs';
import { CONVERSIONS_BY_MAJOR } from '../src/conversions/registry';
import { MIGRATIONS_BY_MAJOR } from '../src/migrations/registry';
import * as AI from '../src/ai';
Expand Down Expand Up @@ -301,7 +302,24 @@ try {
}

const generatedKeys = new Set(generatedSchemas.keys());
const missing = (manifest?.schemas ?? []).filter((key) => !generatedKeys.has(key));

// ─── Declared def renames must describe THIS build ────────────────────
// Both ratchets below consult RENAMED_DEFS, so an entry that no longer matches
// reality (target never emitted, or source still emitted alongside it) would
// weaken them silently. Fail before either one runs. See lib/renamed-defs.ts.
const renameProblems = checkRenameTable(generatedKeys);
if (renameProblems.length > 0) {
console.error(`\n❌ ${renameProblems.length} problem(s) in RENAMED_DEFS (scripts/lib/renamed-defs.ts):`);
for (const p of renameProblems) console.error(` - ${p}`);
process.exit(1);
}

const missing = (manifest?.schemas ?? []).filter(
// A def listed as renamed is not missing — it is published under the new
// name, which `checkRenameTable` just proved this build emits. The manifest
// rewrite below drops the old key, so the entry self-clears on regeneration.
(key) => !generatedKeys.has(key) && !(key in RENAMED_DEFS),
);
if (missing.length > 0) {
console.error(`\n❌ ${missing.length} previously published schema(s) disappeared from this build:`);
for (const key of missing) {
Expand All @@ -319,7 +337,12 @@ if (missing.length > 0) {
}

const added = [...generatedKeys].filter((key) => !(manifest?.schemas ?? []).includes(key));
if (!manifest || added.length > 0) {
// A renamed-away source key must be dropped from the manifest even in the (rare)
// case where the new name adds nothing — e.g. a rename onto a def that already
// existed. Without this the stale key would sit in the manifest forever, kept
// alive only by its RENAMED_DEFS entry.
const renamedAway = (manifest?.schemas ?? []).filter((key) => key in RENAMED_DEFS);
if (!manifest || added.length > 0 || renamedAway.length > 0) {
const updated: SchemaManifest = {
description:
'Ratchet manifest of every JSON Schema emitted by scripts/build-schemas.ts. ' +
Expand Down Expand Up @@ -401,10 +424,46 @@ if (fs.existsSync(AUTHORABLE_SURFACE_PATH)) {
}

if (surfaceDoc) {
const prev = new Map<string, boolean>(
const snapshot = new Map<string, boolean>(
surfaceDoc.keys.map((e) => [e.replace(RETIRED_MARK, ''), e.endsWith(RETIRED_MARK)]),
);

// Carry the snapshot through any declared def rename FIRST, so every check
// below compares like with like. A rename moves keys between defs; it must
// never be able to drop one, and it must never launder a retirement past
// check (b) either — which is why the carried key keeps the OLD key's
// retired state. See scripts/lib/renamed-defs.ts (#4684).
const prev = new Map<string, boolean>();
const carriedFrom = new Map<string, string>(); // new key -> old key
for (const [key, retired] of snapshot) {
const carried = carryAuthorableKey(key);
if (carried !== key) carriedFrom.set(carried, key);
prev.set(carried, retired);
}

// (a0) A declared rename that did not carry one of its keys. Reported apart
// from (a) because the remedy is the opposite one: the key did not leave
// the contract by accident of a deletion, it failed to arrive under the
// new def — restore it there, or stop calling this a rename.
const notCarried = [...carriedFrom.entries()].filter(([to]) => !currentKeys.has(to));
if (notCarried.length > 0) {
console.error(
`\n❌ ${notCarried.length} authorable key(s) were lost by a declared def rename:`,
);
for (const [to, from] of notCarried) console.error(` - ${from} → ${to} (absent)`);
console.error(
`\n RENAMED_DEFS (scripts/lib/renamed-defs.ts) declares that these defs were renamed,\n` +
` and a rename must carry EVERY key: the author-facing contract is unchanged, only\n` +
` an internal schema name moved. A key missing under the new name is a real removal\n` +
` wearing a rename's clothes — and these schemas are NOT .strict(), so Zod would\n` +
` silently strip whatever the author kept writing (#3733, ADR-0104).\n\n` +
` Either re-add the key under the new def, or — if it is genuinely being retired —\n` +
` tombstone it there with \`retiredKey()\` plus its registered D2 conversion, exactly\n` +
` as a retirement without a rename would require.`,
);
process.exit(1);
}

// (a) A key that vanished outright. The silent-strip class — always fatal.
const vanished = [...prev.keys()].filter((k) => !currentKeys.has(k));
if (vanished.length > 0) {
Expand Down
Loading
Loading