Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 64 additions & 0 deletions .changeset/connector-rate-limit-config-removed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
---
"@objectstack/spec": major
---

refactor(spec)!: remove `connector.rateLimitConfig` and the whole outbound rate-limit shape — the engine never existed (#4911, ADR-0049)

`ConnectorSchema.rateLimitConfig` let an author declare an outbound throttle for
their connector — `strategy`, `maxRequests`, `windowSeconds`, `burstCapacity`,
`respectUpstreamLimits`, `rateLimitHeaders` — and nothing anywhere applied it.
This is not the ordinary declared-but-unread case; it is a step worse:
**there is no outbound rate-limiting engine to wire it to.** The only token
bucket the platform owns is `packages/runtime/src/security/rate-limit.ts`, and it
is INBOUND — the dispatcher calls `consume(key)` on a request fingerprint and
answers 429. No connector provider (`connector-rest`, `connector-openapi`,
`connector-mcp`, `connector-slack`) reads the key, and no seam exists that could.

So a well-formed, schema-validated block told the author they had capped their
call rate against a third party's quota, and capped nothing — the false-compliance
class ADR-0049 exists for. With no implementation and no committed roadmap,
`experimental` would be a promise nobody made; **absent** is the honest
disposition. The vocabulary comes back *with* the engine, in one change
(implementation-first — the #4834 / PR #4878 ruling for the plugin-runtime family).

FROM → TO:

| Removed | Replacement |
| :--- | :--- |
| `connector.rateLimitConfig` (key) | **none** — delete it; throttle at the connector provider or upstream gateway |
| `ConnectorRateLimitConfigSchema` / `ConnectorRateLimitConfig` | **none** — importing either is TS2305 in v17 |
| `RateLimitStrategySchema` / `RateLimitStrategy` | **none** — the enum had no other consumer |

**Do NOT substitute `shared`'s `RateLimitConfig`.** That is the INBOUND limiter
(`enabled` / `windowMs` / `maxRequests`) and caps the calls others make to *us* —
the opposite direction. #4684 split the two names for exactly this confusion; the
conversion deliberately does not rewrite one into the other, because that would
silently change behaviour rather than losing a no-op.

The retirement kit:

- **Tombstone.** `ConnectorSchema` is not `.strict()`, so a plain delete would be
a silent strip (ADR-0104). `retiredKey()` makes the removal audible in the two
channels an upgrading author hits — `tsc` (the key types `never`) and the parse
(the prescription itself). It reaches `stack.connectors[]` and
`DeclarativeConnectorEntry`, which is `ConnectorSchema.superRefine(…)`.
- **ADR-0087 D2 conversion + D3 chain step** (`connector-rate-limit-config-removed`,
`retiredFromLoadPath`): `os migrate meta --from 16` deletes the key from author
sources and stored rows replay clean. A lossless delete — the block never had an
effect to lose.
- **The shape goes with the key.** `ConnectorRateLimitConfigSchema` and the
`RateLimitStrategySchema` enum it embedded had no other consumer, and an
exported schema with no consumer reads as a capability to whoever finds it
(#3950).
- **#4684's rename is absorbed.** `integration/RateLimitConfig` →
`integration/ConnectorRateLimitConfig` and this retirement landed in the same
unreleased major; composed they are a plain delete, so the `RENAMED_DEFS` entry
is removed rather than pointing at a def this build no longer emits.
- Baselines updated deliberately: `json-schema.manifest.json` (−2 defs),
`authorable-surface.json` (−6 def lines; `Connector` /
`DeclarativeConnectorEntry` gain `… [RETIRED]`), `api-surface.json` (−4
exports). `api-surface-signatures.json` is unchanged by construction — it hashes
each `defineX` parameter as TypeScript *prints* it, a reference
(`z.input<typeof ConnectorSchema >`), so key-level narrowing never reaches it.

No runtime behaviour changes — that impossibility is the reason for the removal.
38 changes: 4 additions & 34 deletions content/docs/references/integration/connector.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -134,8 +134,8 @@ a dead end of the same class in #4738.)
## TypeScript Usage

```typescript
import { CircuitBreakerConfigSchema, ConnectorSchema, ConnectorActionSchema, ConnectorConflictResolutionSchema, ConnectorErrorCategorySchema, ConnectorFieldMappingSchema, ConnectorHealthSchema, ConnectorRateLimitConfigSchema, ConnectorRetryStrategySchema, ConnectorStatusSchema, ConnectorTriggerSchema, ConnectorTypeSchema, DataSyncConfigSchema, DeclarativeConnectorEntrySchema, ErrorMappingConfigSchema, ErrorMappingRuleSchema, HealthCheckConfigSchema, RateLimitStrategySchema, RetryConfigSchema, SyncStrategySchema, WebhookConfigSchema, WebhookEventSchema, WebhookSignatureAlgorithmSchema } from '@objectstack/spec/integration';
import type { CircuitBreakerConfig, Connector, ConnectorConflictResolution, ConnectorErrorCategory, ConnectorFieldMapping, ConnectorHealth, ConnectorRateLimitConfig, ConnectorRetryStrategy, ConnectorStatus, ConnectorType, DataSyncConfig, DeclarativeConnectorEntry, ErrorMappingConfig, ErrorMappingRule, HealthCheckConfig, RateLimitStrategy, RetryConfig, SyncStrategy, WebhookConfig, WebhookEvent, WebhookSignatureAlgorithm } from '@objectstack/spec/integration';
import { CircuitBreakerConfigSchema, ConnectorSchema, ConnectorActionSchema, ConnectorConflictResolutionSchema, ConnectorErrorCategorySchema, ConnectorFieldMappingSchema, ConnectorHealthSchema, ConnectorRetryStrategySchema, ConnectorStatusSchema, ConnectorTriggerSchema, ConnectorTypeSchema, DataSyncConfigSchema, DeclarativeConnectorEntrySchema, ErrorMappingConfigSchema, ErrorMappingRuleSchema, HealthCheckConfigSchema, RetryConfigSchema, SyncStrategySchema, WebhookConfigSchema, WebhookEventSchema, WebhookSignatureAlgorithmSchema } from '@objectstack/spec/integration';
import type { CircuitBreakerConfig, Connector, ConnectorConflictResolution, ConnectorErrorCategory, ConnectorFieldMapping, ConnectorHealth, ConnectorRetryStrategy, ConnectorStatus, ConnectorType, DataSyncConfig, DeclarativeConnectorEntry, ErrorMappingConfig, ErrorMappingRule, HealthCheckConfig, RetryConfig, SyncStrategy, WebhookConfig, WebhookEvent, WebhookSignatureAlgorithm } from '@objectstack/spec/integration';

// Validate data
const result = CircuitBreakerConfigSchema.parse(data);
Expand Down Expand Up @@ -181,7 +181,7 @@ Circuit breaker configuration
| **syncConfig** | `{ strategy?: Enum<'full' \| 'incremental' \| 'upsert' \| 'append_only'>; direction?: Enum<'import' \| 'export' \| 'bidirectional'>; schedule?: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object }; realtimeSync?: boolean; … }` | optional | Data sync configuration |
| **fieldMappings** | `{ source: string; target: string; transform?: { type: 'constant'; value: any } \| { type: 'cast'; targetType: Enum<'string' \| 'number' \| 'boolean' \| 'date'> } \| { type: 'lookup'; table: string; keyField: string; valueField: string } \| { type: 'javascript'; expression: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object } } \| { type: 'map'; mappings: Record<string, any> }; defaultValue?: any; … }[]` | optional | Field mapping rules |
| **webhooks** | `{ name: string; label?: string; object?: string; triggers?: Enum<'create' \| 'update' \| 'delete' \| 'bulk_update' \| 'bulk_delete'>[]; … }[]` | optional | Webhook configurations (not yet enforced — never read at registration; see #3197) |
| **rateLimitConfig** | `{ strategy?: Enum<'fixed_window' \| 'sliding_window' \| 'token_bucket' \| 'leaky_bucket'>; maxRequests: number; windowSeconds: number; burstCapacity?: number; … }` | optional | Rate limiting configuration |
| **rateLimitConfig** | `any` | optional | [REMOVED] `connector.rateLimitConfig` was removed in @objectstack/spec 17.0.0 (#4911, ADR-0049 D2) — the entire shape is gone, not just this key: `ConnectorRateLimitConfig` and its `RateLimitStrategy` enum were removed with it, because no outbound rate-limiting engine ever existed. The platform's only token bucket (runtime `security/rate-limit.ts`) throttles INBOUND requests to us; nothing throttled the calls a connector makes out, so every knob here was inert while reading like a configured cap. Delete the key. Do NOT substitute `shared` `RateLimitConfig` — that is the inbound limiter and would cap the wrong direction; until an outbound throttle exists, rate-limit at the connector provider or upstream gateway. Run `os migrate meta --from 16` to rewrite it automatically. |
| **retryConfig** | `{ strategy?: Enum<'exponential_backoff' \| 'linear_backoff' \| 'fixed_delay' \| 'no_retry'>; maxAttempts?: number; initialDelayMs?: number; maxDelayMs?: number; … }` | optional | Retry configuration |
| **connectionTimeoutMs** | `number` | optional | Connection timeout in ms |
| **requestTimeoutMs** | `number` | optional | Request timeout in ms |
Expand Down Expand Up @@ -270,22 +270,6 @@ Connector health configuration
| **circuitBreaker** | `{ enabled: boolean; failureThreshold: number; resetTimeoutMs: number; halfOpenMaxRequests: number; … }` | optional | Circuit breaker configuration |


---

## ConnectorRateLimitConfig

### Properties

| Property | Type | Required | Description |
| :--- | :--- | :--- | :--- |
| **strategy** | `Enum<'fixed_window' \| 'sliding_window' \| 'token_bucket' \| 'leaky_bucket'>` | ✅ | Rate limiting strategy |
| **maxRequests** | `number` | ✅ | Maximum requests per window |
| **windowSeconds** | `number` | ✅ | Time window in seconds |
| **burstCapacity** | `number` | optional | Burst capacity |
| **respectUpstreamLimits** | `boolean` | ✅ | Respect external rate limit headers |
| **rateLimitHeaders** | `{ remaining: string; limit: string; reset: string }` | optional | Custom rate limit headers |


---

## ConnectorRetryStrategy
Expand Down Expand Up @@ -386,7 +370,7 @@ Connector type
| **syncConfig** | `{ strategy?: Enum<'full' \| 'incremental' \| 'upsert' \| 'append_only'>; direction?: Enum<'import' \| 'export' \| 'bidirectional'>; schedule?: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object }; realtimeSync?: boolean; … }` | optional | Data sync configuration |
| **fieldMappings** | `{ source: string; target: string; transform?: { type: 'constant'; value: any } \| { type: 'cast'; targetType: Enum<'string' \| 'number' \| 'boolean' \| 'date'> } \| { type: 'lookup'; table: string; keyField: string; valueField: string } \| { type: 'javascript'; expression: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object } } \| { type: 'map'; mappings: Record<string, any> }; defaultValue?: any; … }[]` | optional | Field mapping rules |
| **webhooks** | `{ name: string; label?: string; object?: string; triggers?: Enum<'create' \| 'update' \| 'delete' \| 'bulk_update' \| 'bulk_delete'>[]; … }[]` | optional | Webhook configurations (not yet enforced — never read at registration; see #3197) |
| **rateLimitConfig** | `{ strategy?: Enum<'fixed_window' \| 'sliding_window' \| 'token_bucket' \| 'leaky_bucket'>; maxRequests: number; windowSeconds: number; burstCapacity?: number; … }` | optional | Rate limiting configuration |
| **rateLimitConfig** | `any` | optional | [REMOVED] `connector.rateLimitConfig` was removed in @objectstack/spec 17.0.0 (#4911, ADR-0049 D2) — the entire shape is gone, not just this key: `ConnectorRateLimitConfig` and its `RateLimitStrategy` enum were removed with it, because no outbound rate-limiting engine ever existed. The platform's only token bucket (runtime `security/rate-limit.ts`) throttles INBOUND requests to us; nothing throttled the calls a connector makes out, so every knob here was inert while reading like a configured cap. Delete the key. Do NOT substitute `shared` `RateLimitConfig` — that is the inbound limiter and would cap the wrong direction; until an outbound throttle exists, rate-limit at the connector provider or upstream gateway. Run `os migrate meta --from 16` to rewrite it automatically. |
| **retryConfig** | `{ strategy?: Enum<'exponential_backoff' \| 'linear_backoff' \| 'fixed_delay' \| 'no_retry'>; maxAttempts?: number; initialDelayMs?: number; maxDelayMs?: number; … }` | optional | Retry configuration |
| **connectionTimeoutMs** | `number` | optional | Connection timeout in ms |
| **requestTimeoutMs** | `number` | optional | Request timeout in ms |
Expand Down Expand Up @@ -452,20 +436,6 @@ Health check configuration
| **healthyThreshold** | `number` | ✅ | Consecutive successes before marking healthy |


---

## RateLimitStrategy

Rate limiting strategy

### Allowed Values

* `fixed_window`
* `sliding_window`
* `token_bucket`
* `leaky_bucket`


---

## RetryConfig
Expand Down
3 changes: 3 additions & 0 deletions docs/protocol-upgrade-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,8 @@ That kernel-side tombstone was then SUPERSEDED inside the same unreleased major

Finally it removes the script-body capability token 'crypto.hash' (#4391). Four layers declared it — the `HookBodyCapability` enum, the doc table beside it, the CLI extractor and `ScriptContext.crypto.hash` — and none implemented it: `installCtx` wired only `randomUUID`, so the one call the token authorised threw inside the VM every time. The build-time inference made it worse than an ordinary declared-but-unenforced key: writing `ctx.crypto.hash(...)` made the CLI ADD the capability for you, so `os build` went green on the body that was guaranteed to fail at the first record write. Removed rather than implemented (ADR-0049) — hashing inside the sandbox widens its capability and security-review surface, and a capability that throws on every use yet drew zero complaints in its whole life is its own liveness verdict. This is an enum VALUE, not a key, so there is no `retiredKey()` tombstone: the enum error map carries the prescription, keyed on the received value so that only the spelling which used to be legal is told it "was removed". The conversion strips the dead token from `body.capabilities` on hooks and actions; it deliberately does NOT touch the `ctx.crypto.hash(...)` call the body made under it, which never returned a value and which the author must delete. Hashing returns only WITH an implementation, through the capability admission process.

It also removes `connector.rateLimitConfig` and its whole shape (#4911). This one is not "declared but unread" — it is declared but UNIMPLEMENTED, one step worse. The only token bucket the platform owns (runtime `security/rate-limit.ts`) is INBOUND: the dispatcher calls `consume(key)` on a request fingerprint and answers 429. Nothing anywhere throttles the calls a connector makes OUT, and no provider — `connector-rest`, `connector-openapi`, `connector-mcp`, `connector-slack` — reads the key or has a seam that could. So `strategy`, `maxRequests`, `windowSeconds`, `burstCapacity`, `respectUpstreamLimits` and `rateLimitHeaders` parsed cleanly and capped nothing, on a surface where the author believed they had bounded their spend against a third party's quota. `ConnectorRateLimitConfig` and the `RateLimitStrategy` enum it embedded had no other consumer and are removed with the key, so importing either is TS2305 in v17 — the #4834 shape, and the same implementation-first ruling: the vocabulary comes back WITH the engine, in one change. It is deliberately NOT converted to `shared` `RateLimitConfig`, which limits the calls others make to US; #4684 split their names for precisely this confusion, and rewriting an outbound cap into an inbound one would throttle the wrong direction. Delete the key and rate-limit where the calls are actually made — the connector provider or upstream gateway.

### Mechanical (applied for you)

| Conversion | Surface | Change | Load window |
Expand Down Expand Up @@ -229,6 +231,7 @@ Finally it removes the script-body capability token 'crypto.hash' (#4391). Four
| `object-managed-by-system-to-system-data` | `object.managedBy` | object managedBy 'system' → 'system-data' (#3355 — ADR-0103's residual bucket named the engine-owned half v16 had already moved out to `engine-owned`; the rename leaves the name describing what the bucket actually holds: admin/user-writable platform data) | retired — `migrate meta` only |
| `object-enable-trash-mru-removed` | `object.enable.trash / object.enable.mru` | object capability flags 'enable.trash'/'enable.mru' removed (#3207, #2377 close-out — no recycle bin and no MRU tracking ever ran; both default-true flags gated nothing) | retired — `migrate meta` only |
| `hook-body-crypto-hash-removed` | `hook.body.capabilities / action.body.capabilities` | script-body capability token 'crypto.hash' removed (#4391 — the sandbox never installed ctx.crypto.hash, so the token granted a call that always threw; the CLI inferred it too) | retired — `migrate meta` only |
| `connector-rate-limit-config-removed` | `connector.rateLimitConfig` | connector key 'rateLimitConfig' removed (#4911 — no outbound rate-limiting engine exists; the runtime's only token bucket limits INBOUND requests, so every knob here was inert while reading like a configured cap. The whole ConnectorRateLimitConfig shape went with it) | retired — `migrate meta` only |

### Semantic (delegated to you, with acceptance criteria)

Expand Down
4 changes: 0 additions & 4 deletions packages/spec/api-surface.json
Original file line number Diff line number Diff line change
Expand Up @@ -3865,8 +3865,6 @@
"ConnectorOrigin (type)",
"ConnectorProviderContext (interface)",
"ConnectorProviderFactory (type)",
"ConnectorRateLimitConfig (type)",
"ConnectorRateLimitConfigSchema (const)",
"ConnectorRetryStrategy (type)",
"ConnectorRetryStrategySchema (const)",
"ConnectorSchema (const)",
Expand All @@ -3887,8 +3885,6 @@
"ErrorMappingRuleSchema (const)",
"HealthCheckConfig (type)",
"HealthCheckConfigSchema (const)",
"RateLimitStrategy (type)",
"RateLimitStrategySchema (const)",
"ResolvedConnectorAuth (type)",
"RetryConfig (type)",
"RetryConfigSchema (const)",
Expand Down
10 changes: 2 additions & 8 deletions packages/spec/authorable-surface.json
Original file line number Diff line number Diff line change
Expand Up @@ -4073,7 +4073,7 @@
"integration/Connector:name",
"integration/Connector:provider",
"integration/Connector:providerConfig",
"integration/Connector:rateLimitConfig",
"integration/Connector:rateLimitConfig [RETIRED]",
"integration/Connector:requestTimeoutMs",
"integration/Connector:retryConfig",
"integration/Connector:status",
Expand Down Expand Up @@ -4105,12 +4105,6 @@
"integration/ConnectorInstanceBearerAuth:credentialRef",
"integration/ConnectorInstanceBearerAuth:type",
"integration/ConnectorInstanceNoAuth:type",
"integration/ConnectorRateLimitConfig:burstCapacity",
"integration/ConnectorRateLimitConfig:maxRequests",
"integration/ConnectorRateLimitConfig:rateLimitHeaders",
"integration/ConnectorRateLimitConfig:respectUpstreamLimits",
"integration/ConnectorRateLimitConfig:strategy",
"integration/ConnectorRateLimitConfig:windowSeconds",
"integration/ConnectorTrigger:description",
"integration/ConnectorTrigger:interval",
"integration/ConnectorTrigger:key",
Expand Down Expand Up @@ -4140,7 +4134,7 @@
"integration/DeclarativeConnectorEntry:name",
"integration/DeclarativeConnectorEntry:provider",
"integration/DeclarativeConnectorEntry:providerConfig",
"integration/DeclarativeConnectorEntry:rateLimitConfig",
"integration/DeclarativeConnectorEntry:rateLimitConfig [RETIRED]",
"integration/DeclarativeConnectorEntry:requestTimeoutMs",
"integration/DeclarativeConnectorEntry:retryConfig",
"integration/DeclarativeConnectorEntry:status",
Expand Down
12 changes: 3 additions & 9 deletions packages/spec/docs/SYNC_ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -264,14 +264,8 @@ const sapConnector: Connector = {
}
],

// Rate Limiting
rateLimitConfig: {
strategy: 'token_bucket',
maxRequests: 100,
windowSeconds: 60,
burstCapacity: 150,
respectUpstreamLimits: true
},
// (`rateLimitConfig` sat here until #4911 retired it — no outbound
// rate-limiting engine ever existed. Throttle at the provider/gateway.)

// Retry Configuration
retryConfig: {
Expand Down Expand Up @@ -404,7 +398,7 @@ const pipeline: ETLPipeline = {
const connector: Connector = {
authentication: { type: 'oauth2', ... },
webhooks: [...],
rateLimitConfig: { ... }
retryConfig: { ... }
};
```

Expand Down
Loading
Loading