From 9e7f0bd072946f1d281e3a61b8d7489b9d4fc858 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 7 Aug 2026 15:54:21 +0000 Subject: [PATCH] =?UTF-8?q?docs(spec):=20SYNC=5FARCHITECTURE.md=20L3=20?= =?UTF-8?q?=E6=AE=B5=E5=81=9C=E6=AD=A2=E5=AE=A3=E4=BC=A0=E5=B7=B2=E9=80=80?= =?UTF-8?q?=E5=BD=B9=E7=9A=84=E5=87=BA=E7=AB=99=20Rate=20Limiting=20(#5554?= =?UTF-8?q?)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `connector.rateLimitConfig` 及其整个形状(`ConnectorRateLimitConfig` + `RateLimitStrategy` 枚举)已在 @objectstack/spec 17.0.0 按 #4911 / ADR-0049 D2 退役,理由不是"暂时没人读",而是**出站限流引擎从来就不存在**:平台唯一的令牌桶 `packages/runtime/src/security/rate-limit.ts` 是入站的,没有任何东西节流连接器 发出的调用。文档的示例块早已带上墓碑注释,散文却没跟着改。 正文点名三处,实测为六处,全部改为如实说法(措辞复用 #4911 墓碑现成句: 出站限流请在 connector provider 或上游网关做): - L191 Purpose 导语:`rate limiting` → `retry policies` - L197 Key Features:删掉打勾的 `Rate Limiting: Token bucket, leaky bucket algorithms`(两个从来不存在的算法),改为显式的 ❌ 条目 + 引用块。这里用 显式否定而非静默删除:#4911 注释点名的危害是"作者以为平台替我限流"—— 这一面最像安全承诺,漏掉比编译不过更糟,只有写出来才消得掉。 - L356 Best Practices:限流请在 provider / 网关侧做;`retryConfig` 处理超限 后拿到的 429,它不负责让你不超限 - L374 Decision Matrix:⛔ 未删行(retry 那半是对的)。改写为"retry policies and circuit breaking",保留 `retryConfig` / `health.circuitBreaker`,并写明 出站限流不构成选择任何一层的理由 - L394 Pattern 2 示意图:`Rate Limiting` → `Retry / Circuit Breaker` - L447 Migration Guide L2→L3 引导语:同上 保留的每一条都对着 schema 核过,不是假定: - `retryConfig`(`connector.zod.ts:769`)在,`strategy` 默认 `exponential_backoff`,`retryableStatusCodes` 默认 `[408, 429, 500, 502, 503, 504]` —— 确含 429,故引用块可以这么写 - `health.circuitBreaker`(`ConnectorHealthSchema:526` → `CircuitBreakerConfigSchema:505`)在 纯散文改动,未动任何 ```typescript 块:`etl-author-shape.test.ts` 钉的总块数 仍为 6,`connector-author-shape.test.ts` 钉的 `Connector` 例子仍为 3(两个门禁 30 tests 实跑通过,非假定)。 `packages/spec/docs/` 不在该包 `package.json` 的 `files` 白名单内,不随包发布, 故不写 changeset,改用 `skip-changeset` 标签。 Fixes #5554 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_014wsZeReNTqiceBfLb5Pyf5 --- packages/spec/docs/SYNC_ARCHITECTURE.md | 30 ++++++++++++++++++++----- 1 file changed, 24 insertions(+), 6 deletions(-) diff --git a/packages/spec/docs/SYNC_ARCHITECTURE.md b/packages/spec/docs/SYNC_ARCHITECTURE.md index 9270996b2f..2d8ffbe5e2 100644 --- a/packages/spec/docs/SYNC_ARCHITECTURE.md +++ b/packages/spec/docs/SYNC_ARCHITECTURE.md @@ -188,18 +188,32 @@ const dataWarehousePipeline: ETLPipeline = { ### Purpose -Complete, production-grade integration with external systems. Includes authentication, security, webhooks, rate limiting, and full lifecycle management. +Complete, production-grade integration with external systems. Includes authentication, security, webhooks, retry policies, and full lifecycle management. ### Key Features - ✅ **Authentication**: OAuth2, JWT, SAML, API Key, Basic Auth - ✅ **Webhooks**: Bidirectional event notifications -- ✅ **Rate Limiting**: Token bucket, leaky bucket algorithms - ✅ **Retry Policies**: Exponential backoff, circuit breaker - ✅ **Field Mapping**: With transformations and data type conversion - ✅ **Conflict Resolution**: Multiple strategies (`ConnectorConflictResolution`) - ✅ **Security**: Signature verification, encryption - ✅ **Monitoring**: Health checks, metrics, logging +- ❌ **Outbound rate limiting**: **not provided** — at this or any other level; see below + +> **There is no outbound rate limiting.** This list used to carry a ticked +> "**Rate Limiting**: Token bucket, leaky bucket algorithms" line. It named two +> algorithms that never existed. `connector.rateLimitConfig` — and the entire +> `ConnectorRateLimitConfig` / `RateLimitStrategy` shape behind it — was removed +> in `@objectstack/spec` 17.0.0 (#4911, ADR-0049 D2), because **no outbound +> rate-limiting engine ever existed**. The platform's only token bucket +> (runtime `security/rate-limit.ts`) throttles **INBOUND** requests *to* us; +> nothing throttles the calls a connector makes *out*. Do **not** substitute +> `shared`'s `RateLimitConfig` — that is the inbound limiter and would cap the +> wrong direction. **Until an outbound throttle exists, rate-limit at the +> connector provider or upstream gateway.** What L3 does declare for a +> rate-limited upstream is `retryConfig` — whose `retryableStatusCodes` default +> `[408, 429, 500, 502, 503, 504]` includes `429` — and `health.circuitBreaker`. ### Use Cases @@ -353,7 +367,10 @@ const sapConnector: ConnectorInput = { ### Best Practices - **Security First**: Always use encrypted credentials and secure storage -- **Rate Limiting**: Respect external API rate limits to avoid throttling +- **Rate Limiting**: Respect the upstream API's limits — and enforce that at the + connector provider or upstream gateway, since the connector shape declares no + outbound throttle (#4911). `retryConfig` handles the `429` you get for exceeding a + limit; it does not keep you under one - **Error Handling**: Implement comprehensive retry logic with exponential backoff - **Monitoring**: Set up health checks and alerting for connector failures - **Testing**: Test authentication, sync, and webhook flows thoroughly @@ -371,7 +388,7 @@ const sapConnector: ConnectorInput = { | Do you need multi-source aggregation? | **Yes** → L2 (ETL) | | Do you need real-time webhooks? | **Yes** → L3 (Connector) | | Do you need advanced authentication (OAuth2, SAML)? | **Yes** → L3 (Connector) | -| Do you need rate limiting and retry policies? | **Yes** → L3 (Connector) | +| Do you need retry policies and circuit breaking? | **Yes** → L3 (Connector) — `retryConfig`, `health.circuitBreaker`. Outbound **rate limiting** is not a reason to pick any level: no level provides it (#4911); throttle at the provider or gateway | | Is it a simple point-to-point sync with an external system? | **Yes** → L3 (Connector) with `syncConfig` | | Are you building a data warehouse pipeline? | **Yes** → L2 (ETL) | | Are you integrating with an enterprise system? | **Yes** → L3 (Connector) | @@ -391,7 +408,7 @@ Use **L2 ETL Pipeline** for multi-source data warehousing. ``` ObjectStack ↔ Enterprise Connector ↔ SAP ↓ - Webhooks, Auth, Rate Limiting + Webhooks, Auth, Retry / Circuit Breaker ``` Use **L3 Enterprise Connector** for production-grade integrations — including straightforward point-to-point sync, via a connector instance with simple `auth` @@ -444,7 +461,8 @@ and `ETLPipeline` is the author shape. ### From L2 to L3 -When your ETL pipeline needs webhooks, advanced auth, or rate limiting: +When your ETL pipeline needs webhooks, advanced auth, or retry / circuit-breaker +policies: **Before (L2):** ```typescript