Skip to content

feat(entities): auto-refetch when realtime broadcast is truncated#165

Closed
theunreal wants to merge 2 commits into
base44:mainfrom
theunreal:auto-refetch-truncated-entities-base-40236
Closed

feat(entities): auto-refetch when realtime broadcast is truncated#165
theunreal wants to merge 2 commits into
base44:mainfrom
theunreal:auto-refetch-truncated-entities-base-40236

Conversation

@theunreal
Copy link
Copy Markdown
Collaborator

@theunreal theunreal commented Apr 27, 2026

Why

Realtime entity broadcasts have a transport size limit. Records with very large fields (e.g. base64-encoded media, long markdown) can exceed the limit and get dropped — subscribers see stale state until refresh.

The Base44 backend now slims oversize broadcasts to fit under the limit and signals it with _oversize: true on the event data. Without this SDK change, customer apps that read the truncated fields would render placeholder text until refresh.

This change makes the recovery transparent: the SDK detects _oversize and refetches the full record over HTTP before invoking the user callback. Existing app code keeps working with no changes.

What

In entities.X.subscribe(callback):

  1. Parse the realtime event as before.
  2. If event.type !== "delete" and event.data._oversize, call the entity GET endpoint and replace event.data with the response.
  3. Concurrent subscribers in the same browser fan out to a single HTTP call — debounced by (entityName, id, timestamp) via an in-flight Map<string, Promise<T>>. The cache entry clears 5 s after the promise settles so late subscribers can still piggy-back.
  4. On HTTP failure: log a warning and fall through with the stub payload. UX is no worse than today's behavior on dropped messages.
  5. Delete events skip the refetch — the record no longer exists.

Wire contract is preserved: the server always emits data as an object containing at minimum id, so older SDK clients that don't know about _oversize continue to receive a structurally valid event.

Tests

tests/unit/entities-subscribe.test.ts — 5 new tests, all 11 pass:

  • refetches full record over HTTP when data._oversize is true
  • does NOT refetch on delete events even if _oversize is set
  • does NOT refetch when _oversize is absent
  • falls through with stub payload + warning when refetch fails
  • debounces concurrent refetches for the same (entity, id, timestamp) — only one HTTP call

Full suite: 158 / 158 pass. `npm run lint` and `npm run build` clean.

Backward compatibility

  • Existing subscribers: no behavior change for events without _oversize. Pure passthrough.
  • Older SDK versions: receive the stub payload as-is — render placeholder strings until refresh, same as today's behavior. Customer apps pick up the new behavior on their next rebuild.
  • Server without slimming: _oversize flag never appears, so this code path is dormant.

Versioning

Recommend a minor bump (0.8.x → 0.9.0) since this is additive behavior.

Server-side BASE-40236 slims oversize entity broadcasts to fit under the
Redis pubsub cap, signaling the slim with `_truncated: true` on the event
data. Customer apps that render the truncated fields would otherwise
display placeholder text until refresh.

The SDK now detects `_truncated` in `entities.X.subscribe` and transparently
refetches the full record over HTTP before invoking the user callback, so
deployed customer code keeps working without changes. Concurrent subscribers
in the same browser fan out to a single HTTP call via an in-flight
debounce keyed by `${entityName}:${id}:${timestamp}`. On refetch failure
the SDK falls through with the partial payload and logs a warning, so the
failure mode is no worse than today's drop-and-stale.

Delete events skip refetch — the record no longer exists.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@theunreal theunreal changed the title feat(entities): auto-refetch when realtime broadcast is truncated (BASE-40236) feat(entities): auto-refetch when realtime broadcast is truncated Apr 27, 2026
The wire flag is set only when the server falls back to a stub payload
because the original record was too big for realtime transport. Calling
it "truncated" was misleading — we don't truncate fields in that path,
we replace the whole payload with `{id, _oversize: true}`. `_oversize`
names the actual cause and tells the SDK why a refetch is needed.

Coordinated with the matching backend rename.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@theunreal
Copy link
Copy Markdown
Collaborator Author

Superseded by #167 — recreated from a base-repo branch so that secrets/OIDC-dependent checks (claude-review, claude-docs-draft, publish-preview) can run.

@theunreal theunreal closed this Apr 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant