Repository navigation
fix(studio,shard-engine): decode wire-tagged doc columns on the display read paths - #339
Conversation
…ay read paths
The `__doc__` blob is written through the wire codec, so a `v.bigint()` or
`v.bytes()` column is stored as a tagged array (`["$lunora.wire$","bigint",…]`).
Three display paths parsed that blob with bare `JSON.parse`, so the tag itself
reached the screen where the value belongs — on `paymentSessions.amountMinor`,
that is an amount.
- `introspect.ts`: `safeParseObject` routes through `decodeDocJson`, covering
both its call sites. Its non-throwing contract is preserved deliberately — a
malformed tag degrades to an unexpanded page rather than failing the read.
- `use-data-browser.tsx`: decode in `rowDocument`. Decoding alone would have
broken the JSON editor, which is a pre-existing defect this makes visible:
the rows now carry real bigints and `JSON.stringify` throws on one, so
editing any row of a bigint-bearing table already failed. Fixed
symmetrically — the editor is seeded from `encodeWire(doc)` and `writeRow`
decodes on the way out, so an untouched save re-encodes byte-identically to
what the shard stored.
- `payments-panel.tsx`: decode in `readField` for codec agreement. This one is
inert today and the plan's premise for it was wrong: the panel reads only
`subscriptions`/`events`, and every `v.bigint()` in the payment schema is on
`paymentSessions`, which it never opens.
- `lib/internal.ts`: a decoded `ArrayBuffer` formatted to a bare `{}`, strictly
less informative than the tag it replaced. Render `<bytes: N>` instead.
Studio reaches the codec through the bundler-inlined `shared/wire-codec` (it
already inlines `shared/log-fields`), so no package dependency was added and no
second decoder exists.
Every new behavioural test was demonstrated to fail against the base revision:
the tag surfacing instead of `1000n`, `{}` instead of `<bytes: 3>`, the payments
value vanishing to `""`, and the editor's `TypeError: Do not know how to
serialize a BigInt`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019WfuWNN3cWJE6beZjng347
✅ Deploy Preview for lunorash ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
Thank you for confirming the Contributor License Agreement! 🙏 |
|
Thank you for following the naming conventions! 🙏 |
WalkthroughThe change adds shared wire-format decoding and encoding to shard introspection and Studio document flows. Bigint and binary values now decode for parsing and display, remain encoded during row editing, and decode before write operations. ChangesWire-format handling
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
React Doctor found no new issues. 🎉 Reviewed by React Doctor for commit |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/shard-engine/src/introspect.ts`:
- Around line 861-865: Use one strict plain-record guard for decoded documents:
update safeParseObject in packages/shard-engine/src/introspect.ts (lines
861-865) to reject all non-plain roots, including byte-like objects; apply the
same guard in packages/studio/src/features/data/hooks/use-data-browser.tsx
(lines 145-150) while preserving the column-strip fallback; and validate the
decoded editor value in
packages/studio/src/features/data/hooks/use-data-browser.tsx (lines 853-859)
before assigning parsedDocument or sending the write.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 97fcfbe9-10fe-4875-9e80-c75c27e01b05
⛔ Files ignored due to path filters (6)
packages/shard-engine/__tests__/introspect.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/studio/__tests__/features/data/data-browser.test.tsxis excluded by!**/__tests__/**and included bypackages/**packages/studio/__tests__/features/payments/payments-panel.test.tsxis excluded by!**/__tests__/**and included bypackages/**packages/studio/__tests__/lib/internal.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**plans/300-decode-doc-read-paths.mdis excluded by none and included by noneplans/README.mdis excluded by none and included by none
📒 Files selected for processing (4)
packages/shard-engine/src/introspect.tspackages/studio/src/features/data/hooks/use-data-browser.tsxpackages/studio/src/features/payments/payments-panel.tsxpackages/studio/src/lib/internal.ts
Review catch, and a real regression from the decode. `!Array.isArray` was a
sufficient document check under bare `JSON.parse`, whose only non-plain root is
an array — but a `__doc__` whose ROOT is a tagged value parses as an array and
DECODES to a `Uint8Array` / `Date` / `Map`, which is neither null nor an array.
The old check waved those through:
root ["$lunora.wire$","bytes","AAEC","ArrayBuffer"]
→ decodes to Uint8Array → spreads to {"0":1,"1":2,"2":3}
i.e. a page of junk numeric columns where the expansion should have bailed.
All three read/write sites now narrow by prototype (`null` or
`Object.prototype`), matching `encodeWire`'s own definition of a plain object so
the two ends agree on what a document is. The editor's save path gained the
check outright — it had none, so a free-text `[1,2]` or a root-level tag would
have been handed to the writer as a document.
Tests cover bytes / date / map roots on both sides and are demonstrated to fail
against the array-only check. The bare-array case passes either way and is kept
as the control that shows the new tests discriminate.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019WfuWNN3cWJE6beZjng347
…s' into advisor/300-decode-doc-read-paths
… one document guard
Thermos review found the plan this branch executes was wrong about its own
primary surface, and that the fix for it was a regression.
The data browser never showed a raw tag. `readTablePage`'s result goes back
through `jsonResponse`, and `LunoraClient` decodes the whole response
(`decodeWire(body.result)`), so a tagged array is already a real bigint by the
time `formatCell` runs. Decoding inside `introspect.ts` broke that: `jsonResponse`
is the one DO result path that does NOT `encodeWire`, so a decoded `v.bigint()`
made `JSON.stringify` throw — every bigint table would have returned a redacted
500 — and a decoded `v.bytes()` flattened to `{}` before the client saw it.
That decode is reverted. A test pins the invariant instead: the page must stay
JSON-serializable, and the tagged form must decode to `1000n` across a simulated
wire. It fails with the decode reinstated.
What stays, because it was never on that path:
- `rowDocument` / `readField` parse a raw `__doc__` STRING, which the client's
decode leaves alone (a JSON string stays a string), so they must decode.
- The editor's encode-prefill / decode-save pair, which fixes a pre-existing
defect: `JSON.stringify` throws on a bigint, so editing any row of a
bigint-bearing table already failed.
- `formatCell`'s byte rendering — a decoded ArrayBuffer really does reach it.
Also from review:
- The "is this a document" predicate existed in five near-copies, already drifted
(one rejected null-prototype objects, the others accepted them). It now lives
once in `shared/wire-codec.ts` as `isPlainObject` + `decodeDocument`, next to
the encoder that defines the term. Net deletion.
- `formatCell` reuses the `formatBytes` that already lived 70 lines below it, so
a 2 MB blob reads `<bytes: 2.0 MB>` rather than `<bytes: 2097152>`.
- The editor reports a JSON syntax error and a malformed tag separately, instead
of labelling a codec RangeError "Invalid JSON".
- Three comments corrected: a typed array stringifies to its indices, not `{}`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019WfuWNN3cWJE6beZjng347
…bigint Pre-existing on alpha, surfaced by the review of this branch. `LunoraClient` decodes the wire codec on the way in, so a `v.bigint()` column arrives in the studio as a real `bigint` and `v.bytes()` as an `ArrayBuffer`. Three surfaces then serialize a row without accounting for that: - the JSON view (`data-browser-page.tsx`) — `JSON.stringify` throws during RENDER, so the whole Data page dies rather than just the toggle; - "Export page → JSON" (`grid-features.tsx`) — throws inside the click handler, so the download silently never happens and nothing is surfaced; - the `hash` mask strategy (`mask-preview.ts`) — throws and fails closed to the custom-mask sentinel, so a masked bigint column shows the sentinel instead of a hash. Reachable today: `@lunora/payment`'s `paymentSessions` has three `v.bigint()` columns, so opening it and switching to JSON view is enough. The CSV exporter already handled bigint explicitly, which is how it went unnoticed in the others. One shared `jsonRowReplacer` covers the two stringify sites, rendering a bigint as a decimal string (JSON has no bigint) and bytes as the same `<bytes: N>` summary a cell shows. `mask-preview` only special-cases bigint, so every other value's hash is byte-for-byte unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019WfuWNN3cWJE6beZjng347
Merging this PR will regress 3 benchmarks
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
|
This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
Follow-on to #338. Rewritten after review — the plan this executes was wrong about its own primary surface, and my first cut of the fix was a regression. History kept so the correction is auditable.
Plan:
plans/300-decode-doc-read-paths.md.What the plan claimed, and what is actually true
The plan said three display paths parse
__doc__with bareJSON.parse, so av.bigint()column renders as its raw tag (["$lunora.wire$","bigint","1000"]) where an amount belongs.For the data browser grid that was false.
readTablePage's result goes back throughjsonResponse, andLunoraClientdecodes the entire response (decodeWire(body.result)), so the tagged array is already a realbigintbeforeformatCellruns. The grid has always rendered1000.And decoding there actively breaks it.
jsonResponseis the one DO result path that does notencodeWire(contrast the user dispatch and the WS push, which both do). Verified by execution:The throw becomes a redacted 500, so every
v.bigint()table would have become unbrowsable, and bytes would have been silently emptied. That decode is reverted.A test now pins the invariant rather than trusting a comment: the page must stay JSON-serializable, and the tagged form must survive to decode as
1000nacross a simulated wire. It fails with the decode reinstated (TypeError: Do not know how to serialize a BigInt).What is genuinely fixed
use-data-browser.tsxrowDocument__doc__string. The client's decode leaves strings alone, so this one really does see the tag.payments-panel.tsxreadFieldpaymentSessions), kept for codec agreement — now a one-line call, so it costs nothing.JSON.stringifythrows on a bigint, so editing any row of a bigint table already failed. Prefill now encodes and save decodes, making an untouched save byte-identical to what was stored.formatCellbytesArrayBuffergenuinely reaches it, and rendered as a bare{}.Also from review
isPlainObject+decodeDocumentinshared/wire-codec.ts, next to the encoder that defines the term. Net deletion.formatCellnow reuses theformatBytesthat already lived 70 lines below it — a 2 MB blob reads<bytes: 2.0 MB>, not<bytes: 2097152>.RangeError"Invalid JSON".__doc__whose root is a tagged value parses as an array but decodes to aUint8Array/Date/Map, which!Array.isArraywaves through. A bytes root spread into{"0":1,"1":2,"2":3}.{}.Second commit: a pre-existing crash cluster
The review surfaced three more surfaces that serialize a row without accounting for the client-side decode. All reachable on
alphatoday, sincepaymentSessionshas threev.bigint()columns:data-browser-page.tsx) — throws during render, so the whole Data page dies, not just the toggle;grid-features.tsx) — throws in the click handler; the download silently never happens;hashmask strategy (mask-preview.ts) — throws and fails closed to the sentinel, so a masked bigint shows the sentinel instead of a hash.The CSV exporter already special-cased bigint, which is how the others went unnoticed. One shared
jsonRowReplacercovers the two stringify sites;mask-previewspecial-cases bigint only, so every other value's hash is unchanged.Corrections to the plan's own answers
expandDocumentRows—runReadonlySqlis a separate path — so it still shows raw__doc__text. Pre-existing, out of scope, worth its own plan.v.bigint()in the payment schema is onpaymentSessions, which the payments panel never reads.Known, not fixed
A legacy row carrying a literal
__proto__field loses it on an untouched save —encodeWireassigns by key, so the prototype setter fires instead of the property landing. Only reachable for pre-codec rows (encodeDocJsondrops it on write too).Verification
Gates: shard-engine 981, studio 1012, client 640, do 519,
lint:typesclean, prettier + eslint clean. Every new behavioural test demonstrated to fail against the code it guards.🤖 Generated with Claude Code
https://claude.ai/code/session_019WfuWNN3cWJE6beZjng347