feat(vcffi): write a nil fixed-width pointer as a typed null, and read typed nulls by coderdan · Pull Request #499 · cipherstash/vitaminc

coderdan · 2026-10-20T11:46:29Z

Closes #493

Stacked on #497 (base: feat/aead-typed-null). Review that first; this PR's diff is only the Go side and the shared corpus.

Why

#497 makes Rust seal a fixed-width None as a typed null, the same length as a value, so ciphertext length doesn't show which optional values are absent. The Go codec (vcffi) is what the Go SDK (cipherstash/stack#1298) is meant to share. Until this PR it could neither write a typed null nor read one, so Go-encrypted optional fields still leaked absence by length.

What changes for a Go caller

A nil pointer to a fixed-width type is written as a typed null, the Go side of Rust's Option<T>. That covers *int8 … *int64, *int, the unsigned widths, *float32, *float64, *time.Time, *vcvalue.Date, *vcvalue.Timestamp, *vcvalue.Decimal, *vcvalue.UUID, *vcvalue.Int128, *vcvalue.Uint128, a named int or float type (type Age int32), and through any number of pointers.
```
type Person struct {
    Name string
    Born *vcvalue.Date // nil now seals as long as a date
}
```
A nil with no fixed-width type to go by stays the plain null: a nil interface, *bool (already 1 byte), *string and *[]byte (variable width), *[N]byte, a pointer to a struct (a named type over a vcvalue or time type included, since reflection writes it field by field), and a pointer to an Encryptable, which chooses its own encoding.
For a caller whose Go type can't say what's missing (a nil in map[string]any), there is vcvalue.TypedNull{Type: vcvalue.TypeDate}. An Encryptable can call Encoder.TypedNull(vcvalue.TypeInt32). vcvalue.FixedType names the 16 fixed-width types.
An Encryptable whose EncryptValue always writes one value of one fixed-width type can implement vcffi.FixedWidth (FixedType() vcvalue.FixedType), the Go side of Rust's Encrypt::FIXED. A nil pointer to it is then a typed null of that type. FixedType is called once per type on a pointer to a zero value, and an invalid declaration fails the encode with ErrInvalidFixedType rather than write the plain null.
Unmarshal reads a typed null and decodes it as nil, like every null. Its validation is as strict as Rust's: header, then exactly the zero padding, so every absence has one encoding. Rust's frame writer writes a typed null back as the plain null (feat(aead)!: seal a fixed-width None as a typed null as long as a Some #497), so Go does too.

Wire format

The same as #497 and pinned against the same bytes. For example, *int32 nil gives 15 04 00 00 00, *vcvalue.Date nil gives 15 12 00 00 00, and *vcvalue.UUID nil gives 15 F3 05 00 0B + 15 zeros. TestTypedNullKnownAnswers pins all 16 types and checks each against a value's length.

Shared corpus

testdata/value-conformance.json gains:

9 normalized vectors. A typed null of each kind reads as null and writes back as 00.
25 malformed vectors: padding that is nonzero, short or trailing; a tag with no typed null (null, undefined, bool, text, plain bytes, an array, a nested typed null, 0xF4); an envelope over a shorthand type, id zero, the undefined id, the decimal id, timestamp nanos, a uuid over an int, a uuid with a length prefix, a uuid with a nonzero parameter, a timestamp over a float, an unknown id over bytes or bool; and a typed null inside an envelope.

Rust (vitaminc-aead's leaf test, vitaminc-aead-frame's conformance test), Go (vcffi) and Node (value-conformance.cjs, through the napi addon) all read it directly, and all pass.

Guest rebuild

bindings/go/vcencrypt/wasm/vitaminc_guest.wasm is rebuilt from this tree with cargo build --release --target wasm32-wasip1, as vcencrypt/README.md says. The differential fuzz targets compare Go's decoder with the committed guest, which now accepts typed nulls in the frames it encrypts. Their walkers (rewrite in vcencrypt, canonicalForm in vcffi) learn the projection: a typed null re-encodes as 00 on both sides.

Trade-offs to review

Behaviour change for existing Go callers. A nil *int32 used to seal as the 1-byte null and now seals padded. That is the point, and it matches Rust. A reader built before feat(aead)!: seal a fixed-width None as a typed null as long as a Some #497 refuses the new leaf.
Decrypt then re-encrypt through any loses the type. Since feat(aead)!: seal a fixed-width None as a typed null as long as a Some #497, Rust's frame writer writes typed nulls as the plain null, so a decrypted frame never carries one. A value re-encrypted from any or vcvalue.Object therefore seals its nils as the plain null. Hold optional fields in a typed struct (Born *vcvalue.Date) before encrypting again. The README says so.
No opt-out. A switch back to the plain null would put the leaky encoding one flag away. A binding must instead update its guest together with vcffi; a guest that predates feat(aead)!: seal a fixed-width None as a typed null as long as a Some #497 refuses 0x15. The README says so.
[N]byte stays the plain null. A plain bytes typed null has no length in its header to say where its padding ends in a frame (the same call as feat(aead)!: seal a fixed-width None as a typed null as long as a Some #497).

Review

A high-effort review posted 10 inline findings, with no correctness bugs among them. Eight are addressed in 1283e1b:

interceptMarkers, encodeReflect's numeric arms and fixedTypeOf now share one table, checked against a sample of every fixed-width entry.
The UUID header is shared, and Valid() is used.
The 24-byte uuid-with-length vector replaces the truncated one.
The *string check is added.
The README is corrected, and fixedTypeOf is cached and runs only for nil pointers.

Two are answered without code changes, as above: decrypt then re-encrypt through any, and the opt-out. One more is declined with reasons in its thread: a compile-time-closed FixedType isn't expressible in Go, and an invalid one fails closed with ErrInvalidFixedType.

A second, automated review posted 7 findings and listed 4 more as optional. All 11 are addressed in 8735de7, with a reply on each posted thread:

Marshal hung on a nil value of a pointer type that points at itself (type P *P): fixedTypeOf's walk is now bounded by maxDepth, and such a type keeps the plain null.
The three corpus vectors above (uuid with a parameter, envelope under the undefined and decimal ids), so the param != 0 and id checks have tests in every binding.
vcffi.FixedWidth, so an Encryptable that always writes one fixed-width value can have its nil pointer padded.
Tests: a row for every fixed-width type (*vcvalue.Uint128 was missing), a non-nil pointer to a nil *int32, **Encryptable with either receiver, every case encoded twice so the cache is read, subtests instead of one Fatalf, a sample whose marker lost its fixed type fails the header test, the guest length test covers all 16 types, and a nil fixed-width pointer inside vcvalue.Plain goes through the guest and comes back as the plain null.

Testing

go vet and go test pass for vcvalue, vcffi and vcencrypt. go vet passes for GOOS=linux GOARCH=386, but the 386 tests were not run locally: they need a linux runner, which CI has.
vcencrypt/typed_null_test.go, through the real guest: a struct with a nil field of each of the 16 fixed-width types seals every leaf to the same length as a present one, and decrypts as nil. A nil *bool matches a present bool, a nil *string an empty string, and a nil *int32 inside vcvalue.Plain is the plain null in the ciphertext and after a decrypt.
vcffi/typed_null_test.go: known answers, the nil-pointer table (every fixed-width type, pointers to pointers, Encryptables with either receiver, a self-pointing type, each encoded twice), FixedWidth, struct fields, the explicit channel, and strict decode edges.
Fuzzing: FuzzUnmarshal ran for 20s and FuzzDifferentialValue for 30s against the rebuilt guest, with no failures.
Guest: cargo fmt --check, clippy (wasm32-wasip1 and native) with -D warnings, and cargo test all pass. The userrecord example runs.
Rust: the vitaminc-aead tests (all features) and the vitaminc-aead-frame conformance tests pass. Node conformance reports 34 values and 70 refusals.

github-actions · 2026-10-20T11:47:54Z

🧬 Mutation testing (cargo-mutants, `--in-diff`)

No mutants were generated for the changed lines.

github-actions · 2026-10-20T11:51:36Z

✅ No CRAP threshold violations

1329 function(s) analyzed · threshold 30

chatgpt-codex-connector · 2026-10-21T00:02:53Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-21T00:06:57.373379Z	`1ea3b59`	Draft marked ready
🔒 Security Review	✅ Completed 2026-10-21T00:08:26.859151Z	`1ea3b59`	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

cipherstash-bot

Recommendation: 🟡 merge after changes (1 of 4 review job(s) failed)

Before merging, fix the endless loop in fixedTypeOf for pointer types that point at themselves, such as type P *P. Also add the three small tests that protect the rule "one encoding per absence": the vcvalue.Uint128 marker row, the UUID parameter vector, and the Undefined and Decimal envelope vectors. The other comments can wait for a follow-up.

Other findings not posted as comments

bindings/go/vcencrypt/typed_null_test.go:18: the guest length test covers 6 of 16 fixed-width types. Add int8, int16, int64, Int128, uint8, uint16, uint32, uint64, Uint128 and float32 to optional. Optional.
bindings/go/vcffi/encoder.go (vcvalue.Plain marker): no test sends a nil fixed-width pointer inside vcvalue.Plain through the guest. Plain{V: (*int32)(nil)} writes a typed null inside a passthrough frame. Optional.
bindings/go/vcffi/typed_null_test.go:121: the table tests use t.Fatalf with no t.Run, so the first failure hides the others. Optional.
bindings/go/vcffi/encoder.go:600: no test covers a non-nil outer pointer around a nil fixed-width inner pointer, for example Marshal(&missing) with var missing *int32. The expected result is 1504000000. Optional.

How this review was made

Agent	Model	Review type	Result
claude	claude-opus-5-5	test-gap	7 found, 5 posted
claude	claude-opus-5-5	golang	3 found, 2 posted
codex	gpt-5.6-terra	test-gap	1 found, 0 posted
codex	gpt-5.6-terra	golang	failed

Synthesis: claude-sonnet-5-5 merged the findings, removed duplicates and dropped findings it could not confirm in the code. 0 posted finding(s) were raised by two or more models.

Plain language: claude-sonnet-5-5 read every comment as a new reader would. 2 comment(s) had a problem that stopped the reader acting; it rewrote 2. It also rewrote the review body.

Stack: position 2 of 2 (#497, *️⃣ #499). *️⃣ marks this pull request.

Context loaded: the description, 1 linked issue(s) and 23 discussion entries.

…m alike Nine normalized vectors (a typed null of each kind reads as null and is written back as the plain null, as a frame writer writes it) and 22 malformed ones (padding that is not zero or the wrong length, a tag or an envelope with no typed null). Rust, Go and Node read the corpus directly. Refs #493

…d typed nulls, so Go hides absent values too A nil *int32, *float64, *time.Time, *vcvalue.Date, *vcvalue.Decimal, *vcvalue.UUID (a named type over one included) now encodes as a typed null, padded to the length of a value of its type, as Rust writes an Option<T>'s None. vcvalue.TypedNull and Encoder.TypedNull cover a caller whose Go type cannot say what is missing. A nil with no fixed-width type (an interface, a *bool, a *string, an Encryptable) stays the plain null. Unmarshal reads a typed null as strictly as the Rust codec and decodes it as nil, as Rust's frame writer writes it back as the plain null. The committed guest is rebuilt from this tree so the differential fuzz targets compare Go with the Rust that reads typed nulls, and their walkers learn the projection. Refs #493

…their padding, sharper tests and docs - Drive interceptMarkers and the numeric kinds of encodeReflect from the same tables fixedTypeOf reads, so a new marker type cannot be written one way and padded another; a test checks every fixed-width entry against a sample value, and fails for a marker added without one. fixedTypeOf now runs only for a nil pointer and is cached per type. - Share the UUID envelope header between Encoder.UUID and its typed null, and check Encoder.TypedNull's type with FixedType.Valid. - Make the corpus's uuid-with-length vector the full 24 bytes, so it tests a length prefix rather than truncation. - Check the *string half of the plain-null test. - Say in the README that only named int and float types are padded, that a value re-encrypted from any or vcvalue.Object seals its nils as the plain null, and that a binding must update its guest with this module. - Hoist the differential walker's width tables to one shared table. Refs #493

…l-pointer type walk, and pin the typed-null edges the review found A pointer type that points at itself (type P *P) never reaches a non-pointer, so fixedTypeOf looped forever on a nil value of it and Marshal never returned. The walk is now bounded by the same depth budget that bounds a value cycle, and past it the type keeps the plain null. An Encryptable that always writes one value of one fixed-width type had no way to say so, so its nil pointer leaked absence by length where a plain *int64 did not. FixedWidth is the Go side of Rust's Encrypt::FIXED: a nil pointer to a type that implements it is a typed null of the type it declares, and an invalid declaration fails the encode rather than write the plain null in its place. The shared corpus gains three refusals every binding now checks: a uuid typed null with a nonzero parameter, and a typed null in the envelope under the undefined or decimal id, which Go accepted only because no vector reached those arms. The nil-pointer table covers every fixed-width type, a non-nil pointer to a nil one, a pointer to a pointer to an Encryptable with either receiver, and reads the per-type cache; the header test fails for a sample whose marker lost its fixed type; and the guest tests seal an optional field of all sixteen types and a nil fixed-width pointer inside a passthrough.

auxesis

Thanks @coderdan, approved.

Appreciate the follow up on the review feedback too. ❤️