Skip to content

test: verify a bucket migrated to the tenant-rooted layout #1870 - #1893

Open
DmytroZaichenkoDev wants to merge 3 commits into
feat/issue-1870-2-access-differfrom
feat/issue-1870-3-bucket-verifier
Open

DmytroZaichenkoDev wants to merge 3 commits into
feat/issue-1870-2-access-differfrom
feat/issue-1870-3-bucket-verifier

Conversation

@DmytroZaichenkoDev

Copy link
Copy Markdown
Contributor

Stacked on #1892.

Copies a legacy tree into the tenant-rooted layout, checks everything arrived, then boots a tenant-rooted core onto the copy with an empty cache and reads it all back.

The only instrument that crosses layouts. The other two start empty and each run reads only its own writes, so neither can see a failure that happens only to data written under one layout and read under the other — which is the whole of what P2 is about to do.

What it compares:

Inventory every source path, mapped through the transform, exists at the destination; nothing extra
Content the bytes
Metadata content-encoding, content-type, etag, author, created_at
public/rules/rules by name, not as a row in a count — its absence fails open, and every object count would still add up
Read-back what each resource returned before the move vs after

Read-back is a before/after comparison rather than an assertion of 200: the question is whether migration changed anything, not whether every fixture happens to be readable.

BucketCopier is a stand-in for the migrator P2 will build, not that migrator. It exists so the verifier has something to verify, and so the questions the verifier asks are settled before the real one lands. Swap it when P2 arrives.

What building it turned up: a byte-perfect copy still produces an unreadable bucket. Whether an object is compressed is recorded beside it rather than inside it, along with its etag, author and creation time — so a copy that keeps every byte and drops the metadata reads back as a parse error. Only some resources are affected, since small ones are stored uncompressed, which is what makes it easy to miss.

Inventory and checksums both passed while that was true. That is why the verifier compares metadata as well, and it is the thing P2's migrator most needs to get right.

🤖 Generated with Claude Code

@DmytroZaichenkoDev
DmytroZaichenkoDev force-pushed the feat/issue-1870-3-bucket-verifier branch from 087abbe to 22acbe4 Compare September 2, 2026 14:31
@DmytroZaichenkoDev
DmytroZaichenkoDev force-pushed the feat/issue-1870-2-access-differ branch from 0a6cc22 to 132276a Compare September 2, 2026 14:31
@DmytroZaichenkoDev
DmytroZaichenkoDev force-pushed the feat/issue-1870-3-bucket-verifier branch from 22acbe4 to 0adef3f Compare September 2, 2026 15:07
@DmytroZaichenkoDev
DmytroZaichenkoDev force-pushed the feat/issue-1870-2-access-differ branch 2 times, most recently from 5d65dbb to b4bc5ac Compare September 2, 2026 15:23
@DmytroZaichenkoDev
DmytroZaichenkoDev force-pushed the feat/issue-1870-3-bucket-verifier branch 2 times, most recently from 0e59c3d to aebd9b1 Compare September 2, 2026 15:56
@DmytroZaichenkoDev
DmytroZaichenkoDev force-pushed the feat/issue-1870-2-access-differ branch from b4bc5ac to 480ed3a Compare September 2, 2026 15:56
@DmytroZaichenkoDev
DmytroZaichenkoDev force-pushed the feat/issue-1870-3-bucket-verifier branch from aebd9b1 to 2763d4a Compare September 3, 2026 10:53
@DmytroZaichenkoDev
DmytroZaichenkoDev force-pushed the feat/issue-1870-2-access-differ branch from 480ed3a to 3b03a3a Compare September 3, 2026 10:53
@DmytroZaichenkoDev
DmytroZaichenkoDev force-pushed the feat/issue-1870-3-bucket-verifier branch from 2763d4a to d3ba078 Compare September 3, 2026 11:58
@DmytroZaichenkoDev
DmytroZaichenkoDev force-pushed the feat/issue-1870-2-access-differ branch from 3b03a3a to 4415dcb Compare September 3, 2026 11:58
@DmytroZaichenkoDev
DmytroZaichenkoDev marked this pull request as ready for review September 3, 2026 15:05
Dmytro Zaichenko and others added 3 commits September 7, 2026 14:38
Copies a legacy tree into the tenant-rooted layout, checks everything arrived,
then boots a tenant-rooted core onto the copy with an empty cache and reads it
all back.

This is the only instrument that crosses layouts. The other two start empty and
each run reads only its own writes, so neither can see a failure that happens
only to data written under one layout and read under the other, which is the
whole of what P2 is about to do.

Read-back compares what each resource returned before the move with what it
returns after, rather than asserting a 200: the question is whether migration
changed anything, not whether every fixture is readable.

The copier is a stand-in for the migrator P2 will build, not that migrator. It
exists so the verifier has something to verify, and so the questions the verifier
asks are settled before the real one lands.

Two of those questions came out of building it. A byte-perfect copy still
produces an unreadable bucket: whether an object is compressed is recorded beside
it rather than inside it, along with its etag, author and creation time, so a
copy that keeps every byte and drops the metadata reads back as a parse error.
Inventory and checksums both passed while that was true, which is why the
verifier compares metadata too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The verifier moved bytes but nothing it moved was ciphertext, so the
one hazard specific to encrypted resources — an AAD derived from a path
that migration changes — was never exercised. The seed now stores a
user-authored external service with a client secret, and serving it
after migration is a named check like the rules document: before/after
equality alone would also pass if both reads failed identically.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants