Skip to content

feat(import): full Supabase and Firebase migration — rows, auth, and files - #357

Merged
prisis merged 22 commits into
alphafrom
feat/supabase-firebase-importers
Aug 7, 2026
Merged

prisis merged 22 commits into
alphafrom
feat/supabase-firebase-importers

Conversation

@prisis

@prisis prisis commented Aug 7, 2026 •

Copy link
Copy Markdown
Member

Builds on the Convex blob migration merged in #354, reusing its id
preservation, content-hash keys and verification machinery.

lunora import --from supabase and --from firebase now take a dump the whole
way: rows, auth, and files, with the same id preservation, parity check and
dangling-reference reporting the Convex path already had.

What it adds

Sources. Supabase Postgres CSV and Firestore export JSON both decode into
the one { table, doc } envelope the importer already speaks. The CSV cast
distinguishes an unquoted empty field (NULL) from a quoted "" (empty string)
and from \N (text-format NULL) — collapsing those three is silent data loss.
Firestore's typed-value encoding (integerValue as a string, timestampValue,
mapValue, referenceValue, geoPointValue, bytesValue) decodes to plain
JSON.

Mapping. lunora/import-<source>.json states which file feeds which table,
which column is the id, which columns hold storage paths, and any declared
reshape. --scan proposes one by sampling the dump. A mapping that fails to
parse throws naming the key rather than degrading to "no mapping" — a dropped
mapping turns a declared conversion into a silent pass-through.

Auth. Supabase auth.users + auth.identities and a Firebase auth:export
become better-auth user/account rows. Password hashes and reset tokens are
never emitted; users sign in again.

Storage. Supabase objects transfer live over the Storage REST API with a
service-role key read from the environment (never a flag — a flag is visible in
the process table). Firebase objects come from a gcloud storage cp -r
directory, because gcloud already owns Google's auth. Both then take the
identical path into R2: content-hash key, checksum-verified upload,
skip-if-present, delete-on-mismatch. The transfer is resumable — every
object is checkpointed, so a run that dies at object 40,000 of 50,000 resumes
without re-downloading what already moved. A partial transfer deliberately does
not import rows: every path column would point at an object that is not there
yet.

Advisor. A static lint flags a migrated-away platform's SDK still imported
under lunora/.

Docs. Rewritten data-migration steps in the Supabase and Firebase guides.

Behaviour changes worth a reviewer's attention

  • A broken migration now exits non-zero. Previously a run could report
    success while emitting credential columns as table rows, or import a table
    with no _id at all. Both reproduced before the fix; both now fail closed.
  • Uploads record their digest. storageUpload passes sha256 through to
    R2, so list()/head() report a real checksum instead of degrading every
    downstream integrity check to a size comparison.
  • A ZIP snapshot streams. The directory import read tables line by line
    while the ZIP import inflated each entry whole into a string and split it into
    an array of every line — so the same export succeeded as a directory and ran
    the CLI out of heap as an archive. It now reads the entry's byte range through
    zlib.
  • Storage columns resolve whatever spelling the app stored — bucket-qualified,
    bucket-relative, or a getPublicUrl()/createSignedUrl() URL. A bare filename
    two buckets both carry stays unresolved on purpose: guessing would rewrite rows
    at the wrong object.
  • ImportCommandResult.body is ImportSummary | undefined instead of
    unknown (the one api-snapshot change).

Verification

lint:types, lint:eslint, lint:prettier, lint:package-json, api:check
and dist:check all green. Suites: cli 1168, storage 109, runtime 845, advisor
467, codegen 1054, seed 79, studio 1012.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added lunora import support for Firebase and Supabase exports, including CSV, Firestore JSON, authentication data, and storage files.
    • Added resumable storage transfers, checksum verification, reference remapping, scan-generated mappings, and detailed import reporting.
    • Added streaming support for large Convex snapshot archives.
    • Added advisor warnings for stale platform imports after migration.
    • Added SHA-256 verification and content-type validation for storage uploads.
  • Documentation

    • Added migration guides for Convex, Firebase, and Supabase, including supported formats, workflows, and limitations.

prisis and others added 14 commits August 7, 2026 14:37
`lunora import --from supabase|firebase` turns a foreign dump into the same
`{ table, doc }` NDJSON the Convex importer already feeds, so ids preserve
verbatim and foreign keys survive without a second remapping pass.

Supabase reads a directory of `COPY … TO STDOUT WITH CSV HEADER` files through
`csv-parse` — quoting, embedded newlines, and doubled quotes are exactly what a
hand-rolled split gets wrong on the one file an operator cannot re-create. NULL
is distinguished from the empty string the way Postgres CSV encodes it: an
unquoted empty field is NULL, a quoted `""` is a real empty string. Collapsing
the two would null every empty text column in the database.

Firebase decodes the Firestore typed-value JSON — `{ integerValue: "42" }`,
`{ timestampValue }`, `{ mapValue: { fields } }`, references, geopoints, bytes —
from a directory of per-collection files, in any of the three container shapes
those tools emit.

Reshapes are declared per column in `lunora/import-<source>.json`, from a closed
set with defined lossless targets. The rule the reshape module exists to enforce
is that a conversion which would lose information errors and names the column:
an `int8` past `Number.MAX_SAFE_INTEGER` or a `numeric` with more digits than a
double holds refuses rather than rounding, and points at `int8-string`. A column
the mapping does not name is copied through untouched. An unknown reshape name
fails the run instead of being ignored, and a malformed mapping never degrades
to "no mapping" — dropping it would turn a declared conversion into a silent
pass-through, which is the corruption the rule exists to prevent.

Firestore's `integerValue` is decoded the same way: it arrives as a string
precisely because it may exceed the safe range, and it stays one when it would
not survive the conversion.

Note on the plan: 305's D3 describes `gcloud firestore export` as emitting "JSON
document shards". It does not — that command writes LevelDB-log files wrapping
protobuf entities. What it describes is the Firestore REST/RPC JSON encoding,
which is real, is what every JSON-producing Firestore read path emits, and is
the part worth owning. The decoder targets that; the guide will document how to
produce it.
`--with-storage` now moves a foreign bucket into R2 alongside the rows, and
rewrites the columns a mapping names from the provider-side path to the R2 key.

Supabase objects are listed and downloaded live through its Storage REST API.
The service-role key is read from `SUPABASE_SERVICE_ROLE_KEY` rather than taken
as a flag: it grants full read/write on the project, and a flag puts it in the
process table where any other local process can read it — the same reason
`--token` already carries that warning. Firebase has no listing endpoint the CLI
can reach without owning Google's auth, so its bucket arrives as a directory the
operator downloaded with `gcloud storage cp -r` and passes as `--storage-dir`.

Both then take the identical path into R2 — content-hash key, checksum-verified
upload, skip-if-present, delete-on-mismatch — inherited from the Convex blob
migration rather than rebuilt.

A bucket migration is the one part of an import genuinely expensive to repeat:
every object is a download from the old provider and an upload to the new one, so
a run that dies at object 40,000 of 50,000 has spent real time and bandwidth.
Each completed transfer is appended to a checkpoint, and a re-run skips what is
already recorded — without re-downloading it. The checkpoint is append-only
NDJSON rather than a rewritten blob: appending costs no re-serialisation as the
run grows, and a process killed mid-write leaves at most a torn final line, which
is skipped at the cost of one repeated object rather than a corrupt file that
strands the whole migration. Progress is reported as it goes, so a multi-hour
transfer is visibly moving.

A partial transfer does not then import rows. Every path column would point at an
object that is not there yet, which is the dangling reference the files-first
ordering exists to prevent — so the run stops, says the checkpoint is saved, and
exits non-zero. A path the transfer never saw is reported and left as-is rather
than guessed at, and fails `--verify`.

Deviates from the plan's D5, which specified `@aws-sdk/client-s3`: Supabase's own
REST API takes the service-role key the operator already has, so this needs no
S3 client at all. That removes the plan's own listed bundle-size risk and one
credential the operator would otherwise have to create.
A `lunora/import-<source>.json` `auth` block turns a Supabase `auth.users`
dump (plus `auth.identities` for the linked providers) or a Firebase
`auth:export` JSON into better-auth `user` and `account` rows. Users are emitted
before accounts, so an account row never lands before the user it points at.

Passwords are never carried across. Supabase stores bcrypt, Firebase its own
scrypt variant, and better-auth hashes with neither — there is no honest
conversion, and inventing one locks every user out with an error they cannot act
on. Profile fields and `emailVerified` do carry, so a reset is the only thing a
user redoes.

Enforcing that turned out to need more than not reading the hash column: the
auth dumps were also being picked up as ordinary tables, so `auth.users.csv`
imported as a table named `auth.users` with `encrypted_password` intact. A test
asserting the bcrypt prefix never appears in the output caught it. Files a
mapping claims for auth are now excluded from the table listing.

Two users claiming one email fails the run with both ids named, rather than
silently merging two people into one identity.

`--scan` now covers the new sources. For Supabase it samples each column and
proposes a reshape, requiring every non-null value to agree — one stray value
falls back to "copy it through untouched", which is the safe answer, since an
un-reshaped column keeps its text while a wrong reshape either errors or changes
the data. `int8-string` is only proposed when a value genuinely exceeds the safe
range, so ordinary id columns are left alone. Firestore values are already
self-describing, so its skeleton just lists the collections with the
`storageColumns` an operator still has to declare. Neither overwrites a mapping
that already exists.
`migration_stale_import` catches the half-finished port: the data moved, most
handlers were rewritten, and one still imports `@supabase/supabase-js` or
`firebase/firestore`. Nothing fails — it compiles and typechecks — so the app
quietly keeps reading from the platform it just left and the two stores drift
apart until someone notices stale data.

The finding names the guide that covers finishing the port. Only the three
platforms with migration guides are matched: the lint exists to catch an
unfinished migration, not to police library choice, so a deliberate second data
source just has to live outside `lunora/`.

Evidence comes from a codegen feeder that reads real import declarations rather
than grepping — a specifier is only a specifier when the AST says so, and a
mention in a comment or a string is not an import. Prefix matching is
boundary-aware, so `convex` and `convex/server` match while `convexity` does not.
Both guides told the reader to dump each table, reshape it by hand, and
batch-insert through a mutation or a `lunora run` script — a homework assignment
rather than a migration path. They now walk the importer end to end: dumping,
generating the column mapping with `--scan`, importing with `--verify`, the auth
import, and the storage transfer.

Each guide closes with what the importer does NOT move — RLS policies, edge and
cloud functions, triggers, realtime listeners, sessions — linked back to the step
that covers it, so a reader who followed the import knows exactly what is still
theirs to do. Firebase says plainly that Realtime Database is a different format
with no reader, rather than leaving it to be discovered mid-cutover.

Two things the guides now state that the previous versions could not:

- `gcloud firestore export` is the wrong artifact — it writes protobuf, not
  JSON. The guide gives the Admin SDK script that produces what the reader wants.
- Passwords cannot migrate from either platform, so every user signs in once via
  "forgot password". The announcement needs writing before cutover, not after.

Also adds three engineering posts on the migration work: why preserving ids
makes a Convex import single-pass, why the Supabase reader refuses a conversion
that would drop a digit, and what Firestore's typed-value encoding is actually
for.

Plan 305 records both design deviations (the Firestore format, and dropping the
S3 SDK for Supabase's own REST API) and answers its eight open questions.
Six defects, five of which ended in a green `--verify` line over data that was
already wrong. Every one is now covered by a regression test.

**Credential material reached the wire.** The `auth.*` dump exclusion was built
only from what the mapping file named, but the mapping is optional — so a dump
imported without one sent `auth.users.csv` as an ordinary table, carrying the
bcrypt hash AND the live single-use `recovery_token`, `confirmation_token` and
`email_change_token_*` values. Exit 0. The docs and the code comment both
asserted the opposite guarantee. Exclusion is now by shape (`auth.` prefix) with
the mapping as an addition, and any row carrying a known credential column is
refused outright rather than filtered — a silent filter would hide that the file
is an auth dump.

**Rows silently lost their id.** `toDocument` set `_id` only on finding the id
column, with no check that it ever appeared. A table whose PK is `user_id`
imported with no `_id` at all, the target minted fresh ones, and every foreign
key dangled — with `--verify` green, because the row counts still matched. That
defeats the premise the whole importer rests on, so it is now a hard error
naming the table and the columns present.

**The transfer checkpoint was trusted without checking the target.** It records
source paths and nothing about which deployment it was written against, so a
resume against a different or wiped bucket skipped every upload and rewrote
documents to keys that do not exist. The listing is already in hand; a
checkpoint entry now has to appear in it.

**`--verify` passed over unmigrated storage in three ways**: for Convex when the
export was taken without `--include-file-storage` (no `_storage` table, so the
existing guard never fired and `{ $storage }` objects imported verbatim); for
the foreign sources, which had no equivalent guard at all; and for a plain
string in a column the mapping explicitly declared, which was passed through and
reported nowhere — the self-describing form had that check, the declared column
did not.

Also: the run summary printed `logger.success` on a run exiting 1, so the last
line an operator read disagreed with the exit code.
R2 reports a checksum only for objects whose writer supplied one, and
`upload()` never did. So `list()`/`head()` always returned no `sha256`, and
every downstream integrity check silently degraded to comparing sizes — while
the CLI's tests, whose fake bucket does return one, exercised a hash branch that
was dead in production.

The admin upload route already computes that digest to verify the body before
writing, then discarded it. It now travels through to `bucket.put`, which makes
R2 both verify the write itself and record the checksum. Two checks become real
as a result: the >32 MiB signed-PUT verification, and the idempotency probe,
which compared sizes alone and can now compare content.

`StorageObject` gains the `sha256` field the CLI was already reading. It existed
at runtime but not in the declared contract, so "the host does not report this"
was indistinguishable from "the contract never promised it" — the failure mode
the platform-parity note calls out after `ShardSqlExec`.

Also from review:

- `--scan` was a second implementation of the rewrite walk, kept in step with it
  by a comment. It is now a dry run of the real thing: an identity map and no
  `storageColumns` makes every plain string matching a storage id land in
  `ambiguous`, which is exactly the candidate list. Drift is now impossible
  rather than merely discouraged.
- 8 concurrent uploads × a 32 MiB per-blob ceiling is ~256 MiB of request bodies
  against a 128 MB isolate shared by every concurrent request. The window now
  budgets bytes as well as requests, so small blobs still run eight wide and
  large ones narrow it automatically.
- `contentType` was the one `_storage` field reaching an HTTP header unvalidated,
  so a CR/LF crashed the client with an opaque "Invalid header value" instead of
  naming the offending metadata line.
- A list response claiming more pages without advancing its cursor would have
  looped forever.
- `SUPABASE_URL` is refused unless https — the service-role key travels as a
  Bearer on every request, and over plain HTTP that is the whole credential in
  cleartext.
- A Firestore document with no fields crashed with "Cannot use 'in' operator";
  `documents.list` omits `fields` entirely for one.
- `timestamp-iso` truncated to milliseconds while Postgres defaults to
  microseconds, despite having a lossless target available.
Three structural findings, each deleting a copy rather than moving one.

**The two foreign readers were parallel implementations.** Matching
mapping-named files, excluding the auth dump, erroring on an empty directory,
sorting for determinism, seeding the parity tally, emitting auth rows first —
all written twice. That duplication had already drifted once: the auth CSV
reader lost the `\N` NULL handling, so a `COPY … WITH (FORMAT text)` dump gave
every other table proper nulls and the auth table the literal string "\N".
`dump-directory.ts` now owns the shape; each source contributes only its
decoder, which is the only part that genuinely differs.

**`auth-reader.ts` was one seam too many.** It imported five of the six symbols
`auth.ts` exported and nothing else did, and `AuthRows` was exported solely to
cross that boundary — the header comment defending the split was the tell.
Merged.

**`import-source.ts` and `import.ts` imported each other.** Type-only, so it
erased at build, but it coupled source resolution to the whole command flag
surface. Source resolution now declares the handful of flags it actually reads.
`CONVEX_STORAGE_TABLE` and `isConvexSystemTable` moved to `convex-snapshot.ts`,
which already owns the export layout, instead of being defined in the resolver
and imported back out by two other modules.
The live Supabase Storage path was the only third-party integration in the
branch and had zero tests: bucket enumeration, offset paging, the
folder-placeholder recursion that is the only reason nested objects move at all,
and the https guard. It now has them, including an assertion that the
service-role key never appears in a URL — it travels as a header, and a test is
the only thing keeping it that way.

Also covered: the storage-dir symlink guard (a security check with no test), and
the `conflicts > 0` parity branch — which exists specifically so a re-run does
not fail `--verify`, on the run the docs tell operators to make, and which every
fake worker returning `conflicts: 0` had left unexercised.

The batcher tests moved out of the storage file. Neither had any storage in
them, and both files now describe what they contain.

Two smaller review items: `STORAGE_CHECKSUM_MISMATCH` was inserted out of
alphabetical order in the error catalog, and `migration_stale_import` — a WARN —
sat above the schema-correctness errors that `STATIC_LINTS` documents as coming
first. `AdvisorStaleMigrationImport` is now re-exported like every sibling
feeder type; the snapshot referenced it without declaring it.
The allowlist guard inlined in `upload()` pushed it past the cognitive
complexity ceiling once the sha256 pass-through landed. Lifting it to a
named function keeps `upload()` a readable sequence of checks and gives
the security control its own documented unit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The directory import reads a table's `documents.jsonl` line by line; the
ZIP import inflated the whole entry into a string and then split it into
an array of every line. A snapshot's packaging therefore decided whether
the import fit in memory — the same export succeeded as a directory and
ran the CLI out of heap as an archive.

Read the entry's byte range through `zlib` instead, using the local file
header for the data offset (its name/extra lengths may differ from the
central copy). `readSnapshotText` had one caller left, which now reads
lines too, so it is gone.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A transfer keys objects by their bucket-qualified path, but an app column
holds whichever spelling it stored: bucket-qualified, bucket-relative, or
a getPublicUrl()/createSignedUrl() URL — the last being what most Supabase
apps persist. Only the first matched, so the other two were reported as
untransferred with no mapping knob able to fix them.

Index the transfer by all three. A bucket-relative name carried by more
than one bucket is deliberately left out: resolving it to whichever bucket
listed first would rewrite rows at the wrong object, which is worse than
reporting it unresolved.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- `ImportCommandResult.body` was `unknown`, so a caller wanting the
  conflict count cast its way there. It is now `ImportSummary`, and the
  `--scan` path (whose product is the file it writes) returns none.
- The two mapping files repeated their validation, phrasing the same
  mistake two ways; the three checks now live in one place. The shapes
  stay separate — a Convex export is self-describing, so its mapping has
  no use for a file name, an id column, or a declared reshape.
- The storage-reference walk moves out of the mapping-file module. It has
  two callers that must not diverge: the rewrite and --scan, which runs it
  as a dry run.
- The foreign path rewrite is a hook on the row transformer rather than a
  second transform wrapped around it, so a row is parsed and serialised
  once instead of twice.
- `TransferredObject.source` meant the object path while the sibling
  `source` argument meant the provider; they are now `path` and
  `provider`. The progress reporter is a logging cadence, not checkpoint
  state, so it sits with its caller.
- The three worker fakes in the source tests shared a copied import
  handler; now one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@netlify

netlify Bot commented Aug 7, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for lunorash ready!

Name Link
🔨 Latest commit e3ed7d0
🔍 Latest deploy log https://app.netlify.com/projects/lunorash/deploys/6a75e161a6661a0008ac83f4
😎 Deploy Preview https://deploy-preview-357--lunorash.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@coderabbitai

coderabbitai Bot commented Aug 7, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 8cde42c8-f9cb-41c1-8727-6a74fdfbd115

📥 Commits

Reviewing files that changed from the base of the PR and between b8f03fb and e3ed7d0.

⛔ Files ignored due to path filters (2)
  • packages/cli/__tests__/commands/zip-entry-stream.test.ts is excluded by !**/__tests__/**, !**/*.test.ts and included by packages/**
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml and included by none
📒 Files selected for processing (2)
  • packages/cli/package.json
  • packages/cli/src/commands/zip-entry-stream.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • packages/cli/src/commands/zip-entry-stream.ts
  • packages/cli/package.json

Walkthrough

The PR adds Firebase and Supabase import support, extends Convex storage migration, adds migration documentation, detects stale platform imports, and propagates storage checksums.

Changes

Migration import platform support

Layer / File(s) Summary
Source readers and mappings
packages/cli/src/commands/data-transfer/sources/*, packages/cli/package.json
Adds Firebase and Supabase readers, authentication conversion, mapping validation, type reshaping, scanning, directory discovery, and CSV parsing support.
Storage transfer and remapping
packages/cli/src/commands/data-transfer/storage-*, packages/cli/src/commands/data-transfer/sources/storage-*
Adds path resolution, content-hash uploads, resumable checkpoints, storage-reference rewriting, streaming snapshot reads, and bounded upload concurrency.
Unified import pipeline
packages/cli/src/commands/data-transfer/import*.ts, packages/cli/src/commands/import/*, packages/cli/src/commands/convex-snapshot.ts, packages/cli/src/commands/zip-entry-stream.ts
Adds source selection, storage-first imports, typed summaries, verification reporting, source-aware remapping, and streamed ZIP snapshot reads.
Migration documentation
apps/docs/src/content/blog/*, apps/docs/src/content/docs/migrating/*
Documents Firebase, Supabase, and Convex migration workflows, storage handling, authentication limits, verification, and unsupported features.

Advisor and storage contracts

Layer / File(s) Summary
Stale import advisor
packages/codegen/src/*, packages/advisor/src/*
Adds AST-based detection and lint reporting for migrated platform SDK imports.
Checksum-enabled uploads
packages/runtime/src/*, packages/storage/src/*, packages/codegen/src/emit-app.ts, packages/errors/src/catalog.ts
Adds optional SHA-256 propagation, centralized content-type validation, and storage error catalog reordering.

Estimated code review effort: 5 (Critical) | ~120 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CLI
  participant SourceReader
  participant StorageTransfer
  participant R2
  participant RowImporter
  CLI->>SourceReader: resolve and stream source rows
  CLI->>StorageTransfer: transfer source storage
  StorageTransfer->>R2: upload or resume objects
  StorageTransfer-->>CLI: return path mappings
  CLI->>RowImporter: remap and import rows
  RowImporter-->>CLI: return typed import summary
Loading

Possibly related PRs

  • anolilab/lunora#354: Extends the Convex blob migration, storage remapping, snapshot streaming, and checksum functionality.
  • anolilab/lunora#241: Shares Convex migration import functionality and the Convex migration guide.
  • anolilab/lunora#335: Shares code-generation and advisor-context changes.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the primary change: full Supabase and Firebase migration support for rows, authentication, and files.
Description check ✅ Passed The description provides a detailed summary, verification results, behavior changes, migration details, and documentation scope.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/supabase-firebase-importers

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Thank you for confirming the Contributor License Agreement! 🙏

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Thank you for following the naming conventions! 🙏

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 16

🧹 Nitpick comments (10)
packages/cli/src/commands/data-transfer/sources/reshape.ts (1)

52-53: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Merge the two identical regexes.

EXPONENT_RE and EXPONENT_SPLIT_RE are both /e/i. One constant covers both uses. Keep one name and delete the other.

Note: a /e/i literal has no g flag, so sharing it between test and split carries no lastIndex risk.

♻️ Proposed simplification
-const EXPONENT_RE = /e/i;
-const EXPONENT_SPLIT_RE = /e/i;
+const EXPONENT_RE = /e/i;

Then use EXPONENT_RE in expandExponent:

-    const [mantissa = "0", exponentPart = "0"] = raw.split(EXPONENT_SPLIT_RE);
+    const [mantissa = "0", exponentPart = "0"] = raw.split(EXPONENT_RE);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/cli/src/commands/data-transfer/sources/reshape.ts` around lines 52 -
53, Merge EXPONENT_RE and EXPONENT_SPLIT_RE into a single /e/i constant, remove
the duplicate declaration, and update expandExponent and any other references to
use the retained EXPONENT_RE symbol.
packages/runtime/src/storage-admin-routes.ts (1)

177-180: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

The checksum is recorded only for uploads that request verification.

sha256 stays undefined unless the caller sends expectedSha256 or expectedSize. Every other upload still lands without a recorded checksum, so the degradation the comment describes remains for those objects.

If you want every object to carry a checksum, compute the digest unconditionally. The cost is one SHA-256 over a body that STORAGE_UPLOAD_MAX_BODY_BYTES already bounds. If the current scope is intentional, narrow the comment to say the digest is recorded only when verification is requested.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/runtime/src/storage-admin-routes.ts` around lines 177 - 180, Update
the upload flow around storageUpload so the SHA-256 digest is computed
unconditionally for every request, rather than only when expectedSha256 or
expectedSize is provided, and pass that digest to storageUpload for recording.
If verification-only behavior is intentional instead, narrow the nearby comment
to state that the checksum is recorded only for uploads requesting verification.
packages/cli/src/commands/data-transfer/sources/storage-progress.ts (1)

85-91: 🚀 Performance & Scalability | 🔵 Trivial | 💤 Low value

recordTransfer runs mkdir and opens a file handle for every object.

Each completed object costs one mkdir and one open/write/close cycle. For a 50,000-object bucket that is 100,000 extra syscalls. Durability does not require it: an appended write to a handle opened with "a" gives the same one-line-per-object guarantee.

Open the checkpoint once per run and reuse the handle, or create the directory only when the file is first written.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/cli/src/commands/data-transfer/sources/storage-progress.ts` around
lines 85 - 91, Refactor recordTransfer so the checkpoint directory is created
and the progress file is opened once per transfer run, then reuse the
append-capable file handle for each completed object. Preserve one JSON line per
entry and ensure the handle is closed after the run, rather than performing
mkdir and open/write/close operations for every object.
packages/storage/src/create-storage.ts (2)

323-331: 🩺 Stability & Availability | 🔵 Trivial | 💤 Low value

Validate the sha256 format before handing it to R2.

uploadOptions.sha256 is forwarded unchecked. R2 requires a 64-character hex string or a 32-byte buffer. A truncated or non-hex value produces an opaque binding error instead of the VALIDATION_ERROR the rest of this module raises for bad input.

Add a short guard next to assertContentTypeAllowed so a malformed digest names the field.

♻️ Proposed guard
+const SHA256_HEX_RE = /^[\da-f]{64}$/i;
+
+/** Reject a digest R2 would refuse, so the error names the option rather than the binding. */
+const assertSha256Valid = (sha256: ArrayBuffer | string | undefined): void => {
+    if (sha256 === undefined) {
+        return;
+    }
+
+    if (sha256 instanceof ArrayBuffer) {
+        if (sha256.byteLength !== 32) {
+            throw new LunoraError("VALIDATION_ERROR", "`@lunora/storage`: sha256 buffer must be exactly 32 bytes");
+        }
+
+        return;
+    }
+
+    if (!SHA256_HEX_RE.test(sha256)) {
+        throw new LunoraError("VALIDATION_ERROR", "`@lunora/storage`: sha256 must be 64 hex characters");
+    }
+};
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/storage/src/create-storage.ts` around lines 323 - 331, Validate
uploadOptions.sha256 near assertContentTypeAllowed before calling
options.bucket.put: accept only a 64-character hexadecimal string or a 32-byte
buffer, and otherwise raise the module’s standard VALIDATION_ERROR while naming
sha256. Keep valid digests unchanged when constructing the put options.

249-269: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Document the allowedContentTypes + missing contentType runtime cost.

allowedContentTypes without contentType now rejects uploads with VALIDATION_ERROR; update the Storage docs package example to pass contentType and add a migration note.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/storage/src/create-storage.ts` around lines 249 - 269, Update the
Storage documentation package example to provide a valid contentType whenever
allowedContentTypes is configured, matching assertContentTypeAllowed. Add a
migration note explaining that uploads omitting contentType now fail at runtime
with VALIDATION_ERROR and must be updated to declare an allowed content type.
packages/cli/src/commands/data-transfer/sources/dump-directory.ts (1)

48-52: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Preserve the underlying readdir failure.

catch(() => undefined) collapses ENOENT, ENOTDIR, and EACCES into one message. Attach the original error as cause so the operator can tell "wrong path" from "no permission".

♻️ Proposed refactor
-    const entries = await readDirectory(directory).catch(() => undefined);
-
-    if (entries === undefined) {
-        throw new LunoraError("INTERNAL", `${directory} is not a readable directory`);
-    }
+    const entries = await readDirectory(directory).catch((error: unknown) => {
+        throw new LunoraError("INTERNAL", `${directory} is not a readable directory`, { cause: error });
+    });
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/cli/src/commands/data-transfer/sources/dump-directory.ts` around
lines 48 - 52, Update the directory-reading error handling around readDirectory
so the caught failure is retained and passed as the cause when constructing
LunoraError. Preserve the existing unreadable-directory message and
undefined-check behavior while allowing callers to distinguish failures such as
ENOENT, ENOTDIR, and EACCES.
packages/cli/src/commands/data-transfer/import.ts (2)

104-127: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win

Report foreign transfers in ImportSummary.storage.

buildImportBody adds storage only when storageIdMap is defined. storageIdMap stays undefined for Supabase and Firebase sources, because those sources produce transferredPaths instead. A foreign import with --with-storage therefore returns a summary with no storage section, and a caller cannot see how many objects moved. ImportSummary is exported from packages/cli/src/commands/data-transfer/index.ts, so this is part of the public type surface.

Consider passing the transferred-path count into buildImportBody and emitting storage for foreign sources too.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/cli/src/commands/data-transfer/import.ts` around lines 104 - 127,
Update buildImportBody and its callers to accept the transferred-path count, and
emit ImportSummary.storage for foreign Supabase and Firebase imports when
--with-storage is used, even when storageIdMap is undefined. Populate the
storage summary with the number of transferred objects while preserving the
existing storage remap fields for sources that provide storageIdMap, and keep
the exported ImportSummary type consistent with both paths.

252-272: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Type the runScan return value.

runScan returns Promise<unknown>. The caller only tests it against undefined, so the loose type is not a defect. A union of the scanner return types documents the contract and keeps the --scan exit-code check honest if a scanner later returns a falsy value.

As per path instructions, packages/**/src/**/*.ts requires "Verify TypeScript types are correct".

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/cli/src/commands/data-transfer/import.ts` around lines 252 - 272,
Update the runScan return type from Promise<unknown> to a union of the concrete
return types from scanStorageColumns, scanFirebaseDump, and scanSupabaseDump,
plus undefined for the default branch. Preserve the existing scanner dispatch
and caller’s undefined check.

Source: Path instructions

packages/cli/src/commands/data-transfer/import-verify.ts (1)

135-145: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Log an error line when verify fails on untransferred paths.

reportStorageOutcome logs logger.error before it returns true. reportUntransferredPaths returns true with only warnings logged. The run then exits 1 while no error line names this cause. Add a matching error line so the operator can see why the verification failed.

♻️ Proposed change
-    return verify && paths.size > 0;
+    if (verify && paths.size > 0) {
+        logger.error(`verify: ${String(paths.size)} storage path(s) were never transferred`);
+
+        return true;
+    }
+
+    return false;
 };
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/cli/src/commands/data-transfer/import-verify.ts` around lines 135 -
145, Update reportUntransferredPaths so that when verify is true and paths is
non-empty, it logs an error identifying the untransferred storage paths before
returning true, matching the error reporting behavior of reportStorageOutcome.
Preserve the existing warning details and return behavior for other cases.
packages/cli/src/commands/data-transfer/sources/storage-transfer.ts (1)

215-235: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Record uploaded keys in alreadyStored to avoid re-uploading identical content.

alreadyStored is built once from the target listing and is never updated. Two source objects with identical bytes produce the same content-hash key. The second object then re-uploads the same key. The result is correct, but the transfer pays for a duplicate upload on every duplicate blob.

♻️ Proposed change
     if (alreadyStored.get(key)?.size !== bytes.length) {
         const metadata: StorageMetadataRow = { contentType: entry.contentType, id: entry.path, sha256, size: bytes.length };
 
         await uploadStorageBlob(context, bytes, metadata, options.keyPrefix, logger);
+        alreadyStored.set(key, { size: bytes.length });
     }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/cli/src/commands/data-transfer/sources/storage-transfer.ts` around
lines 215 - 235, Update moveOneObject to add each successfully handled key to
alreadyStored with its byte length after the upload-or-skip check, so later
source objects with the same content hash avoid re-uploading while preserving
the existing recordTransfer behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@apps/docs/src/content/blog/firestore-typed-values-and-the-export-that-isnt-json.mdx`:
- Around line 46-67: Update the Firestore dump-generation example described at
apps/docs/src/content/blog/firestore-typed-values-and-the-export-that-isnt-json.mdx:46-67
and apps/docs/src/content/docs/migrating/from-firebase.mdx:288-295 to emit REST
Value JSON rather than raw document._fieldsProto. Normalize each typed value,
including protobuf timestamps to RFC-3339 timestampValue strings, before writing
the dump, and keep both documented import examples consistent with that output.

In `@apps/docs/src/content/docs/migrating/from-convex.mdx`:
- Around line 304-307: Update the documented Convex export command in the
migration guide to include the file-storage option, so the resulting
snapshot.zip contains blobs that lunora import --with-storage can migrate.

In `@packages/cli/src/commands/convex-snapshot.ts`:
- Around line 162-169: Update the async iteration around createInterface in the
snapshot entry-stream handling to wrap consumption in a finally block that
closes the readline interface and destroys the underlying stream, including when
the consumer breaks or throws. Preserve the existing raw-line yielding and
undefined-stream return behavior.

In `@packages/cli/src/commands/data-transfer/import-rows.ts`:
- Around line 60-61: Update remapEnvelope to catch JSON.parse failures and
rethrow the same row-scoped LunoraError pattern used by wrapBareDocument,
including the provided lineNumber. Preserve successful envelope parsing and
ensure drainIntoBatcher receives the contextual error instead of a raw
SyntaxError.

In `@packages/cli/src/commands/data-transfer/import.ts`:
- Around line 282-324: Update runForeignStorageTransfer so only source.kind ===
"supabase" can proceed to the live Supabase listing and transfer; keep the
storageDir branch source-independent. For Firebase without --storage-dir, log
guidance to download the bucket with gcloud storage cp -r and provide
--storage-dir, then return undefined before reading Supabase environment
variables.

In `@packages/cli/src/commands/data-transfer/sources/auth.ts`:
- Around line 173-184: Update both auth mappers in
packages/cli/src/commands/data-transfer/sources/auth.ts:173-184 and
packages/cli/src/commands/data-transfer/sources/auth.ts:221-233 so each
account’s _id and id include the user ID, provider, and provider account ID. Use
providerAccountId for the Supabase identities and the resolved accountId for
Firebase providerUserInfo entries, preserving distinct rows for multiple
identities under the same provider.

In `@packages/cli/src/commands/data-transfer/sources/dump-directory.ts`:
- Around line 54-72: Update the mapping resolution in the dump-directory flow
around claimed, found, and readDumpFiles so every mapping entry with a file is
validated against directory entries independently of options.matches. Resolve
claimed basenames to matching files, ensure mapped files are present and
eligible for import, and fail or warn when a claimed file is missing or rejected
rather than silently falling back to discovery; preserve the existing table
association and ensure unresolved mapped tables remain visible to --verify.

In `@packages/cli/src/commands/data-transfer/sources/firebase.ts`:
- Around line 88-96: Update the timestampValue handling in the Firebase value
reshaping logic to preserve RFC-3339 fractional-second digits beyond
milliseconds instead of silently converting them through Date.parse. Keep the
original timestamp string for values with sub-millisecond precision while
retaining current validation and millisecond conversion for compatible
timestamps.
- Around line 201-264: Change the .ndjson handling in readFirestoreCollection
and readCollectionDocuments so the file is consumed as a stream and each
non-empty line is parsed and yielded immediately, rather than loading the full
text or document array. Preserve line trimming, 1-based line numbers, and the
existing invalid-JSON LunoraError message and cause. Keep the existing JSON-file
parsing paths unchanged.
- Around line 149-152: Update the wrapped-shape detection in the surrounding
decoder to use the string-valued name field when fallbackId is available, rather
than treating any object-valued fields property as wrapped. Preserve wrapped
decoding for REST documents and ensure bare community documents—including those
with a fields field—are decoded as complete records.

In `@packages/cli/src/commands/data-transfer/sources/scan.ts`:
- Around line 67-77: Update scanCsvTable to parse the CSV from a file read
stream and stop consuming input after collecting the SCAN_ROW_LIMIT sample,
avoiding readFile and whole-file buffering; follow the existing streaming
approach used by readSupabaseTable. While processing rows, replace the byColumn
array reconstruction with in-place push operations on the existing column
arrays.

In `@packages/cli/src/commands/data-transfer/sources/supabase.ts`:
- Around line 159-173: In the generator’s parser loop, adjust the try/catch
around parser reads so the catch covers only obtaining the next row, not the
subsequent yield to the consumer. Preserve the existing LunoraError context for
parser failures while allowing errors thrown by yield or consumer interaction to
propagate unchanged.
- Around line 121-142: Update the row-mapping logic around the id-column loop to
track explicitly whether the declared idColumn was encountered, rather than
using document["_id"] as the presence check. Set that flag only when column ===
idColumn, keep assigning the declared raw id to document["_id"], and make the
final missing-id error trigger when the flag is false so a non-id source column
named "_id" cannot satisfy the guard.

In `@packages/cli/src/commands/import/index.ts`:
- Around line 29-34: Update the option descriptions for “with-storage” and
“storage-dir” in the import command so they mention all supported Convex,
Supabase, and Firebase storage transfer sources, including that storage-dir also
supplies the Supabase transfer path.

In `@packages/cli/src/commands/zip-entry-stream.ts`:
- Around line 63-91: Update openZipEntryStream to compute CRC-32 for every
non-empty entry while streaming, including STORED and DEFLATED entries, and
compare the final checksum with entry.header.crc. Destroy the returned output
stream with a descriptive error when the checksum differs, while preserving
existing stream and decompression error propagation; add fixtures covering
corrupted stored and deflated archives.

In `@packages/codegen/src/discover-stale-migration-imports.ts`:
- Around line 15-18: Update the Firebase entry in MIGRATED_PLATFORMS to include
the firebase-admin package prefix alongside firebase and `@firebase/`. Ensure
platformFor recognizes firebase-admin and subpath imports such as
firebase-admin/auth as stale Firebase imports.

---

Nitpick comments:
In `@packages/cli/src/commands/data-transfer/import-verify.ts`:
- Around line 135-145: Update reportUntransferredPaths so that when verify is
true and paths is non-empty, it logs an error identifying the untransferred
storage paths before returning true, matching the error reporting behavior of
reportStorageOutcome. Preserve the existing warning details and return behavior
for other cases.

In `@packages/cli/src/commands/data-transfer/import.ts`:
- Around line 104-127: Update buildImportBody and its callers to accept the
transferred-path count, and emit ImportSummary.storage for foreign Supabase and
Firebase imports when --with-storage is used, even when storageIdMap is
undefined. Populate the storage summary with the number of transferred objects
while preserving the existing storage remap fields for sources that provide
storageIdMap, and keep the exported ImportSummary type consistent with both
paths.
- Around line 252-272: Update the runScan return type from Promise<unknown> to a
union of the concrete return types from scanStorageColumns, scanFirebaseDump,
and scanSupabaseDump, plus undefined for the default branch. Preserve the
existing scanner dispatch and caller’s undefined check.

In `@packages/cli/src/commands/data-transfer/sources/dump-directory.ts`:
- Around line 48-52: Update the directory-reading error handling around
readDirectory so the caught failure is retained and passed as the cause when
constructing LunoraError. Preserve the existing unreadable-directory message and
undefined-check behavior while allowing callers to distinguish failures such as
ENOENT, ENOTDIR, and EACCES.

In `@packages/cli/src/commands/data-transfer/sources/reshape.ts`:
- Around line 52-53: Merge EXPONENT_RE and EXPONENT_SPLIT_RE into a single /e/i
constant, remove the duplicate declaration, and update expandExponent and any
other references to use the retained EXPONENT_RE symbol.

In `@packages/cli/src/commands/data-transfer/sources/storage-progress.ts`:
- Around line 85-91: Refactor recordTransfer so the checkpoint directory is
created and the progress file is opened once per transfer run, then reuse the
append-capable file handle for each completed object. Preserve one JSON line per
entry and ensure the handle is closed after the run, rather than performing
mkdir and open/write/close operations for every object.

In `@packages/cli/src/commands/data-transfer/sources/storage-transfer.ts`:
- Around line 215-235: Update moveOneObject to add each successfully handled key
to alreadyStored with its byte length after the upload-or-skip check, so later
source objects with the same content hash avoid re-uploading while preserving
the existing recordTransfer behavior.

In `@packages/runtime/src/storage-admin-routes.ts`:
- Around line 177-180: Update the upload flow around storageUpload so the
SHA-256 digest is computed unconditionally for every request, rather than only
when expectedSha256 or expectedSize is provided, and pass that digest to
storageUpload for recording. If verification-only behavior is intentional
instead, narrow the nearby comment to state that the checksum is recorded only
for uploads requesting verification.

In `@packages/storage/src/create-storage.ts`:
- Around line 323-331: Validate uploadOptions.sha256 near
assertContentTypeAllowed before calling options.bucket.put: accept only a
64-character hexadecimal string or a 32-byte buffer, and otherwise raise the
module’s standard VALIDATION_ERROR while naming sha256. Keep valid digests
unchanged when constructing the put options.
- Around line 249-269: Update the Storage documentation package example to
provide a valid contentType whenever allowedContentTypes is configured, matching
assertContentTypeAllowed. Add a migration note explaining that uploads omitting
contentType now fail at runtime with VALIDATION_ERROR and must be updated to
declare an allowed content type.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: ce027a33-318d-4ef2-9b56-2fa50f48f886

📥 Commits

Reviewing files that changed from the base of the PR and between f54396b and e715a6f.

⛔ Files ignored due to path filters (19)
  • api-snapshots/advisor.api.md is excluded by none and included by none
  • api-snapshots/cli.api.md is excluded by none and included by none
  • api-snapshots/codegen.api.md is excluded by none and included by none
  • api-snapshots/errors.api.md is excluded by none and included by none
  • api-snapshots/runtime.api.md is excluded by none and included by none
  • api-snapshots/storage.api.md is excluded by none and included by none
  • examples/blog/lunora/_generated/app.ts is excluded by !**/_generated/** and included by none
  • examples/team-chat/lunora/_generated/app.ts is excluded by !**/_generated/** and included by none
  • packages/advisor/__tests__/migration-stale-import.test.ts is excluded by !**/__tests__/**, !**/*.test.ts and included by packages/**
  • packages/cli/__tests__/commands/import-batcher.test.ts is excluded by !**/__tests__/**, !**/*.test.ts and included by packages/**
  • packages/cli/__tests__/commands/import-sources.test.ts is excluded by !**/__tests__/**, !**/*.test.ts and included by packages/**
  • packages/cli/__tests__/commands/import-storage.test.ts is excluded by !**/__tests__/**, !**/*.test.ts and included by packages/**
  • packages/cli/__tests__/commands/storage-path-index.test.ts is excluded by !**/__tests__/**, !**/*.test.ts and included by packages/**
  • packages/cli/__tests__/commands/zip-entry-stream.test.ts is excluded by !**/__tests__/**, !**/*.test.ts and included by packages/**
  • packages/codegen/__tests__/fixtures/simple/expected/_generated/app.ts is excluded by !**/_generated/**, !**/__tests__/** and included by packages/**
  • packages/runtime/__tests__/storage-admin.test.ts is excluded by !**/__tests__/**, !**/*.test.ts and included by packages/**
  • plans/305-supabase-firebase-importers.md is excluded by none and included by none
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml and included by none
  • pnpm-workspace.yaml is excluded by none and included by none
📒 Files selected for processing (46)
  • apps/docs/src/content/blog/firestore-typed-values-and-the-export-that-isnt-json.mdx
  • apps/docs/src/content/blog/migrate-convex-files-and-all.mdx
  • apps/docs/src/content/blog/supabase-to-lunora-without-a-reshape-script.mdx
  • apps/docs/src/content/docs/migrating/from-convex.mdx
  • apps/docs/src/content/docs/migrating/from-firebase.mdx
  • apps/docs/src/content/docs/migrating/from-supabase.mdx
  • packages/advisor/src/index.ts
  • packages/advisor/src/lints/static/migration-stale-import.ts
  • packages/advisor/src/stale-migration-imports.ts
  • packages/advisor/src/types.ts
  • packages/cli/package.json
  • packages/cli/src/commands/convex-snapshot.ts
  • packages/cli/src/commands/data-transfer/import-rows.ts
  • packages/cli/src/commands/data-transfer/import-source.ts
  • packages/cli/src/commands/data-transfer/import-verify.ts
  • packages/cli/src/commands/data-transfer/import.ts
  • packages/cli/src/commands/data-transfer/index.ts
  • packages/cli/src/commands/data-transfer/mapping-shapes.ts
  • packages/cli/src/commands/data-transfer/shared.ts
  • packages/cli/src/commands/data-transfer/sources/auth.ts
  • packages/cli/src/commands/data-transfer/sources/dump-directory.ts
  • packages/cli/src/commands/data-transfer/sources/firebase.ts
  • packages/cli/src/commands/data-transfer/sources/mapping.ts
  • packages/cli/src/commands/data-transfer/sources/reshape.ts
  • packages/cli/src/commands/data-transfer/sources/scan.ts
  • packages/cli/src/commands/data-transfer/sources/storage-progress.ts
  • packages/cli/src/commands/data-transfer/sources/storage-transfer.ts
  • packages/cli/src/commands/data-transfer/sources/supabase.ts
  • packages/cli/src/commands/data-transfer/storage-blobs.ts
  • packages/cli/src/commands/data-transfer/storage-mapping.ts
  • packages/cli/src/commands/data-transfer/storage-path-index.ts
  • packages/cli/src/commands/data-transfer/storage-remap.ts
  • packages/cli/src/commands/import/handler.ts
  • packages/cli/src/commands/import/index.ts
  • packages/cli/src/commands/seed/handler.ts
  • packages/cli/src/commands/zip-entry-stream.ts
  • packages/codegen/src/advisor.ts
  • packages/codegen/src/discover-stale-migration-imports.ts
  • packages/codegen/src/emit-app.ts
  • packages/codegen/src/ir.ts
  • packages/codegen/src/run-codegen.ts
  • packages/errors/src/catalog.ts
  • packages/runtime/src/create-worker.ts
  • packages/runtime/src/storage-admin-routes.ts
  • packages/storage/src/create-storage.ts
  • packages/storage/src/types.ts

@codecov-commenter

codecov-commenter commented Aug 7, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.18919% with 116 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (alpha@262eccb). Learn more about missing BASE report.

Files with missing lines Patch % Lines
.../cli/src/commands/data-transfer/sources/reshape.ts 78.94% 24 Missing ⚠️
...ges/cli/src/commands/data-transfer/sources/auth.ts 88.73% 13 Missing and 3 partials ⚠️
...cli/src/commands/data-transfer/sources/firebase.ts 90.90% 10 Missing and 1 partial ⚠️
...ges/cli/src/commands/data-transfer/sources/scan.ts 84.93% 11 Missing ⚠️
packages/cli/src/commands/data-transfer/import.ts 88.50% 10 Missing ⚠️
...commands/data-transfer/sources/storage-transfer.ts 90.56% 10 Missing ⚠️
...es/cli/src/commands/data-transfer/storage-blobs.ts 81.81% 6 Missing ⚠️
...es/cli/src/commands/data-transfer/import-source.ts 87.50% 4 Missing ⚠️
.../cli/src/commands/data-transfer/sources/mapping.ts 91.30% 4 Missing ⚠️
...c/commands/data-transfer/sources/dump-directory.ts 89.28% 3 Missing ⚠️
... and 9 more
Additional details and impacted files
@@           Coverage Diff            @@
##             alpha     #357   +/-   ##
========================================
  Coverage         ?   87.04%           
========================================
  Files            ?     1157           
  Lines            ?    62128           
  Branches         ?    15312           
========================================
  Hits             ?    54079           
  Misses           ?     7528           
  Partials         ?      521           
Files with missing lines Coverage Δ
packages/advisor/src/index.ts 100.00% <ø> (ø)
...advisor/src/lints/static/migration-stale-import.ts 100.00% <100.00%> (ø)
packages/cli/src/commands/data-transfer/shared.ts 100.00% <ø> (ø)
.../cli/src/commands/data-transfer/storage-mapping.ts 91.78% <100.00%> (ø)
...es/cli/src/commands/data-transfer/storage-remap.ts 100.00% <100.00%> (ø)
packages/cli/src/commands/import/index.ts 25.00% <ø> (ø)
packages/cli/src/commands/seed/handler.ts 84.09% <100.00%> (ø)
packages/cli/src/commands/zip-entry-stream.ts 100.00% <100.00%> (ø)
packages/codegen/src/advisor.ts 79.54% <ø> (ø)
...es/codegen/src/discover-stale-migration-imports.ts 100.00% <100.00%> (ø)
... and 24 more
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codspeed

codspeed Bot commented Aug 7, 2026 •

Copy link
Copy Markdown

Merging this PR will regress 1 benchmark

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 2 improved benchmarks
❌ 1 regressed benchmark
✅ 250 untouched benchmarks
⏩ 10 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
❌ in-batch: single IN(...) query + id->doc re-projection 812.6 µs 950.9 µs -14.54%
⚡ findMany — plain table (no scope) 87.6 ms 77.8 ms +12.56%
⚡ count, no attributes 67 µs 60.5 µs +10.8%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing feat/supabase-firebase-importers (e3ed7d0) with alpha (8e5d203)2

Open in CodSpeed

Footnotes

  1. 10 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. No successful run was found on alpha (262eccb) during the generation of this report, so 8e5d203 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

prisis and others added 2 commits August 7, 2026 15:09
Seven review findings, each verified against the code first.

`supabase.ts` — the `try` wrapped the `yield`, so a generator resumes at
its yield and an error thrown by the CONSUMER came back relabelled
`<file> row N: …`, a parse-location message for something that never
touched the parser. Reading and decoding a row stay attributed; the yield
now sits outside.

`supabase.ts` — a source column literally named `_id` landed in the
target's reserved id field and satisfied the "did we find an id?" guard,
so a table whose real `idColumn` was absent imported with the wrong value
as its id — the dangling-FK failure that guard exists to prevent. The
guard now tracks whether the id column was actually seen, and a
non-id `_id` column is a named error rather than a silent collision.

`firebase.ts` — a BARE document with a field called `fields` was read as
the wrapped REST form, so every other field was dropped. `name` stays the
reliable signal; without one, the tie-break is what `fields` holds — a map
of name→`Value` (wrapped) versus a single one-key `Value` (bare) — and
anything alongside it that is not wrapper metadata settles it as bare.

`import.ts` — a Firebase source with `--with-storage` and no
`--storage-dir` fell through to the Supabase remote path and was told to
set `SUPABASE_URL`, for a migration with no Supabase in it. There is no
remote Firebase download path, so it now says to pull the bucket first.

`import-rows.ts` — a malformed NDJSON line threw a bare `Unexpected token`
with no way to find it in a multi-GB file, while every other failure on
that path names its line.

`discover-stale-migration-imports.ts` — the matcher is exact-or-`prefix/`,
so `firebase-admin` (the SDK a migrated backend actually imported) never
matched `firebase`. Listed explicitly, with the test that was missing:
the matcher had none, which is why the gap existed.

`import/index.ts` — `--with-storage` still described itself as Convex-only.

cli 1168, codegen 1068.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`convex-snapshot.ts` — a consumer that stops early (a `break`, a throw, an
import failing on row 12 of a million) left the readline interface and the
ZIP entry stream open, holding the archive's file handle for the rest of
the process. Closed in a `finally`.

`auth.ts` — both mappers keyed an account on `${user}:${provider}`, so a
user holding two identities at the SAME provider collapsed into one id:
the second silently replacing the first, or the import failing on a
duplicate. The provider account id is part of the key now, with a test
that imports two github identities for one user and asserts both survive.

`dump-directory.ts` — a mapping entry whose `file` is absent from the
directory, or present but filtered out (wrong extension), was dropped
without a word. The operator named that table expecting it to migrate, and
`--verify` still passed because it compares what WAS imported. Now a named
error listing each unresolved file and its table.

`scan.ts` — `--scan` read the WHOLE dump into memory to look at 200 rows
(`toLine` bounds what the parser emits, not what is read), and rebuilt the
per-column array on every value, so sampling N rows cost O(N²) copies. It
streams, stops at the sample size, destroys the handle, and accumulates in
place.

`firebase.ts` — the `timestampValue` truncation to milliseconds is now
stated where it happens and in the migration guide. Keeping the RFC-3339
string for sub-millisecond rows was the suggested fix and is worse: the
column would be number for most rows and string for a few, which no schema
describes. Firestore stores microseconds; a Lunora timestamp is
milliseconds; an app that needs the rest should carry it in its own column.

cli 1169.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
prisis and others added 4 commits August 7, 2026 15:23
The documented Firestore dump script reads `document._fieldsProto`, and
that is the Admin SDK's own representation — *almost* the REST encoding,
not exactly it. `Timestamp.toProto()` returns
`{ timestampValue: { seconds, nanos } }` where REST returns an RFC-3339
string, and bytes arrive as a `Buffer` rather than base64.

The decoder accepted only the REST spelling, so `Date.parse` ran against
an object, returned NaN, and threw "is not an RFC-3339 timestamp" — on the
first document with a `createdAt`. The fifteen-line script the blog post
advertises did not produce a dump this importer could read.

Both spellings are accepted now, with a test that imports the
`_fieldsProto` shape and asserts the decoded millisecond value and base64.
The blog post and the guide say which is which rather than implying the
two are the same encoding.

Also: the guide's `snapshot.zip` line omitted `--include-file-storage`, so
following it produced an archive with no blobs — and then `--with-storage`
had nothing to migrate, while the CLI's own `--verify` error tells you the
export needed that flag. The directory command already had it; the ZIP one
now matches.

Finally, `.ndjson` collections stream a line at a time instead of reading
the file into a string and then building an array of every document — two
full copies of a collection that can be tens of gigabytes. The `.json`
container shapes are single JSON values and still parse whole.

cli 1170.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Lint follow-up to the previous commit: the added ternaries pushed
`decodeValue` past the cognitive-complexity limit, and each branch had
grown a paragraph of context that reads better next to the code it
explains than inside a fifteen-way dispatch.

No behaviour change — cli 1170.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The archive already carries the checksum its writer computed, and nothing
compared it. A corrupted or truncated entry imported as data — silently,
with row counts that still matched, so `--verify` reported nothing either:
it compares counts, not contents.

Both paths are checked. STORED bytes are the entry's contents already;
DEFLATED ones are checked after inflating, which is what the header's CRC
describes. `zlib.crc32` is the same polynomial ZIP uses (Node ≥22.2, and
this repo requires ^22.15), so this is a comparison rather than a
reimplementation.

The test corrupts a good archive's recorded checksum and asserts the
stream rejects rather than yielding the bytes.

cli 1171.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Presented as literal CLI output, so it should be. The real message joins
both clauses with an em dash; the post had a comma in the second one.

Checked the rest of the three posts against the code rather than reading
them for tone. Accurate: the Postgres NULL-vs-empty-string distinction
(`castPostgresCsv`), the digit-loss refusal, `integerValue` staying a
string outside the safe range (`Number.isSafeInteger(parsed) ? parsed :
raw`), SHA-256 content keying, the 32 MiB verified-upload cap, the
signed-PUT fallback, and deleting an object that fails verification.
`--pg-url` appears only as a rejected alternative, not a claimed flag.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/cli/src/commands/zip-entry-stream.ts`:
- Around line 117-125: Validate zero-sized entries before the existing CRC/read
bypass in the zip-entry stream flow, rejecting any entry whose declared
uncompressedSize is nonzero or whose header.crc is nonzero. Preserve the empty
result only when all zero-size metadata is consistent, and add a test that
modifies a non-empty entry to declare zero compressed bytes.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 23831844-9f54-49a6-b0f5-4fd88f98ae30

📥 Commits

Reviewing files that changed from the base of the PR and between e715a6f and b8f03fb.

⛔ Files ignored due to path filters (3)
  • packages/cli/__tests__/commands/import-sources.test.ts is excluded by !**/__tests__/**, !**/*.test.ts and included by packages/**
  • packages/cli/__tests__/commands/zip-entry-stream.test.ts is excluded by !**/__tests__/**, !**/*.test.ts and included by packages/**
  • packages/codegen/__tests__/discover-stale-migration-imports.test.ts is excluded by !**/__tests__/**, !**/*.test.ts and included by packages/**
📒 Files selected for processing (15)
  • apps/docs/src/content/blog/firestore-typed-values-and-the-export-that-isnt-json.mdx
  • apps/docs/src/content/blog/supabase-to-lunora-without-a-reshape-script.mdx
  • apps/docs/src/content/docs/migrating/from-convex.mdx
  • apps/docs/src/content/docs/migrating/from-firebase.mdx
  • packages/cli/src/commands/convex-snapshot.ts
  • packages/cli/src/commands/data-transfer/import-rows.ts
  • packages/cli/src/commands/data-transfer/import.ts
  • packages/cli/src/commands/data-transfer/sources/auth.ts
  • packages/cli/src/commands/data-transfer/sources/dump-directory.ts
  • packages/cli/src/commands/data-transfer/sources/firebase.ts
  • packages/cli/src/commands/data-transfer/sources/scan.ts
  • packages/cli/src/commands/data-transfer/sources/supabase.ts
  • packages/cli/src/commands/import/index.ts
  • packages/cli/src/commands/zip-entry-stream.ts
  • packages/codegen/src/discover-stale-migration-imports.ts
🚧 Files skipped from review as they are similar to previous changes (12)
  • apps/docs/src/content/docs/migrating/from-convex.mdx
  • packages/cli/src/commands/convex-snapshot.ts
  • packages/codegen/src/discover-stale-migration-imports.ts
  • packages/cli/src/commands/import/index.ts
  • packages/cli/src/commands/data-transfer/import-rows.ts
  • packages/cli/src/commands/data-transfer/sources/supabase.ts
  • packages/cli/src/commands/data-transfer/sources/auth.ts
  • apps/docs/src/content/docs/migrating/from-firebase.mdx
  • packages/cli/src/commands/data-transfer/sources/scan.ts
  • apps/docs/src/content/blog/supabase-to-lunora-without-a-reshape-script.mdx
  • packages/cli/src/commands/data-transfer/import.ts
  • packages/cli/src/commands/data-transfer/sources/firebase.ts

Follow-up to the CRC check in 853122b, which left a hole in itself: the
`compressedSize === 0` early return ran BEFORE any verification, so a
header claiming zero compressed bytes skipped the checksum entirely. The
entry read as empty and imported as no rows — silent wrong data, which is
the case the CRC exists to stop, reached by the one path that never
consults it.

An empty entry is empty in all three fields, so the early return now
requires that: zero compressed, zero uncompressed, zero CRC. Anything else
claiming zero compression is a corrupt header and says so.

The genuinely-empty case still returns `undefined` — its test is
unchanged and still passes. The new one takes a good archive and rewrites
`compressedSize` to 0.

cli 1172.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@prisis
prisis merged commit 8e5d203 into alpha Aug 7, 2026
38 checks passed
@prisis
prisis deleted the feat/supabase-firebase-importers branch August 7, 2026 13:47
@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using our GitHub Discussions tab for questions.

@github-actions github-actions Bot locked as resolved and limited conversation to collaborators Sep 8, 2026

This branch was previously deployed

1 inactive deployment
benchmarks — e3ed7d0e Deployed Aug 7, 2026 by prisis via Benchmarks #1501
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants