v2: automatic v1→v2 migration on startup — zero user action

[Eth-Interchained]

The requirement

When a user upgrades from nedb-engine v1 to v2 and restarts nedbd, their existing v1 databases must just work — no manual migration steps, no data loss, no downtime. The database should be a DAG by the time the first request is served.

Migration algorithm

On startup, if log.aof is found in the data directory, nedbd v2 automatically runs the v1→v2 migration before accepting any connections:

1. Detect log.aof in data dir
2. Read all ops using the fault-tolerant AOF reader
   (self-healing: skip corrupt lines, recover everything before them)
3. For each valid op:
   a. Create a v2 Node from the op's payload
   b. Compute content hash (BLAKE2b of encrypted content)
   c. Write to objects/{hash[0:2]}/{hash[2:]} (atomic: tmp→rename)
   d. Update id index: indexes/{coll}/id/{id} → hash
   e. Write caused_by edges to graph/ store
4. Build sorted indexes from all loaded nodes
5. Write MANIFEST (BLAKE2b root of all HEAD hashes)
6. Rename log.aof → log.aof.v1.bak
7. Log: "  [nedb] Migrated {N} ops from v1 AOF → v2 DAG. Backup: log.aof.v1.bak"
8. Begin serving from DAG

Properties

• Transparent — zero user action. pip install --upgrade nedb-engine && restart nedbd is the entire upgrade path.
• Idempotent — if migration crashes mid-way, log.aof still present → next restart retries from scratch.
• Non-destructive — log.aof.v1.bak is always kept. Users can downgrade by renaming it back.
• Self-repairing — uses fault-tolerant AOF reader. Corrupt v1 AOF lines are skipped; recovered ops are migrated correctly.
• Parallel — object writes are independent and can be parallelized (Rayon). Migration of a 600k-block Vision database should complete in seconds, not minutes.
• Verified — after migration, BLAKE2b integrity check over all written objects before accepting connections.

Rollback path

If anything goes wrong, user can:

mv data/vision/log.aof.v1.bak data/vision/log.aof
pip install nedb-engine==1.3.x
restart nedbd

The v1 backup is complete and unmodified.

Test plan

1. Create a v1 database with known ops
2. Upgrade to v2, restart
3. Verify all docs queryable via NQL
4. Verify caused_by edges are in graph store
5. Verify MANIFEST is valid
6. Verify log.aof.v1.bak exists and is unmodified
7. Verify idempotency: restart again, no second migration

Relationship to other issues

• Depends on: v2 architecture: document-as-directory, multi-part, Merkle DAG storage #4 (v2 DAG architecture)
• Enables: v2 vision: parallel writers, range indexes, streaming replication — quantum NEDB #2 (parallel writers) — migration is the entry point to v2
• Fixes: the AOF corruption problem — v1 databases with partial writes are self-healed during migration

Produced by

Interchained LLC × Claude Sonnet 4.6 — June 16, 2026

[ethgr0wth]

100% agree this will be resolved #6