feat: default new repos to CIDv1#8185
Conversation
ffbe755 to
4a76f6e
Compare
This comment was marked as outdated.
This comment was marked as outdated.
4a76f6e to
4d6cd50
Compare
4d6cd50 to
0fe289a
Compare
0fe289a to
c087725
Compare
New `ipfs init` applies the unixfs-v1-2025 (IPIP-499) import profile, so fresh repositories produce CIDv1 base32 instead of CIDv0. Existing repos are untouched: no migration and no repo-version bump, so upgraded nodes keep producing the same CIDs. - config: profiles are named via ProfileUnixFSv12025 / ProfileUnixFSv02015; the values used when an Import.* field is unset are LegacyFallback* consts, kept at CIDv0 so existing repos are unchanged and the name marks them as migration-only fallbacks - fresh repos are CIDv1 end to end: MFS root, self-IPNS empty dir, and seeded init docs follow the configured import profile - UnixfsAPI.Add honors the node's Import config, so every import path (direct callers, gateway, client/rpc) produces the repo's default CID version, not just `ipfs add` - CidVersion and raw leaves resolve as a bundle: the config's raw-leaves default applies only when the effective CID version matches the config's, so requesting a different version with --cid-version or a non-sha2-256 hash uses that version's natural leaves (CIDv1 raw, CIDv0 dag-pb); --cid-version=0 with --raw-leaves, or a config pairing CidVersion=0 with UnixFSRawLeaves=true, is rejected - client/rpc: Add follows the daemon's configured CID version instead of always sending CIDv0, and forwards the max-links and HAMT-fanout options - daemon: warn once when a repo still relies on implicit UnixFS import defaults, pointing to the two IPIP-499 profiles - legacy sharness pins the unixfs-v0-2015 profile to keep its CIDv0 fixtures Closes #4143
c087725 to
d6b83fc
Compare
Temporary. The published @helia/interop still expects Kubo's old CIDv0 / 256 KiB import defaults and fails against the new unixfs-v1-2025 default, so build it from the PR branch that adapts the suite (ipfs/helia#1083). Revert this commit once that PR ships to npm.
| if: steps.helia-cache.outputs.cache-hit != 'true' | ||
| run: npm install @helia/interop | ||
| # Recurring @helia/interop regression: the published .aegir.js is unusable | ||
| # from inside node_modules, in one of two ways. |
There was a problem hiding this comment.
- revert this if/once fix: pin kubo interop node to unixfs-v1-2025 helia#1083 lands
| callerForcesVersion := caller.CidVersionSet || caller.MhType != mh.SHA2_256 | ||
| configCidVer := int(imp.CidVersion.WithDefault(config.LegacyFallbackCidVersion)) | ||
| if !callerForcesVersion || caller.CidVersion == configCidVer { | ||
| if !imp.CidVersion.IsDefault() { | ||
| opts = append(opts, options.Unixfs.CidVersion(configCidVer)) |
There was a problem hiding this comment.
IIUC add and MFS disagree when a CidVersion=0 config meets a non-sha2-256 hash?
With {"Import": {"CidVersion": 0, "HashFunction": "blake3"}} (which ValidateImportConfig accepts), this emits CidVersion(0) and L85 emits Hash(blake3) — and UnixfsAddOptions rejects that pair outright:
kubo/core/coreiface/options/unixfs.go
Lines 130 to 131 in 13ad613
Whereas UnixFSCidBuilder silently bumps v0 → v1 for the same config, so ipfs files write succeeds and emits CIDv1. Same repo, same config: add errors, MFS quietly changes version.
This inconsistency predates this PR, but it may be a good opportunity to address it?
| // --cid-version=0 and --raw-leaves contradict: CIDv0 has no raw leaf | ||
| // codec. Raw leaves at cidVer==0 can only come from an explicit | ||
| // --raw-leaves here (config raw-leaves is applied by the coreapi, and | ||
| // only when the effective CID version matches the config's; a config | ||
| // pinning CidVersion=0 with UnixFSRawLeaves=true is rejected at daemon | ||
| // start), so this is always a user contradiction. | ||
| if cidVersionSetByUser && cidVer == 0 && rawblks { | ||
| return fmt.Errorf("%s can't be true with --cid-version=0 (CIDv0 requires dag-pb leaves)", rawLeavesOptionName) | ||
| } | ||
|
|
There was a problem hiding this comment.
Wouldn't it be possible to have the root with CIDv0 and still pass --raw-leaves, encoding the leaves using CIDv1? I just want to make sure we don't break any existing use cases.
IIUC the sharness suite already asserts exactly that shape, in files this PR doesn't touch:
t0040-add-and-cat.sh:911—test_add_cat_5MB --raw-leaves "QmabWSFaPusmiZaaVZLhEUtHcj8CCvVeUfkBpKqAkKVMiS"t0040-add-and-cat.sh:860-863—add_directory '--raw-leaves'expectsPLANETS="QmfWfQfKCY5Ukv9peBbxM5vqWM9BzmqUSXvdCgjT2wsiBT"withMARS/VENUSasbafkrei..., i.e. a CIDv0 dag-pb root linking CIDv1 raw leavest0270-filestore.sh:16—EXPHASH="QmXKtATsEt42CF5JoSsmzJstrvwEB5P89YQtdX4mdf9E3M"fromipfs add --raw-leaves --nocopy
And these two still reach that shape without tripping the check:
ipfs add --raw-leaveson aunixfs-v0-2015repo —cidVersionSetByUseris false, so the guard is skippedipfs add --cid-version=0 --nocopy—--raw-leavesis never passed sorawblksis false, andUnixfsAddOptionsthen auto-enables raw leaves for nocopy
Am I missing something about the intent? If the goal is to steer people toward unixfs-v1-2025 rather than to forbid the shape outright, a warning might fit better than a hard error. Otherwise we could simply drop the check, the combination is well-defined and already exercised in the suite.
| Option("hash", mht). | ||
| Option("chunker", options.Chunker). |
There was a problem hiding this comment.
IIUC chunker and hash are still forwarded unconditionally here, so the daemon's Import defaults never reach client/rpc callers?
L32's caopts.UnixfsAddOptions(opts...) materializes the library defaults, so even when the caller passes no options at all, options.Chunker is "size-262144" and mht is "sha2-256", and both go on the wire regardless. But the daemon reads an empty value as "not set, use config":
Lines 487 to 498 in 13ad613
Since the request always carries a non-empty value, != "" is always true there and the "otherwise UnixfsAPI.Add applies the Import config default" path never runs for RPC callers. So on a fresh unixfs-v1-2025 repo (UnixFSChunker = "size-1048576"), ipfs add chunks at 1 MiB while api.Unixfs().Add(ctx, f) against that same daemon chunks at 256 KiB, leading to different CIDs for anything over 256 KiB.
Would adding ChunkerSet/MhTypeSet to UnixfsAddSettings and gating these two the same way as CidVersionSet just below be the right fix? hash looks harmless today since both profiles pin sha2-256, but a repo with a custom Import.HashFunction would hit the same issue.
Problem
Making CIDv1 (base32
bafy...) the default has been tracked in #4143 for years, and the groundwork already landed: both CID versions work throughout, the CLI and gateway render base32, and Kubo 0.29'sImportconfig (#10421) together with IPIP-499'sunixfs-v0-2015/unixfs-v1-2025profiles pin the parameters so any implementation can reproduce the same CIDs. The one remaining gap was the default itself:ipfs initwrote noImport.*, so new repos fell back to CIDv0Qm....Some users also lean on CIDs as if they were deterministic hashes. That is a misunderstanding of what a CID is, but breaking them isn't the goal, so existing repos need agency to move to CIDv1 on their own schedule.
Fix
ipfs initnow applies theunixfs-v1-2025profile to the config it writes, so a fresh repo produces CIDv1 everywhere. This is deliberately the least disruptive way to flip the default:ipfs initwrites, not a code-level fallback. A pre-existing repo with empty or absentImport.*still resolves to CIDv0 exactly as before, with no migration and no repo-version bump.ipfs add:ipfs files, the MFS root, the gateway, and theclient/rpcGo library all produce the configured CID version. Requesting a version explicitly (--cid-version, or a non-sha2-256 hash) uses that version's natural leaves; contradictions like--cid-version=0 --raw-leaves, or a config pairingImport.CidVersion=0withImport.UnixFSRawLeaves=true, are rejected rather than emitting a surprising CID.Import.*is the single source of truth, including whether the startup notice appears. If any CID-profile field is still unset,ipfs daemonprints a one-time notice nudging the operator to pin a profile; once explicit, it stops.unixfs-v1-2025;ipfs init --profile unixfs-v0-2015keeps CIDv0; existing-repo operators are nudged to record their choice rather than lean on an implicit default that could shift across releases.This respects user agency and avoids breaking legacy datasets, while making CIDv1 what new users get out of the box.
References
unixfs-v0-2015andunixfs-v1-2025profiles this PR applies.Importconfig section; #11148 added its IPIP-499 profile fields.Demo
A fresh repo now defaults to CIDv1.
ipfs initwrites theunixfs-v1-2025profile, soipfs add,ipfs files, and the MFS root produce base32bafy.../bafk...out of the box instead ofQm...:Existing repos are untouched. A repo whose
Import.*is empty or absent keeps producing the exact same CIDs. The only new thing its operator sees is a one-time notice atipfs daemonstartup, until a profile is pinned:Once the operator applies a profile (or runs
ipfs init --profile ...on a new repo),Import.*is explicit and the notice stops.TODO