Skip to content

feat: default new repos to CIDv1#8185

Draft
lidel wants to merge 2 commits into
masterfrom
feat/puts-return-cidv1
Draft

feat: default new repos to CIDv1#8185
lidel wants to merge 2 commits into
masterfrom
feat/puts-return-cidv1

Conversation

@lidel

@lidel lidel commented Jun 11, 2021

Copy link
Copy Markdown
Member

Problem

Making CIDv1 (base32 bafy...) the default has been tracked in #4143 for years, and the groundwork already landed: both CID versions work throughout, the CLI and gateway render base32, and Kubo 0.29's Import config (#10421) together with IPIP-499's unixfs-v0-2015 / unixfs-v1-2025 profiles pin the parameters so any implementation can reproduce the same CIDs. The one remaining gap was the default itself: ipfs init wrote no Import.*, so new repos fell back to CIDv0 Qm....

Some users also lean on CIDs as if they were deterministic hashes. That is a misunderstanding of what a CID is, but breaking them isn't the goal, so existing repos need agency to move to CIDv1 on their own schedule.

Fix

ipfs init now applies the unixfs-v1-2025 profile to the config it writes, so a fresh repo produces CIDv1 everywhere. This is deliberately the least disruptive way to flip the default:

  • Only fresh repos change. The switch lives in what ipfs init writes, not a code-level fallback. A pre-existing repo with empty or absent Import.* still resolves to CIDv0 exactly as before, with no migration and no repo-version bump.
  • Every import path follows the repo's config, not just ipfs add: ipfs files, the MFS root, the gateway, and the client/rpc Go library all produce the configured CID version. Requesting a version explicitly (--cid-version, or a non-sha2-256 hash) uses that version's natural leaves; contradictions like --cid-version=0 --raw-leaves, or a config pairing Import.CidVersion=0 with Import.UnixFSRawLeaves=true, are rejected rather than emitting a surprising CID.
  • Import.* is the single source of truth, including whether the startup notice appears. If any CID-profile field is still unset, ipfs daemon prints a one-time notice nudging the operator to pin a profile; once explicit, it stops.
  • Users choose, explicitly. New repos get unixfs-v1-2025; ipfs init --profile unixfs-v0-2015 keeps CIDv0; existing-repo operators are nudged to record their choice rather than lean on an implicit default that could shift across releases.

This respects user agency and avoids breaking legacy datasets, while making CIDv1 what new users get out of the box.

References

Demo

A fresh repo now defaults to CIDv1. ipfs init writes the unixfs-v1-2025 profile, so ipfs add, ipfs files, and the MFS root produce base32 bafy.../bafk... out of the box instead of Qm...:

$ ipfs init          # writes the unixfs-v1-2025 profile
$ ipfs config Import.CidVersion
1
$ echo -n "hello, CIDv1!" | ipfs add -q
bafkreifriwiigahgxhtea4usek2ew2hr4vtnzsj7m6jy6n5iusfporvs2e   # was QmNbyD5G... before

Existing repos are untouched. A repo whose Import.* is empty or absent keeps producing the exact same CIDs. The only new thing its operator sees is a one-time notice at ipfs daemon startup, until a profile is pinned:

$ ipfs daemon
Initializing daemon...
...
⚠️ This repository relies on implicit UnixFS Import.* config defaults,
⚠️ so the CIDs it produces can change between Kubo releases. Pin
⚠️ them by applying a profile (this also silences this notice):
⚠️   $ ipfs config profile apply unixfs-v1-2025   # recommended, modern CIDv1
⚠️   $ ipfs config profile apply unixfs-v0-2015   # legacy, CIDv0 "Qm.." format
⚠️ Use the legacy profile only to keep reproducing existing CIDs.
⚠️ See IPIP-499: https://specs.ipfs.tech/ipips/ipip-0499/
...
Daemon is ready

Once the operator applies a profile (or runs ipfs init --profile ... on a new repo), Import.* is explicit and the notice stops.

TODO

@lidel
lidel force-pushed the feat/puts-return-cidv1 branch from ffbe755 to 4a76f6e Compare April 7, 2022 23:44
@lidel lidel changed the title feat: unixfs imports produce cidv1 test: ipfs add with CIDv1 as default Apr 7, 2022
@schomatis schomatis self-assigned this Apr 14, 2022
@schomatis

This comment was marked as outdated.

@lidel lidel mentioned this pull request Jul 7, 2026
46 tasks
@lidel
lidel force-pushed the feat/puts-return-cidv1 branch from 4a76f6e to 4d6cd50 Compare July 7, 2026 20:27
@lidel lidel changed the title test: ipfs add with CIDv1 as default feat: default new repos to CIDv1 Jul 7, 2026
@lidel
lidel force-pushed the feat/puts-return-cidv1 branch from 4d6cd50 to 0fe289a Compare July 8, 2026 18:51
@lidel
lidel force-pushed the feat/puts-return-cidv1 branch from 0fe289a to c087725 Compare July 8, 2026 20:35
New `ipfs init` applies the unixfs-v1-2025 (IPIP-499) import profile, so
fresh repositories produce CIDv1 base32 instead of CIDv0. Existing repos
are untouched: no migration and no repo-version bump, so upgraded nodes
keep producing the same CIDs.

- config: profiles are named via ProfileUnixFSv12025 / ProfileUnixFSv02015;
  the values used when an Import.* field is unset are LegacyFallback*
  consts, kept at CIDv0 so existing repos are unchanged and the name marks
  them as migration-only fallbacks
- fresh repos are CIDv1 end to end: MFS root, self-IPNS empty dir, and
  seeded init docs follow the configured import profile
- UnixfsAPI.Add honors the node's Import config, so every import path
  (direct callers, gateway, client/rpc) produces the repo's default CID
  version, not just `ipfs add`
- CidVersion and raw leaves resolve as a bundle: the config's raw-leaves
  default applies only when the effective CID version matches the config's,
  so requesting a different version with --cid-version or a non-sha2-256
  hash uses that version's natural leaves (CIDv1 raw, CIDv0 dag-pb);
  --cid-version=0 with --raw-leaves, or a config pairing CidVersion=0 with
  UnixFSRawLeaves=true, is rejected
- client/rpc: Add follows the daemon's configured CID version instead of
  always sending CIDv0, and forwards the max-links and HAMT-fanout options
- daemon: warn once when a repo still relies on implicit UnixFS import
  defaults, pointing to the two IPIP-499 profiles
- legacy sharness pins the unixfs-v0-2015 profile to keep its CIDv0 fixtures

Closes #4143
Temporary. The published @helia/interop still expects Kubo's old CIDv0 /
256 KiB import defaults and fails against the new unixfs-v1-2025 default,
so build it from the PR branch that adapts the suite (ipfs/helia#1083).
Revert this commit once that PR ships to npm.
@lidel
lidel marked this pull request as ready for review July 9, 2026 18:22
@lidel
lidel requested a review from a team as a code owner July 9, 2026 18:22
if: steps.helia-cache.outputs.cache-hit != 'true'
run: npm install @helia/interop
# Recurring @helia/interop regression: the published .aegir.js is unusable
# from inside node_modules, in one of two ways.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread core/coreapi/unixfs.go
Comment on lines +60 to +64
callerForcesVersion := caller.CidVersionSet || caller.MhType != mh.SHA2_256
configCidVer := int(imp.CidVersion.WithDefault(config.LegacyFallbackCidVersion))
if !callerForcesVersion || caller.CidVersion == configCidVer {
if !imp.CidVersion.IsDefault() {
opts = append(opts, options.Unixfs.CidVersion(configCidVer))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC add and MFS disagree when a CidVersion=0 config meets a non-sha2-256 hash?

With {"Import": {"CidVersion": 0, "HashFunction": "blake3"}} (which ValidateImportConfig accepts), this emits CidVersion(0) and L85 emits Hash(blake3) — and UnixfsAddOptions rejects that pair outright:

case 0:
return nil, cid.Prefix{}, errors.New("CIDv0 only supports sha2-256")

Whereas UnixFSCidBuilder silently bumps v0 → v1 for the same config, so ipfs files write succeeds and emits CIDv1. Same repo, same config: add errors, MFS quietly changes version.

This inconsistency predates this PR, but it may be a good opportunity to address it?

Comment thread core/commands/add.go
Comment on lines +407 to +416
// --cid-version=0 and --raw-leaves contradict: CIDv0 has no raw leaf
// codec. Raw leaves at cidVer==0 can only come from an explicit
// --raw-leaves here (config raw-leaves is applied by the coreapi, and
// only when the effective CID version matches the config's; a config
// pinning CidVersion=0 with UnixFSRawLeaves=true is rejected at daemon
// start), so this is always a user contradiction.
if cidVersionSetByUser && cidVer == 0 && rawblks {
return fmt.Errorf("%s can't be true with --cid-version=0 (CIDv0 requires dag-pb leaves)", rawLeavesOptionName)
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it be possible to have the root with CIDv0 and still pass --raw-leaves, encoding the leaves using CIDv1? I just want to make sure we don't break any existing use cases.

IIUC the sharness suite already asserts exactly that shape, in files this PR doesn't touch:

  • t0040-add-and-cat.sh:911test_add_cat_5MB --raw-leaves "QmabWSFaPusmiZaaVZLhEUtHcj8CCvVeUfkBpKqAkKVMiS"
  • t0040-add-and-cat.sh:860-863add_directory '--raw-leaves' expects PLANETS="QmfWfQfKCY5Ukv9peBbxM5vqWM9BzmqUSXvdCgjT2wsiBT" with MARS/VENUS as bafkrei..., i.e. a CIDv0 dag-pb root linking CIDv1 raw leaves
  • t0270-filestore.sh:16EXPHASH="QmXKtATsEt42CF5JoSsmzJstrvwEB5P89YQtdX4mdf9E3M" from ipfs add --raw-leaves --nocopy

And these two still reach that shape without tripping the check:

  • ipfs add --raw-leaves on a unixfs-v0-2015 repo — cidVersionSetByUser is false, so the guard is skipped
  • ipfs add --cid-version=0 --nocopy--raw-leaves is never passed so rawblks is false, and UnixfsAddOptions then auto-enables raw leaves for nocopy

Am I missing something about the intent? If the goal is to steer people toward unixfs-v1-2025 rather than to forbid the shape outright, a warning might fit better than a hard error. Otherwise we could simply drop the check, the combination is well-defined and already exercised in the suite.

Comment thread client/rpc/unixfs.go
Comment on lines 43 to 44
Option("hash", mht).
Option("chunker", options.Chunker).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC chunker and hash are still forwarded unconditionally here, so the daemon's Import defaults never reach client/rpc callers?

L32's caopts.UnixfsAddOptions(opts...) materializes the library defaults, so even when the caller passes no options at all, options.Chunker is "size-262144" and mht is "sha2-256", and both go on the wire regardless. But the daemon reads an empty value as "not set, use config":

kubo/core/commands/add.go

Lines 487 to 498 in 13ad613

// Forward hash and chunker only when set on the CLI; otherwise
// UnixfsAPI.Add applies the Import config default.
if hashFunStr != "" {
hashFunCode, ok := mh.Names[strings.ToLower(hashFunStr)]
if !ok {
return fmt.Errorf("unrecognized hash function: %q", strings.ToLower(hashFunStr))
}
opts = append(opts, options.Unixfs.Hash(hashFunCode))
}
if chunker != "" {
opts = append(opts, options.Unixfs.Chunker(chunker))
}

Since the request always carries a non-empty value, != "" is always true there and the "otherwise UnixfsAPI.Add applies the Import config default" path never runs for RPC callers. So on a fresh unixfs-v1-2025 repo (UnixFSChunker = "size-1048576"), ipfs add chunks at 1 MiB while api.Unixfs().Add(ctx, f) against that same daemon chunks at 256 KiB, leading to different CIDs for anything over 256 KiB.

Would adding ChunkerSet/MhTypeSet to UnixfsAddSettings and gating these two the same way as CidVersionSet just below be the right fix? hash looks harmless today since both profiles pin sha2-256, but a repo with a custom Import.HashFunction would hit the same issue.

@lidel
lidel marked this pull request as draft July 14, 2026 14:38
@lidel lidel mentioned this pull request Jul 16, 2026
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Migration to CIDv1 (default base32) Make base32 CIDv1 the default (Kubo 1.0)

4 participants