chore: merge new changes from ipfs/kubo master#2
Open
alvin-reyes wants to merge 944 commits into
Open
Conversation
Merge release v0.39.0
stop publishing to ipfs/go-ipfs entirely - the deprecation stub introduced in v0.39 is no longer needed. only ipfs/kubo is published. - remove legacy-name job from docker-image.yml workflow - remove .github/legacy/ (Dockerfile.goipfs-stub, goipfs_stub.sh) - update bin scripts to use ipfs/kubo as default
Signed-off-by: rifeplight <rifeplight@outlook.com>
document known 0.39 limitation where sweep provider may fail to estimate DHT size when accelerated client is still crawling the network, resulting in single-region mode without efficiency gains. also remove accelerated client recommendation from changelog since it may mislead users into enabling both together.
add link to Shipyard's "Provide Sweep: Solving the DHT Provide Bottleneck" blogpost and remove stale TODO placeholder
* Add bytes progress tracker for ipfs pin add * upgrade to boxo that has ipfs/boxo#1071
* ipfswatch: fix loading datastore plugins * test: add CLI tests for ipfswatch --------- Co-authored-by: Marcin Rataj <lidel@lidel.org>
* fix: update go-libp2p to v0.46.0 - reduced WebRTC log noise (go-libp2p#3426) - fixed mDNS discovery on Windows/macOS (go-libp2p#3434) - includes quic-go v0.57.1 (v0.56.0 + v0.57.0) * fix(example): kubo-as-a-library test timeout - use custom ports (4010/4011) to avoid conflicts with default 4001 - add 2-minute context timeout to fail fast - get peer addresses dynamically instead of hardcoding wrong port - wait for peer connection synchronously instead of fire-and-forget - update comments to reference autoconf.FallbackBootstrapPeers * chore: update p2p-forge to v0.7.0 * fix(test): wait for DHT readiness in GetClosestPeers test the test was failing for `routing_type=auto` because it only waited for swarm connections but not for the DHT routing table to be populated. added a separate probe loop that waits for GetClosestPeers to succeed before running the actual test assertions.
Bumps [actions/cache](https://github.com/actions/cache) from 4 to 5. - [Release notes](https://github.com/actions/cache/releases) - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - [Commits](actions/cache@v4...v5) --- updated-dependencies: - dependency-name: actions/cache dependency-version: '5' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* datastore: upgrade go-ds-pebble to v0.5.8 - update 'go-ds-pebble` to [v0.5.8](https://github.com/ipfs/go-ds-pebble/releases/tag/v0.5.8) - updates github.com/cockroachdb/pebble to [v2.1.3](https://github.com/cockroachdb/pebble/releases/tag/v2.1.3) - enables Go 1.26 support
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 6 to 7. - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](actions/download-artifact@v6...v7) --- updated-dependencies: - dependency-name: actions/download-artifact dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
addresses https://discuss.ipfs.tech/t/19933 - add docs/developer-guide.md with prerequisites, build, test, and troubleshooting - link from README.md, docs/README.md, and CONTRIBUTING.md - document test suite differences (unit vs e2e, test/cli vs test/sharness) - include tips for running specific tests during development
Change the `ipfs key list` behavior to log an error and continue listing keys when a key cannot be read from the keystore or decoded. Closes: #11102
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 5.5.1 to 5.5.2. - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](codecov/codecov-action@5a10915...671740a) --- updated-dependencies: - dependency-name: codecov/codecov-action dependency-version: 5.5.2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 5 to 6. - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](actions/upload-artifact@v5...v6) --- updated-dependencies: - dependency-name: actions/upload-artifact dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* docs: improve README for first-time users - add Quick Taste section with real CIDv1 example near top - rewrite "What is Kubo?" with technical concepts (CIDs, DAGs, UnixFS, Bitswap) - reorder features to follow user journey (CLI before advanced HTTP features) - streamline install section with links to docs.ipfs.tech - organize package managers in tables with Repology version badges - add supply chain security warning for third-party packages - surface important docs (metrics, debug guide, customizing) - update maintainer info with Shipyard branding Closes #11125 Closes #7298 Closes #5471 Closes #5087 * docs(readme): add changelogs link and fix docs directory URL * docs(readme): add mDNS/DHT links and clarify build steps - link LAN discovery to mDNS spec and WAN to Amino DHT glossary - show both make build and make install with output paths
Use testing/synctest instead of go-clock for artificial time control.
* ci: parallelize gotest by separating test/cli into own job
split the Go Test workflow into two parallel jobs:
- `unit-tests`: runs unit tests (excluding test/cli)
- `cli-tests`: runs test/cli end-to-end tests
test/cli takes ~3 minutes (~50% of total gotest time), so running
it in parallel should reduce wall-clock CI time by ~1.5-2.5 minutes.
both jobs produce JUnit XML and HTML reports for consistent debugging.
* ci(gotest): reduce noise on test timeout panics
add GOTRACEBACK=single to show only one goroutine stack instead of all
when a test timeout panic occurs. this makes CI output much cleaner
when tests hang.
* fix(ci): prevent stderr from corrupting test JSON output
- remove 2>&1 which mixed "go: downloading" stderr messages into JSON
- add JSON validation before parsing
- print failed test names for easier debugging
* ci(gotest): use gotestsum for human-readable test output
- replace per-package coverage loop with single gotestsum invocation
- both unit-tests and cli-tests now show human-readable output
- simplified coverage collection (single -coverprofile, no gocovmerge)
- clarified step names to indicate they run tests
* ci: fix codecov uploads by adding token
- add CODECOV_TOKEN to gotest.yml and sharness.yml
- update codecov-action to v5.5.2
- add fail_ci_if_error: false for robustness
codecov stopped receiving coverage data ~1 year ago when they
started requiring tokens for public repos
* refactor(make): add test_unit and test_cli targets
- add `make test_unit` for unit tests with coverage (used by CI)
- add `make test_cli` for CLI integration tests (used by CI)
- only disable colors when CI env var is set (local dev gets colors)
- remove legacy targets: test_go_test, test_go_short, test_go_race, test_go_expensive
- update gotest.yml to use make targets instead of inline commands
- add test artifacts to .gitignore
* fix(ci): move client/rpc tests to cli-tests job
client/rpc tests use test/cli/harness which requires the ipfs binary.
Move them from test_unit to test_cli where the binary is built.
also:
- update gotestsum to v1.13.0
- simplify workflow step names
* fix(ci): use build tags when listing test packages
go list needs build tags to properly exclude packages like fuse/mfs
when running with TEST_FUSE=0 (nofuse tag).
* fix(ci): move test/integration to cli-tests job
test/integration tests need the ipfs binary, move them from test_unit
to test_cli.
* fix(test): fix flaky kubo-as-a-library and GetClosestPeers tests
kubo-as-a-library: use `Bootstrap()` instead of raw `Swarm().Connect()`
to fix race condition between swarm connection and bitswap peer
discovery. `Bootstrap()` properly integrates peers into the routing
system, ensuring bitswap learns about connected peers synchronously.
GetClosestPeers: simplify retry logic using `EventuallyWithT` with
10-minute timeout. tests all 4 routing types (`auto`, `autoclient`,
`dht`, `dhtclient`) against real bootstrap peers with patient polling.
* fix(example): use bidirectional Swarm().Connect() for reliable bitswap
- connect nodes bidirectionally (A→B and B→A) to simulate mutual peering
- mutual peering protects connection from resource manager culling
- use port 0 for random available ports (avoids CI conflicts)
- enable LoopbackAddressesOnLanDHT for local testing
- move retry logic to test file using require.Eventually
* fix(ci): add test_examples target and parallel example-tests job
- add `make test_examples` target to mk/golang.mk for consistency with test_unit/test_cli
- move example tests to separate parallel CI job (example-tests)
- example: use Bootstrap() with autoconf.FallbackBootstrapPeers for reliable bitswap
- example: increase context timeout to 10 minutes
- test: add 60s per-request timeout to GetClosestPeers (server has 30s routing timeout)
- test: reduce EventuallyWithT to 3 minutes (locally passes in under 1 minute)
* fix(ci): improve test targets, exclusion patterns, and artifact naming
- define COVERPKG_EXCLUDE and UNIT_EXCLUDE as documented variables
- use grep -vE with single regex instead of multiple grep -v calls
- add mkdir -p before rm to ensure directories exist
- add DEPS_GO dependency to test_cli target
- make CLI test timeout configurable via TEST_CLI_TIMEOUT (default 10m)
- fix test_examples cleanup on failure using subshell
- reduce GetClosestPeers test wait time from 3m to 2m
- rename artifacts to match job names: unit-tests-{junit,html}, cli-tests-{junit,html}
- update cli-tests upload-artifact from v5 to v6
* fix(ci): fix unit test exclusion and speed up example test
- fix UNIT_EXCLUDE regex to match client/rpc at end of path
- remove public bootstrap peers from example (only connect to nodeA)
- example test now runs in ~3s instead of timing out
* fix(test): fix flaky TestAddMultipleGCLive race condition
added time.Sleep after spawning GC goroutines to ensure they reach
GCLock() before the test proceeds. without this, the adder's
maybePauseForGC() might check GCRequested() before GC has even
requested the lock, causing the lock to not be released and GC to
block indefinitely.
this matches the existing pattern in TestAddGCLive which already
had this sleep.
also replaced context.Background() with t.Context() in both
TestAddMultipleGCLive and TestAddGCLive for proper test lifecycle
management.
* fix(example): use test harness settings for reliable CI
the kubo-as-a-library example was flaky on CI. applied test-harness-like
settings that match what transports_test.go uses:
- TCP-only on 127.0.0.1 with random port (no QUIC/UDP)
- explicitly disable non-TCP transports (QUIC, Relay, WebTransport, etc)
- use NilRouterOption (no routing) since we connect peers directly
- bitswap works with directly connected peers without DHT lookups
- 2-minute context timeout
- streaming output in test for debugging
* fix(routing): use LegacyProvider for HTTP-only custom routing when `Routing.Type=custom` with only HTTP routers and no DHT, fall back to LegacyProvider instead of SweepingProvider. SweepingProvider requires a DHT client which is unavailable in HTTP-only configurations, causing it to return NoopProvider and breaking provider record announcements to HTTP routers. fixes #11089 * test(routing): verify provide stat works with HTTP-only routing * docs(config): clarify SweepEnabled fallback for HTTP-only routing --------- Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
- add "Running in Production" section to gateway.md - link to specs.ipfs.tech instead of github - update "Protocol Labs" to "IPFS Foundation" - add reverse proxy requirement to config.md PublicGateways section - reference reverse proxy, timeout, rate limiting, and CDN caveats
- add TTY auto-detection for progress display (matching `dag export`) - use single-line progress with carriage return instead of flooding - show human-readable sizes alongside raw bytes in summary - update --progress flag to be auto-detected by default progress format: `Fetched/Processed N blocks, M bytes (X MB)` summary format: `Total Size: 99 (99 B)`
* feat(p2p): add --foreground flag to listen and forward commands adds `-f/--foreground` option that keeps the command running until interrupted (SIGTERM/Ctrl+C) or closed via `ipfs p2p close`. the listener/forwarder is automatically removed when the command exits. useful for systemd services and scripts that need cleanup on exit. * docs: add p2p-tunnels.md with systemd examples - add dedicated docs/p2p-tunnels.md covering: - why p2p tunnels (NAT traversal, no public IP needed) - quick start with netcat - background and foreground modes - systemd integration with path-based activation - security considerations and troubleshooting - document Experimental.Libp2pStreamMounting in docs/config.md - simplify docs/experimental-features.md, link to new doc - add "Learn more" links to ipfs p2p listen/forward --help - update changelog entry with doc link - add cross-reference in misc/README.md * chore: reference kubo#5460 for p2p config Ref. #5460 * fix(daemon): write api/gateway files only after HTTP server is ready fixes race condition where $IPFS_PATH/api and $IPFS_PATH/gateway files were written before the HTTP servers were ready to accept connections. this caused issues for tools like systemd path units that immediately try to connect when these files appear. changes: - add corehttp.ServeWithReady() that signals when server is ready - wait for ready signal before writing address files - use sync.WaitGroup.Go() (Go 1.25) for cleaner goroutine management - add TestAddressFileReady to verify both api and gateway files * fix(daemon): buffer errc channel and wait for all listeners - buffer error channel with len(listeners) to prevent deadlock when multiple servers write errors simultaneously - wait for ALL listeners to be ready before writing api/gateway file, not just the first one Feedback-from: #11099 (review) * docs(changelog): improve p2p tunnel section clarity reframe to lead with user benefit and add example output * docs(p2p): remove obsolete race condition caveat the "First launch fails but restarts work" troubleshooting section described a race where the api file was written before the daemon was ready. this was fixed in 80b703a which ensures api/gateway files are only written after HTTP servers are ready to accept connections. --------- Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
* Do not output keystore error on shutdown. Closes #11127 * fix: add debug log for keystore sync interrupted by shutdown log at DEBUG level when keystore sync is interrupted during shutdown, preserving error details for debugging while keeping normal output clean --------- Co-authored-by: Marcin Rataj <lidel@lidel.org>
Belt-and-suspenders defense against data races where routing subsystem (DHT or delegated routing) may reuse backing array. Clones AddrInfo before publishing to QueryEvent to ensure isolated copy. Closes #11116
* docs: mark custom routing as experimental reorganize Routing.Type section for clarity, group production and experimental options, consolidate DHT explanation, add limitations section to delegated-routing.md documenting that HTTP-only routing cannot provide content reliably * chore(config): reorder Routing sections and improve callout formatting move DelegatedRouters after Type, add config option names to CAUTION headers * docs: address reviewer feedback on config.md - clarify that `auto` can be combined with custom URLs in `Routing.DelegatedRouters` - rename headers for consistency: `Routing.Routers.[name].Type`, `Routing.Routers.[name].Parameters`, `Routing.Methods` - replace deprecated Strategic Providing reference with `Provide.*` config - remove outdated caveat about 0.39 sweep limitation - wording: "likely suffer" → "will be most affected" * docs: remove redundant Summary section from delegated-routing.md the IMPORTANT callout and Motivation section already cover what users need to know. historical version info was noise for researchers trying to configure custom routing. addresses reviewer feedback from #11111. --------- Co-authored-by: Daniel Norman <2color@users.noreply.github.com> Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
Gracefully shutdown the ipfs node to ensure data is saved. Forceful shutdown is done if there is an error sending the interrupt signal to the process, such as on Windows.
* datastore: upgrade go-ds-flatfs to v0.6.0 See: ipfs/go-ds-flatfs#142 * docs(changelog): add go-ds-flatfs atomic batch writes *documents the new flatfs batch implementation that uses atomic operations via temp directory, preventing orphan blocks on interrupted imports and reducing memory usage. * includes improved tests, batch cleanup fixes, and docs * docs(changelog): reframe go-ds-flatfs entry for users focus on user benefits instead of implementation details
…lang-x group across 1 directory (#11346) * chore(deps): bump golang.org/x/crypto Bumps the golang-x group with 1 update in the / directory: [golang.org/x/crypto](https://github.com/golang/crypto). Updates `golang.org/x/crypto` from 0.51.0 to 0.52.0 - [Commits](golang/crypto@v0.51.0...v0.52.0) --- updated-dependencies: - dependency-name: golang.org/x/crypto dependency-version: 0.52.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: golang-x ... Signed-off-by: dependabot[bot] <support@github.com> * chore: run make mod_tidy --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
* fix(l1): return error instead of log.Fatal in HolePunching Signed-off-by: reflecttypefor <reflecttypefor@outlook.com> * test: cover HolePunching flag combinations * docs: highlight clearer hole punching error The Changelog section is generated at release time, so hand-written notes belong under Highlights instead. --------- Signed-off-by: reflecttypefor <reflecttypefor@outlook.com> Co-authored-by: Marcin Rataj <lidel@lidel.org>
Pulls the AutoTLS DNS-01 registration fix: the client now keeps a cookie jar so a load-balanced forge endpoint (registration.libp2p.direct) keeps both PeerID-auth requests on one backend, instead of the second request hitting a backend that never issued the challenge and failing with a 401.
* commands: derive peer ID on config replace Signed-off-by: morning-verlu <258725120+morning-verlu@users.noreply.github.com> * feat: validate Identity.PeerID against node key Setting Identity.PeerID to a value that disagrees with the node's private key produces a config the node refuses to start with. The config command now validates the field against the stored key, the same way config replace re-derives it. - reject a mismatched Identity.PeerID and point at `ipfs key rotate`; accept the node's own PeerID in any base58 or CIDv1 form and store the canonical base58 string - share the PeerID derivation between the set path and config replace - document the identity fields and `ipfs key rotate` in docs/config.md - cover the set-path guard and base36 normalization in test/cli - switch the t0070 sharness probe off Identity.PeerID, now validated --------- Signed-off-by: morning-verlu <258725120+morning-verlu@users.noreply.github.com> Co-authored-by: morning-verlu <258725120+morning-verlu@users.noreply.github.com> Co-authored-by: Guillaume Michel <guillaumemichel@users.noreply.github.com> Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com> Co-authored-by: Marcin Rataj <lidel@lidel.org>
Bumps [actions/checkout](https://github.com/actions/checkout) from 6 to 7. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v6...v7) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [actions/cache](https://github.com/actions/cache) from 5 to 6. - [Release notes](https://github.com/actions/cache/releases) - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - [Commits](actions/cache@v5...v6) --- updated-dependencies: - dependency-name: actions/cache dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* test: cover ipfs get paths containing closing bracket * test: move get punctuation coverage to test/cli ipfs get must retrieve UnixFS paths whose segments contain punctuation that is valid on Linux, macOS, and Windows but is sensitive to a POSIX shell (issue #9369, where a "]" segment failed). AGENTS.md prefers test/cli for new integration tests, and driving ipfs directly avoids the shell-quoting limits of the sharness loop. - add test/cli/get_test.go: add a directory with one file per segment, then get each "<cid>/<segment>" and compare bytes, exercised both offline and against a daemon - cover the apostrophe segment, which the single-quoted sharness test body could not include - drop the now-redundant punctuation block from t0090-get.sh --------- Co-authored-by: Guillaume Michel <guillaumemichel@users.noreply.github.com> Co-authored-by: Marcin Rataj <lidel@lidel.org>
* fix(daemon): return config errors instead of calling log.Fatal Signed-off-by: blackflytech <blackflytech@outlook.com> * Update cmd/ipfs/kubo/daemon_config_test.go --------- Signed-off-by: blackflytech <blackflytech@outlook.com> Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
The hole-punching and daemon-config highlights describe the same user-facing change, a named startup error instead of an abrupt exit, so fold them into one entry and drop the internal `log.Fatal` wording.
* feat: make telemetry opt-in Telemetry no longer runs by default. It stays off unless an operator sets the mode to `on` and configures an `Endpoint`, and Kubo ships with no built-in telemetry URL, so a default node never phones home. - plugin: default mode is off; the legacy `auto` and any unrecognized value also stay off - plugin: the implicit default does no work, not even disk IO; only an explicit `off` removes a stored telemetry identifier - plugin: enabling without an endpoint warns and sends nothing; the destination comes only from config - docs: note the opt-in default in the `IPFS_TELEMETRY` reference, the plugins table, and the changelog - tests: cover off-by-default, the auto alias, missing-endpoint, and explicit opt-out cleanup, and opt in where the send path runs * docs: rewrite telemetry.md for opt-in Rework the telemetry page around the opt-in plugin: how to enable it and point it at your own collector, the modes and config reference, the HTTP endpoint API and payload schema, and the privacy model. Match the plainer style of the sibling docs, with no emoji headers and a table of contents. * chore(telemetry): log opt-in enable event Log when telemetry is enabled, since that's the notable event. The opt-out branch already logs UUID removal, so the redundant disable log is dropped. Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com> --------- Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
* feat: accept native ipfs:// and ipns:// URIs Commands that take a content path or CID now also accept native IPFS URIs (ipfs://cid, ipns://name, and the schemeless ipfs:/ipns: forms), so a URI copied from a browser or another tool works as-is. - cmdutils: PathOrCidPath parses via boxo NewPathFromURI; new CidFromArg for raw-CID commands takes the root CID and rejects sub-paths and mutable IPNS. - files: cp/stat sources and getNodeFromPath accept URIs and content paths; chroot takes its CID via CidFromArg. - resolve and name resolve normalize URIs before the namespace checks; name resolve stays IPNS-only. - routing, provide, filestore, pin remote: raw-CID args via CidFromArg. Depends on boxo NewPathFromURI (ipfs/boxo#1182); go.mod pins the PR commit until it is released. * depend on boxo@main * test: fix telemetry opt-out assertions #11374 made telemetry opt-in and rewrote the explicit "off" mode to no longer log "telemetry disabled via opt-out", but the opt-out subtests still assert that string, so TestTelemetry is red on master. Assert the "telemetry collection skipped: opted out" message the daemon emits whenever telemetry is off. * ci: inject .aegir.js for helia interop @helia/interop v11.0.0+ ships without .aegir.js (ipfs/helia#1049), so aegir test finds no specs and the interop job fails. Inject a minimal config pointing at the prebuilt dist specs when it is missing. Helia's own .aegir.js can't be reused as-is: it globs source .ts specs that Node won't run from node_modules. The same omission regressed before (ipfs/helia#1001, fixed by ipfs/helia#1003); see the comment. * ci: force mocha exit after helia interop run The node interop specs leave kubo daemon and libp2p handles open, so mocha prints "N passing" and then hangs until the job timeout instead of exiting. Pass --exit so mocha quits once the run completes. --------- Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
* chore(deps): bump go-libp2p-kad-dht to v0.41.0 Cuts peak memory during reprovides on nodes announcing many CIDs, so low-memory consumer devices are less likely to be out-of-memory killed (libp2p/go-libp2p-kad-dht#1259). Pulls quic-go v0.59.1 transitively. * chore: tidy secondary go modules for kad-dht 0.41 * ci: fix helia-interop for @helia/interop 11.0.3 The .aegir.js restored upstream in ipfs/helia#1066 globs the TypeScript sources, which node refuses to run from inside node_modules, so the inject-if-missing workaround skipped it and the job failed. Replace the config whenever it is missing or globs .ts files, and leave a usable one untouched so the step self-heals once upstream ships a node_modules-safe config. Also drop the IPIP-499 --grep/--invert exclusion: helia implemented the missing MFS features (ipfs/helia#972) and the test passes now.
#11380) * chore(deps): bump the golang-x group across 1 directory with 5 updates Bumps the golang-x group with 2 updates in the / directory: [golang.org/x/crypto](https://github.com/golang/crypto) and [golang.org/x/mod](https://github.com/golang/mod). Updates `golang.org/x/crypto` from 0.52.0 to 0.53.0 - [Commits](golang/crypto@v0.52.0...v0.53.0) Updates `golang.org/x/mod` from 0.36.0 to 0.37.0 - [Commits](golang/mod@v0.36.0...v0.37.0) Updates `golang.org/x/sync` from 0.20.0 to 0.21.0 - [Commits](golang/sync@v0.20.0...v0.21.0) Updates `golang.org/x/sys` from 0.45.0 to 0.46.0 - [Commits](golang/sys@v0.45.0...v0.46.0) Updates `golang.org/x/term` from 0.43.0 to 0.44.0 - [Commits](golang/term@v0.43.0...v0.44.0) --- updated-dependencies: - dependency-name: golang.org/x/crypto dependency-version: 0.53.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: golang-x - dependency-name: golang.org/x/mod dependency-version: 0.37.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: golang-x - dependency-name: golang.org/x/sync dependency-version: 0.21.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: golang-x - dependency-name: golang.org/x/sys dependency-version: 0.46.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: golang-x - dependency-name: golang.org/x/term dependency-version: 0.44.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: golang-x ... Signed-off-by: dependabot[bot] <support@github.com> * chore: run make mod_tidy --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(mfs): stop repo gc from freezing files ops Running `ipfs repo gc` alongside `ipfs files` writes could leave MFS permanently hung: GC deleted directory-node blocks a write had written to the blockstore but not yet linked into the persisted MFS root, and the next path lookup blocked forever fetching the missing block while holding the MFS directory lock, so every later files command piled up behind it. MFS mutations now take the pin lock, the same lock `ipfs add` uses, and GC computes the MFS root only after it holds the GC lock. A write's blocks are therefore either fully linked into the root before GC runs, or the write waits for GC to finish, so GC never collects data a live write still needs. - core/commands/files.go: hold the pin lock across write, cp, mkdir, mv, rm, flush, chcid, chmod, and touch - gc/gc.go: take the best-effort MFS root snapshot after acquiring the GC lock, closing the snapshot-before-lock window - core/corerepo/gc.go: pass the root as a callback evaluated under the lock - core/node/core.go: document why MFS keeps its online DAG service so lazy `ipfs files cp /ipfs/<cid>` pointers still resolve - gc/gc_test.go: assert the root snapshot is taken under the GC lock Closes #10842 * fix(mfs): extend gc pin lock to add and fuse The pin lock that stops garbage collection from collecting live MFS blocks now covers every path that mutates MFS, not just `ipfs files`, and every live MFS root is part of the GC live set. - core/commands/add.go: hold the pin lock while `ipfs add --to-files` links the added content into the MFS root - fuse/writable, fuse/mfs: hold the pin lock across FUSE `/mfs` structural writes and file flush, fsync, and release - fuse/ipns: same for the per-key `/ipns` mounts, and register their roots so GC keeps their blocks live - core/core.go: track mounted MFS roots for the GC live set - core/corerepo/gc.go: build the live set from the files root plus every registered root, skipping a root that errors instead of aborting the whole GC Closes #6113 Closes #7008 Closes #9553 * fix(mfs): fail fast on a missing block A directory-node block that is missing locally and unreachable would block an MFS operation forever while it held the directory lock, wedging the whole MFS and a clean shutdown until the process was killed. This is what remained after a repo was damaged by an older Kubo, a manual `ipfs block rm`, or a crash (the lockup in #7844). MFS now bounds those network reads: an unreachable block fails the operation with a timeout and releases the lock, while lazily-referenced content (`ipfs files cp /ipfs/<cid>`) still loads as before. - config: add DefaultMFSFetchTimeout and pass it to the MFS root via mfs.WithFetchTimeout in Import.MFSRootOptions - go.mod: bump boxo to pull in mfs.WithFetchTimeout * feat(mfs): honor --timeout in files read and write Adopt boxo's mfs.File.Open(ctx) so MFS file operations carry a context and a read or write stuck on a missing block can be cancelled instead of blocking forever. - go.mod: bump boxo to pick up mfs.File.Open(ctx) - core/commands/files.go: pass the request context to files read and write, so a client --timeout ends a stuck read or write - fuse/writable: bind long-lived FUSE descriptors (Create, Open) to the mount context via the new Config.MountCtx; the transient Setattr truncate uses the per-operation context - fuse/mfs, fuse/ipns: set MountCtx to the node/mount context - test/cli: cover files read and write honoring --timeout on unreachable content * chore(deps): bump boxo to merged mfs fix Switch from the pre-merge branch pseudo-version to the merged commit of ipfs/boxo#1185, the boxo side of the MFS under-lock work this branch depends on. * refactor(fuse): pass request ctx to pin lock Flush, Release, and Fsync passed context.Background() to the pin lock while their own ctx argument was in scope. Thread the passed ctx through instead, matching the other handlers. boxo's default GCLocker discards the context, so this is a consistency change, not a behavior change. #11386 (comment) #11386 (comment) #11386 (comment) Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com> * refactor(gc): const for buffer, simplify roots Name the GC result channel buffer size as a const, drop the temporary in the best-effort root snapshot, and use t.Context() in the snapshot test. The buffer size predates this branch; it only shifted in the diff. #11386 (comment) #11386 (comment) #11386 (comment) #11386 (comment) Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com> * refactor(add): collapse pin lock defer Fold the pin lock's unlock into the defer at both --to-files call sites, matching the one-line style used elsewhere. #11386 (comment) #11386 (comment) Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com> --------- Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
…11387) * fix(key): restore secp256k1 keygen and add PEM PKCS8 import/export * fix(key): validate --size for fixed-size key types ed25519 and secp256k1 keys have a single valid size, so a --size (--bits for init) that does not match it is now an error instead of being accepted and ignored. core/coreapi Generate and config.CreateIdentity share a new options.CheckKeySize helper, so key gen, key rotate, and init accept --size only when it equals the fixed 256 bits. RSA keeps its variable size. * test(cli): cover key lifecycle for all key types Adds end-to-end CLI coverage of the key commands (gen, list, export, import, rename, rm, rotate) for rsa, ed25519, and secp256k1. This mirrors the sharness keystore and rotate suites and extends them to secp256k1, which they never exercised. - OpenSSL fixtures under testdata/ pin byte-identical PKCS#8 export and import in both directions - rotate checks the previous identity survives, usable, under the backup name - reserved-name ('self') and restricted-type imports assert the specific refusal, not just a non-zero exit * chore(deps): note secp256k1 version alignment go-libp2p/core/crypto's key types alias this package, so the direct and transitive pins must stay on one version to avoid two copies in the build. The go.mod comment flags that for future dependency bumps. --------- Co-authored-by: Marcin Rataj <lidel@lidel.org>
Gateway recipes now lead with the decision that matters most for a public gateway: does it fetch any CID from the network, or serve only content the node already hosts? They move from the config reference into the gateway guide, where operators look for them. - gateway.md: new "Gateway recipes" section (serve-only-your-own-content via NoFetch, open recursive gateway, and subdomain/path/DNSLink URL styles as linkable sub-headings), a table of contents, and fixes to the Directories ordering and the /ipns/ subdomain redirect example; public gateways point to the public-utilities and checker pages and the self-hosting guide instead of naming hosts - config.md: recipes replaced by a pointer to the gateway guide - content-blocking.md: suggest an allowlist (NoFetch) over reactive denylist whack-a-mole
* chore(deps): bump go-libp2p and go-libp2p-pubsub Both are pinned to master pseudo-versions ahead of tagged releases, with a TODO in go.mod to switch once the tags ship. go-libp2p adds webrtc-direct v2 handshake support, keeps /certhash stable across restarts, and stops stale addresses from accumulating in the peerstore and in published signed peer records. It also fixes a data race that could take the daemon down mid-response during ipfs routing findprovs, findpeer, and dht query. go-libp2p-pubsub frees topic state once the last peer leaves a topic, closing an unbounded memory growth path (libp2p/go-libp2p-pubsub#705, the Go counterpart of CVE-2026-46679). Only nodes that opt into Pubsub.Enabled or Ipns.UsePubsub run the pubsub stack and are affected. * feat(config): Internal.NonPublicAddrPublishing Exposes go-libp2p's NonPublicAddrPublishing as an Internal flag, so a node can be told to stop publishing addresses the wider internet cannot reach: private, CGNAT, link-local, loopback, ULA, reserved IPv6, and special-use DNS names such as .local. Left unset, kubo passes no option and go-libp2p's default decides. The default has shifted before (libp2p/go-libp2p#3460), so the flag gives operators a way to pin the behavior, and makes it easy to see exactly what a node publishes while chasing a routing problem. Only the peerstore self-entry and the signed peer record are filtered. Listening and dialing are untouched, so `ipfs id` keeps reporting the full set.
Minor dependency updates, no changes to kubo. - go-block-format v0.2.4 - go-cid v0.6.2 - go-cidutil v0.1.2 - go-datastore v0.9.2 - go-ds-flatfs v0.6.1 - go-ipld-format v0.6.4 - golang.org/x/...
Bumps [actions/setup-node](https://github.com/actions/setup-node) from 6 to 7. - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](actions/setup-node@v6...v7) --- updated-dependencies: - dependency-name: actions/setup-node dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
https://github.com/ipfs/ipfs-webui/releases/tag/v4.13.0 Also restores the changelog TOC entry for the IPNS lifetime and TTL highlight, which was left out when that section was added.
- Use new `go-test/random` API - no global seed values - reuse generator where appropriate - Fix tests to match new random data generation - update expected deterministic values - fix t0040-add-and-cat.sh - fix t0043-add-w.sh - fix t0045-ls.sh - fix t0087-repo-robust-gc.sh - fix t0270-filestore.sh - fix t0271-filestore-utils.sh - fix t0272-urlstore.sh - Migrate from `/math/rand' to `/math/rand/v2` - random seeds are now `[32]byte` and not `uint64` - use `StringToSeed` or `Uint64ToSeed` to create random seeds for deterministic output
* feat(autotls): skip issuance when broker is down Bump github.com/ipshipyard/p2p-forge to the head of ipshipyard/p2p-forge#91: before first-time ACME issuance the client now confirms the broker responds with HTTP 204 on /v1/health, after the registration delay and once the node is publicly reachable. While the broker keeps failing the check, certificate setup is postponed with one ERROR and an hourly re-check (respecting Retry-After, capped at 24h) instead of doomed ACME retries for weeks. Ephemeral nodes (CI runners) still produce no broker traffic at all, and nodes with a certificate in storage are unaffected. * chore: update p2p-forge to v0.10.0
* chore: bump go-libp2p-kad-dht to v0.42.0 * bump kad-dht * fix(init): stop publishing an empty-directory IPNS record * bump kad-dht to v0.42.1 * bump boxo * chore: bump boxo, clarify IPNS storage changelog Bumps boxo, bringing the offline router retention change (records are served until their EOL, no receive-age cap by default; ipfs/boxo#1189) and a gateway 304 cache-freshness fix (ipfs/boxo#1188). The v0.43 "Unified IPNS record storage" highlight now spells out how retention works: with the offline router and the DHT sharing the /dht datastore prefix, online nodes drop value records 48h after storage (the DHT's value-store GC) while offline nodes keep them until EOL. * fix: sync DHT purge once after deletion * bump boxo * depend on boxo@master * chore: bump boxo to main, align module pins Pick up boxo main HEAD (4794174d), which includes the merged offline router value-store change (ipfs/boxo#1189). This brings the root module, the kubo-as-a-library example, and test/dependencies onto one boxo pin; they previously pointed at three different commits. - go-libp2p moves to ec408fcc as a transitive floor required by boxo - changelog: note the one-time full-datastore key scan on first start after upgrading, and update the pinned boxo and go-libp2p hashes * chore: require make mod_tidy before commit/push Spell out that the repo has three go.mod files that must stay on the same dependency versions, and that a bare `go mod tidy` only tidies one module and lets pins drift between them. --------- Co-authored-by: Marcin Rataj <lidel@lidel.org>
* chore: upgrade to boxo v0.42.0 * use tagged release * update changelog * docs: expand v0.43 changelog highlights Cover the user-facing changes the boxo v0.42.0 and go-libp2p bumps pull in since v0.42.0, and group the security fixes so operators can decide whether to update. - add a security section for the pubsub, libp2p resource-cap, routing-query crash, and OTLP exporter fixes, with their CVEs - reframe the DNSLink caching note as a TTL and expiration revamp spanning IPNS records, DNSLink, and 304 revalidation - credit boxo v0.41.0 and its bitswap fetch-stall fix - drop dependency-section prose that repeats the highlights --------- Co-authored-by: Marcin Rataj <lidel@lidel.org>
Give a new maintainer the guardrails that keep contributions safe as people come and go. AGENTS.md carries the hard rules a change must not cross; CONTRIBUTING.md carries the why and who the project is for. - AGENTS.md: stability rules for the /api/v0 RPC, the HTTP Gateway, default CIDs, and protocol changes via IPIP, each with a way to verify it; boxo-vs-kubo boundary; configurable-endpoint rule for user agency; note that CLI/RPC reference docs are generated from help text; PRs need a description and tests, test/cli over sharness; release tooling is off-limits outside a release - CONTRIBUTING.md: what Kubo optimizes for (the self-hoster first), and a pointer to AGENTS.md for humans working with an LLM assistant
* chore: bump go to 1.26.5 latest 1.26.x patch; keeps go.mod, sub-module go.mods, and the Dockerfile GO_VERSION default in sync. * chore(deps): bump go-ipld-cbor, go-ds-measure part of the ipfs-ecosystem dependabot group (#11395); go-test in that group is skipped since master is already on v0.4.1. - go-ipld-cbor v0.2.1 -> v0.3.0 - go-ds-measure v0.2.2 -> v0.2.3
Opens development for the next release after v0.43. - version.go -> 0.44.0-dev - add docs/changelogs/v0.44.md stub and link it in CHANGELOG.md
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 6 to 7. - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](actions/setup-go@v6...v7) --- updated-dependencies: - dependency-name: actions/setup-go dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.