From 64190647f784295f309f86e9677b025f48ab15b5 Mon Sep 17 00:00:00 2001 From: Bloxster Date: Wed, 26 Aug 2026 12:22:12 +0200 Subject: [PATCH 1/3] docs: document the v3.6 plain commitment format and its migration paths MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Snapshots Management gets a new "Commitment Snapshot Format" section: what referenced vs plain encoding means, that plain is now the default, that existing referenced files stay readable and convert during merges, and the two ways to adopt plain sooner (`erigon seg reset`, `integration commitment convert`) with their flags, destructiveness, reversibility and cost. Adds the step-size prerequisite for datadirs still recorded at the pre-3.4 step size (`erigon seg step-rebase`). CLI Reference: `--commitment.plainValues` said "Default: false", which reads as "references by default" and is wrong for v3.6. The flag is a tri-state override — omitted means plain, and only an explicit `--commitment.plainValues=false` selects the referenced encoding. --- .../docs/fundamentals/configuring-erigon.mdx | 4 +- .../fundamentals/snapshots-management.mdx | 108 +++++++++++++++++- 2 files changed, 109 insertions(+), 3 deletions(-) diff --git a/docs/site/docs/fundamentals/configuring-erigon.mdx b/docs/site/docs/fundamentals/configuring-erigon.mdx index f838b497e0f..a715ab7a407 100644 --- a/docs/site/docs/fundamentals/configuring-erigon.mdx +++ b/docs/site/docs/fundamentals/configuring-erigon.mdx @@ -114,8 +114,8 @@ These flags control database performance and memory usage. See [Database](/funda * `--erigondb.domain.steps-in-frozen-file value`: Overrides the `steps_in_frozen_file` setting from `erigondb.toml` for the domain merge cap only (history and inverted-index merges are unaffected). Pass a positive integer to set an explicit cap, or `Inf` to leave the domain merge unbounded. * Default: unset (uses the value from `erigondb.toml`) * Use with care — an incorrect value may affect database structure. -* `--commitment.plainValues` *(New in v3.6)*: On the first start of a fresh datadir, write commitment values in plain form (no shortened key references). This is a one-time, per-datadir choice — it is ignored once `erigondb.toml` exists. - * Default: `false` +* `--commitment.plainValues` *(New in v3.6)*: Chooses the commitment branch encoding recorded in `snapshots/erigondb.toml` at the moment Erigon creates that file. `true` writes commitment values in plain form (keys inline); `false` writes shortened key references. This is a one-time, per-datadir choice — it is ignored once `erigondb.toml` exists, and a `erigondb.toml` delivered by the downloader wins over it. See [Commitment Snapshot Format](/fundamentals/snapshots-management#commitment-snapshot-format). + * Default: plain. Leaving the flag out selects plain, which is v3.6's compiled default; passing `--commitment.plainValues=false` explicitly selects the older *referenced* encoding. Before v3.6 the compiled default was referenced. ### Pruning and Snapshots diff --git a/docs/site/docs/fundamentals/snapshots-management.mdx b/docs/site/docs/fundamentals/snapshots-management.mdx index a8f7417d3c5..02456dc695a 100644 --- a/docs/site/docs/fundamentals/snapshots-management.mdx +++ b/docs/site/docs/fundamentals/snapshots-management.mdx @@ -1,6 +1,6 @@ --- title: "Snapshots Management" -description: "Understand and manage Erigon snapshot files — check disk usage by category, compare node type estimates, and migrate to v3.5 snapshot formats." +description: "Understand and manage Erigon snapshot files — check disk usage by category, compare node type estimates, and migrate commitment and step-size snapshot formats." sidebar_position: 5 --- @@ -121,6 +121,112 @@ The `seg du` estimates table shows the projected disk footprint for each mode gi --- +## Commitment Snapshot Format *(New in v3.6)* {#commitment-snapshot-format} + +A commitment snapshot file stores Merkle-Patricia trie branches. Each branch has to refer to the account and storage keys underneath it, and Erigon can encode those references in one of two ways: + +| Encoding | What a branch stores | File version stamp | +|----------|----------------------|--------------------| +| **Referenced** | *Shortened keys* — offsets into the accounts and storage files, which must be dereferenced on read | `v2.1` | +| **Plain** | The keys themselves, inline | `v2.2` | + +Plain files are **larger on disk**, but state reads and snapshot merges are **faster**, because nothing has to be looked up in the accounts and storage files first. + +v3.6 makes **plain the default** for newly written commitment files. In v3.5 and earlier the default was referenced. + +The choice is recorded per datadir as `references_in_commitment_branches` in `snapshots/erigondb.toml`. When that file omits the field, the compiled default applies — `false` (plain) from v3.6, `true` (referenced) before it. On a normal fresh sync the downloader delivers `erigondb.toml` along with the snapshot set, so the published set decides the regime. + +### What happens to an existing datadir + +Nothing breaks, and nothing is rewritten eagerly: + +- **Existing referenced files stay readable.** Whether a commitment file carries shortened references is a property of *that file* — its version stamp and step range — not of the current setting. The commitment domain reads `v2.1` and `v2.2` alike. +- **Conversion happens lazily, during merges.** When a merge has any referenced input, Erigon attaches a value transformer that expands the references; because new files are now written plain, the merged output lands as `v2.2`. Ordinary background merging therefore migrates the datadir on its own, with no operator action and no downtime. + +For most operators upgrading from v3.5 the correct action is **none**. The commands below exist for operators who want the plain format sooner than merging will deliver it, and who are prepared to pay for it. + +:::warning +Downgrading to v3.5 after v3.6 has written new snapshot files is **not supported** — the new snapshot and accessor formats are incompatible with v3.5. Back up your `--datadir` before taking either migration path below. +::: + +### Path 1 — re-download the snapshot set (`erigon seg reset`) + +Discards the local snapshot set and lets the next start pull the published v3.6 set, which is already plain. + +```bash +erigon seg reset --datadir /your/datadir +``` + +| Flag | Description | +|------|-------------| +| `--datadir ` | Path to the Erigon data directory. | +| `--local`, `-l` | A single switch over two behaviours: remove snapshot files that are not described in the preverified set (locally generated ones), **and** remove `chaindata/` together with the Heimdall / Polygon-bridge DBs. **Default: `true`.** `--local=false` keeps all of them. | +| `--dry-run`, `-n` | Print the files that would be removed and exit without removing anything. Default: `false`. | +| `--preverified ` | Which preverified snapshot list to reset against. Default: `remote`. | + +`seg reset` is **destructive and not reversible**: at the default `--local=true` it deletes `chaindata/`, the Heimdall / Polygon-bridge DBs, any locally generated snapshot files, and any `.torrent` file whose infohash does not match the preverified list. Preverified snapshots already on disk are kept and corrected by the downloader on the next start. The node must be **stopped** — the command takes an exclusive lock on the datadir. + +Cost: whatever the downloader has to re-fetch, plus rebuilding state on the next start. Run `--dry-run` first, and back up the datadir. + +`seg reset` has no `--chain` flag of its own; it reads the chain name from `chaindata/`. If `chaindata/` is missing or unreadable, pass the chain as a root-level flag before the subcommand — `erigon --chain mainnet seg reset --datadir /your/datadir`. + +:::note +`erigon seg reset` is not new in v3.6 (`seg` is an alias for `snapshots`, so `erigon snapshots reset` is the same command). What is new is its role as a shortcut to the plain commitment format. See also [Snapshots Upgrade Options](../get-started/installation/upgrading#snapshots-upgrade-options). +::: + +### Path 2 — offline conversion (`integration commitment convert`) *(New in v3.6)* + +Re-encodes the commitment files already on disk instead of re-downloading them. Built from source as `build/bin/integration` (`make integration`). + +```bash +integration commitment convert --datadir /your/datadir --chain mainnet +``` + +Both encoding flags default to `false`, and `false` is the target state that matches what v3.6 writes — so the command above, with no encoding flags, converts referenced files to plain and leaves the key encoding alone. + +| Flag | Description | +|------|-------------| +| `--datadir ` | Path to the Erigon data directory. | +| `--chain ` | Chain name. | +| `--squeeze` | Target state of the value axis: `true` = referenced (offsets), `false` = plain. Default: `false`. | +| `--nibbles.v2` | Target state of the key axis: `true` = V2, `false` = V1 (compact bytes, what Erigon writes). Default: `false`. | +| `--restore` | Move the originals back from `snapshots/backup/domains/` and remove the converted files. Mutually exclusive with `--squeeze` / `--nibbles.v2` and with `--continue`. | +| `--continue` | Resume an interrupted run, reusing complete shards left in `snapshots/rebuild/domain/`. Mutually exclusive with `--restore`. | + +The flag value is the **target** state, not a direction: the tool detects each file's current encoding and converts only what needs it, skipping files already in the target state. + +**Is it reversible?** Yes — this is the one path with a real undo. The originals are moved to `snapshots/backup/domains/`, and `--restore` puts them back. Restore is filesystem-only and runs even when the datadir is too broken for Erigon to open it. Because a leftover backup blocks a second conversion, restore or clear `snapshots/backup/domains/` before converting again. + +**Is the node stopped?** Yes. The command opens the chaindata DB read-write and applies migrations. + +**What does it cost?** Every commitment `.kv` file that is not already plain is rewritten and its accessors (`.kvi` / `.bt` / `.kvei`) rebuilt. Converted output is written into `snapshots/rebuild/domain/` while the originals are still in place, so at peak you need free space for a second copy of the commitment domain — use the `commitment` row of `erigon seg du --datadir -v` to size it. Plain output is larger than the referenced input. + +:::warning +If a run is interrupted and you resume with `--continue`, you **must** pass the same `--squeeze` and `--nibbles.v2` values as the original run. A mismatch silently produces mixed-encoding output. Without `--continue`, a restart simply discards `snapshots/rebuild/domain/` and redoes the conversion from scratch. +::: + +### Prerequisite for pre-3.4 step sizes (`erigon seg step-rebase`) + +Erigon groups state files into *steps*. Up to v3.3 a step was 1,562,500 transactions; from v3.4 the default is **390,625** — a quarter of the old size, and still the default in v3.6. v3.4 and v3.5 offered the smaller step as an optional gain (about 4× smaller `chaindata/`) rather than forcing it, so a datadir first created by v3.3 or earlier can still be recorded at the legacy size. The v3.6 release notes require those datadirs to rebase, or to take the `seg reset` path above, before their first normal v3.6 start. + +Check `step_size` in `snapshots/erigondb.toml`. If it reads `1562500`: + +```bash +erigon seg step-rebase --datadir /your/datadir --new-step-size=390625 +``` + +| Flag | Description | +|------|-------------| +| `--datadir ` | Path to the Erigon data directory. | +| `--new-step-size ` | **Required.** Target step size. `390625` is the current default. Must be a whole multiple or a whole divisor of the datadir's present step size. | +| `--keep-blocks` | Keep `chaindata/` and reset only the execution-state tables inside it, so already-downloaded blocks can be re-executed instead of re-downloaded. Default: delete `chaindata/` entirely. | + +What it does: renames every state snapshot file under `snapshots/domain`, `history`, `accessor`, and `idx` so its step numbers are rescaled by the factor; deletes the `.torrent` files in those directories and `snapshots/erigondb.toml.torrent`, which the rebase invalidates; deletes `chaindata/` (or, with `--keep-blocks`, resets its execution-state tables); and rewrites `snapshots/erigondb.toml` with the new `step_size` and a correspondingly rescaled `steps_in_frozen_file`. + +The command prints the full rename and delete plan and asks for confirmation (`Proceed with these changes? [y/N]`) before touching anything, and exits as a no-op if the datadir is already at the requested step size. It is **destructive and has no built-in undo**, but it is safe to re-run after an interruption. The node must be **stopped** — it locks the datadir and opens `chaindata/` exclusively. The v3.4.0 release notes put the rebase itself at roughly ten seconds; the cost that follows is re-acquiring whatever was in `chaindata/`. + +--- + ## Upgrading Snapshots in v3.5 ### EIP-8252 retention window change From f7cd217ccfa6869afb3cc26ead67be27d4857d38 Mon Sep 17 00:00:00 2001 From: Bloxster Date: Wed, 26 Aug 2026 13:18:38 +0200 Subject: [PATCH 2/3] docs: correct seg reset --local scope and step-rebase lock detail after code fact-check Fact-checked every operator-facing claim in the v3.6 snapshot-format docs against the release/3.6 Go code. Two corrections: - seg reset: removal of a preverified .torrent with a mismatching infohash (and retraction of the data file it vouched for) is unconditional in Reset.decideRemove/doSnapshots (db/datadir/reset/reset.go), not gated by --local as the prose implied. --local=false is therefore not fully non-destructive; say so explicitly. - seg step-rebase: the exclusive chaindata open happens only in the --keep-blocks resetExecState path (cmd/utils/app/step_cmd.go); the unconditional part is the datadir flock. Also drop the space before the custom heading anchor so the generated llms-full.txt line carries no trailing whitespace, and regenerate the llms artifacts for this branch. --- .../fundamentals/snapshots-management.mdx | 6 +- docs/site/static/llms-full.txt | 110 +++++++++++++++++- docs/site/static/llms.txt | 2 +- llms-full.txt | 110 +++++++++++++++++- llms.txt | 2 +- 5 files changed, 221 insertions(+), 9 deletions(-) diff --git a/docs/site/docs/fundamentals/snapshots-management.mdx b/docs/site/docs/fundamentals/snapshots-management.mdx index 02456dc695a..5f3907d7018 100644 --- a/docs/site/docs/fundamentals/snapshots-management.mdx +++ b/docs/site/docs/fundamentals/snapshots-management.mdx @@ -121,7 +121,7 @@ The `seg du` estimates table shows the projected disk footprint for each mode gi --- -## Commitment Snapshot Format *(New in v3.6)* {#commitment-snapshot-format} +## Commitment Snapshot Format *(New in v3.6)*{#commitment-snapshot-format} A commitment snapshot file stores Merkle-Patricia trie branches. Each branch has to refer to the account and storage keys underneath it, and Erigon can encode those references in one of two ways: @@ -164,7 +164,7 @@ erigon seg reset --datadir /your/datadir | `--dry-run`, `-n` | Print the files that would be removed and exit without removing anything. Default: `false`. | | `--preverified ` | Which preverified snapshot list to reset against. Default: `remote`. | -`seg reset` is **destructive and not reversible**: at the default `--local=true` it deletes `chaindata/`, the Heimdall / Polygon-bridge DBs, any locally generated snapshot files, and any `.torrent` file whose infohash does not match the preverified list. Preverified snapshots already on disk are kept and corrected by the downloader on the next start. The node must be **stopped** — the command takes an exclusive lock on the datadir. +`seg reset` is **destructive and not reversible**: at the default `--local=true` it deletes `chaindata/`, the Heimdall / Polygon-bridge DBs, and any locally generated snapshot files. Independently of `--local` — that is, even with `--local=false` — it also deletes any `.torrent` file whose infohash does not match the preverified list, **together with the snapshot data file that torrent vouched for** (a stale local build is unverifiable, so the downloader must re-fetch the canonical copy). All other preverified snapshots already on disk are kept and corrected by the downloader on the next start. The node must be **stopped** — the command takes an exclusive lock on the datadir. Cost: whatever the downloader has to re-fetch, plus rebuilding state on the next start. Run `--dry-run` first, and back up the datadir. @@ -223,7 +223,7 @@ erigon seg step-rebase --datadir /your/datadir --new-step-size=390625 What it does: renames every state snapshot file under `snapshots/domain`, `history`, `accessor`, and `idx` so its step numbers are rescaled by the factor; deletes the `.torrent` files in those directories and `snapshots/erigondb.toml.torrent`, which the rebase invalidates; deletes `chaindata/` (or, with `--keep-blocks`, resets its execution-state tables); and rewrites `snapshots/erigondb.toml` with the new `step_size` and a correspondingly rescaled `steps_in_frozen_file`. -The command prints the full rename and delete plan and asks for confirmation (`Proceed with these changes? [y/N]`) before touching anything, and exits as a no-op if the datadir is already at the requested step size. It is **destructive and has no built-in undo**, but it is safe to re-run after an interruption. The node must be **stopped** — it locks the datadir and opens `chaindata/` exclusively. The v3.4.0 release notes put the rebase itself at roughly ten seconds; the cost that follows is re-acquiring whatever was in `chaindata/`. +The command prints the full rename and delete plan and asks for confirmation (`Proceed with these changes? [y/N]`) before touching anything, and exits as a no-op if the datadir is already at the requested step size. It is **destructive and has no built-in undo**, but it is safe to re-run after an interruption. The node must be **stopped** — it locks the datadir (and, with `--keep-blocks`, additionally opens `chaindata/` exclusively). The v3.4.0 release notes put the rebase itself at roughly ten seconds; the cost that follows is re-acquiring whatever was in `chaindata/`. --- diff --git a/docs/site/static/llms-full.txt b/docs/site/static/llms-full.txt index fc8aba0772a..4f4c5b7d491 100644 --- a/docs/site/static/llms-full.txt +++ b/docs/site/static/llms-full.txt @@ -1968,6 +1968,112 @@ The `seg du` estimates table shows the projected disk footprint for each mode gi --- +## Commitment Snapshot Format *(New in v3.6)* + +A commitment snapshot file stores Merkle-Patricia trie branches. Each branch has to refer to the account and storage keys underneath it, and Erigon can encode those references in one of two ways: + +| Encoding | What a branch stores | File version stamp | +|----------|----------------------|--------------------| +| **Referenced** | *Shortened keys* — offsets into the accounts and storage files, which must be dereferenced on read | `v2.1` | +| **Plain** | The keys themselves, inline | `v2.2` | + +Plain files are **larger on disk**, but state reads and snapshot merges are **faster**, because nothing has to be looked up in the accounts and storage files first. + +v3.6 makes **plain the default** for newly written commitment files. In v3.5 and earlier the default was referenced. + +The choice is recorded per datadir as `references_in_commitment_branches` in `snapshots/erigondb.toml`. When that file omits the field, the compiled default applies — `false` (plain) from v3.6, `true` (referenced) before it. On a normal fresh sync the downloader delivers `erigondb.toml` along with the snapshot set, so the published set decides the regime. + +### What happens to an existing datadir + +Nothing breaks, and nothing is rewritten eagerly: + +- **Existing referenced files stay readable.** Whether a commitment file carries shortened references is a property of *that file* — its version stamp and step range — not of the current setting. The commitment domain reads `v2.1` and `v2.2` alike. +- **Conversion happens lazily, during merges.** When a merge has any referenced input, Erigon attaches a value transformer that expands the references; because new files are now written plain, the merged output lands as `v2.2`. Ordinary background merging therefore migrates the datadir on its own, with no operator action and no downtime. + +For most operators upgrading from v3.5 the correct action is **none**. The commands below exist for operators who want the plain format sooner than merging will deliver it, and who are prepared to pay for it. + +:::warning +Downgrading to v3.5 after v3.6 has written new snapshot files is **not supported** — the new snapshot and accessor formats are incompatible with v3.5. Back up your `--datadir` before taking either migration path below. +::: + +### Path 1 — re-download the snapshot set (`erigon seg reset`) + +Discards the local snapshot set and lets the next start pull the published v3.6 set, which is already plain. + +```bash +erigon seg reset --datadir /your/datadir +``` + +| Flag | Description | +|------|-------------| +| `--datadir ` | Path to the Erigon data directory. | +| `--local`, `-l` | A single switch over two behaviours: remove snapshot files that are not described in the preverified set (locally generated ones), **and** remove `chaindata/` together with the Heimdall / Polygon-bridge DBs. **Default: `true`.** `--local=false` keeps all of them. | +| `--dry-run`, `-n` | Print the files that would be removed and exit without removing anything. Default: `false`. | +| `--preverified ` | Which preverified snapshot list to reset against. Default: `remote`. | + +`seg reset` is **destructive and not reversible**: at the default `--local=true` it deletes `chaindata/`, the Heimdall / Polygon-bridge DBs, and any locally generated snapshot files. Independently of `--local` — that is, even with `--local=false` — it also deletes any `.torrent` file whose infohash does not match the preverified list, **together with the snapshot data file that torrent vouched for** (a stale local build is unverifiable, so the downloader must re-fetch the canonical copy). All other preverified snapshots already on disk are kept and corrected by the downloader on the next start. The node must be **stopped** — the command takes an exclusive lock on the datadir. + +Cost: whatever the downloader has to re-fetch, plus rebuilding state on the next start. Run `--dry-run` first, and back up the datadir. + +`seg reset` has no `--chain` flag of its own; it reads the chain name from `chaindata/`. If `chaindata/` is missing or unreadable, pass the chain as a root-level flag before the subcommand — `erigon --chain mainnet seg reset --datadir /your/datadir`. + +:::note +`erigon seg reset` is not new in v3.6 (`seg` is an alias for `snapshots`, so `erigon snapshots reset` is the same command). What is new is its role as a shortcut to the plain commitment format. See also [Snapshots Upgrade Options](../get-started/installation/upgrading#snapshots-upgrade-options). +::: + +### Path 2 — offline conversion (`integration commitment convert`) *(New in v3.6)* + +Re-encodes the commitment files already on disk instead of re-downloading them. Built from source as `build/bin/integration` (`make integration`). + +```bash +integration commitment convert --datadir /your/datadir --chain mainnet +``` + +Both encoding flags default to `false`, and `false` is the target state that matches what v3.6 writes — so the command above, with no encoding flags, converts referenced files to plain and leaves the key encoding alone. + +| Flag | Description | +|------|-------------| +| `--datadir ` | Path to the Erigon data directory. | +| `--chain ` | Chain name. | +| `--squeeze` | Target state of the value axis: `true` = referenced (offsets), `false` = plain. Default: `false`. | +| `--nibbles.v2` | Target state of the key axis: `true` = V2, `false` = V1 (compact bytes, what Erigon writes). Default: `false`. | +| `--restore` | Move the originals back from `snapshots/backup/domains/` and remove the converted files. Mutually exclusive with `--squeeze` / `--nibbles.v2` and with `--continue`. | +| `--continue` | Resume an interrupted run, reusing complete shards left in `snapshots/rebuild/domain/`. Mutually exclusive with `--restore`. | + +The flag value is the **target** state, not a direction: the tool detects each file's current encoding and converts only what needs it, skipping files already in the target state. + +**Is it reversible?** Yes — this is the one path with a real undo. The originals are moved to `snapshots/backup/domains/`, and `--restore` puts them back. Restore is filesystem-only and runs even when the datadir is too broken for Erigon to open it. Because a leftover backup blocks a second conversion, restore or clear `snapshots/backup/domains/` before converting again. + +**Is the node stopped?** Yes. The command opens the chaindata DB read-write and applies migrations. + +**What does it cost?** Every commitment `.kv` file that is not already plain is rewritten and its accessors (`.kvi` / `.bt` / `.kvei`) rebuilt. Converted output is written into `snapshots/rebuild/domain/` while the originals are still in place, so at peak you need free space for a second copy of the commitment domain — use the `commitment` row of `erigon seg du --datadir -v` to size it. Plain output is larger than the referenced input. + +:::warning +If a run is interrupted and you resume with `--continue`, you **must** pass the same `--squeeze` and `--nibbles.v2` values as the original run. A mismatch silently produces mixed-encoding output. Without `--continue`, a restart simply discards `snapshots/rebuild/domain/` and redoes the conversion from scratch. +::: + +### Prerequisite for pre-3.4 step sizes (`erigon seg step-rebase`) + +Erigon groups state files into *steps*. Up to v3.3 a step was 1,562,500 transactions; from v3.4 the default is **390,625** — a quarter of the old size, and still the default in v3.6. v3.4 and v3.5 offered the smaller step as an optional gain (about 4× smaller `chaindata/`) rather than forcing it, so a datadir first created by v3.3 or earlier can still be recorded at the legacy size. The v3.6 release notes require those datadirs to rebase, or to take the `seg reset` path above, before their first normal v3.6 start. + +Check `step_size` in `snapshots/erigondb.toml`. If it reads `1562500`: + +```bash +erigon seg step-rebase --datadir /your/datadir --new-step-size=390625 +``` + +| Flag | Description | +|------|-------------| +| `--datadir ` | Path to the Erigon data directory. | +| `--new-step-size ` | **Required.** Target step size. `390625` is the current default. Must be a whole multiple or a whole divisor of the datadir's present step size. | +| `--keep-blocks` | Keep `chaindata/` and reset only the execution-state tables inside it, so already-downloaded blocks can be re-executed instead of re-downloaded. Default: delete `chaindata/` entirely. | + +What it does: renames every state snapshot file under `snapshots/domain`, `history`, `accessor`, and `idx` so its step numbers are rescaled by the factor; deletes the `.torrent` files in those directories and `snapshots/erigondb.toml.torrent`, which the rebase invalidates; deletes `chaindata/` (or, with `--keep-blocks`, resets its execution-state tables); and rewrites `snapshots/erigondb.toml` with the new `step_size` and a correspondingly rescaled `steps_in_frozen_file`. + +The command prints the full rename and delete plan and asks for confirmation (`Proceed with these changes? [y/N]`) before touching anything, and exits as a no-op if the datadir is already at the requested step size. It is **destructive and has no built-in undo**, but it is safe to re-run after an interruption. The node must be **stopped** — it locks the datadir (and, with `--keep-blocks`, additionally opens `chaindata/` exclusively). The v3.4.0 release notes put the rebase itself at roughly ten seconds; the cost that follows is re-acquiring whatever was in `chaindata/`. + +--- + ## Upgrading Snapshots in v3.5 ### EIP-8252 retention window change @@ -2151,8 +2257,8 @@ These flags control database performance and memory usage. See [Database](/funda * `--erigondb.domain.steps-in-frozen-file value`: Overrides the `steps_in_frozen_file` setting from `erigondb.toml` for the domain merge cap only (history and inverted-index merges are unaffected). Pass a positive integer to set an explicit cap, or `Inf` to leave the domain merge unbounded. * Default: unset (uses the value from `erigondb.toml`) * Use with care — an incorrect value may affect database structure. -* `--commitment.plainValues` *(New in v3.6)*: On the first start of a fresh datadir, write commitment values in plain form (no shortened key references). This is a one-time, per-datadir choice — it is ignored once `erigondb.toml` exists. - * Default: `false` +* `--commitment.plainValues` *(New in v3.6)*: Chooses the commitment branch encoding recorded in `snapshots/erigondb.toml` at the moment Erigon creates that file. `true` writes commitment values in plain form (keys inline); `false` writes shortened key references. This is a one-time, per-datadir choice — it is ignored once `erigondb.toml` exists, and a `erigondb.toml` delivered by the downloader wins over it. See [Commitment Snapshot Format](/fundamentals/snapshots-management#commitment-snapshot-format). + * Default: plain. Leaving the flag out selects plain, which is v3.6's compiled default; passing `--commitment.plainValues=false` explicitly selects the older *referenced* encoding. Before v3.6 the compiled default was referenced. ### Pruning and Snapshots diff --git a/docs/site/static/llms.txt b/docs/site/static/llms.txt index 6e5927f3913..47095f1b304 100644 --- a/docs/site/static/llms.txt +++ b/docs/site/static/llms.txt @@ -29,7 +29,7 @@ The full text of every page listed below is also available as a single file: [ll - [Architecture](https://docs.erigon.tech/fundamentals/architecture): How Erigon is built — staged sync, modular processes, flat-DB on MDBX, immutable snapshots, and an embedded consensus layer. - [Database](https://docs.erigon.tech/fundamentals/database): How Erigon stores chain data — MDBX engine, datadir layout, snapshot files, and real mainnet sizing numbers. - [Pruning Modes](https://docs.erigon.tech/fundamentals/pruning-modes): Full, minimal, blocks, and archive pruning modes explained — choose the right mode for your use case. -- [Snapshots Management](https://docs.erigon.tech/fundamentals/snapshots-management): Understand and manage Erigon snapshot files — check disk usage by category, compare node type estimates, and migrate to v3.5 snapshot formats. +- [Snapshots Management](https://docs.erigon.tech/fundamentals/snapshots-management): Understand and manage Erigon snapshot files — check disk usage by category, compare node type estimates, and migrate commitment and step-size snapshot formats. - [Caplin](https://docs.erigon.tech/fundamentals/caplin): Erigon's built-in consensus layer — run a full node without an external CL client. - [CLI Reference](https://docs.erigon.tech/fundamentals/configuring-erigon): Complete CLI flag reference for Erigon — all startup options, environment variables, and configuration settings. - [Supported Networks](https://docs.erigon.tech/fundamentals/supported-networks): Mainnet, testnets, Gnosis, Polygon, and all other chains Erigon can sync. diff --git a/llms-full.txt b/llms-full.txt index fc8aba0772a..4f4c5b7d491 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -1968,6 +1968,112 @@ The `seg du` estimates table shows the projected disk footprint for each mode gi --- +## Commitment Snapshot Format *(New in v3.6)* + +A commitment snapshot file stores Merkle-Patricia trie branches. Each branch has to refer to the account and storage keys underneath it, and Erigon can encode those references in one of two ways: + +| Encoding | What a branch stores | File version stamp | +|----------|----------------------|--------------------| +| **Referenced** | *Shortened keys* — offsets into the accounts and storage files, which must be dereferenced on read | `v2.1` | +| **Plain** | The keys themselves, inline | `v2.2` | + +Plain files are **larger on disk**, but state reads and snapshot merges are **faster**, because nothing has to be looked up in the accounts and storage files first. + +v3.6 makes **plain the default** for newly written commitment files. In v3.5 and earlier the default was referenced. + +The choice is recorded per datadir as `references_in_commitment_branches` in `snapshots/erigondb.toml`. When that file omits the field, the compiled default applies — `false` (plain) from v3.6, `true` (referenced) before it. On a normal fresh sync the downloader delivers `erigondb.toml` along with the snapshot set, so the published set decides the regime. + +### What happens to an existing datadir + +Nothing breaks, and nothing is rewritten eagerly: + +- **Existing referenced files stay readable.** Whether a commitment file carries shortened references is a property of *that file* — its version stamp and step range — not of the current setting. The commitment domain reads `v2.1` and `v2.2` alike. +- **Conversion happens lazily, during merges.** When a merge has any referenced input, Erigon attaches a value transformer that expands the references; because new files are now written plain, the merged output lands as `v2.2`. Ordinary background merging therefore migrates the datadir on its own, with no operator action and no downtime. + +For most operators upgrading from v3.5 the correct action is **none**. The commands below exist for operators who want the plain format sooner than merging will deliver it, and who are prepared to pay for it. + +:::warning +Downgrading to v3.5 after v3.6 has written new snapshot files is **not supported** — the new snapshot and accessor formats are incompatible with v3.5. Back up your `--datadir` before taking either migration path below. +::: + +### Path 1 — re-download the snapshot set (`erigon seg reset`) + +Discards the local snapshot set and lets the next start pull the published v3.6 set, which is already plain. + +```bash +erigon seg reset --datadir /your/datadir +``` + +| Flag | Description | +|------|-------------| +| `--datadir ` | Path to the Erigon data directory. | +| `--local`, `-l` | A single switch over two behaviours: remove snapshot files that are not described in the preverified set (locally generated ones), **and** remove `chaindata/` together with the Heimdall / Polygon-bridge DBs. **Default: `true`.** `--local=false` keeps all of them. | +| `--dry-run`, `-n` | Print the files that would be removed and exit without removing anything. Default: `false`. | +| `--preverified ` | Which preverified snapshot list to reset against. Default: `remote`. | + +`seg reset` is **destructive and not reversible**: at the default `--local=true` it deletes `chaindata/`, the Heimdall / Polygon-bridge DBs, and any locally generated snapshot files. Independently of `--local` — that is, even with `--local=false` — it also deletes any `.torrent` file whose infohash does not match the preverified list, **together with the snapshot data file that torrent vouched for** (a stale local build is unverifiable, so the downloader must re-fetch the canonical copy). All other preverified snapshots already on disk are kept and corrected by the downloader on the next start. The node must be **stopped** — the command takes an exclusive lock on the datadir. + +Cost: whatever the downloader has to re-fetch, plus rebuilding state on the next start. Run `--dry-run` first, and back up the datadir. + +`seg reset` has no `--chain` flag of its own; it reads the chain name from `chaindata/`. If `chaindata/` is missing or unreadable, pass the chain as a root-level flag before the subcommand — `erigon --chain mainnet seg reset --datadir /your/datadir`. + +:::note +`erigon seg reset` is not new in v3.6 (`seg` is an alias for `snapshots`, so `erigon snapshots reset` is the same command). What is new is its role as a shortcut to the plain commitment format. See also [Snapshots Upgrade Options](../get-started/installation/upgrading#snapshots-upgrade-options). +::: + +### Path 2 — offline conversion (`integration commitment convert`) *(New in v3.6)* + +Re-encodes the commitment files already on disk instead of re-downloading them. Built from source as `build/bin/integration` (`make integration`). + +```bash +integration commitment convert --datadir /your/datadir --chain mainnet +``` + +Both encoding flags default to `false`, and `false` is the target state that matches what v3.6 writes — so the command above, with no encoding flags, converts referenced files to plain and leaves the key encoding alone. + +| Flag | Description | +|------|-------------| +| `--datadir ` | Path to the Erigon data directory. | +| `--chain ` | Chain name. | +| `--squeeze` | Target state of the value axis: `true` = referenced (offsets), `false` = plain. Default: `false`. | +| `--nibbles.v2` | Target state of the key axis: `true` = V2, `false` = V1 (compact bytes, what Erigon writes). Default: `false`. | +| `--restore` | Move the originals back from `snapshots/backup/domains/` and remove the converted files. Mutually exclusive with `--squeeze` / `--nibbles.v2` and with `--continue`. | +| `--continue` | Resume an interrupted run, reusing complete shards left in `snapshots/rebuild/domain/`. Mutually exclusive with `--restore`. | + +The flag value is the **target** state, not a direction: the tool detects each file's current encoding and converts only what needs it, skipping files already in the target state. + +**Is it reversible?** Yes — this is the one path with a real undo. The originals are moved to `snapshots/backup/domains/`, and `--restore` puts them back. Restore is filesystem-only and runs even when the datadir is too broken for Erigon to open it. Because a leftover backup blocks a second conversion, restore or clear `snapshots/backup/domains/` before converting again. + +**Is the node stopped?** Yes. The command opens the chaindata DB read-write and applies migrations. + +**What does it cost?** Every commitment `.kv` file that is not already plain is rewritten and its accessors (`.kvi` / `.bt` / `.kvei`) rebuilt. Converted output is written into `snapshots/rebuild/domain/` while the originals are still in place, so at peak you need free space for a second copy of the commitment domain — use the `commitment` row of `erigon seg du --datadir -v` to size it. Plain output is larger than the referenced input. + +:::warning +If a run is interrupted and you resume with `--continue`, you **must** pass the same `--squeeze` and `--nibbles.v2` values as the original run. A mismatch silently produces mixed-encoding output. Without `--continue`, a restart simply discards `snapshots/rebuild/domain/` and redoes the conversion from scratch. +::: + +### Prerequisite for pre-3.4 step sizes (`erigon seg step-rebase`) + +Erigon groups state files into *steps*. Up to v3.3 a step was 1,562,500 transactions; from v3.4 the default is **390,625** — a quarter of the old size, and still the default in v3.6. v3.4 and v3.5 offered the smaller step as an optional gain (about 4× smaller `chaindata/`) rather than forcing it, so a datadir first created by v3.3 or earlier can still be recorded at the legacy size. The v3.6 release notes require those datadirs to rebase, or to take the `seg reset` path above, before their first normal v3.6 start. + +Check `step_size` in `snapshots/erigondb.toml`. If it reads `1562500`: + +```bash +erigon seg step-rebase --datadir /your/datadir --new-step-size=390625 +``` + +| Flag | Description | +|------|-------------| +| `--datadir ` | Path to the Erigon data directory. | +| `--new-step-size ` | **Required.** Target step size. `390625` is the current default. Must be a whole multiple or a whole divisor of the datadir's present step size. | +| `--keep-blocks` | Keep `chaindata/` and reset only the execution-state tables inside it, so already-downloaded blocks can be re-executed instead of re-downloaded. Default: delete `chaindata/` entirely. | + +What it does: renames every state snapshot file under `snapshots/domain`, `history`, `accessor`, and `idx` so its step numbers are rescaled by the factor; deletes the `.torrent` files in those directories and `snapshots/erigondb.toml.torrent`, which the rebase invalidates; deletes `chaindata/` (or, with `--keep-blocks`, resets its execution-state tables); and rewrites `snapshots/erigondb.toml` with the new `step_size` and a correspondingly rescaled `steps_in_frozen_file`. + +The command prints the full rename and delete plan and asks for confirmation (`Proceed with these changes? [y/N]`) before touching anything, and exits as a no-op if the datadir is already at the requested step size. It is **destructive and has no built-in undo**, but it is safe to re-run after an interruption. The node must be **stopped** — it locks the datadir (and, with `--keep-blocks`, additionally opens `chaindata/` exclusively). The v3.4.0 release notes put the rebase itself at roughly ten seconds; the cost that follows is re-acquiring whatever was in `chaindata/`. + +--- + ## Upgrading Snapshots in v3.5 ### EIP-8252 retention window change @@ -2151,8 +2257,8 @@ These flags control database performance and memory usage. See [Database](/funda * `--erigondb.domain.steps-in-frozen-file value`: Overrides the `steps_in_frozen_file` setting from `erigondb.toml` for the domain merge cap only (history and inverted-index merges are unaffected). Pass a positive integer to set an explicit cap, or `Inf` to leave the domain merge unbounded. * Default: unset (uses the value from `erigondb.toml`) * Use with care — an incorrect value may affect database structure. -* `--commitment.plainValues` *(New in v3.6)*: On the first start of a fresh datadir, write commitment values in plain form (no shortened key references). This is a one-time, per-datadir choice — it is ignored once `erigondb.toml` exists. - * Default: `false` +* `--commitment.plainValues` *(New in v3.6)*: Chooses the commitment branch encoding recorded in `snapshots/erigondb.toml` at the moment Erigon creates that file. `true` writes commitment values in plain form (keys inline); `false` writes shortened key references. This is a one-time, per-datadir choice — it is ignored once `erigondb.toml` exists, and a `erigondb.toml` delivered by the downloader wins over it. See [Commitment Snapshot Format](/fundamentals/snapshots-management#commitment-snapshot-format). + * Default: plain. Leaving the flag out selects plain, which is v3.6's compiled default; passing `--commitment.plainValues=false` explicitly selects the older *referenced* encoding. Before v3.6 the compiled default was referenced. ### Pruning and Snapshots diff --git a/llms.txt b/llms.txt index 6e5927f3913..47095f1b304 100644 --- a/llms.txt +++ b/llms.txt @@ -29,7 +29,7 @@ The full text of every page listed below is also available as a single file: [ll - [Architecture](https://docs.erigon.tech/fundamentals/architecture): How Erigon is built — staged sync, modular processes, flat-DB on MDBX, immutable snapshots, and an embedded consensus layer. - [Database](https://docs.erigon.tech/fundamentals/database): How Erigon stores chain data — MDBX engine, datadir layout, snapshot files, and real mainnet sizing numbers. - [Pruning Modes](https://docs.erigon.tech/fundamentals/pruning-modes): Full, minimal, blocks, and archive pruning modes explained — choose the right mode for your use case. -- [Snapshots Management](https://docs.erigon.tech/fundamentals/snapshots-management): Understand and manage Erigon snapshot files — check disk usage by category, compare node type estimates, and migrate to v3.5 snapshot formats. +- [Snapshots Management](https://docs.erigon.tech/fundamentals/snapshots-management): Understand and manage Erigon snapshot files — check disk usage by category, compare node type estimates, and migrate commitment and step-size snapshot formats. - [Caplin](https://docs.erigon.tech/fundamentals/caplin): Erigon's built-in consensus layer — run a full node without an external CL client. - [CLI Reference](https://docs.erigon.tech/fundamentals/configuring-erigon): Complete CLI flag reference for Erigon — all startup options, environment variables, and configuration settings. - [Supported Networks](https://docs.erigon.tech/fundamentals/supported-networks): Mainnet, testnets, Gnosis, Polygon, and all other chains Erigon can sync. From 64fcab392b7d8b3ebbb28f1e3f342b1a23374fff Mon Sep 17 00:00:00 2001 From: Bloxster Date: Thu, 27 Aug 2026 08:53:26 +0200 Subject: [PATCH 3/3] docs: correct the commitment encoding rule, seg reset flag order, and reset cost - Referenced encoding needs both a sub-v2.2 stamp and a >=2-step range (CommitmentBranchReferenced), so a single-step v2.1 file is plain. - --chain is a root-level flag and resolves in either position. - Size seg reset by how far the node is past the preverified manifest: segments freeze only to 1,024 behind tip, and the tail returns through Caplin per-block backfill, not a segment fetch. --- .../docs/fundamentals/configuring-erigon.mdx | 2 +- .../fundamentals/snapshots-management.mdx | 22 +++++++++++++---- docs/site/static/llms-full.txt | 24 ++++++++++++++----- llms-full.txt | 24 ++++++++++++++----- 4 files changed, 54 insertions(+), 18 deletions(-) diff --git a/docs/site/docs/fundamentals/configuring-erigon.mdx b/docs/site/docs/fundamentals/configuring-erigon.mdx index a715ab7a407..1c54d81fea8 100644 --- a/docs/site/docs/fundamentals/configuring-erigon.mdx +++ b/docs/site/docs/fundamentals/configuring-erigon.mdx @@ -114,7 +114,7 @@ These flags control database performance and memory usage. See [Database](/funda * `--erigondb.domain.steps-in-frozen-file value`: Overrides the `steps_in_frozen_file` setting from `erigondb.toml` for the domain merge cap only (history and inverted-index merges are unaffected). Pass a positive integer to set an explicit cap, or `Inf` to leave the domain merge unbounded. * Default: unset (uses the value from `erigondb.toml`) * Use with care — an incorrect value may affect database structure. -* `--commitment.plainValues` *(New in v3.6)*: Chooses the commitment branch encoding recorded in `snapshots/erigondb.toml` at the moment Erigon creates that file. `true` writes commitment values in plain form (keys inline); `false` writes shortened key references. This is a one-time, per-datadir choice — it is ignored once `erigondb.toml` exists, and a `erigondb.toml` delivered by the downloader wins over it. See [Commitment Snapshot Format](/fundamentals/snapshots-management#commitment-snapshot-format). +* `--commitment.plainValues` *(New in v3.6)*: Chooses the commitment branch encoding recorded in `snapshots/erigondb.toml` at the moment Erigon creates that file. `true` writes commitment values in plain form (keys inline); `false` writes shortened key references. This is a one-time, per-datadir choice — it is ignored once `erigondb.toml` exists, and an `erigondb.toml` delivered by the downloader wins over it. See [Commitment Snapshot Format](/fundamentals/snapshots-management#commitment-snapshot-format). * Default: plain. Leaving the flag out selects plain, which is v3.6's compiled default; passing `--commitment.plainValues=false` explicitly selects the older *referenced* encoding. Before v3.6 the compiled default was referenced. ### Pruning and Snapshots diff --git a/docs/site/docs/fundamentals/snapshots-management.mdx b/docs/site/docs/fundamentals/snapshots-management.mdx index 5f3907d7018..6913bb8a3ac 100644 --- a/docs/site/docs/fundamentals/snapshots-management.mdx +++ b/docs/site/docs/fundamentals/snapshots-management.mdx @@ -125,10 +125,15 @@ The `seg du` estimates table shows the projected disk footprint for each mode gi A commitment snapshot file stores Merkle-Patricia trie branches. Each branch has to refer to the account and storage keys underneath it, and Erigon can encode those references in one of two ways: -| Encoding | What a branch stores | File version stamp | -|----------|----------------------|--------------------| -| **Referenced** | *Shortened keys* — offsets into the accounts and storage files, which must be dereferenced on read | `v2.1` | -| **Plain** | The keys themselves, inline | `v2.2` | +| Encoding | What a branch stores | When a file uses it | +|----------|----------------------|---------------------| +| **Referenced** | *Shortened keys* — offsets into the accounts and storage files, which must be dereferenced on read | Stamped below `v2.2` **and** spanning at least 2 steps | +| **Plain** | The keys themselves, inline | `v2.2` and later, or a sub-`v2.2` file spanning fewer than 2 steps | + +The version stamp alone does not decide the encoding. `CommitmentBranchReferenced` +(in `db/state/domain_committed.go`) requires both a version below `v2.2` *and* a range of at least +`commitment.DefaultKeyReferencingMinSteps` (2) steps, so a small `v2.1` file — a single unmerged step — is plain +despite its stamp. Plain files are **larger on disk**, but state reads and snapshot merges are **faster**, because nothing has to be looked up in the accounts and storage files first. @@ -168,7 +173,14 @@ erigon seg reset --datadir /your/datadir Cost: whatever the downloader has to re-fetch, plus rebuilding state on the next start. Run `--dry-run` first, and back up the datadir. -`seg reset` has no `--chain` flag of its own; it reads the chain name from `chaindata/`. If `chaindata/` is missing or unreadable, pass the chain as a root-level flag before the subcommand — `erigon --chain mainnet seg reset --datadir /your/datadir`. +Size that cost by how far the node is past the published preverified manifest, not by download time. The manifest can +lag the live tip considerably, and only the range it covers can be restored by fetching a torrent. Blocks past it are +held either in `chaindata/` or in locally built segments — and at `--local=true` `seg reset` removes both (the reset +path in `db/datadir/reset`). That range then comes back through Caplin's per-block beacon backfill rather than a segment +fetch, which is orders of magnitude slower, and a node far enough behind it may need more than one restart before +execution resumes. + +`seg reset` has no `--chain` flag of its own; it reads the chain name from `chaindata/`. If `chaindata/` is missing or unreadable, pass `--chain` — it is a root-level flag and resolves in either position: `erigon --chain mainnet seg reset --datadir /your/datadir` and `erigon seg reset --chain mainnet --datadir /your/datadir` behave the same. :::note `erigon seg reset` is not new in v3.6 (`seg` is an alias for `snapshots`, so `erigon snapshots reset` is the same command). What is new is its role as a shortcut to the plain commitment format. See also [Snapshots Upgrade Options](../get-started/installation/upgrading#snapshots-upgrade-options). diff --git a/docs/site/static/llms-full.txt b/docs/site/static/llms-full.txt index 4f4c5b7d491..c15ea704a90 100644 --- a/docs/site/static/llms-full.txt +++ b/docs/site/static/llms-full.txt @@ -1972,10 +1972,15 @@ The `seg du` estimates table shows the projected disk footprint for each mode gi A commitment snapshot file stores Merkle-Patricia trie branches. Each branch has to refer to the account and storage keys underneath it, and Erigon can encode those references in one of two ways: -| Encoding | What a branch stores | File version stamp | -|----------|----------------------|--------------------| -| **Referenced** | *Shortened keys* — offsets into the accounts and storage files, which must be dereferenced on read | `v2.1` | -| **Plain** | The keys themselves, inline | `v2.2` | +| Encoding | What a branch stores | When a file uses it | +|----------|----------------------|---------------------| +| **Referenced** | *Shortened keys* — offsets into the accounts and storage files, which must be dereferenced on read | Stamped below `v2.2` **and** spanning at least 2 steps | +| **Plain** | The keys themselves, inline | `v2.2` and later, or a sub-`v2.2` file spanning fewer than 2 steps | + +The version stamp alone does not decide the encoding. `CommitmentBranchReferenced` +(in `db/state/domain_committed.go`) requires both a version below `v2.2` *and* a range of at least +`commitment.DefaultKeyReferencingMinSteps` (2) steps, so a small `v2.1` file — a single unmerged step — is plain +despite its stamp. Plain files are **larger on disk**, but state reads and snapshot merges are **faster**, because nothing has to be looked up in the accounts and storage files first. @@ -2015,7 +2020,14 @@ erigon seg reset --datadir /your/datadir Cost: whatever the downloader has to re-fetch, plus rebuilding state on the next start. Run `--dry-run` first, and back up the datadir. -`seg reset` has no `--chain` flag of its own; it reads the chain name from `chaindata/`. If `chaindata/` is missing or unreadable, pass the chain as a root-level flag before the subcommand — `erigon --chain mainnet seg reset --datadir /your/datadir`. +Size that cost by how far the node is past the published preverified manifest, not by download time. The manifest can +lag the live tip considerably, and only the range it covers can be restored by fetching a torrent. Blocks past it are +held either in `chaindata/` or in locally built segments — and at `--local=true` `seg reset` removes both (the reset +path in `db/datadir/reset`). That range then comes back through Caplin's per-block beacon backfill rather than a segment +fetch, which is orders of magnitude slower, and a node far enough behind it may need more than one restart before +execution resumes. + +`seg reset` has no `--chain` flag of its own; it reads the chain name from `chaindata/`. If `chaindata/` is missing or unreadable, pass `--chain` — it is a root-level flag and resolves in either position: `erigon --chain mainnet seg reset --datadir /your/datadir` and `erigon seg reset --chain mainnet --datadir /your/datadir` behave the same. :::note `erigon seg reset` is not new in v3.6 (`seg` is an alias for `snapshots`, so `erigon snapshots reset` is the same command). What is new is its role as a shortcut to the plain commitment format. See also [Snapshots Upgrade Options](../get-started/installation/upgrading#snapshots-upgrade-options). @@ -2257,7 +2269,7 @@ These flags control database performance and memory usage. See [Database](/funda * `--erigondb.domain.steps-in-frozen-file value`: Overrides the `steps_in_frozen_file` setting from `erigondb.toml` for the domain merge cap only (history and inverted-index merges are unaffected). Pass a positive integer to set an explicit cap, or `Inf` to leave the domain merge unbounded. * Default: unset (uses the value from `erigondb.toml`) * Use with care — an incorrect value may affect database structure. -* `--commitment.plainValues` *(New in v3.6)*: Chooses the commitment branch encoding recorded in `snapshots/erigondb.toml` at the moment Erigon creates that file. `true` writes commitment values in plain form (keys inline); `false` writes shortened key references. This is a one-time, per-datadir choice — it is ignored once `erigondb.toml` exists, and a `erigondb.toml` delivered by the downloader wins over it. See [Commitment Snapshot Format](/fundamentals/snapshots-management#commitment-snapshot-format). +* `--commitment.plainValues` *(New in v3.6)*: Chooses the commitment branch encoding recorded in `snapshots/erigondb.toml` at the moment Erigon creates that file. `true` writes commitment values in plain form (keys inline); `false` writes shortened key references. This is a one-time, per-datadir choice — it is ignored once `erigondb.toml` exists, and an `erigondb.toml` delivered by the downloader wins over it. See [Commitment Snapshot Format](/fundamentals/snapshots-management#commitment-snapshot-format). * Default: plain. Leaving the flag out selects plain, which is v3.6's compiled default; passing `--commitment.plainValues=false` explicitly selects the older *referenced* encoding. Before v3.6 the compiled default was referenced. ### Pruning and Snapshots diff --git a/llms-full.txt b/llms-full.txt index 4f4c5b7d491..c15ea704a90 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -1972,10 +1972,15 @@ The `seg du` estimates table shows the projected disk footprint for each mode gi A commitment snapshot file stores Merkle-Patricia trie branches. Each branch has to refer to the account and storage keys underneath it, and Erigon can encode those references in one of two ways: -| Encoding | What a branch stores | File version stamp | -|----------|----------------------|--------------------| -| **Referenced** | *Shortened keys* — offsets into the accounts and storage files, which must be dereferenced on read | `v2.1` | -| **Plain** | The keys themselves, inline | `v2.2` | +| Encoding | What a branch stores | When a file uses it | +|----------|----------------------|---------------------| +| **Referenced** | *Shortened keys* — offsets into the accounts and storage files, which must be dereferenced on read | Stamped below `v2.2` **and** spanning at least 2 steps | +| **Plain** | The keys themselves, inline | `v2.2` and later, or a sub-`v2.2` file spanning fewer than 2 steps | + +The version stamp alone does not decide the encoding. `CommitmentBranchReferenced` +(in `db/state/domain_committed.go`) requires both a version below `v2.2` *and* a range of at least +`commitment.DefaultKeyReferencingMinSteps` (2) steps, so a small `v2.1` file — a single unmerged step — is plain +despite its stamp. Plain files are **larger on disk**, but state reads and snapshot merges are **faster**, because nothing has to be looked up in the accounts and storage files first. @@ -2015,7 +2020,14 @@ erigon seg reset --datadir /your/datadir Cost: whatever the downloader has to re-fetch, plus rebuilding state on the next start. Run `--dry-run` first, and back up the datadir. -`seg reset` has no `--chain` flag of its own; it reads the chain name from `chaindata/`. If `chaindata/` is missing or unreadable, pass the chain as a root-level flag before the subcommand — `erigon --chain mainnet seg reset --datadir /your/datadir`. +Size that cost by how far the node is past the published preverified manifest, not by download time. The manifest can +lag the live tip considerably, and only the range it covers can be restored by fetching a torrent. Blocks past it are +held either in `chaindata/` or in locally built segments — and at `--local=true` `seg reset` removes both (the reset +path in `db/datadir/reset`). That range then comes back through Caplin's per-block beacon backfill rather than a segment +fetch, which is orders of magnitude slower, and a node far enough behind it may need more than one restart before +execution resumes. + +`seg reset` has no `--chain` flag of its own; it reads the chain name from `chaindata/`. If `chaindata/` is missing or unreadable, pass `--chain` — it is a root-level flag and resolves in either position: `erigon --chain mainnet seg reset --datadir /your/datadir` and `erigon seg reset --chain mainnet --datadir /your/datadir` behave the same. :::note `erigon seg reset` is not new in v3.6 (`seg` is an alias for `snapshots`, so `erigon snapshots reset` is the same command). What is new is its role as a shortcut to the plain commitment format. See also [Snapshots Upgrade Options](../get-started/installation/upgrading#snapshots-upgrade-options). @@ -2257,7 +2269,7 @@ These flags control database performance and memory usage. See [Database](/funda * `--erigondb.domain.steps-in-frozen-file value`: Overrides the `steps_in_frozen_file` setting from `erigondb.toml` for the domain merge cap only (history and inverted-index merges are unaffected). Pass a positive integer to set an explicit cap, or `Inf` to leave the domain merge unbounded. * Default: unset (uses the value from `erigondb.toml`) * Use with care — an incorrect value may affect database structure. -* `--commitment.plainValues` *(New in v3.6)*: Chooses the commitment branch encoding recorded in `snapshots/erigondb.toml` at the moment Erigon creates that file. `true` writes commitment values in plain form (keys inline); `false` writes shortened key references. This is a one-time, per-datadir choice — it is ignored once `erigondb.toml` exists, and a `erigondb.toml` delivered by the downloader wins over it. See [Commitment Snapshot Format](/fundamentals/snapshots-management#commitment-snapshot-format). +* `--commitment.plainValues` *(New in v3.6)*: Chooses the commitment branch encoding recorded in `snapshots/erigondb.toml` at the moment Erigon creates that file. `true` writes commitment values in plain form (keys inline); `false` writes shortened key references. This is a one-time, per-datadir choice — it is ignored once `erigondb.toml` exists, and an `erigondb.toml` delivered by the downloader wins over it. See [Commitment Snapshot Format](/fundamentals/snapshots-management#commitment-snapshot-format). * Default: plain. Leaving the flag out selects plain, which is v3.6's compiled default; passing `--commitment.plainValues=false` explicitly selects the older *referenced* encoding. Before v3.6 the compiled default was referenced. ### Pruning and Snapshots