wb | genesis: leaner cache and Plutus cost models#6544
Open
fmaste wants to merge 21 commits into
Open
Conversation
fmaste
force-pushed
the
bench/genesis
branch
3 times, most recently
from
a023ad9 to
1eaa705
Compare
fmaste
force-pushed
the
bench/genesis
branch
9 times, most recently
from
c24ca3d to
fc69ed3
Compare
fmaste
force-pushed
the
bench/genesis
branch
18 times, most recently
from
247e461 to
5dc42e6
Compare
Both genesis backends, "jq" and "modular", now live in the same file. Extract unmodified the code of each to new files.
Resolve `$genesis_backend` once at source time and replace all the 'if WB_MODULAR_GENESIS' with a single `"X-$genesis_backend" "$@"`. Applies to `profile-cache-key-input`, `profile-cache-key`, `spec` and `pool-relays`. Also, the decision between `create-testnet-data` and `create-staked` (WB_CREATE_TESTNET_DATA), now lives in the "jq" backend (the "modular" backend only implements spec, pool-relays, byron and cache key functions, not the create-testnet-data call).
Function `derive-from-cache` becomes a backend implementation. Both `derive-from-cache` and `finalise-cache-entry` are merged and inlined into `derive-from-cache-jq` (`derive-from-cache-modular` delegates to "jq").
Move the preset-dispatch branch out of the per-call helpers so the "from preset" path is resolved once at source time, matching the other backend-dispatch consolidations.
`create-testnet-data` is now the only supported path.
Replaces 'cardano-cli byron genesis genesis'.
This introduces changes to the cache output so the layout version was updated.
Removes genesis-byron-{jq,modular} and the genesis-byron dispatcher.
fmaste
changed the title
…e-testnet-data`
Was converging to `create-staked` output, keep `create-testnet-data` layout:
- pools-keys/poolN/{cold,kes,vrf,opcert}.{skey,vkey,cert,counter}
- stake-delegators/delegatorN/{payment,staking}.{skey,vkey}
- drep-keys/drepN/drep.{skey,vkey}
- utxo-keys/utxoN/utxo.{skey,vkey}
- genesis-keys/genesisN/key.{skey,vkey}
- delegate-keys/delegateN/{key,kes,vrf,opcert}.{skey,vkey,cert,counter}
- byron-gen-command/
Normalises genesis file names. Every era's genesis file is now genesis.<era>.json everywhere:
- genesis.byron.json
- genesis.shelley.json
- genesis.alonzo.json
- genesis.conway.json
- genesis.dijkstra.json
Details:
- create-testnet-data drops its symlink-creation block and the `link_keys` helper (no longer needed under the native layout).
- Removes `Massage_the_key_file_layout_to_match_AWS`, `key_depl` and `key_genesis`.
The names `create-testnet-data` itself emits are unchanged and the workbench's downstream consumers are updated to read the normalised form.
The pool relays written into the Shelley genesis (via `pool-relays.json` and `cardano-cli ... --relays` / `--relay-specification-file`) are only consumed by ledger-peer discovery. With ledger peers off and `publicRoots` empty, the relay records in genesis are never read. Every workbench topology disables it: - the supervisor (local / 127.0.0.1) backend gets `useLedgerAfterSlot: -1` from `cardano-topology projection-for`. - the Nomad backend hardcodes the same value (nix/workbench/backend/nomad-job.nix).
The "ripper" backend assembles a run's genesis by combining: - a cached **dataset**: UTXO, delegators, DReps, pool/staking keys - a cached **protocol**: everything else The existing "jq" backend stays as the default
These cost models were removed when the node started accepting only what mainnet shipped with.
- Structure: cost models are separated from the rest of the parameters. - Consistency: Shelley genesis is the full object, not only the "protocolParams" field. - Consistency: cost models are an object everywhere, no arrays. - Cost model names are updated.
fmaste
changed the title
Going forward, the only way to inject cost models with any number of parameters is through Alonzo genesis "extraConfig" field.
Based on a mainnet db-sync and mainnet genesis files, reconstruct the timeline without baking in cardano-profile-specific assumptions.
Remove the "genesis spec" workbench command that was used for `create-testnet-data --spec-*` and instead use what cardano-profile provides. The profile may have null "conway" or "dijkstra" genesis fields, but the node still needs valid files. Use a "zero" genesis in those cases.
cardano-profile recognises the pparamsEpoch values that target PV 11 and adds a dijkstra zero stub and a v11-preview overlay so profiles can opt in.
Era was previously inferred from .era inside profile.json. Promote it to a first-class workbench-level input.
The info banner sat inside the genesis() function body, so it fired once per `wb genesis <op>` dispatch
Restructure the genesis cache around a small backend interface (cache-hit, create-cache, derive-from-cache) with per-era zero-spec-ERA emitters. Entries now commit atomically: each backend fills a temp dir the caller mv's into place (shared at top level, and per dataset/protocol sub-cache in the ripper). Fix a ripper leak: create-testnet-data bakes shelley minUTxOValue and conway dRepDeposit into dataset fields, but the ripper cached the zero-spec defaults. It now injects the profile's real values into the temp specs and folds them into the dataset cache key, matching the jq output. Also rewrite zero/README.md to match verified behavior.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Workbench genesis
Summary
cat,sed, andjq.extraConfigfield, this PR adds support.What's in this PR
Genesis creation
cardano-cli latest genesis create-testnet-data. The retiredcreate-stakedpath and the separate byron genesis step are gone, byron now comes fromcreate-testnet-datatoo.cache-hit,create-cache,derive-from-cache, with one shared atomic commit (build in a temp dir,mvinto place, so an interrupted run never leaves a half-written cache).create-testnet-databakes shelleyminUTxOValueand conwaydRepDepositinto dataset fields, so both are now part of the dataset cache key and injected into the specs.Ripper backend
cat,sed, andjq, with nojqpass over the large dataset and no pretty-printing.create-testnet-data's output exactly, so the genesis the node sees is unchanged.Plutus cost models and profiles
genesis.alonzo.extraConfig, the only path cardano-node still accepts for non-default parameter counts. The mainnet cost-model genesis fields were locked to fixed entry counts (cardano-ledger#5241, cardano-ledger#5379), andextraConfig.costModelswas added as the bypass for testnets and benchmarks (cardano-ledger#5342, surfaced for the CLI in cardano-cli#1352).cardano-profile.epoch-timeline.jsonfrom a mainnet db-sync run and mainnet genesis, and restore the cost models that had been removed.Era as a first-class input
--era-nameflag onwb startandwb run allocate, validated up front.coay,bage,dira, ...) and written straight tometa.jsonas.meta.era, instead of being reconstructed downstream.Simpler CLI and layout
wb run allocatetakes named flags only, and the profile name is read fromprofile.json(one source of truth, no positional arguments).genesis.<era>.json, and the cache layout matches whatcreate-testnet-dataactually writes.--relaysmetadata, the genesis "spec" sub-command, and several unused key-shuffling helpers.WB_BACKENDbecomesWB_BACKEND_NAME.Compatibility
wb analyseand the run helpers fall back to the previous genesis filenames and layout when the new ones are absent.Breaking changes
wb startandwb run allocatenow require--era-name.wb run allocateno longer accepts positional arguments; use the named flags.WB_BACKENDis renamed toWB_BACKEND_NAME.--relays, the genesis "spec" sub-command, andcreate-stakedare removed.genesis.alonzo.extraConfig.