bench-state-gen: fat accounts and prepaid jetton wallets - #2529
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9c851f03e5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| auto jw_data0 = build_jw_data(kPrepaidJettonBalance, info.w5_addr, minter_addr); | ||
| info.jw_addr = build_state_init(contracts.jw_code, jw_data0)->get_hash().bits(); |
There was a problem hiding this comment.
When the updated bench-spam is used with an existing version-1 manifest/state, load_inputs() validates only w5_code_hash, while this function unconditionally uses the replaced prepaid jetton code and layout. The derived W5 address still exists, so its external message is accepted, but it sends the transfer to a newly derived jetton-wallet address that is absent from the legacy state, silently invalidating benchmark results. Either version the layout, retain the legacy derivation path, or reject manifests whose jw_code_hash does not match the loaded artifact.
Useful? React with 👍 / 👎.
| std::vector<Ref<vm::Cell>> w5_roots{ctx.contracts.w5_code, w5_data}; | ||
| ctx.w5_used = compute_account_storage_used(cfg.v5_balance, w5_roots); | ||
| auto jw_data = build_jw_data(cfg.jw_jetton_balance, sample.w5_addr, ctx.minter_addr, ctx.contracts.jw_code); | ||
| auto jw_data = build_jw_data(kPrepaidJettonBalance, sample.w5_addr, ctx.minter_addr); |
There was a problem hiding this comment.
For every generated V5 pair, this now stores kPrepaidJettonBalance (1,000,000) in the jetton wallet, but Config::total_supply() and manifest.jw_jetton_balance still use the old 1,000,000,000,000,000-per-wallet value. Thus any nonempty generated state claims a minter supply one billion times the sum of its initialized wallet balances and publishes an incorrect per-wallet balance in the manifest; update the supply and manifest values to reflect the prepaid balance.
Useful? React with 👍 / 👎.
| p.add_checked_option('\0', "fats-count", "number of fat load-target accounts (large storage dict)", | ||
| [&](td::Slice arg) { | ||
| TRY_RESULT_ASSIGN(cfg.num_fats, td::to_integer_safe<td::uint64>(arg)); | ||
| return td::Status::OK(); |
There was a problem hiding this comment.
When the newly accepted --fats-count option is passed to the self-test command, it remains set in the base configuration used by self_test_empty_root(), which clears only num_v5 and num_ballast. That test therefore generates fat accounts, fails res.total_balance == 0, and aborts; reset num_fats there so the option does not break the empty-state check.
Useful? React with 👍 / 👎.
…slab per read A bundle record stores, under the root cell's hash, the root plus several levels of its subtree as a flat slab (children-before-parents, internal refs by index, external refs as level_mask+hash+depth like plain-record children). One DB read materializes the whole slab as DataCells; ext cells are created only at the cut. Written only by offline tooling (benchmark state generator); the validator reads them transparently through CellLoader (both V1 and V2 dynamic BoC dbs). refcnt merge and load_refcnt handle the new tag; celldb's compress-depth migration explicitly skips bundle records so it never rewrites them as plain cells. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
9c851f0 to
cc91b94
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cc91b94d36
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| jetton transfers, sends via liteserver at a configured rate, tracks inclusion by | ||
| scanning new blocks, reports TPS/latency/block stats (JSON + CSV). | ||
| 5. **Orchestration**: `benchmark/bench_jetton.py` — builds network on /mnt/bench, | ||
| 5. **Orchestration**: `test/integration/bench_jetton.py` — builds network on /mnt/bench, |
There was a problem hiding this comment.
This commit deletes the only bench_jetton.py implementation while changing the documented component to test/integration/bench_jetton.py; a repository-wide file search shows that no replacement exists there or elsewhere. Consequently, the documented end-to-end workflow for checkpointing the generated state, starting the tontester network, running bench-spam, and collecting results can no longer be invoked. Move the deleted script to the new path or retain it at its original location.
Useful? React with 👍 / 👎.
…epth Fold the whole benchmark line into one benchmark-only change on testnet: - Fat load-target accounts: AccountType::Fat with a large storage dict (--fats-count / --fats-size), fats.addrs/.sizes sidecars, and a `derive` command that dumps the deterministic pubkey / w5 / jetton-wallet / minter / fat addresses for the Go cross-toolchain parity vectors. - Prepaid jetton wallet swap: build_jw_data drops the code ref and derives the address from PREPAID_BALANCE (jetton-wallet.code.boc replaced; fat.code.boc added; contracts.json / README updated). - --bundle-depth (from d5231a1): bundle the accounts-dict interior into celldb records; the two features coexist in build_account_cells (the fat data root is retained into the bundle, deeper storage cells stay external) and in self_test_celldb (bundling-invariance check at depth 0/5 plus fats coverage, num_fats=20). - Drop bench_jetton.py and take the feature/bench spam.cpp / DESIGN.md / RESULTS.md. Formatting normalized with clang-format-22 (repo .clang-format) so the benchmark C/C++ passes the ton CI lint check. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cc91b94 to
5fb40b9
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5fb40b9ad3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // fats.addrs (+ .sizes sidecar): the raw addresses of the fat load-targets, for go-spam --mode fats. | ||
| // Re-derived here (they aren't retained after phase 1); deterministic from the seed, so every host | ||
| // that regenerates the same state writes the identical file. | ||
| if (cfg.num_fats > 0) { |
There was a problem hiding this comment.
When gen --overwrite reuses an output directory that previously contained fat accounts and the new run sets --fats-count 0, this conditional neither rewrites nor removes fats.addrs and fats.sizes; --overwrite only deletes the existing celldb. The regenerated manifest and state then contain no fat accounts while go-spam --mode fats can still consume the stale addresses and target nonexistent accounts, invalidating the benchmark. Remove the sidecars or write empty files when num_fats is zero.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bd27d83ff7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return jw_jetton_balance * num_v5; | ||
| } | ||
| td::uint64 num_accounts() const { | ||
| return 2 * num_v5 + num_ballast + (num_v5 > 0 ? 1 : 0); |
There was a problem hiding this comment.
When --fats-count is nonzero, both derivation and build increment progress.accounts for every fat account, but this total omits num_fats. As a result, progress.json reaches and clamps to 100% before those accounts have been processed, and its accounts_total, phase percentage, and ETA are incorrect for fat-heavy state generation.
Useful? React with 👍 / 👎.
| merge_total_ = run_bytes; | ||
| merge_total_captured_ = true; |
There was a problem hiding this comment.
With progress snapshots enabled, this uses the full run-file byte count as total, including the 36-byte record framing and duplicate records, while merged_bytes advances only by 32 plus the selected distinct value size. The two counters therefore cannot converge even without deduplication, and default bundling adds a discarded plain record for every bundle root, so merge progress remains below 100% and reports a misleading ETA until the phase abruptly changes.
Useful? React with 👍 / 👎.
The ProgressPrinter thread additionally writes an atomic progress.json snapshot each tick (--progress-file / --progress-interval): current phase, per-phase and global weighted percent, rate and ETA. Best-effort side output, wrapped so it can never throw or slow generation, and it does not change the generated CellDB (root hash verified identical with and without the flag). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
bd27d83 to
09d838f
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 09d838f037
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| int num_cells = std::max(1, fats_size / 127); | ||
| auto chain = build_ballast_chain(addr, num_cells); |
There was a problem hiding this comment.
When --fats-size is at least 130,175 bytes, this creates at least 1,025 linearly linked cells; adding the storage root makes its depth exceed TON's CellTraits::max_depth of 1,024, so finalize_novm() throws and generation aborts even though the CLI accepts the value. Build the advertised large storage as a branching dictionary, or reject sizes that cannot fit within the depth limit.
Useful? React with 👍 / 👎.
2 participants
Adds
--fats-count/--fats-sizeto bench-state-gen: fat load-target accounts with a large storage dict (AccountType::Fat, raw addresses written tofats.addrs), and switches the paired jetton wallets to the prepaid layout. Adds aderivesubcommand for cross-toolchain address parity.self-testpasses.