diff --git a/doc/ant_colony_mining.md b/doc/ant_colony_mining.md new file mode 100644 index 00000000..881d6501 --- /dev/null +++ b/doc/ant_colony_mining.md @@ -0,0 +1,417 @@ +# Ant Colony Mining + +This document has two parts: + +- **Part 1 - Overview**: what ant-colony mining is. +- **Part 2 - Miner / pool integration guide**: the exact on-the-wire contract. + +> Config values (threshold, freshness window, child cap) are per-epoch and can change between epochs. +> Always read the live values from the epoch-context query (Part 2, section 2.7a). Numbers quoted in +> this document are current defaults, not constants to hard-code. + +> **Reading this doc.** Part 1 explains the split between the ant-colony *structure* and the mining +> *algorithm*. The parts specific to today's algorithm - the score's range and direction, the nonce +> knobs and their ranges, the mutation walk, and the numeric constants - are labelled **bpp9000** below; +> do not treat them as fixed ant-colony rules. + +--- + +## Part 1 - Overview + +**Ant colony is not a mining algorithm - it is a search *structure*.** It does not decide what a good +solution looks like or how to score one - a **mining algorithm** does. `nonce[0]` selects which +algorithm runs; today the only one is **bpp9000**, which the ant colony runs by default. A future +algorithm could be added the same way, in the same structure. So "ant colony" and "bpp9000" are two +separate things: the structure, and the algorithm currently running in it. + +| The ant-colony **structure** provides (any algorithm) | The **algorithm** provides (bpp9000 today) | +|---|---| +| a per-identity tree of solutions, grown from parents | what a solution *is* | +| the accept rules, the deposit, and the ranking | how to derive a child solution from a parent | +| the request / response queries | how to score a solution | + +With that split in mind: mining is a search for a **solution** that does well on a fixed task, and the +algorithm defines what a solution is and how it scores. Under **bpp9000** a solution is a neural network +(an "ANN"), scored by an **error count** over the task's data windows - range `[0, 8088]`, **lower is +better** (a flawless network makes zero mistakes). The rest of this overview uses bpp9000's terms, but +the tree structure around them is identical for any algorithm. + +Under bpp9000 the network's **wiring** (which neuron reads which) is **global** - the same graph for +everyone, from the epoch's task file - and a miner searches over each neuron's **lookup table (LUT)**, +the ternary function it computes. + +Standalone mining searches alone: every attempt starts from scratch. Ant-colony mining searches +**together, as a tree**: + +- Every **mining identity** (a computor or candidate public key) owns its **own tree** - the colony is + a per-identity forest. A pool's workers extend the tree of the computor they mine for. +- Each tree starts from a **virtual root**: a starting solution derived from that identity's public key + and the epoch's spectrum digest. It is fixed for the epoch, identical every time you derive it, and + is never stored or submitted. +- To mine, you pick a **parent** (the root, or any node already in your tree), **inherit** it, vary it + under your nonce, and score the result. +- If the result **strictly beats the parent** and clears the epoch **threshold**, you **submit** it. On + acceptance it becomes a new tree node that you - or anyone - can extend further. + +So the colony converges toward better solutions: each accepted node beats its parent, and the next +miner starts from there instead of from scratch. The goal of the epoch is the single best solution +found anywhere in the forest. + +``` + virtual root (per identity, not stored) + | + +----+----+ + | | + node A node B each node beats its parent + | | and clears the threshold + node C node D + | + node E <-- best in this tree so far +``` + +Concretely, error gates every attachment: it only falls down a branch (a child must beat its parent), +and a *start* - a depth-1 child of the root - must clear the threshold. + +``` + error = error count, lower is better threshold = 3838 + + root ~4044 raw a fresh root sits above 3838; a start must mutate below it + | + +-- A 3790 <= threshold ACCEPT (depth-1 start) + | | + | +-- B 3540 < 3790, beats A ACCEPT + | | | + | | +-- D 3120 < 3540, beats B ACCEPT + | | +-- E 3560 not < 3540 REJECT (must beat parent) + | | + | +-- C 3700 < 3790, beats A ACCEPT + | + +-- X 3900 > threshold REJECT (over threshold) + + Error only falls as you go deeper. The epoch winner is the single lowest-error node + found in any identity's forest. +``` + +At epoch end the node ranks every identity by its **single best** score and **harvests the top 676** +(the number of computors). + +**Anti-spam deposit.** Each solution a computor publishes on-chain carries a **refundable +1,000,000 QU deposit**, funded by the computor - not the worker. It is returned when the solution is +accepted **and** its claimed score matches the node's recompute; otherwise it is kept. So a computor +only publishes solutions it has already validated, and an honest, correct one costs nothing. + +--- + +## Part 2 - Miner / pool integration guide + +**In short.** A miner works one identity's tree. It reads the epoch context, takes a **parent** (the +identity's virtual root, or a node already in the tree), picks a canonical **nonce**, inherits the +parent's network, and **mutates and scores** it - reproducing the node's score exactly. If the result +**beats its parent** and **clears the threshold**, it hands the solution to the **computor**, which +re-checks it and **publishes it on-chain**; every node then recomputes the score, folds it into +consensus, attaches the node to the tree, and refunds the deposit. A miner's entire job is an **exact +scorer** - the tree, gates, deposit, and queries are the wrapper around it. + +### 2.1 The mining loop + +1. **Epoch context** - `REQUEST_ANT_EPOCH_CONTEXT` (public). Read the threshold, freshness window, + epoch spectrum digest, and child cap for this epoch, and **verify your task file** against the + returned `topologyHash` / `dataHash` (section 2.7a) before doing any work. +2. **Get a starting point** - derive your identity's virtual root, or fetch an existing node you want + to extend (`REQUEST_ANT_PARENT_ANN`). +3. **Pick a parent** - the root, or any node in your own tree. +4. **Search** - choose a nonce (section 2.2), inherit the parent LUT, run the mutation walk, score + (section 2.3). +5. **Submit** - if it passes the local rules (section 2.4), send `AntSolutionBroadcastPayload` to the + computor you mine for (section 2.6, stage 1). +6. **Publish + confirm** - the computor validates and scores it, then publishes it on-chain as an + `AntColonyMiningSolutionTransaction` (section 2.6, stage 2). Every node processes that transaction - + recompute, fold the digest, commit to the computor's tree, refund the deposit. Query the tree + (section 2.7b) to see accepted nodes and extend them. + +### 2.2 The nonce (32 bytes) + +| Byte(s) | Meaning | Valid range | +|-------------|---------|-------------| +| `nonce[0]` | algorithm selector (must select bpp9000) | - | +| `nonce[1]` | `L` = LUT entries rewritten per mutation step | `[1, 10]` | +| `nonce[2]` | `K` = number of **explore** steps | `[0, 100]` | +| `nonce[3..31]` | the walk seed (the actual search space) | any | + +**Canonical-nonce rule.** The scorer **refuses** any non-canonical nonce - it returns no score, and the +node rejects the submission with `RejectNonCanonicalNonce`. For an ant solution the rule is: + +``` +algo == bpp9000 && nonce[1] in [1, 10] && nonce[2] in [0, 100] +``` + +`nonce[0..2]` (the algo / `L` / `K` knobs) are **excluded from the RNG seed** - zeroed before hashing - +so `L` and `K` can be chosen without changing the walk seed. This makes the score-relevant bytes equal +to the identity/dedup bytes: there is no malleability room, and two nonces that differ only in these +knobs are not two solutions. + +### 2.3 Scoring - bpp9000 (must be bit-exact) + +Throughout, `publicKey` is the **mining identity you are extending** - the computor you mine for, which +becomes the transaction's `sourcePublicKey`. Derive the root and the mutation seed from **that** key, +not your worker key, or the node's recompute will not match yours. + +**Root.** `deriveRootANN(publicKey, epochPool)`: `K12(publicKey)` seeds a per-neuron LUT from the +epoch's random pool (the pool comes from the epoch-start spectrum digest). No mutation walk. Never +stored. The same every time for the epoch. + +**Child.** `computeScoreFromParent(parentLUT, publicKey, nonce, anchorTickDigest)`: + +1. Inherit `parentLUT`. +2. `mutationSeed = K12(publicKey || nonce[3..31] || anchorTickDigest)` (`nonce[0..2]` zeroed). +3. Walk `numberOfMutations = 100` steps. Each step rewrites `L` LUT entries. For the first `K` steps + accept a worse-or-equal result (**explore**); after that accept only better-or-equal (**exploit**); + one-step rollback on reject. Keep and return the **best** score seen. The best is seeded with the + inherited network's own score, so a child that fails to improve on its parent is rejected (see + `RejectLeParent`). + +**Anchor digest.** `anchorTickDigest = K12(anchorTick || transactionDigest)`, where `transactionDigest` +is `K12(TickData)` of the anchor tick's `TickData` (`REQUEST_TICK_DATA`). This binds a solution to a tick. + +**Empty ticks carry no anchor.** A tick without `TickData` records no anchor digest, and a published +solution whose `anchorTick` is an empty tick is rejected (`RejectStale`) with the **deposit +forfeited**. Anchor only on ticks that have `TickData`, make sure select a non-empty tick as an anchor tick. + +Score is an error count in `[0, 8088]`; lower is better. + +### 2.4 Accept rules + +A submission is accepted (`Valid` or `ValidNotStored`) only if **all** of these hold. The node checks +in this order; the first failure is the reject reason: + +| Check | Reject reason if it fails | +|-------|---------------------------| +| Parent record exists | `RejectParentNotRegistered` | +| Parent is in the **same** identity's tree (the tx `sourcePublicKey`) | `RejectWrongTree` | +| Nonce is canonical | `RejectNonCanonicalNonce` | +| Anchor not in the future, published within `freshnessWindow` ticks of it | `RejectStale` / `RejectTickOutOfRange` | +| Score `<=` epoch threshold | `RejectBelowThreshold` | +| Score **strictly** below the parent's score | `RejectLeParent` | +| Parent holds fewer than `maxChildrenPerParent` children (`0` = unbounded) | `RejectMaxChildrenPerParent` | +| `(publicKey, parentRef, nonce)` not already committed this epoch | `RejectReplay` | +| Store and miner index not full | `RejectDedupFull` / `RejectMinerIndexFull` | + +Separately, the **claimed score must equal the node's recompute** - if not, the solution may still be +recorded but the **deposit is kept** and the miner is **not ranked**. + +**Starting a tree.** The root's record score is the worst possible value, so a first (depth-1) child +passes the "beats parent" check trivially - the **threshold is the only score gate** for starting a +tree. A random root scores far above the threshold, so a start still requires real mutation. + +**`ValidNotStored`.** Accepted, refunded, and ranked exactly like `Valid`, but the per-epoch store was +full so the node was not persisted for others to extend. Ranking and refund are unaffected. + +### 2.5 The deposit + +Every on-chain `AntColonyMiningSolutionTransaction` carries a **1,000,000 QU** deposit +(`SOLUTION_SECURITY_DEPOSIT`), funded by the **computor** that publishes it - not the miner (see 2.6). +It is refunded **iff** the solution is accepted (`Valid` / `ValidNotStored`) **and** the claimed score +equals the node's recompute; otherwise it is kept. So a computor risks its own deposit and therefore +pre-validates each solution before publishing; a worker posts nothing on-chain. Keep the computor +identity funded above the deposit. + +### 2.6 Submission: broadcast (off-chain), then transaction (on-chain) + +A solution travels in two stages - an off-chain hand-off to a computor, then the on-chain transaction +that is the actual consensus record. + +**Stage 1 - P2P broadcast (`AntSolutionBroadcastPayload`, 48 bytes).** The miner hands its solution to +the computor it mines for, inside the standard `BroadcastMessage` envelope (network type +`BROADCAST_MESSAGE`), message type `MESSAGE_TYPE_ANT_SOLUTION` (`3`): + +``` +BroadcastMessage { // 96-byte envelope + m256i sourcePublicKey; // the worker key - pool off-chain accounting only, NOT the tree + m256i destinationPublicKey; // the COMPUTOR you mine for - THIS identity owns the tree and funds the deposit + m256i gammingNonce; // first gamming byte selects MESSAGE_TYPE_ANT_SOLUTION (3) +} +// then the payload: +AntSolutionBroadcastPayload { // 48 bytes + unsigned int parentTick; // ABSOLUTE tick of the parent node (0 with the index below = root) + unsigned int parentSolutionIndexInTick; + unsigned int anchorTick; // ABSOLUTE tick number + unsigned int claimedScore; + m256i nonce; // the 32-byte nonce from 2.2 +} +``` + +The computor scores and validates on receipt (a non-canonical or already-seen solution is dropped for +free). Nothing is on-chain yet. + +**Stage 2 - on-chain transaction (`AntColonyMiningSolutionTransaction`, `inputType` 12).** When the +computor publishes, it emits a standard transaction into tick data under **its own** key and funds the +deposit from **its own** balance - the computor pays, not the miner. This transaction is the consensus +record: every node processes it in `processTick`, recomputes the score, folds it into +`resourceTestingDigest`, commits the node to the computor's tree, and refunds or keeps the deposit. + +``` +AntColonyMiningSolutionTransaction : Transaction { // 80-byte header + 48-byte payload + 64-byte signature + // --- Transaction header --- + m256i sourcePublicKey; // the COMPUTOR (tree owner); signs the tx and funds the deposit + m256i destinationPublicKey; // zero (NULL_ID) + long long amount; // SOLUTION_SECURITY_DEPOSIT = 1,000,000 QU + unsigned int tick; // publish tick + unsigned short inputType; // ANT_COLONY_MINING_SOLUTION_INPUT_TYPE = 12 + unsigned short inputSize; // 48 + // --- payload (48 bytes) --- + unsigned int parentTick; // ABSOLUTE tick of the parent node + unsigned int parentSolutionIndexInTick; + unsigned int anchorTick; // ABSOLUTE + unsigned int claimedScore; + m256i nonce; + // --- 64-byte signature over header + payload --- +} +``` + +`parentRef = (parentTick, parentSolutionIndexInTick) = (0, 0xFFFFFFFF)` means the **virtual root** +(a depth-1 child). + +**Every tick in the protocol is absolute.** `parentTick`, `selfTick` and `anchorTick` are all real +system tick numbers - the same values `getCurrentTick` or a tick-data query returns. A parent is named +by the absolute tick it was committed in (plus its index within that tick), so the ref you copy from +the identity tree is used verbatim - there is no epoch-relative offset to convert. + +**For a pool.** The tree belongs to the **computor** - `destinationPublicKey` of the broadcast, which +becomes `sourcePublicKey` of the transaction. Workers hold no tree and post no on-chain deposit; they +hand solutions to the pool's computor, which pre-validates them and risks its own deposit only on +solutions it expects to be accepted and refunded. Fund the computor identity, not the workers. + +### 2.7 Read queries + +Three request/response pairs. **Identity tree** and **parent ANN** are **operator-signed**; **epoch +context** is **public**. + +Operator signing: the request payload is followed by a **64-byte signature** over `K12(payload)`, +verified against the node's configured **operator public key**. This lets a pool route and filter its +own miners' reads while keeping untrusted parties from spamming the node. There is no monotonic nonce - +replaying a read only costs a duplicate answer. + +``` +digest = K12(requestPayload) +signature = sign(operatorSubseed, operatorPublicKey, digest) // 64 bytes, appended to the payload +``` + +**(a) Epoch context** - `REQUEST_ANT_EPOCH_CONTEXT` (76) / `RESPOND_ANT_EPOCH_CONTEXT` (77). **Public.** + +Request: empty. Response `RespondAntEpochContext` (120 bytes, packed): + +``` +m256i spectrumDigest; // epoch-start spectrum digest (seeds every root) +m256i topologyHash; // canonical task topology-block hash (BPP9000_TOPOLOGY_HASH) +m256i dataHash; // canonical task data-block hash (BPP9000_DATA_HASH) +unsigned int threshold; // per-epoch accept bound +unsigned int freshnessWindow; // publish within this many ticks of the anchor +unsigned int solutionCount; // accepted solutions so far this epoch +unsigned int freeAnnSlotsCount; // free slots in the live ANN pool +unsigned int maxChildrenPerParent; // per-parent child cap; 0 = unbounded +unsigned short epoch; +unsigned short padding; +``` + +`topologyHash` / `dataHash` identify the exact task the node scores against. After loading your task +file, recompute K12 over your own topology and data blocks and compare against these. If either +differs you are holding the wrong task: **stop**. Mining a stale task wastes work, and the mismatched +score forfeits the computor's deposit on every submission (the node scores with *its* task, so your +claimed score never matches). + +**(b) Identity tree** - `REQUEST_ANT_IDENTITY_TREE` (72) / `RESPOND_ANT_IDENTITY_TREE` (73). +**Operator-signed.** Paginated. + +Request `RequestAntIdentityTree` (40 bytes) + 64-byte signature: + +``` +m256i pubkey; // whose tree to report (usually your own) +unsigned int fromIndex; // resume cursor, 0 on the first call +unsigned int padding; +``` + +Response: `RespondAntIdentityTreeHeader` (12 bytes: `count`, `itemSize`, `nextIndex`) followed by +`count` x `AntIdentityTreeNode`. Up to 64 nodes per response; page with `nextIndex` until it is `0`. + +``` +AntIdentityTreeNode { // 32 bytes + unsigned int selfTick; // ABSOLUTE; set these two as your parentRef to extend THIS node + unsigned int selfSolutionIndexInTick; + unsigned int parentTick; // this node's own parent (ABSOLUTE); (0, 0xFFFFFFFF) = root + unsigned int parentSolutionIndexInTick; + unsigned int score; // error count; a child must score strictly below this + unsigned int childCount; // children already attached (compare vs maxChildrenPerParent) + unsigned int anchorTick; + unsigned int depth; +} +``` + +Paging every node of a pubkey reconstructs the whole tree, edges included, with no further fetches. + +**(c) Parent ANN** - `REQUEST_ANT_PARENT_ANN` (74) / `RESPOND_ANT_PARENT_ANN` (75). **Operator-signed.** + +Request `RequestAntParentAnn` (8 bytes) + 64-byte signature: + +``` +unsigned int parentRefTick; +unsigned int parentRefSolutionIndexInTick; +``` + +Response `RespondAntParentAnnHeader` (16 bytes), then `annSizeBytes` of **canonical ANN** (one trit per +byte, the exact form the scorer consumes - no unpacking needed): + +``` +unsigned int parentRefTick; +unsigned int parentRefSolutionIndexInTick; +unsigned int annSizeBytes; // ANN LUT size when status is OK, else 0 +unsigned char status; // 0 = OK, 1 = NOT_FOUND, 2 = IS_ROOT (derive your own root instead) +unsigned char padding[3]; +``` + +**Canonical ANN layout.** The same byte form is used everywhere ANN bytes leave the node: this response, the snapshot pool, and the epoch export. Under the current bpp9000 parameters it is 1'728 bytes = 64 rows of 27: + +``` +row k, k = 0..45 LUT of neuron updatedNeuronIndices[k]: the k-th NON-INPUT neuron in + ascending absolute index (input neurons have no LUT; which indices are + inputs comes from the task topology) +row k, k = 46..63 zero (the row count is fixed at the population size, the live count is + population minus inputs and so task-dependent) + +byte[line] of a row, line = t0 + 3*t1 + 9*t2 + the neuron's next trit for that neighbor state; t0, t1, t2 are the + current trits {0,1,2} of its three wired neighbors in task wiring + order (2 = UNKNOWN is an ordinary value) +``` + +Rows are ordered **by updated-neuron position, not by absolute neuron index**. A miner or tool that keeps LUTs indexed by absolute neuron number must convert through the task's `updatedNeuronIndices` mapping before comparing or reusing these bytes. + +### 2.8 Network message + transaction types + +| Type | Value | Signed | Direction | +|------|-------|--------|-----------| +| `BROADCAST_MESSAGE` + `MESSAGE_TYPE_ANT_SOLUTION` | `3` | envelope-signed | miner -> computor (submit) | +| `AntColonyMiningSolutionTransaction` (`inputType`) | `12` | computor-signed | computor -> tick data (consensus) | +| `REQUEST_ANT_IDENTITY_TREE` / `RESPOND_ANT_IDENTITY_TREE` | `72` / `73` | operator | read tree | +| `REQUEST_ANT_PARENT_ANN` / `RESPOND_ANT_PARENT_ANN` | `74` / `75` | operator | read one node's ANN | +| `REQUEST_ANT_EPOCH_CONTEXT` / `RESPOND_ANT_EPOCH_CONTEXT` | `76` / `77` | public | read epoch params | + +## Part 3 - Node files + +The ant colony adds the following files on the node's disk. File names are defined in +`public_settings.h`; `{epoch}` is the epoch number. + +| File | Content | Handling | +|------|---------|----------| +| `snapshotAntColonyHeader.{epoch}` | colony meta + anchor ring + export set | snapshot set | +| `snapshotAntColonyRecords.{epoch}` | accepted solution records | snapshot set | +| `snapshotAntColonyPool.{epoch}` | packed ANN pool | snapshot set | +| `snapshotAntSolutionFlag` | ant solution flags bitmap | snapshot set | +| `antColonyReplayCache.{epoch}` | score cache | optional, recommended | +| `antColonySolutions.eoe` | end-of-epoch export: best 676 networks (pubkey, score, depth, LUT) | output only | +| `bpp9000.task` | the epoch's pinned task | required at boot | + +**Snapshot set**: the four snapshot files are part of the node snapshot. When you back up, copy, or restore node state, take them TOGETHER with the other snapshot files (spectrum, universe, contracts, system) from the same point. On load the node cross-checks the colony snapshot's rootSeed, error threshold, and initial tick against the restored node state, a colony snapshot from a different moment is refused and the node will not start from it. + +**Replay cache**: a score memo, not consensus state, same behavior with standalone score. Without it a restart recomputes every stored solution, so keep it for fast restarts; losing it only costs time. + +**Export**: written at the epoch transition for offline analysis. It is not read back by the node and is not needed for a restart. diff --git a/doc/protocol.md b/doc/protocol.md index 70fcc0c8..85121758 100644 --- a/doc/protocol.md +++ b/doc/protocol.md @@ -29,6 +29,7 @@ The following transaction types (`tx->inputType`) are defined: - `ExecutionFeeReportTransactionPrefix`, type 9, defined in `src/network_messages/execution_fees.h`. - `OracleUserQueryTransactionPrefix`, type 10, defined in `src/oracle_core/oracle_transactions.h`. - `DogeMiningShareTransaction`, type 11, defined in `src/mining/mining.h`. +- `AntColonyMiningSolutionTransaction`, type 12, defined in `src/mining/mining.h`. ## Peer Sharing diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index bd31a318..eb4cb27c 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -207,6 +207,11 @@ if(NO_RPC) target_compile_definitions(Qubic PRIVATE NO_RPC) endif() +# Peer port override, so a second node can run where the default is taken. +if(DEFINED PORT) + target_compile_definitions(Qubic PRIVATE PORT=${PORT}) +endif() + if(NO_ENABLE_QUBIC_LOGGING_EVENT) target_compile_definitions(Qubic PRIVATE NO_ENABLE_QUBIC_LOGGING_EVENT) endif() diff --git a/src/Qubic.vcxproj b/src/Qubic.vcxproj index e10600ba..0594fe83 100644 --- a/src/Qubic.vcxproj +++ b/src/Qubic.vcxproj @@ -44,6 +44,7 @@ + @@ -69,6 +70,9 @@ + + + @@ -76,9 +80,11 @@ + + diff --git a/src/Qubic.vcxproj.filters b/src/Qubic.vcxproj.filters index 10b72223..930f9d4a 100644 --- a/src/Qubic.vcxproj.filters +++ b/src/Qubic.vcxproj.filters @@ -50,6 +50,9 @@ network_messages + + network_messages + network_messages @@ -123,6 +126,9 @@ contracts + + contracts + contracts @@ -185,6 +191,18 @@ mining + + mining\ant_colony + + + mining\ant_colony + + + mining\ant_colony + + + mining + logging @@ -459,6 +477,9 @@ {df525479-7504-470c-a25a-de4af8be0e5d} + + {7b1f3a52-9c4e-4d2a-b8e6-2a5c9d417f60} + {d334594b-f24d-440e-949a-c791aa13f867} diff --git a/src/contract_core/contract_def.h b/src/contract_core/contract_def.h index 074a6dd6..7d3f7e53 100644 --- a/src/contract_core/contract_def.h +++ b/src/contract_core/contract_def.h @@ -209,7 +209,11 @@ #define CONTRACT_INDEX QRAFFLE_CONTRACT_INDEX #define CONTRACT_STATE_TYPE QRAFFLE #define CONTRACT_STATE2_TYPE QRAFFLE2 +#ifdef OLD_QRAFFLE +#include "contracts/QRaffle_old.h" +#else #include "contracts/QRaffle.h" +#endif #undef CONTRACT_INDEX #undef CONTRACT_STATE_TYPE diff --git a/src/contracts/QRaffle.h b/src/contracts/QRaffle.h index 14c138f7..839ad112 100644 --- a/src/contracts/QRaffle.h +++ b/src/contracts/QRaffle.h @@ -13,10 +13,20 @@ constexpr uint32 QRAFFLE_REGISTER_FEE = 5; // percent constexpr uint32 QRAFFLE_FEE = 1; // percent constexpr uint32 QRAFFLE_CHARITY_FEE = 1; // percent constexpr uint32 QRAFFLE_SHAREHOLDER_FEE = 8; // percent +// Retained in the contract's own balance (never transferred out) to fund RANDOM +// entropy purchases in END_EPOCH; builds a self-sustaining reserve over time. +constexpr uint32 QRAFFLE_ENTROPY_FEE = 1; // percent + +// RANDOM smart contract entropy purchase: mixed into the winner-selection seed +// once per END_EPOCH. Mirrors RL_RANDOM_* in RandomLottery.h. +constexpr uint16 QRAFFLE_RANDOM_ENTROPY_BITS = 256; +constexpr uint8 QRAFFLE_RANDOM_COLLATERAL_TIER = 0; +constexpr uint64 QRAFFLE_RANDOM_ENTROPY_FEE = RANDOM_BITFEE * QRAFFLE_RANDOM_ENTROPY_BITS; + constexpr uint32 QRAFFLE_MAX_EPOCH = 65536; constexpr uint32 QRAFFLE_MAX_PROPOSAL_EPOCH = 128; constexpr uint32 QRAFFLE_MAX_MEMBER = 65536; -constexpr uint32 QRAFFLE_DEFAULT_QRAFFLE_AMOUNT = 10000000ull; +constexpr uint32 QRAFFLE_DEFAULT_QRAFFLE_AMOUNT = 1000000ull; constexpr uint32 QRAFFLE_MIN_QRAFFLE_AMOUNT = 1000000ull; constexpr uint32 QRAFFLE_MAX_QRAFFLE_AMOUNT = 1000000000ull; // Ended token-raffle ring: 16 384 slots × ~96 B ≈ 1.5 MB. @@ -1409,16 +1419,7 @@ struct QRAFFLE : public ContractBase LOG_INFO(locals.log); return; } - // QRAFFLE and QXMR are reserved for dividends/registration; disallow in bundles. - if ((locals.item.asset.assetName == QRAFFLE_ASSET_NAME && locals.item.asset.issuer == NULL_ID) - || (locals.item.asset.assetName == QRAFFLE_QXMR_ASSET_NAME && locals.item.asset.issuer == state.get().QXMRIssuer)) - { - qpi.transfer(qpi.invocator(), qpi.invocationReward()); - output.returnCode = QRAFFLE_INVALID_BUNDLE; - locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_invalidBundle, 0 }; - LOG_INFO(locals.log); - return; - } + // Duplicate-asset check within the bundle; O(N²) acceptable for N ≤ 4. locals.dupFound = 0; for (locals.j = 0; locals.j < locals.i; locals.j++) @@ -2218,6 +2219,16 @@ struct QRAFFLE : public ContractBase sint64 transferResult; uint64 sumOfEntryAmountSubmitted, r, winnerRevenue, burnAmount, charityRevenue, shareholderRevenue, registerRevenue, fee, oneShareholderRev; uint64 tokenPool; + // RANDOM entropy purchase (winner-selection seed strengthening). + Entity entity; + RANDOM::BuyEntropy_input buyEntropyInput; + RANDOM::BuyEntropy_output buyEntropyOutput; + uint64 entropyReserveAmount; // per-block carve-out, recomputed fresh in each of the 3 fee blocks + struct EntropyMixData + { + id baseSeed; + bit_4096 entropy; + } entropyMixData; uint64 shareholderPerShareUnit; uint64 registerPerShareUnit; uint64 actualShareholderTotal; @@ -2281,6 +2292,28 @@ struct QRAFFLE : public ContractBase locals.digest.u64._2 ^ locals.computerDigest.u64._2, locals.digest.u64._3 ^ locals.computerDigest.u64._3)); + // Strengthen the seed with entropy from the RANDOM smart contract's independent + // commit-reveal miner pool, on top of the digest-based value above. This is + // additive, not a replacement: if the purchase fails (insufficient reserve, or + // RANDOM's pool is empty this cycle) baseSeed simply stays digest-only and + // raffle settlement proceeds unchanged -- member funds are already committed, + // so degrading gracefully is safer than blocking settlement. + qpi.getEntity(SELF, locals.entity); + if (locals.entity.incomingAmount - locals.entity.outgoingAmount >= (sint64)QRAFFLE_RANDOM_ENTROPY_FEE) + { + locals.buyEntropyInput.collateralTier = QRAFFLE_RANDOM_COLLATERAL_TIER; + locals.buyEntropyInput.numberOfBits = QRAFFLE_RANDOM_ENTROPY_BITS; + locals.buyEntropyInput.trustee = id::zero(); + INVOKE_OTHER_CONTRACT_PROCEDURE(RANDOM, BuyEntropy, locals.buyEntropyInput, locals.buyEntropyOutput, QRAFFLE_RANDOM_ENTROPY_FEE); + + if (interContractCallError == NoCallError && !(locals.buyEntropyOutput.entropy == BIT4096_ZERO)) + { + locals.entropyMixData.baseSeed = locals.baseSeed; + locals.entropyMixData.entropy = locals.buyEntropyOutput.entropy; + locals.baseSeed = qpi.K12(locals.entropyMixData); + } + } + // Asset descriptor reused by the QXMR distribution and per-token-raffle shareholder // payout loops below; both iterate QRAFFLE_ASSET possessors directly via locals.iter. locals.QraffleAsset.assetName = QRAFFLE_ASSET_NAME; @@ -2301,12 +2334,15 @@ struct QRAFFLE : public ContractBase locals.shareholderRevenue = div(locals.tokenPool * QRAFFLE_SHAREHOLDER_FEE, 100); locals.registerRevenue = div(locals.tokenPool * QRAFFLE_REGISTER_FEE, 100); locals.fee = div(locals.tokenPool * QRAFFLE_FEE, 100); + // Entropy reserve: retained in the contract's own balance (never transferred), + // funding future RANDOM entropy purchases in END_EPOCH. + locals.entropyReserveAmount = div(locals.tokenPool * QRAFFLE_ENTROPY_FEE, 100); // Round down per-share amounts; winner gets the remainder. locals.shareholderPerShareUnit = div(locals.shareholderRevenue, NUMBER_OF_COMPUTORS); locals.actualShareholderTotal = locals.shareholderPerShareUnit * NUMBER_OF_COMPUTORS; locals.registerPerShareUnit = div(locals.registerRevenue, state.get().numberOfRegisters); locals.actualRegisterTotal = locals.registerPerShareUnit * state.get().numberOfRegisters; - locals.winnerRevenue = locals.tokenPool - locals.burnAmount - locals.charityRevenue - locals.actualShareholderTotal - locals.actualRegisterTotal - locals.fee; + locals.winnerRevenue = locals.tokenPool - locals.burnAmount - locals.charityRevenue - locals.actualShareholderTotal - locals.actualRegisterTotal - locals.fee - locals.entropyReserveAmount; locals.revenueLog = RevenueLogger{ QRAFFLE_CONTRACT_INDEX, @@ -2566,7 +2602,7 @@ struct QRAFFLE : public ContractBase } } - // Qu pool distribution: 80% to creator, 20% to fee buckets. + // Qu pool distribution: 79% to creator, 21% to fee buckets (incl. entropy reserve). // Always executed when reserve is met, regardless of per-item delivery outcome. // (Assets already delivered to winner; we cannot recall them from a user's wallet.) locals.arBurn = div(locals.arGross * (uint64)QRAFFLE_BURN_FEE, 100ull); @@ -2574,6 +2610,9 @@ struct QRAFFLE : public ContractBase locals.arShareholderRev = div(locals.arGross * (uint64)QRAFFLE_SHAREHOLDER_FEE, 100ull); locals.arRegisterRev = div(locals.arGross * (uint64)QRAFFLE_REGISTER_FEE, 100ull); locals.arFee = div(locals.arGross * (uint64)QRAFFLE_FEE, 100ull); + // Entropy reserve: retained in the contract's own balance (never transferred), + // funding future RANDOM entropy purchases in END_EPOCH. + locals.entropyReserveAmount = div(locals.arGross * (uint64)QRAFFLE_ENTROPY_FEE, 100ull); // Round down per-share amounts; all rounding dust goes to creator. locals.arShareholderPerShare = div(locals.arShareholderRev, (uint64)NUMBER_OF_COMPUTORS); locals.arRegisterPerShare = (state.get().numberOfRegisters > 0) @@ -2586,7 +2625,8 @@ struct QRAFFLE : public ContractBase - locals.arCharity - (locals.arShareholderPerShare * (uint64)NUMBER_OF_COMPUTORS) - locals.arRegisterPerShareActual - - locals.arFee; + - locals.arFee + - locals.entropyReserveAmount; qpi.transfer(locals.arInfo.creator, locals.arCreatorPay); qpi.burn(locals.arBurn); diff --git a/src/contracts/QRaffle_old.h b/src/contracts/QRaffle_old.h new file mode 100644 index 00000000..453612b3 --- /dev/null +++ b/src/contracts/QRaffle_old.h @@ -0,0 +1,2836 @@ +using namespace QPI; + +constexpr uint64 QRAFFLE_REGISTER_AMOUNT = 1000000000ull; +constexpr uint64 QRAFFLE_QXMR_REGISTER_AMOUNT = 250000000ull; +constexpr uint64 QRAFFLE_MAX_QRE_AMOUNT = 1000000000ull; +constexpr uint64 QRAFFLE_ASSET_NAME = 19505638103142993; +constexpr uint64 QRAFFLE_QXMR_ASSET_NAME = 1380800593; // QXMR token asset name +constexpr uint32 QRAFFLE_LOGOUT_FEE = div(QRAFFLE_REGISTER_AMOUNT, 20); +constexpr uint32 QRAFFLE_QXMR_LOGOUT_FEE = div(QRAFFLE_QXMR_REGISTER_AMOUNT, 20); // QXMR logout fee +constexpr uint32 QRAFFLE_TRANSFER_SHARE_FEE = 100; +constexpr uint32 QRAFFLE_BURN_FEE = 5; // percent +constexpr uint32 QRAFFLE_REGISTER_FEE = 5; // percent +constexpr uint32 QRAFFLE_FEE = 1; // percent +constexpr uint32 QRAFFLE_CHARITY_FEE = 1; // percent +constexpr uint32 QRAFFLE_SHAREHOLDER_FEE = 8; // percent +constexpr uint32 QRAFFLE_MAX_EPOCH = 65536; +constexpr uint32 QRAFFLE_MAX_PROPOSAL_EPOCH = 128; +constexpr uint32 QRAFFLE_MAX_MEMBER = 65536; +constexpr uint32 QRAFFLE_DEFAULT_QRAFFLE_AMOUNT = 10000000ull; +constexpr uint32 QRAFFLE_MIN_QRAFFLE_AMOUNT = 1000000ull; +constexpr uint32 QRAFFLE_MAX_QRAFFLE_AMOUNT = 1000000000ull; +// Ended token-raffle ring: 16 384 slots × ~96 B ≈ 1.5 MB. +// At most QRAFFLE_MAX_PROPOSAL_EPOCH (128) raffles/epoch → covers ~128 epochs of history. +constexpr uint32 QRAFFLE_MAX_TOKEN_RAFFLES = 16384; +constexpr uint32 QRAFFLE_TOKEN_RAFFLE_SLOT_SIZE = 512; // 2^9, max members per token raffle +constexpr uint8 QRAFFLE_MAX_PROPOSALS_PER_PROPOSER = 3; // max proposals per user per epoch + +constexpr sint32 QRAFFLE_SUCCESS = 0; +constexpr sint32 QRAFFLE_INSUFFICIENT_FUND = 1; +constexpr sint32 QRAFFLE_ALREADY_REGISTERED = 2; +constexpr sint32 QRAFFLE_UNREGISTERED = 3; +constexpr sint32 QRAFFLE_MAX_PROPOSAL_EPOCH_REACHED = 4; +constexpr sint32 QRAFFLE_INVALID_PROPOSAL = 5; +constexpr sint32 QRAFFLE_FAILED_TO_DEPOSIT = 6; +constexpr sint32 QRAFFLE_ALREADY_VOTED = 7; +constexpr sint32 QRAFFLE_INVALID_TOKEN_RAFFLE = 8; +constexpr sint32 QRAFFLE_INVALID_OFFSET_OR_LIMIT = 9; +constexpr sint32 QRAFFLE_INVALID_EPOCH = 10; +constexpr sint32 QRAFFLE_MAX_MEMBER_REACHED = 11; +constexpr sint32 QRAFFLE_INITIAL_REGISTER_CANNOT_LOGOUT = 12; +constexpr sint32 QRAFFLE_INSUFFICIENT_QXMR = 13; +constexpr sint32 QRAFFLE_INVALID_TOKEN_TYPE = 14; +constexpr sint32 QRAFFLE_USER_NOT_FOUND = 15; +constexpr sint32 QRAFFLE_INVALID_ENTRY_AMOUNT = 16; +constexpr sint32 QRAFFLE_EMPTY_QU_RAFFLE = 17; +constexpr sint32 QRAFFLE_EMPTY_TOKEN_RAFFLE = 18; +constexpr sint32 QRAFFLE_MAX_PROPOSAL_PER_USER_REACHED = 19; + +// Asset Raffle return codes +constexpr sint32 QRAFFLE_INVALID_BUNDLE = 20; +constexpr sint32 QRAFFLE_INVALID_RESERVE_PRICE = 21; +constexpr sint32 QRAFFLE_BUNDLE_ESCROW_FAILED = 22; +constexpr sint32 QRAFFLE_INVALID_ASSET_RAFFLE = 23; +constexpr sint32 QRAFFLE_ASSET_RAFFLE_FULL = 24; +constexpr sint32 QRAFFLE_TICKET_LIMIT_REACHED = 25; +constexpr sint32 QRAFFLE_MAX_ASSET_RAFFLES_REACHED = 26; +constexpr sint32 QRAFFLE_CANCEL_NOT_ALLOWED = 27; + +// Asset Raffle configuration +constexpr uint64 QRAFFLE_ASSET_RAFFLE_PROPOSAL_FEE = 500000ull; // 500K Qu; non-refundable anti-spam +constexpr uint32 QRAFFLE_MAX_ASSET_RAFFLES_PER_EPOCH = 64; // concurrent active raffles +constexpr uint32 QRAFFLE_MAX_ASSETS_PER_BUNDLE = 4; // items per bundle +constexpr uint32 QRAFFLE_MAX_ASSET_TICKET_BUYERS = 1024; // distinct buyers per raffle +constexpr uint32 QRAFFLE_MAX_TICKETS_PER_BUYER = 100; // per-buyer cap (anti-griefing) +constexpr uint32 QRAFFLE_MAX_ASSET_RAFFLES_PER_CREATOR = 2; // per creator per epoch +constexpr uint32 QRAFFLE_MAX_ENDED_ASSET_RAFFLES = 8192; // history ring buffer +constexpr uint64 QRAFFLE_MIN_ASSET_TICKET_AMOUNT = 1000000ull; // 1M Qu +constexpr uint64 QRAFFLE_MAX_ASSET_TICKET_AMOUNT = 1000000000000ull;// 1T Qu +// Flat array strides: raffle i occupies [i*stride .. i*stride+count) +constexpr uint32 QRAFFLE_ASSET_RAFFLE_BUNDLE_FLAT_SIZE = QRAFFLE_MAX_ASSET_RAFFLES_PER_EPOCH * QRAFFLE_MAX_ASSETS_PER_BUNDLE; // 256 +constexpr uint32 QRAFFLE_ASSET_RAFFLE_BUYERS_FLAT_SIZE = QRAFFLE_MAX_ASSET_RAFFLES_PER_EPOCH * QRAFFLE_MAX_ASSET_TICKET_BUYERS; // 65536 + + +constexpr uint32 QRAFFLE_MAX_SHAREHOLDERS_OLD = 1024; + + +struct QRAFFLE2 +{ +}; + +struct QRAFFLE : public ContractBase +{ +public: + enum LogInfo { + QRAFFLE_success = 0, + QRAFFLE_insufficientQubic = 1, + QRAFFLE_insufficientQXMR = 2, + QRAFFLE_alreadyRegistered = 3, + QRAFFLE_unregistered = 4, + QRAFFLE_maxMemberReached = 5, + QRAFFLE_maxProposalEpochReached = 6, + QRAFFLE_invalidProposal = 7, + QRAFFLE_failedToDeposit = 8, + QRAFFLE_alreadyVoted = 9, + QRAFFLE_invalidTokenRaffle = 10, + QRAFFLE_invalidOffsetOrLimit = 11, + QRAFFLE_invalidEpoch = 12, + QRAFFLE_initialRegisterCannotLogout = 13, + QRAFFLE_invalidTokenType = 14, + QRAFFLE_invalidEntryAmount = 15, + QRAFFLE_maxMemberReachedForQuRaffle = 16, + QRAFFLE_proposalNotFound = 17, + QRAFFLE_proposalAlreadyEnded = 18, + QRAFFLE_notEnoughShares = 19, + QRAFFLE_transferFailed = 20, + QRAFFLE_epochEnded = 21, + QRAFFLE_winnerSelected = 22, + QRAFFLE_revenueDistributed = 23, + QRAFFLE_tokenRaffleCreated = 24, + QRAFFLE_tokenRaffleEnded = 25, + QRAFFLE_proposalSubmitted = 26, + QRAFFLE_proposalVoted = 27, + QRAFFLE_quRaffleDeposited = 28, + QRAFFLE_tokenRaffleDeposited = 29, + QRAFFLE_shareManagementRightsTransferred = 30, + QRAFFLE_emptyQuRaffle = 31, + QRAFFLE_emptyTokenRaffle = 32, + QRAFFLE_maxProposalPerUserReached = 33, + // Asset Raffle log types + QRAFFLE_assetRaffleCreated = 34, + QRAFFLE_assetRaffleTicketBought = 35, + QRAFFLE_assetRaffleSucceeded = 36, + QRAFFLE_assetRaffleRefunded = 37, + QRAFFLE_assetRaffleBundleEscrowFailed = 38, + QRAFFLE_assetRaffleBundleDeliveryFailed = 39, + QRAFFLE_assetRaffleCancelled = 40, + QRAFFLE_invalidBundle = 41, + QRAFFLE_invalidReservePrice = 42, + QRAFFLE_assetRaffleFull = 43, + QRAFFLE_ticketLimitReached = 44, + QRAFFLE_maxAssetRafflesReached = 45, + QRAFFLE_cancelNotAllowed = 46 + }; + + struct Logger + { + uint32 _contractIndex; + uint32 _type; // Assign a random unique (per contract) number to distinguish messages of different types + sint8 _terminator; // Only data before "_terminator" are logged + }; + + // Enhanced logger for END_EPOCH with detailed information + struct EndEpochLogger + { + uint32 _contractIndex; + uint32 _type; + uint32 _epoch; // Current epoch number + uint32 _memberCount; // Number of QuRaffle members + uint64 _totalAmount; // Total amount being processed + uint64 _winnerAmount; // Amount won by winner + uint32 _winnerIndex; // Index of the winner + sint8 _terminator; + }; + + // Enhanced logger for revenue distribution + struct RevenueLogger + { + uint32 _contractIndex; + uint32 _type; + uint64 _burnAmount; // Amount burned + uint64 _charityAmount; // Amount sent to charity + uint64 _shareholderAmount; // Amount distributed to shareholders + uint64 _registerAmount; // Amount distributed to registers + uint64 _feeAmount; // Amount sent to fee address + uint64 _winnerAmount; // Amount sent to winner + sint8 _terminator; + }; + + // Enhanced logger for token raffle processing + struct TokenRaffleLogger + { + uint32 _contractIndex; + uint32 _type; + uint32 _raffleIndex; // Index of the token raffle + uint64 _assetName; // Asset name of the token + uint32 _memberCount; // Number of members in this raffle + uint64 _entryAmount; // Entry amount for this raffle + uint32 _winnerIndex; // Winner index for this raffle + uint64 _winnerAmount; // Amount won in this raffle + sint8 _terminator; + }; + + // Enhanced logger for proposal processing + struct ProposalLogger + { + uint32 _contractIndex; + uint32 _type; + uint32 _proposalIndex; // Index of the proposal + id _proposer; // Proposer of the proposal + uint32 _yesVotes; // Number of yes votes + uint32 _noVotes; // Number of no votes + uint64 _assetName; // Asset name if approved + uint64 _entryAmount; // Entry amount if approved + sint8 _terminator; + }; + + struct EmptyTokenRaffleLogger + { + uint32 _contractIndex; + uint32 _type; + uint32 _tokenRaffleIndex; // Index of the token raffle per epoch + sint8 _terminator; + }; + + struct AssetRaffleCreatedLogger + { + uint32 _contractIndex; + uint32 _type; + uint32 _raffleIndex; + id _creator; + uint64 _reservePriceQu; + uint64 _entryTicketQu; + uint32 _bundleSize; + sint8 _terminator; + }; + + struct AssetRaffleTicketLogger + { + uint32 _contractIndex; + uint32 _type; + uint32 _raffleIndex; + id _buyer; + uint32 _tickets; + uint64 _cost; + sint8 _terminator; + }; + + struct AssetRaffleEndedLogger + { + uint32 _contractIndex; + uint32 _type; + uint32 _raffleIndex; + id _creator; + id _winner; + uint64 _grossPoolQu; + uint64 _creatorPaidQu; + uint8 _reserveMet; + sint8 _terminator; + }; + + // One item in an asset raffle bundle (token or SC share + quantity). + struct AssetRaffleItem + { + Asset asset; + sint64 numberOfShares; + }; + + // Active asset raffle state (live during the epoch it was created). + struct AssetRaffleInfo + { + id creator; + uint64 reservePriceQu; // net Qu creator wants AFTER 20% fee + uint64 entryTicketQu; // Qu per ticket + uint64 totalTicketsPaidQu; // gross Qu pool so far + uint32 numberOfBuyers; + uint32 totalTickets; + uint32 bundleSize; + uint32 epoch; + }; + + // Historical record written at END_EPOCH for each settled asset raffle. + // Field order keeps the largest types first so trailing padding is minimal and + // deterministic across compilers (no explicit pad needed → no plain C arrays). + struct EndedAssetRaffleInfo + { + id creator; + id epochWinner; // NULL_ID if reserve was missed + uint64 reservePriceQu; + uint64 entryTicketQu; + uint64 grossPoolQu; + uint64 creatorPaidQu; // 0 if reserve missed + uint32 totalTickets; + uint32 numberOfBuyers; + uint32 bundleSize; + uint32 epoch; + uint32 reserveMet; // 1 = reserve met and winner paid; 0 = refunded (uint32 keeps natural alignment) + }; + + struct ProposalInfo { + Asset token; + id proposer; + uint64 entryAmount; + uint32 nYes; + uint32 nNo; + }; + + struct QuRaffleInfo + { + id epochWinner; + uint64 receivedAmount; + uint64 entryAmount; + uint32 numberOfMembers; + uint32 winnerIndex; + }; + + struct TokenRaffleInfo + { + id epochWinner; + Asset token; + uint64 entryAmount; + uint32 numberOfMembers; + uint32 winnerIndex; + uint32 epoch; + }; + + struct ActiveTokenRaffleInfo { + Asset token; + uint64 entryAmount; + }; + + struct OldStateData + { + HashMap registers; + Array proposals; + + HashMap , QRAFFLE_MAX_MEMBER> voteParticipation; + HashMap , QRAFFLE_MAX_MEMBER> voteValues; + Array numberOfVotedInProposal; + Array quRaffleMembers; + HashSet quRaffleMemberSet; + + Array activeTokenRaffle; + HashMap , QRAFFLE_MAX_MEMBER> tokenRaffleParticipation; + Array tokenRaffleMemberSlots; + Array numberOfTokenRaffleMembers; + + Array QuRaffles; + Array tokenRaffle; + HashMap quRaffleEntryAmount; + HashSet shareholdersList; + + id initialRegister1, initialRegister2, initialRegister3, initialRegister4, initialRegister5; + id charityAddress, feeAddress, QXMRIssuer; + uint64 epochRevenue, epochQXMRRevenue, qREAmount, totalBurnAmount, totalCharityAmount, totalShareholderAmount, totalRegisterAmount, totalFeeAmount, totalWinnerAmount, largestWinnerAmount; + uint32 numberOfRegisters, numberOfQuRaffleMembers, numberOfEntryAmountSubmitted, numberOfProposals, numberOfActiveTokenRaffle, numberOfEndedTokenRaffle; + Array daoMemberCount; + HashMap proposalsPerProposer; + }; + + struct StateData + { + HashMap registers; + Array proposals; + + // Per-user vote tracking with dual BitArray (qRWA pattern). + // O(1) lookup via id hash, 1 bit per proposal. ~4 MB each, ~8 MB total. + HashMap , QRAFFLE_MAX_MEMBER> voteParticipation; // bit=1 if user has voted + HashMap , QRAFFLE_MAX_MEMBER> voteValues; // bit=1 for yes, bit=0 for no + Array numberOfVotedInProposal; + Array quRaffleMembers; + // O(1) duplicate guard for quRaffle entries; mirrors quRaffleMembers for membership tests. + HashSet quRaffleMemberSet; + + Array activeTokenRaffle; + // Per-user O(1) duplicate check for token raffle deposits. ~4 MB. + HashMap , QRAFFLE_MAX_MEMBER> tokenRaffleParticipation; + // Flat indexed member storage: raffle i occupies slots [i*SLOT_SIZE .. i*SLOT_SIZE+count). ~2 MB. + Array tokenRaffleMemberSlots; + Array numberOfTokenRaffleMembers; + + Array QuRaffles; + Array tokenRaffle; + HashMap quRaffleEntryAmount; + + id initialRegister1, initialRegister2, initialRegister3, initialRegister4, initialRegister5; + id charityAddress, feeAddress, QXMRIssuer; + uint64 epochRevenue, epochQXMRRevenue, qREAmount, totalBurnAmount, totalCharityAmount, totalShareholderAmount, totalRegisterAmount, totalFeeAmount, totalWinnerAmount, largestWinnerAmount; + uint32 numberOfRegisters, numberOfQuRaffleMembers, numberOfEntryAmountSubmitted, numberOfProposals, numberOfActiveTokenRaffle, numberOfEndedTokenRaffle; + Array daoMemberCount; // Number of DAO members (registers) at each epoch + HashMap proposalsPerProposer; + + // ── Asset Raffle state ──────────────────────────────────────────────────────── + Array activeAssetRaffles; + uint32 numberOfActiveAssetRaffles; + + // Bundle items: raffle i occupies [i*4 .. i*4+bundleSize). + Array activeAssetRaffleItems; + + // Buyer lists: raffle i occupies [i*1024 .. i*1024+numberOfBuyers). + Array activeAssetRaffleBuyers; + Array activeAssetRaffleBuyerTickets; + + // O(1) has-bought check: bit[i]=1 means this user has tickets in raffle i. + HashMap, QRAFFLE_MAX_MEMBER> assetRaffleParticipation; + + // O(1) slot lookup: entry[i] = buyer's 0-based position in raffle i's buyer region. + // Sentinel 0xFFFF = not present. ~8 MB (64 raffles × 2 B × 65536 buyers). + // Reset each epoch alongside the buyer arrays. + HashMap, QRAFFLE_MAX_MEMBER> assetRaffleBuyerSlotIndex; + + // Per-creator raffle count; reset each epoch to enforce QRAFFLE_MAX_ASSET_RAFFLES_PER_CREATOR. + HashMap assetRafflesPerCreator; + + // Settled raffle history ring buffer. + Array endedAssetRaffles; + uint32 numberOfEndedAssetRaffles; + + // Accumulated proposal fees destined for DAO registers (50% of each 500K fee); + // distributed in one O(R) pass at END_EPOCH alongside the register share bucket. + uint64 epochAssetRaffleDaoBucket; + + // Aggregate analytics (monotonically increasing). + uint64 totalAssetRaffleProposalFees; + uint64 totalAssetRaffleCreatorPaid; + uint64 totalAssetRaffleRefunded; + uint32 totalAssetRafflesCreated; + uint32 totalAssetRafflesSucceeded; + uint32 totalAssetRafflesFailed; + }; + + struct registerInSystem_input + { + bit useQXMR; // 0 = use qubic, 1 = use QXMR tokens + }; + + struct registerInSystem_output + { + sint32 returnCode; + }; + + struct logoutInSystem_input + { + }; + + struct logoutInSystem_output + { + sint32 returnCode; + }; + + struct submitEntryAmount_input + { + uint64 amount; + }; + + struct submitEntryAmount_output + { + sint32 returnCode; + }; + + struct submitProposal_input + { + id tokenIssuer; + uint64 tokenName; + uint64 entryAmount; + }; + + struct submitProposal_output + { + sint32 returnCode; + }; + + struct voteInProposal_input + { + uint32 indexOfProposal; + bit yes; + }; + + struct voteInProposal_output + { + sint32 returnCode; + }; + + struct depositInQuRaffle_input + { + + }; + + struct depositInQuRaffle_output + { + sint32 returnCode; + }; + + struct depositInTokenRaffle_input + { + uint32 indexOfTokenRaffle; + }; + + struct depositInTokenRaffle_output + { + sint32 returnCode; + }; + + struct TransferShareManagementRights_input + { + id tokenIssuer; + uint64 tokenName; + sint64 numberOfShares; + uint32 newManagingContractIndex; + }; + + struct TransferShareManagementRights_output + { + sint64 transferredNumberOfShares; + }; + + struct getRegisters_input + { + uint32 offset; + uint32 limit; + }; + + struct getRegisters_output + { + id register1, register2, register3, register4, register5, register6, register7, register8, register9, register10, register11, register12, register13, register14, register15, register16, register17, register18, register19, register20; + sint32 returnCode; + }; + + struct getAnalytics_input + { + }; + + struct getAnalytics_output + { + uint64 currentQuRaffleAmount; + uint64 totalBurnAmount; + uint64 totalCharityAmount; + uint64 totalShareholderAmount; + uint64 totalRegisterAmount; + uint64 totalFeeAmount; + uint64 totalWinnerAmount; + uint64 largestWinnerAmount; + uint32 numberOfRegisters; + uint32 numberOfProposals; + uint32 numberOfQuRaffleMembers; + uint32 numberOfActiveTokenRaffle; + uint32 numberOfEndedTokenRaffle; + uint32 numberOfEntryAmountSubmitted; + sint32 returnCode; + }; + + struct getActiveProposal_input + { + uint32 indexOfProposal; + }; + + struct getActiveProposal_output + { + id tokenIssuer; + id proposer; + uint64 tokenName; + uint64 entryAmount; + uint32 nYes; + uint32 nNo; + sint32 returnCode; + }; + + struct getEndedTokenRaffle_input + { + uint32 indexOfRaffle; + }; + + struct getEndedTokenRaffle_output + { + id epochWinner; + id tokenIssuer; + uint64 tokenName; + uint64 entryAmount; + uint32 numberOfMembers; + uint32 winnerIndex; + uint32 epoch; + sint32 returnCode; + }; + + struct getEpochRaffleIndexes_input + { + uint32 epoch; + }; + + struct getEpochRaffleIndexes_output + { + uint32 StartIndex; + uint32 EndIndex; + sint32 returnCode; + }; + + struct getEndedQuRaffle_input + { + uint32 epoch; + }; + + struct getEndedQuRaffle_output + { + id epochWinner; + uint64 receivedAmount; + uint64 entryAmount; + uint32 numberOfMembers; + uint32 winnerIndex; + uint32 numberOfDaoMembers; + sint32 returnCode; + }; + + struct getActiveTokenRaffle_input + { + uint32 indexOfTokenRaffle; + }; + + struct getActiveTokenRaffle_output + { + id tokenIssuer; + uint64 tokenName; + uint64 entryAmount; + uint32 numberOfMembers; + sint32 returnCode; + }; + + struct getQuRaffleEntryAmountPerUser_input + { + id user; + }; + + struct getQuRaffleEntryAmountPerUser_output + { + uint64 entryAmount; + sint32 returnCode; + }; + + struct getQuRaffleEntryAverageAmount_input + { + }; + + struct getQuRaffleEntryAverageAmount_output + { + uint64 entryAverageAmount; + sint32 returnCode; + }; + + // ── Asset Raffle I/O structs ─────────────────────────────────────────────── + + struct createAssetRaffle_input + { + // Bundle: fixed-capacity QPI Array; only [0..bundleSize) are used. + Array bundleItems; + uint32 bundleSize; + uint64 reservePriceQu; // min Qu creator wants AFTER 20% service fee + uint64 entryTicketQu; // Qu per ticket + }; + + struct createAssetRaffle_output + { + uint32 raffleIndex; + sint32 returnCode; + }; + + struct buyAssetRaffleTicket_input + { + uint32 indexOfAssetRaffle; + uint32 numberOfTickets; + }; + + struct buyAssetRaffleTicket_output + { + uint32 ticketsBought; + sint32 returnCode; + }; + + struct cancelAssetRaffle_input + { + uint32 indexOfAssetRaffle; + }; + + struct cancelAssetRaffle_output + { + sint32 returnCode; + }; + + struct getActiveAssetRaffle_input + { + uint32 indexOfAssetRaffle; + }; + + struct getActiveAssetRaffle_output + { + id creator; + uint64 reservePriceQu; + uint64 entryTicketQu; + uint64 totalTicketsPaidQu; + uint32 numberOfBuyers; + uint32 totalTickets; + uint32 bundleSize; + uint32 epoch; + sint32 returnCode; + }; + + struct getActiveAssetRaffleBundleItem_input + { + uint32 indexOfAssetRaffle; + uint32 itemIndex; + }; + + struct getActiveAssetRaffleBundleItem_output + { + id assetIssuer; + uint64 assetName; + sint64 numberOfShares; + sint32 returnCode; + }; + + struct getActiveAssetRaffleBuyer_input + { + uint32 indexOfAssetRaffle; + uint32 buyerIndex; + }; + + struct getActiveAssetRaffleBuyer_output + { + id buyer; + uint32 ticketCount; + sint32 returnCode; + }; + + struct getEndedAssetRaffle_input + { + uint32 indexOfRaffle; + }; + + struct getEndedAssetRaffle_output + { + id creator; + id epochWinner; + uint64 reservePriceQu; + uint64 entryTicketQu; + uint64 grossPoolQu; + uint64 creatorPaidQu; + uint32 totalTickets; + uint32 numberOfBuyers; + uint32 bundleSize; + uint32 epoch; + uint8 reserveMet; + sint32 returnCode; + }; + + struct getAssetRaffleAnalytics_input + { + }; + + struct getAssetRaffleAnalytics_output + { + uint64 totalAssetRaffleProposalFees; + uint64 totalAssetRaffleCreatorPaid; + uint64 totalAssetRaffleRefunded; + uint32 numberOfActiveAssetRaffles; + uint32 numberOfEndedAssetRaffles; + uint32 totalAssetRafflesCreated; + uint32 totalAssetRafflesSucceeded; + uint32 totalAssetRafflesFailed; + sint32 returnCode; + }; + +protected: + + + struct registerInSystem_locals + { + Logger log; + }; + + PUBLIC_PROCEDURE_WITH_LOCALS(registerInSystem) + { + if (state.get().registers.contains(qpi.invocator())) + { + if (qpi.invocationReward() > 0) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + } + output.returnCode = QRAFFLE_ALREADY_REGISTERED; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_alreadyRegistered, 0 }; + LOG_INFO(locals.log); + return ; + } + if (state.get().numberOfRegisters >= QRAFFLE_MAX_MEMBER) + { + if (qpi.invocationReward() > 0) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + } + output.returnCode = QRAFFLE_MAX_MEMBER_REACHED; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_maxMemberReached, 0 }; + LOG_INFO(locals.log); + return ; + } + + if (input.useQXMR) + { + // refund the invocation reward if the user uses QXMR for registration + if (qpi.invocationReward() > 0) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + } + // Use QXMR tokens for registration + if (qpi.numberOfPossessedShares(QRAFFLE_QXMR_ASSET_NAME, state.get().QXMRIssuer, qpi.invocator(), qpi.invocator(), SELF_INDEX, SELF_INDEX) < QRAFFLE_QXMR_REGISTER_AMOUNT) + { + output.returnCode = QRAFFLE_INSUFFICIENT_QXMR; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_insufficientQXMR, 0 }; + LOG_INFO(locals.log); + return ; + } + + // Transfer QXMR tokens to the contract + if (qpi.transferShareOwnershipAndPossession(QRAFFLE_QXMR_ASSET_NAME, state.get().QXMRIssuer, qpi.invocator(), qpi.invocator(), QRAFFLE_QXMR_REGISTER_AMOUNT, SELF) < 0) + { + output.returnCode = QRAFFLE_INSUFFICIENT_QXMR; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_insufficientQXMR, 0 }; + LOG_INFO(locals.log); + return ; + } + state.mut().registers.set(qpi.invocator(), 2); + } + else + { + // Use qubic for registration + if (qpi.invocationReward() < QRAFFLE_REGISTER_AMOUNT) + { + if (qpi.invocationReward() > 0) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + } + output.returnCode = QRAFFLE_INSUFFICIENT_FUND; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_insufficientQubic, 0 }; + LOG_INFO(locals.log); + return ; + } + qpi.transfer(qpi.invocator(), qpi.invocationReward() - QRAFFLE_REGISTER_AMOUNT); + state.mut().registers.set(qpi.invocator(), 1); + } + + state.mut().numberOfRegisters++; + output.returnCode = QRAFFLE_SUCCESS; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_success, 0 }; + LOG_INFO(locals.log); + } + + struct logoutInSystem_locals + { + sint64 refundAmount; + uint8 tokenType; + Logger log; + }; + + PUBLIC_PROCEDURE_WITH_LOCALS(logoutInSystem) + { + if (qpi.invocationReward() > 0) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + } + if (qpi.invocator() == state.get().initialRegister1 || qpi.invocator() == state.get().initialRegister2 || qpi.invocator() == state.get().initialRegister3 || qpi.invocator() == state.get().initialRegister4 || qpi.invocator() == state.get().initialRegister5) + { + output.returnCode = QRAFFLE_INITIAL_REGISTER_CANNOT_LOGOUT; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_initialRegisterCannotLogout, 0 }; + LOG_INFO(locals.log); + return ; + } + if (state.get().registers.contains(qpi.invocator()) == 0) + { + output.returnCode = QRAFFLE_UNREGISTERED; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_unregistered, 0 }; + LOG_INFO(locals.log); + return ; + } + + state.get().registers.get(qpi.invocator(), locals.tokenType); + + if (locals.tokenType == 1) + { + // Use qubic for logout + locals.refundAmount = QRAFFLE_REGISTER_AMOUNT - QRAFFLE_LOGOUT_FEE; + qpi.transfer(qpi.invocator(), locals.refundAmount); + state.mut().epochRevenue += QRAFFLE_LOGOUT_FEE; + } + else if (locals.tokenType == 2) + { + // Use QXMR tokens for logout + locals.refundAmount = QRAFFLE_QXMR_REGISTER_AMOUNT - QRAFFLE_QXMR_LOGOUT_FEE; + + // Check if contract has enough QXMR tokens + if (qpi.numberOfPossessedShares(QRAFFLE_QXMR_ASSET_NAME, state.get().QXMRIssuer, SELF, SELF, SELF_INDEX, SELF_INDEX) < locals.refundAmount) + { + output.returnCode = QRAFFLE_INSUFFICIENT_QXMR; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_insufficientQXMR, 0 }; + LOG_INFO(locals.log); + return ; + } + + // Transfer QXMR tokens back to user + if (qpi.transferShareOwnershipAndPossession(QRAFFLE_QXMR_ASSET_NAME, state.get().QXMRIssuer, SELF, SELF, locals.refundAmount, qpi.invocator()) < 0) + { + output.returnCode = QRAFFLE_INSUFFICIENT_QXMR; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_insufficientQXMR, 0 }; + LOG_INFO(locals.log); + return ; + } + + state.mut().epochQXMRRevenue += QRAFFLE_QXMR_LOGOUT_FEE; + } + + state.mut().registers.removeByKey(qpi.invocator()); + state.mut().numberOfRegisters--; + output.returnCode = QRAFFLE_SUCCESS; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_success, 0 }; + LOG_INFO(locals.log); + } + + struct submitEntryAmount_locals + { + Logger log; + }; + + PUBLIC_PROCEDURE_WITH_LOCALS(submitEntryAmount) + { + if (qpi.invocationReward() > 0) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + } + if (input.amount < QRAFFLE_MIN_QRAFFLE_AMOUNT || input.amount > QRAFFLE_MAX_QRAFFLE_AMOUNT) + { + output.returnCode = QRAFFLE_INVALID_ENTRY_AMOUNT; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_invalidEntryAmount, 0 }; + LOG_INFO(locals.log); + return ; + } + if (state.get().registers.contains(qpi.invocator()) == 0) + { + output.returnCode = QRAFFLE_UNREGISTERED; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_unregistered, 0 }; + LOG_INFO(locals.log); + return ; + } + if (state.get().quRaffleEntryAmount.contains(qpi.invocator()) == 0) + { + state.mut().numberOfEntryAmountSubmitted++; + } + state.mut().quRaffleEntryAmount.set(qpi.invocator(), input.amount); + output.returnCode = QRAFFLE_SUCCESS; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_success, 0 }; + LOG_INFO(locals.log); + } + + struct submitProposal_locals + { + ProposalInfo proposal; + uint8 countThisEpoch; + Logger log; + }; + + PUBLIC_PROCEDURE_WITH_LOCALS(submitProposal) + { + if (qpi.invocationReward() > 0) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + } + if (state.get().registers.contains(qpi.invocator()) == 0) + { + output.returnCode = QRAFFLE_UNREGISTERED; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_unregistered, 0 }; + LOG_INFO(locals.log); + return ; + } + if (state.get().numberOfProposals >= QRAFFLE_MAX_PROPOSAL_EPOCH) + { + output.returnCode = QRAFFLE_MAX_PROPOSAL_EPOCH_REACHED; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_maxProposalEpochReached, 0 }; + LOG_INFO(locals.log); + return ; + } + locals.countThisEpoch = 0; + if (state.get().proposalsPerProposer.contains(qpi.invocator())) + { + state.get().proposalsPerProposer.get(qpi.invocator(), locals.countThisEpoch); + } + if (locals.countThisEpoch >= QRAFFLE_MAX_PROPOSALS_PER_PROPOSER) + { + output.returnCode = QRAFFLE_MAX_PROPOSAL_PER_USER_REACHED; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_maxProposalPerUserReached, 0 }; + LOG_INFO(locals.log); + return ; + } + if (input.entryAmount <= 0) + { + output.returnCode = QRAFFLE_INVALID_ENTRY_AMOUNT; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_invalidEntryAmount, 0 }; + LOG_INFO(locals.log); + return ; + } + if (!qpi.isAssetIssued(input.tokenIssuer, input.tokenName)) + { + output.returnCode = QRAFFLE_INVALID_TOKEN_TYPE; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_invalidTokenType, 0 }; + LOG_INFO(locals.log); + return ; + } + + locals.proposal.token.issuer = input.tokenIssuer; + locals.proposal.token.assetName = input.tokenName; + locals.proposal.entryAmount = input.entryAmount; + locals.proposal.proposer = qpi.invocator(); + state.mut().proposals.set(state.get().numberOfProposals, locals.proposal); + state.mut().numberOfProposals++; + state.mut().proposalsPerProposer.set(qpi.invocator(), locals.countThisEpoch + 1); + output.returnCode = QRAFFLE_SUCCESS; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_proposalSubmitted, 0 }; + LOG_INFO(locals.log); + } + + struct voteInProposal_locals + { + ProposalInfo proposal; + BitArray participation; + BitArray values; + uint32 votedCount; + Logger log; + }; + + PUBLIC_PROCEDURE_WITH_LOCALS(voteInProposal) + { + if (qpi.invocationReward() > 0) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + } + if (state.get().registers.contains(qpi.invocator()) == 0) + { + output.returnCode = QRAFFLE_UNREGISTERED; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_unregistered, 0 }; + LOG_INFO(locals.log); + return ; + } + if (input.indexOfProposal >= state.get().numberOfProposals) + { + output.returnCode = QRAFFLE_INVALID_PROPOSAL; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_proposalNotFound, 0 }; + LOG_INFO(locals.log); + return ; + } + locals.proposal = state.get().proposals.get(input.indexOfProposal); + + // O(1) vote lookup via per-user bitfield (id hash, no K12 overhead). + state.get().voteParticipation.get(qpi.invocator(), locals.participation); + if (locals.participation.get(input.indexOfProposal)) + { + // Already voted — check if same direction. + state.get().voteValues.get(qpi.invocator(), locals.values); + if (locals.values.get(input.indexOfProposal) == input.yes) + { + output.returnCode = QRAFFLE_ALREADY_VOTED; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_alreadyVoted, 0 }; + LOG_INFO(locals.log); + return ; + } + // Flip the vote: update counters with underflow guard. + if (input.yes) + { + locals.proposal.nYes++; + if (locals.proposal.nNo > 0) { locals.proposal.nNo--; } + } + else + { + locals.proposal.nNo++; + if (locals.proposal.nYes > 0) { locals.proposal.nYes--; } + } + state.mut().proposals.set(input.indexOfProposal, locals.proposal); + locals.values.set(input.indexOfProposal, input.yes); + state.mut().voteValues.replace(qpi.invocator(), locals.values); + output.returnCode = QRAFFLE_SUCCESS; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_proposalVoted, 0 }; + LOG_INFO(locals.log); + return ; + } + + // New vote: check capacity before inserting. + locals.votedCount = state.get().numberOfVotedInProposal.get(input.indexOfProposal); + if (locals.votedCount >= QRAFFLE_MAX_MEMBER) + { + output.returnCode = QRAFFLE_MAX_MEMBER_REACHED; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_maxMemberReached, 0 }; + LOG_INFO(locals.log); + return ; + } + if (input.yes) + { + locals.proposal.nYes++; + } + else + { + locals.proposal.nNo++; + } + state.mut().proposals.set(input.indexOfProposal, locals.proposal); + + // Mark participation and record vote direction. + locals.participation.set(input.indexOfProposal, 1); + state.mut().voteParticipation.set(qpi.invocator(), locals.participation); + state.get().voteValues.get(qpi.invocator(), locals.values); + locals.values.set(input.indexOfProposal, input.yes); + state.mut().voteValues.set(qpi.invocator(), locals.values); + + state.mut().numberOfVotedInProposal.set(input.indexOfProposal, locals.votedCount + 1); + output.returnCode = QRAFFLE_SUCCESS; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_proposalVoted, 0 }; + LOG_INFO(locals.log); + } + + struct depositInQuRaffle_locals + { + Logger log; + }; + + PUBLIC_PROCEDURE_WITH_LOCALS(depositInQuRaffle) + { + if (state.get().numberOfQuRaffleMembers >= QRAFFLE_MAX_MEMBER) + { + if (qpi.invocationReward() > 0) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + } + output.returnCode = QRAFFLE_MAX_MEMBER_REACHED; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_maxMemberReachedForQuRaffle, 0 }; + LOG_INFO(locals.log); + return ; + } + if (qpi.invocationReward() < (sint64)state.get().qREAmount) + { + if (qpi.invocationReward() > 0) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + } + output.returnCode = QRAFFLE_INSUFFICIENT_FUND; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_insufficientQubic, 0 }; + LOG_INFO(locals.log); + return ; + } + // O(1) duplicate check via HashSet (replaces former O(N) linear scan over quRaffleMembers). + if (state.get().quRaffleMemberSet.contains(qpi.invocator())) + { + if (qpi.invocationReward() > 0) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + } + output.returnCode = QRAFFLE_ALREADY_REGISTERED; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_alreadyRegistered, 0 }; + LOG_INFO(locals.log); + return ; + } + qpi.transfer(qpi.invocator(), qpi.invocationReward() - state.get().qREAmount); + state.mut().quRaffleMembers.set(state.get().numberOfQuRaffleMembers, qpi.invocator()); + state.mut().quRaffleMemberSet.add(qpi.invocator()); + state.mut().numberOfQuRaffleMembers++; + output.returnCode = QRAFFLE_SUCCESS; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_quRaffleDeposited, 0 }; + LOG_INFO(locals.log); + } + + struct depositInTokenRaffle_locals + { + ActiveTokenRaffleInfo raffleInfo; + BitArray participation; + uint32 currentMembers; + Logger log; + }; + + PUBLIC_PROCEDURE_WITH_LOCALS(depositInTokenRaffle) + { + if (qpi.invocationReward() < QRAFFLE_TRANSFER_SHARE_FEE) + { + if (qpi.invocationReward() > 0) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + } + output.returnCode = QRAFFLE_INSUFFICIENT_FUND; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_insufficientQubic, 0 }; + LOG_INFO(locals.log); + return ; + } + + if (input.indexOfTokenRaffle >= state.get().numberOfActiveTokenRaffle) + { + if (qpi.invocationReward() > 0) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + } + output.returnCode = QRAFFLE_INVALID_TOKEN_RAFFLE; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_invalidTokenRaffle, 0 }; + LOG_INFO(locals.log); + return ; + } + locals.currentMembers = state.get().numberOfTokenRaffleMembers.get(input.indexOfTokenRaffle); + if (locals.currentMembers >= QRAFFLE_TOKEN_RAFFLE_SLOT_SIZE) + { + if (qpi.invocationReward() > 0) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + } + output.returnCode = QRAFFLE_MAX_MEMBER_REACHED; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_maxMemberReached, 0 }; + LOG_INFO(locals.log); + return ; + } + // O(1) duplicate check via per-user bitfield. + state.get().tokenRaffleParticipation.get(qpi.invocator(), locals.participation); + if (locals.participation.get(input.indexOfTokenRaffle)) + { + if (qpi.invocationReward() > 0) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + } + output.returnCode = QRAFFLE_ALREADY_REGISTERED; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_alreadyRegistered, 0 }; + LOG_INFO(locals.log); + return ; + } + locals.raffleInfo = state.get().activeTokenRaffle.get(input.indexOfTokenRaffle); + if (qpi.transferShareOwnershipAndPossession(locals.raffleInfo.token.assetName, locals.raffleInfo.token.issuer, qpi.invocator(), qpi.invocator(), locals.raffleInfo.entryAmount, SELF) < 0) + { + if (qpi.invocationReward() > 0) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + } + output.returnCode = QRAFFLE_FAILED_TO_DEPOSIT; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_transferFailed, 0 }; + LOG_INFO(locals.log); + return ; + } + // Keep QRAFFLE_TRANSFER_SHARE_FEE as service revenue; refund any excess. + if (qpi.invocationReward() > QRAFFLE_TRANSFER_SHARE_FEE) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward() - QRAFFLE_TRANSFER_SHARE_FEE); + } + state.mut().epochRevenue += QRAFFLE_TRANSFER_SHARE_FEE; + + // Store member in flat slot array and mark participation. + state.mut().tokenRaffleMemberSlots.set(input.indexOfTokenRaffle * QRAFFLE_TOKEN_RAFFLE_SLOT_SIZE + locals.currentMembers, qpi.invocator()); + state.mut().numberOfTokenRaffleMembers.set(input.indexOfTokenRaffle, locals.currentMembers + 1); + locals.participation.set(input.indexOfTokenRaffle, 1); + state.mut().tokenRaffleParticipation.set(qpi.invocator(), locals.participation); + output.returnCode = QRAFFLE_SUCCESS; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_tokenRaffleDeposited, 0 }; + LOG_INFO(locals.log); + } + + struct TransferShareManagementRights_locals + { + Asset asset; + sint64 offeredFee; + sint64 paidFee; + Logger log; + }; + + PUBLIC_PROCEDURE_WITH_LOCALS(TransferShareManagementRights) + { + // Requires QRAFFLE_TRANSFER_SHARE_FEE minimum. The rest is offered to the destination + // contract as transfer fee; the unused portion is refunded after releaseShares. + if (qpi.invocationReward() < QRAFFLE_TRANSFER_SHARE_FEE) + { + if (qpi.invocationReward() > 0) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + } + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_insufficientQubic, 0 }; + LOG_INFO(locals.log); + return ; + } + + if (qpi.numberOfPossessedShares(input.tokenName, input.tokenIssuer,qpi.invocator(), qpi.invocator(), SELF_INDEX, SELF_INDEX) < input.numberOfShares) + { + // Not enough shares — refund in full. + output.transferredNumberOfShares = 0; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_notEnoughShares, 0 }; + LOG_INFO(locals.log); + if (qpi.invocationReward() > 0) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + } + } + else + { + locals.asset.assetName = input.tokenName; + locals.asset.issuer = input.tokenIssuer; + locals.offeredFee = qpi.invocationReward() - QRAFFLE_TRANSFER_SHARE_FEE; + locals.paidFee = qpi.releaseShares(locals.asset, qpi.invocator(), qpi.invocator(), input.numberOfShares, + input.newManagingContractIndex, input.newManagingContractIndex, locals.offeredFee); + if (locals.paidFee < 0) + { + // Transfer rejected by the destination — refund everything. + output.transferredNumberOfShares = 0; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_transferFailed, 0 }; + LOG_INFO(locals.log); + if (qpi.invocationReward() > 0) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + } + } + else + { + // Success — keep service fee as revenue, refund unused transfer fee. + output.transferredNumberOfShares = input.numberOfShares; + state.mut().epochRevenue += QRAFFLE_TRANSFER_SHARE_FEE; + if (locals.offeredFee > locals.paidFee) + { + qpi.transfer(qpi.invocator(), locals.offeredFee - locals.paidFee); + } + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_shareManagementRightsTransferred, 0 }; + LOG_INFO(locals.log); + } + } + } + + // ── createAssetRaffle ────────────────────────────────────────────────────── + struct createAssetRaffle_locals + { + AssetRaffleItem item; + AssetRaffleItem dupItem; + AssetRaffleItem rollbackItem; + AssetRaffleInfo info; + AssetRaffleCreatedLogger clog; + Logger log; + uint64 proposalFeeHalf; + uint8 creatorCount; + uint32 slot; + uint32 i; + uint32 j; + sint64 escrowResult; + bit dupFound; + }; + + PUBLIC_PROCEDURE_WITH_LOCALS(createAssetRaffle) + { + if (qpi.invocationReward() < (sint64)QRAFFLE_ASSET_RAFFLE_PROPOSAL_FEE) + { + if (qpi.invocationReward() > 0) { qpi.transfer(qpi.invocator(), qpi.invocationReward()); } + output.returnCode = QRAFFLE_INSUFFICIENT_FUND; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_insufficientQubic, 0 }; + LOG_INFO(locals.log); + return; + } + if (!state.get().registers.contains(qpi.invocator())) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + output.returnCode = QRAFFLE_UNREGISTERED; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_unregistered, 0 }; + LOG_INFO(locals.log); + return; + } + locals.creatorCount = 0; + if (state.get().assetRafflesPerCreator.contains(qpi.invocator())) + { + state.get().assetRafflesPerCreator.get(qpi.invocator(), locals.creatorCount); + } + if (locals.creatorCount >= QRAFFLE_MAX_ASSET_RAFFLES_PER_CREATOR) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + output.returnCode = QRAFFLE_MAX_ASSET_RAFFLES_REACHED; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_maxAssetRafflesReached, 0 }; + LOG_INFO(locals.log); + return; + } + if (state.get().numberOfActiveAssetRaffles >= QRAFFLE_MAX_ASSET_RAFFLES_PER_EPOCH) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + output.returnCode = QRAFFLE_MAX_ASSET_RAFFLES_REACHED; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_maxAssetRafflesReached, 0 }; + LOG_INFO(locals.log); + return; + } + if (input.bundleSize == 0 || input.bundleSize > QRAFFLE_MAX_ASSETS_PER_BUNDLE) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + output.returnCode = QRAFFLE_INVALID_BUNDLE; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_invalidBundle, 0 }; + LOG_INFO(locals.log); + return; + } + if (input.entryTicketQu < QRAFFLE_MIN_ASSET_TICKET_AMOUNT || input.entryTicketQu > QRAFFLE_MAX_ASSET_TICKET_AMOUNT) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + output.returnCode = QRAFFLE_INVALID_ENTRY_AMOUNT; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_invalidEntryAmount, 0 }; + LOG_INFO(locals.log); + return; + } + if (input.reservePriceQu == 0) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + output.returnCode = QRAFFLE_INVALID_RESERVE_PRICE; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_invalidReservePrice, 0 }; + LOG_INFO(locals.log); + return; + } + // Guard against uint64 overflow in the END_EPOCH reserve check (reservePriceQu * 100). + if (input.reservePriceQu > div(0xFFFFFFFFFFFFFFFFull, 100ull)) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + output.returnCode = QRAFFLE_INVALID_RESERVE_PRICE; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_invalidReservePrice, 0 }; + LOG_INFO(locals.log); + return; + } + for (locals.i = 0; locals.i < input.bundleSize; locals.i++) + { + locals.item = input.bundleItems.get(locals.i); + if (!qpi.isAssetIssued(locals.item.asset.issuer, locals.item.asset.assetName)) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + output.returnCode = QRAFFLE_INVALID_BUNDLE; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_invalidBundle, 0 }; + LOG_INFO(locals.log); + return; + } + if (locals.item.numberOfShares <= 0) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + output.returnCode = QRAFFLE_INVALID_BUNDLE; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_invalidBundle, 0 }; + LOG_INFO(locals.log); + return; + } + // QRAFFLE and QXMR are reserved for dividends/registration; disallow in bundles. + if ((locals.item.asset.assetName == QRAFFLE_ASSET_NAME && locals.item.asset.issuer == NULL_ID) + || (locals.item.asset.assetName == QRAFFLE_QXMR_ASSET_NAME && locals.item.asset.issuer == state.get().QXMRIssuer)) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + output.returnCode = QRAFFLE_INVALID_BUNDLE; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_invalidBundle, 0 }; + LOG_INFO(locals.log); + return; + } + // Duplicate-asset check within the bundle; O(N²) acceptable for N ≤ 4. + locals.dupFound = 0; + for (locals.j = 0; locals.j < locals.i; locals.j++) + { + locals.dupItem = input.bundleItems.get(locals.j); + if (locals.dupItem.asset.assetName == locals.item.asset.assetName + && locals.dupItem.asset.issuer == locals.item.asset.issuer) + { + locals.dupFound = 1; + break; + } + } + if (locals.dupFound) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + output.returnCode = QRAFFLE_INVALID_BUNDLE; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_invalidBundle, 0 }; + LOG_INFO(locals.log); + return; + } + } + + // Atomic escrow: if any item fails, roll back all previously transferred items. + locals.slot = state.get().numberOfActiveAssetRaffles; + for (locals.i = 0; locals.i < input.bundleSize; locals.i++) + { + locals.item = input.bundleItems.get(locals.i); + locals.escrowResult = qpi.transferShareOwnershipAndPossession( + locals.item.asset.assetName, locals.item.asset.issuer, + qpi.invocator(), qpi.invocator(), + locals.item.numberOfShares, SELF); + if (locals.escrowResult < 0) + { + // Rollback: return all previously escrowed items to the invocator. + // These transfers move shares we just received from the same invocator back to + // them, so they should not fail in normal circumstances. If a rollback transfer + // does fail (e.g. spectrum entry evicted), residual shares stay under contract + // management for governance recovery. + for (locals.j = 0; locals.j < locals.i; locals.j++) + { + locals.rollbackItem = input.bundleItems.get(locals.j); + qpi.transferShareOwnershipAndPossession( + locals.rollbackItem.asset.assetName, locals.rollbackItem.asset.issuer, + SELF, SELF, + locals.rollbackItem.numberOfShares, qpi.invocator()); + } + qpi.transfer(qpi.invocator(), qpi.invocationReward()); + output.returnCode = QRAFFLE_BUNDLE_ESCROW_FAILED; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_assetRaffleBundleEscrowFailed, 0 }; + LOG_INFO(locals.log); + return; + } + state.mut().activeAssetRaffleItems.set(locals.slot * QRAFFLE_MAX_ASSETS_PER_BUNDLE + locals.i, locals.item); + } + + locals.info.creator = qpi.invocator(); + locals.info.reservePriceQu = input.reservePriceQu; + locals.info.entryTicketQu = input.entryTicketQu; + locals.info.totalTicketsPaidQu = 0; + locals.info.numberOfBuyers = 0; + locals.info.totalTickets = 0; + locals.info.bundleSize = input.bundleSize; + locals.info.epoch = qpi.epoch(); + state.mut().activeAssetRaffles.set(locals.slot, locals.info); + state.mut().numberOfActiveAssetRaffles++; + state.mut().assetRafflesPerCreator.set(qpi.invocator(), locals.creatorCount + 1); + + // 50% of proposal fee → shareholders via epochRevenue; 50% → DAO bucket distributed in END_EPOCH. + locals.proposalFeeHalf = div(QRAFFLE_ASSET_RAFFLE_PROPOSAL_FEE, 2); + state.mut().epochRevenue += locals.proposalFeeHalf; + state.mut().epochAssetRaffleDaoBucket += (QRAFFLE_ASSET_RAFFLE_PROPOSAL_FEE - locals.proposalFeeHalf); + state.mut().totalAssetRaffleProposalFees += QRAFFLE_ASSET_RAFFLE_PROPOSAL_FEE; + state.mut().totalAssetRafflesCreated++; + + if (qpi.invocationReward() > (sint64)QRAFFLE_ASSET_RAFFLE_PROPOSAL_FEE) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward() - (sint64)QRAFFLE_ASSET_RAFFLE_PROPOSAL_FEE); + } + + output.raffleIndex = locals.slot; + output.returnCode = QRAFFLE_SUCCESS; + locals.clog = AssetRaffleCreatedLogger{ + QRAFFLE_CONTRACT_INDEX, QRAFFLE_assetRaffleCreated, + locals.slot, qpi.invocator(), + input.reservePriceQu, input.entryTicketQu, + input.bundleSize, 0 + }; + LOG_INFO(locals.clog); + } + + // ── buyAssetRaffleTicket ─────────────────────────────────────────────────── + struct buyAssetRaffleTicket_locals + { + AssetRaffleInfo info; + AssetRaffleTicketLogger tlog; + Logger log; + BitArray participation; + Array slotIndexArr; + uint64 cost; + uint32 baseSlot; + uint32 existingTickets; + uint32 buyerSlot; + bit isNewBuyer; + }; + + PUBLIC_PROCEDURE_WITH_LOCALS(buyAssetRaffleTicket) + { + if (input.indexOfAssetRaffle >= state.get().numberOfActiveAssetRaffles) + { + if (qpi.invocationReward() > 0) { qpi.transfer(qpi.invocator(), qpi.invocationReward()); } + output.returnCode = QRAFFLE_INVALID_ASSET_RAFFLE; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_invalidTokenRaffle, 0 }; + LOG_INFO(locals.log); + return; + } + if (input.numberOfTickets == 0 || input.numberOfTickets > QRAFFLE_MAX_TICKETS_PER_BUYER) + { + // Reject zero or absurd ticket counts up-front so the cost multiplication below cannot overflow. + if (qpi.invocationReward() > 0) { qpi.transfer(qpi.invocator(), qpi.invocationReward()); } + output.returnCode = QRAFFLE_INVALID_ENTRY_AMOUNT; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_invalidEntryAmount, 0 }; + LOG_INFO(locals.log); + return; + } + locals.info = state.get().activeAssetRaffles.get(input.indexOfAssetRaffle); + // Overflow-safe cost: entryTicketQu ≤ 1e12 (QRAFFLE_MAX_ASSET_TICKET_AMOUNT) and tickets ≤ 100, + // so cost ≤ 1e14, safely under uint64 max (~1.8e19). totalTicketsPaidQu accumulates across + // up to 1024 buyers, max ~1e17, also safe. + locals.cost = locals.info.entryTicketQu * (uint64)input.numberOfTickets; + if (qpi.invocationReward() < (sint64)locals.cost) + { + if (qpi.invocationReward() > 0) { qpi.transfer(qpi.invocator(), qpi.invocationReward()); } + output.returnCode = QRAFFLE_INSUFFICIENT_FUND; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_insufficientQubic, 0 }; + LOG_INFO(locals.log); + return; + } + + locals.baseSlot = input.indexOfAssetRaffle * QRAFFLE_MAX_ASSET_TICKET_BUYERS; + locals.existingTickets = 0; + locals.isNewBuyer = 1; + locals.buyerSlot = locals.baseSlot + locals.info.numberOfBuyers; + + state.get().assetRaffleParticipation.get(qpi.invocator(), locals.participation); + if (locals.participation.get(input.indexOfAssetRaffle)) + { + // Returning buyer: resolve slot via index map (O(1)). + locals.isNewBuyer = 0; + state.get().assetRaffleBuyerSlotIndex.get(qpi.invocator(), locals.slotIndexArr); + locals.buyerSlot = locals.baseSlot + (uint32)locals.slotIndexArr.get(input.indexOfAssetRaffle); + locals.existingTickets = state.get().activeAssetRaffleBuyerTickets.get(locals.buyerSlot); + } + + if (locals.existingTickets + input.numberOfTickets > QRAFFLE_MAX_TICKETS_PER_BUYER) + { + if (qpi.invocationReward() > 0) { qpi.transfer(qpi.invocator(), qpi.invocationReward()); } + output.returnCode = QRAFFLE_TICKET_LIMIT_REACHED; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_ticketLimitReached, 0 }; + LOG_INFO(locals.log); + return; + } + + if (locals.isNewBuyer) + { + if (locals.info.numberOfBuyers >= QRAFFLE_MAX_ASSET_TICKET_BUYERS) + { + if (qpi.invocationReward() > 0) { qpi.transfer(qpi.invocator(), qpi.invocationReward()); } + output.returnCode = QRAFFLE_ASSET_RAFFLE_FULL; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_assetRaffleFull, 0 }; + LOG_INFO(locals.log); + return; + } + state.mut().activeAssetRaffleBuyers.set(locals.buyerSlot, qpi.invocator()); + // Record slot position so future purchases skip the buyer-list scan. + state.get().assetRaffleBuyerSlotIndex.get(qpi.invocator(), locals.slotIndexArr); + locals.slotIndexArr.set(input.indexOfAssetRaffle, (uint16)locals.info.numberOfBuyers); + state.mut().assetRaffleBuyerSlotIndex.set(qpi.invocator(), locals.slotIndexArr); + locals.participation.set(input.indexOfAssetRaffle, 1); + state.mut().assetRaffleParticipation.set(qpi.invocator(), locals.participation); + locals.info.numberOfBuyers++; + } + + state.mut().activeAssetRaffleBuyerTickets.set(locals.buyerSlot, locals.existingTickets + input.numberOfTickets); + locals.info.totalTickets += input.numberOfTickets; + locals.info.totalTicketsPaidQu += locals.cost; + state.mut().activeAssetRaffles.set(input.indexOfAssetRaffle, locals.info); + + if (qpi.invocationReward() > (sint64)locals.cost) + { + qpi.transfer(qpi.invocator(), qpi.invocationReward() - (sint64)locals.cost); + } + + output.ticketsBought = input.numberOfTickets; + output.returnCode = QRAFFLE_SUCCESS; + locals.tlog = AssetRaffleTicketLogger{ + QRAFFLE_CONTRACT_INDEX, QRAFFLE_assetRaffleTicketBought, + input.indexOfAssetRaffle, qpi.invocator(), + input.numberOfTickets, locals.cost, 0 + }; + LOG_INFO(locals.tlog); + } + + // ── cancelAssetRaffle ────────────────────────────────────────────────────── + struct cancelAssetRaffle_locals + { + AssetRaffleInfo info; + AssetRaffleItem item; + AssetRaffleInfo lastInfo; + AssetRaffleItem lastItem; + Array slotIndexArr; + BitArray participation; + Logger log; + id movedBuyer; + uint8 creatorCount; + uint32 lastSlot; + uint32 i; + }; + + PUBLIC_PROCEDURE_WITH_LOCALS(cancelAssetRaffle) + { + if (qpi.invocationReward() > 0) { qpi.transfer(qpi.invocator(), qpi.invocationReward()); } + + if (input.indexOfAssetRaffle >= state.get().numberOfActiveAssetRaffles) + { + output.returnCode = QRAFFLE_INVALID_ASSET_RAFFLE; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_invalidTokenRaffle, 0 }; + LOG_INFO(locals.log); + return; + } + locals.info = state.get().activeAssetRaffles.get(input.indexOfAssetRaffle); + if (locals.info.creator != qpi.invocator()) + { + output.returnCode = QRAFFLE_CANCEL_NOT_ALLOWED; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_cancelNotAllowed, 0 }; + LOG_INFO(locals.log); + return; + } + if (locals.info.numberOfBuyers > 0) + { + output.returnCode = QRAFFLE_CANCEL_NOT_ALLOWED; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_cancelNotAllowed, 0 }; + LOG_INFO(locals.log); + return; + } + + for (locals.i = 0; locals.i < locals.info.bundleSize; locals.i++) + { + locals.item = state.get().activeAssetRaffleItems.get(input.indexOfAssetRaffle * QRAFFLE_MAX_ASSETS_PER_BUNDLE + locals.i); + qpi.transferShareOwnershipAndPossession( + locals.item.asset.assetName, locals.item.asset.issuer, + SELF, SELF, + locals.item.numberOfShares, qpi.invocator()); + } + + // Swap-and-pop: fill the vacated slot with the last active raffle. + // Three parallel arrays must be kept in sync: raffle info, bundle items, and buyer lists. + locals.lastSlot = state.get().numberOfActiveAssetRaffles - 1; + if (input.indexOfAssetRaffle < locals.lastSlot) + { + locals.lastInfo = state.get().activeAssetRaffles.get(locals.lastSlot); + state.mut().activeAssetRaffles.set(input.indexOfAssetRaffle, locals.lastInfo); + + for (locals.i = 0; locals.i < locals.lastInfo.bundleSize; locals.i++) + { + locals.lastItem = state.get().activeAssetRaffleItems.get(locals.lastSlot * QRAFFLE_MAX_ASSETS_PER_BUNDLE + locals.i); + state.mut().activeAssetRaffleItems.set(input.indexOfAssetRaffle * QRAFFLE_MAX_ASSETS_PER_BUNDLE + locals.i, locals.lastItem); + } + + // Cancelled raffle already has numberOfBuyers==0, so the destination region is safe to overwrite. + for (locals.i = 0; locals.i < locals.lastInfo.numberOfBuyers; locals.i++) + { + locals.movedBuyer = state.get().activeAssetRaffleBuyers.get(locals.lastSlot * QRAFFLE_MAX_ASSET_TICKET_BUYERS + locals.i); + state.mut().activeAssetRaffleBuyers.set( + input.indexOfAssetRaffle * QRAFFLE_MAX_ASSET_TICKET_BUYERS + locals.i, + locals.movedBuyer); + state.mut().activeAssetRaffleBuyerTickets.set( + input.indexOfAssetRaffle * QRAFFLE_MAX_ASSET_TICKET_BUYERS + locals.i, + state.get().activeAssetRaffleBuyerTickets.get(locals.lastSlot * QRAFFLE_MAX_ASSET_TICKET_BUYERS + locals.i)); + // Update slot-index map: raffle index changed from lastSlot to indexOfAssetRaffle; + // relative position within the buyer region is preserved. + state.get().assetRaffleBuyerSlotIndex.get(locals.movedBuyer, locals.slotIndexArr); + locals.slotIndexArr.set(input.indexOfAssetRaffle, locals.slotIndexArr.get(locals.lastSlot)); + locals.slotIndexArr.set(locals.lastSlot, 0xFFFF); + state.mut().assetRaffleBuyerSlotIndex.replace(locals.movedBuyer, locals.slotIndexArr); + + // Keep participation bitfield aligned with moved raffle index. + state.get().assetRaffleParticipation.get(locals.movedBuyer, locals.participation); + locals.participation.set(input.indexOfAssetRaffle, 1); + locals.participation.set(locals.lastSlot, 0); + state.mut().assetRaffleParticipation.replace(locals.movedBuyer, locals.participation); + } + } + state.mut().numberOfActiveAssetRaffles--; + + // Decrement per-creator counter so the creator can replace the cancelled raffle + // within the same epoch. The 500K proposal fee is *not* refunded (anti-spam). + locals.creatorCount = 0; + if (state.get().assetRafflesPerCreator.contains(qpi.invocator())) + { + state.get().assetRafflesPerCreator.get(qpi.invocator(), locals.creatorCount); + if (locals.creatorCount > 0) + { + state.mut().assetRafflesPerCreator.set(qpi.invocator(), (uint8)(locals.creatorCount - 1)); + } + } + + output.returnCode = QRAFFLE_SUCCESS; + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_assetRaffleCancelled, 0 }; + LOG_INFO(locals.log); + } + + struct getRegisters_locals + { + id user; + sint64 idx; + uint32 i; + }; + + PUBLIC_FUNCTION_WITH_LOCALS(getRegisters) + { + if (input.limit > 20) + { + output.returnCode = QRAFFLE_INVALID_OFFSET_OR_LIMIT; + return ; + } + if (input.offset >= state.get().numberOfRegisters) + { + output.returnCode = QRAFFLE_SUCCESS; + return ; + } + locals.idx = state.get().registers.nextElementIndex(NULL_INDEX); + while (locals.idx != NULL_INDEX) + { + locals.user = state.get().registers.key(locals.idx); + if (locals.i >= input.offset && locals.i < input.offset + input.limit) + { + if (locals.i - input.offset == 0) + { + output.register1 = locals.user; + } + else if (locals.i - input.offset == 1) + { + output.register2 = locals.user; + } + else if (locals.i - input.offset == 2) + { + output.register3 = locals.user; + } + else if (locals.i - input.offset == 3) + { + output.register4 = locals.user; + } + else if (locals.i - input.offset == 4) + { + output.register5 = locals.user; + } + else if (locals.i - input.offset == 5) + { + output.register6 = locals.user; + } + else if (locals.i - input.offset == 6) + { + output.register7 = locals.user; + } + else if (locals.i - input.offset == 7) + { + output.register8 = locals.user; + } + else if (locals.i - input.offset == 8) + { + output.register9 = locals.user; + } + else if (locals.i - input.offset == 9) + { + output.register10 = locals.user; + } + else if (locals.i - input.offset == 10) + { + output.register11 = locals.user; + } + else if (locals.i - input.offset == 11) + { + output.register12 = locals.user; + } + else if (locals.i - input.offset == 12) + { + output.register13 = locals.user; + } + else if (locals.i - input.offset == 13) + { + output.register14 = locals.user; + } + else if (locals.i - input.offset == 14) + { + output.register15 = locals.user; + } + else if (locals.i - input.offset == 15) + { + output.register16 = locals.user; + } + else if (locals.i - input.offset == 16) + { + output.register17 = locals.user; + } + else if (locals.i - input.offset == 17) + { + output.register18 = locals.user; + } + else if (locals.i - input.offset == 18) + { + output.register19 = locals.user; + } + else if (locals.i - input.offset == 19) + { + output.register20 = locals.user; + } + } + if (locals.i >= input.offset + input.limit) + { + break; + } + locals.i++; + locals.idx = state.get().registers.nextElementIndex(locals.idx); + } + output.returnCode = QRAFFLE_SUCCESS; + } + + PUBLIC_FUNCTION(getAnalytics) + { + output.currentQuRaffleAmount = state.get().qREAmount; + output.totalBurnAmount = state.get().totalBurnAmount; + output.totalCharityAmount = state.get().totalCharityAmount; + output.totalShareholderAmount = state.get().totalShareholderAmount; + output.totalRegisterAmount = state.get().totalRegisterAmount; + output.totalFeeAmount = state.get().totalFeeAmount; + output.totalWinnerAmount = state.get().totalWinnerAmount; + output.largestWinnerAmount = state.get().largestWinnerAmount; + output.numberOfRegisters = state.get().numberOfRegisters; + output.numberOfProposals = state.get().numberOfProposals; + output.numberOfQuRaffleMembers = state.get().numberOfQuRaffleMembers; + output.numberOfActiveTokenRaffle = state.get().numberOfActiveTokenRaffle; + output.numberOfEndedTokenRaffle = state.get().numberOfEndedTokenRaffle; + output.numberOfEntryAmountSubmitted = state.get().numberOfEntryAmountSubmitted; + output.returnCode = QRAFFLE_SUCCESS; + } + + PUBLIC_FUNCTION(getActiveProposal) + { + if (input.indexOfProposal >= state.get().numberOfProposals) + { + output.returnCode = QRAFFLE_INVALID_PROPOSAL; + return ; + } + output.tokenName = state.get().proposals.get(input.indexOfProposal).token.assetName; + output.tokenIssuer = state.get().proposals.get(input.indexOfProposal).token.issuer; + output.proposer = state.get().proposals.get(input.indexOfProposal).proposer; + output.entryAmount = state.get().proposals.get(input.indexOfProposal).entryAmount; + output.nYes = state.get().proposals.get(input.indexOfProposal).nYes; + output.nNo = state.get().proposals.get(input.indexOfProposal).nNo; + output.returnCode = QRAFFLE_SUCCESS; + } + + PUBLIC_FUNCTION(getEndedTokenRaffle) + { + if (input.indexOfRaffle >= state.get().numberOfEndedTokenRaffle) + { + output.returnCode = QRAFFLE_INVALID_TOKEN_RAFFLE; + return ; + } + // Reject indices that have been overwritten by the ring buffer. + if (state.get().numberOfEndedTokenRaffle > QRAFFLE_MAX_TOKEN_RAFFLES + && input.indexOfRaffle < state.get().numberOfEndedTokenRaffle - QRAFFLE_MAX_TOKEN_RAFFLES) + { + output.returnCode = QRAFFLE_INVALID_TOKEN_RAFFLE; + return ; + } + output.epochWinner = state.get().tokenRaffle.get(input.indexOfRaffle).epochWinner; + output.tokenName = state.get().tokenRaffle.get(input.indexOfRaffle).token.assetName; + output.tokenIssuer = state.get().tokenRaffle.get(input.indexOfRaffle).token.issuer; + output.entryAmount = state.get().tokenRaffle.get(input.indexOfRaffle).entryAmount; + output.numberOfMembers = state.get().tokenRaffle.get(input.indexOfRaffle).numberOfMembers; + output.winnerIndex = state.get().tokenRaffle.get(input.indexOfRaffle).winnerIndex; + output.epoch = state.get().tokenRaffle.get(input.indexOfRaffle).epoch; + output.returnCode = QRAFFLE_SUCCESS; + } + + struct getEpochRaffleIndexes_locals + { + uint32 ringStart; + uint32 ringEnd; + uint32 i; + }; + + PUBLIC_FUNCTION_WITH_LOCALS(getEpochRaffleIndexes) + { + output.StartIndex = 0; + output.EndIndex = 0; + if (input.epoch > qpi.epoch()) + { + output.returnCode = QRAFFLE_INVALID_EPOCH; + return ; + } + if (input.epoch == qpi.epoch()) + { + output.StartIndex = 0; + output.EndIndex = state.get().numberOfActiveTokenRaffle; + output.returnCode = QRAFFLE_SUCCESS; + return ; + } + // Only scan the valid ring window to avoid re-reading overwritten slots. + locals.ringEnd = state.get().numberOfEndedTokenRaffle; + locals.ringStart = (locals.ringEnd > QRAFFLE_MAX_TOKEN_RAFFLES) ? (locals.ringEnd - QRAFFLE_MAX_TOKEN_RAFFLES) : 0; + for (locals.i = locals.ringStart; locals.i < locals.ringEnd; locals.i++) + { + if (state.get().tokenRaffle.get(locals.i).epoch == input.epoch) + { + output.StartIndex = locals.i; + break; + } + } + locals.i = locals.ringEnd; + while (locals.i > locals.ringStart) + { + locals.i--; + if (state.get().tokenRaffle.get(locals.i).epoch == input.epoch) + { + output.EndIndex = locals.i; + break; + } + } + output.returnCode = QRAFFLE_SUCCESS; + } + + PUBLIC_FUNCTION(getEndedQuRaffle) + { + // Note: indices are masked by the underlying Array (capacity is power-of-two), so + // any uint32 epoch value is safely bounded inside the QuRaffles ring. Slots that + // were never written return a zero-initialised QuRaffleInfo, which the caller can + // detect via numberOfMembers == 0 / epochWinner == NULL_ID. + output.epochWinner = state.get().QuRaffles.get(input.epoch).epochWinner; + output.receivedAmount = state.get().QuRaffles.get(input.epoch).receivedAmount; + output.entryAmount = state.get().QuRaffles.get(input.epoch).entryAmount; + output.numberOfMembers = state.get().QuRaffles.get(input.epoch).numberOfMembers; + output.winnerIndex = state.get().QuRaffles.get(input.epoch).winnerIndex; + output.numberOfDaoMembers = state.get().daoMemberCount.get(input.epoch); + output.returnCode = QRAFFLE_SUCCESS; + } + + PUBLIC_FUNCTION(getActiveTokenRaffle) + { + if (input.indexOfTokenRaffle >= state.get().numberOfActiveTokenRaffle) + { + output.returnCode = QRAFFLE_INVALID_TOKEN_RAFFLE; + return ; + } + output.tokenName = state.get().activeTokenRaffle.get(input.indexOfTokenRaffle).token.assetName; + output.tokenIssuer = state.get().activeTokenRaffle.get(input.indexOfTokenRaffle).token.issuer; + output.entryAmount = state.get().activeTokenRaffle.get(input.indexOfTokenRaffle).entryAmount; + output.numberOfMembers = state.get().numberOfTokenRaffleMembers.get(input.indexOfTokenRaffle); + output.returnCode = QRAFFLE_SUCCESS; + } + + PUBLIC_FUNCTION(getQuRaffleEntryAmountPerUser) + { + if (state.get().quRaffleEntryAmount.contains(input.user) == 0) + { + output.entryAmount = 0; + output.returnCode = QRAFFLE_USER_NOT_FOUND; + } + else + { + state.get().quRaffleEntryAmount.get(input.user, output.entryAmount); + output.returnCode = QRAFFLE_SUCCESS; + } + } + + struct getQuRaffleEntryAverageAmount_locals + { + uint64 entryAmount; + uint64 totalEntryAmount; + sint64 idx; + }; + + PUBLIC_FUNCTION_WITH_LOCALS(getQuRaffleEntryAverageAmount) + { + locals.entryAmount = 0; + locals.totalEntryAmount = 0; + locals.idx = state.get().quRaffleEntryAmount.nextElementIndex(NULL_INDEX); + while (locals.idx != NULL_INDEX) + { + locals.entryAmount = state.get().quRaffleEntryAmount.value(locals.idx); + locals.totalEntryAmount += locals.entryAmount; + locals.idx = state.get().quRaffleEntryAmount.nextElementIndex(locals.idx); + } + if (state.get().numberOfEntryAmountSubmitted > 0) + { + output.entryAverageAmount = div(locals.totalEntryAmount, state.get().numberOfEntryAmountSubmitted); + } + else + { + output.entryAverageAmount = 0; + } + output.returnCode = QRAFFLE_SUCCESS; + } + + // ── Asset Raffle view functions ──────────────────────────────────────────── + + struct getActiveAssetRaffle_locals + { + AssetRaffleInfo info; + }; + + PUBLIC_FUNCTION_WITH_LOCALS(getActiveAssetRaffle) + { + if (input.indexOfAssetRaffle >= state.get().numberOfActiveAssetRaffles) + { + output.returnCode = QRAFFLE_INVALID_ASSET_RAFFLE; + return; + } + locals.info = state.get().activeAssetRaffles.get(input.indexOfAssetRaffle); + output.creator = locals.info.creator; + output.reservePriceQu = locals.info.reservePriceQu; + output.entryTicketQu = locals.info.entryTicketQu; + output.totalTicketsPaidQu = locals.info.totalTicketsPaidQu; + output.numberOfBuyers = locals.info.numberOfBuyers; + output.totalTickets = locals.info.totalTickets; + output.bundleSize = locals.info.bundleSize; + output.epoch = locals.info.epoch; + output.returnCode = QRAFFLE_SUCCESS; + } + + struct getActiveAssetRaffleBundleItem_locals + { + AssetRaffleInfo info; + AssetRaffleItem item; + }; + + PUBLIC_FUNCTION_WITH_LOCALS(getActiveAssetRaffleBundleItem) + { + if (input.indexOfAssetRaffle >= state.get().numberOfActiveAssetRaffles) + { + output.returnCode = QRAFFLE_INVALID_ASSET_RAFFLE; + return; + } + locals.info = state.get().activeAssetRaffles.get(input.indexOfAssetRaffle); + if (input.itemIndex >= locals.info.bundleSize) + { + output.returnCode = QRAFFLE_INVALID_BUNDLE; + return; + } + locals.item = state.get().activeAssetRaffleItems.get( + input.indexOfAssetRaffle * QRAFFLE_MAX_ASSETS_PER_BUNDLE + input.itemIndex); + output.assetIssuer = locals.item.asset.issuer; + output.assetName = locals.item.asset.assetName; + output.numberOfShares = locals.item.numberOfShares; + output.returnCode = QRAFFLE_SUCCESS; + } + + struct getActiveAssetRaffleBuyer_locals + { + AssetRaffleInfo info; + uint32 slot; + }; + + PUBLIC_FUNCTION_WITH_LOCALS(getActiveAssetRaffleBuyer) + { + if (input.indexOfAssetRaffle >= state.get().numberOfActiveAssetRaffles) + { + output.returnCode = QRAFFLE_INVALID_ASSET_RAFFLE; + return; + } + locals.info = state.get().activeAssetRaffles.get(input.indexOfAssetRaffle); + if (input.buyerIndex >= locals.info.numberOfBuyers) + { + output.returnCode = QRAFFLE_INVALID_OFFSET_OR_LIMIT; + return; + } + locals.slot = input.indexOfAssetRaffle * QRAFFLE_MAX_ASSET_TICKET_BUYERS + input.buyerIndex; + output.buyer = state.get().activeAssetRaffleBuyers.get(locals.slot); + output.ticketCount = state.get().activeAssetRaffleBuyerTickets.get(locals.slot); + output.returnCode = QRAFFLE_SUCCESS; + } + + struct getEndedAssetRaffle_locals + { + EndedAssetRaffleInfo r; + uint32 slot; + }; + + PUBLIC_FUNCTION_WITH_LOCALS(getEndedAssetRaffle) + { + if (input.indexOfRaffle >= state.get().numberOfEndedAssetRaffles) + { + output.returnCode = QRAFFLE_INVALID_ASSET_RAFFLE; + return; + } + // Reject indices that have been overwritten by the ring buffer. + if (state.get().numberOfEndedAssetRaffles > QRAFFLE_MAX_ENDED_ASSET_RAFFLES + && input.indexOfRaffle < state.get().numberOfEndedAssetRaffles - QRAFFLE_MAX_ENDED_ASSET_RAFFLES) + { + output.returnCode = QRAFFLE_INVALID_ASSET_RAFFLE; + return; + } + locals.slot = mod(input.indexOfRaffle, QRAFFLE_MAX_ENDED_ASSET_RAFFLES); + locals.r = state.get().endedAssetRaffles.get(locals.slot); + output.creator = locals.r.creator; + output.epochWinner = locals.r.epochWinner; + output.reservePriceQu = locals.r.reservePriceQu; + output.entryTicketQu = locals.r.entryTicketQu; + output.grossPoolQu = locals.r.grossPoolQu; + output.creatorPaidQu = locals.r.creatorPaidQu; + output.totalTickets = locals.r.totalTickets; + output.numberOfBuyers = locals.r.numberOfBuyers; + output.bundleSize = locals.r.bundleSize; + output.epoch = locals.r.epoch; + output.reserveMet = locals.r.reserveMet; + output.returnCode = QRAFFLE_SUCCESS; + } + + PUBLIC_FUNCTION(getAssetRaffleAnalytics) + { + output.totalAssetRaffleProposalFees = state.get().totalAssetRaffleProposalFees; + output.totalAssetRaffleCreatorPaid = state.get().totalAssetRaffleCreatorPaid; + output.totalAssetRaffleRefunded = state.get().totalAssetRaffleRefunded; + output.numberOfActiveAssetRaffles = state.get().numberOfActiveAssetRaffles; + output.numberOfEndedAssetRaffles = state.get().numberOfEndedAssetRaffles; + output.totalAssetRafflesCreated = state.get().totalAssetRafflesCreated; + output.totalAssetRafflesSucceeded = state.get().totalAssetRafflesSucceeded; + output.totalAssetRafflesFailed = state.get().totalAssetRafflesFailed; + output.returnCode = QRAFFLE_SUCCESS; + } + + REGISTER_USER_FUNCTIONS_AND_PROCEDURES() + { + REGISTER_USER_FUNCTION(getRegisters, 1); + REGISTER_USER_FUNCTION(getAnalytics, 2); + REGISTER_USER_FUNCTION(getActiveProposal, 3); + REGISTER_USER_FUNCTION(getEndedTokenRaffle, 4); + REGISTER_USER_FUNCTION(getEndedQuRaffle, 5); + REGISTER_USER_FUNCTION(getActiveTokenRaffle, 6); + REGISTER_USER_FUNCTION(getEpochRaffleIndexes, 7); + REGISTER_USER_FUNCTION(getQuRaffleEntryAmountPerUser, 8); + REGISTER_USER_FUNCTION(getQuRaffleEntryAverageAmount, 9); + // Asset Raffle view functions + REGISTER_USER_FUNCTION(getActiveAssetRaffle, 10); + REGISTER_USER_FUNCTION(getActiveAssetRaffleBundleItem, 11); + REGISTER_USER_FUNCTION(getActiveAssetRaffleBuyer, 12); + REGISTER_USER_FUNCTION(getEndedAssetRaffle, 13); + REGISTER_USER_FUNCTION(getAssetRaffleAnalytics, 14); + + REGISTER_USER_PROCEDURE(registerInSystem, 1); + REGISTER_USER_PROCEDURE(logoutInSystem, 2); + REGISTER_USER_PROCEDURE(submitEntryAmount, 3); + REGISTER_USER_PROCEDURE(submitProposal, 4); + REGISTER_USER_PROCEDURE(voteInProposal, 5); + REGISTER_USER_PROCEDURE(depositInQuRaffle, 6); + REGISTER_USER_PROCEDURE(depositInTokenRaffle, 7); + REGISTER_USER_PROCEDURE(TransferShareManagementRights, 8); + // Asset Raffle procedures + REGISTER_USER_PROCEDURE(createAssetRaffle, 9); + REGISTER_USER_PROCEDURE(buyAssetRaffleTicket, 10); + REGISTER_USER_PROCEDURE(cancelAssetRaffle, 11); + } + + INITIALIZE() + { + state.mut().qREAmount = QRAFFLE_DEFAULT_QRAFFLE_AMOUNT; + state.mut().charityAddress = ID(_D, _P, _Q, _R, _L, _S, _Z, _S, _S, _C, _X, _I, _Y, _F, _I, _Q, _G, _B, _F, _B, _X, _X, _I, _S, _D, _D, _E, _B, _E, _G, _Q, _N, _W, _N, _T, _Q, _U, _E, _I, _F, _S, _C, _U, _W, _G, _H, _V, _X, _J, _P, _L, _F, _G, _M, _Y, _D); + state.mut().initialRegister1 = ID(_I, _L, _N, _J, _X, _V, _H, _A, _U, _X, _D, _G, _G, _B, _T, _T, _U, _O, _I, _T, _O, _Q, _G, _P, _A, _Y, _U, _C, _F, _T, _N, _C, _P, _X, _D, _K, _O, _C, _P, _U, _O, _C, _D, _O, _T, _P, _U, _W, _X, _B, _I, _G, _R, _V, _Q, _D); + state.mut().initialRegister2 = ID(_L, _S, _D, _A, _A, _C, _L, _X, _X, _G, _I, _P, _G, _G, _L, _S, _O, _C, _L, _M, _V, _A, _Y, _L, _N, _T, _G, _D, _V, _B, _N, _O, _S, _S, _Y, _E, _Q, _D, _R, _K, _X, _D, _Y, _W, _B, _C, _G, _J, _I, _K, _C, _M, _Z, _K, _M, _F); + state.mut().initialRegister3 = ID(_G, _H, _G, _R, _L, _W, _S, _X, _Z, _X, _W, _D, _A, _A, _O, _M, _T, _X, _Q, _Y, _U, _P, _R, _L, _P, _N, _K, _C, _W, _G, _H, _A, _E, _F, _I, _R, _J, _I, _Z, _A, _K, _C, _A, _U, _D, _G, _N, _M, _C, _D, _E, _Q, _R, _O, _Q, _B); + state.mut().initialRegister4 = ID(_E, _U, _O, _N, _A, _Z, _J, _U, _A, _G, _V, _D, _C, _E, _I, _B, _A, _H, _J, _E, _T, _G, _U, _U, _H, _M, _N, _D, _J, _C, _S, _E, _T, _T, _Q, _V, _G, _Y, _F, _H, _M, _D, _P, _X, _T, _A, _L, _D, _Y, _U, _V, _E, _P, _F, _C, _A); + state.mut().initialRegister5 = ID(_S, _L, _C, _J, _C, _C, _U, _X, _G, _K, _N, _V, _A, _D, _F, _B, _E, _A, _Y, _V, _L, _S, _O, _B, _Z, _P, _A, _B, _H, _K, _S, _G, _M, _H, _W, _H, _S, _H, _G, _G, _B, _A, _P, _J, _W, _F, _V, _O, _K, _Z, _J, _P, _F, _L, _X, _D); + state.mut().QXMRIssuer = ID(_Q, _X, _M, _R, _T, _K, _A, _I, _I, _G, _L, _U, _R, _E, _P, _I, _Q, _P, _C, _M, _H, _C, _K, _W, _S, _I, _P, _D, _T, _U, _Y, _F, _C, _F, _N, _Y, _X, _Q, _L, _T, _E, _C, _S, _U, _J, _V, _Y, _E, _M, _M, _D, _E, _L, _B, _M, _D); + state.mut().feeAddress = ID(_H, _H, _R, _L, _C, _Z, _Q, _V, _G, _O, _M, _G, _X, _G, _F, _P, _H, _T, _R, _H, _H, _D, _W, _A, _E, _U, _X, _C, _N, _D, _L, _Z, _S, _Z, _J, _R, _M, _O, _R, _J, _K, _A, _I, _W, _S, _U, _Y, _R, _N, _X, _I, _H, _H, _O, _W, _D); + + state.mut().registers.set(state.get().initialRegister1, 0); + state.mut().registers.set(state.get().initialRegister2, 0); + state.mut().registers.set(state.get().initialRegister3, 0); + state.mut().registers.set(state.get().initialRegister4, 0); + state.mut().registers.set(state.get().initialRegister5, 0); + state.mut().numberOfRegisters = 5; + } + + struct END_EPOCH_locals + { + ProposalInfo proposal; + QuRaffleInfo qraffle; + TokenRaffleInfo tRaffle; + ActiveTokenRaffleInfo acTokenRaffle; + AssetPossessionIterator iter; + Asset QraffleAsset; + id digest, computerDigest, winner, shareholder, baseSeed, raffleSeed; + sint64 idx; + sint64 sharesHeld; + sint64 perShare; + sint64 transferResult; + uint64 sumOfEntryAmountSubmitted, r, winnerRevenue, burnAmount, charityRevenue, shareholderRevenue, registerRevenue, fee, oneShareholderRev; + uint64 tokenPool; + uint64 shareholderPerShareUnit; + uint64 registerPerShareUnit; + uint64 actualShareholderTotal; + uint64 actualRegisterTotal; + uint64 qxmrPerShare; + uint64 qxmrDistributedTotal; + uint64 qxmrContractBalance; + uint32 i, j, winnerIndex; + Logger log; + EmptyTokenRaffleLogger emptyTokenRafflelog; + EndEpochLogger endEpochLog; + RevenueLogger revenueLog; + TokenRaffleLogger tokenRaffleLog; + ProposalLogger proposalLog; + // Asset raffle settlement locals + AssetRaffleInfo arInfo; + AssetRaffleItem arItem; + EndedAssetRaffleInfo arEnded; + AssetRaffleEndedLogger arLog; + uint64 arGross; + uint64 arCreatorPay; + uint64 arBurn; + uint64 arCharity; + uint64 arShareholderRev; + uint64 arRegisterRev; + uint64 arFee; + uint64 arShareholderPerShare; + uint64 arRegisterPerShare; + uint64 arRegisterPerShareActual; + uint64 arRegisterBucket; // accumulated register share across all successful asset raffles + uint64 arRegisterBucketPerReg; // per-register amount distributed after the settlement loop + uint64 arDaoBucketPerRegister; + uint64 arTicketAcc; + uint64 arRefund; // per-buyer refund amount (used in reserve-missed path) + uint32 arI; + uint32 arJ; + uint32 arBuyerSlot; + uint32 arWinnerIndex; + uint32 arEndedIdx; // ring-buffer slot (masked) the settled raffle is written to + uint32 arEndedGlobalIdx; // monotonic global index callers pass to getEndedAssetRaffle + bit arReserveMet; + }; + + END_EPOCH_WITH_LOCALS() + { + // Distribute logout-fee revenue to shareholders. + locals.oneShareholderRev = div(state.get().epochRevenue, NUMBER_OF_COMPUTORS); + if (locals.oneShareholderRev > 0) + { + qpi.distributeDividends(locals.oneShareholderRev); + state.mut().epochRevenue -= locals.oneShareholderRev * NUMBER_OF_COMPUTORS; + } + + // RNG seed: XOR of prevSpectrumDigest and prevComputerDigest, hashed with K12. + // Tick-level inputs are excluded so a computor cannot grind the seed. + locals.digest = qpi.getPrevSpectrumDigest(); + locals.computerDigest = qpi.getPrevComputerDigest(); + locals.baseSeed = qpi.K12(m256i( + locals.digest.u64._0 ^ locals.computerDigest.u64._0, + locals.digest.u64._1 ^ locals.computerDigest.u64._1, + locals.digest.u64._2 ^ locals.computerDigest.u64._2, + locals.digest.u64._3 ^ locals.computerDigest.u64._3)); + + // Asset descriptor reused by the QXMR distribution and per-token-raffle shareholder + // payout loops below; both iterate QRAFFLE_ASSET possessors directly via locals.iter. + locals.QraffleAsset.assetName = QRAFFLE_ASSET_NAME; + locals.QraffleAsset.issuer = NULL_ID; + + if (state.get().numberOfQuRaffleMembers > 0) + { + // Pick winner. + locals.raffleSeed = qpi.K12(m256i(locals.baseSeed.u64._0, locals.baseSeed.u64._1, locals.baseSeed.u64._2, locals.baseSeed.u64._3)); + locals.r = locals.raffleSeed.u64._0; + locals.winnerIndex = (uint32)mod(locals.r, state.get().numberOfQuRaffleMembers * 1ull); + locals.winner = state.get().quRaffleMembers.get(locals.winnerIndex); + + // Calculate fee distributions. + locals.tokenPool = state.get().qREAmount * state.get().numberOfQuRaffleMembers; + locals.burnAmount = div(locals.tokenPool * QRAFFLE_BURN_FEE, 100); + locals.charityRevenue = div(locals.tokenPool * QRAFFLE_CHARITY_FEE, 100); + locals.shareholderRevenue = div(locals.tokenPool * QRAFFLE_SHAREHOLDER_FEE, 100); + locals.registerRevenue = div(locals.tokenPool * QRAFFLE_REGISTER_FEE, 100); + locals.fee = div(locals.tokenPool * QRAFFLE_FEE, 100); + // Round down per-share amounts; winner gets the remainder. + locals.shareholderPerShareUnit = div(locals.shareholderRevenue, NUMBER_OF_COMPUTORS); + locals.actualShareholderTotal = locals.shareholderPerShareUnit * NUMBER_OF_COMPUTORS; + locals.registerPerShareUnit = div(locals.registerRevenue, state.get().numberOfRegisters); + locals.actualRegisterTotal = locals.registerPerShareUnit * state.get().numberOfRegisters; + locals.winnerRevenue = locals.tokenPool - locals.burnAmount - locals.charityRevenue - locals.actualShareholderTotal - locals.actualRegisterTotal - locals.fee; + + locals.revenueLog = RevenueLogger{ + QRAFFLE_CONTRACT_INDEX, + QRAFFLE_revenueDistributed, + locals.burnAmount, + locals.charityRevenue, + locals.actualShareholderTotal, + locals.actualRegisterTotal, + locals.fee, + locals.winnerRevenue, + 0 + }; + LOG_INFO(locals.revenueLog); + + qpi.transfer(locals.winner, locals.winnerRevenue); + qpi.burn(locals.burnAmount); + qpi.transfer(state.get().charityAddress, locals.charityRevenue); + if (locals.shareholderPerShareUnit > 0) + { + qpi.distributeDividends(locals.shareholderPerShareUnit); + } + qpi.transfer(state.get().feeAddress, locals.fee); + + state.mut().totalBurnAmount += locals.burnAmount; + state.mut().totalCharityAmount += locals.charityRevenue; + state.mut().totalShareholderAmount += locals.actualShareholderTotal; + state.mut().totalRegisterAmount += locals.actualRegisterTotal; + state.mut().totalFeeAmount += locals.fee; + state.mut().totalWinnerAmount += locals.winnerRevenue; + if (locals.winnerRevenue > state.get().largestWinnerAmount) + { + state.mut().largestWinnerAmount = locals.winnerRevenue; + } + + if (locals.registerPerShareUnit > 0) + { + locals.idx = state.get().registers.nextElementIndex(NULL_INDEX); + while (locals.idx != NULL_INDEX) + { + qpi.transfer(state.get().registers.key(locals.idx), locals.registerPerShareUnit); + locals.idx = state.get().registers.nextElementIndex(locals.idx); + } + } + + locals.qraffle.epochWinner = locals.winner; + locals.qraffle.receivedAmount = locals.winnerRevenue; + locals.qraffle.entryAmount = state.get().qREAmount; + locals.qraffle.numberOfMembers = state.get().numberOfQuRaffleMembers; + locals.qraffle.winnerIndex = locals.winnerIndex; + state.mut().QuRaffles.set(qpi.epoch(), locals.qraffle); + + locals.endEpochLog = EndEpochLogger{ + QRAFFLE_CONTRACT_INDEX, + QRAFFLE_revenueDistributed, + qpi.epoch(), + state.get().numberOfQuRaffleMembers, + locals.tokenPool, + locals.winnerRevenue, + locals.winnerIndex, + 0 + }; + LOG_INFO(locals.endEpochLog); + } + else + { + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_emptyQuRaffle, 0 }; + LOG_INFO(locals.log); + } + + locals.qxmrPerShare = div(state.get().epochQXMRRevenue, NUMBER_OF_COMPUTORS); + locals.qxmrDistributedTotal = 0; + if (locals.qxmrPerShare > 0) + { + locals.qxmrContractBalance = (uint64)qpi.numberOfPossessedShares(QRAFFLE_QXMR_ASSET_NAME, state.get().QXMRIssuer, SELF, SELF, SELF_INDEX, SELF_INDEX); + locals.iter.begin(locals.QraffleAsset); + while (!locals.iter.reachedEnd()) + { + locals.sharesHeld = locals.iter.numberOfPossessedShares(); + if (locals.sharesHeld > 0) + { + locals.shareholder = locals.iter.possessor(); + locals.perShare = (sint64)(locals.qxmrPerShare * (uint64)locals.sharesHeld); + if ((uint64)locals.perShare <= locals.qxmrContractBalance - locals.qxmrDistributedTotal) + { + locals.transferResult = qpi.transferShareOwnershipAndPossession(QRAFFLE_QXMR_ASSET_NAME, state.get().QXMRIssuer, SELF, SELF, locals.perShare, locals.shareholder); + if (locals.transferResult >= 0) + { + locals.qxmrDistributedTotal += (uint64)locals.perShare; + } + } + } + locals.iter.next(); + } + } + state.mut().epochQXMRRevenue -= locals.qxmrDistributedTotal; + + // Process each active token raffle. + for (locals.i = 0 ; locals.i < state.get().numberOfActiveTokenRaffle; locals.i++) + { + if (state.get().numberOfTokenRaffleMembers.get(locals.i) > 0) + { + locals.raffleSeed = qpi.K12(m256i(locals.baseSeed.u64._0, locals.baseSeed.u64._1, locals.baseSeed.u64._2, locals.baseSeed.u64._3 ^ ((uint64)locals.i + 1ULL))); + locals.r = locals.raffleSeed.u64._0; + locals.winnerIndex = (uint32)mod(locals.r, state.get().numberOfTokenRaffleMembers.get(locals.i) * 1ull); + locals.winner = state.get().tokenRaffleMemberSlots.get(locals.i * QRAFFLE_TOKEN_RAFFLE_SLOT_SIZE + locals.winnerIndex); + + locals.acTokenRaffle = state.get().activeTokenRaffle.get(locals.i); + + locals.tokenPool = locals.acTokenRaffle.entryAmount * state.get().numberOfTokenRaffleMembers.get(locals.i); + locals.burnAmount = div(locals.tokenPool * QRAFFLE_BURN_FEE, 100); + locals.charityRevenue = div(locals.tokenPool * QRAFFLE_CHARITY_FEE, 100); + locals.shareholderRevenue = div(locals.tokenPool * QRAFFLE_SHAREHOLDER_FEE, 100); + locals.registerRevenue = div(locals.tokenPool * QRAFFLE_REGISTER_FEE, 100); + locals.fee = div(locals.tokenPool * QRAFFLE_FEE, 100); + // Round down per-share amounts; winner gets the remainder. + locals.shareholderPerShareUnit = div(locals.shareholderRevenue, NUMBER_OF_COMPUTORS); + locals.actualShareholderTotal = locals.shareholderPerShareUnit * NUMBER_OF_COMPUTORS; + locals.registerPerShareUnit = div(locals.registerRevenue, state.get().numberOfRegisters); + locals.actualRegisterTotal = locals.registerPerShareUnit * state.get().numberOfRegisters; + locals.winnerRevenue = locals.tokenPool - locals.burnAmount - locals.charityRevenue - locals.actualShareholderTotal - locals.actualRegisterTotal - locals.fee; + + // Send winner share. Skip the whole raffle if this fails to prevent partial distribution. + locals.transferResult = qpi.transferShareOwnershipAndPossession(locals.acTokenRaffle.token.assetName, locals.acTokenRaffle.token.issuer, SELF, SELF, locals.winnerRevenue, locals.winner); + if (locals.transferResult < 0) + { + // Winner transfer failed — skip raffle, leave funds for next epoch. + locals.emptyTokenRafflelog = EmptyTokenRaffleLogger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_transferFailed, locals.i, 0 }; + LOG_INFO(locals.emptyTokenRafflelog); + state.mut().numberOfTokenRaffleMembers.set(locals.i, 0); + continue; + } + + // Burn shares (NULL_ID); fall back to charity if burn is rejected. + if (locals.burnAmount > 0) + { + locals.transferResult = qpi.transferShareOwnershipAndPossession(locals.acTokenRaffle.token.assetName, locals.acTokenRaffle.token.issuer, SELF, SELF, locals.burnAmount, NULL_ID); + if (locals.transferResult < 0) + { + qpi.transferShareOwnershipAndPossession(locals.acTokenRaffle.token.assetName, locals.acTokenRaffle.token.issuer, SELF, SELF, locals.burnAmount, state.get().charityAddress); + } + } + if (locals.charityRevenue > 0) + { + qpi.transferShareOwnershipAndPossession(locals.acTokenRaffle.token.assetName, locals.acTokenRaffle.token.issuer, SELF, SELF, locals.charityRevenue, state.get().charityAddress); + } + if (locals.fee > 0) + { + qpi.transferShareOwnershipAndPossession(locals.acTokenRaffle.token.assetName, locals.acTokenRaffle.token.issuer, SELF, SELF, locals.fee, state.get().feeAddress); + } + + // Pay shareholders proportional to possessed shares. + if (locals.shareholderPerShareUnit > 0) + { + locals.iter.begin(locals.QraffleAsset); + while (!locals.iter.reachedEnd()) + { + locals.sharesHeld = locals.iter.numberOfPossessedShares(); + if (locals.sharesHeld > 0) + { + qpi.transferShareOwnershipAndPossession(locals.acTokenRaffle.token.assetName, locals.acTokenRaffle.token.issuer, SELF, SELF, (sint64)(locals.shareholderPerShareUnit * (uint64)locals.sharesHeld), locals.iter.possessor()); + } + locals.iter.next(); + } + } + + if (locals.registerPerShareUnit > 0) + { + locals.idx = state.get().registers.nextElementIndex(NULL_INDEX); + while (locals.idx != NULL_INDEX) + { + qpi.transferShareOwnershipAndPossession(locals.acTokenRaffle.token.assetName, locals.acTokenRaffle.token.issuer, SELF, SELF, (sint64)locals.registerPerShareUnit, state.get().registers.key(locals.idx)); + locals.idx = state.get().registers.nextElementIndex(locals.idx); + } + } + + locals.tRaffle.epochWinner = locals.winner; + locals.tRaffle.token.assetName = locals.acTokenRaffle.token.assetName; + locals.tRaffle.token.issuer = locals.acTokenRaffle.token.issuer; + locals.tRaffle.entryAmount = locals.acTokenRaffle.entryAmount; + locals.tRaffle.numberOfMembers = state.get().numberOfTokenRaffleMembers.get(locals.i); + locals.tRaffle.winnerIndex = locals.winnerIndex; + locals.tRaffle.epoch = qpi.epoch(); + state.mut().tokenRaffle.set(state.get().numberOfEndedTokenRaffle, locals.tRaffle); + + locals.tokenRaffleLog = TokenRaffleLogger{ + QRAFFLE_CONTRACT_INDEX, + QRAFFLE_tokenRaffleEnded, + state.mut().numberOfEndedTokenRaffle++, + locals.acTokenRaffle.token.assetName, + state.get().numberOfTokenRaffleMembers.get(locals.i), + locals.acTokenRaffle.entryAmount, + locals.winnerIndex, + locals.winnerRevenue, + 0 + }; + LOG_INFO(locals.tokenRaffleLog); + + state.mut().numberOfTokenRaffleMembers.set(locals.i, 0); + } + else + { + locals.emptyTokenRafflelog = EmptyTokenRaffleLogger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_emptyTokenRaffle, locals.i, 0 }; + LOG_INFO(locals.emptyTokenRafflelog); + } + } + + // ── Asset Raffle settlement ──────────────────────────────────────────── + locals.arRegisterBucket = 0; + for (locals.arI = 0; locals.arI < state.get().numberOfActiveAssetRaffles; locals.arI++) + { + locals.arInfo = state.get().activeAssetRaffles.get(locals.arI); + locals.arGross = locals.arInfo.totalTicketsPaidQu; + + // Reserve test: gross * 80 >= reservePriceQu * 100 + locals.arReserveMet = (locals.arInfo.totalTickets > 0) + && (locals.arGross * 80ull >= locals.arInfo.reservePriceQu * 100ull); + + if (locals.arReserveMet) + { + // Weighted winner selection by ticket count. + locals.raffleSeed = qpi.K12(m256i( + locals.baseSeed.u64._0, locals.baseSeed.u64._1, + locals.baseSeed.u64._2, + locals.baseSeed.u64._3 ^ (0xA55E7000ULL + (uint64)locals.arI))); + locals.r = locals.raffleSeed.u64._0; + locals.r = mod(locals.r, (uint64)locals.arInfo.totalTickets); + locals.arTicketAcc = 0; + locals.arWinnerIndex = 0; + locals.winner = NULL_ID; + for (locals.arJ = 0; locals.arJ < locals.arInfo.numberOfBuyers; locals.arJ++) + { + locals.arBuyerSlot = locals.arI * QRAFFLE_MAX_ASSET_TICKET_BUYERS + locals.arJ; + locals.arTicketAcc += (uint64)state.get().activeAssetRaffleBuyerTickets.get(locals.arBuyerSlot); + if (locals.r < locals.arTicketAcc) + { + locals.arWinnerIndex = locals.arJ; + locals.winner = state.get().activeAssetRaffleBuyers.get(locals.arBuyerSlot); + break; + } + } + + // Transfer each bundle item to the winner. If an item fails, log and continue; + // we cannot pull assets back from a user's wallet, so the winner keeps whatever + // was delivered and the remaining escrowed items stay in the contract. + for (locals.arJ = 0; locals.arJ < locals.arInfo.bundleSize; locals.arJ++) + { + locals.arItem = state.get().activeAssetRaffleItems.get( + locals.arI * QRAFFLE_MAX_ASSETS_PER_BUNDLE + locals.arJ); + locals.transferResult = qpi.transferShareOwnershipAndPossession( + locals.arItem.asset.assetName, locals.arItem.asset.issuer, + SELF, SELF, + locals.arItem.numberOfShares, locals.winner); + if (locals.transferResult < 0) + { + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_assetRaffleBundleDeliveryFailed, 0 }; + LOG_INFO(locals.log); + } + } + + // Qu pool distribution: 80% to creator, 20% to fee buckets. + // Always executed when reserve is met, regardless of per-item delivery outcome. + // (Assets already delivered to winner; we cannot recall them from a user's wallet.) + locals.arBurn = div(locals.arGross * (uint64)QRAFFLE_BURN_FEE, 100ull); + locals.arCharity = div(locals.arGross * (uint64)QRAFFLE_CHARITY_FEE, 100ull); + locals.arShareholderRev = div(locals.arGross * (uint64)QRAFFLE_SHAREHOLDER_FEE, 100ull); + locals.arRegisterRev = div(locals.arGross * (uint64)QRAFFLE_REGISTER_FEE, 100ull); + locals.arFee = div(locals.arGross * (uint64)QRAFFLE_FEE, 100ull); + // Round down per-share amounts; all rounding dust goes to creator. + locals.arShareholderPerShare = div(locals.arShareholderRev, (uint64)NUMBER_OF_COMPUTORS); + locals.arRegisterPerShare = (state.get().numberOfRegisters > 0) + ? div(locals.arRegisterRev, (uint64)state.get().numberOfRegisters) + : 0; + locals.arRegisterPerShareActual = locals.arRegisterPerShare * (uint64)state.get().numberOfRegisters; + // Creator gets gross minus all deductions; rounding dust stays with creator. + locals.arCreatorPay = locals.arGross + - locals.arBurn + - locals.arCharity + - (locals.arShareholderPerShare * (uint64)NUMBER_OF_COMPUTORS) + - locals.arRegisterPerShareActual + - locals.arFee; + + qpi.transfer(locals.arInfo.creator, locals.arCreatorPay); + qpi.burn(locals.arBurn); + qpi.transfer(state.get().charityAddress, locals.arCharity); + if (locals.arShareholderPerShare > 0) + { + qpi.distributeDividends(locals.arShareholderPerShare); + } + qpi.transfer(state.get().feeAddress, locals.arFee); + // Accumulate register share; distributed in a single pass after the loop. + locals.arRegisterBucket += locals.arRegisterPerShareActual; + + state.mut().totalAssetRaffleCreatorPaid += locals.arCreatorPay; + state.mut().totalAssetRafflesSucceeded++; + } + + if (!locals.arReserveMet) + { + // Reserve not met: refund all Qu to buyers and return bundle to creator. + for (locals.arJ = 0; locals.arJ < locals.arInfo.numberOfBuyers; locals.arJ++) + { + locals.arBuyerSlot = locals.arI * QRAFFLE_MAX_ASSET_TICKET_BUYERS + locals.arJ; + locals.arRefund = (uint64)state.get().activeAssetRaffleBuyerTickets.get(locals.arBuyerSlot) * locals.arInfo.entryTicketQu; + qpi.transfer(state.get().activeAssetRaffleBuyers.get(locals.arBuyerSlot), locals.arRefund); + state.mut().totalAssetRaffleRefunded += locals.arRefund; + } + for (locals.arJ = 0; locals.arJ < locals.arInfo.bundleSize; locals.arJ++) + { + locals.arItem = state.get().activeAssetRaffleItems.get( + locals.arI * QRAFFLE_MAX_ASSETS_PER_BUNDLE + locals.arJ); + qpi.transferShareOwnershipAndPossession( + locals.arItem.asset.assetName, locals.arItem.asset.issuer, + SELF, SELF, + locals.arItem.numberOfShares, locals.arInfo.creator); + } + locals.arCreatorPay = 0; + locals.winner = NULL_ID; + locals.arWinnerIndex = 0; + state.mut().totalAssetRafflesFailed++; + } + + // Write to ended ring buffer. + // arEndedGlobalIdx is the monotonic logical index (pre-increment); this is the + // value callers pass to getEndedAssetRaffle. arEndedIdx is its ring-masked slot. + locals.arEndedGlobalIdx = state.get().numberOfEndedAssetRaffles; + locals.arEndedIdx = mod(locals.arEndedGlobalIdx, QRAFFLE_MAX_ENDED_ASSET_RAFFLES); + locals.arEnded.creator = locals.arInfo.creator; + locals.arEnded.epochWinner = locals.winner; + locals.arEnded.reservePriceQu = locals.arInfo.reservePriceQu; + locals.arEnded.entryTicketQu = locals.arInfo.entryTicketQu; + locals.arEnded.grossPoolQu = locals.arGross; + locals.arEnded.creatorPaidQu = locals.arCreatorPay; + locals.arEnded.totalTickets = locals.arInfo.totalTickets; + locals.arEnded.numberOfBuyers = locals.arInfo.numberOfBuyers; + locals.arEnded.bundleSize = locals.arInfo.bundleSize; + locals.arEnded.epoch = locals.arInfo.epoch; + locals.arEnded.reserveMet = locals.arReserveMet ? 1u : 0u; + state.mut().endedAssetRaffles.set(locals.arEndedIdx, locals.arEnded); + state.mut().numberOfEndedAssetRaffles++; + + locals.arLog = AssetRaffleEndedLogger{ + QRAFFLE_CONTRACT_INDEX, + locals.arReserveMet ? (uint32)QRAFFLE_assetRaffleSucceeded : (uint32)QRAFFLE_assetRaffleRefunded, + locals.arEndedGlobalIdx, + locals.arInfo.creator, + locals.winner, + locals.arGross, + locals.arCreatorPay, + locals.arReserveMet ? (uint8)1 : (uint8)0, + 0 + }; + LOG_INFO(locals.arLog); + } + + // Distribute accumulated register share from all successful asset raffles in one O(R) pass. + // Any integer-division remainder (up to numberOfRegisters-1 Qu) is folded into the DAO + // bucket below so the dust either pays out this same epoch via the DAO distribution that + // follows, or carries forward — never silently leaks into untracked contract balance. + if (locals.arRegisterBucket > 0 && state.get().numberOfRegisters > 0) + { + locals.arRegisterBucketPerReg = div(locals.arRegisterBucket, (uint64)state.get().numberOfRegisters); + if (locals.arRegisterBucketPerReg > 0) + { + locals.idx = state.get().registers.nextElementIndex(NULL_INDEX); + while (locals.idx != NULL_INDEX) + { + qpi.transfer(state.get().registers.key(locals.idx), locals.arRegisterBucketPerReg); + locals.idx = state.get().registers.nextElementIndex(locals.idx); + } + } + state.mut().epochAssetRaffleDaoBucket += locals.arRegisterBucket - locals.arRegisterBucketPerReg * (uint64)state.get().numberOfRegisters; + } + + // Distribute DAO proposal-fee bucket evenly to registers. + // Subtract only what was actually paid out so the integer-division remainder carries + // forward to the next epoch instead of being silently dropped into contract balance. + if (state.get().epochAssetRaffleDaoBucket > 0 && state.get().numberOfRegisters > 0) + { + locals.arDaoBucketPerRegister = div(state.get().epochAssetRaffleDaoBucket, (uint64)state.get().numberOfRegisters); + if (locals.arDaoBucketPerRegister > 0) + { + locals.idx = state.get().registers.nextElementIndex(NULL_INDEX); + while (locals.idx != NULL_INDEX) + { + qpi.transfer(state.get().registers.key(locals.idx), locals.arDaoBucketPerRegister); + locals.idx = state.get().registers.nextElementIndex(locals.idx); + } + } + state.mut().epochAssetRaffleDaoBucket -= locals.arDaoBucketPerRegister * (uint64)state.get().numberOfRegisters; + } + + // Reset asset raffle per-epoch state. + state.mut().numberOfActiveAssetRaffles = 0; + state.mut().assetRaffleParticipation.reset(); + state.mut().assetRaffleBuyerSlotIndex.reset(); + state.mut().assetRafflesPerCreator.reset(); + + // Calculate new qREAmount and log + locals.log = Logger{ QRAFFLE_CONTRACT_INDEX, QRAFFLE_revenueDistributed, 0 }; + LOG_INFO(locals.log); + + locals.sumOfEntryAmountSubmitted = 0; + locals.idx = state.get().quRaffleEntryAmount.nextElementIndex(NULL_INDEX); + while (locals.idx != NULL_INDEX) + { + locals.sumOfEntryAmountSubmitted += state.get().quRaffleEntryAmount.value(locals.idx); + locals.idx = state.get().quRaffleEntryAmount.nextElementIndex(locals.idx); + } + if (state.get().numberOfEntryAmountSubmitted > 0) + { + state.mut().qREAmount = div(locals.sumOfEntryAmountSubmitted, state.get().numberOfEntryAmountSubmitted); + } + else + { + state.mut().qREAmount = QRAFFLE_DEFAULT_QRAFFLE_AMOUNT; + } + + state.mut().numberOfActiveTokenRaffle = 0; + + // Process approved proposals and create new token raffles + for (locals.i = 0 ; locals.i < state.get().numberOfProposals; locals.i++) + { + locals.proposal = state.get().proposals.get(locals.i); + + // Log proposal processing with detailed information + locals.proposalLog = ProposalLogger{ + QRAFFLE_CONTRACT_INDEX, + QRAFFLE_proposalSubmitted, + locals.i, + locals.proposal.proposer, + locals.proposal.nYes, + locals.proposal.nNo, + locals.proposal.token.assetName, + locals.proposal.entryAmount, + 0 + }; + LOG_INFO(locals.proposalLog); + + if (locals.proposal.nYes > locals.proposal.nNo) + { + locals.acTokenRaffle.token.assetName = locals.proposal.token.assetName; + locals.acTokenRaffle.token.issuer = locals.proposal.token.issuer; + locals.acTokenRaffle.entryAmount = locals.proposal.entryAmount; + + state.mut().activeTokenRaffle.set(state.mut().numberOfActiveTokenRaffle++, locals.acTokenRaffle); + } + } + + // Record DAO member count for this epoch before resetting per-epoch state. + state.mut().daoMemberCount.set(qpi.epoch(), state.get().numberOfRegisters); + + state.mut().numberOfVotedInProposal.setAll(0); + state.mut().tokenRaffleParticipation.reset(); + state.mut().proposalsPerProposer.reset(); + state.mut().quRaffleEntryAmount.reset(); + state.mut().voteParticipation.reset(); + state.mut().voteValues.reset(); + state.mut().numberOfEntryAmountSubmitted = 0; + state.mut().numberOfProposals = 0; + state.mut().numberOfQuRaffleMembers = 0; + state.mut().quRaffleMemberSet.reset(); + if (state.get().registers.needsCleanup()) { state.mut().registers.cleanup(); } + } + + MIGRATE() + { + copyMemory(state.mut().registers, oldState.registers); + copyMemory(state.mut().voteParticipation, oldState.voteParticipation); + copyMemory(state.mut().voteValues, oldState.voteValues); + copyMemory(state.mut().quRaffleMemberSet, oldState.quRaffleMemberSet); + copyMemory(state.mut().tokenRaffleParticipation, oldState.tokenRaffleParticipation); + copyMemory(state.mut().quRaffleEntryAmount, oldState.quRaffleEntryAmount); + copyMemory(state.mut().proposalsPerProposer, oldState.proposalsPerProposer); + + state.mut().proposals = oldState.proposals; + state.mut().numberOfVotedInProposal = oldState.numberOfVotedInProposal; + state.mut().quRaffleMembers = oldState.quRaffleMembers; + state.mut().activeTokenRaffle = oldState.activeTokenRaffle; + state.mut().tokenRaffleMemberSlots = oldState.tokenRaffleMemberSlots; + state.mut().numberOfTokenRaffleMembers = oldState.numberOfTokenRaffleMembers; + state.mut().QuRaffles = oldState.QuRaffles; + state.mut().tokenRaffle = oldState.tokenRaffle; + state.mut().daoMemberCount = oldState.daoMemberCount; + + state.mut().initialRegister1 = oldState.initialRegister1; + state.mut().initialRegister2 = oldState.initialRegister2; + state.mut().initialRegister3 = oldState.initialRegister3; + state.mut().initialRegister4 = oldState.initialRegister4; + state.mut().initialRegister5 = oldState.initialRegister5; + state.mut().charityAddress = oldState.charityAddress; + state.mut().feeAddress = oldState.feeAddress; + state.mut().QXMRIssuer = oldState.QXMRIssuer; + state.mut().epochRevenue = oldState.epochRevenue; + state.mut().epochQXMRRevenue = oldState.epochQXMRRevenue; + state.mut().qREAmount = oldState.qREAmount; + state.mut().totalBurnAmount = oldState.totalBurnAmount; + state.mut().totalCharityAmount = oldState.totalCharityAmount; + state.mut().totalShareholderAmount = oldState.totalShareholderAmount; + state.mut().totalRegisterAmount = oldState.totalRegisterAmount; + state.mut().totalFeeAmount = oldState.totalFeeAmount; + state.mut().totalWinnerAmount = oldState.totalWinnerAmount; + state.mut().largestWinnerAmount = oldState.largestWinnerAmount; + state.mut().numberOfRegisters = oldState.numberOfRegisters; + state.mut().numberOfQuRaffleMembers = oldState.numberOfQuRaffleMembers; + state.mut().numberOfEntryAmountSubmitted = oldState.numberOfEntryAmountSubmitted; + state.mut().numberOfProposals = oldState.numberOfProposals; + state.mut().numberOfActiveTokenRaffle = oldState.numberOfActiveTokenRaffle; + state.mut().numberOfEndedTokenRaffle = oldState.numberOfEndedTokenRaffle; + } + + PRE_ACQUIRE_SHARES() + { + // Accept all incoming share management transfers for free. + // Service fees are collected in user procedures (depositInTokenRaffle, TransferShareManagementRights). + output.requestedFee = 0; + output.allowTransfer = true; + } + + POST_ACQUIRE_SHARES() + { + // Credit any received fee to epochRevenue. + if (input.receivedFee > 0) + { + state.mut().epochRevenue += (uint64)input.receivedFee; + } + } +}; diff --git a/src/extensions/overload.h b/src/extensions/overload.h index 1bf2747d..0e9f6c3d 100644 --- a/src/extensions/overload.h +++ b/src/extensions/overload.h @@ -2,6 +2,8 @@ ////////////////// Extensions \\\\\\\\\\\\ +#include + #if defined(_WIN32) #include #include @@ -264,13 +266,40 @@ inline void* qVirtualCommit(void* address, const unsigned long long size) { return VirtualAlloc(address, (SIZE_T)size, MEM_COMMIT, PAGE_READWRITE); } +inline unsigned long long qGetPageSize() { + SYSTEM_INFO systemInfo; + GetSystemInfo(&systemInfo); + return (unsigned long long)systemInfo.dwPageSize; +} + inline bool qVirtualFreeAndRecommit(void* address, const unsigned long long size) { - VirtualFree(address, (SIZE_T)size, MEM_DECOMMIT); - bool commitMem = commitMemMap[(unsigned long long)address]; - if (!commitMem) { - return true; - } - return VirtualAlloc(address, (SIZE_T)size, MEM_COMMIT, PAGE_READWRITE) != address; + static const unsigned long long pageSize = qGetPageSize(); + const bool commitMem = commitMemMap[(unsigned long long)address]; + + // MEM_DECOMMIT rounds the length up to a page, so decommitting a non-page-multiple size would + // also drop whatever region shares the last page; zero that tail in place instead. + const unsigned long long decommitSize = size & ~(pageSize - 1); + if (decommitSize) + { + VirtualFree(address, (SIZE_T)decommitSize, MEM_DECOMMIT); + if (commitMem && VirtualAlloc(address, (SIZE_T)decommitSize, MEM_COMMIT, PAGE_READWRITE) != address) + { + return false; + } + } + + const unsigned long long tailSize = size - decommitSize; + if (tailSize) + { + char* tail = (char*)address + decommitSize; + if (!VirtualAlloc(tail, (SIZE_T)tailSize, MEM_COMMIT, PAGE_READWRITE)) + { + return false; + } + memset(tail, 0, (size_t)tailSize); + } + + return true; } // Emulate demand-zero overcommit by committing Windows pages on first access. @@ -387,9 +416,30 @@ inline void* qVirtualCommit(void* address, const unsigned long long size) { } inline bool qVirtualFreeAndRecommit(void* address, const unsigned long long size) { - bool commitMem = commitMemMap[(unsigned long long)address]; - int prot = commitMem ? (PROT_READ | PROT_WRITE) : PROT_NONE; - return mmap(address, size, prot, MAP_PRIVATE | MAP_ANONYMOUS | MAP_FIXED, -1, 0) == address; + static const unsigned long long pageSize = (unsigned long long)sysconf(_SC_PAGESIZE); + const bool commitMem = commitMemMap[(unsigned long long)address]; + const int prot = commitMem ? (PROT_READ | PROT_WRITE) : PROT_NONE; + + // MAP_FIXED rounds the length up to a page, so remapping a non-page-multiple size would also + // wipe whatever region shares the last page; zero that tail in place instead of remapping it. + const unsigned long long remapSize = size & ~(pageSize - 1); + if (remapSize && mmap(address, remapSize, prot, MAP_PRIVATE | MAP_ANONYMOUS | MAP_FIXED, -1, 0) != address) + { + return false; + } + + const unsigned long long tailSize = size - remapSize; + if (tailSize) + { + char* tail = (char*)address + remapSize; + if (mprotect(tail, tailSize, PROT_READ | PROT_WRITE) != 0) + { + return false; + } + memset(tail, 0, tailSize); + } + + return true; } #endif @@ -1604,4 +1654,5 @@ struct Overload { } }; +void logToConsole_1(const CHAR16* message); #define logToConsole logToConsole_1 diff --git a/src/extensions/test_invalid_solution.h b/src/extensions/test_invalid_solution.h index 7ae66bf5..dd248400 100644 --- a/src/extensions/test_invalid_solution.h +++ b/src/extensions/test_invalid_solution.h @@ -2,6 +2,7 @@ #include "platform/m256.h" #include "mining/mining.h" +#include "mining/ant_colony/ant_colony_bpp9000.h" #include "mining/score_common.h" #include "spectrum/special_entities.h" #include "network_core/peers.h" @@ -127,4 +128,250 @@ inline bool broadcastRandom(const m256i& currentMiningSeed, unsigned int txTick, return true; } + +// Ant-colony injector. The node mines against its own colony, so this drives the whole inputType-12 +// path on one machine: broadcast, pre-score, publish, commit, deposit, ranking. Each mode aims at one +// branch of the accept rules, so every ValidityResult is reachable without a second node. +// Same value as MIN_MINING_SOLUTIONS_PUBLICATION_OFFSET, which is defined after this header is included. +static constexpr unsigned int ANT_INJECT_PUBLICATION_OFFSET = 3; + +enum class AntInjectMode +{ + Valid, // honest solution: accepted, deposit refunded, ranked + BadClaim, // right nonce, wrong claimedScore: committed and folded, deposit kept + NonCanonical, // nonce[1] out of range + WrongTree, // parent belonging to another identity + Stale, // anchor older than the publish window + FutureParent, // parent ref into the current tick + LeParent, // child that does not beat its parent +}; + +namespace detail +{ + +inline void signAndBroadcastAntSolution(unsigned int computorIdx, + const SolutionRef& parentRef, + unsigned int anchorTick, + unsigned int claimedScore, + const m256i& nonce, + unsigned int txTick) +{ + AntColonyMiningSolutionTransaction payload; + setMem(&payload, sizeof(payload), 0); + payload.sourcePublicKey = computorPublicKeys[computorIdx]; + payload.destinationPublicKey = m256i::zero(); + payload.amount = AntColonyMiningSolutionTransaction::minAmount(); + payload.tick = txTick; + payload.inputType = AntColonyMiningSolutionTransaction::transactionType(); + payload.inputSize = AntColonyMiningSolutionTransaction::minInputSize(); + payload.parentTick = parentRef.tick; + payload.parentSolutionIndexInTick = parentRef.solutionIndexInTick; + payload.anchorTick = anchorTick; + payload.claimedScore = claimedScore; + payload.nonce = nonce; + + unsigned char digest[32]; + KangarooTwelve(&payload, + sizeof(Transaction) + AntColonyMiningSolutionTransaction::minInputSize(), + digest, + sizeof(digest)); + sign(computorSubseeds[computorIdx].m256i_u8, + computorPublicKeys[computorIdx].m256i_u8, + digest, + payload.signature); + + enqueueResponse(NULL, sizeof(payload), BROADCAST_TRANSACTION, 0, &payload); +} + +} // namespace detail + +// ColonyT and ScoreT stay template parameters so this header keeps compiling where it is included, +// which is before qubic.cpp declares gAntColony and score. +template +// The publish tick is read from system.tick when the transaction is signed, not when the walk +// starts: a walk takes seconds and the chain may have moved dozens of ticks meanwhile. +inline bool broadcastAntSolution(ColonyT& colony, + ScoreT& scoreFn, + unsigned long long processorNumber, + unsigned int anchorTick, + AntInjectMode mode, + unsigned int attempts = 8) +{ + if (computorSeedsCount == 0) + { + return false; + } + + // One identity, so successive solutions chain into a deepening tree rather than 676 depth-1 stubs. + // Seat 0's root network fails to self-clock under this pool, so mine seat 1. + const unsigned int computorIdx = 1; + const m256i& minerKey = computorPublicKeys[computorIdx]; + + // Extend this identity's best node when it has one, otherwise start its tree from the root. + SolutionRef parentRef = ROOT_REF; + const AntSolutionRecord* parentRec = nullptr; + unsigned int parentScore = 0xFFFFFFFFU; + for (unsigned int i = 0; i < colony.solutionCount(); i++) + { + const AntSolutionRecord* rec = colony.recordAt((long long)i); + if (rec != nullptr && rec->pubkey == minerKey && rec->score < parentScore) + { + parentScore = rec->score; + parentRef = rec->selfRef; + parentRec = rec; + } + } + + if (mode == AntInjectMode::WrongTree) + { + // Any node owned by somebody else. Without one the rule is not reachable yet. + parentRec = nullptr; + for (unsigned int i = 0; i < colony.solutionCount(); i++) + { + const AntSolutionRecord* rec = colony.recordAt((long long)i); + if (rec != nullptr && !(rec->pubkey == minerKey)) + { + parentRef = rec->selfRef; + parentRec = rec; + break; + } + } + if (parentRec == nullptr) + { + return false; + } + } + else if (mode == AntInjectMode::FutureParent) + { + parentRef.tick = system.tick + ANT_INJECT_PUBLICATION_OFFSET; + parentRef.solutionIndexInTick = 0; + parentRec = nullptr; + } + + unsigned int usedAnchorTick = anchorTick; + if (mode == AntInjectMode::Stale) + { + // Far enough back that the ring no longer holds it. + usedAnchorTick = (anchorTick > ANT_PUBLISH_WINDOW_TICKS + 1) + ? (anchorTick - ANT_PUBLISH_WINDOW_TICKS - 1) + : 0; + } + + // Anchors are only recorded for non-empty ticks, so an idle testnet has none and the injector + // would never fire. Walk back a short window, and if nothing is there put a transfer on chain to + // make this tick non-empty - that seeds the ring for the next attempt. + m256i anchorDigest = m256i::zero(); + if (mode != AntInjectMode::Stale) + { + bool haveAnchor = false; + for (unsigned int back = 0; back < 16 && back < usedAnchorTick; back++) + { + if (colony.getAnchorDigest(usedAnchorTick - back, anchorDigest)) + { + usedAnchorTick -= back; + haveAnchor = true; + break; + } + } + if (!haveAnchor) + { + detail::broadcastTransfer(computorIdx, computorPublicKeys[computorIdx], 1, + system.tick + ANT_INJECT_PUBLICATION_OFFSET); + return false; + } + } + + // The parent's network is what the child inherits; the scorer derives the root itself when the + // parent is ROOT_REF. + typename ColonyT::Ann parentAnn; + typename ColonyT::Ann childAnn; + const typename ColonyT::Ann* parentAnnPtr = nullptr; + if (parentRec != nullptr) + { + if (!colony.annOfNonRoot(*parentRec, parentAnn)) + { + return false; + } + parentAnnPtr = &parentAnn; + } + + m256i nonce; + nonce.setRandomValue(); + nonce.m256i_u8[0] = (unsigned char)score_engine::AlgoType::Bpp9000; + + if (mode == AntInjectMode::NonCanonical) + { + nonce.m256i_u8[1] = 0; // L below range; the scorer refuses before walking + detail::signAndBroadcastAntSolution(computorIdx, parentRef, usedAnchorTick, 0, nonce, + system.tick + ANT_INJECT_PUBLICATION_OFFSET); + return true; + } + + unsigned int childScore = score_engine::INVALID_SCORE_VALUE; + bool found = false; + for (unsigned int attempt = 0; attempt < attempts && !found; attempt++) + { + nonce.setRandomValue(); + nonce.m256i_u8[0] = (unsigned char)score_engine::AlgoType::Bpp9000; + nonce.m256i_u8[1] = (unsigned char)(1 + (nonce.m256i_u8[1] % score_engine::MAX_LUT_ENTRIES_PER_STEP)); + nonce.m256i_u8[2] = 0; // no explore steps: pure descent gives the best odds of beating the parent + + const unsigned long long walkStart = __rdtsc(); + childScore = scoreFn.computeAntChildScore(processorNumber, parentAnnPtr, minerKey, nonce, + anchorDigest, childAnn); + { + CHAR16 line[192]; + setText(line, L"ANT-INJECT attempt score="); + appendNumber(line, childScore, FALSE); + appendText(line, L" parentScore="); + appendNumber(line, parentScore, FALSE); + appendText(line, L" threshold="); + appendNumber(line, colony.errorThreshold(), FALSE); + appendText(line, L" ms="); + appendNumber(line, (__rdtsc() - walkStart) / (frequency / 1000), FALSE); + logToConsole(line); + } + if (childScore == score_engine::INVALID_SCORE_VALUE) + { + continue; + } + + const bool beatsParent = (childScore < parentScore); + const bool clearsThreshold = (childScore <= colony.errorThreshold()); + // Every walk beats the root's WORST_SCORE, so LeParent is unreachable until the identity has a real parent: seed one first. + found = (mode == AntInjectMode::LeParent && parentRec != nullptr) + ? (clearsThreshold && !beatsParent) + : (beatsParent && clearsThreshold); + } + + if (!found) + { + return false; + } + + // BadClaim keeps the honest nonce so the node's recompute succeeds and then disagrees, which is + // what forfeits the deposit. + const unsigned int claimedScore = + (mode == AntInjectMode::BadClaim) ? (childScore + 1) : childScore; + + const unsigned int publishTick = system.tick + ANT_INJECT_PUBLICATION_OFFSET; + detail::signAndBroadcastAntSolution(computorIdx, parentRef, usedAnchorTick, claimedScore, nonce, publishTick); + + CHAR16 line[192]; + setText(line, L"ANT-INJECT published score="); + appendNumber(line, childScore, FALSE); + appendText(line, L" claimed="); + appendNumber(line, claimedScore, FALSE); + appendText(line, L" parent="); + appendNumber(line, parentRef.tick, FALSE); + appendText(line, L"/"); + appendNumber(line, parentRef.solutionIndexInTick, FALSE); + appendText(line, L" anchor="); + appendNumber(line, usedAnchorTick, FALSE); + appendText(line, L" for tick "); + appendNumber(line, publishTick, FALSE); + logToConsole(line); + return true; +} + } // namespace TestInvalidSolution diff --git a/src/extensions/tick_fork_rollback.h b/src/extensions/tick_fork_rollback.h index e6febd5a..c443f1d6 100644 --- a/src/extensions/tick_fork_rollback.h +++ b/src/extensions/tick_fork_rollback.h @@ -58,6 +58,9 @@ inline long gForkRssBeforeKb = 0; // parent RSS just before fork namespace tickFork { + // Trusting a claimed score is only safe where a tick can be undone. + inline constexpr bool gRollbackAvailable = true; + inline std::atomic gForkRequest{ false }; // tickProcessor -> BSP: fork now inline std::atomic gChildPid{ -2 }; // BSP -> tickProcessor: child pid (>=0) / -1 fail / -2 idle inline int gPipe[2] = { -1, -1 }; // verdict channel: parent writes [1], child reads [0] @@ -81,7 +84,7 @@ namespace tickFork gWinState.store((int)state, std::memory_order_release); } - // Only ticks carrying a mining-solution tx can mismatch quorum. + // Only ticks carrying a mining-solution tx can mismatch quorum. Ant counts too: AUX commits it on the claimed score. inline bool tickHasSolution(unsigned int tick) { TickData tickDataCopy; @@ -101,7 +104,7 @@ namespace tickFork Transaction* transaction = ts.tickTransactions(offsets[i]); if (!transaction->checkValidity()) continue; - if (MiningSolutionTransaction::isSolutionTransaction(transaction)) + if (MiningSolutionTransaction::isSolutionTransaction(transaction) || AntColonyMiningSolutionTransaction::isSolutionTransaction(transaction)) return true; } return false; @@ -521,6 +524,9 @@ namespace tickFork #include namespace tickFork { + // No checkpoint on this build, so every optimistic shortcut stays off. + inline constexpr bool gRollbackAvailable = false; + inline std::atomic gIsForkChild{ false }; inline std::atomic gForkRequest{ false }; inline void maybeForkBeforeTick(unsigned long long) {} diff --git a/src/logging/logging.h b/src/logging/logging.h index add088cd..ddca7106 100644 --- a/src/logging/logging.h +++ b/src/logging/logging.h @@ -69,6 +69,7 @@ struct Peer; #define CUSTOM_MESSAGE_OP_END_DISTRIBUTE_DIVIDENDS 6217575821008457285ULL //END_DDIV #define CUSTOM_MESSAGE_OP_START_EPOCH 4850183582582395987ULL // STA_EPOC #define CUSTOM_MESSAGE_OP_END_EPOCH 4850183582582591045ULL //END_EPOC +#define CUSTOM_MESSAGE_ANT_SOLUTION 6146374810954124865ULL // ANT_SOLU /* * STRUCTS FOR LOGGING */ @@ -194,6 +195,23 @@ struct DummyCustomMessage char _terminator; // Only data before "_terminator" are logged }; +// On-chain outcome of one ant-colony solution transaction (accepted or rejected), +// identified by its dedup key (sourcePublicKey, parentRef, nonce). +struct AntSolutionLogMessage +{ + unsigned long long _type; // CUSTOM_MESSAGE_ANT_SOLUTION + m256i sourcePublicKey; + m256i nonce; + unsigned int parentTick; + unsigned int parentSolutionIndexInTick; + unsigned int anchorTick; + unsigned int score; + // ValidityResult of the commit + unsigned int result; + + char _terminator; // Only data before "_terminator" are logged +}; + struct Burning { m256i sourcePublicKey; diff --git a/src/mining/ant_colony/ant_colony.h b/src/mining/ant_colony/ant_colony.h new file mode 100644 index 00000000..4f8af706 --- /dev/null +++ b/src/mining/ant_colony/ant_colony.h @@ -0,0 +1,1742 @@ +#pragma once + +#include "platform/assert.h" +#include "platform/concurrency.h" +#include "platform/m256.h" +#include "platform/memory.h" +#include "platform/memory_util.h" +#include "kangaroo_twelve.h" +#include "platform/file_io.h" +#include "platform/debugging.h" +#include "contract_core/pre_qpi_def.h" +#include "qpi/qpi.h" +#include "qpi/impl/qpi_hash_map_impl.h" +#include "public_settings.h" +#include "mining/mining.h" +#include "mining/trit_pack.h" + +// The keyed structures are twice the population they index. +static constexpr unsigned long long ANT_DEDUP_SIZE = 2ULL * ANT_MAX_NODES_PER_EPOCH; +// At most one key per record, and records are capped, so load stays at or below 50% and set() cannot fail. +static constexpr unsigned long long ANT_CHILD_HEAD_BY_PARENT_SIZE = 2ULL * ANT_MAX_NODES_PER_EPOCH; +static_assert(ANT_CHILD_HEAD_BY_PARENT_SIZE >= 2ULL * ANT_MAX_NODES_PER_EPOCH, + "child-head-by-parent map must stay at or below 50% load so its set() cannot fail"); +// One entry per identity holding a tree. Unlike the map above, nothing caps how many identities +// submit, so this one CAN fill - commit() fails closed with RejectMinerIndexFull. +static constexpr unsigned long long ANT_CHILD_HEAD_BY_MINER_SIZE = 2ULL * MAX_NUMBER_OF_MINERS; + +// How many ANN the epoch's harvest keeps. The target is the LUT with the best error, so this is +// simply the lowest N scores of the epoch - not one per identity, and not tied to the ranking +static constexpr unsigned int ANT_EXPORT_MAX_SOLUTIONS = NUMBER_OF_COMPUTORS; + +// Serial scratch for the save/load header (meta + anchor ring + export set) and the solution export. +// In case of this grow to large, consider use the one in common buffer +static constexpr unsigned long long ANT_SNAPSHOT_SCRATCH_BYTES = 2ULL * 1024 * 1024; // 2MB + +static constexpr unsigned int NO_SIBLING = 0xFFFFFFFFU; +static constexpr unsigned int WORST_SCORE = 0xFFFFFFFFU; + +// annStateSlot values for a record committed without its network. Both sit above ANT_MAX_NODES_PER_EPOCH, +// so annOfNonRoot()'s bounds check already reads them as "no network". MATERIALISING is a walk in progress. +static constexpr unsigned int ANT_ANN_UNMATERIALISED = 0xFFFFFFFFU; +static constexpr unsigned int ANT_ANN_MATERIALISING = 0xFFFFFFFEU; +static constexpr long long ANT_INVALID_INDEX = -1; + +// Anchor digests for recent ticks, indexed by tick & (size - 1). Smallest power of two holding +// 2*(N+1) entries so a lookup inside the freshness window can never be aliased by a newer tick. +static constexpr unsigned int antAnchorRingSize(unsigned int window) +{ + unsigned int size = 1; + while (size < 2u * (window + 1u)) + { + size <<= 1; + } + return size; +} +static constexpr unsigned int ANT_ANCHOR_RING_SIZE = antAnchorRingSize(ANT_PUBLISH_WINDOW_TICKS); +static constexpr unsigned int ANT_ANCHOR_TICK_NONE = 0xFFFFFFFFU; + +// (tick, solutionIndexInTick), ABSOLUTE system tick plus the solution transaction's index in tick. +// Every tick in the subsystem is absolute; slotOf() is the only place a tick becomes a tick-index offset. +struct SolutionRef +{ + unsigned int tick; // ABSOLUTE system tick + unsigned int solutionIndexInTick; + + bool operator==(const SolutionRef& other) const + { + return (tick == other.tick) && (solutionIndexInTick == other.solutionIndexInTick); + } + + bool isRoot() const + { + return (tick == 0) && (solutionIndexInTick == 0xFFFFFFFFu); + } +}; + +// A root of all trees +static constexpr SolutionRef ROOT_REF = { 0u, 0xFFFFFFFFu }; + +// A solution is uniquely identified by (pubkey, parentRef, nonce). +struct AntDedupKey +{ + m256i pubkey; + m256i nonce; + SolutionRef parentRef; + + bool operator==(const AntDedupKey& other) const + { + return (pubkey == other.pubkey) && (nonce == other.nonce) && (parentRef == other.parentRef); + } +}; + +struct AntSolutionRecord +{ + m256i pubkey; + m256i nonce; + SolutionRef parentRef; // this solution's parent, or ROOT_REF + SolutionRef selfRef; // this solution's own address (ABSOLUTE tick inside) + unsigned int score; // error count, lower is better + unsigned int anchorTick; // ABSOLUTE. tick whose digest seeded the RNG + unsigned int depth; // a child of the root is depth 1; the root itself is never stored + unsigned int childAnnHash; // K12 of the canonical ANN at commit; digest-fold input + unsigned int annStateSlot; // index into the ANN pool; always equals the record index + unsigned int nextSiblingIdx; // next child of the same parent, NO_SIBLING terminates +}; +static_assert(sizeof(AntSolutionRecord) == 104, "AntSolutionRecord unexpected padding"); + +// ANN that will be saved for the epoch +template +struct AntExportSlotT +{ + m256i pubkey; + unsigned int score; + unsigned int depth; + PackedAnnT ann; +}; + +// slotOf(tick) -> the run of records committed in that tick, so findIndexBySolutionRef() resolves a +// SolutionRef without scanning the store. The run is unbroken because the store is append-only and +// a tick's solutions all commit while that tick is processed +struct AntTickSlot +{ + unsigned int startIdx; // this tick's first record + unsigned int count; // records this tick produced; written last, so it gates the run +}; + +// Results and diagnostics +enum ValidityResult +{ + Valid, + // Passed every rule but the store is full, so it was not recorded. Still a valid solution: its + // score is already folded into resourceTestingDigest, and the caller must refund and rank it. + ValidNotStored, + RejectParentNotRegistered, + RejectStale, // anchor in the future, or published more than N ticks after it + RejectWrongTree, // parent belongs to a different identity + RejectBelowThreshold, // score above the per-epoch error bound + RejectLeParent, // did not strictly beat the parent + RejectMaxChildrenPerParent, // the parent already holds ANT_MAX_CHILDREN_PER_PARENT children + RejectTickOutOfRange, + RejectReplay, // (pubkey, parentRef, nonce) already committed this epoch + RejectDedupFull, + RejectMinerIndexFull, // more than MAX_NUMBER_OF_MINERS identities hold a tree this epoch + RejectNonCanonicalNonce, // scorer refused the nonce; no score was produced +}; + +struct AntColonyDiagnostics +{ + unsigned long long rejectParentNotRegistered; + unsigned long long rejectStale; + unsigned long long rejectWrongTree; + unsigned long long rejectThreshold; + unsigned long long rejectLeParent; + unsigned long long rejectMaxChildren; + unsigned long long rejectTickOutOfRange; + unsigned long long rejectReplay; + unsigned long long rejectDedupFull; + unsigned long long rejectMinerIndexFull; + unsigned long long rejectNonCanonicalNonce; + + unsigned long long acceptedSolutions; + unsigned long long acceptedNotStored; + unsigned long long treeDepthMax; + unsigned long long treeSizeCurrent; + + void reset() + { + setMem(this, sizeof(*this), 0); + } + + void appendLog(CHAR16* message) const + { + appendText(message, L"tree "); + appendNumber(message, treeSizeCurrent, TRUE); + appendText(message, L"/"); + appendNumber(message, ANT_MAX_NODES_PER_EPOCH, TRUE); + appendText(message, L" depth "); + appendNumber(message, treeDepthMax, FALSE); + appendText(message, L" | accepted "); + appendNumber(message, acceptedSolutions, TRUE); + appendText(message, L" (not stored "); + appendNumber(message, acceptedNotStored, TRUE); + appendText(message, L")"); + + appendText(message, L" | rejected: parent "); + appendNumber(message, rejectParentNotRegistered, TRUE); + appendText(message, L", stale "); + appendNumber(message, rejectStale, TRUE); + appendText(message, L", wrongTree "); + appendNumber(message, rejectWrongTree, TRUE); + appendText(message, L", threshold "); + appendNumber(message, rejectThreshold, TRUE); + appendText(message, L", leParent "); + appendNumber(message, rejectLeParent, TRUE); + appendText(message, L", maxChildren "); + appendNumber(message, rejectMaxChildren, TRUE); + appendText(message, L", tickRange "); + appendNumber(message, rejectTickOutOfRange, TRUE); + appendText(message, L", replay "); + appendNumber(message, rejectReplay, TRUE); + appendText(message, L", dedupFull "); + appendNumber(message, rejectDedupFull, TRUE); + appendText(message, L", minerIndexFull "); + appendNumber(message, rejectMinerIndexFull, TRUE); + appendText(message, L", nonCanonicalNonce "); + appendNumber(message, rejectNonCanonicalNonce, TRUE); + } + + void count(ValidityResult r) + { + switch (r) + { + case ValidityResult::RejectParentNotRegistered: rejectParentNotRegistered++; break; + case ValidityResult::RejectStale: rejectStale++; break; + case ValidityResult::RejectWrongTree: rejectWrongTree++; break; + case ValidityResult::RejectBelowThreshold: rejectThreshold++; break; + case ValidityResult::RejectLeParent: rejectLeParent++; break; + case ValidityResult::RejectMaxChildrenPerParent: rejectMaxChildren++; break; + case ValidityResult::RejectTickOutOfRange: rejectTickOutOfRange++; break; + case ValidityResult::RejectReplay: rejectReplay++; break; + case ValidityResult::RejectDedupFull: rejectDedupFull++; break; + case ValidityResult::RejectMinerIndexFull: rejectMinerIndexFull++; break; + case ValidityResult::RejectNonCanonicalNonce: rejectNonCanonicalNonce++; break; + default: break; + } + } +}; + +// A proposed child, reduced to what the admission rules read. Deliberately narrower than +// AntCommitInput so validateChild() stays a pure predicate. +struct ChildCandidate +{ + m256i pubkey; + unsigned int score; // error count, lower is better + unsigned int anchorTick; // ABSOLUTE + unsigned int publishTick; // ABSOLUTE +}; + +// Every tick here is an ABSOLUTE system tick: selfRef/parentRef ticks, anchorTick and publishTick all +// share one basis (publishTick equals selfRef.tick), so no cross-basis comparison can slip in. +struct AntCommitInput +{ + m256i pubkey; + m256i nonce; + SolutionRef parentRef; + SolutionRef selfRef; + unsigned int anchorTick; // ABSOLUTE + unsigned int publishTick; // ABSOLUTE +}; + +struct AnchorRing +{ + unsigned int ticks[ANT_ANCHOR_RING_SIZE]; + m256i digests[ANT_ANCHOR_RING_SIZE]; +}; + +// --------------------------------------------------------------------------------------------- + +template +class AntColony +{ +public: + // The ANN state will depend on score type + using Ann = typename ScoreT::ANN; + + // In-store form of Ann: 2 bits per trit + using PackedAnn = score_engine::PackedTrits; + static_assert(sizeof(PackedAnn) == PackedAnn::groupCount * sizeof(unsigned long long), + "PackedAnn must not be padded"); + // Catches sizing from a scorer's padded genome (bpp9000: lutSize 27 vs PaddedLut 32). + static_assert(PackedAnn::tritCount == sizeof(Ann), "PackedAnn must cover exactly one ANN"); + + using ExportSlot = AntExportSlotT; + + // The epoch's best ANN, maintained as solutions arrive + struct ExportSet + { + ExportSlot slots[ANT_EXPORT_MAX_SOLUTIONS]; + // Indices into slots, ascending by score. Kept separate so an insert shifts 4-byte indices + // rather than 552-byte slots. + unsigned int order[ANT_EXPORT_MAX_SOLUTIONS]; + unsigned int count; + unsigned int padding; + }; + + static constexpr unsigned long long ANT_RECORDS_BYTES = + (unsigned long long)ANT_MAX_NODES_PER_EPOCH * sizeof(AntSolutionRecord); + static constexpr unsigned long long ANT_ANN_POOL_BYTES = + (unsigned long long)ANT_MAX_NODES_PER_EPOCH * sizeof(PackedAnn); + + // The score is actually a function of below + struct ReplayKey + { + m256i pubkey; + m256i nonce; + // The parent's address, not a hash of its network: both name the same score, but only this one + // can be built without holding that network, which a record accepted on a claimed score lacks. + m256i parentKey; + m256i anchorDigest; // the digest the walk consumed, not the tick it came from + + bool operator==(const ReplayKey& other) const + { + return (pubkey == other.pubkey) && (nonce == other.nonce) + && (parentKey == other.parentKey) && (anchorDigest == other.anchorDigest); + } + }; + + // ROOT_REF encodes distinctly from every real ref, so a root child never shares a key with one. + static m256i replayParentKey(const SolutionRef& parentRef) + { + m256i out = m256i::zero(); + copyMem(&out, &parentRef, sizeof(parentRef)); + return out; + } + static_assert(sizeof(ReplayKey) == 4 * sizeof(m256i), "ReplayKey must be padding-free"); + + struct ReplayEntry + { + ReplayKey key; + PackedAnn ann; + unsigned int score; + unsigned int occupied; + }; + + // Padding-free, so the on-disk entry matches the in-memory one byte for byte. + static_assert(sizeof(ReplayEntry) == + sizeof(ReplayKey) + sizeof(PackedAnn) + 2 * sizeof(unsigned int), + "ReplayEntry unexpected padding"); + + static constexpr unsigned long long ANT_REPLAY_CACHE_BYTES = + (unsigned long long)ANT_REPLAY_CACHE_SIZE * sizeof(ReplayEntry); + + bool init(); + void deinit(); + + // Wipe the whole tree. A new epoch starts empty and reseeded. + void reset(); + + void beginEpoch(const m256i& rootSeed, unsigned int initialTick) + { + reset(); + clearReplayCache(); + _rootSeed = rootSeed; + _initialTick = initialTick; + } + + const m256i& rootSeed() const + { + return _rootSeed; + } + + void setErrorThreshold(unsigned int t) + { + _errorThreshold = t; + } + + unsigned int errorThreshold() const + { + return _errorThreshold; + } + + unsigned int solutionCount() const + { + return _solutionCount; + } + + // Slots a miner can still claim. Reaching zero does not stop acceptance, a valid solution is + // still scored, folded, refunded and ranked, but no NEW branch point can be created, which is + // what a miner needs to know before planning a lineage. + unsigned int freeAnnSlotsCount() const + { + return (_solutionCount < ANT_MAX_NODES_PER_EPOCH) ? (ANT_MAX_NODES_PER_EPOCH - _solutionCount) : 0; + } + + const AntColonyDiagnostics& stats() const + { + return _stats; + } + + void recordReject(ValidityResult r) + { + ASSERT(r != ValidityResult::Valid); + _stats.count(r); + } + + // Only records, the ANN pool and the anchor ring are written; + // the tick index, both head maps and the dedup set are DERIVED and are + // rebuilt from the records on load + bool saveSnapshot(unsigned short epoch, CHAR16* directory, unsigned int initialTick) const; + // rootSeed and errorThreshold are the NODE's values, not the file's. The snapshot must agree + // with them or it is refused + bool loadSnapshot(unsigned short epoch, CHAR16* directory, + const m256i& rootSeed, unsigned int errorThreshold, unsigned int initialTick); + + void putReplayScore(const ReplayKey& key, unsigned int score, const Ann& ann); + bool tryGetReplayScore(const ReplayKey& key, unsigned int& outScore, Ann& outAnn); + void clearReplayCache(); + unsigned int replayCacheOccupancy() const + { + return _replayCacheOccupancy; + } + bool saveReplayCache(unsigned short epoch, CHAR16* directory); + bool loadReplayCache(unsigned short epoch, CHAR16* directory); + + // Writes the ANT_EXPORT_MAX_SOLUTIONS lowest-scoring networks of the epoch to a file for offline + // extraction. MUST be called between endEpoch() and the reset that starts the next epoch + bool exportBestSolutions(unsigned short epoch, CHAR16* directory); + + // Children already recorded under this parent, capped at ANT_MAX_CHILDREN_PER_PARENT, for query + // purposes. Off-thread safe: the head map is read under the lock, the chain walk after it is not. + unsigned int childCountForQuery(const SolutionRef& parentRef, const m256i& childPubkey) + { + unsigned int head = NO_SIBLING; + // Take the latest child first with lock to touch the head map + { + LockGuard guard(_headMapLock); + if (!chainHead(parentRef, childPubkey, head)) + { + return 0; + } + } + + // Escape the lock, then count from head, in which the record is imutable + return childCountFromHead(head); + } + + // Anchor digests. Both take an ABSOLUTE system tick. + // Called from tick processor only + void recordAnchorDigest(unsigned int tick, const m256i& digest); + // Can be called from any processors + bool getAnchorDigest(unsigned int tick, m256i& digest) const; + + // Tree access + + // nullptr when idx is out of range + const AntSolutionRecord* recordAt(long long idx) const + { + if (idx < 0 || (unsigned long long)idx >= _solutionCount) + { + return nullptr; + } + return &_records[idx]; + } + + // Unpacks a stored network into the caller's buffer. ROOT is never a record, so callers must + // handle parentRef.isRoot() before reaching here. False also means the record has no network yet. + bool annOfNonRoot(const AntSolutionRecord& rec, Ann& out) const + { + if (rec.annStateSlot >= _solutionCount) + { + return false; + } + _annPool[rec.annStateSlot].unpack(out.lut); + return true; + } + + bool isAnnMaterialised(unsigned int idx) const + { + return (idx < _solutionCount) && (ATOMIC_LOAD32(_records[idx].annStateSlot) == idx); + } + + bool isAnnClaimHeld(unsigned int idx) const + { + return (idx < _solutionCount) && (ATOMIC_LOAD32(_records[idx].annStateSlot) == ANT_ANN_MATERIALISING); + } + + // Outcome of claiming a record's network for rebuild. + enum AnnClaim + { + AnnClaimReady, // the network is already in the pool + AnnClaimOwned, // this caller owns the rebuild and must end it with publishAnn or releaseAnnClaim + AnnClaimBusy, // another thread is rebuilding it + AnnClaimInvalid, // no such record + }; + + AnnClaim tryClaimAnn(unsigned int idx) + { + if (idx >= _solutionCount) + { + return AnnClaimInvalid; + } + LockGuard guard(_annClaimLock); + const unsigned int state = _records[idx].annStateSlot; + if (state == idx) + { + return AnnClaimReady; + } + if (state == ANT_ANN_MATERIALISING) + { + return AnnClaimBusy; + } + _records[idx].annStateSlot = ANT_ANN_MATERIALISING; + return AnnClaimOwned; + } + + // The network lands before the slot index, so a reader that sees the index never reads a half-written + // one. The hash is written here too: a record committed without a network had none to hash. + void publishAnn(unsigned int idx, const Ann& ann, unsigned int annHash) + { + _annPool[idx].pack(ann.lut); + _records[idx].childAnnHash = annHash; + ATOMIC_STORE32(_records[idx].annStateSlot, (long)idx); + } + + // Drops a claim whose rebuild produced nothing, so a later attempt retries instead of waiting on it. + void releaseAnnClaim(unsigned int idx) + { + if (idx >= _solutionCount) + { + return; + } + LockGuard guard(_annClaimLock); + if (_records[idx].annStateSlot == ANT_ANN_MATERIALISING) + { + _records[idx].annStateSlot = ANT_ANN_UNMATERIALISED; + } + } + + long long findIndexBySolutionRef(const SolutionRef& ref) const; + + // Constraint specific functions + + // Resolves a parent for scoring. outParentRec is null for ROOT_REF, the caller derives the + // per-identity root from the submitter's pubkey instead. + ValidityResult tryGetParent(const SolutionRef& parentRef, + const AntSolutionRecord** outParentRec) const; + + // Admission rules for a proposed child: freshness, tree ownership, threshold, parent, per-parent + // child cap. Static and pure, so the rule set is testable without a colony. Lower score is better. + // trustedScore drops the rules that judge the score itself, for a caller that took it on trust. + static ValidityResult validateChild(const ChildCandidate& child, + const AntSolutionRecord* parentRecord, unsigned int childCount, unsigned int threshold, + bool trustedScore = false); + + // Validates and, if accepted, appends the record and its network. A null childAnn marks the slot + // unmaterialised, for a score accepted without walking. + ValidityResult commit(const AntCommitInput& in, const AntSolutionRecord* parentRec, + unsigned int score, const Ann* childAnn, unsigned int childAnnHash, + bool trustedScore = false); + +private: + // Children already recorded under this parent, capped at ANT_MAX_CHILDREN_PER_PARENT. Root + // children are keyed by miner, deeper ones by parent. + unsigned int countChildren(const SolutionRef& parentRef, const m256i& childPubkey) const; + + // The map read, and the ONLY part of a child count that touches a head map. Split out because the + // walk that follows it does not, which is what lets the query hold the lock for a constant time + // instead of a whole chain. + // Depth-1 nodes chain per identity; deeper nodes chain per parent, which is single-identity by the + // wrong-tree check. + bool chainHead(const SolutionRef& parentRef, const m256i& childPubkey, unsigned int& out) const + { + if (parentRef.isRoot()) + { + return _childHeadByMiner->get(childPubkey, out); + } + return _childHeadByParent->get(parentRef, out); + } + + // The walk half, from a chain head already in hand. Takes no lock: _records is append-only and a + // record's nextSiblingIdx is written once at commit and never touched again, so a chain is stable + // to follow even while the tick processor is appending elsewhere. + unsigned int childCountFromHead(unsigned int head) const; + + // Offers a solution to the epoch's best-N set. Called for EVERY solution that passes the rules + void noteExportCandidate(const m256i& pubkey, unsigned int score, unsigned int depth, const Ann& ann) + { + ExportSet& set = *_exportSet; + unsigned int slot; + if (set.count < ANT_EXPORT_MAX_SOLUTIONS) + { + slot = set.count; + } + else if (score >= set.slots[set.order[ANT_EXPORT_MAX_SOLUTIONS - 1]].score) + { + // The common case once the set is full + return; + } + else + { + // Reuse the worst entry's storage; its index leaves the order below. + slot = set.order[ANT_EXPORT_MAX_SOLUTIONS - 1]; + } + + set.slots[slot].pubkey = pubkey; + set.slots[slot].score = score; + set.slots[slot].depth = depth; + set.slots[slot].ann.pack(ann.lut); + + // Insert into the order, shifting indices only. Equal scores keep the incumbent ahead, so + // among equals the earlier solution ranks first + const unsigned int end = (set.count < ANT_EXPORT_MAX_SOLUTIONS) ? set.count : (ANT_EXPORT_MAX_SOLUTIONS - 1); + unsigned int i = end; + while (i > 0 && set.slots[set.order[i - 1]].score > score) + { + set.order[i] = set.order[i - 1]; + i--; + } + set.order[i] = slot; + if (set.count < ANT_EXPORT_MAX_SOLUTIONS) + { + set.count++; + } + } + + // loadSnapshot() helper: rebuild the tick index, head maps and dedup set from the loaded + // records, treating them as untrusted input. Uses _initialTick, set by the caller beforehand. + bool rebuildDerivedState(); + + static unsigned int replaySlotOf(const ReplayKey& key) + { + unsigned long long digest; + KangarooTwelve(&key, sizeof(key), &digest, sizeof(digest)); + return (unsigned int)(digest & (ANT_REPLAY_CACHE_SIZE - 1)); + } + + // The sole place an absolute tick becomes a tick-index offset. Returns false when the tick falls + // outside this epoch's window (before initialTick, or past the per-epoch tick cap); callers treat + // that as "no such record" - RejectParentNotRegistered on lookup, RejectTickOutOfRange on commit. + bool slotOf(unsigned int tick, unsigned int& slot) const + { + if (tick < _initialTick) + { + return false; + } + slot = tick - _initialTick; + return slot < MAX_NUMBER_OF_TICKS_PER_EPOCH; + } + + AntSolutionRecord* _records; + PackedAnn* _annPool; + AntTickSlot* _tickIndex; + AnchorRing* _anchors; + + // Solutions already committed this epoch, so a resend is rejected instead of re-added. + QPI::HashSet* _dedup; + ExportSet* _exportSet; + + // The two head maps have no reader/writer protocol of their own: QPI::HashMap::set() makes a + // slot's key visible before its value, so a reader asking for the key being inserted can get a + // garbage index + volatile char _headMapLock; + // Both give countChildren() a parent's children without scanning the store: the value is the + // newest child's record index, and nextSiblingIdx chains back to the older ones. + // parent's address -> newest child + QPI::HashMap* _childHeadByParent; + // miner's pubkey -> newest depth-1 node (a child OF that miner's root, not a root itself). + // Keyed by miner because ROOT_REF is shared by everyone. + QPI::HashMap* _childHeadByMiner; + + ReplayEntry* _replayCache; + volatile char _replayCacheLock; + + // Guards the annStateSlot transitions only; the walk itself runs with this released. + volatile char _annClaimLock; + + // Serial scratch for save/load and the solution export + unsigned char* _snapshotScratch; + unsigned int _replayCacheOccupancy; + + unsigned int _solutionCount; + unsigned int _errorThreshold; + // This epoch's first tick. slotOf() maps an absolute tick to a tick-index offset against it. + unsigned int _initialTick; + m256i _rootSeed; + AntColonyDiagnostics _stats; +}; + +template +inline bool AntColony::init() +{ + setMem(this, sizeof(*this), 0); + + if (!allocPoolWithErrorLog(L"AntColony::_records", + ANT_RECORDS_BYTES, (void**)&_records, __LINE__)) + { + return false; + } + if (!allocPoolWithErrorLog(L"AntColony::_annPool", + ANT_ANN_POOL_BYTES, (void**)&_annPool, __LINE__)) + { + return false; + } + if (!allocPoolWithErrorLog(L"AntColony::_tickIndex", + (unsigned long long)MAX_NUMBER_OF_TICKS_PER_EPOCH * sizeof(AntTickSlot), + (void**)&_tickIndex, __LINE__)) + { + return false; + } + if (!allocPoolWithErrorLog(L"AntColony::_anchors", + sizeof(AnchorRing), (void**)&_anchors, __LINE__)) + { + return false; + } + if (!allocPoolWithErrorLog(L"AntColony::_childHeadByParent", + sizeof(QPI::HashMap), + (void**)&_childHeadByParent, __LINE__)) + { + return false; + } + if (!allocPoolWithErrorLog(L"AntColony::_childHeadByMiner", + sizeof(QPI::HashMap), + (void**)&_childHeadByMiner, __LINE__)) + { + return false; + } + if (!allocPoolWithErrorLog(L"AntColony::_exportSet", + sizeof(ExportSet), (void**)&_exportSet, __LINE__)) + { + return false; + } + if (!allocPoolWithErrorLog(L"AntColony::_dedup", + sizeof(QPI::HashSet), + (void**)&_dedup, __LINE__)) + { + return false; + } + if (!allocPoolWithErrorLog(L"AntColony::_replayCache", + ANT_REPLAY_CACHE_BYTES, (void**)&_replayCache, __LINE__)) + { + return false; + } + if (!allocPoolWithErrorLog(L"AntColony::_snapshotScratch", + ANT_SNAPSHOT_SCRATCH_BYTES, (void**)&_snapshotScratch, __LINE__)) + { + return false; + } + + reset(); + clearReplayCache(); + return true; +} + +template +inline void AntColony::deinit() +{ + if (_snapshotScratch) + { + freePool(_snapshotScratch); + } + if (_replayCache) + { + freePool(_replayCache); + } + if (_exportSet) + { + freePool(_exportSet); + } + if (_dedup) + { + freePool(_dedup); + } + if (_childHeadByMiner) + { + freePool(_childHeadByMiner); + } + if (_childHeadByParent) + { + freePool(_childHeadByParent); + } + if (_anchors) + { + freePool(_anchors); + } + if (_tickIndex) + { + freePool(_tickIndex); + } + if (_annPool) + { + freePool(_annPool); + } + if (_records) + { + freePool(_records); + } + + _replayCache = nullptr; + _exportSet = nullptr; + _dedup = nullptr; + _childHeadByMiner = nullptr; + _childHeadByParent = nullptr; + _anchors = nullptr; + _tickIndex = nullptr; + _annPool = nullptr; + _records = nullptr; +} + +template +inline void AntColony::reset() +{ + ASSERT(_records != nullptr); + ASSERT(_annPool != nullptr); + ASSERT(_tickIndex != nullptr); + ASSERT(_anchors != nullptr); + ASSERT(_childHeadByParent != nullptr); + ASSERT(_childHeadByMiner != nullptr); + ASSERT(_dedup != nullptr); + ASSERT(_exportSet != nullptr); + + setMem(_records, ANT_RECORDS_BYTES, 0); + setMem(_tickIndex, + (unsigned long long)MAX_NUMBER_OF_TICKS_PER_EPOCH * sizeof(AntTickSlot), 0); + _childHeadByParent->reset(); + _childHeadByMiner->reset(); + _dedup->reset(); + setMem(_exportSet, sizeof(ExportSet), 0); + + // ANT_ANCHOR_TICK_NONE is used rather than zero + for (unsigned int i = 0; i < ANT_ANCHOR_RING_SIZE; i++) + { + _anchors->ticks[i] = ANT_ANCHOR_TICK_NONE; + _anchors->digests[i] = m256i::zero(); + } + + _solutionCount = 0; + _rootSeed = m256i::zero(); + + // Forgets setErrorThreshold rejects everything but a perfect score, rather than silently reusing the previous epoch's bound. + _errorThreshold = 0; + + _stats.reset(); +} + +template +inline void AntColony::clearReplayCache() +{ + if (_replayCache == nullptr) + { + return; + } + LockGuard guard(_replayCacheLock); + for (unsigned int i = 0; i < ANT_REPLAY_CACHE_SIZE; i++) + { + _replayCache[i].occupied = 0; + } + _replayCacheOccupancy = 0; +} + +template +inline void AntColony::putReplayScore(const ReplayKey& key, unsigned int score, const Ann& ann) +{ + if (_replayCache == nullptr) + { + return; + } + ReplayEntry staged; + staged.key = key; + staged.ann.pack(ann.lut); + staged.score = score; + staged.occupied = 1; + + ReplayEntry& slot = _replayCache[replaySlotOf(key)]; + LockGuard guard(_replayCacheLock); + if (!slot.occupied) + { + _replayCacheOccupancy++; + } + copyMem(&slot, &staged, sizeof(ReplayEntry)); +} + +template +inline bool AntColony::tryGetReplayScore(const ReplayKey& key, unsigned int& outScore, Ann& outAnn) +{ + if (_replayCache == nullptr) + { + return false; + } + ReplayEntry& slot = _replayCache[replaySlotOf(key)]; + LockGuard guard(_replayCacheLock); + if (!slot.occupied || !(slot.key == key)) + { + return false; + } + outScore = slot.score; + slot.ann.unpack(outAnn.lut); + return true; +} + +// Cache the already computed score for the ant colony + +template +inline bool AntColony::saveReplayCache(unsigned short epoch, CHAR16* directory) +{ +#if ANT_USE_SCORE_CACHE + if (_replayCache == nullptr) + { + return false; + } + addEpochToFileName(ANT_COLONY_REPLAY_CACHE_FILENAME, + sizeof(ANT_COLONY_REPLAY_CACHE_FILENAME) / sizeof(ANT_COLONY_REPLAY_CACHE_FILENAME[0]), epoch); + + // Held across the file IO so the table cannot change under the write. That blocks the solution + // processors for the duration + LockGuard guard(_replayCacheLock); + if (saveLargeFile(ANT_COLONY_REPLAY_CACHE_FILENAME, ANT_REPLAY_CACHE_BYTES, + (unsigned char*)_replayCache, directory, false) != (long long)ANT_REPLAY_CACHE_BYTES) + { + logToConsole(L"[ant-colony] failed to save replay cache"); + return false; + } + return true; +#else + return true; +#endif +} + +template +inline bool AntColony::loadReplayCache(unsigned short epoch, CHAR16* directory) +{ +#if ANT_USE_SCORE_CACHE + if (_replayCache == nullptr) + { + return false; + } + addEpochToFileName(ANT_COLONY_REPLAY_CACHE_FILENAME, + sizeof(ANT_COLONY_REPLAY_CACHE_FILENAME) / sizeof(ANT_COLONY_REPLAY_CACHE_FILENAME[0]), epoch); + + LockGuard guard(_replayCacheLock); + if (loadLargeFile(ANT_COLONY_REPLAY_CACHE_FILENAME, ANT_REPLAY_CACHE_BYTES, + (unsigned char*)_replayCache, directory) != (long long)ANT_REPLAY_CACHE_BYTES) + { + // Absent at the start of an epoch, and a wrong size means another build wrote it. Either way + // the table is zeroed and every solution gets computed honestly. + logToConsole(L"[ant-colony] no usable replay cache, solutions will be recomputed"); + setMem(_replayCache, ANT_REPLAY_CACHE_BYTES, 0); + _replayCacheOccupancy = 0; + return false; + } + + unsigned int occupied = 0; + for (unsigned int i = 0; i < ANT_REPLAY_CACHE_SIZE; i++) + { + if (_replayCache[i].occupied) + { + occupied++; + } + } + _replayCacheOccupancy = occupied; + + CHAR16 message[192]; + setText(message, L"[ant-colony] replay cache loaded, entries "); + appendNumber(message, occupied, FALSE); + logToConsole(message); + return true; +#else + return true; +#endif +} + +// The epoch's harvest file + + +// Written once at the front of the file +struct AntColonyExportHeader +{ + unsigned int epoch; + unsigned int entryCount; + unsigned int entrySizeBytes; // of AntColonyExportEntry, not of a store record + unsigned int annSizeBytes; + unsigned int solutionCount; // the whole epoch, of which entryCount are exported + unsigned int errorThreshold; + unsigned char topologyHash[32]; // == BPP9000_TOPOLOGY_HASH of the build that wrote this + unsigned char dataHash[32]; // == BPP9000_DATA_HASH + m256i rootSeed; +}; +static_assert(sizeof(AntColonyExportHeader) == 24 + 64 + 32, "AntColonyExportHeader unexpected padding"); + +// What the harvest needs to reproduce the best error +struct AntColonyExportEntry +{ + // Names the identity that found this network, log only + m256i pubkey; + unsigned int score; // error count, lower is better - how good this network is + unsigned int depth; // generations of strict improvement behind it, so the chain length is visible +}; +static_assert(sizeof(AntColonyExportEntry) == 32 + 8, "AntColonyExportEntry unexpected padding"); + +template +inline bool AntColony::exportBestSolutions(unsigned short epoch, CHAR16* directory) +{ + ASSERT(_exportSet != nullptr); + + struct Entry + { + AntColonyExportEntry meta; + Ann ann; + }; + const ExportSet& set = *_exportSet; + + AntColonyExportHeader header; + setMem(&header, sizeof(header), 0); + header.epoch = epoch; + header.entryCount = set.count; + header.entrySizeBytes = (unsigned int)sizeof(AntColonyExportEntry); + header.annSizeBytes = (unsigned int)sizeof(Ann); + header.solutionCount = _solutionCount; + header.errorThreshold = _errorThreshold; + copyMem(header.topologyHash, BPP9000_TOPOLOGY_HASH, sizeof(header.topologyHash)); + copyMem(header.dataHash, BPP9000_DATA_HASH, sizeof(header.dataHash)); + header.rootSeed = _rootSeed; + + if (set.count == 0) + { + // gAsyncFileIO is NULL only during early init and in NO_UEFI tests; otherwise route the write + // through the async worker so the tick-processor thread never touches the EFI file protocol. + const long long headerSaved = gAsyncFileIO + ? asyncSave(ANT_COLONY_SOLUTIONS_EOE_FILENAME, sizeof(header), (unsigned char*)&header, directory) + : save(ANT_COLONY_SOLUTIONS_EOE_FILENAME, sizeof(header), (unsigned char*)&header, directory); + return headerSaved == (long long)sizeof(header); + } + + static_assert(sizeof(AntColonyExportHeader) + (unsigned long long)ANT_EXPORT_MAX_SOLUTIONS * sizeof(Entry) + <= ANT_SNAPSHOT_SCRATCH_BYTES, "ant solution export exceeds the scratch buffer"); + const unsigned long long totalBytes = sizeof(header) + (unsigned long long)set.count * sizeof(Entry); + unsigned char* buffer = _snapshotScratch; + + copyMem(buffer, &header, sizeof(header)); + Entry* out = (Entry*)(buffer + sizeof(header)); + for (unsigned int i = 0; i < set.count; i++) + { + const ExportSlot& slot = set.slots[set.order[i]]; + out[i].meta.pubkey = slot.pubkey; + out[i].meta.score = slot.score; + out[i].meta.depth = slot.depth; + slot.ann.unpack(out[i].ann.lut); + } + + const long long saved = gAsyncFileIO + ? asyncSave(ANT_COLONY_SOLUTIONS_EOE_FILENAME, totalBytes, buffer, directory) + : save(ANT_COLONY_SOLUTIONS_EOE_FILENAME, totalBytes, buffer, directory); + + if (saved != (long long)totalBytes) + { +#ifndef NDEBUG + addDebugMessage(L"[ant-colony] failed to write the solution export"); +#endif + return false; + } + +#ifndef NDEBUG + CHAR16 message[192]; + setText(message, L"[ant-colony] exported best networks, entries "); + appendNumber(message, set.count, FALSE); + appendText(message, L", best error "); + appendNumber(message, set.slots[set.order[0]].score, FALSE); + appendText(message, L", worst kept "); + appendNumber(message, set.slots[set.order[set.count - 1]].score, FALSE); + addDebugMessage(message); +#endif + return true; +} + +template +inline void AntColony::recordAnchorDigest(unsigned int tick, const m256i& digest) +{ + const unsigned int slot = tick & (ANT_ANCHOR_RING_SIZE - 1); + // Invalidate first + ATOMIC_STORE32(_anchors->ticks[slot], (long)ANT_ANCHOR_TICK_NONE); + _anchors->digests[slot] = digest; + // The tick marker is written last, a reader that sees the tick must already see its digest. + ATOMIC_STORE32(_anchors->ticks[slot], (long)tick); +} + +template +inline bool AntColony::getAnchorDigest(unsigned int tick, m256i& digest) const +{ + if (tick == ANT_ANCHOR_TICK_NONE) + { + return false; + } + const unsigned int slot = tick & (ANT_ANCHOR_RING_SIZE - 1); + // Seqlock read, the tick is loaded atomically before and after the digest copy, so a re-check + // that still sees the same tick means the digest was not overwritten mid-copy. + if ((unsigned int)ATOMIC_LOAD32(_anchors->ticks[slot]) != tick) + { + return false; // never recorded, or aged out and overwritten by a newer tick + } + digest = _anchors->digests[slot]; + return ((unsigned int)ATOMIC_LOAD32(_anchors->ticks[slot]) == tick); +} + +template +inline long long AntColony::findIndexBySolutionRef(const SolutionRef& ref) const +{ + unsigned int tickSlot = 0; + if (ref.isRoot() || !slotOf(ref.tick, tickSlot)) + { + return ANT_INVALID_INDEX; + } + // Start from tick' begin index in the record and loop total of record in the tick + const AntTickSlot& slot = _tickIndex[tickSlot]; + for (unsigned int i = 0; i < slot.count; i++) + { + const unsigned int idx = slot.startIdx + i; + if (idx >= _solutionCount) + { + break; + } + if (_records[idx].selfRef == ref) + { + return (long long)idx; + } + } + return ANT_INVALID_INDEX; +} + +template +inline unsigned int AntColony::countChildren(const SolutionRef& parentRef, const m256i& childPubkey) const +{ + // Tick processor only, so the head map read needs no lock here - nothing else writes it. + unsigned int head = NO_SIBLING; + if (!chainHead(parentRef, childPubkey, head)) + { + return 0; + } + return childCountFromHead(head); +} + +template +inline unsigned int AntColony::childCountFromHead(unsigned int head) const +{ + // Count only up to the cap: past it the child is rejected regardless, so the walk and with it + // the whole per-commit cost, is bounded by ANT_MAX_CHILDREN_PER_PARENT. A cap of 0 (unbound) + // leaves the loop empty and returns 0, since the count is then never used. + unsigned int count = 0; + unsigned int idx = head; + while (idx != NO_SIBLING && count < ANT_MAX_CHILDREN_PER_PARENT) + { + // NOT a redundant bounds check - it is the publication barrier this walk relies on. commit() + // points the head map at a new index BEFORE it writes that record, and only bumps + // _solutionCount once the record is complete. An off-thread walk that reaches the new index + // early therefore stops here instead of reading half a record. Removing this reintroduces a + // torn read that would only ever show up under load. + if (idx >= _solutionCount) + { + break; + } + count++; + idx = _records[idx].nextSiblingIdx; + } + return count; +} + +template +inline ValidityResult AntColony::tryGetParent(const SolutionRef& parentRef, + const AntSolutionRecord** outParentRec) const +{ + *outParentRec = nullptr; + if (parentRef.isRoot()) + { + return ValidityResult::Valid; // root is not a record; a null parent is the valid answer + } + + const long long parentIdx = findIndexBySolutionRef(parentRef); + if (parentIdx == ANT_INVALID_INDEX) + { + return ValidityResult::RejectParentNotRegistered; + } + const AntSolutionRecord* rec = recordAt(parentIdx); + if (rec == nullptr) + { + return ValidityResult::RejectParentNotRegistered; + } + *outParentRec = rec; + return ValidityResult::Valid; +} + +template +inline ValidityResult AntColony::validateChild(const ChildCandidate& child, + const AntSolutionRecord* parentRecord, unsigned int childCount, unsigned int threshold, + bool trustedScore) +{ + // Freshness, the anchor cannot be in the future, and publication cannot lag it by more than N. + if (child.anchorTick > child.publishTick + || (child.publishTick - child.anchorTick) > ANT_PUBLISH_WINDOW_TICKS) + { + return ValidityResult::RejectStale; + } + + // A null parent record means ROOT, it has no score of its own, so seed WORST_SCORE and any child + // improves on it. A non-root parent must belong to the same identity + unsigned int parentScore = WORST_SCORE; + if (parentRecord != nullptr) + { + if (!(parentRecord->pubkey == child.pubkey)) + { + return ValidityResult::RejectWrongTree; + } + parentScore = parentRecord->score; + } + + // A trusted score cannot be judged against the threshold or the parent: rejecting on a number this node + // did not compute leaves no refund for the quorum to disagree with, so the lie would go uncaught. The + // metadata rules around this stay on, since every node reads those the same way. + if (!trustedScore) + { + if (child.score > threshold) + { + return ValidityResult::RejectBelowThreshold; + } + if (child.score >= parentScore) + { + return ValidityResult::RejectLeParent; + } + } + // Per-parent breadth cap. 0 means unbound - no cap. + if (ANT_MAX_CHILDREN_PER_PARENT != 0 && childCount >= ANT_MAX_CHILDREN_PER_PARENT) + { + return ValidityResult::RejectMaxChildrenPerParent; + } + return ValidityResult::Valid; +} + +template +inline ValidityResult AntColony::commit(const AntCommitInput& in, const AntSolutionRecord* parentRec, + unsigned int score, const Ann* childAnn, unsigned int childAnnHash, bool trustedScore) +{ + const unsigned int childCount = countChildren(in.parentRef, in.pubkey); + const ChildCandidate child{ in.pubkey, score, in.anchorTick, in.publishTick }; + + const ValidityResult result = validateChild(child, parentRec, childCount, _errorThreshold, trustedScore); + if (result != ValidityResult::Valid) + { + recordReject(result); + return result; + } + + const AntDedupKey dedupKey{ in.pubkey, in.nonce, in.parentRef }; + if (_dedup->contains(dedupKey)) + { + recordReject(ValidityResult::RejectReplay); + return ValidityResult::RejectReplay; + } + // Store full. Every rule above already passed, so the solution is honest work and its score is + // in the digest whatever happens here - rejecting it would burn the deposit for a valid answer. + // Honour it and stop storing: the tree freezes, the leaderboard does not. + if (_solutionCount >= ANT_MAX_NODES_PER_EPOCH) + { + // The record is dropped, the network is not, still note this sols for end of epoch exppot + if (childAnn != nullptr) + { + noteExportCandidate(in.pubkey, score, (parentRec != nullptr) ? (parentRec->depth + 1) : 1, *childAnn); + } + _stats.acceptedNotStored++; + return ValidityResult::ValidNotStored; + } + unsigned int selfSlot = 0; + if (!slotOf(in.selfRef.tick, selfSlot)) + { + recordReject(ValidityResult::RejectTickOutOfRange); + return ValidityResult::RejectTickOutOfRange; + } + // never commit a solution without recording its replay key. Cannot fire under the + // cap (population <= ANT_MAX_NODES_PER_EPOCH = 50% of ANT_DEDUP_SIZE), kept as a defensive check + if (_dedup->add(dedupKey) == QPI::NULL_INDEX) + { + recordReject(ValidityResult::RejectDedupFull); + return ValidityResult::RejectDedupFull; + } + + const unsigned int newIdx = _solutionCount; + + // Claim the sibling-chain head before writing the record + unsigned int prevHead = NO_SIBLING; + bool headClaimed = true; + { + LockGuard guard(_headMapLock); + if (in.parentRef.isRoot()) + { + _childHeadByMiner->get(in.pubkey, prevHead); + headClaimed = (_childHeadByMiner->set(in.pubkey, newIdx) != QPI::NULL_INDEX); + } + else + { + _childHeadByParent->get(in.parentRef, prevHead); + _childHeadByParent->set(in.parentRef, newIdx); // cannot fail, see the static_assert on its size + } + } + if (!headClaimed) + { + // Fail closed. Degrading instead, accepting the node but leaving the identity without a + // chain head, would leave its children uncounted and silently disable its cap. Released + // first: this touches _dedup, which must not be reached under the head-map lock. + _dedup->remove(dedupKey); + recordReject(ValidityResult::RejectMinerIndexFull); + return ValidityResult::RejectMinerIndexFull; + } + + // The record and its network share an index, which keeps the used portion of the allocation a + // contiguous prefix. + if (childAnn != nullptr) + { + _annPool[newIdx].pack(childAnn->lut); + } + + AntSolutionRecord& newRec = _records[newIdx]; + newRec.pubkey = in.pubkey; + newRec.nonce = in.nonce; + newRec.parentRef = in.parentRef; + newRec.selfRef = in.selfRef; + newRec.score = score; + newRec.anchorTick = in.anchorTick; + newRec.depth = (parentRec != nullptr) ? (parentRec->depth + 1) : 1; + newRec.childAnnHash = childAnnHash; + newRec.annStateSlot = (childAnn != nullptr) ? newIdx : ANT_ANN_UNMATERIALISED; + newRec.nextSiblingIdx = prevHead; + + AntTickSlot& tslot = _tickIndex[selfSlot]; + if (tslot.count == 0) + { + tslot.startIdx = newIdx; + } + + // PUBLICATION ORDER, load-bearing. Readers on other threads gate on tslot.count, so everything + // they may then read must already be visible: record fields, then _solutionCount, then + // tslot.count last. _solutionCount must rise before tslot.count or findIndexBySolutionRef can + // resolve an index that recordAt() rejects. + // ATOMIC_STORE32 is here for the ordering barrier, not for atomicity of the value: these are + // plain unsigned ints written only by the tick processor + ATOMIC_STORE32(_solutionCount, (long)(newIdx + 1)); + ATOMIC_STORE32(tslot.count, (long)(tslot.count + 1)); + + if (childAnn != nullptr) + { + noteExportCandidate(in.pubkey, score, newRec.depth, *childAnn); + } + + _stats.acceptedSolutions++; + _stats.treeSizeCurrent = _solutionCount; + if (newRec.depth > _stats.treeDepthMax) + { + _stats.treeDepthMax = newRec.depth; + } + return ValidityResult::Valid; +} + +// Only what cannot be derived is written. The tick index, both head maps and the dedup set are +// rebuilt from the records + +struct AntColonySnapshotMeta +{ + unsigned int magic; + unsigned int version; + unsigned int epoch; + unsigned int solutionCount; + // Layout guards. A snapshot written by a build with a different record or ANN size must be + // refused rather than reinterpreted + unsigned int recordSizeBytes; + unsigned int annPoolEntryBytes; + unsigned int errorThreshold; + // This epoch's first tick. Records hold absolute ticks; the base must still match so slotOf() maps + // them into this node's tick index, and a snapshot from another epoch is refused rather than mis-read. + unsigned int initialTick; + unsigned int anchorRingBytes; + unsigned int exportSetBytes; + m256i rootSeed; + + static constexpr unsigned int MAGIC = 0x414E5443; // "ANTC" + static constexpr unsigned int VERSION = 1; // SolutionRef holds absolute ticks +}; +static_assert(sizeof(AntColonySnapshotMeta) == 40 + 32, "AntColonySnapshotMeta unexpected padding"); + +static void antSnapshotFailure(const CHAR16* what, unsigned long long a, unsigned long long b) +{ + CHAR16 message[256]; + setText(message, L"[ant-colony] snapshot: "); + appendText(message, what); + appendText(message, L" "); + appendNumber(message, a, FALSE); + appendText(message, L" / "); + appendNumber(message, b, FALSE); + logToConsole(message); +} + +// Records and pool are sized by this, never by the raw count. At least one slot is always written, +// so no snapshot file is ever zero length +static unsigned long long antSnapshotSlotCount(unsigned int solutionCount) +{ + return (solutionCount > 0) ? (unsigned long long)solutionCount : 1ULL; +} + +static void antSnapshotNameForEpoch(unsigned short epoch) +{ + addEpochToFileName(ANT_SNAPSHOT_HEADER_FILENAME, sizeof(ANT_SNAPSHOT_HEADER_FILENAME) / sizeof(ANT_SNAPSHOT_HEADER_FILENAME[0]), epoch); + addEpochToFileName(ANT_SNAPSHOT_RECORDS_FILENAME, sizeof(ANT_SNAPSHOT_RECORDS_FILENAME) / sizeof(ANT_SNAPSHOT_RECORDS_FILENAME[0]), epoch); + addEpochToFileName(ANT_SNAPSHOT_POOL_FILENAME, sizeof(ANT_SNAPSHOT_POOL_FILENAME) / sizeof(ANT_SNAPSHOT_POOL_FILENAME[0]), epoch); +} + +template +inline bool AntColony::saveSnapshot(unsigned short epoch, CHAR16* directory, + unsigned int initialTick) const +{ + ASSERT(_records != nullptr); + ASSERT(_annPool != nullptr); + ASSERT(_anchors != nullptr); + + antSnapshotNameForEpoch(epoch); + + AntColonySnapshotMeta meta; + setMem(&meta, sizeof(meta), 0); + meta.magic = AntColonySnapshotMeta::MAGIC; + meta.version = AntColonySnapshotMeta::VERSION; + meta.epoch = epoch; + meta.solutionCount = _solutionCount; + meta.recordSizeBytes = (unsigned int)sizeof(AntSolutionRecord); + meta.annPoolEntryBytes = (unsigned int)sizeof(PackedAnn); + meta.errorThreshold = _errorThreshold; + meta.initialTick = initialTick; + meta.anchorRingBytes = (unsigned int)sizeof(AnchorRing); + meta.exportSetBytes = (unsigned int)sizeof(ExportSet); + meta.rootSeed = _rootSeed; + + // The meta, anchor ring and export set share one file. The file API writes a single contiguous + // buffer, so the three are gathered into the serial scratch: meta, then anchors, then export. + const unsigned long long headerBytes = sizeof(meta) + sizeof(AnchorRing) + sizeof(ExportSet); + static_assert(sizeof(AntColonySnapshotMeta) + sizeof(AnchorRing) + sizeof(ExportSet) <= ANT_SNAPSHOT_SCRATCH_BYTES, + "ant snapshot header exceeds the scratch buffer"); + unsigned char* headerBuffer = _snapshotScratch; + copyMem(headerBuffer, &meta, sizeof(meta)); + copyMem(headerBuffer + sizeof(meta), _anchors, sizeof(AnchorRing)); + copyMem(headerBuffer + sizeof(meta) + sizeof(AnchorRing), _exportSet, sizeof(ExportSet)); + if (save(ANT_SNAPSHOT_HEADER_FILENAME, headerBytes, headerBuffer, directory) != (long long)headerBytes) + { + logToConsole(L"[ant-colony] failed to save snapshot header"); + return false; + } + + // Written even when the colony is empty, so the operator's snapshot is always the same number of files + const unsigned long long slots = antSnapshotSlotCount(_solutionCount); + const unsigned long long recordBytes = slots * sizeof(AntSolutionRecord); + const unsigned long long poolBytes = slots * sizeof(PackedAnn); + if (saveLargeFile(ANT_SNAPSHOT_RECORDS_FILENAME, recordBytes, (unsigned char*)_records, directory, false) + != (long long)recordBytes) + { + logToConsole(L"[ant-colony] failed to save snapshot records"); + return false; + } + if (saveLargeFile(ANT_SNAPSHOT_POOL_FILENAME, poolBytes, (unsigned char*)_annPool, directory, false) + != (long long)poolBytes) + { + logToConsole(L"[ant-colony] failed to save snapshot pool"); + return false; + } + return true; +} + +template +inline bool AntColony::loadSnapshot(unsigned short epoch, CHAR16* directory, + const m256i& rootSeed, unsigned int errorThreshold, unsigned int initialTick) +{ + ASSERT(_records != nullptr); + ASSERT(_annPool != nullptr); + ASSERT(_anchors != nullptr); + + antSnapshotNameForEpoch(epoch); + + // The meta, anchor ring and export set share one file. Read it whole, validate the meta before + // any colony state is touched, then copy the two sections into place. + const unsigned long long headerBytes = sizeof(AntColonySnapshotMeta) + sizeof(AnchorRing) + sizeof(ExportSet); + static_assert(sizeof(AntColonySnapshotMeta) + sizeof(AnchorRing) + sizeof(ExportSet) <= ANT_SNAPSHOT_SCRATCH_BYTES, + "ant snapshot header exceeds the scratch buffer"); + unsigned char* headerBuffer = _snapshotScratch; + if (load(ANT_SNAPSHOT_HEADER_FILENAME, headerBytes, headerBuffer, directory) != (long long)headerBytes) + { + logToConsole(L"[ant-colony] failed to load snapshot header"); + return false; + } + + AntColonySnapshotMeta meta; + copyMem(&meta, headerBuffer, sizeof(meta)); + if (meta.magic != AntColonySnapshotMeta::MAGIC || meta.version != AntColonySnapshotMeta::VERSION) + { + antSnapshotFailure(L"bad magic/version", meta.magic, meta.version); + return false; + } + if (meta.epoch != epoch) + { + antSnapshotFailure(L"epoch mismatch, file/expected", meta.epoch, epoch); + return false; + } + // A differently sized record or ANN would parse cleanly and produce a tree that is silently wrong. + if (meta.recordSizeBytes != sizeof(AntSolutionRecord) || meta.annPoolEntryBytes != sizeof(PackedAnn)) + { + antSnapshotFailure(L"layout mismatch, record/ann", meta.recordSizeBytes, meta.annPoolEntryBytes); + return false; + } + // The two sections after the meta must be exactly the size this build lays them out at, or the + // copies below would read them at the wrong offset. + if (meta.anchorRingBytes != sizeof(AnchorRing) || meta.exportSetBytes != sizeof(ExportSet)) + { + antSnapshotFailure(L"layout mismatch, anchors/export", meta.anchorRingBytes, meta.exportSetBytes); + return false; + } + if (meta.solutionCount > ANT_MAX_NODES_PER_EPOCH) + { + antSnapshotFailure(L"solutionCount exceeds the cap, count/cap", meta.solutionCount, ANT_MAX_NODES_PER_EPOCH); + return false; + } + // Cross-check against the node state restored alongside this file. A mismatch means the two + // snapshots are not from the same moment - loading the tree anyway would derive every root from + // a seed the rest of the network is not using. + if (!(meta.rootSeed == rootSeed)) + { + antSnapshotFailure(L"root seed does not match the restored node state, record/0", 0, 0); + return false; + } + if (meta.errorThreshold != errorThreshold) + { + antSnapshotFailure(L"threshold does not match the node, file/node", meta.errorThreshold, errorThreshold); + return false; + } + // Records hold absolute ticks; slotOf() maps them against initialTick, so a snapshot taken at a + // different base would resolve parent references to the wrong records. Refuse it. + if (meta.initialTick != initialTick) + { + antSnapshotFailure(L"initial tick does not match the node, file/node", meta.initialTick, initialTick); + return false; + } + + // Everything above only read the meta, so a refusal there leaves the colony untouched. From here + // on the state is being overwritten + reset(); + + copyMem(_anchors, headerBuffer + sizeof(meta), sizeof(AnchorRing)); + copyMem(_exportSet, headerBuffer + sizeof(meta) + sizeof(AnchorRing), sizeof(ExportSet)); + + // Read unconditionally and at the same sizing the save used, so an incomplete copy is refused + // here rather than booting a node with an empty tree. + const unsigned long long slots = antSnapshotSlotCount(meta.solutionCount); + const unsigned long long recordBytes = slots * sizeof(AntSolutionRecord); + const unsigned long long poolBytes = slots * sizeof(PackedAnn); + if (loadLargeFile(ANT_SNAPSHOT_RECORDS_FILENAME, recordBytes, (unsigned char*)_records, directory) + != (long long)recordBytes) + { + logToConsole(L"[ant-colony] failed to load snapshot records"); + reset(); + return false; + } + if (loadLargeFile(ANT_SNAPSHOT_POOL_FILENAME, poolBytes, (unsigned char*)_annPool, directory) + != (long long)poolBytes) + { + logToConsole(L"[ant-colony] failed to load snapshot pool"); + reset(); + return false; + } + if (_exportSet->count > ANT_EXPORT_MAX_SOLUTIONS) + { + antSnapshotFailure(L"export count exceeds the cap, count/cap", _exportSet->count, ANT_EXPORT_MAX_SOLUTIONS); + reset(); + return false; + } + for (unsigned int i = 0; i < _exportSet->count; i++) + { + const unsigned int slot = _exportSet->order[i]; + if (slot >= _exportSet->count) + { + antSnapshotFailure(L"export order out of range, position/slot", i, slot); + reset(); + return false; + } + for (unsigned int j = 0; j < i; j++) + { + if (_exportSet->order[j] == slot) + { + antSnapshotFailure(L"export order duplicate, position/slot", i, slot); + reset(); + return false; + } + } + } + + // The caller's values, which the checks above proved the file agrees with. + _rootSeed = rootSeed; + _errorThreshold = errorThreshold; + _solutionCount = meta.solutionCount; + _initialTick = initialTick; + + // Rebuild the intermediate data + if (!rebuildDerivedState()) + { + reset(); + return false; + } + return true; +} + +template +inline bool AntColony::rebuildDerivedState() +{ + // Hoisted out of the loop: unpacking each stored network to verify its hash needs somewhere to + // put it, and this path runs once at boot. + Ann annBuffer; + + for (unsigned int i = 0; i < _solutionCount; i++) + { + const AntSolutionRecord& rec = _records[i]; + + unsigned int selfSlot = 0; + if (!slotOf(rec.selfRef.tick, selfSlot)) + { + antSnapshotFailure(L"tick out of range, record/tick", i, rec.selfRef.tick); + return false; + } + // commit() writes the record and its network at the same index, and findIndexBySolutionRef + // and annOfNonRoot both rely on it + // annStateSlot point to the ANN that must have similar index to the record, unless the + // record was committed without one + const bool annMaterialised = (rec.annStateSlot == i); + if (!annMaterialised + && rec.annStateSlot != ANT_ANN_UNMATERIALISED + && rec.annStateSlot != ANT_ANN_MATERIALISING) + { + antSnapshotFailure(L"annStateSlot is not the record index, record/slot", i, rec.annStateSlot); + return false; + } + // A claim saved mid-walk is held by no thread in this process, so drop it back to + // unmaterialised rather than leaving the slot permanently unbuildable. + if (rec.annStateSlot == ANT_ANN_MATERIALISING) + { + _records[i].annStateSlot = ANT_ANN_UNMATERIALISED; + } + + // The freshness rule validateChild() applied at admission, re-checked here so a tampered + // snapshot cannot smuggle in a record that was never admissible. anchorTick seeds the score's + // RNG, so it is consensus-relevant. The record's own absolute tick is its publish tick. + const unsigned int publishTick = rec.selfRef.tick; + if (rec.anchorTick > publishTick + || publishTick - rec.anchorTick > ANT_PUBLISH_WINDOW_TICKS) + { + antSnapshotFailure(L"anchor tick outside the freshness window, record/anchorTick", i, rec.anchorTick); + return false; + } + + // Rebuild the tick index + AntTickSlot& tslot = _tickIndex[selfSlot]; + if (tslot.count == 0) + { + tslot.startIdx = i; + } + else if (tslot.startIdx + tslot.count != i) + { + antSnapshotFailure(L"record breaks tick contiguity, record/tick", i, rec.selfRef.tick); + return false; + } + tslot.count++; + + // The parent must be ROOT or an EARLIER record. The tick index built so far covers only + // records before this one, so a forward or self reference fails to resolve - which is what + // rejects a cycle. + const AntSolutionRecord* parentRec = nullptr; + if (!rec.parentRef.isRoot()) + { + const long long parentIdx = findIndexBySolutionRef(rec.parentRef); + if (parentIdx == ANT_INVALID_INDEX || (unsigned long long)parentIdx >= i) + { + antSnapshotFailure(L"parent not an earlier record, record/parentTick", i, rec.parentRef.tick); + return false; + } + parentRec = &_records[parentIdx]; + if (!(parentRec->pubkey == rec.pubkey)) + { + antSnapshotFailure(L"parent belongs to another identity, record", i, 0); + return false; + } + } + const unsigned int expectedDepth = (parentRec != nullptr) ? (parentRec->depth + 1) : 1; + if (rec.depth != expectedDepth) + { + antSnapshotFailure(L"depth does not match the parent, record/depth", i, rec.depth); + return false; + } + + // The two score rules validateChild() enforced when this record was admitted. A corrupt + // score would otherwise set a wrong bar for its own children. + if (rec.score > _errorThreshold) + { + antSnapshotFailure(L"score above the epoch threshold, record/score", i, rec.score); + return false; + } + if (parentRec != nullptr && rec.score >= parentRec->score) + { + antSnapshotFailure(L"score does not beat the parent, record/score", i, rec.score); + return false; + } + + // Re-derive the hash from the stored network. An unmaterialised record has none yet, and + // gets its hash when the walk that builds one publishes it. + if (annMaterialised) + { + _annPool[i].unpack(annBuffer.lut); + unsigned int annHash; + KangarooTwelve(&annBuffer, sizeof(annBuffer), &annHash, sizeof(annHash)); + if (annHash != rec.childAnnHash) + { + antSnapshotFailure(L"stored network does not match childAnnHash, record", i, 0); + return false; + } + } + + const AntDedupKey key{ rec.pubkey, rec.nonce, rec.parentRef }; + if (_dedup->contains(key)) + { + antSnapshotFailure(L"duplicate solution, record", i, 0); + return false; + } + if (_dedup->add(key) == QPI::NULL_INDEX) + { + antSnapshotFailure(L"dedup set full, record", i, 0); + return false; + } + + // Rebuild the _childHeadByMiner and _childHeadByParent + unsigned int prevHead = NO_SIBLING; + if (rec.parentRef.isRoot()) + { + _childHeadByMiner->get(rec.pubkey, prevHead); + _records[i].nextSiblingIdx = prevHead; + if (_childHeadByMiner->set(rec.pubkey, i) == QPI::NULL_INDEX) + { + antSnapshotFailure(L"miner index full, record", i, 0); + return false; + } + } + else + { + _childHeadByParent->get(rec.parentRef, prevHead); + _records[i].nextSiblingIdx = prevHead; + _childHeadByParent->set(rec.parentRef, i); + } + + // Restore the stats also + _stats.acceptedSolutions++; + if (rec.depth > _stats.treeDepthMax) + { + _stats.treeDepthMax = rec.depth; + } + } + + _stats.treeSizeCurrent = _solutionCount; + return true; +} diff --git a/src/mining/ant_colony/ant_colony_bpp9000.h b/src/mining/ant_colony/ant_colony_bpp9000.h new file mode 100644 index 00000000..da88d0ab --- /dev/null +++ b/src/mining/ant_colony/ant_colony_bpp9000.h @@ -0,0 +1,8 @@ +#pragma once + +#include "mining/ant_colony/ant_colony.h" +#include "score.h" + +// Binds the colony to bpp9000. This is the only place a concrete scorer is named, which is why +// ant_colony.h itself can stay free of score.h and everything it drags in. +using AntColonyBpp9000T = AntColony; diff --git a/src/mining/ant_colony/ant_pending_solutions.h b/src/mining/ant_colony/ant_pending_solutions.h new file mode 100644 index 00000000..b3658cb1 --- /dev/null +++ b/src/mining/ant_colony/ant_pending_solutions.h @@ -0,0 +1,434 @@ +#pragma once + +#include "platform/m256.h" +#include "platform/concurrency.h" +#include "platform/memory.h" +#include "mining/ant_colony/ant_colony.h" + +// Solutions waiting to be published as transactions signed by the node's own computors +// A queue is needed because a broadcast can be lost with nothing reporting it. An entry stores the +// tick its transaction was targeted at, if it is not on-chain by then, publish again. +struct AntPendingSolution +{ + m256i computorPublicKey; + m256i nonce; + SolutionRef parentRef; + unsigned int anchorTick; // ABSOLUTE. Bounds how long this entry is worth publishing. + unsigned int score; // computed at receipt; the publisher uses it without re-scoring +}; +static_assert(sizeof(AntPendingSolution) == 32 + 32 + 8 + 8, "AntPendingSolution unexpected padding"); + +class AntPendingSolutions +{ +public: + static constexpr unsigned int CAPACITY = 65536; + static_assert((CAPACITY & (CAPACITY - 1)) == 0, "CAPACITY must be a power of two"); + static constexpr unsigned int NO_ENTRY = 0xFFFFFFFFU; + + // publicationTick[] states. A positive value is the tick the transaction was targeted at, which + // is also the deadline to see it on-chain + static constexpr int NOT_SCHEDULED = 0; + static constexpr int RECORDED = -1; // observed on-chain; never publish again + static constexpr int OBSOLETE = -2; // can never land; stop occupying the retry slot + + struct Stats + { + unsigned long long received; + unsigned long long droppedNonCanonical; + unsigned long long droppedBadAnchor; // anchor in the future, or aged out of the ring + unsigned long long droppedParentUnknown; // parentRef names a node this node does not hold + unsigned long long droppedUnscorable; // the scorer returned no usable value + unsigned long long droppedUnacceptable; // scored, but the colony would reject it now + unsigned long long droppedDuplicate; + unsigned long long droppedFull; + unsigned long long published; + unsigned long long recorded; + unsigned long long obsoleteParentGone; + unsigned long long obsoleteExpired; + unsigned long long obsoleteGateRejected; + unsigned long long claimMismatch; + }; + + bool init() + { + setMem(this, sizeof(*this), 0); + if (!allocPoolWithErrorLog(L"AntPendingSolutions::_entries", + CAPACITY * sizeof(AntPendingSolution), (void**)&_entries, __LINE__)) + { + return false; + } + if (!allocPoolWithErrorLog(L"AntPendingSolutions::_publicationTick", + CAPACITY * sizeof(int), (void**)&_publicationTick, __LINE__)) + { + return false; + } + if (!allocPoolWithErrorLog(L"AntPendingSolutions::_index", + INDEX_CAPACITY * sizeof(unsigned int), (void**)&_index, __LINE__)) + { + return false; + } + reset(); + return true; + } + + void deinit() + { + if (_index) + { + freePool(_index); + } + if (_publicationTick) + { + freePool(_publicationTick); + } + if (_entries) + { + freePool(_entries); + } + _index = nullptr; + _publicationTick = nullptr; + _entries = nullptr; + } + + void reset() + { + ASSERT(_entries != nullptr); + LockGuard guard(_lock); + setMem(_entries, CAPACITY * sizeof(AntPendingSolution), 0); + setMem(_publicationTick, CAPACITY * sizeof(int), 0); + setMem(_index, INDEX_CAPACITY * sizeof(unsigned int), 0xFF); + _count = 0; + _nextFree = 0; + _touchedSlots = 0; + setMem(&_stats, sizeof(_stats), 0); + } + + // Ingress only counts what it drops, the caller decides what is worth dropping, because the + // reasons live where the colony can be consulted. + void noteReceived() + { + LockGuard guard(_lock); + _stats.received++; + } + void noteDroppedNonCanonical() + { + LockGuard guard(_lock); + _stats.droppedNonCanonical++; + } + void noteDroppedBadAnchor() + { + LockGuard guard(_lock); + _stats.droppedBadAnchor++; + } + void noteDroppedParentUnknown() + { + LockGuard guard(_lock); + _stats.droppedParentUnknown++; + } + void noteDroppedDuplicate() + { + LockGuard guard(_lock); + _stats.droppedDuplicate++; + } + void noteDroppedUnscorable() + { + LockGuard guard(_lock); + _stats.droppedUnscorable++; + } + void noteDroppedUnacceptable() + { + LockGuard guard(_lock); + _stats.droppedUnacceptable++; + } + + void getStats(Stats& outStats, unsigned int& outCount) const + { + LockGuard guard(_lock); + outStats = _stats; + outCount = _count; + } + + // Queue a solution for publication, called from request processors. + bool add(const m256i& computorPublicKey, const SolutionRef& parentRef, + unsigned int anchorTick, unsigned int score, const m256i& nonce) + { + LockGuard guard(_lock); + const unsigned int slot = indexSlotFor(computorPublicKey, parentRef, nonce); + unsigned int entryIdx = NO_ENTRY; + if (_index[slot] != INDEX_EMPTY) + { + // An OBSOLETE entry is not a duplicate + // Every other state is a real duplicate: NOT_SCHEDULED and scheduled are still live, and + // RECORDED already landed, so a resend would be rejected as a replay anyway. + if (_publicationTick[_index[slot]] != OBSOLETE) + { + _stats.droppedDuplicate++; + return false; + } + entryIdx = _index[slot]; + } + else + { + entryIdx = findFreeEntry(); + if (entryIdx == NO_ENTRY) + { + _stats.droppedFull++; + return false; + } + } + + AntPendingSolution& e = _entries[entryIdx]; + e.computorPublicKey = computorPublicKey; + e.nonce = nonce; + e.parentRef = parentRef; + e.anchorTick = anchorTick; + e.score = score; + _publicationTick[entryIdx] = NOT_SCHEDULED; + if (_index[slot] == INDEX_EMPTY) + { + _index[slot] = entryIdx; + _count++; + } + return true; + } + + // Pick the next solution this computor should publish, or NO_ENTRY. + // Anything already past its publish window is retired here rather than published + unsigned int selectForPublish(const m256i& computorPublicKey, unsigned int currentTick, + AntPendingSolution& outEntry) + { + LockGuard guard(_lock); + + unsigned int found = scanForPublish(computorPublicKey, currentTick, true); + if (found == NO_ENTRY) + { + found = scanForPublish(computorPublicKey, currentTick, false); + } + if (found != NO_ENTRY) + { + outEntry = _entries[found]; + } + return found; + } + + void markScheduled(unsigned int index, int publicationTick) + { + LockGuard guard(_lock); + if (index >= CAPACITY || _publicationTick[index] < 0) + { + return; + } + _publicationTick[index] = publicationTick; + _stats.published++; + } + + void markObsoleteParentGone(unsigned int index) + { + retire(index, _stats.obsoleteParentGone); + } + void markObsoleteExpired(unsigned int index) + { + retire(index, _stats.obsoleteExpired); + } + void markObsoleteGateRejected(unsigned int index) + { + retire(index, _stats.obsoleteGateRejected); + } + + void noteClaimMismatch() + { + LockGuard guard(_lock); + _stats.claimMismatch++; + } + + void markRecorded(const m256i& computorPublicKey, const SolutionRef& parentRef, const m256i& nonce) + { + LockGuard guard(_lock); + + const unsigned int slot = indexSlotFor(computorPublicKey, parentRef, nonce); + if (_index[slot] != INDEX_EMPTY) + { + _publicationTick[_index[slot]] = RECORDED; + _stats.recorded++; + return; + } + + const unsigned int entryIdx = findFreeEntry(); + if (entryIdx == NO_ENTRY) + { + // Nothing to reclaim + return; + } + + AntPendingSolution& e = _entries[entryIdx]; + e.computorPublicKey = computorPublicKey; + e.nonce = nonce; + e.parentRef = parentRef; + e.anchorTick = 0; + e.score = 0; + _publicationTick[entryIdx] = RECORDED; + _index[slot] = entryIdx; + _count++; + _stats.recorded++; + } + +private: + static constexpr unsigned int INDEX_CAPACITY = 2 * CAPACITY; + // Not INDEX_EMPTY: network_messages/assets.h defines that as a macro, and this header is included + // after it in qubic.cpp. + static constexpr unsigned int INDEX_EMPTY = 0xFFFFFFFFU; + + // A slot is free if it has never been used or if its entry is finished. Reuse is IN PLACE: + // nothing is ever moved, so an index the tick processor is holding across the publish gate + // stays valid. + unsigned int findFreeEntry() + { + for (unsigned int i = 0; i < CAPACITY; i++) + { + const unsigned int idx = (_nextFree + i) & (CAPACITY - 1); + if (isZero(_entries[idx].computorPublicKey)) + { + claim(idx); + return idx; + } + if (_publicationTick[idx] == RECORDED || _publicationTick[idx] == OBSOLETE) + { + indexRemove(_entries[idx]); + _count--; + claim(idx); + return idx; + } + } + return NO_ENTRY; + } + + void claim(unsigned int idx) + { + _nextFree = (idx + 1) & (CAPACITY - 1); + if (idx + 1 > _touchedSlots) + { + _touchedSlots = idx + 1; + } + } + + unsigned int scanForPublish(const m256i& computorPublicKey, unsigned int currentTick, bool retries) + { + for (unsigned int i = 0; i < _touchedSlots; i++) + { + const int state = _publicationTick[i]; + if (retries) + { + if (state <= NOT_SCHEDULED || state > (int)currentTick) + { + continue; + } + } + else if (state != NOT_SCHEDULED) + { + continue; + } + if (isZero(_entries[i].computorPublicKey) || !(_entries[i].computorPublicKey == computorPublicKey)) + { + continue; + } + if (currentTick - _entries[i].anchorTick > ANT_PUBLISH_WINDOW_TICKS) + { + retireLocked(i, _stats.obsoleteExpired); + continue; + } + return i; + } + return NO_ENTRY; + } + + void retire(unsigned int index, unsigned long long& counter) + { + LockGuard guard(_lock); + retireLocked(index, counter); + } + + void retireLocked(unsigned int index, unsigned long long& counter) + { + if (index >= CAPACITY || _publicationTick[index] < 0) + { + return; + } + _publicationTick[index] = OBSOLETE; + counter++; + } + + static AntDedupKey keyOf(const m256i& computorPublicKey, const SolutionRef& parentRef, + const m256i& nonce) + { + AntDedupKey key; + key.pubkey = computorPublicKey; + key.nonce = nonce; + key.parentRef = parentRef; + return key; + } + + // Reads only the low words would put every nonce differing above them in one slot, which is the + // clustering the index exists to avoid. + static unsigned int hashOf(const m256i& computorPublicKey, const SolutionRef& parentRef, + const m256i& nonce) + { + const AntDedupKey key = keyOf(computorPublicKey, parentRef, nonce); + unsigned long long digest; + KangarooTwelve(&key, sizeof(key), &digest, sizeof(digest)); + return (unsigned int)(digest & (INDEX_CAPACITY - 1)); + } + + // Returns the slot holding this key, or the first free slot if it is absent. Linear probing over + // a table at most half full, so the walk always terminates. + unsigned int indexSlotFor(const m256i& computorPublicKey, const SolutionRef& parentRef, + const m256i& nonce) const + { + unsigned int slot = hashOf(computorPublicKey, parentRef, nonce); + for (unsigned int probe = 0; probe < INDEX_CAPACITY; probe++) + { + const unsigned int e = _index[slot]; + if (e == INDEX_EMPTY) + { + return slot; + } + if (keyOf(_entries[e].computorPublicKey, _entries[e].parentRef, _entries[e].nonce) + == keyOf(computorPublicKey, parentRef, nonce)) + { + return slot; + } + slot = (slot + 1) & (INDEX_CAPACITY - 1); + } + return 0; + } + + void indexRemove(const AntPendingSolution& entry) + { + const unsigned int slot = indexSlotFor(entry.computorPublicKey, entry.parentRef, entry.nonce); + if (_index[slot] == INDEX_EMPTY) + { + return; + } + _index[slot] = INDEX_EMPTY; + + // Re-place the run that followed it, or a probe would stop early at the hole just made. + unsigned int next = (slot + 1) & (INDEX_CAPACITY - 1); + while (_index[next] != INDEX_EMPTY) + { + const unsigned int moved = _index[next]; + _index[next] = INDEX_EMPTY; + const unsigned int target = indexSlotFor(_entries[moved].computorPublicKey, + _entries[moved].parentRef, _entries[moved].nonce); + _index[target] = moved; + next = (next + 1) & (INDEX_CAPACITY - 1); + } + } + + AntPendingSolution* _entries; + int* _publicationTick; + unsigned int* _index; + unsigned int _count; + unsigned int _nextFree; + unsigned int _touchedSlots; + Stats _stats; + mutable volatile char _lock; +}; diff --git a/src/mining/mining.h b/src/mining/mining.h index 6b7c7372..2bd63b17 100644 --- a/src/mining/mining.h +++ b/src/mining/mining.h @@ -10,6 +10,11 @@ #include +// Miners tracked in the ranking table that feeds computor selection. A hard cap rather than mere +// sizing: once full, a newcomer is admitted only if it outranks the current worst entry. Lives here +// rather than in qubic.cpp so mining headers can size per-miner structures against it. +#define MAX_NUMBER_OF_MINERS 8192 + static unsigned int getTickInDogeBroadcastCycle() { #ifdef REAL_NODE @@ -353,3 +358,41 @@ struct CustomMiningStats }; static CustomMiningStats gDogeMiningStats; + +// Ant colony solution transaction +constexpr int ANT_COLONY_MINING_SOLUTION_INPUT_TYPE = 12; +struct AntColonyMiningSolutionTransaction : public Transaction +{ + static constexpr unsigned char transactionType() + { + return ANT_COLONY_MINING_SOLUTION_INPUT_TYPE; + } + + static constexpr long long minAmount() + { + return SOLUTION_SECURITY_DEPOSIT; // same anti-spam deposit as legacy + } + + static constexpr unsigned short minInputSize() + { + return sizeof(parentTick) + sizeof(parentSolutionIndexInTick) + sizeof(anchorTick) + sizeof(claimedScore) + sizeof(nonce); // 4 + 4 + 4 + 4 + 32 = 48 bytes + } + + static bool isSolutionTransaction(const Transaction* tx) + { + return isZero(tx->destinationPublicKey) + && tx->inputType == transactionType() + && tx->amount >= minAmount() + && tx->inputSize == minInputSize(); + } + + unsigned int parentTick; // ABSOLUTE tick of the parent ref + unsigned int parentSolutionIndexInTick; // dense within-tick index of the parent ref + unsigned int anchorTick; // ABSOLUTE tick whose digest the solution anchored to (RNG seed + freshness) + // The score the submitter claims this solution reaches. The deposit is refunded only when it + // matches the score the node computes + unsigned int claimedScore; + m256i nonce; + unsigned char signature[SIGNATURE_SIZE]; +}; +static_assert(sizeof(AntColonyMiningSolutionTransaction) == sizeof(Transaction) + 4 + 4 + 4 + 4 + 32 + SIGNATURE_SIZE, "AntColonyMiningSolutionTransaction unexpected padding"); diff --git a/src/mining/score_bpp9000.h b/src/mining/score_bpp9000.h index b7f308b4..40468c8c 100644 --- a/src/mining/score_bpp9000.h +++ b/src/mining/score_bpp9000.h @@ -45,6 +45,15 @@ struct ScoreBpp9000 static_assert(lutSize <= lutStride, "LUT rows must fit the padded stride"); + // K is a real degree of freedom here: the walk restores K = nonce[2] as its explore-step count + static bool isCanonicalAntNonce(const unsigned char* nonce) + { + return (getAlgoType(nonce) == AlgoType::Bpp9000) + && (nonce[1] >= 1) + && (nonce[1] <= MAX_LUT_ENTRIES_PER_STEP) + && (nonce[2] <= numberOfMutations); + } + // random2 draw sizes padded up to a multiple of 64 bytes; leading bytes bit-exact with reference. static constexpr unsigned long long lutInitBytes = maxNumberOfNeurons * lutSize; static constexpr unsigned long long lutInitPaddedBytes = ((lutInitBytes + 63) / 64) * 64; @@ -79,13 +88,26 @@ struct ScoreBpp9000 kEvolution, }; - // Rollback/snapshot state: just the per-neuron LUT. + // An ANN is its per-neuron LUT: maxNumberOfNeurons rows of lutSize entries + // resourceTestingDigest, written to snapshots, sent to miners... struct ANN + { + unsigned char lut[maxNumberOfNeurons * lutSize]; + }; + + // padding LUT for a SIMD + struct PaddedLut { alignas(64) unsigned char lut[maxNumberOfNeurons * lutStride]; }; - ANN currentANN; - ANN prevANN; + + PaddedLut currentANN; + PaddedLut prevANN; + + // LUT that produced the score returned by the walk, in working layout. The ant colony stores this + // as the child's inherited state, so a child branches from the best-ever LUT rather than the last + // one walked; read it with getBestANN(). + PaddedLut bestANN; struct InitValue { @@ -403,7 +425,8 @@ struct ScoreBpp9000 #endif } - // Sliding-window self-clocked score via the window-batched SIMD kernel (AVX-512 or AVX2, bit-exact). + // Sliding-window self-clocked score via the window-batched SIMD kernel + // Apply on the curANN unsigned int score() { // PROFILE_NAMED_SCOPE("bpp9000:score"); @@ -972,44 +995,102 @@ struct ScoreBpp9000 currentANN.lut[storageIdx] = newTrit; } - // Seed the ANN: root LUT from the pubkey alone (each computor's fixed root); mutation seeds from - // pubkey+nonce (nonce[0..2] are the algo/L/K knobs, excluded from the RNG). Returns the start score. - unsigned int initializeANN(const unsigned char* publicKey, const unsigned char* nonce, const unsigned char* pRandom2Pool) + // Derive the root LUT material + void deriveRootLut(const unsigned char* publicKey, const unsigned char* pRandom2Pool) { - // PROFILE_NAMED_SCOPE("bpp9000:initializeANN"); unsigned char rootHash[32]; KangarooTwelve(publicKey, 32, rootHash, 32); random2(rootHash, pRandom2Pool, (unsigned char*)&initValue.lutInit, lutInitPaddedBytes); + } + // Derive the mutation-walk seeds. nonce[0..2] are the algo/L/K knobs and stay excluded from the RNG, + // so K and L can be chosen freely without reseeding the walk. anchorTickDigest == nullptr for the + // standalone walk; the ant colony binds a child's walk to the tick it anchors on. + void deriveMutationSeeds( + const unsigned char* publicKey, + const unsigned char* nonce, + const unsigned char* anchorTickDigest, + const unsigned char* pRandom2Pool) + { unsigned char searchHash[32]; - unsigned char combined[64]; + unsigned char combined[96]; copyMem(combined, publicKey, 32); copyMem(combined + 32, nonce, 32); combined[32] = 0; combined[33] = 0; combined[34] = 0; - KangarooTwelve(combined, 64, searchHash, 32); + unsigned int combinedSize = 64; + if (anchorTickDigest != nullptr) + { + copyMem(combined + 64, anchorTickDigest, 32); + combinedSize = 96; + } + KangarooTwelve(combined, combinedSize, searchHash, 32); random2(searchHash, pRandom2Pool, (unsigned char*)&initValue.mutationSeed, mutationSeedPaddedBytes); + } + + // Store the LUT densely by updated-neuron position k (row k): row k holds neuron + // updatedNeuronIndices[k]'s LUT. RNG draw into initValue.lutInit unchanged (bit-exact). + void applyRootLut(PaddedLut& target) + { + // The loop below writes only rows [0, numberOfUpdatedNeurons) columns [0, lutSize), and the + // SIMD path loads whole rows, so the rest has to be initialised rather than left as whatever + // the buffer previously held. + setMem(&target, sizeof(target), 0); - // Store the LUT densely by updated-neuron position k (row k): row k holds neuron - // updatedNeuronIndices[k]'s LUT. RNG draw into initValue.lutInit unchanged (bit-exact). for (unsigned long long k = 0; k < numberOfUpdatedNeurons; ++k) { const unsigned long long n = updatedNeuronIndices[k]; for (unsigned long long line = 0; line < lutSize; ++line) { - currentANN.lut[k * lutStride + line] = (unsigned char)(initValue.lutInit[n * lutSize + line] % 3); + target.lut[k * lutStride + line] = (unsigned char)(initValue.lutInit[n * lutSize + line] % 3); } } + } + + // Working layout to ANN remove the stride padding. + void compact(const PaddedLut& src, ANN& out) const + { + for (unsigned long long k = 0; k < maxNumberOfNeurons; ++k) + { + copyMem(out.lut + k * lutSize, src.lut + k * lutStride, lutSize); + } + } + + // Restores the stride and zeroes the padding. + void expand(const ANN& src, PaddedLut& out) const + { + setMem(&out, sizeof(out), 0); + for (unsigned long long k = 0; k < maxNumberOfNeurons; ++k) + { + copyMem(out.lut + k * lutStride, src.lut + k * lutSize, lutSize); + } + } + + // The LUT behind the score the last walk returned. + void getBestANN(ANN& out) const + { + compact(bestANN, out); + } + + // Seed the ANN: root LUT from the pubkey alone (each computor's fixed root); mutation seeds from + // pubkey+nonce (nonce[0..2] are the algo/L/K knobs, excluded from the RNG). Returns the start score. + unsigned int initializeANN( + const unsigned char* publicKey, + const unsigned char* nonce, + const unsigned char* pRandom2Pool) + { + // PROFILE_NAMED_SCOPE("bpp9000:initializeANN"); + deriveRootLut(publicKey, pRandom2Pool); + deriveMutationSeeds(publicKey, nonce, nullptr, pRandom2Pool); + applyRootLut(currentANN); return score(); } - // Anti-attractor search: L mutations/step; accept worse-or-equal for the first K steps (explore), - // then better-or-equal (exploit); one-step rollback; keep and return the best score found. - unsigned int computeScore(const unsigned char* publicKey, const unsigned char* nonce, const unsigned char* pRandom2Pool) + // Miner-chosen number of LUT entries rewritten per step, clamped to the verifiable range. + static unsigned int lutEntriesPerStep(const unsigned char* nonce) { - // PROFILE_NAMED_SCOPE("bpp9000:computeScore"); unsigned int L = nonce[1]; if (L < 1) { @@ -1019,11 +1100,17 @@ struct ScoreBpp9000 { L = MAX_LUT_ENTRIES_PER_STEP; } - // Explore disabled pre-ant-colony (K=0); restore K = nonce[2] when ants return. - const unsigned long long K = 0; + return L; + } - unsigned int cur = initializeANN(publicKey, nonce, pRandom2Pool); + // Anti-attractor walk starting from the LUT already in currentANN: L mutations/step; accept + // worse-or-equal for the first K steps (explore), then better-or-equal (exploit); one-step rollback. + // Returns the best score found and leaves the LUT that produced it in bestANN. + unsigned int computeScoreFromCurrent(unsigned int L, unsigned long long K, unsigned int startScore) + { + unsigned int cur = startScore; unsigned int best = cur; + copyMem(&bestANN, ¤tANN, sizeof(bestANN)); for (unsigned long long s = 0; s < numberOfMutations; ++s) { @@ -1058,11 +1145,68 @@ struct ScoreBpp9000 if (cur < best) { best = cur; + copyMem(&bestANN, ¤tANN, sizeof(bestANN)); } } return best; } + // Anti-attractor search: L mutations/step; accept worse-or-equal for the first K steps (explore), + // then better-or-equal (exploit); one-step rollback; keep and return the best score found. + unsigned int computeScore( + const unsigned char* publicKey, + const unsigned char* nonce, + const unsigned char* pRandom2Pool) + { + // PROFILE_NAMED_SCOPE("bpp9000:computeScore"); + const unsigned int L = lutEntriesPerStep(nonce); + // Explore disabled for the standalone algorithm (K=0); the ant colony passes K = nonce[2]. + const unsigned long long K = 0; + + const unsigned int cur = initializeANN(publicKey, nonce, pRandom2Pool); + + return computeScoreFromCurrent(L, K, cur); + } + + // Ant colony: the network every one of an identity's lineages starts from. Written to a buffer the + // caller owns, so two roots can be derived on one engine without the first silently becoming the + // second - a child scored against the wrong root would differ only in resourceTestingDigest. + // Uses currentANN as its working buffer, so it destroys whatever the engine was holding. Callers + // derive a root and then score from it, which overwrites currentANN anyway. + void deriveRootANN(const unsigned char* publicKey, const unsigned char* pRandom2Pool, ANN& out) + { + deriveRootLut(publicKey, pRandom2Pool); + applyRootLut(currentANN); + compact(currentANN, out); + } + + // Ant colony: score a child by inheriting the parent's LUT and walking it with the child's own seeds + unsigned int computeScoreFromParent( + const ANN& parentANN, + const unsigned char* publicKey, + const unsigned char* nonce, + const unsigned char* anchorTickDigest, + const unsigned char* pRandom2Pool) + { + // The canonical rule + if (!isCanonicalAntNonce(nonce)) + { + return INVALID_SCORE_VALUE; + } + + // Get the ANN from parent, also init the new mutation starting point + expand(parentANN, currentANN); + deriveMutationSeeds(publicKey, nonce, anchorTickDigest, pRandom2Pool); + + // Both knobs are already in range: the check above is what puts them there. + const unsigned int L = lutEntriesPerStep(nonce); + const unsigned long long K = nonce[2]; + + const unsigned int cur = score(); + + return computeScoreFromCurrent(L, K, cur); + } + int getLastOutput(unsigned char* requestedOutput, int requestedSizeInBytes) { return 0; diff --git a/src/mining/score_engine.h b/src/mining/score_engine.h index d6752a80..a95cb048 100644 --- a/src/mining/score_engine.h +++ b/src/mining/score_engine.h @@ -61,6 +61,20 @@ struct ScoreEngine } } + // Each engine owns its canonical ant-nonce rule; this switch is the algorithm seam, so ingress + // code stays algorithm-agnostic. Neuraxon is reserved and not ant-minable, so no nonce in its + // slot is canonical. + static bool isCanonicalAntNonce(const unsigned char* nonce) + { + switch (getAlgoType(nonce)) + { + case AlgoType::Bpp9000: + return ScoreBpp9000::isCanonicalAntNonce(nonce); + default: + return false; + } + } + // returns last computed output neurons of the active bpp9000 slot m256i getLastOutput() { diff --git a/src/mining/trit_pack.h b/src/mining/trit_pack.h new file mode 100644 index 00000000..bd3d86ef --- /dev/null +++ b/src/mining/trit_pack.h @@ -0,0 +1,51 @@ +#pragma once + +// Ternary storage: values {0,1,2} at 2 bits each. +// +// Qubic's mining networks store their genome as trits +// One byte per trit wastes six bits of eight; two bits per trit cuts the stored genome to a quarter. +// Callers hash and transmit the unpacked bytes + +namespace score_engine +{ + +template +struct PackedTrits +{ + static_assert(GROUPS > 0, "need at least one group"); + static_assert(TRITS_PER_GROUP > 0, "a group needs at least one trit"); + static_assert(TRITS_PER_GROUP * 2 <= 64, "a group must fit at 2 bits per trit in one uint64"); + + static constexpr unsigned long long groupCount = GROUPS; + static constexpr unsigned long long tritsPerGroup = TRITS_PER_GROUP; + static constexpr unsigned long long tritCount = GROUPS * TRITS_PER_GROUP; + + unsigned long long word[GROUPS]; + + void pack(const unsigned char* src) + { + for (unsigned long long g = 0; g < GROUPS; g++) + { + unsigned long long packed = 0; + for (unsigned long long i = 0; i < TRITS_PER_GROUP; i++) + { + packed |= ((unsigned long long)(src[g * TRITS_PER_GROUP + i] & 3u)) << (i * 2); + } + word[g] = packed; + } + } + + void unpack(unsigned char* dst) const + { + for (unsigned long long g = 0; g < GROUPS; g++) + { + const unsigned long long packed = word[g]; + for (unsigned long long i = 0; i < TRITS_PER_GROUP; i++) + { + dst[g * TRITS_PER_GROUP + i] = (unsigned char)((packed >> (i * 2)) & 3ull); + } + } + } +}; + +} diff --git a/src/network_messages/all.h b/src/network_messages/all.h index edd8e66e..21424292 100644 --- a/src/network_messages/all.h +++ b/src/network_messages/all.h @@ -17,3 +17,4 @@ #include "transactions.h" #include "system_info.h" #include "revenue_data.h" +#include "ant_colony_message.h" diff --git a/src/network_messages/ant_colony_message.h b/src/network_messages/ant_colony_message.h new file mode 100644 index 00000000..98b123eb --- /dev/null +++ b/src/network_messages/ant_colony_message.h @@ -0,0 +1,177 @@ +#pragma once + +#include "common_def.h" + +// Asks for the parents one identity can branch a child from. Scoped by pubkey because a child must +// name a parent in its OWN tree - validate() rejects anything else with RejectWrongTree - +// Operator-signed: the request payload is followed by SIGNATURE_SIZE bytes signed by +// operatorPublicKey. Signature only, with no monotonic nonce. A nonce exists to make an operator +// ACTION execute exactly once; replaying a read just costs a duplicate answer, while consuming the +// nonce would put a polling miner in contention with every other operator command. +// +// Paginated via fromIndex / nextIndex. +struct RequestAntIdentityTree +{ + // Whose tree to report. Usually the caller's own. + m256i pubkey; + // Record index to resume scanning from (0 on the first call). + unsigned int fromIndex; + unsigned int padding; + static constexpr unsigned char type() + { + return REQUEST_ANT_IDENTITY_TREE; + } +}; +static_assert(sizeof(RequestAntIdentityTree) == 40, "RequestAntIdentityTree unexpected size"); + +// A pool miner hands its computor a solution over BroadcastMessage(MESSAGE_TYPE_ANT_SOLUTION); this +// is the payload that follows the header. +struct AntSolutionBroadcastPayload +{ + unsigned int parentTick; // ABSOLUTE + unsigned int parentSolutionIndexInTick; + unsigned int anchorTick; // ABSOLUTE + unsigned int claimedScore; + m256i nonce; +}; +static_assert(sizeof(AntSolutionBroadcastPayload) == 48, "AntSolutionBroadcastPayload unexpected size"); + +// Max identity-tree nodes returned per response. Miners page through the +// rest via the nextIndex cursor. +constexpr unsigned int ANT_IDENTITY_TREE_NODES_PER_RESPONSE = 64; + +// Max records scanned per request +constexpr unsigned int ANT_IDENTITY_TREE_SCAN_BUDGET = 1024; + +// One stored node of the requested identity's tree. selfTick/selfSolutionIndexInTick is the +// ref a child sets as its own parentRef to extend this node; parentTick/parentSolutionIndexInTick +// is this node's OWN parent - (0, 0xFFFFFFFF) means the root - so paging every node of a pubkey +// reconstructs the whole tree, edges included, without fetching any network bytes. +// The score is an error count, so smaller is better: a child must score strictly below score. +// childCount is how many children this node already holds, capped at ANT_MAX_CHILDREN_PER_PARENT; at +// the cap it takes no more children (0 for the cap means unbound). +struct AntIdentityTreeNode +{ + unsigned int selfTick; + unsigned int selfSolutionIndexInTick; + unsigned int parentTick; + unsigned int parentSolutionIndexInTick; + unsigned int score; + unsigned int childCount; + unsigned int anchorTick; // this node's own anchor tick number (ABSOLUTE) + unsigned int depth; +}; +static_assert(sizeof(AntIdentityTreeNode) == 32, "AntIdentityTreeNode unexpected size"); + +// Metadata header only; followed by count * AntIdentityTreeNode (count * itemSize +// bytes). itemSize lets the receiver validate the payload without hardcoding the +// entry size. +struct RespondAntIdentityTreeHeader +{ + // Number of AntIdentityTreeNode entries that follow this header. + unsigned int count; + // Size in bytes of one AntIdentityTreeNode entry. + unsigned int itemSize; + // Resume cursor for the next request; 0 means no more records. + unsigned int nextIndex; + static constexpr unsigned char type() + { + return RESPOND_ANT_IDENTITY_TREE; + } +}; +static_assert(sizeof(RespondAntIdentityTreeHeader) == 12, "RespondAntIdentityTreeHeader unexpected size"); + +// The largest an identity-tree response can be, the header followed by a full page of entries +struct AntIdentityTreeResponse +{ + RespondAntIdentityTreeHeader header; + AntIdentityTreeNode items[ANT_IDENTITY_TREE_NODES_PER_RESPONSE]; +}; +static_assert(sizeof(AntIdentityTreeResponse) + == sizeof(RespondAntIdentityTreeHeader) + + ANT_IDENTITY_TREE_NODES_PER_RESPONSE * sizeof(AntIdentityTreeNode), + "AntIdentityTreeResponse must have no padding between the header and the items"); + +// RespondAntParentAnnHeader.status values. +constexpr unsigned char ANT_PARENT_ANN_STATUS_OK = 0; // ANN bytes follow the header +constexpr unsigned char ANT_PARENT_ANN_STATUS_NOT_FOUND = 1; // parentRef has no record +constexpr unsigned char ANT_PARENT_ANN_STATUS_IS_ROOT = 2; // ROOT_REF; no ANN payload - miner derives its own per-identity root + +// ONE tree node's stored network, named by parentRef - the ANN state a miner mutates to extend +// that node. The tree itself is listed by the identity-tree query; this fetches the material for a +// single chosen parent. +// Operator-signed: the request payload is followed by SIGNATURE_SIZE bytes signed by +// operatorPublicKey +struct RequestAntParentAnn +{ + unsigned int parentRefTick; + unsigned int parentRefSolutionIndexInTick; + static constexpr unsigned char type() + { + return REQUEST_ANT_PARENT_ANN; + } +}; +static_assert(sizeof(RequestAntParentAnn) == 8, "RequestAntParentAnn unexpected size"); + +// Metadata header, when status is Ok, annSizeBytes bytes of CANONICAL ANN follow it - one trit per +// byte, the form the scorer consumes, so the receiver does no unpacking. annSizeBytes is 0 for every +// other status. Kept ANN-agnostic here to avoid a heavy include; the receiver reads the trailing +// blob by annSizeBytes. +struct RespondAntParentAnnHeader +{ + unsigned int parentRefTick; + unsigned int parentRefSolutionIndexInTick; + // Bytes of canonical ANN that follow this header: ANN LUT size when status is Ok, 0 for every other + // status. + unsigned int annSizeBytes; + unsigned char status; + unsigned char padding[3]; + static constexpr unsigned char type() + { + return RESPOND_ANT_PARENT_ANN; + } +}; +static_assert(sizeof(RespondAntParentAnnHeader) == 16, "RespondAntParentAnnHeader unexpected size"); + +struct RequestAntEpochContext +{ + static constexpr unsigned char type() + { + return REQUEST_ANT_EPOCH_CONTEXT; + } +}; + +// Per-epoch ant-colony parameters a miner needs to start building solutions: +// the score threshold, the freshness window, the epoch's root seed, pool occupancy, +// and the per-parent child cap. +// The anchor digest is not included; a miner derives it from the anchor tick's TickData +// (REQUEST_TICK_DATA): transactionDigest = K12(TickData), then K12(anchorTick || transactionDigest). +#pragma pack(push, 1) +struct RespondAntEpochContext +{ + // The epoch-start spectrum digest + m256i spectrumDigest; + // confirm its task file matches the one the node scores against. + m256i topologyHash; + m256i dataHash; + // score threshold for this epoch + unsigned int threshold; + // ANT_PUBLISH_WINDOW_TICKS: publish within this many ticks of the anchor. + unsigned int freshnessWindow; + // accepted solutions so far this epoch + unsigned int solutionCount; + // free slots in the live ANN pool + unsigned int freeAnnSlotsCount; + // ANT_MAX_CHILDREN_PER_PARENT: max children a parent takes; 0 = unbound + unsigned int maxChildrenPerParent; + // epoch this context is for + unsigned short epoch; + unsigned short padding; + + static constexpr unsigned char type() + { + return RESPOND_ANT_EPOCH_CONTEXT; + } +}; +#pragma pack(pop) +static_assert(sizeof(RespondAntEpochContext) == 120, "RespondAntEpochContext unexpected size"); diff --git a/src/network_messages/broadcast_message.h b/src/network_messages/broadcast_message.h index fe4c6c5d..a06a52e9 100644 --- a/src/network_messages/broadcast_message.h +++ b/src/network_messages/broadcast_message.h @@ -5,6 +5,7 @@ #define MESSAGE_TYPE_SOLUTION 0 #define MESSAGE_TYPE_CUSTOM_MINING_TASK 1 #define MESSAGE_TYPE_CUSTOM_MINING_SOLUTION 2 +#define MESSAGE_TYPE_ANT_SOLUTION 3 // TODO: documentation needed: // "A General Message type used to send/receive messages from/to peers." -> right? diff --git a/src/network_messages/network_message_type.h b/src/network_messages/network_message_type.h index a9b3993b..e297d37b 100644 --- a/src/network_messages/network_message_type.h +++ b/src/network_messages/network_message_type.h @@ -51,6 +51,12 @@ enum NetworkMessageType : unsigned char BROADCAST_CUSTOM_MINING_SOLUTION = 69, REQUEST_REVENUE_DATA = 70, RESPOND_REVENUE_DATA = 71, + REQUEST_ANT_IDENTITY_TREE = 72, + RESPOND_ANT_IDENTITY_TREE = 73, + REQUEST_ANT_PARENT_ANN = 74, + RESPOND_ANT_PARENT_ANN = 75, + REQUEST_ANT_EPOCH_CONTEXT = 76, + RESPOND_ANT_EPOCH_CONTEXT = 77, ORACLE_MACHINE_QUERY = 190, // only on communication channel Core node <-> OM node ORACLE_MACHINE_REPLY = 191, // only on communication channel Core node <-> OM node OC_MACHINE_INVOCATION = 192, // only on communication channel Core node <-> OC machine diff --git a/src/platform/concurrency.h b/src/platform/concurrency.h index 81518320..41670cbe 100644 --- a/src/platform/concurrency.h +++ b/src/platform/concurrency.h @@ -309,8 +309,12 @@ struct LockGuard #ifdef _MSC_VER static_assert(sizeof(long) == 4, "Size of long for _InterlockedExchange is 4 bytes"); #define ATOMIC_STORE32(target, val) _InterlockedExchange((volatile long*)&target, val) +#define ATOMIC_LOAD32(target) _InterlockedCompareExchange((volatile long*)&target, 0, 0) #else #define ATOMIC_STORE32(target, val) _InterlockedExchange((volatile int*)&target, val) +// A real load, not a CAS: routing this through the _InterlockedCompareExchange shim would issue +// an 8-byte operation on a 4-byte field, since long is 8 bytes here. +#define ATOMIC_LOAD32(target) __atomic_load_n((volatile unsigned int*)&(target), __ATOMIC_SEQ_CST) #endif #define ATOMIC_INC64(target) _InterlockedIncrement64(&target) #define ATOMIC_AND64(target, val) _InterlockedAnd64(&target, val) diff --git a/src/platform/file_io.h b/src/platform/file_io.h index 8d710331..07b13273 100644 --- a/src/platform/file_io.h +++ b/src/platform/file_io.h @@ -353,6 +353,19 @@ static bool removeDir(CHAR16* dirName) #endif } +static bool renameDir(CHAR16* fromDirName, CHAR16* toDirName) +{ +#ifdef NO_UEFI + ASSERT(isMainProcessor()); + std::error_code error; + std::filesystem::rename(getHostPath(fromDirName), getHostPath(toDirName), error); + return !error; +#else + logToConsole(L"renameDir is not supported in UEFI mode"); + return false; +#endif +} + static long long load(const CHAR16* fileName, unsigned long long totalSize, unsigned char* buffer, const CHAR16* directory = NULL) { #ifdef NO_UEFI diff --git a/src/public_settings.h b/src/public_settings.h index e2c408c6..8284b552 100644 --- a/src/public_settings.h +++ b/src/public_settings.h @@ -39,6 +39,9 @@ #endif #define SCORE_CACHE_COLLISION_RETRIES 20 // number of retries to find entry in cache in case of hash collision +// Persist the ant-colony replay cache (mirror of USE_SCORE_CACHE for the standalone score cache). +#define ANT_USE_SCORE_CACHE 1 + // Number of ticks from prior epoch that are kept after seamless epoch transition. These can be requested after transition. #define TICKS_TO_KEEP_FROM_PRIOR_EPOCH 100 @@ -135,6 +138,13 @@ static wchar_t REVENUE_DATA_END_OF_EPOCH_FILE_NAME[] = L"revenue_data.eoe"; static wchar_t REVENUE_DATA_SNAPSHOT_FILE_NAME[] = L"revenue_data.???"; static wchar_t MULTIDIM_REVENUE_SNAPSHOT_FILE_NAME[] = L"revenue_data_multi.???"; static wchar_t MULTIDIM_REVENUE_END_OF_EPOCH_FILE_NAME[] = L"revenue_data_multi.eoe"; +// Ant colony files. The header file carries the meta, the anchor ring and the export set together +static wchar_t ANT_SNAPSHOT_HEADER_FILENAME[] = L"snapshotAntColonyHeader.???"; +static wchar_t ANT_SNAPSHOT_RECORDS_FILENAME[] = L"snapshotAntColonyRecords.???"; +static wchar_t ANT_SNAPSHOT_POOL_FILENAME[] = L"snapshotAntColonyPool.???"; +static wchar_t ANT_COLONY_REPLAY_CACHE_FILENAME[] = L"antColonyReplayCache.???"; +static wchar_t ANT_COLONY_SOLUTIONS_EOE_FILENAME[] = L"antColonySolutions.eoe"; +static wchar_t ANT_SOL_FLAG_FILE_NAME[] = L"snapshotAntSolutionFlag"; // Neuraxon (even-nonce slot) - reserved for a future algorithm, not yet implemented. static constexpr unsigned long long NEURAXON_NUMBER_OF_INPUT_NEURONS = 1; @@ -166,7 +176,39 @@ static constexpr unsigned long long BPP9000_NUMBER_OF_MUTATIONS = 100; // Number of graded windows. The score is an error count in [0, BPP9000_NUMBER_OF_WINDOWS], smaller is // better, and a solution passes when score <= threshold. static constexpr unsigned long long BPP9000_NUMBER_OF_WINDOWS = BPP9000_SEQUENCE_LENGTH - BPP9000_WINDOW_WIDTH; -static constexpr unsigned int BPP9000_SOLUTION_THRESHOLD_DEFAULT = 3838; +#ifdef TESTNET +// A fresh root scores around 5500 and one walk reaches about 4500, so the mainnet bound is +// unreachable at depth 1 and the local tree never starts. Raise it on testnet so every walk qualifies +// and the tree deepens; deeper nodes must still strictly beat their parent. +static constexpr unsigned int BPP9000_SOLUTION_THRESHOLD_DEFAULT = 6500; +#else +static constexpr unsigned int BPP9000_SOLUTION_THRESHOLD_DEFAULT = 4000; +#endif + +// Ant colony: a solution must be published within this many ticks of the anchor its walk seeded from. +static constexpr unsigned int ANT_PUBLISH_WINDOW_TICKS = 15000; + +// Per-parent child cap: a parent accepts at most this many children - a miner's parallel branches +// off one node. 0 means unbound (no cap). A child's score must still strictly beat its parent's. +// A child over the cap is rejected without a refund, so miners should stop submitting to a full parent. +static constexpr unsigned int ANT_MAX_CHILDREN_PER_PARENT = 0; + +// Ant colony: tree nodes recorded per epoch; one per accepted solution. +#if defined(TESTNET) && defined(TESTNET_LITE_RAM) +static constexpr unsigned int ANT_MAX_NODES_PER_EPOCH = 1u << 16; +#else +static constexpr unsigned int ANT_MAX_NODES_PER_EPOCH = 1u << 23; +#endif + +// Ant colony: replay-cache entries, scores this node already computed so a restart does not +// recompute them. Node-local, not consensus; a miss only costs time. +#if defined(TESTNET) && defined(TESTNET_LITE_RAM) +static constexpr unsigned int ANT_REPLAY_CACHE_SIZE = 1u << 14; +#else +static constexpr unsigned int ANT_REPLAY_CACHE_SIZE = 1u << 20; +#endif +static_assert((ANT_REPLAY_CACHE_SIZE & (ANT_REPLAY_CACHE_SIZE - 1)) == 0, + "ANT_REPLAY_CACHE_SIZE must be a power of two, the slot index masks with it"); // Multipler of score static constexpr unsigned int NEURAXON_SOLUTION_MULTIPLER = 1; diff --git a/src/qubic.cpp b/src/qubic.cpp index 18785bde..0086be9a 100644 --- a/src/qubic.cpp +++ b/src/qubic.cpp @@ -88,6 +88,7 @@ // #define INCLUDE_CONTRACT_TEST_EXAMPLES +// #define OLD_QRAFFLE // contract_def.h needs to be included first to make sure that contracts have minimal access #include "contract_core/contract_def.h" @@ -163,10 +164,12 @@ #include "mining/mining.h" #include "mining/custom_qubic_mining_storage.h" #include "mining/bpp9000_task.generated.h" +#include "mining/ant_colony/ant_colony_bpp9000.h" #include "oracle_core/oracle_engine.h" #include "oracle_core/net_msg_impl.h" #include "oracle_core/snapshot_files.h" +#include "mining/ant_colony/ant_pending_solutions.h" #include "oracle_core/oracle_interfaces_def.h" #include "qpi/impl/qpi_oracle_impl.h" @@ -227,20 +230,24 @@ TickStorage::TransactionsDigestAccess TickStorage::transactionsDigestAccess; #define CONTRACT_STATES_DEPTH 10 // Is derived from MAX_NUMBER_OF_CONTRACTS (=N) #define TICK_REQUESTING_PERIOD 500ULL #define MAX_NUMBER_EPOCH 1000ULL -#define MAX_NUMBER_OF_MINERS 8192 #if defined(TESTNET) && defined(TESTNET_LITE_RAM) #define NUMBER_OF_MINER_SOLUTION_FLAGS 0x10000000 // 16 MB bitmap — LITE testnet +#define NUMBER_OF_ANT_SOLUTION_FLAGS 0x10000000 // 16 MB bitmap — LITE testnet #else #define NUMBER_OF_MINER_SOLUTION_FLAGS 0x100000000 +#define NUMBER_OF_ANT_SOLUTION_FLAGS 0x100000000 #endif #define MAX_MESSAGE_PAYLOAD_SIZE MAX_TRANSACTION_SIZE #define MAX_UNIVERSE_SIZE 1073741824 #define MESSAGE_DISSEMINATION_THRESHOLD 1000000000 +// Overridable so a second node can run on a box where the default port is already taken. +#ifndef PORT #ifdef TESTNET #define PORT 31841 #else #define PORT 21841 #endif +#endif #define SYSTEM_DATA_SAVING_PERIOD 300000ULL #define TICK_TRANSACTIONS_PUBLICATION_OFFSET 2 // Must be only 2 #define MIN_MINING_SOLUTIONS_PUBLICATION_OFFSET 3 // Must be 3+ @@ -293,6 +300,14 @@ static int misalignedState = 0; static bool forceVerifySolutions = false; static bool forceBroadcastInvalidSolution = false; +static bool forceBroadcastAntSolution = false; +static unsigned int forceAntSolutionBudget = 3; +// Honest solutions published before the mode under test, so it has a tree to aim at. +static unsigned int forceAntInjectWarmup = 0; +// Ticks left between publishes. A gap wider than the checkpoint window lets each window retire, which is +// what leaves an optimistically committed record in place to parent a later one. +static unsigned int forceAntInjectGapTicks = 0; +static TestInvalidSolution::AntInjectMode forceAntInjectMode = TestInvalidSolution::AntInjectMode::Valid; static unsigned int gFbisCount = 1; // test: number of solution txs to inject per tick static bool gFbisSameComputor = false; // test: all from one computor (drains it -> out-of-qus) static int gTestSolutionThreshold = -1; // test: override runtime Bpp9000 threshold (-1 = off) @@ -384,8 +399,20 @@ static ScoreFunction< NUMBER_OF_SOLUTION_PROCESSORS > * score = nullptr; static unsigned char* gBpp9000TaskBuffer = nullptr; + +// The payload is the { publicKey, miningSeed, nonce } +static_assert(3 * sizeof(m256i) <= ScoreFunction::TASK_PAYLOAD_MAX, + "A legacy solution must fit the task queue payload"); + +static void scoreLegacySolutionTask(unsigned long long processorNumber, void* payload) +{ + const m256i* data = (const m256i*)payload; + (*score)(processorNumber, data[0], data[1], data[2]); +} + static volatile char solutionsLock = 0; static unsigned long long* minerSolutionFlags = NULL; +static unsigned long long* gAntSolutionFlags = NULL; static volatile m256i minerPublicKeys[MAX_NUMBER_OF_MINERS + 1]; static volatile unsigned int minerScores[MAX_NUMBER_OF_MINERS + 1]; static volatile unsigned int minerBestScoreTicks[MAX_NUMBER_OF_MINERS + 1]; @@ -408,6 +435,9 @@ static constexpr unsigned int gScoreMultiplier[score_engine::AlgoType::MaxAlgoCo NEURAXON_SOLUTION_MULTIPLER, // Neuraxon (reserved) BPP9000_SOLUTION_MULTIPLER // Bpp9000 }; +// Bpp9000's score is a raw error count consumed directly by the minimum-is-best ranking; scaling it +// serves no purpose and a large multiplier would overflow the ranking score. Pin it to 1. +static_assert(BPP9000_SOLUTION_MULTIPLER == 1, "Bpp9000 error score is ranked by minimum; its multiplier must be 1"); // Active solution threshold for an algorithm static int getSolutionThreshold(score_engine::AlgoType selectedAlgo) @@ -422,6 +452,538 @@ static int getSolutionThreshold(score_engine::AlgoType selectedAlgo) } static bool applyBpp9000Task(); +static AntColonyBpp9000T gAntColony; +static AntPendingSolutions gAntPendingSolutions; +static AntColonyBpp9000T::Ann gAntParentAnnScratch[MAX_NUMBER_OF_PROCESSORS]; +static AntColonyBpp9000T::Ann gAntChildAnnScratch[MAX_NUMBER_OF_PROCESSORS]; +// Separate from the two above: a rebuild runs underneath a caller already holding those. +static AntColonyBpp9000T::Ann gAntRebuildParentScratch[MAX_NUMBER_OF_PROCESSORS]; +static AntColonyBpp9000T::Ann gAntRebuildChildScratch[MAX_NUMBER_OF_PROCESSORS]; + +// Compiled into every build and switched on with --ant-debug, so a release node can be traced without +// rebuilding it. Debug builds keep it on by default. +#ifndef NDEBUG +static bool gAntDebugEnabled = true; +#else +static bool gAntDebugEnabled = false; +#endif +static constexpr unsigned int ANT_DEBUG_PRINTS_PER_EPOCH = 512; +static unsigned int gAntDebugPrintBudget = ANT_DEBUG_PRINTS_PER_EPOCH; +static bool antDebugCanPrint() +{ + if (!gAntDebugEnabled || gAntDebugPrintBudget == 0) + { + return false; + } + gAntDebugPrintBudget--; + if (gAntDebugPrintBudget == 0) + { + logToConsole(L"[ant-colony] debug print budget exhausted, silent until next epoch"); + } + return true; +} + +static void antDebugLine(const CHAR16* text) +{ + if (antDebugCanPrint()) + { + logToConsole(text); + } +} + +static void antDebugPoolDrop(const CHAR16* reason, const AntSolutionBroadcastPayload& payload) +{ + if (!antDebugCanPrint()) + { + return; + } + CHAR16 msg[256]; + setText(msg, L"[ant-colony] pool drop "); + appendText(msg, reason); + appendText(msg, L": parent="); + appendNumber(msg, payload.parentTick, FALSE); + appendText(msg, L"/"); + appendNumber(msg, payload.parentSolutionIndexInTick, FALSE); + appendText(msg, L" anchor="); + appendNumber(msg, payload.anchorTick, FALSE); + appendText(msg, L" nonce0="); + appendNumber(msg, payload.nonce.m256i_u64[0], FALSE); + logToConsole(msg); +} + +static void antDebugPending(const CHAR16* outcome, const AntPendingSolution& entry, unsigned int targetTick) +{ + if (!antDebugCanPrint()) + { + return; + } + CHAR16 msg[256]; + setText(msg, L"[ant-colony] "); + appendText(msg, outcome); + appendText(msg, L": parent="); + appendNumber(msg, entry.parentRef.tick, FALSE); + appendText(msg, L"/"); + appendNumber(msg, entry.parentRef.solutionIndexInTick, FALSE); + appendText(msg, L" anchor="); + appendNumber(msg, entry.anchorTick, FALSE); + appendText(msg, L" score="); + appendNumber(msg, entry.score, FALSE); + appendText(msg, L" target="); + appendNumber(msg, targetTick, FALSE); + logToConsole(msg); +} + +static void antDebugAccepted(const AntColonyMiningSolutionTransaction* transaction, unsigned int score, + unsigned int depth, unsigned int transactionIndex, ValidityResult result, bool trustedScore) +{ + if (result != ValidityResult::Valid && result != ValidityResult::ValidNotStored) + { + return; + } + if (!antDebugCanPrint()) + { + return; + } + CHAR16 msg[256]; + setText(msg, L"[ant-colony] accepted: tick="); + appendNumber(msg, system.tick, FALSE); + appendText(msg, L" idx="); + appendNumber(msg, transactionIndex, FALSE); + appendText(msg, L" score="); + appendNumber(msg, score, FALSE); + appendText(msg, L" depth="); + appendNumber(msg, depth, FALSE); + appendText(msg, L" stored="); + appendNumber(msg, (result == ValidityResult::Valid) ? 1 : 0, FALSE); + appendText(msg, L" trusted="); + appendNumber(msg, trustedScore ? 1 : 0, FALSE); + logToConsole(msg); +} + +// Pre-scored ant solutions for the current tick, indexed by TRANSACTION index. Each transaction is +// enqueued at most once, so no two workers ever write the same slot and no lock is needed +static bool gAntScoredReady[NUMBER_OF_TRANSACTIONS_PER_TICK]; +static unsigned int gAntScoredValue[NUMBER_OF_TRANSACTIONS_PER_TICK]; +static AntColonyBpp9000T::Ann gAntScoredAnn[NUMBER_OF_TRANSACTIONS_PER_TICK]; + +// Enqueued per ant solution transaction. 80 bytes, inside ScoreFunction::TASK_PAYLOAD_MAX. +struct AntScoreTaskPayload +{ + m256i pubkey; + m256i nonce; + SolutionRef parentRef; + unsigned int anchorTick; + unsigned int txIdx; +}; +static_assert(sizeof(AntScoreTaskPayload) <= 128, "ant score payload must fit TASK_PAYLOAD_MAX"); + +// The parent is named by its on-chain address, not a hash of its network: within an epoch a ref selects +// the same parent on every node, and unlike the hash it can be formed without holding that network. +// The cache file is epoch-scoped, so a ref never means something else in a later epoch. +static AntColonyBpp9000T::ReplayKey makeAntReplayKey(const m256i& pubkey, const m256i& nonce, + const SolutionRef& parentRef, const m256i& anchorDigest) +{ + AntColonyBpp9000T::ReplayKey key; + key.pubkey = pubkey; + key.nonce = nonce; + key.parentKey = AntColonyBpp9000T::replayParentKey(parentRef); + key.anchorDigest = anchorDigest; + return key; +} + +// Defined below, next to the anchor-digest fallback it needs. +static bool ensureAntRecordAnn(unsigned long long processorNumber, unsigned int recordIdx, + AntColonyBpp9000T::Ann& out); + +// Score one ant solution ahead of the transaction loop, on whichever processor drains the queue. +static void scoreAntSolutionTask(unsigned long long processorNumber, void* payload) +{ + const AntScoreTaskPayload* task = (const AntScoreTaskPayload*)payload; + + const AntSolutionRecord* parentRec = nullptr; + if (gAntColony.tryGetParent(task->parentRef, &parentRec) != ValidityResult::Valid) + { + return; + } + + m256i anchorDigest; + if (!gAntColony.getAnchorDigest(task->anchorTick, anchorDigest)) + { + return; + } + + const AntColonyBpp9000T::Ann* parentAnn = nullptr; + if (parentRec != nullptr) + { + const long long parentIdx = gAntColony.findIndexBySolutionRef(task->parentRef); + if (parentIdx == ANT_INVALID_INDEX + || !ensureAntRecordAnn(processorNumber, (unsigned int)parentIdx, + gAntParentAnnScratch[processorNumber])) + { + return; + } + parentAnn = &gAntParentAnnScratch[processorNumber]; + } + + const AntColonyBpp9000T::ReplayKey replayKey = + makeAntReplayKey(task->pubkey, task->nonce, task->parentRef, anchorDigest); + + // Check in the cache first if this sol was computed + if (!gAntColony.tryGetReplayScore(replayKey, gAntScoredValue[task->txIdx], gAntScoredAnn[task->txIdx])) + { + // Straight into this transaction's own result slot, so nothing is copied afterwards. + gAntScoredValue[task->txIdx] = score->computeAntChildScore( + processorNumber, parentAnn, task->pubkey, task->nonce, + anchorDigest, gAntScoredAnn[task->txIdx]); + gAntColony.putReplayScore(replayKey, gAntScoredValue[task->txIdx], gAntScoredAnn[task->txIdx]); + } + + // Last, so the transaction loop never sees a slot whose score or network is half written. + gAntScoredReady[task->txIdx] = true; +} + +// The two independent bits an ant solution occupies in gAntSolutionFlags. Hashed over the same +// triple AntDedupKey uses, so "seen" and "already in the tree" agree on what counts as one solution. +// Bytes are laid out explicitly rather than hashing a struct, so no padding can make the digest +// differ between compilers. +static void computeAntSolutionFlagIndices(const m256i& pubkey, const m256i& nonce, + const SolutionRef& parentRef, unsigned int* outFlagIndices) +{ + unsigned char preimage[sizeof(m256i) + sizeof(m256i) + sizeof(SolutionRef)]; + copyMem(preimage, &pubkey, sizeof(pubkey)); + copyMem(preimage + sizeof(pubkey), &nonce, sizeof(nonce)); + copyMem(preimage + sizeof(pubkey) + sizeof(nonce), &parentRef, sizeof(parentRef)); + KangarooTwelve(preimage, sizeof(preimage), outFlagIndices, 2 * sizeof(unsigned int)); + // mask hash into allocated gAntSolutionFlags bit-range (no-op at full size; LITE-safe) + outFlagIndices[0] &= (unsigned int)(NUMBER_OF_ANT_SOLUTION_FLAGS - 1); + outFlagIndices[1] &= (unsigned int)(NUMBER_OF_ANT_SOLUTION_FLAGS - 1); +} + +// Seen means BOTH bits are set, matching the legacy filter: one bit alone is a collision with some +// other solution, and requiring two drops the false-positive rate from ~N/2^32 to ~N^2/2^64. +static bool isAntSolutionSeen(const unsigned int* flagIndices) +{ + return (gAntSolutionFlags[flagIndices[0] >> 6] & (1ULL << (flagIndices[0] & 63))) + && (gAntSolutionFlags[flagIndices[1] >> 6] & (1ULL << (flagIndices[1] & 63))); +} + +static void markAntSolutionSeen(const unsigned int* flagIndices) +{ + gAntSolutionFlags[flagIndices[0] >> 6] |= (1ULL << (flagIndices[0] & 63)); + gAntSolutionFlags[flagIndices[1] >> 6] |= (1ULL << (flagIndices[1] & 63)); +} + +// The anchor digest a solution's mutation walk seeds from, K12(tick || transactionDigest) +static void computeAntAnchorDigest(unsigned int tick, const m256i& transactionDigest, m256i& out) +{ + unsigned char preimage[sizeof(unsigned int) + sizeof(m256i)]; + copyMem(preimage, &tick, sizeof(tick)); + copyMem(preimage + sizeof(tick), &transactionDigest, sizeof(transactionDigest)); + KangarooTwelve(preimage, sizeof(preimage), &out, sizeof(out)); +} + +// A record outlives the anchor ring, so a rebuild needs its anchor after the ring has wrapped past it. +// Tick storage keeps the whole epoch; only non-empty ticks recorded an anchor, which this reproduces. +static bool recomputeAntAnchorDigest(unsigned int tick, m256i& out) +{ + m256i anchorTxDigest; + ts.tickData.acquireLock(); + const TickData* storedTickData = ts.tickData.getByTickIfNotEmpty(tick); + const bool usable = (storedTickData != nullptr) && (storedTickData->epoch == system.epoch); + if (usable) + { + KangarooTwelve(storedTickData, sizeof(TickData), &anchorTxDigest, sizeof(anchorTxDigest)); + } + ts.tickData.releaseLock(); + if (!usable) + { + return false; + } + computeAntAnchorDigest(tick, anchorTxDigest, out); + return true; +} + +// Ring first, tick storage once it has wrapped past the tick. Rebuild paths only: the live paths must +// keep rejecting an anchor outside the freshness window, which the bare ring lookup already does. +static bool getAntAnchorDigestForRebuild(unsigned int tick, m256i& out) +{ + if (gAntColony.getAnchorDigest(tick, out)) + { + return true; + } + return recomputeAntAnchorDigest(tick, out); +} + +// Rebuilds one record's network. Its parent must already have one, or be the root. +static bool materialiseOneAntRecord(unsigned long long processorNumber, unsigned int idx) +{ + const AntColonyBpp9000T::AnnClaim claim = gAntColony.tryClaimAnn(idx); + if (claim == AntColonyBpp9000T::AnnClaimReady) + { + return true; + } + if (claim == AntColonyBpp9000T::AnnClaimInvalid) + { + return false; + } + if (claim == AntColonyBpp9000T::AnnClaimBusy) + { + // Waiting costs what the walk costs; walking it here as well would pay that twice. + while (gAntColony.isAnnClaimHeld(idx)) + { + sleepMilliseconds(20); + } + return gAntColony.isAnnMaterialised(idx); + } + + // Owned from here on, so every exit below either publishes or releases the claim. + const unsigned long long rebuildStart = __rdtsc(); + const AntSolutionRecord* rec = gAntColony.recordAt(idx); + if (rec == nullptr) + { + gAntColony.releaseAnnClaim(idx); + return false; + } + + const AntColonyBpp9000T::Ann* parentAnn = nullptr; + if (!rec->parentRef.isRoot()) + { + const long long parentIdx = gAntColony.findIndexBySolutionRef(rec->parentRef); + const AntSolutionRecord* parentRec = + (parentIdx == ANT_INVALID_INDEX) ? nullptr : gAntColony.recordAt(parentIdx); + if (parentRec == nullptr + || !gAntColony.annOfNonRoot(*parentRec, gAntRebuildParentScratch[processorNumber])) + { + gAntColony.releaseAnnClaim(idx); + return false; + } + parentAnn = &gAntRebuildParentScratch[processorNumber]; + } + + m256i anchorDigest; + if (!getAntAnchorDigestForRebuild(rec->anchorTick, anchorDigest)) + { + gAntColony.releaseAnnClaim(idx); + return false; + } + + AntColonyBpp9000T::Ann& childAnn = gAntRebuildChildScratch[processorNumber]; + const unsigned int rebuiltScore = score->computeAntChildScore(processorNumber, parentAnn, + rec->pubkey, rec->nonce, anchorDigest, childAnn); + + // This walk is the computation the record was admitted without. A disagreement means the acceptance + // was wrong, so publish nothing: a network built for a score no other node holds is worse than none. + if (rebuiltScore != rec->score) + { + gAntColony.releaseAnnClaim(idx); + CHAR16 msg[256]; + setText(msg, L"[ant-colony] rebuilt score disagrees with the accepted record, index "); + appendNumber(msg, idx, FALSE); + appendText(msg, L" accepted "); + appendNumber(msg, rec->score, FALSE); + appendText(msg, L" rebuilt "); + appendNumber(msg, rebuiltScore, FALSE); + logToConsole(msg); + return false; + } + + unsigned int annHash; + KangarooTwelve(&childAnn, sizeof(childAnn), &annHash, sizeof(annHash)); + gAntColony.publishAnn(idx, childAnn, annHash); + + // A rebuild costs a full walk, so a tick that takes tens of seconds is attributable here. + CHAR16 okLine[224]; + setText(okLine, L"[ant-colony] rebuilt the network of record "); + appendNumber(okLine, idx, FALSE); + appendText(okLine, L" score "); + appendNumber(okLine, rebuiltScore, FALSE); + appendText(okLine, L" in "); + appendNumber(okLine, (__rdtsc() - rebuildStart) / (frequency / 1000), FALSE); + appendText(okLine, L" ms"); + logToConsole(okLine); + return true; +} + +// Supplies the network of a record committed without one, walking its lineage down from the nearest +// ancestor that still has one and publishing each level for later readers. Unbounded in depth on +// purpose: a record this node cannot rebuild would be rejected here and accepted elsewhere. +static bool ensureAntRecordAnn(unsigned long long processorNumber, unsigned int recordIdx, + AntColonyBpp9000T::Ann& out) +{ + if (!gAntColony.isAnnMaterialised(recordIdx)) + { + antDebugLine(L"[ant-colony] rebuilding a missing parent network, this costs a full walk"); + } + + while (!gAntColony.isAnnMaterialised(recordIdx)) + { + // The shallowest record still missing a network: its parent is the root or already rebuilt. + unsigned int target = recordIdx; + for (;;) + { + const AntSolutionRecord* rec = gAntColony.recordAt(target); + if (rec == nullptr) + { + return false; + } + if (rec->parentRef.isRoot()) + { + break; + } + const long long parentIdx = gAntColony.findIndexBySolutionRef(rec->parentRef); + if (parentIdx == ANT_INVALID_INDEX) + { + return false; + } + if (gAntColony.isAnnMaterialised((unsigned int)parentIdx)) + { + break; + } + target = (unsigned int)parentIdx; + } + + if (!materialiseOneAntRecord(processorNumber, target)) + { + // The caller turns this into RejectParentNotRegistered, which no other node reaches, so say + // so rather than failing quietly. + CHAR16 failLine[224]; + setText(failLine, L"[ant-colony] could not rebuild the network of record "); + appendNumber(failLine, target, FALSE); + appendText(failLine, L" needed by record "); + appendNumber(failLine, recordIdx, FALSE); + logToConsole(failLine); + return false; + } + } + + const AntSolutionRecord* rec = gAntColony.recordAt(recordIdx); + return (rec != nullptr) && gAntColony.annOfNonRoot(*rec, out); +} + +// A pool miner's solution, arriving over BroadcastMessage +static void queueAntSolution(unsigned long long processorNumber, const m256i& computorPublicKey, + const AntSolutionBroadcastPayload& payload) +{ + gAntPendingSolutions.noteReceived(); + + // A non-canonical nonce is rejected by the scorer without producing a score, so the transaction + // would forfeit the deposit with nothing to show. The computor pays that, not the miner. + if (!score_engine::ScoreEngineT::isCanonicalAntNonce(payload.nonce.m256i_u8)) + { + antDebugPoolDrop(L"nonCanonical", payload); + gAntPendingSolutions.noteDroppedNonCanonical(); + return; + } + + // The same bits the commit path checks before RejectReplay, so a hit here is a solution this + // node has already processed on-chain - publishing it again would forfeit the deposit. The + // filter is restored from the snapshot, so unlike this buffer the check survives a restart. + const SolutionRef parentRef = { payload.parentTick, payload.parentSolutionIndexInTick }; + unsigned int seenFlagIndices[2]; + computeAntSolutionFlagIndices(computorPublicKey, payload.nonce, parentRef, seenFlagIndices); + if (isAntSolutionSeen(seenFlagIndices)) + { + gAntPendingSolutions.noteDroppedDuplicate(); + return; + } + + // An anchor in the future is malformed, and one the ring no longer holds cannot be scored. + m256i anchorDigest; + if (payload.anchorTick > system.tick + || system.tick - payload.anchorTick > ANT_PUBLISH_WINDOW_TICKS + || !gAntColony.getAnchorDigest(payload.anchorTick, anchorDigest)) + { + antDebugPoolDrop(L"badAnchor", payload); + gAntPendingSolutions.noteDroppedBadAnchor(); + return; + } + + const AntSolutionRecord* parentRec = nullptr; + if (gAntColony.tryGetParent(parentRef, &parentRec) != ValidityResult::Valid) + { + antDebugPoolDrop(L"parentUnknown", payload); + gAntPendingSolutions.noteDroppedParentUnknown(); + return; + } + + const score_engine::ScoreBpp9000T::ANN* parentAnn = nullptr; + if (parentRec != nullptr) + { + if (!gAntColony.annOfNonRoot(*parentRec, gAntParentAnnScratch[processorNumber])) + { + antDebugPoolDrop(L"parentUnknown", payload); + gAntPendingSolutions.noteDroppedParentUnknown(); + return; + } + parentAnn = &gAntParentAnnScratch[processorNumber]; + } + + // Building the key is far cheaper than a miss, so the cache is consulted first. + const AntColonyBpp9000T::ReplayKey replayKey = + makeAntReplayKey(computorPublicKey, payload.nonce, parentRef, anchorDigest); + unsigned int childScore = 0; + if (!gAntColony.tryGetReplayScore(replayKey, childScore, gAntChildAnnScratch[processorNumber])) + { + childScore = score->computeAntChildScore(processorNumber, parentAnn, computorPublicKey, + payload.nonce, anchorDigest, gAntChildAnnScratch[processorNumber]); + // Cached whatever the outcome: a timed-out network scores invalid, and the walk is + // deterministic, so the cached rejection stays right - without the entry the same doomed + // solution costs a full walk again on every path that sees it. + gAntColony.putReplayScore(replayKey, childScore, gAntChildAnnScratch[processorNumber]); + } + if (!score->isValidScore(childScore, score_engine::AlgoType::Bpp9000)) + { + antDebugPoolDrop(L"unscorable", payload); + gAntPendingSolutions.noteDroppedUnscorable(); + return; + } + + // The sender's own number, checked where it can still prevent work rather than merely be counted. + if (payload.claimedScore != childScore) + { + antDebugPoolDrop(L"claimMismatch", payload); + gAntPendingSolutions.noteClaimMismatch(); + return; + } + + // The child count only grows, so passing now is not a promise it will pass at publication - the + // publisher re-checks. Failing now is final enough to refuse the slot. + const unsigned int childCount = gAntColony.childCountForQuery(parentRef, computorPublicKey); + const ChildCandidate candidate{ computorPublicKey, childScore, payload.anchorTick, system.tick }; + if (AntColonyBpp9000T::validateChild(candidate, parentRec, childCount, + gAntColony.errorThreshold()) != ValidityResult::Valid) + { + antDebugPoolDrop(L"unacceptable", payload); + gAntPendingSolutions.noteDroppedUnacceptable(); + return; + } + + gAntPendingSolutions.add(computorPublicKey, parentRef, payload.anchorTick, childScore, + payload.nonce); +} + +// Reseed the colony for a new epoch. The root seed is score->currentRandomSeed, the epoch-start +// spectrum digest +static void antColonyBeginEpoch() +{ +#ifndef NDEBUG + gAntDebugPrintBudget = ANT_DEBUG_PRINTS_PER_EPOCH; +#endif + gAntPendingSolutions.reset(); + gAntColony.beginEpoch(score->currentRandomSeed, system.initialTick); + gAntColony.setErrorThreshold((unsigned int)getSolutionThreshold(score_engine::AlgoType::Bpp9000)); + + // Every identity's root derives from this one value, so a node that seeded differently builds a + // different forest and diverges. Logged as an identity so operators can compare it across nodes + // by eye at epoch start, which is cheaper than finding out from a digest split later. + CHAR16 digestChars[60 + 1]; + getIdentity(gAntColony.rootSeed().m256i_u8, digestChars, true); + CHAR16 msg[128]; + setText(msg, L"[ant-colony] Root seed = "); + appendText(msg, digestChars); + logToConsole(msg); +} + // DOGE merged-mining shares static volatile char gDogeMiningSharesCountLock = 0; static unsigned int gDogeMiningSharesCount[NUMBER_OF_COMPUTORS] = { 0 }; @@ -641,6 +1203,15 @@ static bool isLastTickInEpoch() { #endif } +// AUX outside the strict paths takes a solution's claimed score instead of computing it, and the fork +// checkpoint undoes the tick if that disagrees with quorum. Every path that fails to establish one sets +// gReRunStrict, so this is never true without a checkpoint behind it, nor on a build with no rollback. +static bool isTrustingClaimedSolutionScore() +{ + return tickFork::gRollbackAvailable + && !isMainMode() && !gReRunStrict && !forceVerifySolutions && !isLastTickInEpoch(); +} + // NOTE: this function doesn't work well on a few CPUs, some bits will be flipped after calling this. It's probably microcode bug. static void enableAVX() { @@ -889,6 +1460,12 @@ static void processBroadcastMessage(const unsigned long long processorNumber, Re const m256i& solution_miningSeed = *(m256i*)((unsigned char*)request + sizeof(BroadcastMessage)); const m256i& solution_nonce = *(m256i*)((unsigned char*)request + sizeof(BroadcastMessage) + 32); + // standalone mining disabled for bpp9000 + if (score_engine::getAlgoType(solution_nonce.m256i_u8) == score_engine::AlgoType::Bpp9000) + { + break; + } + const unsigned int solution_claimedScore = *(unsigned int*)((unsigned char*)request + sizeof(BroadcastMessage) + 64); unsigned int k; for (k = 0; k < system.numberOfSolutions; k++) @@ -935,6 +1512,19 @@ static void processBroadcastMessage(const unsigned long long processorNumber, Re } } break; + + case MESSAGE_TYPE_ANT_SOLUTION: + { + // Exact size, not a minimum: the payload is fixed, so a longer + // one is a different message rather than a forward-compatible + // variant of this one. + if (messagePayloadSize == sizeof(AntSolutionBroadcastPayload)) + { + queueAntSolution(processorNumber, request->destinationPublicKey, + *(AntSolutionBroadcastPayload*)((unsigned char*)request + sizeof(BroadcastMessage))); + } + } + break; } } } @@ -977,8 +1567,11 @@ static void processBroadcastComputors(Peer* peer, RequestResponseHeader* header) enqueueResponse(NULL, header); } - // Copy computor list - copyMem(&broadcastedComputors.computors, &request->computors, sizeof(Computors)); + // Copy computor list. epoch is the flag the tick processor gates on without a lock, so + // copy publicKeys + signature first and publish epoch last + copyMem(broadcastedComputors.computors.publicKeys, request->computors.publicKeys, + sizeof(broadcastedComputors.computors.publicKeys) + sizeof(broadcastedComputors.computors.signature)); + broadcastedComputors.computors.epoch = request->computors.epoch; // Update ownComputorIndices and minerPublicKeys if (request->computors.epoch == system.epoch) @@ -1338,6 +1931,53 @@ static void processBroadcastTransaction(Peer* peer, RequestResponseHeader* heade } } + // Same latency hiding for ant solution transactions, we do simple check first then the last + // is the score engine that where the heavy load stay + if (preprocessSolutionFlags[processorNumber] + && !isTrustingClaimedSolutionScore() + && AntColonyMiningSolutionTransaction::isSolutionTransaction(request)) + { + const AntColonyMiningSolutionTransaction* antTx = (const AntColonyMiningSolutionTransaction*)request; + const SolutionRef preParentRef = { antTx->parentTick, antTx->parentSolutionIndexInTick }; + unsigned int preFlagIndices[2]; + computeAntSolutionFlagIndices(antTx->sourcePublicKey, antTx->nonce, preParentRef, preFlagIndices); + const int spectrumIdx = spectrumIndex(antTx->sourcePublicKey); + if (spectrumIdx >= 0 + && energy(spectrumIdx) >= AntColonyMiningSolutionTransaction::minAmount() + && !isAntSolutionSeen(preFlagIndices) + && score_engine::ScoreEngineT::isCanonicalAntNonce(antTx->nonce.m256i_u8)) + { + const AntSolutionRecord* preParentRec = nullptr; + m256i preAnchorDigest; + if (gAntColony.tryGetParent(preParentRef, &preParentRec) == ValidityResult::Valid + && gAntColony.getAnchorDigest(antTx->anchorTick, preAnchorDigest)) + { + const AntColonyBpp9000T::Ann* preParentAnn = nullptr; + bool preParentOk = true; + if (preParentRec != nullptr) + { + preParentOk = gAntColony.annOfNonRoot(*preParentRec, gAntParentAnnScratch[processorNumber]); + preParentAnn = &gAntParentAnnScratch[processorNumber]; + } + if (preParentOk) + { + const AntColonyBpp9000T::ReplayKey preKey = makeAntReplayKey( + antTx->sourcePublicKey, antTx->nonce, preParentRef, preAnchorDigest); + unsigned int preScore = 0; + if (!gAntColony.tryGetReplayScore(preKey, preScore, gAntChildAnnScratch[processorNumber])) + { + preScore = score->computeAntChildScore(processorNumber, preParentAnn, + antTx->sourcePublicKey, antTx->nonce, preAnchorDigest, + gAntChildAnnScratch[processorNumber]); + // cache this score so later can skip the heavy score computation, + // invalid ones included + gAntColony.putReplayScore(preKey, preScore, gAntChildAnnScratch[processorNumber]); + } + } + } + } + } + // shortcut: oracle reply reveal transactions are analyzed immediately after receiving them (before execution of the tx), // in order to minimize the number of reveal transaction (one per oracle query is enough, so no reveal tx is generated // after one has been seen) @@ -1693,6 +2333,171 @@ static void processRequestContractFunction(Peer* peer, const unsigned long long } } +// One response buffer per processor +static AntIdentityTreeResponse gAntIdentityTreeResponseBuffer[MAX_NUMBER_OF_PROCESSORS]; + +struct AntParentAnnResponse +{ + RespondAntParentAnnHeader header; + AntColonyBpp9000T::Ann ann; +}; +static_assert(sizeof(AntParentAnnResponse) + == sizeof(RespondAntParentAnnHeader) + sizeof(AntColonyBpp9000T::Ann), + "AntParentAnnResponse must have no padding between the header and the network"); +static AntParentAnnResponse gAntParentAnnResponseBuffer[MAX_NUMBER_OF_PROCESSORS]; + +// Request ant colony in epoch contex +static void processRequestAntEpochContext(Peer* peer, RequestResponseHeader* header) +{ + RespondAntEpochContext respond; + setMem(&respond, sizeof(respond), 0); + + respond.spectrumDigest = gAntColony.rootSeed(); + respond.threshold = gAntColony.errorThreshold(); + respond.freshnessWindow = ANT_PUBLISH_WINDOW_TICKS; + respond.solutionCount = gAntColony.solutionCount(); + respond.freeAnnSlotsCount = gAntColony.freeAnnSlotsCount(); + respond.maxChildrenPerParent = ANT_MAX_CHILDREN_PER_PARENT; + respond.epoch = system.epoch; + respond.topologyHash = *(const m256i*)BPP9000_TOPOLOGY_HASH; + respond.dataHash = *(const m256i*)BPP9000_DATA_HASH; + + enqueueResponse(peer, sizeof(respond), RespondAntEpochContext::type(), header->dejavu(), &respond); +} + +// The parents ONE identity can branch from, each with the bar a child of it must beat. Scoped by +// pubkey: a child must name a parent in its own tree, so an unscoped answer would be mostly nodes the +// caller can never use. +// +// Paged, because the store holds millions and a response is one datagram - the cursor is a record +// index, and ANT_IDENTITY_TREE_SCAN_BUDGET caps how far one request may scan, so a caller cannot +// walk the whole store in a single call. Sweeping a full store therefore costs many requests; the +// alternative, walking the identity's tree through the head maps, needs a resumable cursor that does +// not fit a record index, and the scan is cache-friendly where a chain walk is not. +// +// Operator-signed +static void processRequestAntIdentityTree(unsigned long long processorNumber, Peer* peer, RequestResponseHeader* header) +{ + if (processorNumber >= MAX_NUMBER_OF_PROCESSORS) + { + return; + } + if (header->size() != sizeof(RequestResponseHeader) + sizeof(RequestAntIdentityTree) + SIGNATURE_SIZE) + { + return; + } + const RequestAntIdentityTree* request = header->getPayload(); + + // Signature check + unsigned char digest[32]; + KangarooTwelve(request, header->size() - sizeof(RequestResponseHeader) - SIGNATURE_SIZE, digest, sizeof(digest)); + if (!verify(operatorPublicKey.m256i_u8, digest, ((const unsigned char*)header + (header->size() - SIGNATURE_SIZE)))) + { + antDebugLine(L"[ant-colony] query signature rejected"); + return; + } + + AntIdentityTreeResponse& response = gAntIdentityTreeResponseBuffer[processorNumber]; + setMem(&response, sizeof(response), 0); + response.header.itemSize = (unsigned int)sizeof(AntIdentityTreeNode); + + const unsigned int total = gAntColony.solutionCount(); + + unsigned int idx = request->fromIndex; + unsigned int scanned = 0; + while (idx < total + && response.header.count < ANT_IDENTITY_TREE_NODES_PER_RESPONSE + && scanned < ANT_IDENTITY_TREE_SCAN_BUDGET) + { + const AntSolutionRecord* rec = gAntColony.recordAt(idx); + if (rec == nullptr) + { + break; + } + scanned++; + if (!(rec->pubkey == request->pubkey)) + { + idx++; + continue; + } + + AntIdentityTreeNode& item = response.items[response.header.count]; + item.selfTick = rec->selfRef.tick; + item.selfSolutionIndexInTick = rec->selfRef.solutionIndexInTick; + item.parentTick = rec->parentRef.tick; + item.parentSolutionIndexInTick = rec->parentRef.solutionIndexInTick; + item.score = rec->score; + item.childCount = gAntColony.childCountForQuery(rec->selfRef, rec->pubkey); + item.anchorTick = rec->anchorTick; + item.depth = rec->depth; + response.header.count++; + idx++; + } + + // Zero means the caller reached the end of what this node holds, per the protocol. + response.header.nextIndex = (idx < total) ? idx : 0; + + enqueueResponse(peer, + (unsigned int)sizeof(response.header) + response.header.count * (unsigned int)sizeof(AntIdentityTreeNode), + RespondAntIdentityTreeHeader::type(), header->dejavu(), &response); +} + +// One stored node's network, for the pool that is about to mine a child of it +static void processRequestAntParentAnn(unsigned long long processorNumber, Peer* peer, RequestResponseHeader* header) +{ + if (processorNumber >= MAX_NUMBER_OF_PROCESSORS) + { + return; + } + if (header->size() != sizeof(RequestResponseHeader) + sizeof(RequestAntParentAnn) + SIGNATURE_SIZE) + { + return; + } + const RequestAntParentAnn* request = header->getPayload(); + + // Signature check + unsigned char digest[32]; + KangarooTwelve(request, header->size() - sizeof(RequestResponseHeader) - SIGNATURE_SIZE, digest, sizeof(digest)); + if (!verify(operatorPublicKey.m256i_u8, digest, ((const unsigned char*)header + (header->size() - SIGNATURE_SIZE)))) + { + antDebugLine(L"[ant-colony] query signature rejected"); + return; + } + + AntParentAnnResponse& response = gAntParentAnnResponseBuffer[processorNumber]; + setMem(&response, sizeof(response), 0); + response.header.parentRefTick = request->parentRefTick; + response.header.parentRefSolutionIndexInTick = request->parentRefSolutionIndexInTick; + + const SolutionRef ref = { request->parentRefTick, request->parentRefSolutionIndexInTick }; + if (ref.isRoot()) + { + // Roots are never stored; the miner derives its own from the epoch context's seed. + response.header.status = ANT_PARENT_ANN_STATUS_IS_ROOT; + enqueueResponse(peer, sizeof(response.header), RespondAntParentAnnHeader::type(), header->dejavu(), &response); + return; + } + + const AntSolutionRecord* rec = nullptr; + const ValidityResult parentResult = gAntColony.tryGetParent(ref, &rec); + bool annLoaded = false; + if (parentResult == ValidityResult::Valid && rec != nullptr) + { + annLoaded = gAntColony.annOfNonRoot(*rec, response.ann); + } + if (!annLoaded) + { + response.header.status = ANT_PARENT_ANN_STATUS_NOT_FOUND; + enqueueResponse(peer, sizeof(response.header), RespondAntParentAnnHeader::type(), header->dejavu(), &response); + return; + } + + response.header.status = ANT_PARENT_ANN_STATUS_OK; + response.header.annSizeBytes = (unsigned int)sizeof(response.ann); + enqueueResponse(peer, (unsigned int)(sizeof(response.header) + sizeof(response.ann)), + RespondAntParentAnnHeader::type(), header->dejavu(), &response); +} + static void processRequestSystemInfo(Peer* peer, RequestResponseHeader* header) { RespondSystemInfo respondedSystemInfo; @@ -2141,6 +2946,7 @@ static void checkAndSwitchMiningPhase(short tickEpoch, TimeDate tickDate, bool r if (resetPhase) { setNewMiningSeed(); + antColonyBeginEpoch(); } // Roll DOGE per-phase stats at broadcast-cycle boundaries (display only). @@ -2236,7 +3042,7 @@ static void requestProcessor(void* ProcedureArgument, unsigned long long process if (solutionProcessorFlags[processorNumber] && processorNumber != (numberOfProcessors - 1)) { PROFILE_NAMED_SCOPE("requestProcessor(): solution processing"); - score->tryProcessSolution(processorNumber); + score->tryProcessOneTask(processorNumber); } if (requestQueueElementTail == requestQueueElementHead) @@ -2471,6 +3277,24 @@ static void requestProcessor(void* ProcedureArgument, unsigned long long process } break; + case RequestAntEpochContext::type(): + { + processRequestAntEpochContext(peer, header); + } + break; + + case RequestAntIdentityTree::type(): + { + processRequestAntIdentityTree(processorNumber, peer, header); + } + break; + + case RequestAntParentAnn::type(): + { + processRequestAntParentAnn(processorNumber, peer, header); + } + break; + #if ADDON_TX_STATUS_REQUEST /* qli: process RequestTxStatus message */ case RequestTxStatus::type(): @@ -2824,6 +3648,145 @@ static bool ranksBelow(unsigned int scoreA, unsigned int tickA, unsigned int sco return tickA > tickB; } +// Tick-processor only +static void updateMinerRankingAndFutureComputors( + const m256i& sourcePublicKey, + unsigned int newScore, + unsigned int newTick) +{ + ACQUIRE(minerScoreArrayLock); + bool minerEntryChanged = false; + unsigned int minerIndex; + for (minerIndex = 0; minerIndex < numberOfMiners; minerIndex++) + { + if (sourcePublicKey == minerPublicKeys[minerIndex]) + { + if (newScore < minerScores[minerIndex]) + { + minerScores[minerIndex] = newScore; + minerBestScoreTicks[minerIndex] = newTick; + minerEntryChanged = true; + } + + break; + } + } + if (minerIndex == numberOfMiners) + { + if (numberOfMiners < MAX_NUMBER_OF_MINERS) + { + minerPublicKeys[numberOfMiners] = sourcePublicKey; + minerBestScoreTicks[numberOfMiners] = newTick; + minerScores[numberOfMiners++] = newScore; + minerEntryChanged = true; + } + else + { + // The table is full. Entries beyond the computor block are kept sorted, so the + // worst-ranked one sits at the end and is replaced only if the newcomer outranks it. + const unsigned int worstIndex = numberOfMiners - 1; + if (ranksBelow(minerScores[worstIndex], minerBestScoreTicks[worstIndex], newScore, newTick)) + { + minerPublicKeys[worstIndex] = sourcePublicKey; + minerScores[worstIndex] = newScore; + minerBestScoreTicks[worstIndex] = newTick; + minerIndex = worstIndex; + minerEntryChanged = true; + } + } + } + + if (minerEntryChanged) + { + const m256i tmpPublicKey = minerPublicKeys[minerIndex]; + const unsigned int tmpScore = minerScores[minerIndex]; + const unsigned int tmpTick = minerBestScoreTicks[minerIndex]; + while (minerIndex > (unsigned int)(minerIndex < NUMBER_OF_COMPUTORS ? 0 : NUMBER_OF_COMPUTORS) + && ranksBelow(minerScores[minerIndex - 1], minerBestScoreTicks[minerIndex - 1], minerScores[minerIndex], minerBestScoreTicks[minerIndex])) + { + minerPublicKeys[minerIndex] = minerPublicKeys[minerIndex - 1]; + minerScores[minerIndex] = minerScores[minerIndex - 1]; + minerBestScoreTicks[minerIndex] = minerBestScoreTicks[minerIndex - 1]; + minerPublicKeys[--minerIndex] = tmpPublicKey; + minerScores[minerIndex] = tmpScore; + minerBestScoreTicks[minerIndex] = tmpTick; + } + } + + // combine 225 worst current computors with 225 best candidates + for (unsigned int i = 0; i < NUMBER_OF_COMPUTORS - QUORUM; i++) + { + competitorPublicKeys[i] = minerPublicKeys[QUORUM + i]; + competitorScores[i] = minerScores[QUORUM + i]; + competitorTicks[i] = minerBestScoreTicks[QUORUM + i]; + competitorComputorStatuses[i] = true; + + if (NUMBER_OF_COMPUTORS + i < numberOfMiners) + { + competitorPublicKeys[i + (NUMBER_OF_COMPUTORS - QUORUM)] = minerPublicKeys[NUMBER_OF_COMPUTORS + i]; + competitorScores[i + (NUMBER_OF_COMPUTORS - QUORUM)] = minerScores[NUMBER_OF_COMPUTORS + i]; + competitorTicks[i + (NUMBER_OF_COMPUTORS - QUORUM)] = minerBestScoreTicks[NUMBER_OF_COMPUTORS + i]; + } + else + { + competitorScores[i + (NUMBER_OF_COMPUTORS - QUORUM)] = NO_MINER_SCORE; + competitorTicks[i + (NUMBER_OF_COMPUTORS - QUORUM)] = 0; + } + competitorComputorStatuses[i + (NUMBER_OF_COMPUTORS - QUORUM)] = false; + } + RELEASE(minerScoreArrayLock); + + // bubble sorting -> top 225 from competitorPublicKeys have computors and candidates which are the best from that subset + for (unsigned int i = NUMBER_OF_COMPUTORS - QUORUM; i < (NUMBER_OF_COMPUTORS - QUORUM) * 2; i++) + { + int j = i; + const m256i tmpPublicKey = competitorPublicKeys[j]; + const unsigned int tmpScore = competitorScores[j]; + const unsigned int tmpTick = competitorTicks[j]; + const bool tmpComputorStatus = false; + while (j + && ranksBelow(competitorScores[j - 1], competitorTicks[j - 1], competitorScores[j], competitorTicks[j])) + { + competitorPublicKeys[j] = competitorPublicKeys[j - 1]; + competitorScores[j] = competitorScores[j - 1]; + competitorTicks[j] = competitorTicks[j - 1]; + competitorComputorStatuses[j] = competitorComputorStatuses[j - 1]; + competitorPublicKeys[--j] = tmpPublicKey; + competitorScores[j] = tmpScore; + competitorTicks[j] = tmpTick; + competitorComputorStatuses[j] = tmpComputorStatus; + } + } + + minimumComputorScore = competitorScores[NUMBER_OF_COMPUTORS - QUORUM - 1]; + + unsigned char candidateCounter = 0; + for (unsigned int i = 0; i < (NUMBER_OF_COMPUTORS - QUORUM) * 2; i++) + { + if (!competitorComputorStatuses[i]) + { + minimumCandidateScore = competitorScores[i]; + candidateCounter++; + } + } + if (candidateCounter < NUMBER_OF_COMPUTORS - QUORUM) + { + minimumCandidateScore = minimumComputorScore; + } + + ACQUIRE(minerScoreArrayLock); + for (unsigned int i = 0; i < QUORUM; i++) + { + system.futureComputors[i] = minerPublicKeys[i]; + } + RELEASE(minerScoreArrayLock); + + for (unsigned int i = QUORUM; i < NUMBER_OF_COMPUTORS; i++) + { + system.futureComputors[i] = competitorPublicKeys[i - QUORUM]; + } +} + static void processTickTransactionSolution(const MiningSolutionTransaction* transaction, unsigned int transactionIndex, const unsigned long long processorNumber) { PROFILE_SCOPE(); @@ -2836,6 +3799,11 @@ static void processTickTransactionSolution(const MiningSolutionTransaction* tran ASSERT(transaction->amount >=MiningSolutionTransaction::minAmount() && transaction->inputSize == MiningSolutionTransaction::minInputSize() && transaction->inputType == MiningSolutionTransaction::transactionType()); + // Standalone mining is disabled; bpp9000 is mined through the ant colony only. + if (score_engine::getAlgoType(transaction->nonce.m256i_u8) == score_engine::AlgoType::Bpp9000) + { + return; + } m256i data[3] = { transaction->sourcePublicKey, transaction->miningSeed, transaction->nonce }; static_assert(sizeof(data) == 3 * 32, "Unexpected array size"); @@ -2853,7 +3821,7 @@ static void processTickTransactionSolution(const MiningSolutionTransaction* tran score_engine::AlgoType selectedAlgo = score_engine::getAlgoType(transaction->nonce.m256i_u8); const int threshold = getSolutionThreshold(selectedAlgo); unsigned int solutionScore; - if (isMainMode() || gReRunStrict || isLastTickInEpoch() || forceVerifySolutions) + if (!isTrustingClaimedSolutionScore()) { solutionScore = (*::score)(processorNumber, transaction->sourcePublicKey, transaction->miningSeed, transaction->nonce); } @@ -2870,184 +3838,53 @@ static void processTickTransactionSolution(const MiningSolutionTransaction* tran if (transaction->score == solutionScore && score->isGoodScore(solutionScore, threshold, selectedAlgo)) { - // Solution deposit return - { - increaseEnergy(transaction->sourcePublicKey, transaction->amount); - - const QuTransfer quTransfer = { m256i::zero(), transaction->sourcePublicKey, transaction->amount }; - logger.logQuTransfer(quTransfer); - } - - for (unsigned int i = 0; i < computorSeedsCount; i++) - { - if (transaction->sourcePublicKey == computorPublicKeys[i]) - { - ACQUIRE(solutionsLock); - - unsigned int j; - for (j = 0; j < system.numberOfSolutions; j++) - { - if (transaction->nonce == system.solutions[j].nonce - && transaction->miningSeed == system.solutions[j].miningSeed - && transaction->sourcePublicKey == system.solutions[j].computorPublicKey) - { - solutionPublicationTicks[j] = SOLUTION_RECORDED_FLAG; - - break; - } - } - if (j == system.numberOfSolutions - && system.numberOfSolutions < MAX_NUMBER_OF_SOLUTIONS) - { - system.solutions[system.numberOfSolutions].computorPublicKey = transaction->sourcePublicKey; - system.solutions[system.numberOfSolutions].miningSeed = transaction->miningSeed; - system.solutions[system.numberOfSolutions].nonce = transaction->nonce; - system.solutions[system.numberOfSolutions].score = solutionScore; - solutionPublicationTicks[system.numberOfSolutions++] = SOLUTION_RECORDED_FLAG; - } - - RELEASE(solutionsLock); - - break; - } - } - - // A miner is ranked by its single best score of the epoch, not by the number of - // accepted solutions - const unsigned int newScore = solutionScore * gScoreMultiplier[selectedAlgo]; - const unsigned int newTick = system.tick; - - ACQUIRE(minerScoreArrayLock); - bool minerEntryChanged = false; - unsigned int minerIndex; - for (minerIndex = 0; minerIndex < numberOfMiners; minerIndex++) - { - if (transaction->sourcePublicKey == minerPublicKeys[minerIndex]) - { - if (newScore < minerScores[minerIndex]) - { - minerScores[minerIndex] = newScore; - minerBestScoreTicks[minerIndex] = newTick; - minerEntryChanged = true; - } - - break; - } - } - if (minerIndex == numberOfMiners) - { - if (numberOfMiners < MAX_NUMBER_OF_MINERS) - { - minerPublicKeys[numberOfMiners] = transaction->sourcePublicKey; - minerBestScoreTicks[numberOfMiners] = newTick; - minerScores[numberOfMiners++] = newScore; - minerEntryChanged = true; - } - else - { - // The table is full. Entries beyond the computor block are kept sorted, so the - // worst-ranked one sits at the end and is replaced only if the newcomer outranks it. - const unsigned int worstIndex = numberOfMiners - 1; - if (ranksBelow(minerScores[worstIndex], minerBestScoreTicks[worstIndex], newScore, newTick)) - { - minerPublicKeys[worstIndex] = transaction->sourcePublicKey; - minerScores[worstIndex] = newScore; - minerBestScoreTicks[worstIndex] = newTick; - minerIndex = worstIndex; - minerEntryChanged = true; - } - } - } - - if (minerEntryChanged) - { - const m256i tmpPublicKey = minerPublicKeys[minerIndex]; - const unsigned int tmpScore = minerScores[minerIndex]; - const unsigned int tmpTick = minerBestScoreTicks[minerIndex]; - while (minerIndex > (unsigned int)(minerIndex < NUMBER_OF_COMPUTORS ? 0 : NUMBER_OF_COMPUTORS) - && ranksBelow(minerScores[minerIndex - 1], minerBestScoreTicks[minerIndex - 1], minerScores[minerIndex], minerBestScoreTicks[minerIndex])) - { - minerPublicKeys[minerIndex] = minerPublicKeys[minerIndex - 1]; - minerScores[minerIndex] = minerScores[minerIndex - 1]; - minerBestScoreTicks[minerIndex] = minerBestScoreTicks[minerIndex - 1]; - minerPublicKeys[--minerIndex] = tmpPublicKey; - minerScores[minerIndex] = tmpScore; - minerBestScoreTicks[minerIndex] = tmpTick; - } - } - - // combine 225 worst current computors with 225 best candidates - for (unsigned int i = 0; i < NUMBER_OF_COMPUTORS - QUORUM; i++) - { - competitorPublicKeys[i] = minerPublicKeys[QUORUM + i]; - competitorScores[i] = minerScores[QUORUM + i]; - competitorTicks[i] = minerBestScoreTicks[QUORUM + i]; - competitorComputorStatuses[i] = true; - - if (NUMBER_OF_COMPUTORS + i < numberOfMiners) - { - competitorPublicKeys[i + (NUMBER_OF_COMPUTORS - QUORUM)] = minerPublicKeys[NUMBER_OF_COMPUTORS + i]; - competitorScores[i + (NUMBER_OF_COMPUTORS - QUORUM)] = minerScores[NUMBER_OF_COMPUTORS + i]; - competitorTicks[i + (NUMBER_OF_COMPUTORS - QUORUM)] = minerBestScoreTicks[NUMBER_OF_COMPUTORS + i]; - } - else - { - competitorScores[i + (NUMBER_OF_COMPUTORS - QUORUM)] = NO_MINER_SCORE; - competitorTicks[i + (NUMBER_OF_COMPUTORS - QUORUM)] = 0; - } - competitorComputorStatuses[i + (NUMBER_OF_COMPUTORS - QUORUM)] = false; + // Solution deposit return + { + increaseEnergy(transaction->sourcePublicKey, transaction->amount); + + const QuTransfer quTransfer = { m256i::zero(), transaction->sourcePublicKey, transaction->amount }; + logger.logQuTransfer(quTransfer); } - RELEASE(minerScoreArrayLock); - // bubble sorting -> top 225 from competitorPublicKeys have computors and candidates which are the best from that subset - for (unsigned int i = NUMBER_OF_COMPUTORS - QUORUM; i < (NUMBER_OF_COMPUTORS - QUORUM) * 2; i++) + for (unsigned int i = 0; i < computorSeedsCount; i++) { - int j = i; - const m256i tmpPublicKey = competitorPublicKeys[j]; - const unsigned int tmpScore = competitorScores[j]; - const unsigned int tmpTick = competitorTicks[j]; - const bool tmpComputorStatus = false; - while (j - && ranksBelow(competitorScores[j - 1], competitorTicks[j - 1], competitorScores[j], competitorTicks[j])) + if (transaction->sourcePublicKey == computorPublicKeys[i]) { - competitorPublicKeys[j] = competitorPublicKeys[j - 1]; - competitorScores[j] = competitorScores[j - 1]; - competitorTicks[j] = competitorTicks[j - 1]; - competitorComputorStatuses[j] = competitorComputorStatuses[j - 1]; - competitorPublicKeys[--j] = tmpPublicKey; - competitorScores[j] = tmpScore; - competitorTicks[j] = tmpTick; - competitorComputorStatuses[j] = tmpComputorStatus; - } - } + ACQUIRE(solutionsLock); - minimumComputorScore = competitorScores[NUMBER_OF_COMPUTORS - QUORUM - 1]; + unsigned int j; + for (j = 0; j < system.numberOfSolutions; j++) + { + if (transaction->nonce == system.solutions[j].nonce + && transaction->miningSeed == system.solutions[j].miningSeed + && transaction->sourcePublicKey == system.solutions[j].computorPublicKey) + { + solutionPublicationTicks[j] = SOLUTION_RECORDED_FLAG; - unsigned char candidateCounter = 0; - for (unsigned int i = 0; i < (NUMBER_OF_COMPUTORS - QUORUM) * 2; i++) - { - if (!competitorComputorStatuses[i]) - { - minimumCandidateScore = competitorScores[i]; - candidateCounter++; - } - } - if (candidateCounter < NUMBER_OF_COMPUTORS - QUORUM) - { - minimumCandidateScore = minimumComputorScore; - } + break; + } + } + if (j == system.numberOfSolutions + && system.numberOfSolutions < MAX_NUMBER_OF_SOLUTIONS) + { + system.solutions[system.numberOfSolutions].computorPublicKey = transaction->sourcePublicKey; + system.solutions[system.numberOfSolutions].miningSeed = transaction->miningSeed; + system.solutions[system.numberOfSolutions].nonce = transaction->nonce; + system.solutions[system.numberOfSolutions].score = solutionScore; + solutionPublicationTicks[system.numberOfSolutions++] = SOLUTION_RECORDED_FLAG; + } - ACQUIRE(minerScoreArrayLock); - for (unsigned int i = 0; i < QUORUM; i++) - { - system.futureComputors[i] = minerPublicKeys[i]; - } - RELEASE(minerScoreArrayLock); + RELEASE(solutionsLock); - for (unsigned int i = QUORUM; i < NUMBER_OF_COMPUTORS; i++) - { - system.futureComputors[i] = competitorPublicKeys[i - QUORUM]; + break; + } } + + // A miner is ranked by its single best score of the epoch, not by the number of + // accepted solutions + const unsigned int newScore = solutionScore * gScoreMultiplier[selectedAlgo]; + const unsigned int newTick = system.tick; + updateMinerRankingAndFutureComputors(transaction->sourcePublicKey, newScore, newTick); } else { @@ -3095,6 +3932,214 @@ static void processTickTransactionSolution(const MiningSolutionTransaction* tran } } +// One ant solution: resolve its parent, score the child against it, and commit. Every rejection +// forfeits the deposit by simply not refunding it +// One line per ant solution transaction, whatever became of it. The body follows the logger's own +// convention: compiled out with LOG_CUSTOM_MESSAGES, so a build without qlogging pays nothing here. +static void logAntSolutionOutcome(const AntColonyMiningSolutionTransaction* transaction, + unsigned int score, ValidityResult result) +{ +#if LOG_CUSTOM_MESSAGES + AntSolutionLogMessage logMsg; + logMsg._type = CUSTOM_MESSAGE_ANT_SOLUTION; + logMsg.sourcePublicKey = transaction->sourcePublicKey; + logMsg.nonce = transaction->nonce; + logMsg.parentTick = transaction->parentTick; + logMsg.parentSolutionIndexInTick = transaction->parentSolutionIndexInTick; + logMsg.anchorTick = transaction->anchorTick; + logMsg.score = score; + logMsg.result = (unsigned int)result; + logger.logCustomMessage(logMsg); +#endif +} + +static void processTickTransactionAntColonySolution( + const AntColonyMiningSolutionTransaction* transaction, + unsigned int transactionIndex, + const unsigned long long processorNumber) +{ + AntColonyBpp9000T::Ann& parentAnnScratch = gAntParentAnnScratch[processorNumber]; + AntColonyBpp9000T::Ann& childAnnScratch = gAntChildAnnScratch[processorNumber]; + + const SolutionRef parentRef = { transaction->parentTick, transaction->parentSolutionIndexInTick }; + + // Already looked at, accepted or not. Marked BEFORE the walk below, so one solution costs at + // most one walk for the whole epoch - _dedup alone would only cover the accepted ones. + unsigned int antFlagIndices[2]; + computeAntSolutionFlagIndices(transaction->sourcePublicKey, transaction->nonce, parentRef, antFlagIndices); + if (isAntSolutionSeen(antFlagIndices)) + { + gAntColony.recordReject(ValidityResult::RejectReplay); + logAntSolutionOutcome(transaction, 0, ValidityResult::RejectReplay); + return; + } + markAntSolutionSeen(antFlagIndices); + + // Reject a parent ref into the current or a later tick: a real parent is always on-chain from an + // earlier tick. Root is exempt, it is derived rather than stored. + if (!parentRef.isRoot() && parentRef.tick >= system.tick) + { + gAntColony.recordReject(ValidityResult::RejectParentNotRegistered); + logAntSolutionOutcome(transaction, 0, ValidityResult::RejectParentNotRegistered); + return; + } + + const AntSolutionRecord* parentRec = nullptr; + ValidityResult result = gAntColony.tryGetParent(parentRef, &parentRec); + if (result != ValidityResult::Valid) + { + gAntColony.recordReject(result); + logAntSolutionOutcome(transaction, 0, result); + return; + } + + // The anchor digest seeds the child's mutation walk, so an anchor the ring no longer holds cannot + // be scored + m256i anchorDigest; + if (!gAntColony.getAnchorDigest(transaction->anchorTick, anchorDigest)) + { + gAntColony.recordReject(ValidityResult::RejectStale); + logAntSolutionOutcome(transaction, 0, ValidityResult::RejectStale); + return; + } + + // No walk and no network for this solution. A lie shows up as a refund this node makes and the + // quorum does not, so the disagreement lands in the same tick. + const bool trustClaimedScore = isTrustingClaimedSolutionScore(); + + unsigned int childScore; + const AntColonyBpp9000T::Ann* childAnn = nullptr; + if (trustClaimedScore) + { + childScore = transaction->claimedScore; + } + else if (gAntScoredReady[transactionIndex]) + { + childScore = gAntScoredValue[transactionIndex]; + childAnn = &gAntScoredAnn[transactionIndex]; + } + else + { + // A null parent record means root, the scorer derives the submitter's own root, since roots + // are never stored and so cannot be handed in. + const AntColonyBpp9000T::Ann* parentAnn = nullptr; + if (parentRec != nullptr) + { + const long long parentIdx = gAntColony.findIndexBySolutionRef(parentRef); + if (parentIdx == ANT_INVALID_INDEX + || !ensureAntRecordAnn(processorNumber, (unsigned int)parentIdx, parentAnnScratch)) + { + gAntColony.recordReject(ValidityResult::RejectParentNotRegistered); + logAntSolutionOutcome(transaction, 0, ValidityResult::RejectParentNotRegistered); + return; + } + parentAnn = &parentAnnScratch; + } + + // Same cache the async path uses. Reached when the pre-scan did not enqueue this one or the + // queue did not drain in time, which is exactly the catch-up case the cache exists for. + const AntColonyBpp9000T::ReplayKey replayKey = + makeAntReplayKey(transaction->sourcePublicKey, transaction->nonce, parentRef, anchorDigest); + if (!gAntColony.tryGetReplayScore(replayKey, childScore, childAnnScratch)) + { + childScore = score->computeAntChildScore( + processorNumber, parentAnn, transaction->sourcePublicKey, transaction->nonce, + anchorDigest, childAnnScratch); + gAntColony.putReplayScore(replayKey, childScore, childAnnScratch); + } + childAnn = &childAnnScratch; + } + + // This and the two score rules inside commit() are the only checks decided by the score itself, and none + // may fire on a trusted one: rejecting leaves no refund for the quorum to disagree with, so the tree + // diverges in silence. Accepting turns the lie into a spectrum disagreement instead. + if (!trustClaimedScore && !score->isValidScore(childScore, score_engine::AlgoType::Bpp9000)) + { + gAntColony.recordReject(ValidityResult::RejectNonCanonicalNonce); + logAntSolutionOutcome(transaction, 0, ValidityResult::RejectNonCanonicalNonce); + return; + } + + // Keep previous behavior, we fold both good and bad score into resource testing digest + unsigned int childAnnHash = 0; + if (childAnn != nullptr) + { + KangarooTwelve(childAnn, sizeof(*childAnn), &childAnnHash, sizeof(childAnnHash)); + } + resourceTestingDigest ^= childScore; + resourceTestingDigest ^= childAnnHash; + KangarooTwelve(&resourceTestingDigest, sizeof(resourceTestingDigest), &resourceTestingDigest, sizeof(resourceTestingDigest)); + + const AntCommitInput in = { + transaction->sourcePublicKey, + transaction->nonce, + parentRef, + { system.tick, transactionIndex }, // selfRef, ABSOLUTE tick + transaction->anchorTick, // ABSOLUTE + system.tick }; // publishTick, ABSOLUTE (== selfRef.tick) + // Seeing this transaction execute, that mean those solution is recognized on-chain, mark them as recorded + for (unsigned int ownIdx = 0; ownIdx < computorSeedsCount; ownIdx++) + { + if (transaction->sourcePublicKey == computorPublicKeys[ownIdx]) + { + gAntPendingSolutions.markRecorded(transaction->sourcePublicKey, parentRef, transaction->nonce); + break; + } + } + + result = gAntColony.commit(in, parentRec, childScore, childAnn, childAnnHash, trustClaimedScore); + logAntSolutionOutcome(transaction, childScore, result); + antDebugAccepted(transaction, childScore, (parentRec != nullptr) ? (parentRec->depth + 1) : 1, + transactionIndex, result, trustClaimedScore); + + // An accept that only stood because the score was trusted is the one that will disagree with quorum. + if (trustClaimedScore + && (result == ValidityResult::Valid || result == ValidityResult::ValidNotStored)) + { + const unsigned int parentScore = (parentRec != nullptr) ? parentRec->score : WORST_SCORE; + const bool wouldHaveRejected = (childScore > gAntColony.errorThreshold()) + || (childScore >= parentScore) + || !score->isValidScore(childScore, score_engine::AlgoType::Bpp9000); + if (wouldHaveRejected && antDebugCanPrint()) + { + CHAR16 msg[256]; + setText(msg, L"[ant-colony] over-accepted on a trusted score, tick "); + appendNumber(msg, system.tick, FALSE); + appendText(msg, L" idx "); + appendNumber(msg, transactionIndex, FALSE); + appendText(msg, L" score "); + appendNumber(msg, childScore, FALSE); + appendText(msg, L" threshold "); + appendNumber(msg, gAntColony.errorThreshold(), FALSE); + appendText(msg, L" parentScore "); + appendNumber(msg, parentScore, FALSE); + appendText(msg, L" - expect a spectrum disagreement this tick"); + logToConsole(msg); + } + } + // ValidNotStored is the store being full: the solution passed every rule and only missed a slot, + // so it earns its refund and its ranking exactly like a stored one + if (result != ValidityResult::Valid && result != ValidityResult::ValidNotStored) + { + return; // commit() counted this one itself + } + + // Refund AND ranking. A valid solution is refunded whether or not it improved this miner's best, + // and whether or not the store had room for it, ranking is best-score-only + if (transaction->claimedScore == childScore) + { + // Refund if this score == its claimed score and the ann tree check + increaseEnergy(transaction->sourcePublicKey, transaction->amount); + + const QuTransfer quTransfer = { m256i::zero(), transaction->sourcePublicKey, transaction->amount }; + logger.logQuTransfer(quTransfer); + + // A miner is ranked by its single best score of the epoch + const unsigned int newScore = childScore * gScoreMultiplier[score_engine::AlgoType::Bpp9000]; + updateMinerRankingAndFutureComputors(transaction->sourcePublicKey, newScore, system.tick); + } +} + static void processTickTransaction(const Transaction* transaction, unsigned int transactionIndex, const unsigned long long txOffset, unsigned long long processorNumber) { PROFILE_SCOPE(); @@ -3223,6 +4268,19 @@ static void processTickTransaction(const Transaction* transaction, unsigned int } break; + case AntColonyMiningSolutionTransaction::transactionType(): + { + // Exact inputSize, not >=: the payload is fixed and a longer one is not a + // forward-compatible variant, it is a different transaction. + if (transaction->amount >= AntColonyMiningSolutionTransaction::minAmount() + && transaction->inputSize == AntColonyMiningSolutionTransaction::minInputSize()) + { + processTickTransactionAntColonySolution( + (AntColonyMiningSolutionTransaction*)transaction, transactionIndex, processorNumber); + } + } + break; + case OracleReplyCommitTransactionPrefix::transactionType(): { oracleEngine.processOracleReplyCommitTransaction((OracleReplyCommitTransactionPrefix*)transaction); @@ -3504,6 +4562,90 @@ static bool makeAndBroadcastExecutionFeeTransaction(int i, BroadcastFutureTickDa return true; } +OPTIMIZE_OFF() + +// Publish at most one queued ant solution for computor i, retrying anything whose transaction never +// landed. Mirrors the legacy solution publisher: the tick the transaction is targeted at is also the +// deadline to see it on-chain, so missing it is what triggers the retry. +static void publishAntSolutionFor(unsigned long long processorNumber, unsigned int computorIndex) +{ + AntPendingSolution entry; + const unsigned int idx = gAntPendingSolutions.selectForPublish( + computorPublicKeys[computorIndex], system.tick, entry); + if (idx == AntPendingSolutions::NO_ENTRY) + { + return; + } + + const AntSolutionRecord* parentRec = nullptr; + if (gAntColony.tryGetParent(entry.parentRef, &parentRec) != ValidityResult::Valid) + { + gAntPendingSolutions.markObsoleteParentGone(idx); + antDebugPending(L"retire parentGone", entry, 0); + return; + } + + m256i anchorDigest; + if (!gAntColony.getAnchorDigest(entry.anchorTick, anchorDigest)) + { + // The ring no longer holds it, so this node could not score the transaction it is about to + // publish - and neither could anyone else. + gAntPendingSolutions.markObsoleteExpired(idx); + antDebugPending(L"retire expired", entry, 0); + return; + } + + // Last point before the node signs with its own computor key and funds the deposit from its own + // balance. The commit path forfeits the deposit on a non-canonical nonce, and a check this cheap + // belongs on both sides of the queue. + if (!score_engine::ScoreEngineT::isCanonicalAntNonce(entry.nonce.m256i_u8)) + { + gAntPendingSolutions.markObsoleteGateRejected(idx); + antDebugPending(L"retire gateRejected", entry, 0); + return; + } + + // The score was computed at receipt, on a request processor, and the entry has + // carried it since. + // Judged at the tick the transaction will EXECUTE in, not the current one, so this gate sees + // what commit will see - a solution at the window boundary now would commit stale. + const unsigned int publishTick = system.tick + MIN_MINING_SOLUTIONS_PUBLICATION_OFFSET; + const unsigned int childCount = gAntColony.childCountForQuery(entry.parentRef, + entry.computorPublicKey); + const ChildCandidate candidate{ entry.computorPublicKey, entry.score, entry.anchorTick, publishTick }; + if (AntColonyBpp9000T::validateChild(candidate, parentRec, childCount, + gAntColony.errorThreshold()) != ValidityResult::Valid) + { + gAntPendingSolutions.markObsoleteGateRejected(idx); + antDebugPending(L"retire gateRejected", entry, 0); + return; + } + + AntColonyMiningSolutionTransaction payload; + setMem(&payload, sizeof(payload), 0); + payload.sourcePublicKey = computorPublicKeys[computorIndex]; + payload.destinationPublicKey = m256i::zero(); + payload.amount = AntColonyMiningSolutionTransaction::minAmount(); + payload.tick = publishTick; + payload.inputType = AntColonyMiningSolutionTransaction::transactionType(); + payload.inputSize = AntColonyMiningSolutionTransaction::minInputSize(); + payload.parentTick = entry.parentRef.tick; + payload.parentSolutionIndexInTick = entry.parentRef.solutionIndexInTick; + payload.anchorTick = entry.anchorTick; + payload.claimedScore = entry.score; + payload.nonce = entry.nonce; + + unsigned char digest[32]; + KangarooTwelve(&payload, sizeof(Transaction) + AntColonyMiningSolutionTransaction::minInputSize(), + digest, sizeof(digest)); + sign(computorSubseeds[computorIndex].m256i_u8, computorPublicKeys[computorIndex].m256i_u8, + digest, payload.signature); + + enqueueResponse(NULL, sizeof(payload), BROADCAST_TRANSACTION, 0, &payload); + gAntPendingSolutions.markScheduled(idx, (int)payload.tick); + antDebugPending(L"published", entry, payload.tick); +} + static void processTick(unsigned long long processorNumber) { PROFILE_SCOPE(); @@ -3626,6 +4768,7 @@ static void processTick(unsigned long long processorNumber) ts.tickData.acquireLock(); copyMem(&nextTickData, &ts.tickData[tickIndex], sizeof(TickData)); ts.tickData.releaseLock(); + unsigned long long solutionProcessStartTick = __rdtsc(); // for tracking the time processing solutions if (nextTickData.epoch == system.epoch) { @@ -3633,13 +4776,17 @@ static void processTick(unsigned long long processorNumber) #if ADDON_TX_STATUS_REQUEST txStatusData.tickTxIndexStart[system.tick - system.initialTick] = numberOfTransactions; // qli: part of tx_status_request add-on #endif - // Only apply skipping compute solution when in Mainnet with Aux node (except for last tick) - if (isMainMode() || isTestnet() || isLastTickInEpoch() || forceVerifySolutions) + // Only apply skipping compute solution when in Mainnet with Aux node (except for last tick). + // A strict replay computes too, and this is what clears the per-transaction pre-score gates so it + // cannot read the previous tick's answers. + if (isMainMode() || isTestnet() || isLastTickInEpoch() || forceVerifySolutions || gReRunStrict) { PROFILE_NAMED_SCOPE_BEGIN("processTick(): pre-scan solutions"); unsigned long long _bPrescanStart = __rdtsc(); // reset solution task queue score->resetTaskQueue(); + // Only the gate needs clearing; the score and the network are written before it is set. + setMem(gAntScoredReady, sizeof(gAntScoredReady), 0); // pre-scan any solution tx and add them to solution task queue for (unsigned int transactionIndex = 0; transactionIndex < NUMBER_OF_TRANSACTIONS_PER_TICK; transactionIndex++) { @@ -3672,10 +4819,34 @@ static void processTick(unsigned long long processorNumber) if (!(minerSolutionFlags[flagIndices[0] >> 6] & (1ULL << (flagIndices[0] & 63))) || !(minerSolutionFlags[flagIndices[1] >> 6] & (1ULL << (flagIndices[1] & 63)))) { - score->addTask(transaction->sourcePublicKey, solution_miningSeed, solution_nonce); + score->addTask(scoreLegacySolutionTask, data, sizeof(data)); } } } + // Ant solutions ride the same async queue + else if (isZero(transaction->destinationPublicKey) + && transaction->amount >= AntColonyMiningSolutionTransaction::minAmount() + && transaction->inputType == AntColonyMiningSolutionTransaction::transactionType() + && transaction->inputSize == AntColonyMiningSolutionTransaction::minInputSize()) + { + const AntColonyMiningSolutionTransaction* antTx = + (const AntColonyMiningSolutionTransaction*)transaction; + AntScoreTaskPayload task; + task.pubkey = antTx->sourcePublicKey; + task.nonce = antTx->nonce; + task.parentRef.tick = antTx->parentTick; + task.parentRef.solutionIndexInTick = antTx->parentSolutionIndexInTick; + task.anchorTick = antTx->anchorTick; + task.txIdx = transactionIndex; + // Skip anything this node has already looked at, accepted or not, + // and anything this tick will take on its claimed score anyway + unsigned int antFlagIndices[2]; + computeAntSolutionFlagIndices(task.pubkey, task.nonce, task.parentRef, antFlagIndices); + if (!isAntSolutionSeen(antFlagIndices) && !isTrustingClaimedSolutionScore()) + { + score->addTask(scoreAntSolutionTask, &task, sizeof(task)); + } + } } } } @@ -3684,15 +4855,11 @@ static void processTick(unsigned long long processorNumber) TickBench::add(TickBench::PRESCAN_SOLUTIONS, _bPrescanStart, __rdtsc()); PROFILE_SCOPE_END(); { - // Process solutions in this tick and store in cache. In parallel, score->tryProcessSolution() is called by - // request processors to speed up solution processing. + // Process solutions in this tick and store in cache. In parallel, request processors call + // score->tryProcessOneTask() from their idle path to speed this up. PROFILE_NAMED_SCOPE("processTick(): process solutions"); TickBench::Scope _bProcSol(TickBench::PROCESS_SOLUTIONS); - score->startProcessTaskQueue(); - while (!score->isTaskQueueProcessed()) { - score->tryProcessSolution(processorNumber); - } - score->stopProcessTaskQueue(); + score->runUntilDone(processorNumber); } } @@ -3797,6 +4964,25 @@ static void processTick(unsigned long long processorNumber) } TickBench::add(TickBench::PROCESS_TXS, _bProcTxsStart, __rdtsc()); PROFILE_SCOPE_END(); + + // Ant colony anchor for this non-empty tick, K12(tick || transactionDigest) + { + m256i anchorTxDigest; + KangarooTwelve(&nextTickData, sizeof(TickData), &anchorTxDigest, 32); + m256i anchorDigest; + computeAntAnchorDigest(system.tick, anchorTxDigest, anchorDigest); + gAntColony.recordAnchorDigest(system.tick, anchorDigest); + +#if !defined(NDEBUG) + // Rebuilds recompute this from tick storage; check the two agree while both are available. + m256i recomputedAnchorDigest; + if (!recomputeAntAnchorDigest(system.tick, recomputedAnchorDigest) + || recomputedAnchorDigest != anchorDigest) + { + addDebugMessage(L"ant anchor recomputed from tick storage disagrees with the ring"); + } +#endif + } } else { @@ -4490,6 +5676,9 @@ static void processTick(unsigned long long processorNumber) enqueueResponse(NULL, sizeof(payload), BROADCAST_TRANSACTION, 0, &payload); } + + // Re-publish the solutions that are not on chain + publishAntSolutionFor(processorNumber, i); } } @@ -4509,6 +5698,52 @@ static void processTick(unsigned long long processorNumber) } } + // TEST: mine ant solutions against our own colony to drive the inputType-12 path. A walk + // takes seconds, so it runs on its own thread like an external miner would, never on the + // tick processor. + if (forceBroadcastAntSolution && computorSeedsCount > 0) + { + static bool antMinerStarted = false; + if (!antMinerStarted) + { + antMinerStarted = true; + std::thread([]() + { + unsigned int lastMinedTick = 0; + unsigned int published = 0; + unsigned int lastPublishTick = 0; + while (!shutDownNode && published < forceAntSolutionBudget) + { + const unsigned int tick = system.tick; + const bool gapElapsed = (lastPublishTick == 0) + || (tick - lastPublishTick >= forceAntInjectGapTicks); + if (tick != lastMinedTick && tick > system.initialTick && gapElapsed) + { + lastMinedTick = tick; + const TestInvalidSolution::AntInjectMode mode = + (published < forceAntInjectWarmup) + ? TestInvalidSolution::AntInjectMode::Valid + : forceAntInjectMode; + // Engine slot is processorNumber % solutionBufferCount; the tick processor is + // processor 1, so use 0 to keep the miner's walk off the verifier's lock. + if (TestInvalidSolution::broadcastAntSolution(gAntColony, *score, 0, + tick - 1, mode, 1)) + { + published++; + lastPublishTick = tick; + } + } + std::this_thread::sleep_for(std::chrono::milliseconds(50)); + } + CHAR16 doneLine[128]; + setText(doneLine, L"ANT-INJECT budget spent, miner thread exiting after "); + appendNumber(doneLine, published, FALSE); + appendText(doneLine, L" solution(s)"); + logToConsole(doneLine); + }).detach(); + } + } + #ifndef NDEBUG // Check that continuous updating of spectrum info is consistent with counting from scratch SpectrumInfo si; @@ -4640,6 +5875,7 @@ static void beginEpoch() score->initMemory(); score->resetTaskQueue(); setMem(minerSolutionFlags, NUMBER_OF_MINER_SOLUTION_FLAGS / 8, 0); + setMem(gAntSolutionFlags, NUMBER_OF_ANT_SOLUTION_FLAGS / 8, 0); setMem((void*)minerPublicKeys, sizeof(minerPublicKeys), 0); setMem((void*)minerScores, sizeof(minerScores), 0xFF); setMem((void*)minerBestScoreTicks, sizeof(minerBestScoreTicks), 0); @@ -4973,9 +6209,18 @@ static bool saveAllNodeStates() forceLogToConsoleAsAddDebugMessage = true; #endif + CHAR16 snapshotDirectory[16]; + setText(snapshotDirectory, L"ep"); + appendNumber(snapshotDirectory, system.epoch, false); + + // Stage the save in a side directory so an interrupted save leaves the previous snapshot loadable. CHAR16 directory[16]; - setText(directory, L"ep"); - appendNumber(directory, system.epoch, false); + setText(directory, snapshotDirectory); + appendText(directory, L".tmp"); + + CHAR16 previousDirectory[16]; + setText(previousDirectory, snapshotDirectory); + appendText(previousDirectory, L".old"); logToConsole(L"Start saving node states from main thread"); @@ -5060,6 +6305,7 @@ static bool saveAllNodeStates() } score->saveScoreCache(system.epoch, directory); + gAntColony.saveReplayCache(system.epoch, directory); copyMem(&nodeStateBuffer.etalonTick, &etalonTick, sizeof(etalonTick)); copyMem(nodeStateBuffer.minerPublicKeys, (void*)minerPublicKeys, sizeof(minerPublicKeys)); @@ -5145,6 +6391,14 @@ static bool saveAllNodeStates() return false; } + logToConsole(L"Saving ant solution flags"); + savedSize = save(ANT_SOL_FLAG_FILE_NAME, NUMBER_OF_ANT_SOLUTION_FLAGS / 8, (unsigned char*)gAntSolutionFlags, directory); + if (savedSize != NUMBER_OF_ANT_SOLUTION_FLAGS / 8) + { + logToConsole(L"Failed to save ant solution flag"); + return false; + } + setText(message, L"Saving tick storage "); logToConsole(message); if (ts.trySaveToFile(system.epoch, system.tick, directory) != 0) @@ -5156,6 +6410,11 @@ static bool saveAllNodeStates() #if !defined(NDEBUG) oracleEngine.checkStateConsistencyWithAssert(); #endif + if (!gAntColony.saveSnapshot(system.epoch, directory, system.initialTick)) + { + return false; + } + if (!oracleEngine.saveSnapshot(system.epoch, directory)) { return false; @@ -5184,6 +6443,18 @@ static bool saveAllNodeStates() forceLogToConsoleAsAddDebugMessage = false; #endif + // Promote the staged directory only now that every component is on disk. Two renames instead of + // a remove: deleting the previous snapshot takes seconds, and a kill in that window leaves nothing. + removeDir(previousDirectory); + renameDir(snapshotDirectory, previousDirectory); // absent on the first save + if (!renameDir(directory, snapshotDirectory)) + { + renameDir(previousDirectory, snapshotDirectory); + logToConsole(L"Failed to promote snapshot"); + return false; + } + removeDir(previousDirectory); + return true; } @@ -5408,6 +6679,32 @@ static bool loadAllNodeStates() return false; } + logToConsole(L"Loading ant solution flags"); + loadedSize = load(ANT_SOL_FLAG_FILE_NAME, NUMBER_OF_ANT_SOLUTION_FLAGS / 8, (unsigned char*)gAntSolutionFlags, directory); + if (loadedSize != NUMBER_OF_ANT_SOLUTION_FLAGS / 8) + { + logToConsole(L"Failed to load ant solution flag"); + return false; + } + + // initialRandomSeedFromPersistingState, not score->currentRandomSeed, the scorer is not reseeded + // until initialize() finishes, so this is the only restored copy available here. + if (!gAntColony.loadSnapshot(system.epoch, directory, + initialRandomSeedFromPersistingState, + (unsigned int)getSolutionThreshold(score_engine::AlgoType::Bpp9000), + system.initialTick)) + { + return false; + } +#ifndef NDEBUG + { + CHAR16 dbg[128]; + setText(dbg, L"[ant-colony] snapshot loaded, solutions="); + appendNumber(dbg, gAntColony.solutionCount(), FALSE); + logToConsole(dbg); + } +#endif + if (!oracleEngine.loadSnapshot(system.epoch, directory)) { return false; @@ -6597,6 +7894,7 @@ static void tickProcessor(void*, unsigned long long processorNumber) gReRunStrict = false; latestProcessedTick = system.tick; + // safety check for contract locks // after processing a tick, all contract locks should be released checkAllContractLocksReleased(); @@ -7103,6 +8401,16 @@ static void tickProcessor(void*, unsigned long long processorNumber) asyncSave(REVENUE_DATA_END_OF_EPOCH_FILE_NAME, sizeof(gEpochRevenueData), (unsigned char*)&gEpochRevenueData); // Multi-dim revenue (shadow) - for offline comparison against the additive asyncSave(MULTIDIM_REVENUE_END_OF_EPOCH_FILE_NAME, sizeof(gMultiDimRevenue), (unsigned char*)&gMultiDimRevenue); + // The epoch's best networks, for offline extraction +#ifndef NDEBUG + { + CHAR16 dbg[768]; + setText(dbg, L"[ant-colony] epoch end: "); + gAntColony.stats().appendLog(dbg); + logToConsole(dbg); + } +#endif + gAntColony.exportBestSolutions(system.epoch, NULL); // Reorder futureComputors so requalifying computors keep their index // This is needed for correct execution fee reporting across epoch boundaries @@ -7680,11 +8988,24 @@ static bool initialize() setMem(score_qpi, sizeof(*score_qpi), 0); #endif + if (!gAntPendingSolutions.init()) + { + return false; + } + if (!gAntColony.init()) + { + return false; + } + setMem(&solutionThreshold[0][0], sizeof(int) * MAX_NUMBER_EPOCH * score_engine::AlgoType::MaxAlgoCount, 0); if (!allocPoolWithErrorLog(L"minserSolutionFlag", NUMBER_OF_MINER_SOLUTION_FLAGS / 8, (void**)&minerSolutionFlags, __LINE__)) { return false; } + if (!allocPoolWithErrorLog(L"antSolutionFlag", NUMBER_OF_ANT_SOLUTION_FLAGS / 8, (void**)&gAntSolutionFlags, __LINE__)) + { + return false; + } if (!customQubicMiningStorage.init()) { @@ -7950,16 +9271,34 @@ static bool initialize() { score->initMiningData(initialRandomSeedFromPersistingState); loadMiningSeedFromFile = false;; + // Skipped entirely when a snapshot was restored + if (!loadAllNodeStateFromFile) + { + antColonyBeginEpoch(); + } } else { - short tickEpoch = -1; + short tickEpoch = -1; TimeDate tickDate; setMem((void*)&tickDate, sizeof(TimeDate), 0); checkAndSwitchMiningPhase(tickEpoch, tickDate, true); - } + } score->loadScoreCache(system.epoch); + // After the branch above, never before: both paths can call antColonyBeginEpoch(), which clears + // the cache. A memo, not state - absence or any load failure just means the solutions get + // computed honestly. + gAntColony.loadReplayCache(system.epoch, NULL); +#ifndef NDEBUG + { + CHAR16 dbg[128]; + setText(dbg, L"[ant-colony] replay cache loaded, occupancy="); + appendNumber(dbg, gAntColony.replayCacheOccupancy(), FALSE); + logToConsole(dbg); + } +#endif + // Load + hash-verify the bpp9000 task once at init if (!loadBpp9000Task()) { @@ -8150,6 +9489,8 @@ static void deinitialize() pendingTxsPool.deinit(); fastTxWindow.deinit(); + gAntPendingSolutions.deinit(); + gAntColony.deinit(); if (score) { @@ -8159,6 +9500,10 @@ static void deinitialize() { freePoolOrVirtual(minerSolutionFlags); } + if (gAntSolutionFlags) + { + freePool(gAntSolutionFlags); + } if (dejavu0) { @@ -8921,6 +10266,47 @@ static void processKeyPresses() setText(message, L"DogeMining: "); gDogeMiningStats.appendLog(message); logToConsole(message); + + setText(message, L"AntColony: "); + gAntColony.stats().appendLog(message); + appendText(message, L" | replay cache "); + appendNumber(message, gAntColony.replayCacheOccupancy(), TRUE); + logToConsole(message); + + AntPendingSolutions::Stats pending; + unsigned int pendingCount = 0; + gAntPendingSolutions.getStats(pending, pendingCount); + setText(message, L"AntPool: queued "); + appendNumber(message, pendingCount, TRUE); + appendText(message, L" | received "); + appendNumber(message, pending.received, TRUE); + appendText(message, L" | published "); + appendNumber(message, pending.published, TRUE); + appendText(message, L" | recorded "); + appendNumber(message, pending.recorded, TRUE); + appendText(message, L" | dropped: nonCanonical "); + appendNumber(message, pending.droppedNonCanonical, TRUE); + appendText(message, L", badAnchor "); + appendNumber(message, pending.droppedBadAnchor, TRUE); + appendText(message, L", parentUnknown "); + appendNumber(message, pending.droppedParentUnknown, TRUE); + appendText(message, L", unscorable "); + appendNumber(message, pending.droppedUnscorable, TRUE); + appendText(message, L", unacceptable "); + appendNumber(message, pending.droppedUnacceptable, TRUE); + appendText(message, L", duplicate "); + appendNumber(message, pending.droppedDuplicate, TRUE); + appendText(message, L", full "); + appendNumber(message, pending.droppedFull, TRUE); + appendText(message, L" | obsolete: parentGone "); + appendNumber(message, pending.obsoleteParentGone, TRUE); + appendText(message, L", expired "); + appendNumber(message, pending.obsoleteExpired, TRUE); + appendText(message, L", gateRejected "); + appendNumber(message, pending.obsoleteGateRejected, TRUE); + appendText(message, L" | claim mismatch "); + appendNumber(message, pending.claimMismatch, TRUE); + logToConsole(message); } break; @@ -9243,6 +10629,17 @@ static void tickForkChildPromote(unsigned int strictUntilTick) // ── strict-replay window ── gReRunStrict = true; gReRunStrictUntilTick = strictUntilTick; + { + // The range explains the catch-up that follows: every solution in it is recomputed, and a parent + // accepted on trust is rebuilt too. + CHAR16 strictLine[192]; + setText(strictLine, L"[FORK] CHILD: replaying ticks "); + appendNumber(strictLine, (unsigned long long)system.tick, FALSE); + appendText(strictLine, L".."); + appendNumber(strictLine, (unsigned long long)strictUntilTick, FALSE); + appendText(strictLine, L" strict"); + logToConsole(strictLine); + } // ── networking — drop parent connection state, keep COW-shared buffers ── Overload::resetForChildPromote(); @@ -9781,7 +11178,8 @@ EFI_STATUS efi_main(EFI_HANDLE imageHandle, EFI_SYSTEM_TABLE* systemTable) } #if !TICK_STORAGE_AUTOSAVE_MODE - // Only save system + score cache to file regularly here if on AUX and snapshot auto-save is disabled + // Only save system + score cache + ant replay cache to file regularly here if on AUX + // and snapshot auto-save is disabled if ((!isMainMode()) && curTimeTick - systemDataSavingTick >= SYSTEM_DATA_SAVING_PERIOD * frequency / 1000) { @@ -9789,6 +11187,7 @@ EFI_STATUS efi_main(EFI_HANDLE imageHandle, EFI_SYSTEM_TABLE* systemTable) saveSystem(); score->saveScoreCache(system.epoch); + gAntColony.saveReplayCache(system.epoch, NULL); } #endif tryResendTickVotes(); @@ -10152,6 +11551,7 @@ EFI_STATUS efi_main(EFI_HANDLE imageHandle, EFI_SYSTEM_TABLE* systemTable) saveSystem(); score->saveScoreCache(system.epoch); + gAntColony.saveReplayCache(system.epoch, NULL); #ifdef ENABLE_PROFILING gProfilingDataCollector.writeToFile(); #endif @@ -10253,6 +11653,20 @@ unsigned long long getTotalRam() // minerSolutionFlags (qubic.cpp:7068) add("minerSolutionFlags", NUMBER_OF_MINER_SOLUTION_FLAGS / 8); + add("gAntSolutionFlags", NUMBER_OF_ANT_SOLUTION_FLAGS / 8); + + // ant colony pools (allocPoolWithErrorLog in AntColony::init) + add("antColony", + AntColonyBpp9000T::ANT_RECORDS_BYTES + + AntColonyBpp9000T::ANT_ANN_POOL_BYTES + + AntColonyBpp9000T::ANT_REPLAY_CACHE_BYTES + + (unsigned long long)MAX_NUMBER_OF_TICKS_PER_EPOCH * sizeof(AntTickSlot) + + sizeof(AnchorRing) + + sizeof(QPI::HashMap) + + sizeof(QPI::HashMap) + + sizeof(QPI::HashSet) + + sizeof(AntColonyBpp9000T::ExportSet) + + ANT_SNAPSHOT_SCRATCH_BYTES); // contractLocalsStack array (contract_exec.h:45) add("contractLocalsStack", NUMBER_OF_CONTRACT_EXECUTION_BUFFERS * (unsigned long long)ContractLocalsStack::capacity()); @@ -10343,6 +11757,10 @@ void processArgs(int argc, const char* argv[]) { ("fbis-count", "TEST: number of solution txs per tick with --fbis", cxxopts::value()->default_value("1")) ("fbis-same", "TEST: inject all --fbis solutions from one computor", cxxopts::value()) ("test-solution-threshold", "TEST: override runtime Bpp9000 solution threshold", cxxopts::value()->default_value("-1")) + ("fbas,force-broadcast-ant-solution", "TEST: mine ant-colony solutions against our own colony and publish them, as or : (default 3; a walk costs seconds of CPU and each publish burns a deposit). Mode selects which accept rule it aims at: valid, badclaim, noncanon, wrongtree, stale, futureparent, leparent", cxxopts::value()) + ("fbas-warmup", "TEST: publish this many valid ant solutions before switching to the --fbas mode", cxxopts::value()->default_value("0")) + ("fbas-gap", "TEST: minimum ticks between ant publishes; a gap wider than the fork window makes each window retire", cxxopts::value()->default_value("0")) + ("ant-debug", "Trace ant-colony accepts, over-accepts and network rebuilds (budgeted per epoch)", cxxopts::value()) #if defined(__linux__) && !defined(LITE_WASM_SC) ("verify-fork-rollback", "TEST: assert fork re-run reproduces quorum digest", cxxopts::value()) ("fork-force-fork", "TEST: fork every tick (exercise MATCH path)", cxxopts::value()) @@ -10557,6 +11975,13 @@ void processArgs(int argc, const char* argv[]) { logColorToScreen("INFO", "Using testnet go behind trick"); } + if (result.count("ant-debug")) + { + gAntDebugEnabled = true; + logColorToScreen("INFO", "Ant colony tracing enabled, " + + std::to_string(ANT_DEBUG_PRINTS_PER_EPOCH) + " lines per epoch"); + } + if (result.count("rebuild-tx-hashmap")) { rebuildTxHashmap = true; @@ -10710,6 +12135,51 @@ void processArgs(int argc, const char* argv[]) { forceBroadcastInvalidSolution = true; logColorToScreen("INFO", "Force broadcast invalid solution enabled (TEST ONLY)"); } + + if (result.count("force-broadcast-ant-solution")) + { + std::string mode = result["force-broadcast-ant-solution"].as(); + // "mode" or "mode:count" - a walk costs ~25s of CPU and each publish burns a deposit, so the + // injector stops after a small budget rather than mining every tick forever. + const size_t colonPos = mode.find(':'); + if (colonPos != std::string::npos) + { + forceAntSolutionBudget = (unsigned int)std::stoul(mode.substr(colonPos + 1)); + mode = mode.substr(0, colonPos); + } + forceBroadcastAntSolution = true; + if (mode == "valid") forceAntInjectMode = TestInvalidSolution::AntInjectMode::Valid; + else if (mode == "badclaim") forceAntInjectMode = TestInvalidSolution::AntInjectMode::BadClaim; + else if (mode == "noncanon") forceAntInjectMode = TestInvalidSolution::AntInjectMode::NonCanonical; + else if (mode == "wrongtree") forceAntInjectMode = TestInvalidSolution::AntInjectMode::WrongTree; + else if (mode == "stale") forceAntInjectMode = TestInvalidSolution::AntInjectMode::Stale; + else if (mode == "futureparent") forceAntInjectMode = TestInvalidSolution::AntInjectMode::FutureParent; + else if (mode == "leparent") forceAntInjectMode = TestInvalidSolution::AntInjectMode::LeParent; + else + { + forceBroadcastAntSolution = false; + logColorToScreen("ERROR", "Unknown --force-broadcast-ant-solution mode: " + mode); + } + // Clamped by comparison, not std::max: the legacy Qubic.sln build has no NOMINMAX, so + // 's max macro would eat the call. + if (result.count("fbas-warmup")) + { + const int warmup = result["fbas-warmup"].as(); + forceAntInjectWarmup = (warmup > 0) ? (unsigned int)warmup : 0u; + } + if (result.count("fbas-gap")) + { + const int gapTicks = result["fbas-gap"].as(); + forceAntInjectGapTicks = (gapTicks > 0) ? (unsigned int)gapTicks : 0u; + } + if (forceBroadcastAntSolution) + { + logColorToScreen("INFO", "Force broadcast ant solution enabled, mode " + mode + + ", budget " + std::to_string(forceAntSolutionBudget) + " solution(s)" + + ", warmup " + std::to_string(forceAntInjectWarmup) + + ", gap " + std::to_string(forceAntInjectGapTicks) + " tick(s) (TEST ONLY)"); + } + } } #if defined(__linux__) && !defined(NO_RPC) && !defined(TESTNET) diff --git a/src/score.h b/src/score.h index e2e04681..480213e4 100644 --- a/src/score.h +++ b/src/score.h @@ -18,20 +18,9 @@ enum ScoreStatus ScoreStatusTaskNotLoaded, }; -template -struct ScoreFunction +namespace score_engine { - score_engine::ScoreEngine< - score_engine::NeuraxonParams< - NEURAXON_NUMBER_OF_INPUT_NEURONS, - NEURAXON_NUMBER_OF_OUTPUT_NEURONS, - NEURAXON_NUMBER_OF_TICKS, - NEURAXON_NUMBER_OF_NEIGHBORS, - NEURAXON_POPULATION_THRESHOLD, - NEURAXON_NUMBER_OF_MUTATIONS, - NEURAXON_SOLUTION_THRESHOLD_DEFAULT>, - - score_engine::Bpp9000Params< + using Bpp9000ParamsT = Bpp9000Params< BPP9000_NUMBER_OF_INPUT_NEURONS, BPP9000_NUMBER_OF_OUTPUT_NEURONS, BPP9000_SEQUENCE_LENGTH, @@ -40,9 +29,37 @@ struct ScoreFunction BPP9000_NUMBER_OF_NEIGHBORS, BPP9000_POPULATION_THRESHOLD, BPP9000_NUMBER_OF_MUTATIONS, - BPP9000_SOLUTION_THRESHOLD_DEFAULT> - > _computeBuffer[solutionBufferCount]; + BPP9000_SOLUTION_THRESHOLD_DEFAULT>; + using NeuraxonParamsT = NeuraxonParams< + NEURAXON_NUMBER_OF_INPUT_NEURONS, + NEURAXON_NUMBER_OF_OUTPUT_NEURONS, + NEURAXON_NUMBER_OF_TICKS, + NEURAXON_NUMBER_OF_NEIGHBORS, + NEURAXON_POPULATION_THRESHOLD, + NEURAXON_NUMBER_OF_MUTATIONS, + NEURAXON_SOLUTION_THRESHOLD_DEFAULT>; + + // The bpp9000 scorer the ant colony branches on; exposes ANN (the inheritable per-neuron LUT). + using ScoreBpp9000T = ScoreBpp9000; + + using ScoreEngineT = ScoreEngine; +} + +template +struct ScoreFunction +{ +private: + // The engine scratch buffers and the locks guarding them. Private on purpose: a work function run + // by the task queue cannot reach them, so it cannot take a slot lock and then call a method that + // takes the same one. Every route into the engine locks exactly once, inside this class. + score_engine::ScoreEngineT _computeBuffer[solutionBufferCount]; + volatile char solutionEngineLock[solutionBufferCount]; + + // Scratch for the ant root derivation, one per engine slot and covered by that slot's own lock + score_engine::ScoreBpp9000T::ANN _antRootScratch[solutionBufferCount]; + +public: volatile char random2PoolLock; unsigned char state[score_engine::STATE_SIZE]; unsigned char externalPoolVec[score_engine::POOL_VEC_PADDING_SIZE]; @@ -64,8 +81,6 @@ struct ScoreFunction m256i currentRandomSeed; - volatile char solutionEngineLock[solutionBufferCount]; - #if USE_SCORE_CACHE volatile char scoreCacheLock; ScoreCache scoreCache; @@ -86,9 +101,8 @@ struct ScoreFunction } currentRandomSeed = randomSeed; // persist the initial random seed to be able to send it back on system info response - ACQUIRE(random2PoolLock); + LockGuard guard(random2PoolLock); copyMem(poolVec, externalPoolVec, score_engine::POOL_VEC_PADDING_SIZE); - RELEASE(random2PoolLock); } // Load the task blocks into every compute buffer; returns false if any leaf rejects them. @@ -141,12 +155,11 @@ struct ScoreFunction void saveScoreCache(int epoch, CHAR16* directory = NULL) { #if USE_SCORE_CACHE - ACQUIRE(scoreCacheLock); + LockGuard guard(scoreCacheLock); SCORE_CACHE_FILE_NAME[sizeof(SCORE_CACHE_FILE_NAME) / sizeof(SCORE_CACHE_FILE_NAME[0]) - 4] = epoch / 100 + L'0'; SCORE_CACHE_FILE_NAME[sizeof(SCORE_CACHE_FILE_NAME) / sizeof(SCORE_CACHE_FILE_NAME[0]) - 3] = (epoch % 100) / 10 + L'0'; SCORE_CACHE_FILE_NAME[sizeof(SCORE_CACHE_FILE_NAME) / sizeof(SCORE_CACHE_FILE_NAME[0]) - 2] = epoch % 10 + L'0'; scoreCache.save(SCORE_CACHE_FILE_NAME, directory); - RELEASE(scoreCacheLock); #endif } @@ -155,12 +168,13 @@ struct ScoreFunction { bool success = true; #if USE_SCORE_CACHE - ACQUIRE(scoreCacheLock); - SCORE_CACHE_FILE_NAME[sizeof(SCORE_CACHE_FILE_NAME) / sizeof(SCORE_CACHE_FILE_NAME[0]) - 4] = epoch / 100 + L'0'; - SCORE_CACHE_FILE_NAME[sizeof(SCORE_CACHE_FILE_NAME) / sizeof(SCORE_CACHE_FILE_NAME[0]) - 3] = (epoch % 100) / 10 + L'0'; - SCORE_CACHE_FILE_NAME[sizeof(SCORE_CACHE_FILE_NAME) / sizeof(SCORE_CACHE_FILE_NAME[0]) - 2] = epoch % 10 + L'0'; - success = scoreCache.load(SCORE_CACHE_FILE_NAME); - RELEASE(scoreCacheLock); + { + LockGuard guard(scoreCacheLock); + SCORE_CACHE_FILE_NAME[sizeof(SCORE_CACHE_FILE_NAME) / sizeof(SCORE_CACHE_FILE_NAME[0]) - 4] = epoch / 100 + L'0'; + SCORE_CACHE_FILE_NAME[sizeof(SCORE_CACHE_FILE_NAME) / sizeof(SCORE_CACHE_FILE_NAME[0]) - 3] = (epoch % 100) / 10 + L'0'; + SCORE_CACHE_FILE_NAME[sizeof(SCORE_CACHE_FILE_NAME) / sizeof(SCORE_CACHE_FILE_NAME[0]) - 2] = epoch % 10 + L'0'; + success = scoreCache.load(SCORE_CACHE_FILE_NAME); + } #endif return success; } @@ -188,22 +202,63 @@ struct ScoreFunction m256i getLastOutput(const unsigned long long processor_Number) { - ACQUIRE(solutionEngineLock[processor_Number]); + LockGuard guard(solutionEngineLock[processor_Number]); + return _computeBuffer[processor_Number].getLastOutput(); + } - m256i result = _computeBuffer[processor_Number].getLastOutput(); + // Ant colony main score function + // score a child by inheriting its parent's network and walking it with the child's own seeds. + // parentAnn == nullptr means the parent is the submitter's root, which is derived here from the pubkey + // Returns INVALID_SCORE_VALUE for a non-canonical nonce, in which case outChildAnn is not written + // bestANN would still hold the previous call's network, and committing that would put one node's + // stale bytes into childAnnHash. + unsigned int computeAntChildScore( + const unsigned long long processor_Number, + const score_engine::ScoreBpp9000T::ANN* parentAnn, + const m256i& publicKey, + const m256i& nonce, + const m256i& anchorDigest, + score_engine::ScoreBpp9000T::ANN& outChildAnn) + { + const int solutionBufIdx = (int)(processor_Number % solutionBufferCount); + LockGuard guard(solutionEngineLock[solutionBufIdx]); + score_engine::ScoreBpp9000T& engine = _computeBuffer[solutionBufIdx]._bpp9000Score; + + // Derived into this slot's scratch rather than the engine's own buffer: deriveRootANN() uses + // currentANN as working space + const score_engine::ScoreBpp9000T::ANN* parent = parentAnn; + // Depth 1, the start node of every public key + if (parent == nullptr) + { + engine.deriveRootANN(publicKey.m256i_u8, poolVec, _antRootScratch[solutionBufIdx]); + parent = &_antRootScratch[solutionBufIdx]; + } - RELEASE(solutionEngineLock[processor_Number]); - return result; + const unsigned int childScore = engine.computeScoreFromParent( + *parent, publicKey.m256i_u8, nonce.m256i_u8, anchorDigest.m256i_u8, poolVec); + if (childScore == score_engine::INVALID_SCORE_VALUE) + { + return childScore; + } + engine.getBestANN(outChildAnn); + return childScore; } // main score function unsigned int operator()(const unsigned long long processor_Number, const m256i& publicKey, const m256i& miningSeed, const m256i& nonce) { PROFILE_SCOPE(); - // TODO: When neuraxon's going, this check need to be modified - if (!score_engine::isCanonicalBpp9000Nonce(nonce.m256i_u8)) + switch (score_engine::getAlgoType(nonce.m256i_u8)) { - return score_engine::INVALID_SCORE_VALUE; + case score_engine::AlgoType::Bpp9000: + if (!score_engine::isCanonicalBpp9000Nonce(nonce.m256i_u8)) + { + return score_engine::INVALID_SCORE_VALUE; + } + break; + default: + // Unsupported algo + return score_engine::INVALID_SCORE_VALUE; } if (isZero(miningSeed) || miningSeed != currentRandomSeed) @@ -223,11 +278,11 @@ struct ScoreFunction #endif const int solutionBufIdx = (int)(processor_Number % solutionBufferCount); - ACQUIRE(solutionEngineLock[solutionBufIdx]); - - score = computeScore(solutionBufIdx, publicKey, nonce); - - RELEASE(solutionEngineLock[solutionBufIdx]); + { + // Scoped so the cache write below happens with the engine slot released. + LockGuard guard(solutionEngineLock[solutionBufIdx]); + score = computeScore(solutionBufIdx, publicKey, nonce); + } #if USE_SCORE_CACHE scoreCache.addEntry(publicKey, miningSeed, nonce, scoreCacheIndex, score); #endif @@ -242,107 +297,151 @@ struct ScoreFunction unsigned long long stackSize = 0; #endif - // Multithreaded solutions verification: - // This module mainly serve tick processor in qubic core node, thus the queue size is limited at NUMBER_OF_TRANSACTIONS_PER_TICK - // for future use for somewhere else, you can only increase the size. + // Multithreaded solutions verification. + // + // A task is a (work function, payload) pair rather than a fixed tuple, so different kinds of + // scoring work can share one queue and one drain: the queue arbitrates nothing except who runs + // next. The payload is COPIED in, so the caller may reuse or discard its buffer immediately - a + // pointer here would make every caller responsible for keeping data alive across a drain. + // + // The work function is responsible for taking whatever locks it needs, including + // solutionEngineLock. The queue must not take it: solutionEngineLock is a non-reentrant spinlock + // and operator() takes it itself, so a queue that pre-acquired would deadlock any work function + // that reuses operator(). + typedef void (*WorkFunc)(unsigned long long processorNumber, void* payload); + + static constexpr unsigned int TASK_PAYLOAD_MAX = 128; + +private: + static constexpr unsigned int TASK_QUEUE_CAPACITY = NUMBER_OF_TRANSACTIONS_PER_TICK; + + struct Task + { + WorkFunc func; + // 8-byte aligned: m256i is a plain union accessed with unaligned intrinsics, so it needs no more. + unsigned long long payload[TASK_PAYLOAD_MAX / sizeof(unsigned long long)]; + }; volatile char taskQueueLock = 0; - struct - { - m256i publicKey[NUMBER_OF_TRANSACTIONS_PER_TICK]; - m256i miningSeed[NUMBER_OF_TRANSACTIONS_PER_TICK]; - m256i nonce[NUMBER_OF_TRANSACTIONS_PER_TICK]; - } taskQueue; + Task taskQueue[TASK_QUEUE_CAPACITY]; unsigned int _nTask; unsigned int _nProcessing; unsigned int _nFinished; - bool _nIsTaskQueueReady; + volatile bool _nIsTaskQueueReady; +public: + // Queued tasks not finished yet; diagnostics only. + unsigned int pendingTaskCount() const + { + return _nTask - _nFinished; + } + +public: void resetTaskQueue() { - ACQUIRE(taskQueueLock); + LockGuard guard(taskQueueLock); _nTask = 0; _nProcessing = 0; _nFinished = 0; _nIsTaskQueueReady = false; - RELEASE(taskQueueLock); } - // add task to the queue - // queue size is limited at NUMBER_OF_TRANSACTIONS_PER_TICK - void addTask(m256i publicKey, m256i miningSeed, m256i nonce) + // Copies size bytes of data. Returns false if the queue is full or the payload does not fit. + bool addTask(WorkFunc func, const void* data, unsigned int size) { - ACQUIRE(taskQueueLock); - if (_nTask < NUMBER_OF_TRANSACTIONS_PER_TICK) + if (size > TASK_PAYLOAD_MAX) { - unsigned int index = _nTask++; - taskQueue.publicKey[index] = publicKey; - taskQueue.miningSeed[index] = miningSeed; - taskQueue.nonce[index] = nonce; + return false; } - RELEASE(taskQueueLock); - } - void startProcessTaskQueue() - { - ACQUIRE(taskQueueLock); - _nIsTaskQueueReady = true; - RELEASE(taskQueueLock); + LockGuard guard(taskQueueLock); + if (_nTask >= TASK_QUEUE_CAPACITY) + { + return false; + } + Task& t = taskQueue[_nTask++]; + t.func = func; + copyMem(t.payload, data, size); + return true; } - void stopProcessTaskQueue() + // Outcome of one dispatch attempt, so a caller waiting for the batch does not need a second + // lock acquisition just to ask whether it is over. + enum TaskDispatchResult { - ACQUIRE(taskQueueLock); - _nIsTaskQueueReady = false; - RELEASE(taskQueueLock); - } + TaskRan, // a task was taken and executed + TaskNonePending, // nothing left to take, but tasks are still running elsewhere + TaskAllDone // every queued task has finished + }; - // get a task, can call on any thread - bool getTask(m256i* publicKey, m256i* miningSeed, m256i* nonce) + // Run one task if any is pending. Called from request processors' idle path and from the drain. + TaskDispatchResult tryProcessOneTask(unsigned long long processorNumber) { if (!_nIsTaskQueueReady) { - return false; + // No thing to process + return TaskNonePending; } - bool result = false; - ACQUIRE(taskQueueLock); - if (_nProcessing < _nTask) + + WorkFunc func = nullptr; + unsigned long long payload[TASK_PAYLOAD_MAX / sizeof(unsigned long long)]; + TaskDispatchResult result = TaskNonePending; + + // The task itself must run with the lock released { - unsigned int index = _nProcessing++; - *publicKey = taskQueue.publicKey[index]; - *miningSeed = taskQueue.miningSeed[index]; - *nonce = taskQueue.nonce[index]; - result = true; + LockGuard guard(taskQueueLock); + if (_nFinished >= _nTask) + { + result = TaskAllDone; + } + else if (_nIsTaskQueueReady && _nProcessing < _nTask) + { + const Task& t = taskQueue[_nProcessing++]; + func = t.func; + copyMem(payload, t.payload, TASK_PAYLOAD_MAX); + result = TaskRan; + } } - else + + if (func == nullptr) + { + return result; + } + func(processorNumber, payload); + { - result = false; + LockGuard guard(taskQueueLock); + _nFinished++; } - RELEASE(taskQueueLock); return result; } - void finishTask() - { - ACQUIRE(taskQueueLock); - _nFinished++; - RELEASE(taskQueueLock); - } - bool isTaskQueueProcessed() + // Open the queue and work it down. The caller participates rather than spinning idle, and returns + // only once every task has finished - including those running on other threads + void runUntilDone(unsigned long long processorNumber) { - return _nFinished == _nTask; - } + { + LockGuard guard(taskQueueLock); + _nIsTaskQueueReady = true; + } + + // Wait for task queue finish + for (;;) + { + const TaskDispatchResult result = tryProcessOneTask(processorNumber); + if (result == TaskAllDone) + { + break; + } + if (result == TaskNonePending) + { + _mm_pause(); + } + } - void tryProcessSolution(unsigned long long processorNumber) - { - m256i publicKey; - m256i miningSeed; - m256i nonce; - bool res = this->getTask(&publicKey, &miningSeed, &nonce); - if (res) { - (*this)(processorNumber, publicKey, miningSeed, nonce); - this->finishTask(); + LockGuard guard(taskQueueLock); + _nIsTaskQueueReady = false; } } }; diff --git a/src/ticking/tick_storage.h b/src/ticking/tick_storage.h index 06822369..35390005 100644 --- a/src/ticking/tick_storage.h +++ b/src/ticking/tick_storage.h @@ -73,7 +73,6 @@ class TickStorage static constexpr unsigned long long tickTransactionOffsetsSizeCurrentEpoch = tickTransactionOffsetsLengthCurrentEpoch * sizeof(unsigned long long); static constexpr unsigned long long tickTransactionOffsetsSizePreviousEpoch = tickTransactionOffsetsLengthPreviousEpoch * sizeof(unsigned long long); static constexpr unsigned long long tickTransactionOffsetsSize = tickTransactionOffsetsLength * sizeof(unsigned long long); - static constexpr unsigned long long oldTickTransactionsPadding = 4096 * 2; // Tick number range of current epoch storage @@ -828,7 +827,7 @@ class TickStorage // if we don't use swap, these memory will be commited on the fly while core is running (below is just reserve space, not physical memory allocation) if (!allocPoolWithErrorLog(L"tickDataPtr ", tickDataSize, (void**)&tickDataPtr, __LINE__, true, false) || !allocPoolWithErrorLog(L"tickPtr", ticksSize, (void**)&ticksPtr, __LINE__, true, false) - || !allocPoolWithErrorLog(L"tickTransactionPtr", tickTransactionsSize + oldTickTransactionsPadding, (void**)&tickTransactionsPtr, __LINE__, true, false)) + || !allocPoolWithErrorLog(L"tickTransactionPtr", tickTransactionsSize, (void**)&tickTransactionsPtr, __LINE__, true, false)) { return false; } @@ -848,7 +847,7 @@ class TickStorage oldTickDataPtr = tickDataPtr + MAX_NUMBER_OF_TICKS_PER_EPOCH; oldTicksPtr = ticksPtr + ticksLengthCurrentEpoch; - oldTickTransactionsPtr = tickTransactionsPtr + tickTransactionsSizeCurrentEpoch + oldTickTransactionsPadding; + oldTickTransactionsPtr = tickTransactionsPtr + tickTransactionsSizeCurrentEpoch; oldTickTransactionOffsetsPtr = tickTransactionOffsetsPtr + tickTransactionOffsetsLengthCurrentEpoch; tickBegin = 0; @@ -940,7 +939,7 @@ class TickStorage qVirtualCommit(oldTickTransactionsPtr, tickTransactionsSizePreviousEpoch); unsigned long long currentOldTickTransactionsOffet = 0; - const unsigned long long offsetDelta = (tickTransactionsSizeCurrentEpoch + keepTransactionSizesSum) - nextTickTransactionOffset + oldTickTransactionsPadding; + const unsigned long long offsetDelta = (tickTransactionsSizeCurrentEpoch + keepTransactionSizesSum) - nextTickTransactionOffset; for (unsigned int tickId = oldTickBegin; tickId < oldTickEnd; ++tickId) { @@ -1000,7 +999,7 @@ class TickStorage copyMem(oldTickTransactionsPtr, tickTransactionsPtr + firstToKeepOffset, keepTransactionSizesSum); // adjust offsets (based on end of transactions) - const unsigned long long offsetDelta = (tickTransactionsSizeCurrentEpoch + keepTransactionSizesSum) - nextTickTransactionOffset + oldTickTransactionsPadding; + const unsigned long long offsetDelta = (tickTransactionsSizeCurrentEpoch + keepTransactionSizesSum) - nextTickTransactionOffset; for (unsigned int tickId = oldTickBegin; tickId < oldTickEnd; ++tickId) { PinScope _pinScope; // bound swap-page pins during epoch-transition copy @@ -1128,13 +1127,11 @@ class TickStorage ASSERT(tickTransactionOffsetsPtr != nullptr); ASSERT(oldTickDataPtr == tickDataPtr + MAX_NUMBER_OF_TICKS_PER_EPOCH); ASSERT(oldTicksPtr == ticksPtr + ticksLengthCurrentEpoch); - ASSERT(oldTickTransactionsPtr == tickTransactionsPtr + tickTransactionsSizeCurrentEpoch + oldTickTransactionsPadding); + ASSERT(oldTickTransactionsPtr == tickTransactionsPtr + tickTransactionsSizeCurrentEpoch); ASSERT(oldTickTransactionOffsetsPtr == tickTransactionOffsetsPtr + tickTransactionOffsetsLengthCurrentEpoch); ASSERT(nextTickTransactionOffset >= FIRST_TICK_TRANSACTION_OFFSET); ASSERT(nextTickTransactionOffset <= tickTransactionsSizeCurrentEpoch); - const unsigned long long* tickOffsets = TickTransactionOffsetsAccess::getByTickInPreviousEpoch(oldTickBegin+2); - unsigned long long offset = tickOffsets[0]; // Check previous epoch data for (unsigned int tickId = oldTickBegin; tickId < oldTickEnd; ++tickId) { diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt index 3b736f9e..5fd3c8dd 100644 --- a/test/CMakeLists.txt +++ b/test/CMakeLists.txt @@ -29,6 +29,7 @@ include_directories(${CMAKE_CURRENT_SOURCE_DIR}/../src) include_directories(${PROJECT_ROOT_DIR}) file(COPY data/example_task_bpp9000.bin data/samples_bpp9000.csv data/scores_bpp9000.csv + data/bpp9000.task data/gt_production.csv data/gt_ant_production.csv DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/data) add_executable( @@ -54,6 +55,9 @@ add_executable( contract_qrwa.cpp contract_qip.cpp contract_qusino.cpp + ant_colony.cpp + ant_pending_solutions.cpp + trit_pack.cpp custom_qubic_mining_storage.cpp fast_tx_window.cpp file_io.cpp diff --git a/test/ant_colony.cpp b/test/ant_colony.cpp new file mode 100644 index 00000000..6a93959e --- /dev/null +++ b/test/ant_colony.cpp @@ -0,0 +1,1148 @@ +#define NO_UEFI + +#include "gtest/gtest.h" + +#define ENABLE_PROFILING 0 + +// The bound colony, not the bare template: these tests check bpp9000's binding as well as the rules. +#include "../src/mining/ant_colony/ant_colony_bpp9000.h" + +#include + +static constexpr unsigned int TEST_THRESHOLD = 3838; // BPP9000_SOLUTION_THRESHOLD_DEFAULT +// Ticks are absolute. A commit at TEST_PUBLISH_TICK lands in tick-index slot (TEST_PUBLISH_TICK - +// TEST_INITIAL_TICK), which must stay under MAX_NUMBER_OF_TICKS_PER_EPOCH (3005 on the testnet setting). +static constexpr unsigned int TEST_INITIAL_TICK = 99000; +static constexpr unsigned int TEST_PUBLISH_TICK = 100000; + +static m256i makeKey(unsigned long long n) +{ + m256i k = m256i::zero(); + k.m256i_u64[0] = n + 1; + return k; +} + +// A parent sitting at the given score, owned by the given identity. +static AntSolutionRecord makeParent(const m256i& owner, unsigned int score, unsigned int depth = 1) +{ + AntSolutionRecord r; + setMem(&r, sizeof(r), 0); + r.pubkey = owner; + r.score = score; + r.depth = depth; + r.parentRef = ROOT_REF; + r.nextSiblingIdx = NO_SIBLING; + return r; +} + +// A candidate from `owner` at `score`, anchored and published in the same tick unless the test is +// about freshness. +static ChildCandidate makeChild(const m256i& owner, unsigned int score, + unsigned int anchorTick = 1000, unsigned int publishTick = 1000) +{ + ChildCandidate c; + c.pubkey = owner; + c.score = score; + c.anchorTick = anchorTick; + c.publishTick = publishTick; + return c; +} + +// Every test runs at TEST_THRESHOLD, so wrapping it keeps the assertions on one line. +static ValidityResult admit(const ChildCandidate& child, const AntSolutionRecord* parent, + unsigned int childCount) +{ + return AntColonyBpp9000T::validateChild(child, parent, childCount, TEST_THRESHOLD); +} + +// Same wrapper for a caller that took the score on trust instead of computing it. +static ValidityResult admitTrusted(const ChildCandidate& child, const AntSolutionRecord* parent, + unsigned int childCount) +{ + return AntColonyBpp9000T::validateChild(child, parent, childCount, TEST_THRESHOLD, true); +} + +// The packing itself is generic and tested exhaustively +TEST(TestAntColonyPackedAnn, CoversAWholeAnnAtTheUnpaddedStride) +{ + AntColonyBpp9000T::Ann src; + for (unsigned long long i = 0; i < sizeof(src); i++) + { + src.lut[i] = (unsigned char)(i % 3); // mutate() only ever writes 0, 1 or 2 + } + + AntColonyBpp9000T::PackedAnn packed; + packed.pack(src.lut); + + AntColonyBpp9000T::Ann back; + setMem(&back, sizeof(back), 0xFF); + packed.unpack(back.lut); + + for (unsigned long long i = 0; i < sizeof(src); i++) + { + ASSERT_EQ(back.lut[i], src.lut[i]) << "entry " << i; + } +} + +// The threshold is checked before the parent comparison, so nodes worse than it are never stored +TEST(TestAntColonyValidate, ThresholdIsAnUpperBoundOnError) +{ + const m256i me = makeKey(1); + + // A parent that would otherwise admit anything, so only the threshold can reject. + const AntSolutionRecord looseParent = makeParent(me, WORST_SCORE); + EXPECT_EQ(admit(makeChild(me, 3839), &looseParent, 0), + ValidityResult::RejectBelowThreshold); + + // Exactly at the bound is accepted: the rule is score > threshold, not >=. + EXPECT_EQ(admit(makeChild(me, TEST_THRESHOLD), &looseParent, 0), ValidityResult::Valid); +} + +TEST(TestAntColonyValidate, MustStrictlyBeatParent) +{ + const m256i me = makeKey(2); + const AntSolutionRecord parent = makeParent(me, 3800); + + EXPECT_EQ(admit(makeChild(me, 3799), &parent, 0), ValidityResult::Valid); + EXPECT_EQ(admit(makeChild(me, 3800), &parent, 0), ValidityResult::RejectLeParent); + EXPECT_EQ(admit(makeChild(me, 3801), &parent, 0), ValidityResult::RejectLeParent); +} + +// Neither rule that judges the score may run on a trusted one: rejecting leaves no refund for the +// quorum to disagree with, so the tree would diverge with nothing to detect it. +TEST(TestAntColonyValidate, TrustedScoreSkipsBothScoreRules) +{ + const m256i me = makeKey(20); + const AntSolutionRecord parent = makeParent(me, 3800); + + EXPECT_EQ(admit(makeChild(me, 3900), &parent, 0), ValidityResult::RejectBelowThreshold); + EXPECT_EQ(admitTrusted(makeChild(me, 3900), &parent, 0), ValidityResult::Valid); + + EXPECT_EQ(admit(makeChild(me, 3800), &parent, 0), ValidityResult::RejectLeParent); + EXPECT_EQ(admitTrusted(makeChild(me, 3800), &parent, 0), ValidityResult::Valid); + + // Even a score the scorer would never return at all. + EXPECT_EQ(admitTrusted(makeChild(me, WORST_SCORE), &parent, 0), ValidityResult::Valid); +} + +// Metadata rules still apply: every node reads those the same way, so over-accepting them would +// manufacture a disagreement with nothing behind it. +TEST(TestAntColonyValidate, TrustedScoreStillHonoursTheMetadataRules) +{ + const m256i me = makeKey(21); + const m256i someoneElse = makeKey(22); + + const AntSolutionRecord theirNode = makeParent(someoneElse, 3800); + EXPECT_EQ(admitTrusted(makeChild(me, 3900), &theirNode, 0), ValidityResult::RejectWrongTree); + + const AntSolutionRecord myNode = makeParent(me, 3800); + EXPECT_EQ(admitTrusted(makeChild(me, 3900, 1000, 999), &myNode, 0), ValidityResult::RejectStale); + + const unsigned int cap = ANT_MAX_CHILDREN_PER_PARENT; + if (cap != 0) + { + EXPECT_EQ(admitTrusted(makeChild(me, 3900), &myNode, cap), + ValidityResult::RejectMaxChildrenPerParent); + } +} + +// A root has no score of its own, so any threshold-passing child improves on it. This is what lets a +// lineage start at all. +TEST(TestAntColonyValidate, RootParentAdmitsAnyPassingScore) +{ + const m256i me = makeKey(3); + + EXPECT_EQ(admit(makeChild(me, TEST_THRESHOLD), nullptr, 0), ValidityResult::Valid); + EXPECT_EQ(admit(makeChild(me, 0), nullptr, 0), ValidityResult::Valid); + EXPECT_EQ(admit(makeChild(me, TEST_THRESHOLD + 1), nullptr, 0), + ValidityResult::RejectBelowThreshold); +} + +// Trees are isolated per identity: a miner cannot branch off someone else's node. +TEST(TestAntColonyValidate, CannotBranchFromAnotherIdentity) +{ + const m256i me = makeKey(4); + const m256i someoneElse = makeKey(5); + const AntSolutionRecord theirNode = makeParent(someoneElse, 3800); + + EXPECT_EQ(admit(makeChild(me, 3700), &theirNode, 0), ValidityResult::RejectWrongTree); + + const AntSolutionRecord myNode = makeParent(me, 3800); + EXPECT_EQ(admit(makeChild(me, 3700), &myNode, 0), ValidityResult::Valid); +} + +// Per-parent child cap. The cap is compile-time; 0 means unbound. +TEST(TestAntColonyValidate, RejectsAtTheChildCap) +{ + const m256i me = makeKey(6); + const AntSolutionRecord parent = makeParent(me, WORST_SCORE); + + // Below the cap - and always, when unbound - a passing child is admitted. + EXPECT_EQ(admit(makeChild(me, 3799), &parent, 0), ValidityResult::Valid); + + // At the cap it is refused. Skipped when unbound (0). The runtime copy keeps the compile-time + // zero from tripping a constant-condition warning. + const unsigned int cap = ANT_MAX_CHILDREN_PER_PARENT; + if (cap != 0) + { + EXPECT_EQ(admit(makeChild(me, 3799), &parent, cap), + ValidityResult::RejectMaxChildrenPerParent); + } +} + +// Freshness +TEST(TestAntColonyValidate, FreshnessWindowBoundaries) +{ + const m256i me = makeKey(7); + const AntSolutionRecord parent = makeParent(me, WORST_SCORE); + const unsigned int anchor = 100000; + + // Published in the same tick it anchored to: the tightest legal case. + EXPECT_EQ(admit(makeChild(me, 3700, anchor, anchor), &parent, 0), + ValidityResult::Valid); + + // Exactly at the window edge is still legal; one past it is not. + EXPECT_EQ(admit(makeChild(me, 3700, anchor, anchor + ANT_PUBLISH_WINDOW_TICKS), + &parent, 0), ValidityResult::Valid); + EXPECT_EQ(admit(makeChild(me, 3700, anchor, anchor + ANT_PUBLISH_WINDOW_TICKS + 1), + &parent, 0), ValidityResult::RejectStale); + + // An anchor in the future is rejected rather than wrapping the unsigned subtraction. + EXPECT_EQ(admit(makeChild(me, 3700, anchor + 1, anchor), &parent, 0), + ValidityResult::RejectStale); +} + +// Order of checks: Freshness first, then tree isolation, then threshold, then +// parent, then the child cap. +TEST(TestAntColonyValidate, ReportsTheFirstFailingRule) +{ + const m256i me = makeKey(8); + const m256i other = makeKey(9); + const unsigned int anchor = 100000; + const unsigned int stalePublish = anchor + ANT_PUBLISH_WINDOW_TICKS + 1; + + const AntSolutionRecord theirs = makeParent(other, 3000); + + // Stale AND wrong tree AND above threshold AND worse than parent -> reports Stale. + EXPECT_EQ(admit(makeChild(me, 9999, anchor, stalePublish), &theirs, 0), + ValidityResult::RejectStale); + + // Fresh, but wrong tree AND above threshold -> reports WrongTree. + EXPECT_EQ(admit(makeChild(me, 9999, anchor, anchor), &theirs, 0), + ValidityResult::RejectWrongTree); + + // Own tree, above threshold AND worse than parent -> reports the threshold. + const AntSolutionRecord mine = makeParent(me, 3000); + EXPECT_EQ(admit(makeChild(me, 9999, anchor, anchor), &mine, 0), + ValidityResult::RejectBelowThreshold); + + // Passes the threshold but worse than parent -> reports the parent. + EXPECT_EQ(admit(makeChild(me, 3500, anchor, anchor), &mine, 0), + ValidityResult::RejectLeParent); +} + +// For a fixed parent and threshold, acceptance +// must be monotone in the score - every score at or below the tightest bound is accepted, every +// score above it is rejected. An inverted comparison anywhere breaks this even if the individual +// boundary tests above were adjusted to match it. +TEST(TestAntColonyValidate, AcceptanceIsMonotoneInScore) +{ + const m256i me = makeKey(10); + const unsigned int parentScore = 3800; + const AntSolutionRecord parent = makeParent(me, parentScore); + + // Tightest of: <= threshold, < parent. + const unsigned int bestRejected = parentScore; + + bool sawAccept = false; + for (unsigned int score = 3700; score <= 3900; score++) + { + const ValidityResult r = admit(makeChild(me, score), &parent, 0); + const bool accepted = (r == ValidityResult::Valid); + if (score < bestRejected && score <= TEST_THRESHOLD) + { + ASSERT_TRUE(accepted) << "score " << score << " should be accepted, got " << (int)r; + sawAccept = true; + } + else + { + ASSERT_FALSE(accepted) << "score " << score << " should be rejected"; + } + } + EXPECT_TRUE(sawAccept) << "the sweep must cover the accepting region"; +} + +// init() allocates ~6.2 GB, so the colony is built once for the file and re-seeded between tests. +// Lazy rather than SetUpTestSuite: that does not exist before gtest 1.10, and test.vcxproj builds +// against 1.8.1, where it would compile clean and never run. +static AntColonyBpp9000T* freshColony() +{ + static AntColonyBpp9000T colony; + static bool allocated = false; + static bool allocationFailed = false; + + if (!allocated && !allocationFailed) + { + allocationFailed = !colony.init(); + allocated = !allocationFailed; + } + if (allocationFailed) + { + return nullptr; + } + + colony.beginEpoch(makeKey(999), TEST_INITIAL_TICK); + colony.setErrorThreshold(TEST_THRESHOLD); + return &colony; +} + +// Commits one child of the root, returns its index or ANT_INVALID_INDEX. nonceSeed keeps calls +// distinct, since (pubkey, nonce, parentRef) is the replay key. +static long long commitRootChild(AntColonyBpp9000T* colony, const m256i& owner, unsigned int score, + unsigned int txIdx, unsigned long long nonceSeed, unsigned int tick = 100000) +{ + AntCommitInput in; + in.pubkey = owner; + in.nonce = makeKey(nonceSeed); + in.parentRef = ROOT_REF; + in.selfRef.tick = tick; + in.selfRef.solutionIndexInTick = txIdx; + in.anchorTick = tick; + in.publishTick = tick; + + AntColonyBpp9000T::Ann ann; + setMem(&ann, sizeof(ann), 0); + ann.lut[0] = (unsigned char)(score % 3); + + // The real hash, not a stand-in: the snapshot rebuild re-derives it from the stored network. + unsigned int annHash; + KangarooTwelve(&ann, sizeof(ann), &annHash, sizeof(annHash)); + + const long long landsAt = (long long)colony->solutionCount(); + if (colony->commit(in, nullptr, score, &ann, annHash) != ValidityResult::Valid) + { + return ANT_INVALID_INDEX; + } + return landsAt; +} + +// A child of an existing node, so a test can build a lineage rather than a flat set of root children. +static long long commitChild(AntColonyBpp9000T* colony, const m256i& owner, const SolutionRef& parentRef, + unsigned int score, unsigned int txIdx, unsigned long long nonceSeed, unsigned int tick = 100000) +{ + const AntSolutionRecord* parentRec = nullptr; + if (colony->tryGetParent(parentRef, &parentRec) != ValidityResult::Valid) + { + return ANT_INVALID_INDEX; + } + + AntCommitInput in; + in.pubkey = owner; + in.nonce = makeKey(nonceSeed); + in.parentRef = parentRef; + in.selfRef.tick = tick; + in.selfRef.solutionIndexInTick = txIdx; + in.anchorTick = tick; + in.publishTick = tick; + + AntColonyBpp9000T::Ann ann; + setMem(&ann, sizeof(ann), 0); + ann.lut[0] = (unsigned char)(score % 3); + unsigned int annHash; + KangarooTwelve(&ann, sizeof(ann), &annHash, sizeof(annHash)); + + const long long landsAt = (long long)colony->solutionCount(); + if (colony->commit(in, parentRec, score, &ann, annHash) != ValidityResult::Valid) + { + return ANT_INVALID_INDEX; + } + return landsAt; +} + +// commit() head-inserts, so children chain from newest to oldest. countChildren() walks this chain +// from the head, so it must stay intact and terminate. +// Commits one root child with no network, the way an AUX node commits a solution it took on trust. +static long long commitRootChildWithoutAnn(AntColonyBpp9000T* colony, const m256i& owner, + unsigned int score, unsigned int txIdx, unsigned long long nonceSeed, unsigned int tick = 100000) +{ + AntCommitInput in; + in.pubkey = owner; + in.nonce = makeKey(nonceSeed); + in.parentRef = ROOT_REF; + in.selfRef.tick = tick; + in.selfRef.solutionIndexInTick = txIdx; + in.anchorTick = tick; + in.publishTick = tick; + + const long long landsAt = (long long)colony->solutionCount(); + if (colony->commit(in, nullptr, score, nullptr, 0, true) != ValidityResult::Valid) + { + return ANT_INVALID_INDEX; + } + return landsAt; +} + +// The record is addressable but has no network yet, so every reader must see that rather than pool garbage. +TEST(TestAntColonyStore, RecordCommitsWithoutItsNetwork) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB"; + + const m256i me = makeKey(30); + const long long idx = commitRootChildWithoutAnn(colony, me, 3800, 0, 700); + ASSERT_NE(idx, ANT_INVALID_INDEX); + + EXPECT_FALSE(colony->isAnnMaterialised((unsigned int)idx)); + EXPECT_EQ(colony->recordAt(idx)->annStateSlot, ANT_ANN_UNMATERIALISED); + EXPECT_EQ(colony->recordAt(idx)->score, 3800u); + + AntColonyBpp9000T::Ann out; + EXPECT_FALSE(colony->annOfNonRoot(*colony->recordAt(idx), out)); +} + +// One claim wins and the loser is told to wait; the published network reads back byte for byte. +TEST(TestAntColonyStore, ClaimThenPublishSuppliesTheNetwork) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB"; + + const m256i me = makeKey(31); + const long long idx = commitRootChildWithoutAnn(colony, me, 3800, 0, 701); + ASSERT_NE(idx, ANT_INVALID_INDEX); + const unsigned int slot = (unsigned int)idx; + + ASSERT_EQ(colony->tryClaimAnn(slot), AntColonyBpp9000T::AnnClaimOwned); + EXPECT_TRUE(colony->isAnnClaimHeld(slot)); + EXPECT_EQ(colony->tryClaimAnn(slot), AntColonyBpp9000T::AnnClaimBusy); + + // A claim that produces nothing must be releasable, or the slot is never rebuildable again. + colony->releaseAnnClaim(slot); + EXPECT_FALSE(colony->isAnnClaimHeld(slot)); + ASSERT_EQ(colony->tryClaimAnn(slot), AntColonyBpp9000T::AnnClaimOwned); + + AntColonyBpp9000T::Ann rebuilt; + setMem(&rebuilt, sizeof(rebuilt), 0); + rebuilt.lut[0] = 2; + rebuilt.lut[5] = 1; + unsigned int annHash; + KangarooTwelve(&rebuilt, sizeof(rebuilt), &annHash, sizeof(annHash)); + colony->publishAnn(slot, rebuilt, annHash); + + EXPECT_TRUE(colony->isAnnMaterialised(slot)); + EXPECT_EQ(colony->tryClaimAnn(slot), AntColonyBpp9000T::AnnClaimReady); + EXPECT_EQ(colony->recordAt(idx)->childAnnHash, annHash); + + AntColonyBpp9000T::Ann out; + ASSERT_TRUE(colony->annOfNonRoot(*colony->recordAt(idx), out)); + for (unsigned long long i = 0; i < sizeof(out); i++) + { + ASSERT_EQ(out.lut[i], rebuilt.lut[i]) << "entry " << i; + } +} + +TEST(TestAntColonyStore, SiblingsChainNewestFirst) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB"; + + const m256i me = makeKey(1); + // Same anchor tick, so all three are inside the freshness window and coexist. + const long long a = commitRootChild(colony, me, 3800, 0, 500); + const long long b = commitRootChild(colony, me, 3810, 1, 501); + const long long c = commitRootChild(colony, me, 3820, 2, 502); + ASSERT_NE(a, ANT_INVALID_INDEX); + ASSERT_NE(b, ANT_INVALID_INDEX); + ASSERT_NE(c, ANT_INVALID_INDEX); + + EXPECT_EQ(colony->recordAt(c)->nextSiblingIdx, (unsigned int)b); + EXPECT_EQ(colony->recordAt(b)->nextSiblingIdx, (unsigned int)a); + EXPECT_EQ(colony->recordAt(a)->nextSiblingIdx, NO_SIBLING); +} + +// parentRef is a logical address, so it must map back to the record index. +TEST(TestAntColonyStore, SolutionRefResolvesToItsRecord) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB"; + + const m256i me = makeKey(2); + const long long idx = commitRootChild(colony, me, 3800, 42, 600); + ASSERT_NE(idx, ANT_INVALID_INDEX); + + const SolutionRef ref = { TEST_PUBLISH_TICK,42 }; + EXPECT_EQ(colony->findIndexBySolutionRef(ref), idx); + + // An uncommitted ref must not resolve to a neighbour. + const SolutionRef missing = { TEST_PUBLISH_TICK,43 }; + EXPECT_EQ(colony->findIndexBySolutionRef(missing), ANT_INVALID_INDEX); +} + +// Same (pubkey, nonce, parentRef) is a replay whatever its score. +TEST(TestAntColonyStore, SameSolutionCannotCommitTwice) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB"; + + const m256i me = makeKey(3); + ASSERT_NE(commitRootChild(colony, me, 3800, 0, 700), ANT_INVALID_INDEX); + + EXPECT_EQ(commitRootChild(colony, me, 3700, 1, 700), ANT_INVALID_INDEX); + EXPECT_EQ(colony->stats().rejectReplay, 1u); + EXPECT_EQ(colony->solutionCount(), 1u); +} + +// ROOT is never stored, so resolving it is Valid with a null record, not a lookup failure. +TEST(TestAntColonyStore, RootRefResolvesToValidWithNoRecord) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB"; + + const AntSolutionRecord* parent = (const AntSolutionRecord*)1; // must be overwritten + EXPECT_EQ(colony->tryGetParent(ROOT_REF, &parent), ValidityResult::Valid); + EXPECT_EQ(parent, nullptr); + + const SolutionRef missing = { TEST_PUBLISH_TICK,0 }; + EXPECT_EQ(colony->tryGetParent(missing, &parent), ValidityResult::RejectParentNotRegistered); +} + +// Anchor ring +TEST(TestAntColonyStore, AnchorDigestRoundTrips) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB"; + + const m256i digest = makeKey(4242); + colony->recordAnchorDigest(100000, digest); + + m256i out = m256i::zero(); + EXPECT_TRUE(colony->getAnchorDigest(100000, out)); + EXPECT_TRUE(out == digest); + + // An unrecorded tick is a miss, not whatever sits in that slot. + EXPECT_FALSE(colony->getAnchorDigest(100001, out)); +} + +// A tick ANT_ANCHOR_RING_SIZE later lands in the same slot. The evicted one must miss - returning +// the new digest would score against a network the miner never used. +TEST(TestAntColonyStore, AgedOutAnchorIsAMissNotTheWrongDigest) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB"; + + const unsigned int oldTick = 100000; + const unsigned int newTick = oldTick + ANT_ANCHOR_RING_SIZE; + const m256i oldDigest = makeKey(11); + const m256i newDigest = makeKey(22); + + colony->recordAnchorDigest(oldTick, oldDigest); + colony->recordAnchorDigest(newTick, newDigest); + + m256i out = m256i::zero(); + EXPECT_FALSE(colony->getAnchorDigest(oldTick, out)); + EXPECT_TRUE(colony->getAnchorDigest(newTick, out)); + EXPECT_TRUE(out == newDigest); +} + +// beginEpoch() wipes the ring. It fills with ANT_ANCHOR_TICK_NONE, not zero, so tick 0 does not +// look recorded. +TEST(TestAntColonyStore, EpochResetClearsTheRing) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB"; + + colony->recordAnchorDigest(100000, makeKey(7)); + m256i out = m256i::zero(); + ASSERT_TRUE(colony->getAnchorDigest(100000, out)); + + colony->beginEpoch(makeKey(999), TEST_INITIAL_TICK); + EXPECT_FALSE(colony->getAnchorDigest(100000, out)); + EXPECT_FALSE(colony->getAnchorDigest(0, out)); +} + + +// Snapshot test cases + +static constexpr unsigned short TEST_EPOCH = 200; +static const m256i TEST_ROOT_SEED = makeKey(999); // what freshColony() seeds with + +// Save, wipe, load. beginEpoch() clears everything the load has to bring back, so anything that +// survives came out of the files. +static bool saveWipeLoad(AntColonyBpp9000T* colony) +{ + if (!colony->saveSnapshot(TEST_EPOCH, NULL, TEST_INITIAL_TICK)) + { + return false; + } + colony->beginEpoch(TEST_ROOT_SEED, TEST_INITIAL_TICK); + colony->setErrorThreshold(TEST_THRESHOLD); + return colony->loadSnapshot(TEST_EPOCH, NULL, TEST_ROOT_SEED, TEST_THRESHOLD, TEST_INITIAL_TICK); +} + +// Records and the tick index come back, and the sibling chain is rebuilt to the same shape commit() +// built. Only the records are on disk - nextSiblingIdx is replayed, so matching the pre-save chain +// is what proves the replay reproduces the head-insert. +TEST(TestAntColonySnapshot, RoundTripRestoresTheTree) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB"; + + const m256i me = makeKey(1); + ASSERT_NE(commitRootChild(colony, me, 3800, 0, 500), ANT_INVALID_INDEX); + ASSERT_NE(commitRootChild(colony, me, 3810, 1, 501), ANT_INVALID_INDEX); + ASSERT_NE(commitRootChild(colony, me, 3820, 2, 502), ANT_INVALID_INDEX); + + ASSERT_TRUE(saveWipeLoad(colony)); + + ASSERT_EQ(colony->solutionCount(), 3u); + EXPECT_EQ(colony->recordAt(2)->nextSiblingIdx, 1u); + EXPECT_EQ(colony->recordAt(1)->nextSiblingIdx, 0u); + EXPECT_EQ(colony->recordAt(0)->nextSiblingIdx, NO_SIBLING); + EXPECT_EQ(colony->recordAt(1)->score, 3810u); + EXPECT_TRUE(colony->recordAt(1)->pubkey == me); + + // The tick index is derived too, so resolving a logical ref proves it was rebuilt. + const SolutionRef ref = { TEST_PUBLISH_TICK,1 }; + EXPECT_EQ(colony->findIndexBySolutionRef(ref), 1LL); +} + +// The stored network must come back byte for byte, otherwise children score against a parent the +// rest of the network does not have. +TEST(TestAntColonySnapshot, RoundTripRestoresTheStoredNetwork) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB"; + + const m256i me = makeKey(2); + ASSERT_NE(commitRootChild(colony, me, 3800, 0, 900), ANT_INVALID_INDEX); + + AntColonyBpp9000T::Ann before; + ASSERT_TRUE(colony->annOfNonRoot(*colony->recordAt(0), before)); + + ASSERT_TRUE(saveWipeLoad(colony)); + + AntColonyBpp9000T::Ann after; + ASSERT_TRUE(colony->annOfNonRoot(*colony->recordAt(0), after)); + for (unsigned long long i = 0; i < sizeof(before); i++) + { + ASSERT_EQ(after.lut[i], before.lut[i]) << "entry " << i; + } +} + +// A record with no network must survive the round trip, since the loader re-derives childAnnHash from +// a network that is not there. A claim saved mid-rebuild must come back rebuildable, not stuck. +TEST(TestAntColonySnapshot, UnmaterialisedRecordsSurviveTheRoundTrip) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB"; + + const m256i me = makeKey(32); + ASSERT_NE(commitRootChild(colony, me, 3800, 0, 800), ANT_INVALID_INDEX); + ASSERT_NE(commitRootChildWithoutAnn(colony, me, 3810, 1, 801), ANT_INVALID_INDEX); + ASSERT_NE(commitRootChildWithoutAnn(colony, me, 3820, 2, 802), ANT_INVALID_INDEX); + + // The third one is saved mid-rebuild. + ASSERT_EQ(colony->tryClaimAnn(2), AntColonyBpp9000T::AnnClaimOwned); + + ASSERT_TRUE(saveWipeLoad(colony)); + + ASSERT_EQ(colony->solutionCount(), 3u); + EXPECT_TRUE(colony->isAnnMaterialised(0)); + EXPECT_FALSE(colony->isAnnMaterialised(1)); + + EXPECT_FALSE(colony->isAnnMaterialised(2)); + EXPECT_FALSE(colony->isAnnClaimHeld(2)); + EXPECT_EQ(colony->tryClaimAnn(2), AntColonyBpp9000T::AnnClaimOwned); + + // The tree itself is intact, so these records still resolve and still parent children. + EXPECT_EQ(colony->recordAt(1)->score, 3810u); + const SolutionRef ref = { TEST_PUBLISH_TICK, 2 }; + EXPECT_EQ(colony->findIndexBySolutionRef(ref), 2LL); +} + +// The dedup set is not written to disk. If the rebuild misses it, a restarted node re-accepts +// solutions it already has. +TEST(TestAntColonySnapshot, DedupIsRebuiltSoReplaysStillFail) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB"; + + const m256i me = makeKey(3); + ASSERT_NE(commitRootChild(colony, me, 3800, 0, 600), ANT_INVALID_INDEX); + ASSERT_TRUE(saveWipeLoad(colony)); + + // Same (pubkey, nonce, parentRef) as before the restart. + EXPECT_EQ(commitRootChild(colony, me, 3700, 1, 600), ANT_INVALID_INDEX); + EXPECT_EQ(colony->solutionCount(), 1u); +} + +// A cold ring would reject solutions anchored before the restart that peers accept. +TEST(TestAntColonySnapshot, AnchorRingSurvivesTheRoundTrip) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB"; + + const m256i digest = makeKey(4242); + colony->recordAnchorDigest(100000, digest); + ASSERT_TRUE(saveWipeLoad(colony)); + + m256i out = m256i::zero(); + EXPECT_TRUE(colony->getAnchorDigest(100000, out)); + EXPECT_TRUE(out == digest); + EXPECT_FALSE(colony->getAnchorDigest(100001, out)); +} + +// The seed and threshold are supplied by the node, not read from the file. A disagreement means the +// colony files and the node state are from different moments, so the tree is refused. +TEST(TestAntColonySnapshot, FileMustAgreeWithTheNodeState) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB"; + + ASSERT_NE(commitRootChild(colony, makeKey(4), 3800, 0, 700), ANT_INVALID_INDEX); + ASSERT_TRUE(colony->saveSnapshot(TEST_EPOCH, NULL, TEST_INITIAL_TICK)); + + EXPECT_FALSE(colony->loadSnapshot(TEST_EPOCH, NULL, makeKey(12345), TEST_THRESHOLD, TEST_INITIAL_TICK)); + EXPECT_FALSE(colony->loadSnapshot(TEST_EPOCH, NULL, TEST_ROOT_SEED, TEST_THRESHOLD + 1, TEST_INITIAL_TICK)); + + // A different base would resolve every parentRef to the wrong record. + EXPECT_FALSE(colony->loadSnapshot(TEST_EPOCH, NULL, TEST_ROOT_SEED, TEST_THRESHOLD, TEST_INITIAL_TICK + 1)); + + // The epoch is part of the file name, so a different one finds no snapshot at all. + EXPECT_FALSE(colony->loadSnapshot((unsigned short)(TEST_EPOCH + 1), NULL, TEST_ROOT_SEED, TEST_THRESHOLD, TEST_INITIAL_TICK)); + + // And the matching one still loads, so the refusals above were the checks and not a bad file. + EXPECT_TRUE(colony->loadSnapshot(TEST_EPOCH, NULL, TEST_ROOT_SEED, TEST_THRESHOLD, TEST_INITIAL_TICK)); +} + +// Those refusals all happen while only the meta has been read, so the tree the node is already +// running on must be left alone. +TEST(TestAntColonySnapshot, RefusedLoadLeavesTheRunningTreeIntact) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB"; + + const m256i me = makeKey(5); + ASSERT_NE(commitRootChild(colony, me, 3800, 0, 800), ANT_INVALID_INDEX); + ASSERT_TRUE(colony->saveSnapshot(TEST_EPOCH, NULL, TEST_INITIAL_TICK)); + ASSERT_NE(commitRootChild(colony, me, 3790, 1, 801), ANT_INVALID_INDEX); + ASSERT_EQ(colony->solutionCount(), 2u); + + EXPECT_FALSE(colony->loadSnapshot(TEST_EPOCH, NULL, makeKey(12345), TEST_THRESHOLD, TEST_INITIAL_TICK)); + EXPECT_EQ(colony->solutionCount(), 2u); +} + +// An empty colony still writes all three files, so an operator's backup is always the same set and a +// short one means a lost file rather than an empty epoch. +TEST(TestAntColonySnapshot, EmptyColonyWritesTheFullFileSet) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB"; + + ASSERT_EQ(colony->solutionCount(), 0u); + + // Clear whatever an earlier test left at this epoch, so the files below can only come from the + // save under test. + antSnapshotNameForEpoch(TEST_EPOCH); + removeFile(nullptr, ANT_SNAPSHOT_HEADER_FILENAME); + removeFile(nullptr, ANT_SNAPSHOT_RECORDS_FILENAME); + removeFile(nullptr, ANT_SNAPSHOT_POOL_FILENAME); + + ASSERT_TRUE(colony->saveSnapshot(TEST_EPOCH, NULL, TEST_INITIAL_TICK)); + + // All three are written. Records and pool hold a full slot even when empty, so neither is zero + // length; the header always carries the meta, anchor ring and export set. + AntColonySnapshotMeta metaSlot; + AntSolutionRecord recordSlot; + AntColonyBpp9000T::PackedAnn poolSlot; + EXPECT_EQ(load(ANT_SNAPSHOT_HEADER_FILENAME, sizeof(metaSlot), (unsigned char*)&metaSlot), + (long long)sizeof(metaSlot)); + EXPECT_EQ(load(ANT_SNAPSHOT_RECORDS_FILENAME, sizeof(recordSlot), (unsigned char*)&recordSlot), + (long long)sizeof(recordSlot)); + EXPECT_EQ(load(ANT_SNAPSHOT_POOL_FILENAME, sizeof(poolSlot), (unsigned char*)&poolSlot), + (long long)sizeof(poolSlot)); + + EXPECT_TRUE(colony->loadSnapshot(TEST_EPOCH, NULL, TEST_ROOT_SEED, TEST_THRESHOLD, TEST_INITIAL_TICK)); + EXPECT_EQ(colony->solutionCount(), 0u); +} + +// childAnnHash is the only thing tying a record to its stored network, so it is the only check on +// the pool file. Overwrite the pool behind the colony's back and the load must refuse. +TEST(TestAntColonySnapshot, CorruptedPoolIsRefused) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB"; + + // score % 3 == 2, so an all-zero network is not the one this record hashes to. + ASSERT_NE(commitRootChild(colony, makeKey(6), 3800, 0, 1000), ANT_INVALID_INDEX); + ASSERT_TRUE(colony->saveSnapshot(TEST_EPOCH, NULL, TEST_INITIAL_TICK)); + + AntColonyBpp9000T::PackedAnn junk; + setMem(&junk, sizeof(junk), 0); + ASSERT_EQ(save(ANT_SNAPSHOT_POOL_FILENAME, sizeof(junk), (unsigned char*)&junk), + (long long)sizeof(junk)); + + EXPECT_FALSE(colony->loadSnapshot(TEST_EPOCH, NULL, TEST_ROOT_SEED, TEST_THRESHOLD, TEST_INITIAL_TICK)); + + // The pool is read after the meta checks pass, so a refusal here does clear the colony. + EXPECT_EQ(colony->solutionCount(), 0u); +} + +// --------------------------------------------------------------------------------------------- +// Replay cache + +// A key whose four components are all distinct, so a slot function that ignores one still separates +// these. +static AntColonyBpp9000T::ReplayKey makeReplayKey(unsigned long long n) +{ + AntColonyBpp9000T::ReplayKey k; + k.pubkey = makeKey(n); + k.nonce = makeKey(n + 1000); + k.parentKey = makeKey(n + 2000); + k.anchorDigest = makeKey(n + 3000); + return k; +} + +// lut[0] carries n so two networks are distinguishable; the rest stays a legal trit. +static AntColonyBpp9000T::Ann makeAnn(unsigned char n) +{ + AntColonyBpp9000T::Ann a; + setMem(&a, sizeof(a), 0); + a.lut[0] = (unsigned char)(n % 3); + a.lut[1] = (unsigned char)((n / 3) % 3); + return a; +} + +static bool annEquals(const AntColonyBpp9000T::Ann& a, const AntColonyBpp9000T::Ann& b) +{ + for (unsigned long long i = 0; i < sizeof(a); i++) + { + if (a.lut[i] != b.lut[i]) + { + return false; + } + } + return true; +} + +// The score and the network both come back. The network matters as much as the score: commit() +// stores it and childAnnHash folds it into resourceTestingDigest. +TEST(TestAntColonyReplayCache, StoresAndReturnsScoreAndNetwork) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.9 GB"; + + const AntColonyBpp9000T::ReplayKey key = makeReplayKey(1); + const AntColonyBpp9000T::Ann ann = makeAnn(7); + colony->putReplayScore(key, 3800, ann); + + unsigned int score = 0; + AntColonyBpp9000T::Ann out; + setMem(&out, sizeof(out), 0xFF); + ASSERT_TRUE(colony->tryGetReplayScore(key, score, out)); + EXPECT_EQ(score, 3800u); + EXPECT_TRUE(annEquals(out, ann)); +} + +// Every component is part of the key, so changing any one of them must miss. Missing one would +// return a score computed from different inputs. +TEST(TestAntColonyReplayCache, EveryKeyComponentIsPartOfTheLookup) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.9 GB"; + + const AntColonyBpp9000T::ReplayKey key = makeReplayKey(2); + colony->putReplayScore(key, 3800, makeAnn(1)); + + unsigned int score = 0; + AntColonyBpp9000T::Ann out; + for (int component = 0; component < 4; component++) + { + AntColonyBpp9000T::ReplayKey altered = key; + switch (component) + { + case 0: altered.pubkey = makeKey(90001); break; + case 1: altered.nonce = makeKey(90002); break; + case 2: altered.parentKey = makeKey(90003); break; + case 3: altered.anchorDigest = makeKey(90004); break; + } + EXPECT_FALSE(colony->tryGetReplayScore(altered, score, out)) << "component " << component; + } + EXPECT_TRUE(colony->tryGetReplayScore(key, score, out)); +} + +// A new epoch changes every root and anchor digest, so no entry could hit anyway; keeping them would +// just hold slots. +TEST(TestAntColonyReplayCache, BeginEpochClearsIt) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.9 GB"; + + const AntColonyBpp9000T::ReplayKey key = makeReplayKey(3); + colony->putReplayScore(key, 3800, makeAnn(2)); + ASSERT_EQ(colony->replayCacheOccupancy(), 1u); + + colony->beginEpoch(TEST_ROOT_SEED, TEST_INITIAL_TICK); + + unsigned int score = 0; + AntColonyBpp9000T::Ann out; + EXPECT_FALSE(colony->tryGetReplayScore(key, score, out)); + EXPECT_EQ(colony->replayCacheOccupancy(), 0u); +} + +// loadSnapshot() calls reset(), and the catch-up that follows a restore is the one moment the cache +// is worth most. Losing it there would defeat the feature. +TEST(TestAntColonyReplayCache, SurvivesResetAndSnapshotLoad) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.9 GB"; + + const AntColonyBpp9000T::ReplayKey key = makeReplayKey(4); + colony->putReplayScore(key, 3800, makeAnn(3)); + ASSERT_TRUE(colony->saveSnapshot(TEST_EPOCH, NULL, TEST_INITIAL_TICK)); + ASSERT_TRUE(colony->loadSnapshot(TEST_EPOCH, NULL, TEST_ROOT_SEED, TEST_THRESHOLD, TEST_INITIAL_TICK)); + + unsigned int score = 0; + AntColonyBpp9000T::Ann out; + EXPECT_TRUE(colony->tryGetReplayScore(key, score, out)); + EXPECT_EQ(score, 3800u); +} + +// The file is the table verbatim, so this checks that entries survive the write and stay findable +// under their own keys. Enough of them that collisions and evictions are in play. One save writes +// the whole ANT_REPLAY_CACHE_BYTES table, so this is the only test here that touches a file. +TEST(TestAntColonyReplayCache, RoundTripsThroughAFile) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.9 GB"; + + constexpr unsigned int COUNT = 500; + for (unsigned int i = 0; i < COUNT; i++) + { + colony->putReplayScore(makeReplayKey(10000 + i), 3000 + i, makeAnn((unsigned char)i)); + } + ASSERT_EQ(colony->replayCacheOccupancy(), COUNT); + ASSERT_TRUE(colony->saveReplayCache(TEST_EPOCH, NULL)); + + colony->clearReplayCache(); + ASSERT_EQ(colony->replayCacheOccupancy(), 0u); + + ASSERT_TRUE(colony->loadReplayCache(TEST_EPOCH, NULL)); + EXPECT_EQ(colony->replayCacheOccupancy(), COUNT); + + unsigned int score = 0; + AntColonyBpp9000T::Ann out; + for (unsigned int i = 0; i < COUNT; i++) + { + ASSERT_TRUE(colony->tryGetReplayScore(makeReplayKey(10000 + i), score, out)) << "entry " << i; + ASSERT_EQ(score, 3000 + i) << "entry " << i; + ASSERT_TRUE(annEquals(out, makeAnn((unsigned char)i))) << "entry " << i; + } +} + +// No cache is the normal state at the start of an epoch, so it must report a miss and leave an empty +// table rather than fail the boot. +TEST(TestAntColonyReplayCache, AbsentFileIsNotAnError) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.9 GB"; + + colony->putReplayScore(makeReplayKey(7), 3800, makeAnn(6)); + EXPECT_FALSE(colony->loadReplayCache((unsigned short)(TEST_EPOCH + 77), NULL)); + EXPECT_EQ(colony->replayCacheOccupancy(), 0u); + + unsigned int score = 0; + AntColonyBpp9000T::Ann out; + EXPECT_FALSE(colony->tryGetReplayScore(makeReplayKey(7), score, out)); +} + +// --------------------------------------------------------------------------------------------- +// Export best ANN at the end of epoch + +// The file layout: one header, then entryCount of these. +struct ExportFileEntry +{ + AntColonyExportEntry meta; + AntColonyBpp9000T::Ann ann; +}; + +// Reads antColonySolutions.eoe back. Header first, since only it says how long the body is. +static bool readExport(AntColonyExportHeader& header, std::vector& entries) +{ + if (load(ANT_COLONY_SOLUTIONS_EOE_FILENAME, sizeof(header), (unsigned char*)&header) + != (long long)sizeof(header)) + { + return false; + } + entries.clear(); + if (header.entryCount == 0) + { + return true; + } + + const unsigned long long total = sizeof(header) + + (unsigned long long)header.entryCount * sizeof(ExportFileEntry); + std::vector raw(total); + if (load(ANT_COLONY_SOLUTIONS_EOE_FILENAME, total, raw.data()) != (long long)total) + { + return false; + } + entries.resize(header.entryCount); + copyMem(entries.data(), raw.data() + sizeof(header), total - sizeof(header)); + return true; +} + +// More solutions than the file holds, so the cap, the eviction and the ordering are all exercised. +TEST(TestAntColonyExport, KeepsTheLowestScoresInOrder) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.6 GB"; + + constexpr unsigned int COMMITTED = ANT_EXPORT_MAX_SOLUTIONS + 24; + // One identity per solution so the per-parent child cap never binds - the export set is what is + // under test here, not the tree shape. + for (unsigned int i = 0; i < COMMITTED; i++) + { + ASSERT_NE(commitRootChild(colony, makeKey(1 + i), 3000 + i, i, 5000 + i), ANT_INVALID_INDEX) << "commit " << i; + } + ASSERT_TRUE(colony->exportBestSolutions(TEST_EPOCH, NULL)); + + AntColonyExportHeader header; + std::vector entries; + ASSERT_TRUE(readExport(header, entries)); + + EXPECT_EQ(header.entryCount, ANT_EXPORT_MAX_SOLUTIONS); + EXPECT_EQ(header.solutionCount, COMMITTED); + EXPECT_EQ(header.entrySizeBytes, (unsigned int)sizeof(AntColonyExportEntry)); + EXPECT_EQ(header.annSizeBytes, (unsigned int)sizeof(AntColonyBpp9000T::Ann)); + ASSERT_EQ(entries.size(), (size_t)ANT_EXPORT_MAX_SOLUTIONS); + + // The 676 lowest of 3000..3699, so exactly 3000..3675, ascending. + EXPECT_EQ(entries[0].meta.score, 3000u) << "entry 0 must be the best network of the epoch"; + EXPECT_EQ(entries[ANT_EXPORT_MAX_SOLUTIONS - 1].meta.score, 3000u + ANT_EXPORT_MAX_SOLUTIONS - 1); + for (unsigned int i = 1; i < entries.size(); i++) + { + ASSERT_LE(entries[i - 1].meta.score, entries[i].meta.score) << "not ascending at " << i; + } +} + +// Below the cap the file holds everything, still ordered - and the scores are committed descending +// here, so every insert lands at the front and the shift path is the one being used. +TEST(TestAntColonyExport, OrdersFewerThanTheCap) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.6 GB"; + + constexpr unsigned int COUNT = 40; + // One identity per solution so the per-parent child cap never binds. + for (unsigned int i = 0; i < COUNT; i++) + { + ASSERT_NE(commitRootChild(colony, makeKey(2000 + i), 3800 - i, i, 6000 + i), ANT_INVALID_INDEX); + } + ASSERT_TRUE(colony->exportBestSolutions(TEST_EPOCH, NULL)); + + AntColonyExportHeader header; + std::vector entries; + ASSERT_TRUE(readExport(header, entries)); + + ASSERT_EQ(header.entryCount, COUNT); + EXPECT_EQ(entries[0].meta.score, 3800u - (COUNT - 1)); + for (unsigned int i = 1; i < entries.size(); i++) + { + ASSERT_LE(entries[i - 1].meta.score, entries[i].meta.score) << "not ascending at " << i; + } +} + +// Equal scores keep the incumbent, so the earlier solution ranks first. Without a total order two +// nodes with the same solutions could write different files. +TEST(TestAntColonyExport, TiesKeepTheEarlierSolution) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.6 GB"; + + const m256i first = makeKey(3); + const m256i second = makeKey(4); + ASSERT_NE(commitRootChild(colony, first, 3500, 0, 6100), ANT_INVALID_INDEX); + ASSERT_NE(commitRootChild(colony, second, 3500, 1, 6101), ANT_INVALID_INDEX); + ASSERT_TRUE(colony->exportBestSolutions(TEST_EPOCH, NULL)); + + AntColonyExportHeader header; + std::vector entries; + ASSERT_TRUE(readExport(header, entries)); + + ASSERT_EQ(header.entryCount, 2u); + EXPECT_TRUE(entries[0].meta.pubkey == first); + EXPECT_TRUE(entries[1].meta.pubkey == second); +} + +// The stored network has to survive the round trip, a wrong ANN here is a wrong harvest, and +// nothing downstream would notice. +TEST(TestAntColonyExport, CarriesTheNetworkAndItsDepth) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.6 GB"; + + const m256i me = makeKey(5); + ASSERT_NE(commitRootChild(colony, me, 3800, 0, 6200), ANT_INVALID_INDEX); + const SolutionRef aRef = { TEST_PUBLISH_TICK,0 }; + ASSERT_NE(commitChild(colony, me, aRef, 3700, 1, 6201), ANT_INVALID_INDEX); + ASSERT_TRUE(colony->exportBestSolutions(TEST_EPOCH, NULL)); + + AntColonyExportHeader header; + std::vector entries; + ASSERT_TRUE(readExport(header, entries)); + ASSERT_EQ(header.entryCount, 2u); + + // Best first: the depth-2 child at 3700, then its depth-1 parent at 3800. + EXPECT_EQ(entries[0].meta.score, 3700u); + EXPECT_EQ(entries[0].meta.depth, 2u); + EXPECT_EQ(entries[1].meta.score, 3800u); + EXPECT_EQ(entries[1].meta.depth, 1u); + + AntColonyBpp9000T::Ann expected; + ASSERT_TRUE(colony->annOfNonRoot(*colony->recordAt(1), expected)); + for (unsigned long long i = 0; i < sizeof(expected); i++) + { + ASSERT_EQ(entries[0].ann.lut[i], expected.lut[i]) << "genome byte " << i; + } +} + +// The set holds networks the records cannot reproduce once the store is full, so it is saved rather +// than rebuilt. If that file went missing the export would come back empty after a restart. +TEST(TestAntColonyExport, SurvivesASnapshot) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.6 GB"; + + const m256i me = makeKey(6); + for (unsigned int i = 0; i < 10; i++) + { + ASSERT_NE(commitRootChild(colony, me, 3700 + i, i, 6300 + i), ANT_INVALID_INDEX); + } + ASSERT_TRUE(saveWipeLoad(colony)); + ASSERT_TRUE(colony->exportBestSolutions(TEST_EPOCH, NULL)); + + AntColonyExportHeader header; + std::vector entries; + ASSERT_TRUE(readExport(header, entries)); + + ASSERT_EQ(header.entryCount, 10u); + EXPECT_EQ(entries[0].meta.score, 3700u); + EXPECT_EQ(entries[9].meta.score, 3709u); +} + +// A new epoch starts with nothing to export, and the file must say so rather than carry last epoch's. +TEST(TestAntColonyExport, BeginEpochClearsIt) +{ + AntColonyBpp9000T* colony = freshColony(); + ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.6 GB"; + + ASSERT_NE(commitRootChild(colony, makeKey(7), 3500, 0, 6400), ANT_INVALID_INDEX); + colony->beginEpoch(TEST_ROOT_SEED, TEST_INITIAL_TICK); + ASSERT_TRUE(colony->exportBestSolutions(TEST_EPOCH, NULL)); + + AntColonyExportHeader header; + std::vector entries; + ASSERT_TRUE(readExport(header, entries)); + EXPECT_EQ(header.entryCount, 0u); + EXPECT_EQ(header.solutionCount, 0u); +} diff --git a/test/ant_pending_solutions.cpp b/test/ant_pending_solutions.cpp new file mode 100644 index 00000000..d26c084c --- /dev/null +++ b/test/ant_pending_solutions.cpp @@ -0,0 +1,233 @@ +#define NO_UEFI + +#include "gtest/gtest.h" + +#include "../src/mining/ant_colony/ant_pending_solutions.h" + +static m256i key(unsigned long long n) +{ + m256i k = m256i::zero(); + k.m256i_u64[0] = n + 1; // never zero: a zero pubkey marks an unused slot + return k; +} + +static SolutionRef ref(unsigned int tick, unsigned int idx) +{ + SolutionRef r; + r.tick = tick; + r.solutionIndexInTick = idx; + return r; +} + +// 5.75 MB, so one buffer for the file, reset between tests. +static AntPendingSolutions* freshPool() +{ + static AntPendingSolutions pool; + static bool allocated = false; + static bool failed = false; + if (!allocated && !failed) + { + failed = !pool.init(); + allocated = !failed; + } + if (failed) + { + return nullptr; + } + pool.reset(); + return &pool; +} + +// The key is (computor, parentRef, nonce). Same triple twice is one solution, whatever else differs. +TEST(TestAntColonyPending, DedupsOnTheConsensusKey) +{ + AntPendingSolutions* pool = freshPool(); + ASSERT_NE(pool, nullptr); + + EXPECT_TRUE(pool->add(key(1), ref(100, 0), 5000, 0, key(900))); + EXPECT_FALSE(pool->add(key(1), ref(100, 0), 5000, 0, key(900))); + + // A different anchor is the SAME solution - anchorTick is deliberately not in the key, so a + // re-anchored resend cannot be published twice. + EXPECT_FALSE(pool->add(key(1), ref(100, 0), 9999, 0, key(900))); + + // Any other field differing makes it a different solution. + EXPECT_TRUE(pool->add(key(2), ref(100, 0), 5000, 0, key(900))); + EXPECT_TRUE(pool->add(key(1), ref(100, 1), 5000, 0, key(900))); + EXPECT_TRUE(pool->add(key(1), ref(100, 0), 5000, 0, key(901))); +} + +// A fresh entry is selectable, and only by the computor it belongs to. +TEST(TestAntColonyPending, SelectsOnlyForItsOwnComputor) +{ + AntPendingSolutions* pool = freshPool(); + ASSERT_NE(pool, nullptr); + ASSERT_TRUE(pool->add(key(1), ref(100, 0), 5000, 0, key(900))); + + AntPendingSolution out; + EXPECT_EQ(pool->selectForPublish(key(2), 5000, out), AntPendingSolutions::NO_ENTRY); + + const unsigned int idx = pool->selectForPublish(key(1), 5000, out); + ASSERT_NE(idx, AntPendingSolutions::NO_ENTRY); + EXPECT_TRUE(out.nonce == key(900)); + EXPECT_EQ(out.anchorTick, 5000u); +} + +// Scheduling records a deadline. Before it passes the entry must not come back, or the node would +// republish a transaction that is still in flight. +TEST(TestAntColonyPending, ScheduledEntryIsNotReselectedBeforeItsDeadline) +{ + AntPendingSolutions* pool = freshPool(); + ASSERT_NE(pool, nullptr); + ASSERT_TRUE(pool->add(key(1), ref(100, 0), 5000, 0, key(900))); + + AntPendingSolution out; + const unsigned int idx = pool->selectForPublish(key(1), 5000, out); + ASSERT_NE(idx, AntPendingSolutions::NO_ENTRY); + pool->markScheduled(idx, 5003); + + EXPECT_EQ(pool->selectForPublish(key(1), 5001, out), AntPendingSolutions::NO_ENTRY); + EXPECT_EQ(pool->selectForPublish(key(1), 5002, out), AntPendingSolutions::NO_ENTRY); + + // Deadline reached with no acknowledgement: republish. + EXPECT_EQ(pool->selectForPublish(key(1), 5003, out), idx); +} + +// The whole reason the state is a tick and not a flag. +TEST(TestAntColonyPending, RetriesOutrankFreshEntries) +{ + AntPendingSolutions* pool = freshPool(); + ASSERT_NE(pool, nullptr); + + ASSERT_TRUE(pool->add(key(1), ref(100, 0), 5000, 0, key(900))); + AntPendingSolution out; + const unsigned int stale = pool->selectForPublish(key(1), 5000, out); + ASSERT_NE(stale, AntPendingSolutions::NO_ENTRY); + pool->markScheduled(stale, 5003); + + // Newer solutions keep arriving while the first one's transaction is lost. + ASSERT_TRUE(pool->add(key(1), ref(100, 1), 5001, 0, key(901))); + ASSERT_TRUE(pool->add(key(1), ref(100, 2), 5002, 0, key(902))); + + // Past the deadline the retry must win, or a steady stream of new work starves it forever. + EXPECT_EQ(pool->selectForPublish(key(1), 5003, out), stale); +} + +// RECORDED comes from observing the chain, and must both stop republication and suppress a resend. +TEST(TestAntColonyPending, RecordedStopsRepublishingAndSuppressesResend) +{ + AntPendingSolutions* pool = freshPool(); + ASSERT_NE(pool, nullptr); + ASSERT_TRUE(pool->add(key(1), ref(100, 0), 5000, 0, key(900))); + + AntPendingSolution out; + const unsigned int idx = pool->selectForPublish(key(1), 5000, out); + ASSERT_NE(idx, AntPendingSolutions::NO_ENTRY); + pool->markScheduled(idx, 5003); + pool->markRecorded(key(1), ref(100, 0), key(900)); + + EXPECT_EQ(pool->selectForPublish(key(1), 9000, out), AntPendingSolutions::NO_ENTRY); + EXPECT_FALSE(pool->add(key(1), ref(100, 0), 5000, 0, key(900))); +} + +// A transaction the node never queued still has to be remembered, or a miner resending a solution +// that is already on-chain makes the pool publish it a second time and pay a second deposit. +TEST(TestAntColonyPending, RecordingAnUnqueuedSolutionSuppressesALaterSubmission) +{ + AntPendingSolutions* pool = freshPool(); + ASSERT_NE(pool, nullptr); + + pool->markRecorded(key(1), ref(100, 0), key(900)); + EXPECT_FALSE(pool->add(key(1), ref(100, 0), 5000, 0, key(900))); +} + +// Past the publish window the commit path rejects it as stale, so publishing spends the deposit for +// nothing. Selection has to drop it rather than hand it over. +TEST(TestAntColonyPending, ExpiredEntriesAreRetiredNotPublished) +{ + AntPendingSolutions* pool = freshPool(); + ASSERT_NE(pool, nullptr); + ASSERT_TRUE(pool->add(key(1), ref(100, 0), 5000, 0, key(900))); + + AntPendingSolution out; + EXPECT_EQ(pool->selectForPublish(key(1), 5000 + ANT_PUBLISH_WINDOW_TICKS, out), 0u); + EXPECT_EQ(pool->selectForPublish(key(1), 5000 + ANT_PUBLISH_WINDOW_TICKS + 1, out), AntPendingSolutions::NO_ENTRY); + + AntPendingSolutions::Stats stats; + unsigned int count = 0; + pool->getStats(stats, count); + EXPECT_EQ(stats.obsoleteExpired, 1u); +} + +// An entry retired for expiry was never published, so the seen filter was never marked and the key +// is still usable. A resubmission with a fresh anchor must replace it rather than be refused as a +// duplicate - otherwise the solution is stranded and the miner is never told why. +TEST(TestAntColonyPending, ExpiredEntryCanBeResubmittedWithAFreshAnchor) +{ + AntPendingSolutions* pool = freshPool(); + ASSERT_NE(pool, nullptr); + ASSERT_TRUE(pool->add(key(1), ref(100, 0), 5000, 0, key(900))); + + // Selection retires it instead of publishing: publishing a stale one would mark the seen filter + // and kill this key permanently. + AntPendingSolution out; + ASSERT_EQ(pool->selectForPublish(key(1), 5000 + ANT_PUBLISH_WINDOW_TICKS + 1, out), AntPendingSolutions::NO_ENTRY); + + // Same triple, newer anchor. This is the replacement, not a duplicate. + EXPECT_TRUE(pool->add(key(1), ref(100, 0), 40000, 0, key(900))); + + const unsigned int idx = pool->selectForPublish(key(1), 40000, out); + ASSERT_NE(idx, AntPendingSolutions::NO_ENTRY); + EXPECT_EQ(out.anchorTick, 40000u); +} + +// The score is computed once, at receipt, and the publisher reads it back from the entry. +TEST(TestAntColonyPending, CarriesTheScore) +{ + AntPendingSolutions* pool = freshPool(); + ASSERT_NE(pool, nullptr); + ASSERT_TRUE(pool->add(key(1), ref(100, 0), 5000, 3771, key(900))); + + AntPendingSolution out; + ASSERT_NE(pool->selectForPublish(key(1), 5000, out), AntPendingSolutions::NO_ENTRY); + EXPECT_EQ(out.score, 3771u); +} + +// A live entry is a real duplicate, whether it has been scheduled or not. +TEST(TestAntColonyPending, LiveEntryStillRejectsAResubmission) +{ + AntPendingSolutions* pool = freshPool(); + ASSERT_NE(pool, nullptr); + ASSERT_TRUE(pool->add(key(1), ref(100, 0), 5000, 0, key(900))); + + EXPECT_FALSE(pool->add(key(1), ref(100, 0), 6000, 0, key(900))); + + AntPendingSolution out; + const unsigned int idx = pool->selectForPublish(key(1), 5000, out); + ASSERT_NE(idx, AntPendingSolutions::NO_ENTRY); + pool->markScheduled(idx, 5003); + EXPECT_FALSE(pool->add(key(1), ref(100, 0), 6000, 0, key(900))); +} + +// Finished slots are reused in place, so a pool that has published for a whole epoch does not fill. +TEST(TestAntColonyPending, FinishedSlotsAreReclaimed) +{ + AntPendingSolutions* pool = freshPool(); + ASSERT_NE(pool, nullptr); + + for (unsigned int i = 0; i < 4; i++) + { + ASSERT_TRUE(pool->add(key(1), ref(100, i), 5000, 0, key(900 + i))); + pool->markRecorded(key(1), ref(100, i), key(900 + i)); + } + + AntPendingSolutions::Stats stats; + unsigned int count = 0; + pool->getStats(stats, count); + EXPECT_EQ(stats.recorded, 4u); + + // Nothing left to publish, and new work still fits. + AntPendingSolution out; + EXPECT_EQ(pool->selectForPublish(key(1), 5000, out), AntPendingSolutions::NO_ENTRY); + EXPECT_TRUE(pool->add(key(1), ref(200, 0), 5000, 0, key(1000))); +} diff --git a/test/contract_qraffle.cpp b/test/contract_qraffle.cpp index 4c40fad5..98c6d889 100644 --- a/test/contract_qraffle.cpp +++ b/test/contract_qraffle.cpp @@ -6,9 +6,10 @@ #include "contract_testing.h" // Test-local expectation for the asset-raffle creator share. The contract derives the -// creator payout implicitly by subtracting the 20% fee split (5% burn + 1% charity + -// 8% shareholder + 5% register + 1% fee), so 80% is the expected creator percentage. -static constexpr uint32 QRAFFLE_TEST_ASSET_RAFFLE_CREATOR_PCT = 80; +// creator payout implicitly by subtracting the 21% fee split (5% burn + 1% charity + +// 8% shareholder + 5% register + 1% fee + 1% entropy reserve), so 79% is the expected +// creator percentage. +static constexpr uint32 QRAFFLE_TEST_ASSET_RAFFLE_CREATOR_PCT = 79; static std::mt19937_64 rand64; @@ -297,6 +298,11 @@ class ContractTestingQraffle : protected ContractTesting { initEmptySpectrum(); initEmptyUniverse(); + // RANDOM must be constructed before QRAFFLE so END_EPOCH's cross-contract + // BuyEntropy call has an active contract to invoke. + system.epoch = contractDescriptions[RANDOM_CONTRACT_INDEX].constructionEpoch; + INIT_CONTRACT(RANDOM); + callSystemProcedure(RANDOM_CONTRACT_INDEX, INITIALIZE); system.epoch = contractDescriptions[QRAFFLE_CONTRACT_INDEX].constructionEpoch; INIT_CONTRACT(QRAFFLE); callSystemProcedure(QRAFFLE_CONTRACT_INDEX, INITIALIZE); @@ -309,6 +315,26 @@ class ContractTestingQraffle : protected ContractTesting return (QRaffleChecker*)contractStates[QRAFFLE_CONTRACT_INDEX]; } + RANDOM::StateData* randomState() + { + return reinterpret_cast(contractStates[RANDOM_CONTRACT_INDEX]); + } + + // Seeds RANDOM's finalized entropy for the stream BuyEntropy will read when called at the + // *next* endEpoch() (mirrors the +2 offset BuyEntropy itself uses to read the last-finalized + // stream). Lets tests make the entropy purchase deterministically succeed with a known value. + QPI::bit_4096 seedRandomEntropy(uint64 seed) + { + QPI::bit_4096 entropy{}; + for (uint64 i = 0; i < QRAFFLE_RANDOM_ENTROPY_BITS; ++i) + { + entropy.set(i, ((seed + i) & 1ULL) != 0); + } + const uint32 stream = (system.tick + 2u) % 3u; + randomState()->entropy.set(stream * 10u + QRAFFLE_RANDOM_COLLATERAL_TIER, entropy); + return entropy; + } + void endEpoch(bool expectSuccess = true) { callSystemProcedure(QRAFFLE_CONTRACT_INDEX, END_EPOCH, expectSuccess); @@ -1174,11 +1200,15 @@ TEST(ContractQraffle, GetFunctions) } // Deposit in QuRaffle + // Captured once, before END_EPOCH resets/recalculates it for the next epoch, so the + // expected-value math below reflects whatever amount members actually deposited at + // (rather than hardcoding QRAFFLE_DEFAULT_QRAFFLE_AMOUNT's current value, which drifts). + const uint64 actualQREAmount = qraffle.getState()->getQuRaffleEntryAmount(); uint32 memberCount = 0; for (size_t i = 0; i < users.size() / 3; ++i) { - increaseEnergy(users[i], qraffle.getState()->getQuRaffleEntryAmount()); - auto result = qraffle.depositInQuRaffle(users[i], qraffle.getState()->getQuRaffleEntryAmount()); + increaseEnergy(users[i], actualQREAmount); + auto result = qraffle.depositInQuRaffle(users[i], actualQREAmount); EXPECT_EQ(result.returnCode, QRAFFLE_SUCCESS); memberCount++; } @@ -1284,18 +1314,20 @@ TEST(ContractQraffle, GetFunctions) // Calculate expected values from QuRaffle (if any members participated) if (memberCount > 0) { - uint64 qREAmount = 10000000; // initial entry amount - uint64 totalQuRaffleAmount = qREAmount * memberCount; + uint64 totalQuRaffleAmount = actualQREAmount * memberCount; expectedTotalBurnAmount += (totalQuRaffleAmount * QRAFFLE_BURN_FEE) / 100; expectedTotalCharityAmount += (totalQuRaffleAmount * QRAFFLE_CHARITY_FEE) / 100; expectedTotalShareholderAmount += ((totalQuRaffleAmount * QRAFFLE_SHAREHOLDER_FEE) / 100) / 676 * 676; expectedTotalRegisterAmount += ((totalQuRaffleAmount * QRAFFLE_REGISTER_FEE) / 100) / registerCount * registerCount; expectedTotalFeeAmount += (totalQuRaffleAmount * QRAFFLE_FEE) / 100; - - // Winner amount calculation (after all fees) - uint64 winnerAmount = totalQuRaffleAmount - expectedTotalBurnAmount - expectedTotalCharityAmount - - expectedTotalShareholderAmount - expectedTotalRegisterAmount - expectedTotalFeeAmount; + uint64 expectedTotalEntropyAmount = (totalQuRaffleAmount * QRAFFLE_ENTROPY_FEE) / 100; + + // Winner amount calculation (after all fees, including the entropy reserve + // carve-out that funds RANDOM entropy purchases -- retained, not transferred) + uint64 winnerAmount = totalQuRaffleAmount - expectedTotalBurnAmount - expectedTotalCharityAmount + - expectedTotalShareholderAmount - expectedTotalRegisterAmount - expectedTotalFeeAmount + - expectedTotalEntropyAmount; expectedTotalWinnerAmount += winnerAmount; expectedLargestWinnerAmount = winnerAmount; // First winner sets the largest } @@ -1374,7 +1406,7 @@ TEST(ContractQraffle, GetFunctions) { EXPECT_NE(endedQuRaffle.epochWinner, id(0, 0, 0, 0)); EXPECT_GT(endedQuRaffle.receivedAmount, 0); - EXPECT_EQ(endedQuRaffle.entryAmount, 10000000); + EXPECT_EQ(endedQuRaffle.entryAmount, actualQREAmount); } // Test with future epoch @@ -4206,4 +4238,150 @@ TEST(ContractQraffle, AssetRaffle_MultipleEpochs_PerCreatorCounterReset) makeBundle(t201, 100)); EXPECT_EQ(r.returnCode, QRAFFLE_SUCCESS) << "epoch 201 raffle " << i; } +} + +// --------------------------------------------------------------------------- +// RANDOM entropy mixing: END_EPOCH buys a small amount of entropy from the +// RANDOM smart contract and mixes it into the digest-based winner-selection +// seed. The purchase is additive (never blocks settlement) and is funded by +// a 1% reserve carved out of QuRaffle/AssetRaffle pools -- retained in the +// contract's own balance, never transferred out. TokenRaffle pools are +// token-denominated, not Qu, so they don't fund the reserve. +// --------------------------------------------------------------------------- + +TEST(ContractQraffle, EntropyReserveAccumulatesFromQuRaffleSettlement) +{ + ContractTestingQraffle qraffle; + const id qraffleSelf = id(QRAFFLE_CONTRACT_INDEX, 0, 0, 0); + id registerUser = getUser(50001); + increaseEnergy(registerUser, QRAFFLE_REGISTER_AMOUNT); + qraffle.registerInSystem(registerUser, QRAFFLE_REGISTER_AMOUNT, 0); + + // Snapshot after registration: registerInSystem's fee is retained in the contract's own + // balance too, so it must not be mistaken for the entropy reserve carve-out below. + const long long contractBalanceBefore = getBalance(qraffleSelf); + + const uint64 entryAmount = qraffle.getState()->getQuRaffleEntryAmount(); + const uint32 memberCount = 3; + for (uint32 i = 0; i < memberCount; ++i) + { + id member = getUser(50100 + i); + increaseEnergy(member, entryAmount); + EXPECT_EQ(qraffle.depositInQuRaffle(member, entryAmount).returnCode, QRAFFLE_SUCCESS); + } + + const uint64 earnedBefore = qraffle.randomState()->earnedAmount; + + qraffle.endEpoch(); + + // No prior reserve was funded, so the purchase must not have been attempted. + EXPECT_EQ(qraffle.randomState()->earnedAmount, earnedBefore) + << "with zero reserve, END_EPOCH must not attempt a BuyEntropy purchase"; + + const uint64 pool = entryAmount * memberCount; + const uint64 expectedEntropyReserve = (pool * QRAFFLE_ENTROPY_FEE) / 100; + EXPECT_EQ(getBalance(qraffleSelf), contractBalanceBefore + (long long)expectedEntropyReserve) + << "the entropy reserve carve-out must be retained in the contract's own balance, " + "not transferred, netting out to just the reserve once the rest of the pool is paid out"; +} + +TEST(ContractQraffle, BuyEntropyPurchaseSucceedsWhenReserveFunded) +{ + ContractTestingQraffle qraffle; + + id registerUser = getUser(50002); + increaseEnergy(registerUser, QRAFFLE_REGISTER_AMOUNT); + qraffle.registerInSystem(registerUser, QRAFFLE_REGISTER_AMOUNT, 0); + + const uint64 entryAmount = qraffle.getState()->getQuRaffleEntryAmount(); + id member = getUser(50200); + increaseEnergy(member, entryAmount); + EXPECT_EQ(qraffle.depositInQuRaffle(member, entryAmount).returnCode, QRAFFLE_SUCCESS); + + // Pre-fund the reserve directly (bypasses needing several epochs to accumulate it + // organically) and seed RANDOM with a known, non-zero entropy value. + increaseEnergy(id(QRAFFLE_CONTRACT_INDEX, 0, 0, 0), QRAFFLE_RANDOM_ENTROPY_FEE); + qraffle.seedRandomEntropy(0xA11CE); + + const uint64 earnedBefore = qraffle.randomState()->earnedAmount; + + qraffle.endEpoch(); + + EXPECT_EQ(qraffle.randomState()->earnedAmount, earnedBefore + QRAFFLE_RANDOM_ENTROPY_FEE) + << "with a funded reserve and available RANDOM entropy, END_EPOCH must complete the purchase"; + + auto ended = qraffle.getEndedQuRaffle((uint16)contractDescriptions[QRAFFLE_CONTRACT_INDEX].constructionEpoch); + EXPECT_EQ(ended.returnCode, QRAFFLE_SUCCESS); + EXPECT_EQ(ended.epochWinner, member); +} + +TEST(ContractQraffle, EndEpochSettlesNormallyWhenEntropyReserveIsEmpty) +{ + ContractTestingQraffle qraffle; + + id registerUser = getUser(50003); + increaseEnergy(registerUser, QRAFFLE_REGISTER_AMOUNT); + qraffle.registerInSystem(registerUser, QRAFFLE_REGISTER_AMOUNT, 0); + + const uint64 entryAmount = qraffle.getState()->getQuRaffleEntryAmount(); + id member = getUser(50300); + increaseEnergy(member, entryAmount); + EXPECT_EQ(qraffle.depositInQuRaffle(member, entryAmount).returnCode, QRAFFLE_SUCCESS); + + // No reserve funding at all -- purchase must be skipped, not block settlement. + const uint64 earnedBefore = qraffle.randomState()->earnedAmount; + + qraffle.endEpoch(); + + EXPECT_EQ(qraffle.randomState()->earnedAmount, earnedBefore); + + auto ended = qraffle.getEndedQuRaffle((uint16)contractDescriptions[QRAFFLE_CONTRACT_INDEX].constructionEpoch); + EXPECT_EQ(ended.returnCode, QRAFFLE_SUCCESS); + EXPECT_EQ(ended.epochWinner, member) + << "raffle settlement must proceed normally even when the entropy purchase is skipped"; +} + +TEST(ContractQraffle, EndEpochFallsBackGracefullyWhenRandomPoolIsEmpty) +{ + ContractTestingQraffle qraffle; + const id qraffleSelf = id(QRAFFLE_CONTRACT_INDEX, 0, 0, 0); + + id registerUser = getUser(50004); + increaseEnergy(registerUser, QRAFFLE_REGISTER_AMOUNT); + qraffle.registerInSystem(registerUser, QRAFFLE_REGISTER_AMOUNT, 0); + + // Snapshot after registration: registerInSystem's fee is retained in the contract's own + // balance too, so it must not be mistaken for the entropy reserve carve-out below. + const long long contractBalanceBefore = getBalance(qraffleSelf); + + const uint64 entryAmount = qraffle.getState()->getQuRaffleEntryAmount(); + id member = getUser(50400); + increaseEnergy(member, entryAmount); + EXPECT_EQ(qraffle.depositInQuRaffle(member, entryAmount).returnCode, QRAFFLE_SUCCESS); + + // Fund the reserve so the balance check passes, but leave RANDOM's entropy pool at + // its default zero state so BuyEntropy itself refunds instead of delivering entropy. + increaseEnergy(qraffleSelf, QRAFFLE_RANDOM_ENTROPY_FEE); + + const uint64 earnedBefore = qraffle.randomState()->earnedAmount; + + qraffle.endEpoch(); + + // BuyEntropy refunds the full invocation reward when its entropy pool is empty, so + // RANDOM must not have earned anything from this attempt. + EXPECT_EQ(qraffle.randomState()->earnedAmount, earnedBefore); + + // Net balance: +QRAFFLE_RANDOM_ENTROPY_FEE (pre-funded, refunded back unspent) + // +entropyReserveAmount (this epoch's carve-out) -pool +pool (paid out to winner/fees, + // net of the reserve) -- collapses to just the pre-funded fee plus the reserve. + const uint64 pool = entryAmount; + const uint64 expectedEntropyReserve = (pool * QRAFFLE_ENTROPY_FEE) / 100; + EXPECT_EQ(getBalance(qraffleSelf), + contractBalanceBefore + (long long)QRAFFLE_RANDOM_ENTROPY_FEE + (long long)expectedEntropyReserve) + << "a failed BuyEntropy purchase (empty RANDOM pool) must refund in full, " + "leaving only the pre-funded fee and the entropy reserve carve-out as the net change"; + + auto ended = qraffle.getEndedQuRaffle((uint16)contractDescriptions[QRAFFLE_CONTRACT_INDEX].constructionEpoch); + EXPECT_EQ(ended.returnCode, QRAFFLE_SUCCESS); + EXPECT_EQ(ended.epochWinner, member); } \ No newline at end of file diff --git a/test/data/bpp9000.task b/test/data/bpp9000.task new file mode 100644 index 00000000..8ffa5e17 Binary files /dev/null and b/test/data/bpp9000.task differ diff --git a/test/data/gt_ant_production.csv b/test/data/gt_ant_production.csv new file mode 100644 index 00000000..664c7574 --- /dev/null +++ b/test/data/gt_ant_production.csv @@ -0,0 +1,65 @@ +chain, depth, pubkey, nonce, anchor, seed, score +2, 0, 886ef7f14d42287f8a74aaa92a409eaf901e25b6f9cfbedb5f72e1297bcad551, 01084ed07b9200fe2bb401df72b52ba819fd5c09e436f5f185594f8b8afcbc66, 54ba8fded70f55d660977d169d6a2ab6d7711b11f2b49596fbab1d5bf968a961, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4576 +5, 0, 2df29076f8969739636dff083eb1f54bfe7e5d8fcac5b2946d0871d40ba495aa, 0106216c6da947657f74a629a59d5b81da7a4a6ecce8a3abb4b23079b5f1c56e, 21c6ec71261f8027ee7fe4629982e12c50facf8d0acc1f8f297264db6bc2dc2c, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 6400 +11, 0, 83b379182516db53587b3c553468e1af0ca4c21f13b95150024e68ebb2bb9550, 01085ebfe74c8324c42214dde9c3c71f691e327cd8a764e4c396d7431deaae03, 12545454084db67b036475332b9a7ebd7c7ed73b13e64bc17d87ec2e758dcb22, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 5308 +1, 0, 17620b9920ef596dfa46753b355247b1641eaeda1246fa7bc43884630cf61bf2, 010523bbe90fa21b263946830cc53e8744e270cf6e944b1cf255e85ac74a7887, eae546338da0b54a3b4e10f859b6293f1d285a0e51e4881f10d1379ff511f819, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 6295 +0, 0, 4657de22da43ad6402c8cfdd45ba54542213486481988bdd28c82182ec2714af, 01023378b9bf89fae25fdcd165c8739da9dbba04ff92982345a623031d23b0fb, 91a1d4fc06689515127b01f116fd64c4c2ca4c02a4692675c1b0dd4fc1bf3589, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4800 +8, 0, e0b3b0f9b66fe475eaf17c6be38a2759ff254debd5485c4d4a27bc45486fa0ee, 010733ccce2e806a4aea54c0dcc386ebfd4efc6f3027f63c1de301686508f403, a2050b44a20027fb98984b156632263a8bccefd34e4c1a06d516ac807ea19433, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4414 +6, 0, dc40c51265eff259556066d427dbeebb4109a7e79690d61d6349ef650b29dcb9, 010a33855fd20d0f1c910f0f7f7bc4eef7dfde506c7184d12bb6e50dc095b9de, 0509c75356fc65087c969b8eb8602a28c903a34d8f0648394e29dbeaa9cb4892, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4206 +10, 0, eda9ead70a463d6bf5858faf2a8caa3c3d20b3c3e0e994167e685200bc750276, 01080bf6551a06f4826c9dfb047814381042f281b322ab9452623183daab9dd3, a0651819278fad70d8d2e24db8fa595f305cb879b400fb0c270ecad1361cc25c, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4352 +4, 0, e5e05648b1c70c4deea4c7d7aefccd49b24defadceb45dd033caf0f16407808b, 0105337415ffe5aae3f1e4816712ba7ca6e4df3db6b10276868d4e2861cb69e8, a18598ffcfd37339a922658ea450e94f2e4cd083c4e64d5013d002c4fcb17743, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4379 +6, 1, dc40c51265eff259556066d427dbeebb4109a7e79690d61d6349ef650b29dcb9, 010a54a6afb54d321417a986b3e8ce05e51823ea0aceef2681392fcda42010b0, bb849545500e6716bdbbf6fbbdcf6bfe1fd2b4cc20c1024cb419be77dae27df0, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4206 +5, 1, 2df29076f8969739636dff083eb1f54bfe7e5d8fcac5b2946d0871d40ba495aa, 010354eaf1f371ea113434301d5e15b04d24b92fc11dd20f5e03fb6e6f497d30, e9bf8e92a20da221725f0897cf111f326c7754ead360f7aba074459ad99ec7f9, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 5998 +3, 0, 072fef44ac2f7667c366f067b374016bdd39581dbe8acb0f8aadf98502d13646, 010329371769a20be8033c7868d60327ad8b345b190f69f6a6b54f55c7acde8c, a3e0ad1361d551b2ce13025624602a4ebbcb0bc37c93348a88d23d394c155bd9, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4569 +2, 1, 886ef7f14d42287f8a74aaa92a409eaf901e25b6f9cfbedb5f72e1297bcad551, 01081abdbc10a2b70d301fe0bb0afa298f90a5d9bdd0777d24949228abc63106, 02724836ecf8a9bd54e8f78867ea5b4e5c13dc6c41566094a6b41fa36e219f09, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4576 +8, 1, e0b3b0f9b66fe475eaf17c6be38a2759ff254debd5485c4d4a27bc45486fa0ee, 010229cda723c097587f19e30ee090700b7da9bfc9a24e2fb73d83d55b4ba107, c55bea5399c1ca80efe9b9b6c51991ce99b6cc34f804984957cc7a110a1e03cd, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4373 +0, 1, 4657de22da43ad6402c8cfdd45ba54542213486481988bdd28c82182ec2714af, 010615c66cb4c65a74382fce9000de43248a4425ced3bc79929139502e97c705, 4a564c2984b914fda7e68a1c80fd1bfdf6a75ac2443f348e98645d42fbaa052a, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4450 +10, 1, eda9ead70a463d6bf5858faf2a8caa3c3d20b3c3e0e994167e685200bc750276, 010a5e540d6e35170556e51d0a25400b8e7dce3a21872d56e397633411fc7656, eb23ade51e8662688b5e069f59e64c9049c7896058ee4460bab704180f8be6e6, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4352 +1, 1, 17620b9920ef596dfa46753b355247b1641eaeda1246fa7bc43884630cf61bf2, 01050a3c2127a028962e38ab80d9eb6848790106a3809637f8b1eb849c261a1b, 8dc4c241e049f0be5341d03b0d1023ae5340cd8cc506dd6f03a998743e2fed75, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4663 +2, 2, 886ef7f14d42287f8a74aaa92a409eaf901e25b6f9cfbedb5f72e1297bcad551, 010246c5bb2fa79a2bc5f2ed1fbb2877ba2de90011ee439383347c684cfb80a6, ef9bab5a83f63baadbd79d9e3680a36eeb65d8c9c1c52c6dce0e47f807ccb465, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4394 +11, 1, 83b379182516db53587b3c553468e1af0ca4c21f13b95150024e68ebb2bb9550, 01025ded1ea31c0763dd89595fce37c703c886203ef104ccea9979bf6041b511, 967e0547550da817764ba0a7b51e33556c2435e9a910e4b16d9300bd785a06e1, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 5308 +4, 1, e5e05648b1c70c4deea4c7d7aefccd49b24defadceb45dd033caf0f16407808b, 0106282d2480193df99350d4ef0c3b31b9a4314b38cc3d0e86c3c9564b9fc77b, 9ba91514fec4c80a40201e7fbc9bf7d08ee446b8f3efaff16e32ae54c5afdc92, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4310 +6, 2, dc40c51265eff259556066d427dbeebb4109a7e79690d61d6349ef650b29dcb9, 01052e993e876de4beeed6b7e5f674acf08d15f44a41dc59e080b50b5e9ee9ee, a570a04a90af0b23bf9ea5b4be54314a853f517a060c39df1d983e9665f4c45b, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4206 +3, 1, 072fef44ac2f7667c366f067b374016bdd39581dbe8acb0f8aadf98502d13646, 0109595f50ab0a4d9c0c2209dc2e020058a6a3fcaa4ab687f5338d46a4259031, b8a420a950c50a97f8876a589cc1a4f2822eed80efc2e7ef6bd44b2c780061c6, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4569 +5, 2, 2df29076f8969739636dff083eb1f54bfe7e5d8fcac5b2946d0871d40ba495aa, 01030cc6084cbef2f92c8fc481db9b3b36ec91723770bb5c6aaea1dfb2104376, b3310ce38053f82c73d26eed58ad00897a8d0de995d60c4eeb8d4755afc518c9, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4836 +10, 2, eda9ead70a463d6bf5858faf2a8caa3c3d20b3c3e0e994167e685200bc750276, 01040809aeb747d430f914d0ceb1970d3ff7e50b44ac1db1de8ac6a910795423, 5df3e71634f680908f5954ce538b65f470d65779dfcae051f1d42b43f1527abe, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4241 +10, 3, eda9ead70a463d6bf5858faf2a8caa3c3d20b3c3e0e994167e685200bc750276, 01075358c48d3c17682f8c69578d2db1710168ebad922b591b764b0937353be2, 190832364108015065a051e38badc3535be4faff42a319132ebfecea4bf03416, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4241 +8, 2, e0b3b0f9b66fe475eaf17c6be38a2759ff254debd5485c4d4a27bc45486fa0ee, 010307ae588e0d66befcfb11099713189200bfeab97cb37d4f18bfa0d877cd2a, 2032dcba5a6c53d1cb68ad800ae8c7b455aef9aa90e1c14cb5591bbe2c8e1927, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4307 +6, 3, dc40c51265eff259556066d427dbeebb4109a7e79690d61d6349ef650b29dcb9, 01014f1dc6bcc0311403fe50177fe2b69d702b1109db30aab04be2b6cd964375, 7f2fbd2dca697fee9169049bd06a5cadb6932db37cc8190b7f781e2859a13ddf, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4206 +0, 2, 4657de22da43ad6402c8cfdd45ba54542213486481988bdd28c82182ec2714af, 01011280b730b2317044f434b8e56ca881f7efeaf8dd4b29b12e877ea43fb7a1, d260aab327663b1197f602584a74c80dfdc3915d487440e21d59b6d795ba284c, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4450 +2, 3, 886ef7f14d42287f8a74aaa92a409eaf901e25b6f9cfbedb5f72e1297bcad551, 0107159b5267736895605a21efe7bfa20f676707dd2bfdccabc6d67ed211d511, 32fa4cc87d7f7ba35dfcc9fad0256dd64bd1c3a340e1fb1ce92a2e3102641be8, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4291 +11, 2, 83b379182516db53587b3c553468e1af0ca4c21f13b95150024e68ebb2bb9550, 01040f59f2e3d85b897c1997ae4cd6a1aa20f0397795106ec35742efa2f60207, 70efb4b8a77579ec350179a71f89e517c378ec87896a526d5bca985e45e6a2fc, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4581 +1, 2, 17620b9920ef596dfa46753b355247b1641eaeda1246fa7bc43884630cf61bf2, 0103333ea8e0bc20c0f2cfb7cc93e9d3ae302f4b5e933e605ca8d5f2b101f7c8, 39cb97cbe391fff147ecffe6bc7b6cfcf920726974c9ad5394b54e7a86d3e752, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4663 +14, 0, 8c2cce72fa295e618dada27cd9dce2a8f24a4596d676edf11e096bcf6f5c078f, 0103553c86f3dcc0e0a8c63a9004bc5719b09b04348d77a9de594f5b8bdffaae, db7e8851e5e2b45c072bf7cd15040a9bc313b6d4e8e568f7a50a62810a181842, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 5808 +5, 3, 2df29076f8969739636dff083eb1f54bfe7e5d8fcac5b2946d0871d40ba495aa, 01022aa2e9c6bf0d46e240165a772be18804f052b8f76111e0c22af538b332e6, ed0147d60ab884ab994eca1b25f319349437e4106b21068d0f8ae9c357102e14, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4779 +4, 2, e5e05648b1c70c4deea4c7d7aefccd49b24defadceb45dd033caf0f16407808b, 01080ccfc0cce11c4229f4dd023630330c17f3f495c3db73a30ed50107882a5e, eb7729568d616b19004a42dab7f8b237defd6eb3a0a5448037ecf4ec56044bd7, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4137 +3, 2, 072fef44ac2f7667c366f067b374016bdd39581dbe8acb0f8aadf98502d13646, 01013064dacd09cf7f1898ac0b669d25f4c0db3c2bcea9b3c9cd51f3fc8a7414, 5a107488a0caf086ae94acfefca9b36f6ae84691b11a001ef3999a673d7b160b, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4394 +0, 3, 4657de22da43ad6402c8cfdd45ba54542213486481988bdd28c82182ec2714af, 0109151f9d11847474e333460ea73646a31663c84162313db3cea328a74efa82, c10184404849ef460dd734d49d22cad1b25aafa77bbd3f22073dc88885f644ec, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4450 +4, 3, e5e05648b1c70c4deea4c7d7aefccd49b24defadceb45dd033caf0f16407808b, 01044b182869be4ba5670092661abac68606a2f4b06130d101882d236e482358, b7fc5ca1cb869e33b3725a1f3a4c96e8545200d0f263d099351016b6baf46039, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4137 +14, 1, 8c2cce72fa295e618dada27cd9dce2a8f24a4596d676edf11e096bcf6f5c078f, 01076440117d3ff4d99c831ad310105cb1217c9bfe4bbd7a8d98df63fde9dac1, c01ee3f86edc2dbb38b5fcabf8dca2e5f63e80cfc7161b897783b71fb24eb07b, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 5808 +8, 3, e0b3b0f9b66fe475eaf17c6be38a2759ff254debd5485c4d4a27bc45486fa0ee, 01092583572dbd57774dc64402af3254c4e9cfc97a76b686d690e728f4946882, c1003cacbb633a682fa1587cc0a59535501e3d8511b896df7431c320da790724, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4294 +11, 3, 83b379182516db53587b3c553468e1af0ca4c21f13b95150024e68ebb2bb9550, 010522c21f13bb19c26a46e5798d2ce597bfd035261c4a3e9e2d38963910dba2, e692e800bb6e620f5108e70950e6f65c46997a3dd8aef402a30d44870ba94440, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4344 +12, 0, 2814b082c6683e9490338df3e42c8d89ca9e2f889735013dcbb7ddbdf7d846c4, 010618c41d7d80fdeb077f54165537de721e39ee32e5f8d11dd16e5edff70d91, 9c65303cf21df1f289cd5d7e1d431ddd79016d87c6e8ec57b2c23c15c69c119f, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4862 +1, 3, 17620b9920ef596dfa46753b355247b1641eaeda1246fa7bc43884630cf61bf2, 0108598ac27cff46dd5635d43a3ba7b32bf7c0f844c95418fb359f4bb8b14a48, cdb0470fe9f772124a6d158869b4bed2451b4445293ffa2cc56f96917b108ebc, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4663 +12, 1, 2814b082c6683e9490338df3e42c8d89ca9e2f889735013dcbb7ddbdf7d846c4, 010a33039aba1207b1d63d559fbe40220eaa30b56fd25e5d0767eed2538ed35c, 27ee5997236575efdb19c9845b330da3e969ab8b69af32b6cbace636afbca8cc, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4862 +13, 0, e36507d27632e2d6cc70480f6322a4469d20773ff0d047f6f70285e9d44beb5b, 010a602d4d374fe77a551445a50be97aa8ff00a8e6d1bc87eb6a05a7ca9d5bac, 5e65aac82ee69c59e00bd8bc6b688b45c6a30fc14a97c50f631f6255eaaf7327, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4916 +14, 2, 8c2cce72fa295e618dada27cd9dce2a8f24a4596d676edf11e096bcf6f5c078f, 010203df7f6c6ec32fb907ddd1bfe342d0e638248cf885e02534d40ee5b0fdc7, 7f25490bb25c2a15a4ae4b09834af66b32f6c747cd410f205ebb5f26157628b3, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 5475 +3, 3, 072fef44ac2f7667c366f067b374016bdd39581dbe8acb0f8aadf98502d13646, 01011fba8c525ade36b44b91c10f763cc452b9f359138a17b7095defba3de81f, ddbcfaeaeefae22924db28ad1ac6dbab0878009449fa125d1e072edfd29bfe29, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4337 +12, 2, 2814b082c6683e9490338df3e42c8d89ca9e2f889735013dcbb7ddbdf7d846c4, 01081ae266672ca194549f553909f744cc0043134958ecc17e1cb23b8b321b81, eabf4471ea17eeb0e6b8ef2868bdf533e538e9117d89ee7be0506b5b87100c31, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4862 +13, 1, e36507d27632e2d6cc70480f6322a4469d20773ff0d047f6f70285e9d44beb5b, 01061e74c28eab539b083cc334b3d8f49d500d3a0efaf0ddacfeac3c3d7db59c, a1f90ec26c1e922437b5dce2cb7d5aa74a73dd6b2a633e0aa1a87e181ef4e493, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4183 +12, 3, 2814b082c6683e9490338df3e42c8d89ca9e2f889735013dcbb7ddbdf7d846c4, 0107093cee10017f16cccbff4c1bf264ad41d8416654ce13f71d06d40a068d45, 4728ca3016635dbb5fed672ad264689e97f39372d87fdc8751d24d305147e50f, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4808 +15, 0, dac78edf70e526e26a94ac4982c2abded02fe049c5038196738d66d80849ca3d, 010a4622db8bdc6f2d2a53c4d093a7db2eed1511435c0c9d5f3ad99ccc89461f, 782f1d06c5a143065b84a674c8c957ec1f6c335b9b9d1818aa440358490f9d42, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 5165 +15, 1, dac78edf70e526e26a94ac4982c2abded02fe049c5038196738d66d80849ca3d, 010758cd460961f2865307025c35b5c371a152aa887aa79a5ed5db1562554b51, d949da8e7b84d066ea4e0d44064dfc4d8c58b4c4f6580a817de564a74500ce2e, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 5165 +14, 3, 8c2cce72fa295e618dada27cd9dce2a8f24a4596d676edf11e096bcf6f5c078f, 01051655e2f51a514371801ef25660a391477f0d6c492b613c5a83bb30f490f6, 66473bfc883091c7d6b1ddcb8753db715e7a9568aca4b2f870f155edf0b952b3, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4839 +15, 2, dac78edf70e526e26a94ac4982c2abded02fe049c5038196738d66d80849ca3d, 01034a2930d5d739d27f0a4b65bee1bd19ee6dccc2da5612aa3e380d60e263a3, 61cda7fa7dd9cb6b817139a51b7d17f3fef1ddb31db2a506082cd49411d0df57, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4541 +13, 2, e36507d27632e2d6cc70480f6322a4469d20773ff0d047f6f70285e9d44beb5b, 0102482e077b5bbcf0c1f5da9a77803663ade42ccf6667a2d07366ec25c05373, 7cdb704cdc5df918699d7bd399008f249f79ab8b4ca08689f8d3010c5813de38, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4183 +15, 3, dac78edf70e526e26a94ac4982c2abded02fe049c5038196738d66d80849ca3d, 0104501feb2927fd13760f5c39dc277827c125b069804c6ac18e6021b1811868, 61078520e97db6e828ce6465404fe3456dc27deb16516cfdc3c34c8293c52fd0, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4541 +13, 3, e36507d27632e2d6cc70480f6322a4469d20773ff0d047f6f70285e9d44beb5b, 01024f0896557a0084a1d9a2937130ccb2711a8cb810e512ec9e2c97c3c56db6, d4cbdaa2fb4e3c1e8ef809ed628f0abf16d3bf4a93f81ce03bf77b68e07fb516, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4183 +9, 0, a14d543d43592ce38ef38f600a2dba801557fd924638de966d2725dabc5e2472, 0102152a605e81991fe51e4c8964a98462f6a78ab03dfabfd30552d39e9d4eeb, c17c5c2242fd08302a00dddf39bfe32952a319f4816dbc540b991d8ee33180b4, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 5099 +9, 1, a14d543d43592ce38ef38f600a2dba801557fd924638de966d2725dabc5e2472, 01012c56d755fe8e1c3c18ce6c13e4fb74a62f4dbf6b4d88a4d4d5f12a82111f, aec69cdd848a1b7ff0f89c9ea97feb759270ed6719ec8a675ceba9cc0877a1de, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 5035 +9, 2, a14d543d43592ce38ef38f600a2dba801557fd924638de966d2725dabc5e2472, 010918e449d18c7ea382562025280bd33f3dea08e299eb92eaa7413d9b42c367, 1cb9b9ffb834abc7b8bbfeb01e006fd3cc412717f8b9456e5e4d663466417e99, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4281 +9, 3, a14d543d43592ce38ef38f600a2dba801557fd924638de966d2725dabc5e2472, 010a107f47f98b2357d358f56f2a9cb9759714f3bec8ae28dee41e9ce170a5db, c6d51cd991be84979a28aa4b4e51de3ef0f585aadbab7b66a0593805a5085546, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4132 +7, 0, 158b59e2f807c4b05160ad5dae50f96f079b79a7f7467e11d86c37a808515c1a, 010303ab82e286a8adbe84e8b7d72f1c627058d749eb7fee957227ae354736a9, e1b75477f812f034862401a8ecf7c9b7ffed63e904ebff99b4662bb3157d55f7, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4382 +7, 1, 158b59e2f807c4b05160ad5dae50f96f079b79a7f7467e11d86c37a808515c1a, 01061f0f60aa46bb3d3db810b004f0d63a0666f4304c7902c2d0de7116fd4c3a, a6453d96ed2963d2f8073faf29405223c3b9de7b9f0bf925ad5b5807f0df29ec, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4382 +7, 2, 158b59e2f807c4b05160ad5dae50f96f079b79a7f7467e11d86c37a808515c1a, 010a01e51b9910f599cf59ad2c563f0dedb61dbc3c0226bad673b48d1b538419, 1485b5332779be0d64bd08c69344580358b320cf01b9f38ce70badc069e94ef1, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4032 +7, 3, 158b59e2f807c4b05160ad5dae50f96f079b79a7f7467e11d86c37a808515c1a, 01050c4baf9613939235ebdaa17d71ffd8523d368247815fb47fa0fa6334e99b, 182c36a0398009d6b6e58301e13c805cd5acfc37412f55cadee180e90b27afdc, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4032 diff --git a/test/data/gt_production.csv b/test/data/gt_production.csv new file mode 100644 index 00000000..80569d03 --- /dev/null +++ b/test/data/gt_production.csv @@ -0,0 +1,97 @@ +pubkey, nonce, miningseed, score +039cc1f1560aa96daa994a2b296f22d7f2fc9503ce95321d0b8193079e5f93dc, bee1e55fc2c967601827dd80f09a6b0a47ceab37b920b57efbd23c8fdc49e38f, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4254 +9b3d041660f08f692ae1005118f85c35bec491c33b51c94ef126663f4402fb1e, 1a314eb596f55aa79e7f61334072380042ea4e351ca59dc09f536f8224a39afa, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4787 +5ddde3533e040840b737304dd92b4b48d62ab209419a8bf4506e7b0497e9c614, 419569517e0908f9bae5b07159d322204053afd4564b4079fe160f775e214993, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4329 +66950904cb50245726a50f9039e2d384bb878ce3e39e0575f64eb61bf9963325, 158191c4be8a6382d9eff13a448550c7742568909abc76d8213c958e85bb02f5, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4021 +beb0cbbaab3cc54e87e4b6ed4a5f6c4af2fa4f452a02252a8ec910c43fb71a43, 96725650ea0a2052e3175c0a1047f51fcc7031f1455fe168630f0ecbfce1a180, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4642 +c7f5024050c450389d341eabcea04b16e79b7a51f18bcb59284eaf40000cb7a6, 77b3965cec126453cfeb23b5b44f20d652058f7726f0e43e8c265eafba6e8418, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4138 +a6f49eb0c3fd08eeb56b4ab83bbfb0b415b02233101baff5eec26bb2d1455e34, ac8db67e608af98f13f70b611c71f527dbb626afe41128cc6dbb27da921a6db5, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4323 +5c49beb4e29c921d8fad67d9b1e09fd693254f9275dfd77ded5a66acf721b4fd, 5e80e9f095fadb379b12130c4796f74ec967fc526b1face4af813a13b029fcd3, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4596 +d8470f83e18ebd6bd221423f03b0e17a87872f6ef25ca8909f28078a6c300ff0, 7989ab5f03db1a93b51a8629ac6d569ff0299ee1744ddce813013e9a64ff6497, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4254 +4c83417da5f166accd29107292d5c0b7502811b4e0eef6d252c01474dce157bb, 59245fdd6fb52df651c4b3228596175e327aaaf4f4405cd37bf84ac1a9e51747, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4476 +aee5c8edcf3dfdab7ec84b7c580be8a64c0b902be6b365fe916733f05e238ac2, 56dddea0c9052587a47db7f5deaa16727663725edcbd0c5d78f9b1193bef12da, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4179 +f2112f60356004b9fe52840e06e9047d2020ba228b7e1987663491a7d90e3e85, 4893d1aeb7591ce464e26ed43e34501acf450a6d6ff3a4dbb85a3920d9011308, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4227 +050a8c2acdcc99c71b1f6e04f3785291f2980fcfbc0a2355a200eec5287a18d8, cd2744b32506fc199fb1aaf0b9edaa553d05f0f4c66e9f7d2e47140fbc7973e6, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4112 +5bb6da0fd6e14135dcaee492185a223c5495936b29cabecf415094e1cf0b64a4, 1b169a9f717bd61508b009a9e149c3bf3bf192acbc2a11af6da31214ca0c1be2, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4367 +ce103ac663da7295f0013f33b1eadb5295e32dbf8392f67b8085caca9d8291da, 44fdb9bc59d865bd9d87f049ea4a7a702d1a0fca20cddb7172bb10d7c17dcb06, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4373 +da39c1bc6effaa3edb88cc805e6e7bd7aad1726ade6b0dd38d54b52957f14125, 50ee325b50af29c82a0ac06b3ec85885f9e36e08f1342a938acfd9a002ce524e, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4538 +2f2a57841de709ea79ec678d1848a2dc13cb1d96d1d029278181e925ce6ce281, 29b9f9db6a5001480686637bb3f187868d23a8881b32482000a4e3b217074d54, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 5044 +7bf6c1e4d203b633f0f5f5421d2d16e66db924cccc51092f1e89ededa54ad322, 490f7927f8f91dbeab6434e4f0ca660642d9157267613ded72a8315df821d838, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4272 +a6f0ae22305a9a0336a2c7d97e1568697cdb62e1aedfc9874a1ac1915b4e3c67, 332f47bf272e90de5befc330c0ae65b88a981c6c28819c9df8bbc7fccd7176c8, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 5126 +1f710bfadc49f5a1bed75fb10054979a3b4856a48e3b75184f7b056d3a23ac41, 6d5adaad8e0ea9208390e5649d2e0c6365ee84cc1611ae5f765424d7992b8322, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4227 +36f621af4c726d3e5bc3cce3d26a4418c2272024fd8177110a6a64fcc9071afd, 6fa998c1ee8efba299b8b51e865a13b939df3189d94c0b939f86c8b2d9fb7893, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4002 +8117882a9cbfce4ca0df28f72a818773ba80377417e8c73d2c0a56e8c6e4ae22, c762e691c4918cc826e72b5277fa2c11f4ff9cb2a01efefabe6ab8f6c8c5be3e, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4124 +a7a69484aeb270fda2ce28d9b6d4ef6f1e73edde9adea5404f81b7ae265a24bc, 4b18c6bf59aeaf1f96a11956eba1c2ac7bbddbed8e0f12ec5a94dcc8107a5cb6, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4500 +a84e3c009575dd77299c18d600d0252f9d0480d34c5da189a7625a0274f83cc8, e0c96c11ba7a7137aa7fe213dd34dee838a598acf0d76776e5c3778307c1e631, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4334 +dc100c8d058c0cf6df88dbf4fe01f6023b173be3bedd6b86dee4b8eecc72e058, 179fdf1f611d6c27a08eb533b826d169a0bd3d33e6c329ded1eee76fcefcd2dd, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4259 +087fa2a7ada94299393d118b19c6360699845db46f559eba3ee5b8bc494f2a8f, 0f200d7799483ecf52a4af5c3a69111fd29a14bd7a03a82f9f39f1913b29e5f8, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4347 +8f9069a8eb962dc47afac9ea7d4a77d7ab801c3e932183f9dcda9c8cd7fdaf9b, ab4f54740589f62c4e35c2b7a2508a3f7fb23872afd62771e57deea64bc540d9, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4477 +ecbce54faf01fbf9015cf9b19be9bbf42e9530be81fe09fc92c666eace8bfc15, 86acf64f19dc27151febf20c2549d49b05f4dc87290e108e935ebd6d49950cee, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4513 +36e28fe48780115b126255d893208b16f04505687e9aafd17ba73dbe543a8319, ab908ba2a064b7381f26c7a02cfaa2d8b7689d5f24877aaad99e25e0714ca19b, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4711 +ee4fca45aa951e2a2c0e535fddda71cd1c0722f1b30bf74416d9ac6df8787e52, b53606d34971e204c260e1c5c2bcb88450f429b76340d0860046bf2a67cc359f, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4634 +25b3ace18ee066c5cb561d32bfc89f69f20199741519d148a1d00312191e1f0f, 06ab93f88f0fe2aaa54efe1abf60cceef2b21d15c378d66096398c8b82947c7d, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4032 +a7149ab5f6c9f3ec2dc997564839a050de78dcf54e255d0019b4d81c3675bb56, accab93bf12ddf0a8b2ee43bcb94fe01d265eacd6df6aa8bcf133a5d359a79f6, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4964 +d4aaeaf020007d1349590d77e2f779ad48f9a6cf720d04ca6a65fbbad81dc050, 7a40d7032c899ac35f224bd0a7be2c12244e9b2b4a32852b2ec880d526929ece, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 5357 +5fd98a26b2bcd498daef005d5035336777164fd594690d083ffc07c2750a00bf, f8c4c01377b01fb07d914d37301d2eee2349e015128adac766c676913f7c056e, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4538 +acc5d85d4bdb5d3e63879ee9975db8dd89474eaa143fd3a3b0dd9b406e6fb19a, 2097e387f6f27d6f630532f3407189adfebfae9bfe4308f0a4a718b72756cb64, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4251 +5329b924cde1d5be74ae7b98fd5dcd6c6cfc497abaabcfd13ba6ad76b3e80a73, 409da8d39041e54f20851bc0ae095cb8b80ba393af8531ea6f1977e14e900bca, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4435 +bd50121a1a9982b2e6f1becff5beced13d3aeb3b8292b6d64eaa641a85c6af36, e2c0de9cf3488fb4db674d9b6bdb7261044556a5ff21d66781c975316624ce8b, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4064 +f09fa2dd45236ebd36dfca4443787119dcefdd152723cf5f4346f16c33611ca4, ab4f459cae0455dcee65258b8c8340e3429cb43d826001a52542b0b10b2d3051, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4618 +5df6390b68e67e1cff09fea890e14b204f8b5e9ef847b14a05b7b32f83576c73, d2c779399ff772a44024e4991f32463193e7d21e958b926ead5186fad44d276e, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4658 +0c5c814529d514ed2084f87554a1278e303aa627c720c21502509e6453b46e75, 4d2317b4d7645a12954929dd48e1541ab6c333eeb590fed06bc795876d399b86, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4805 +1ec0460a6d6ecba87d61bde51c2b27c758cefecc568cd807b5d34c51d4d9f427, d0aaeea09c9dee00f7062fabcf47fd61f2c288b80295a820ec4222a18c2f67cd, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 5385 +e91795a4cc4ea7a134da695bd2ca1638e3510d2ed21a6207c34b25c920925556, 2cfdfbd4047379ca02f323115287c8132cbd1ab84ae94bb4142147ccfaabcde9, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4104 +547ec5ed8156d06f369b8642d81f542aeb06d8be0fddfb82d499722def67ab8d, 58315e20f5e05abcbb1c225435e985dc08edb01a51bb472e5a2caa1ebea54636, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4293 +d8e5cac99aa0e4d53d13a2a5efcb44533def7116cd8cab23b3e8ffe2e6e36033, db5d8b11d57d382bb4a174aeaeea1934089ca1ce87f695b02e969aa2d804323b, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4490 +11e6f5455923f425403174d26957f3c4343082676f2786a504b175a37e095257, 3d2b359ee811655af3afa43d4b6bcef022d9b086bf7b90196f8d2fd319cef2e9, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4101 +3c5904c5eef6ae6bd1c5f04b329bf8263f5de55ed27b416caec5d1c7ba8a1563, 4595621029e319dd6a712f40b4694bb41a198b97fcb5e1431e36b43d2a49f70c, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4229 +bd57f4064ecfd309f405a10536844d1ba5ee33ef367100f8a7bb8c0d30564c09, 3e3fe3eb8a8e57fd02bf0aed9362a8fa922792c23466f23910a07f5ecd6a5891, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4399 +8cd870e8a74f72c9293b449aff6f2703eee771a2e804ab3b0e85e90110a70ebe, e4043488f279cd9c732d5074e615a25c2684e84c82d86c742627ea7a5eacca33, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4427 +2aad58dee6a39af963379f65fe80379c9787fa0ef39ac6cfca83883b39d88f80, fe7024fbe4ed4989b6004ecc78ef92db203631b606d3bcc0266406bc2dc2c95d, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4211 +2944a74d098bc8cb2c49c7058fa5c1749b6b0762e7953d2909d081a621322bc9, a4251d12a25da7049ac54781cfcf729548fba470abd9aab5071854aa33651f48, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4392 +59423970ab4e01dcd75b21c9bb28039cc754cbd9504f1baf7690ad67f653d9bc, 8cb0ca3bc6616437eac5c040fcbc82de45a9374a686c68be73e1ece507fbe7a2, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4234 +4111344f7654a20f63f205251e233949ced281978307fe80f5994101ba30fff5, e0cb33733e52c5cfe2f342b834e6be21a4c8ba142184a5a712fbe01d2f321ca0, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4192 +cb2e9d7b14e92a7acd3606295181a85b3a3834812c85a58c5d8e8fb966d52e80, ffe82f2f47efff7e3c7c1f8489126be98246f789a0d5cff4e62e251d7e9637b5, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4246 +6acfa4e805379c4ad2c8fed920eb94bea5c66753f76c9e8903cb9f061b9a8568, 3291da84832c5e9fd2ec14ab4bc3852a9f501635962490595f92fccdb66104aa, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4277 +5950a4ccc1ee66af76e4ddec06caa7a0992cb2f59b7fe0468edd64f7bf7ac3e5, 8f6afeae5b25d78e7a9105fe200525e0012cd20ccf172dd0f610dbf804c1d731, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4140 +971e704c6ef92be3bedbdf77b8829804ee21011c6a2a3e79a68b1d4ad1cf826f, 19be39d709b2b1cd553f4ade9553d982a5ebc2028c7d61fa2693ce7c701ccf2b, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4122 +95ebe78a1bec9a12aa57462a7818c047b9cae0376666d0640905e81ff6e9e356, 7a418455ac94a63d6726f6fdecbf32d9426dd990ba0548e31e710703639980f2, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4418 +8782eea2fe0934ef90467ece2754218f16e02f731277ec7f244217f0ee0bb764, 7904e219e0265db109364c873d81f45e7e7522e881f9646faeba35fcc49985af, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4189 +7097a76021713afceeaa610bb79cffa218a8f6235ea1aefe078e957210170a47, 4068216c2a24e21043646f0f6b411628892c9350517451f528e632dc757ddabd, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4174 +e2c85076c6767b991f71921fa6e3bdd669a3e1e739385b4f0330250c828bf8ec, 017471a30b0129216db7a28a5da7dd2b16c4bc856c85d5135985119e4a99e6d3, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4336 +983e19c755f902f0026754de7fc89dbfba6ccc17f161a6f69c94e27ae5f3b8f4, 63301609c9ae6610af2aa1282c70531380a6d90e2b87d36ced6a84108f64755f, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4686 +66f9328e9c1a79595d52798a25d3459552afa6af9aa52ceb7ec76f38a8f3bd39, 0b71ad39ef27577b06bceab5f264201b2e8087143e0df90c69b0e206042470f1, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4307 +befe30abf1c5e24965d39b7ff3f2fc33fc4698be8a33c87aa4f8536ddb8a1996, ad87664522487ee7986f5ce7c26ff962b5feba171551b15b047d9a7053dedca5, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4235 +0363966c6d6ea832ddb1df165841a3479df9ec8d2ce0cea8a9400360f3cecc34, 391a80f81e72894605743026c249995103834322ce94fd953107a781583c2f23, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4472 +2cfd43630593e07902ff52948ce387aaf0e2bb0e11952f8a0c91a33672203bcc, 157ebdf68c866bef7f93adafa1490ce1c15c3b951f23461f5327b53b3546726a, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4874 +f99a06858d3d9dd741b0d1ce08556aab5c4b383b8199920d776daea35634b1b6, 494feb7026395e0ce15b603a2e424b79cea1d46da11fead0d84c29cae57299a5, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4657 +09bf69a84c357ff346577e5d17748c2b3d5b3ed84e9329759b131c2b94e7f4b5, 108075e0dfc82821119e0f323ab789ef809dc39b203db23ee3242f3a42ec0a80, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4728 +7ba937604ac1bdd1cdda1954b6b6a93eda265f468a49f4941f8de0980caed213, eaf8c0959fdd124b1756144818ba32a60f4fe12454bc342e25a74b13b745241a, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4274 +060f7e1a8c8cb6f938b3116ff25884b6eb87fafe9ec957d07ca28ee58fea68ee, fdead1f49e206d1892dc77b38a8b0fc8a286d01683ee994f7fe591c6f66b2bdd, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4212 +377741f3abd82ac5b90df9edf3d59ca9ab1fea5376dadb40242c87f55a6c287a, 9669b5d4bf3ef9184daac77ab44f0e3ce4e46752e2a7fd68ebecfbcf6109fe3a, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 5152 +7a4fcbb032e872b47f2a33fbb2d27fa06965165a91ce284a8d29a43bb4d118aa, edd6bd4e7bff26c7e441011422fcfe753bded55005a5221046884efb2afafaa9, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4144 +6ceb9db060f76ff7a170037cf3fb0413f3bcb5b2cbdf7f42106d30fe6516be95, 9c83bb2e063e73e7338944f8babdfd18d01ed1d13c9baa43fa7489fb52f176c2, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 3956 +43e55c8788cad0c02cdf142a7f7e4683b332ed6a8efcc831c580aab3e9a1b27e, d9990b01cbe2746aafedc17d14f5a84675938f952eaeee0b3b5764e4bf054074, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4236 +cd0e79b31ed4e699811ca0fcbaf81cb3b0e08188acfc38b6a98e3f6d077e6fe7, 3625fb00a429fd89c4ce1c4f92303cececfbccfd7bc10b1f00d98e8e8540fbd6, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4421 +61217450605f85dfd56bb5c1c046db8018bec5c533f3d5dd92732ffdf8786e60, a165d267e0fa10de03d6149b6853930ba3f9a244f3dd07a66b202f064535dc6a, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4330 +c7f29ca4e434a0283049a26b8dba34072bf7a90e2116a5374171b0137789fa34, cce63e8a939e4dbbcff17212af5f390f7b5d5c7ac7c1c4bdc9db353b324b092c, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 5074 +19c0127c42618672cda53ef8998cf7a79bce5eda48aba533c4c157db73bf1430, 63f09dee35a7ad227266f20295f406311a3563765e51dcab2135f589127c5750, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4916 +93970bbe1875f9bd9766a225f95aa3d3e748ae4ff7a629e8d8067582ae6874be, 4e9fc8003d5384be5dde14ed04c9bb256dfc34882ae7c51873a74aba28e300c5, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4419 +6b17204cc22a12de5171f59978a5598908a3778e9d635e925190e312f27a5104, eb631efad97aa19034488b9b19fd26ac670fb584ed0f7eeb0bff2d4f1e29f0c4, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4692 +d0d2996accf533fac438f273ab65edc50eb5a756a727fa9c51308f8b8608583a, fd0ad8a263a4cae7cc1cba87fa8c5af6b0c74694cf6dcc9c56e4867d1467b53e, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4544 +f075dc821b7f48bb638dd898b9c1336abe8ac4b8ba5c964b5900c8b26fb89223, 88ed66223b41c277d464e33e1eca9e022df91b476a316050c9fe296c2d27a55a, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4127 +c14ff23a8dd058c87e49edb5b6e0433e24238911ec1c7c179cd8338ad3f1eea5, c35e26a31a309a160cd348f3540aef3277f3ca07eb869c59634b6986f44ae8d0, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4488 +7baf3e8717e443d1f3a2459c6e4368b79a6233f75e236aa6367b5eaaa2deb8cc, 853d8bc62f96344aed6f69a683ff7bb15f82839fecadbbad8b2eeda3713b73d6, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 5043 +aaab6229bd572c77a7a89b3b9a7bd17cfd085528ac5471fc358b1a4c6d011c4a, 1276013c419de03614ac9da61dc28c732b5d706529e5492caab405db613d6340, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4216 +f73ab8675d3eef76ffd330ad2ef8c2dbdb2f5fe8e76bb7c5458639f2e2cdb7b4, 5c3dba3a97e7cf0d9b92e5cbb8228d5b6ea3f9dc54fba6c288e6181b03ba369f, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4014 +adede9708eecf496012fb63c6dba5926c896d077ff1bd2e8a911b4eced495d82, 66cbfcfbd434f0a4c066c6e42c54c8a5ff544eb81834c14781850421d3b87d28, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4549 +0c5a86ecee90b2275137aead69f1005ef9f1edc420845fa667ea020f98c8d064, b033d35f89a05b58388d4cfd3f2cb855ae4a3925353926b6f267a06d63fc9420, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4144 +f2ce52d93723e7c386d4604876a3c3ae7a06a295411ea2e3719f0d5b5e2a5e44, f3b1935402df86ee59dd2913a11fb327451542645a62bb8e5217a459e242e820, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4309 +f067e50164c5eab1194c3b77cc119365c01ec9501a7adf5becc7804c46695dff, f342f24d14e52fb7441f409386ab457e353caa06ee395beaa25e27be5e1a7c7b, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4517 +7a33c852aef3009f1a8096124c4e166115fd14c1619bf8a1275b94292787c274, 1f704e695d2b807cd9b3f877ab12fa6d5cabe77127a589fb2ae6df94470e7717, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4352 +34cd3c37f0cbda82af0b62b0b3c75887a1332c5a855218c75fe47e2d6bc6aeec, 35df8cd36c809e50e74990932a316fe5ecefcbb19fe3f4a8d027fff912202998, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4042 +dc679985b8b7d22ca9dcf52c947f826d9cf6f36582b70f7282ac1b67a0c46ae8, 80c2186977f4180ad43db66bb7d5596d7e42410dbaf7e84734b71d589f46724e, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4054 +17f3b29ae0d930361208d16169894ee4ed9b5846bd01147d0c7bcf05ae7d25e4, 5ef965812a3918a2dec24cedfb23a306a0305e40e52d9b38ff00b2a0d2ebe616, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4480 +836eb6226f84ecc4a270ea559f2b3418cdc0bb9eb66a26bcceaa7e45c802cdee, 9a7512dc2a55c148ee1900e85f9bec223bdd73ba25b650873433164d196f2349, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4849 +6e8090bf760bbd8dd5146ebe3420223b8be6b30b65e714220c27d1c2ab62423e, 29ad5c73e1f39167a370020bc2ed3af84e640bfcde167a6981f0114723fd7ac9, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4296 +9c256502b40844cd9f2b11ed9eb5472441731b66797542837dbc68ee2524cb98, 5627705fd251600ddff839d02077e646f85dbb08fc08947e9098d0e2c289022a, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4454 diff --git a/test/score.cpp b/test/score.cpp index 67cb0e4d..519210be 100644 --- a/test/score.cpp +++ b/test/score.cpp @@ -9,6 +9,7 @@ #include "../src/public_settings.h" #include "../src/mining/score_bpp9000.h" #include "../src/mining/task_file.h" +#include "../src/score.h" #include "score_bpp9000_reference.h" #include "score_params.h" @@ -25,7 +26,9 @@ #include #include #include +#include #include +#include using namespace score_params; using namespace test_utils; @@ -34,13 +37,17 @@ static const std::string TASK_FILE_NAME = "data/example_task_bpp9000.bin"; static const std::string SAMPLES_FILE_NAME = "data/samples_bpp9000.csv"; static const std::string SCORES_FILE_NAME = "data/scores_bpp9000.csv"; +static const std::string PRODUCTION_TASK_FILE_NAME = "data/bpp9000.task"; +static const std::string PRODUCTION_FILE_NAME = "data/gt_production.csv"; +static const std::string PRODUCTION_ANT_FILE_NAME = "data/gt_ant_production.csv"; + // true = ALSO run the engine-vs-reference cross-check on random tasks, for isolating a divergence. static bool gCompareReference = false; // Samples run per config static constexpr unsigned long long TEST_NUMBER_OF_SAMPLES = 32; -// Worker threads for the parallel path; 0 uses hardware_concurrency. -static constexpr unsigned int TEST_NUMBER_OF_THREADS = 0; +// Worker threads for the parallel path; min with hardware_concurrency and numSamples. 0 falls back to 1 (serial). +static constexpr unsigned int TEST_NUMBER_OF_THREADS = 4; // Samples and worker threads for the Bpp9000Profile timing run. static constexpr unsigned long long PROFILING_NUMBER_OF_SAMPLES = 48; @@ -198,8 +205,9 @@ static unsigned int workerThreadCount(unsigned long long numSamples) { hw = 1; } - const unsigned int chosen = TEST_NUMBER_OF_THREADS == 0 ? hw : std::min(hw, TEST_NUMBER_OF_THREADS); - return std::max(1u, (unsigned int)std::min(numSamples, (unsigned long long)chosen)); + unsigned int requested = (TEST_NUMBER_OF_THREADS == 0) ? 1u : TEST_NUMBER_OF_THREADS; // 0 falls back to 1 (serial) + unsigned int chosen = (hw < requested) ? hw : requested; // min(hardware_concurrency, requested) + return (unsigned int)((numSamples < (unsigned long long)chosen) ? numSamples : (unsigned long long)chosen); // no more than one thread per sample } template @@ -380,6 +388,179 @@ TEST(TestQubicScoreFunction, Bpp9000Regression) runRegression(seeds, pubkeys, nonces, taskBytes, golden); } +TEST(TestQubicScoreFunction, Bpp9000ProductionRegression) +{ + auto rows = readCSV(PRODUCTION_FILE_NAME); + ASSERT_GT(rows.size(), 1u) << "missing/empty " << PRODUCTION_FILE_NAME; + + std::vector pubkeys; + std::vector nonces; + std::vector golden; + std::vector uniqueSeeds; // distinct mining seeds -> one pool each + std::vector poolIndex; // per row: index into uniqueSeeds + for (unsigned long long i = 1; i < rows.size(); ++i) + { + pubkeys.push_back(hexTo32Bytes(trim(rows[i][0]), 32)); + nonces.push_back(hexTo32Bytes(trim(rows[i][1]), 32)); + const m256i seed = hexTo32Bytes(trim(rows[i][2]), 32); + golden.push_back((unsigned int)std::stoul(trim(rows[i][3]))); + + unsigned int idx = (unsigned int)uniqueSeeds.size(); + for (unsigned int k = 0; k < uniqueSeeds.size(); ++k) + { + if (memcmp(uniqueSeeds[k].m256i_u8, seed.m256i_u8, 32) == 0) + { + idx = k; + break; + } + } + if (idx == uniqueSeeds.size()) + { + uniqueSeeds.push_back(seed); + } + poolIndex.push_back(idx); + } + ASSERT_FALSE(pubkeys.empty()); + + std::vector> pools(uniqueSeeds.size()); + for (size_t k = 0; k < uniqueSeeds.size(); ++k) + { + generatePool(uniqueSeeds[k], pools[k]); + } + + // The production task + auto taskBytes = readBinaryFile(PRODUCTION_TASK_FILE_NAME); + ASSERT_GT(taskBytes.size(), sizeof(score_task_file::TaskFileHeader)) << "missing/short " << PRODUCTION_TASK_FILE_NAME; + const TaskBlocks tb = taskSubview(taskBytes); + + runWorkers(workerThreadCount(pubkeys.size()), [&](unsigned int threadIdx, unsigned int numThreads) + { + auto engine = makeEngine(tb.topo, tb.data); + if (!engine) + { + return; + } + for (unsigned long long s = threadIdx; s < pubkeys.size(); s += numThreads) + { + const unsigned int score = engine->computeScore(pubkeys[s].m256i_u8, nonces[s].m256i_u8, pools[poolIndex[s]].data()); + EXPECT_EQ(score, golden[s]) << "gt_production row " << s; + } + }); +} + +// Ant-colony score seam (deriveRootANN + computeScoreFromParent) +TEST(TestQubicScoreFunction, Bpp9000AntColonyRegression) +{ + // Group by chain each chain is a lineage - level 0 extends the derived root, level i extends level i-1's bestANN. + auto rows = readCSV(PRODUCTION_ANT_FILE_NAME); + ASSERT_GT(rows.size(), 1u) << "missing/empty " << PRODUCTION_ANT_FILE_NAME; + + struct AntNode + { + m256i nonce; + m256i anchor; + unsigned int score; + }; + struct AntChain + { + m256i pubkey; + unsigned int poolIndex; + std::vector nodes; // indexed by depth + }; + std::vector chains; + std::vector chainIds; // chain id per slot, first-seen order + std::vector uniqueSeeds; + + for (unsigned long long i = 1; i < rows.size(); ++i) + { + const int chainId = std::stoi(trim(rows[i][0])); + const int depth = std::stoi(trim(rows[i][1])); + const m256i pubkey = hexTo32Bytes(trim(rows[i][2]), 32); + const m256i seed = hexTo32Bytes(trim(rows[i][5]), 32); + + unsigned int sidx = (unsigned int)uniqueSeeds.size(); + for (unsigned int k = 0; k < uniqueSeeds.size(); ++k) + { + if (memcmp(uniqueSeeds[k].m256i_u8, seed.m256i_u8, 32) == 0) + { + sidx = k; + break; + } + } + if (sidx == uniqueSeeds.size()) + { + uniqueSeeds.push_back(seed); + } + + size_t cidx = chains.size(); + for (size_t k = 0; k < chainIds.size(); ++k) + { + if (chainIds[k] == chainId) + { + cidx = k; + break; + } + } + if (cidx == chains.size()) + { + chainIds.push_back(chainId); + AntChain created; + created.pubkey = pubkey; + created.poolIndex = sidx; + chains.push_back(created); + } + + AntNode node; + node.nonce = hexTo32Bytes(trim(rows[i][3]), 32); + node.anchor = hexTo32Bytes(trim(rows[i][4]), 32); + node.score = (unsigned int)std::stoul(trim(rows[i][6])); + + AntChain& chain = chains[cidx]; + if ((size_t)depth >= chain.nodes.size()) + { + chain.nodes.resize((size_t)depth + 1); + } + chain.nodes[(size_t)depth] = node; + } + ASSERT_FALSE(chains.empty()); + + std::vector> pools(uniqueSeeds.size()); + for (size_t k = 0; k < uniqueSeeds.size(); ++k) + { + generatePool(uniqueSeeds[k], pools[k]); + } + + auto taskBytes = readBinaryFile(PRODUCTION_TASK_FILE_NAME); + ASSERT_GT(taskBytes.size(), sizeof(score_task_file::TaskFileHeader)) << "missing/short " << PRODUCTION_TASK_FILE_NAME; + const TaskBlocks tb = taskSubview(taskBytes); + + // Thread across chains; a chain is sequential (each node's bestANN feeds the next depth's parent). + runWorkers(workerThreadCount(chains.size()), [&](unsigned int threadIdx, unsigned int numThreads) + { + auto engine = makeEngine(tb.topo, tb.data); + if (!engine) + { + return; + } + for (size_t ci = threadIdx; ci < chains.size(); ci += numThreads) + { + const AntChain& chain = chains[ci]; + const unsigned char* pool = pools[chain.poolIndex].data(); + + score_engine::ScoreBpp9000::ANN parent; + engine->deriveRootANN(chain.pubkey.m256i_u8, pool, parent); // depth 0's parent = the derived root + for (size_t d = 0; d < chain.nodes.size(); ++d) + { + const AntNode& node = chain.nodes[d]; + const unsigned int score = engine->computeScoreFromParent( + parent, chain.pubkey.m256i_u8, node.nonce.m256i_u8, node.anchor.m256i_u8, pool); + EXPECT_EQ(score, node.score) << "gt_ant chain " << ci << " depth " << d; + engine->getBestANN(parent); // this node becomes the next depth's parent + } + } + }); +} + // TestBpp9000, internal score vs the score reference from Qiner TEST(TestQubicScoreFunction, Bpp9000EngineVsReference) { @@ -507,3 +688,679 @@ TEST(TestQubicScoreFunction, Bpp9000Profile) runBpp9000Profile(); } #endif + +// ============================================================================= +// Ant-colony related + +namespace +{ +using AntCfg = ProductionConfig; +using AntEngine = score_engine::ScoreBpp9000; + +// Pool + synthetic task + loaded engine. Every ant test starts from one of these. +template +struct AntFixtureT +{ + std::vector pool; + std::vector taskBytes; + std::unique_ptr> engine; +}; +using AntFixture = AntFixtureT; + +template +static bool makeAntFixtureT(AntFixtureT& f) +{ + std::vector seeds; + std::vector pubkeys; + std::vector nonces; + loadSamples(seeds, pubkeys, nonces, 1); + if (seeds.empty()) + { + ADD_FAILURE() << "missing/short " << SAMPLES_FILE_NAME; + return false; + } + generatePool(seeds[0], f.pool); + + f.taskBytes = readBinaryFile(TASK_FILE_NAME); + if (f.taskBytes.size() <= sizeof(score_task_file::TaskFileHeader)) + { + ADD_FAILURE() << "missing/short " << TASK_FILE_NAME; + return false; + } + const TaskBlocks tb = taskSubview(f.taskBytes); + f.engine = makeEngine(tb.topo, tb.data); + return f.engine != nullptr; +} + +static bool makeAntFixture(AntFixture& f) +{ + return makeAntFixtureT(f); +} + +static m256i makePubkey(unsigned char tag) +{ + m256i k = m256i::zero(); + k.m256i_u8[0] = tag; + k.m256i_u8[31] = (unsigned char)(tag * 3 + 1); + return k; +} + +// Canonical ant nonce: nonce[0] selects bpp9000, nonce[1] = L, nonce[2] = K, rest is the walk seed. +static m256i makeAntNonce(unsigned char L, unsigned char K, unsigned char tag) +{ + m256i n = m256i::zero(); + n.m256i_u8[0] = (unsigned char)score_engine::AlgoType::Bpp9000; + n.m256i_u8[1] = L; + n.m256i_u8[2] = K; + n.m256i_u8[3] = tag; + n.m256i_u8[17] = (unsigned char)(tag ^ 0x5A); + return n; +} + +// Find a nonce whose walk from this parent actually improves on the parent's score. Needed only by +// tests that compare two children of the SAME parent +static bool findImprovingNonce(AntEngine& engine, const AntEngine::ANN& parent, const m256i& pk, + const m256i& anchor, const unsigned char* pool, m256i& outNonce) +{ + for (unsigned char tag = 1; tag <= 6; ++tag) + { + const m256i n = makeAntNonce(6, 5, (unsigned char)(50 + tag)); + const unsigned int sc = engine.computeScoreFromParent(parent, pk.m256i_u8, n.m256i_u8, + anchor.m256i_u8, pool); + if (sc != score_engine::INVALID_SCORE_VALUE) + { + outNonce = n; + return true; + } + } + return false; +} +} + +// Make sure the score at the full computeScore flow have the same score with +// the engine start directly from the best/final LUT +TEST(TestQubicScoreAntColony, BestAnnReproducesReturnedScore) +{ + AntFixture f; + ASSERT_TRUE(makeAntFixture(f)); + + const m256i pk = makePubkey(1); + const m256i nonce = makeAntNonce(3, 0, 11); + + const unsigned int best = f.engine->computeScore(pk.m256i_u8, nonce.m256i_u8, f.pool.data()); + // Re-score the LUT the walk kept, taken out and put back through the public form - this also + // exercises the compact/expand round trip the tree relies on. + AntEngine::ANN bestLut; + f.engine->getBestANN(bestLut); + f.engine->expand(bestLut, f.engine->currentANN); + EXPECT_EQ(f.engine->score(), best); +} + +// Make sure the score at the full computeScoreFromParent flow have the same score with +// the engine start directly from the best/final LUT +TEST(TestQubicScoreAntColony, BestAnnReproducesScoreFromParent) +{ + AntFixture f; + ASSERT_TRUE(makeAntFixture(f)); + + const m256i pk = makePubkey(2); + const m256i nonce = makeAntNonce(4, 2, 23); + const m256i anchor = makePubkey(9); + + AntEngine::ANN root; + f.engine->deriveRootANN(pk.m256i_u8, f.pool.data(), root); + + const unsigned int childScore = f.engine->computeScoreFromParent( + root, pk.m256i_u8, nonce.m256i_u8, anchor.m256i_u8, f.pool.data()); + + // Re-score the LUT the walk kept, taken out and put back through the public form - this also + // exercises the compact/expand round trip the tree relies on. + AntEngine::ANN bestLut; + f.engine->getBestANN(bestLut); + f.engine->expand(bestLut, f.engine->currentANN); + EXPECT_EQ(f.engine->score(), childScore); +} + +// An ANN is exactly its LUT: no storage padding escapes the engine, so hashing or shipping one is +// just sizeof(ANN) and a future change to lutStride cannot alter a digest or the wire format. +TEST(TestQubicScoreAntColony, AnnCarriesOnlyTheLut) +{ + static_assert(sizeof(AntEngine::ANN) == AntCfg::populationThreshold * AntEngine::lutSize, + "ANN must be the LUT and nothing else"); + static_assert(sizeof(AntEngine::ANN) < sizeof(AntEngine::PaddedLut), + "the working layout is the padded one, not the other way round"); + + AntFixture f; + ASSERT_TRUE(makeAntFixture(f)); + + const m256i pk = makePubkey(3); + AntEngine::ANN root; + f.engine->deriveRootANN(pk.m256i_u8, f.pool.data(), root); + + // Every byte handed out is a trit; nothing from the padded rows leaked in. + for (unsigned long long i = 0; i < sizeof(root.lut); ++i) + { + ASSERT_LT(root.lut[i], 3) << "byte " << i << " of the returned ANN is not a trit"; + } + + // Scribbling on the working layout's padding cannot change what comes out of it. + AntEngine::PaddedLut working; + f.engine->expand(root, working); + for (unsigned long long k = 0; k < AntEngine::maxNumberOfNeurons; ++k) + { + for (unsigned long long b = AntEngine::lutSize; b < AntEngine::lutStride; ++b) + { + working.lut[k * AntEngine::lutStride + b] = (unsigned char)(0xA5 + k + b); + } + } + AntEngine::ANN again; + f.engine->compact(working, again); + EXPECT_EQ(memcmp(&again, &root, sizeof(root)), 0) << "storage padding reached the ANN"; +} + +// expand/compact must be lossless, since every parent read from the tree goes through expand and +// every child written back goes through compact. +TEST(TestQubicScoreAntColony, AnnSurvivesExpandAndCompact) +{ + AntFixture f; + ASSERT_TRUE(makeAntFixture(f)); + + AntEngine::ANN original; + f.engine->deriveRootANN(makePubkey(44).m256i_u8, f.pool.data(), original); + + AntEngine::PaddedLut working; + f.engine->expand(original, working); + AntEngine::ANN restored; + f.engine->compact(working, restored); + + EXPECT_EQ(memcmp(&restored, &original, sizeof(original)), 0) << "expand/compact is not lossless"; +} + +TEST(TestQubicScoreAntColony, RootAnnIsDeterministicAndPerIdentity) +{ + AntFixture f; + ASSERT_TRUE(makeAntFixture(f)); + + const m256i pkA = makePubkey(4); + const m256i pkB = makePubkey(5); + + AntEngine::ANN a1; + AntEngine::ANN a2; + AntEngine::ANN b1; + + f.engine->deriveRootANN(pkA.m256i_u8, f.pool.data(), a1); + // Deriving another identity's root overwrites initValue.lutInit, which is the state a1 came from. + f.engine->deriveRootANN(pkB.m256i_u8, f.pool.data(), b1); + f.engine->deriveRootANN(pkA.m256i_u8, f.pool.data(), a2); + + EXPECT_EQ(memcmp(&a1, &a2, sizeof(a1)), 0) << "root depends on engine state"; + EXPECT_NE(memcmp(&a1, &b1, sizeof(a1)), 0) << "two identities share a root"; +} + +// A child is a function of (parent, pubkey, nonce, anchor). Same inputs must give the same score AND +// the same inherited LUT; a different parent must not give the same child. +TEST(TestQubicScoreAntColony, ChildIsDeterministicAndInheritsParent) +{ + AntFixture f; + ASSERT_TRUE(makeAntFixture(f)); + + const m256i pk = makePubkey(6); + const m256i nonce = makeAntNonce(5, 3, 41); + const m256i anchor = makePubkey(12); + + AntEngine::ANN parentA; + AntEngine::ANN parentB; + f.engine->deriveRootANN(pk.m256i_u8, f.pool.data(), parentA); + f.engine->deriveRootANN(makePubkey(7).m256i_u8, f.pool.data(), parentB); + + const unsigned int s1 = f.engine->computeScoreFromParent( + parentA, pk.m256i_u8, nonce.m256i_u8, anchor.m256i_u8, f.pool.data()); + AntEngine::ANN child1; + f.engine->getBestANN(child1); + + const unsigned int s2 = f.engine->computeScoreFromParent( + parentA, pk.m256i_u8, nonce.m256i_u8, anchor.m256i_u8, f.pool.data()); + + EXPECT_EQ(s1, s2) << "same inputs gave different scores"; + AntEngine::ANN child2; + f.engine->getBestANN(child2); + EXPECT_EQ(memcmp(&child1, &child2, sizeof(child1)), 0) << "same inputs gave a different child LUT"; + + f.engine->computeScoreFromParent(parentB, pk.m256i_u8, nonce.m256i_u8, anchor.m256i_u8, f.pool.data()); + AntEngine::ANN child3; + f.engine->getBestANN(child3); + EXPECT_NE(memcmp(&child1, &child3, sizeof(child1)), 0) << "the parent LUT was not inherited"; +} + +// The anchor digest is part of the child's walk seed, so the same nonce on the same parent must not +// produce the same child at a different anchor. +TEST(TestQubicScoreAntColony, ChildDependsOnAnchorDigest) +{ + AntFixture f; + ASSERT_TRUE(makeAntFixture(f)); + + const m256i pk = makePubkey(8); + const m256i anchorA = makePubkey(20); + const m256i anchorB = makePubkey(21); + + AntEngine::ANN parent; + f.engine->deriveRootANN(pk.m256i_u8, f.pool.data(), parent); + + m256i nonce; + ASSERT_TRUE(findImprovingNonce(*f.engine, parent, pk, anchorA, f.pool.data(), nonce)) + << "no nonce improved on this parent, so bestANN would not move and the comparison below " + "would be vacuous"; + + f.engine->computeScoreFromParent(parent, pk.m256i_u8, nonce.m256i_u8, anchorA.m256i_u8, f.pool.data()); + AntEngine::ANN c1; + f.engine->getBestANN(c1); + + f.engine->computeScoreFromParent(parent, pk.m256i_u8, nonce.m256i_u8, anchorB.m256i_u8, f.pool.data()); + AntEngine::ANN c2; + f.engine->getBestANN(c2); + + EXPECT_NE(memcmp(&c1, &c2, sizeof(c1)), 0) << "anchor digest does not reach the walk"; +} + +// Non-canonical nonces are refused by the scorer itself, so no caller can score first and check after. +TEST(TestQubicScoreAntColony, NonCanonicalNonceIsRejected) +{ + AntFixture f; + ASSERT_TRUE(makeAntFixture(f)); + + const m256i pk = makePubkey(10); + const m256i anchor = makePubkey(30); + AntEngine::ANN parentA; + AntEngine::ANN parentB; + f.engine->deriveRootANN(pk.m256i_u8, f.pool.data(), parentA); + f.engine->deriveRootANN(makePubkey(11).m256i_u8, f.pool.data(), parentB); + + constexpr unsigned char maxK = (unsigned char)AntCfg::numberOfMutations; + const m256i good = makeAntNonce(3, 5, 62); + + // A rejected nonce and a timed-out walk + f.engine->computeScoreFromParent(parentA, pk.m256i_u8, good.m256i_u8, anchor.m256i_u8, f.pool.data()); + AntEngine::ANN afterA; + f.engine->getBestANN(afterA); + + // L below range, L above range, K above numberOfMutations, wrong algorithm slot. + static constexpr unsigned int numberOfBadNonces = 4; + m256i bad[numberOfBadNonces]; + bad[0] = makeAntNonce(0, 0, 64); + bad[1] = makeAntNonce((unsigned char)(score_engine::MAX_LUT_ENTRIES_PER_STEP + 1), 0, 65); + bad[2] = makeAntNonce(3, (unsigned char)(maxK + 1), 66); + bad[3] = makeAntNonce(3, 0, 67); + bad[3].m256i_u8[0] = (unsigned char)score_engine::AlgoType::Neuraxon; + + for (unsigned int i = 0; i < numberOfBadNonces; i++) + { + // The bad nonce is early rejected in computeScoreFromParent() + EXPECT_EQ(f.engine->computeScoreFromParent(parentB, pk.m256i_u8, bad[i].m256i_u8, anchor.m256i_u8, f.pool.data()), + score_engine::INVALID_SCORE_VALUE) << "non-canonical nonce " << i << " accepted"; + + AntEngine::ANN now; + f.engine->getBestANN(now); + EXPECT_EQ(memcmp(&now, &afterA, sizeof(now)), 0) << "rejected nonce " << i << " still ran the walk"; + } + + // Now after bad nonce, we feed good nonce, we expect this is ok + f.engine->computeScoreFromParent(parentB, pk.m256i_u8, good.m256i_u8, anchor.m256i_u8, f.pool.data()); + AntEngine::ANN afterB; + f.engine->getBestANN(afterB); + EXPECT_NE(memcmp(&afterB, &afterA, sizeof(afterB)), 0) << "canonical nonce was not scored"; +} + + +// L and K boundaries of the canonical rule, checked as a pure predicate so no walk is needed. +TEST(TestQubicScoreAntColony, NonceCanonicalRuleBoundaries) +{ + using AntScorer = score_engine::ScoreBpp9000; + constexpr unsigned char maxL = (unsigned char)score_engine::MAX_LUT_ENTRIES_PER_STEP; + constexpr unsigned char maxK = (unsigned char)AntScorer::numberOfMutations; + + EXPECT_TRUE(AntScorer::isCanonicalAntNonce(makeAntNonce(1, 0, 70).m256i_u8)); + EXPECT_TRUE(AntScorer::isCanonicalAntNonce(makeAntNonce(maxL, 0, 71).m256i_u8)); + EXPECT_TRUE(AntScorer::isCanonicalAntNonce(makeAntNonce(3, maxK, 72).m256i_u8)); + + EXPECT_FALSE(AntScorer::isCanonicalAntNonce(makeAntNonce(0, 0, 73).m256i_u8)); + EXPECT_FALSE(AntScorer::isCanonicalAntNonce(makeAntNonce((unsigned char)(maxL + 1), 0, 74).m256i_u8)); + EXPECT_FALSE(AntScorer::isCanonicalAntNonce(makeAntNonce(3, (unsigned char)(maxK + 1), 75).m256i_u8)); +} + + +// --------------------------------------------------------------------------- +// ScoreFunction task queue. + +typedef ScoreFunction<1> TaskQueueScoreFunction; + +static constexpr unsigned int TASK_QUEUE_PROBE_CAPACITY = 256; + +// What the work functions record, so a test can see which tasks ran and what they received. +struct TaskQueueProbe +{ + std::atomic runCount[TASK_QUEUE_PROBE_CAPACITY]; + std::atomic altRunCount; + std::atomic payloadMismatches; + std::atomic started; + std::atomic finished; + + void reset() + { + for (unsigned int i = 0; i < TASK_QUEUE_PROBE_CAPACITY; i++) + { + runCount[i].store(0); + } + altRunCount.store(0); + payloadMismatches.store(0); + started.store(0); + finished.store(0); + } +}; + +struct TaskQueuePayload +{ + TaskQueueProbe* probe; + unsigned int id; + unsigned int patternSize; + unsigned char pattern[64]; +}; +static_assert(sizeof(TaskQueuePayload) <= TaskQueueScoreFunction::TASK_PAYLOAD_MAX, + "TaskQueuePayload must fit one queue slot"); + +// Bigger than one slot, but starts with a valid payload so a wrongly accepted task records the run +// instead of dereferencing garbage. +struct TaskQueueOversizedPayload +{ + TaskQueuePayload base; + unsigned char extra[TaskQueueScoreFunction::TASK_PAYLOAD_MAX]; +}; + +static TaskQueueProbe gTaskQueueProbe; +static std::unique_ptr gTaskQueueOwner; +static std::atomic gTaskQueueHelpersStop; + +static TaskQueuePayload makeTaskQueuePayload(unsigned int id, unsigned int patternSize = sizeof(TaskQueuePayload::pattern)) +{ + TaskQueuePayload task; + setMem(&task, sizeof(task), 0); + task.probe = &gTaskQueueProbe; + task.id = id; + task.patternSize = patternSize; + // Fill the pattern with id+i, so it varies by task and by position + for (unsigned int i = 0; i < patternSize; i++) + { + task.pattern[i] = (unsigned char)(id + i); + } + return task; +} + +// Records the run and checks the payload survived the copy into and out of the queue. +static void countTaskRun(unsigned long long, void* payload) +{ + const TaskQueuePayload* task = (const TaskQueuePayload*)payload; + if (task->id >= TASK_QUEUE_PROBE_CAPACITY) + { + // Surfaces as a failed test rather than a write past runCount. + task->probe->payloadMismatches.fetch_add(1); + return; + } + if (task->patternSize > sizeof(task->pattern)) + { + // A scalar that did not survive the copy is itself a mismatch, and it must not be trusted as + // the loop bound below. + task->probe->payloadMismatches.fetch_add(1); + return; + } + for (unsigned int i = 0; i < task->patternSize; i++) + { + if (task->pattern[i] != (unsigned char)(task->id + i)) + { + task->probe->payloadMismatches.fetch_add(1); + break; + } + } + task->probe->runCount[task->id].fetch_add(1); +} + +class TestQubicScoreTaskQueue : public ::testing::Test +{ +protected: + void SetUp() override + { + if (gTaskQueueOwner.get() == nullptr) + { + gTaskQueueOwner.reset(new TaskQueueScoreFunction()); + } + gTaskQueueOwner->resetTaskQueue(); + gTaskQueueProbe.reset(); + gTaskQueueHelpersStop.store(false); + } + + TaskQueueScoreFunction& queue() + { + return *gTaskQueueOwner; + } +}; + +// Task run once. Normal case +TEST_F(TestQubicScoreTaskQueue, EveryTaskRunsExactlyOnce) +{ + const unsigned int taskCount = TASK_QUEUE_PROBE_CAPACITY; + for (unsigned int i = 0; i < taskCount; i++) + { + const TaskQueuePayload task = makeTaskQueuePayload(i); + EXPECT_TRUE(queue().addTask(countTaskRun, &task, sizeof(task))); + } + + // Try to process every task in queue until all done + queue().runUntilDone(0); + + for (unsigned int i = 0; i < taskCount; i++) + { + // Each task is expected run once + EXPECT_EQ(gTaskQueueProbe.runCount[i].load(), 1u) << "task " << i; + } +} + +// Mixed mutiple size of tasks +TEST_F(TestQubicScoreTaskQueue, PayloadArrivesIntact) +{ + // Bytes a task of this pattern length hands to addTask. + const auto taskQueuePayloadBytes = [](unsigned int patternSize) -> unsigned int + { + return (unsigned int)offsetof(TaskQueuePayload, pattern) + patternSize; + }; + + const unsigned int patternSizes[] = { 0, sizeof(TaskQueuePayload::pattern) }; + const unsigned int sizeCount = (unsigned int)(sizeof(patternSizes) / sizeof(patternSizes[0])); + const unsigned int perSize = 4; + + unsigned int id = 0; + for (unsigned int s = 0; s < sizeCount; s++) + { + for (unsigned int i = 0; i < perSize; i++) + { + const TaskQueuePayload task = makeTaskQueuePayload(id, patternSizes[s]); + EXPECT_TRUE(queue().addTask(countTaskRun, &task, taskQueuePayloadBytes(patternSizes[s]))); + id++; + } + } + + // Try to process every task in queue until all done + queue().runUntilDone(0); + + EXPECT_EQ(gTaskQueueProbe.payloadMismatches.load(), 0u); + for (unsigned int i = 0; i < id; i++) + { + EXPECT_EQ(gTaskQueueProbe.runCount[i].load(), 1u) << "task " << i; + } +} + +// A payload larger than one slot must be refused, not truncated into the slot or written past it. +TEST_F(TestQubicScoreTaskQueue, OversizedPayloadIsRejected) +{ + TaskQueueOversizedPayload oversized; + setMem(&oversized, sizeof(oversized), 0); + oversized.base = makeTaskQueuePayload(0); + + EXPECT_FALSE(queue().addTask(countTaskRun, &oversized, sizeof(oversized))); + + // Nothing was queued, so the drain has nothing to run. + queue().runUntilDone(0); + EXPECT_EQ(gTaskQueueProbe.runCount[0].load(), 0u); +} + +// The queue is bounded. Filling it until addTask refuses shows where the bound is, and that going +// past it fails instead of writing off the end of the array. +TEST_F(TestQubicScoreTaskQueue, QueueRejectsOverflow) +{ + unsigned long long accepted = 0; + for (unsigned long long i = 0; i < NUMBER_OF_TRANSACTIONS_PER_TICK + 16; i++) + { + const TaskQueuePayload task = makeTaskQueuePayload(0); + const bool added = queue().addTask(countTaskRun, &task, sizeof(task)); + if (!added) + { + break; + } + accepted++; + } + + EXPECT_EQ(accepted, NUMBER_OF_TRANSACTIONS_PER_TICK); +} + +// The drain must return only after every task has finished, including the ones other threads picked +// up. Returning once the last task was merely taken would leave work still running. +TEST_F(TestQubicScoreTaskQueue, DrainWaitsForTasksRunningOnOtherThreads) +{ + // Stays in flight long enough that a drain returning on tasks taken, rather than tasks finished, + // would be visible. + const TaskQueueScoreFunction::WorkFunc slowTaskRun = [](unsigned long long, void* payload) + { + const TaskQueuePayload* task = (const TaskQueuePayload*)payload; + task->probe->started.fetch_add(1); + std::this_thread::sleep_for(std::chrono::milliseconds(2)); + task->probe->finished.fetch_add(1); + }; + + const unsigned int taskCount = 64; + const unsigned int helperCount = 4; + for (unsigned int i = 0; i < taskCount; i++) + { + const TaskQueuePayload task = makeTaskQueuePayload(i); + EXPECT_TRUE(queue().addTask(slowTaskRun, &task, sizeof(task))); + } + + // Create another threads for process some tasks in queues + std::vector helpers; + for (unsigned int t = 0; t < helperCount; t++) + { + const unsigned long long helperProcessorNumber = t + 1; + helpers.emplace_back([helperProcessorNumber]() + { + // What a request processor does: keep offering to run queued work until told to stop. + while (!gTaskQueueHelpersStop.load()) + { + gTaskQueueOwner->tryProcessOneTask(helperProcessorNumber); + } + }); + } + + // Mark the task queue ready and process remained task + queue().runUntilDone(0); + const unsigned int finishedOnReturn = gTaskQueueProbe.finished.load(); + + gTaskQueueHelpersStop.store(true); + for (unsigned int t = 0; t < helperCount; t++) + { + helpers[t].join(); + } + + // Expect all task are done + EXPECT_EQ(finishedOnReturn, taskCount); + EXPECT_EQ(gTaskQueueProbe.started.load(), taskCount); +} + +// Tasks are queued before the drain opens the queue. Until it does, a helper must pick up nothing, so +// a half-built batch is never started. +TEST_F(TestQubicScoreTaskQueue, ClosedQueueHandsOutNothing) +{ + const unsigned int taskCount = 8; + for (unsigned int i = 0; i < taskCount; i++) + { + const TaskQueuePayload task = makeTaskQueuePayload(i); + EXPECT_TRUE(queue().addTask(countTaskRun, &task, sizeof(task))); + } + + // Try to run many task but no thing run because the queue is not ready + for (unsigned int i = 0; i < 32; i++) + { + queue().tryProcessOneTask(0); + } + for (unsigned int i = 0; i < taskCount; i++) + { + EXPECT_EQ(gTaskQueueProbe.runCount[i].load(), 0u) << "task " << i << " ran before the drain"; + } + + // Process all items + queue().runUntilDone(0); + for (unsigned int i = 0; i < taskCount; i++) + { + EXPECT_EQ(gTaskQueueProbe.runCount[i].load(), 1u) << "task " << i; + } +} + +// Every tick resets the queue and refills it, so a second batch must behave like the first. It will +// not if reset leaves any of the three counters behind. +TEST_F(TestQubicScoreTaskQueue, QueueIsReusableAfterReset) +{ + const unsigned int taskCount = 16; + for (unsigned int batch = 0; batch < 2; batch++) + { + queue().resetTaskQueue(); + gTaskQueueProbe.reset(); + + for (unsigned int i = 0; i < taskCount; i++) + { + const TaskQueuePayload task = makeTaskQueuePayload(i); + EXPECT_TRUE(queue().addTask(countTaskRun, &task, sizeof(task))); + } + + queue().runUntilDone(0); + + for (unsigned int i = 0; i < taskCount; i++) + { + EXPECT_EQ(gTaskQueueProbe.runCount[i].load(), 1u) << "batch " << batch << " task " << i; + } + } +} + +// Each task carries its own work function, so one batch can mix kinds. This is what lets a second +// caller share the queue without changing it. +TEST_F(TestQubicScoreTaskQueue, OneBatchCarriesDifferentWorkFunctions) +{ + // A second work function, so a batch can be shown to carry more than one kind of task. + const TaskQueueScoreFunction::WorkFunc countAltTaskRun = [](unsigned long long, void* payload) + { + const TaskQueuePayload* task = (const TaskQueuePayload*)payload; + task->probe->altRunCount.fetch_add(1); + }; + + const unsigned int pairCount = 32; + for (unsigned int i = 0; i < pairCount; i++) + { + const TaskQueuePayload counted = makeTaskQueuePayload(i); + EXPECT_TRUE(queue().addTask(countTaskRun, &counted, sizeof(counted))); + const TaskQueuePayload alt = makeTaskQueuePayload(i); + EXPECT_TRUE(queue().addTask(countAltTaskRun, &alt, sizeof(alt))); + } + + queue().runUntilDone(0); + + for (unsigned int i = 0; i < pairCount; i++) + { + EXPECT_EQ(gTaskQueueProbe.runCount[i].load(), 1u) << "task " << i; + } + EXPECT_EQ(gTaskQueueProbe.altRunCount.load(), pairCount); +} + diff --git a/test/stdlib_impl.cpp b/test/stdlib_impl.cpp index 665513dc..1fd71744 100644 --- a/test/stdlib_impl.cpp +++ b/test/stdlib_impl.cpp @@ -89,13 +89,40 @@ void* qVirtualCommit(void* address, const unsigned long long size) { return VirtualAlloc(address, (SIZE_T)size, MEM_COMMIT, PAGE_READWRITE); } +unsigned long long qGetPageSize() { + SYSTEM_INFO systemInfo; + GetSystemInfo(&systemInfo); + return (unsigned long long)systemInfo.dwPageSize; +} + bool qVirtualFreeAndRecommit(void* address, const unsigned long long size) { - VirtualFree(address, (SIZE_T)size, MEM_DECOMMIT); - bool commitMem = commitMemMap[(unsigned long long)address]; - if (!commitMem) { - return true; - } - return VirtualAlloc(address, (SIZE_T)size, MEM_COMMIT, PAGE_READWRITE) != address; + static const unsigned long long pageSize = qGetPageSize(); + const bool commitMem = commitMemMap[(unsigned long long)address]; + + // MEM_DECOMMIT rounds the length up to a page, so decommitting a non-page-multiple size would + // also drop whatever region shares the last page; zero that tail in place instead. + const unsigned long long decommitSize = size & ~(pageSize - 1); + if (decommitSize) + { + VirtualFree(address, (SIZE_T)decommitSize, MEM_DECOMMIT); + if (commitMem && VirtualAlloc(address, (SIZE_T)decommitSize, MEM_COMMIT, PAGE_READWRITE) != address) + { + return false; + } + } + + const unsigned long long tailSize = size - decommitSize; + if (tailSize) + { + char* tail = (char*)address + decommitSize; + if (!VirtualAlloc(tail, (SIZE_T)tailSize, MEM_COMMIT, PAGE_READWRITE)) + { + return false; + } + memset(tail, 0, (size_t)tailSize); + } + + return true; } #else void* qVirtualAlloc(const unsigned long long size, bool commitMem = false) { @@ -127,9 +154,30 @@ void* qVirtualCommit(void* address, const unsigned long long size) { } bool qVirtualFreeAndRecommit(void* address, const unsigned long long size) { - bool commitMem = commitMemMap[(unsigned long long)address]; - int prot = commitMem ? (PROT_READ | PROT_WRITE) : PROT_NONE; - return mmap(address, size, prot, MAP_PRIVATE | MAP_ANONYMOUS | MAP_FIXED, -1, 0) == address; + static const unsigned long long pageSize = (unsigned long long)sysconf(_SC_PAGESIZE); + const bool commitMem = commitMemMap[(unsigned long long)address]; + const int prot = commitMem ? (PROT_READ | PROT_WRITE) : PROT_NONE; + + // MAP_FIXED rounds the length up to a page, so remapping a non-page-multiple size would also + // wipe whatever region shares the last page; zero that tail in place instead of remapping it. + const unsigned long long remapSize = size & ~(pageSize - 1); + if (remapSize && mmap(address, remapSize, prot, MAP_PRIVATE | MAP_ANONYMOUS | MAP_FIXED, -1, 0) != address) + { + return false; + } + + const unsigned long long tailSize = size - remapSize; + if (tailSize) + { + char* tail = (char*)address + remapSize; + if (mprotect(tail, tailSize, PROT_READ | PROT_WRITE) != 0) + { + return false; + } + memset(tail, 0, tailSize); + } + + return true; } #endif diff --git a/test/test.vcxproj b/test/test.vcxproj index 2e22b39f..367f208c 100644 --- a/test/test.vcxproj +++ b/test/test.vcxproj @@ -176,9 +176,12 @@ + + + diff --git a/test/trit_pack.cpp b/test/trit_pack.cpp new file mode 100644 index 00000000..e8aef529 --- /dev/null +++ b/test/trit_pack.cpp @@ -0,0 +1,137 @@ +#define NO_UEFI + +#include "gtest/gtest.h" + +#include "../src/mining/trit_pack.h" + +// Trit packing is a STORAGE format: the ant colony's ANN pool is packed in memory and written to +// disk that way, so a layout change silently breaks snapshot loads rather than failing to compile. +// The layout is therefore pinned by golden values here, not just round-tripped. +// +// trit_pack.h includes nothing, so this file does too - if that ever stops being true the test +// stops building and the header has quietly grown a dependency. + +using score_engine::PackedTrits; + +// Two groups of five trits is small enough that every value the structure can hold fits in a loop, +// so this is exhaustive rather than a sample: 3^10 assignments, each packed and unpacked. +TEST(TestTritPack, RoundTripsEveryPossibleValue) +{ + using P = PackedTrits<2, 5>; + static constexpr unsigned int COMBINATIONS = 59049; // 3^10 + + for (unsigned int v = 0; v < COMBINATIONS; v++) + { + unsigned char src[P::tritCount]; + unsigned int rest = v; + for (unsigned long long i = 0; i < P::tritCount; i++) + { + src[i] = (unsigned char)(rest % 3); + rest /= 3; + } + + P packed; + packed.pack(src); + + unsigned char back[P::tritCount]; + for (unsigned long long i = 0; i < P::tritCount; i++) + { + back[i] = 0xFF; // so a trit the unpack never writes fails loudly + } + packed.unpack(back); + + for (unsigned long long i = 0; i < P::tritCount; i++) + { + ASSERT_EQ(back[i], src[i]) << "value " << v << ", trit " << i; + } + } +} + +// The documented layout - trit i of group g at bits [2i, 2i+2), lowest index in the lowest bits. +// Round-trip tests pass under any self-consistent layout, so only fixed words catch a reordering +// that would leave existing snapshot files unreadable. +TEST(TestTritPack, LayoutIsTwoBitsPerTritLowestIndexFirst) +{ + PackedTrits<2, 4> packed; + const unsigned char src[8] = { 1, 2, 0, 1, 2, 2, 1, 0 }; + packed.pack(src); + + EXPECT_EQ(packed.word[0], 1ull + (2ull << 2) + (0ull << 4) + (1ull << 6)); // 73 + EXPECT_EQ(packed.word[1], 2ull + (2ull << 2) + (1ull << 4) + (0ull << 6)); // 26 +} + +// A group is one scorer row and gets its own word, so editing a row must not touch any other. This +// is what lets the colony reason about a neuron's LUT independently. +TEST(TestTritPack, GroupsAreIndependent) +{ + using P = PackedTrits<4, 6>; + unsigned char src[P::tritCount]; + for (unsigned long long i = 0; i < P::tritCount; i++) + { + src[i] = 0; + } + + P base; + base.pack(src); + + for (unsigned long long g = 0; g < P::groupCount; g++) + { + src[g * P::tritsPerGroup] = 2; + P edited; + edited.pack(src); + src[g * P::tritsPerGroup] = 0; + + for (unsigned long long m = 0; m < P::groupCount; m++) + { + if (m == g) + { + ASSERT_NE(edited.word[m], base.word[m]) << "group " << m << " should have changed"; + } + else + { + ASSERT_EQ(edited.word[m], base.word[m]) << "group " << m << " must not change"; + } + } + } +} + +// 32 trits is the widest group a uint64 holds, so the top trit sits at bits 62-63. A shift written +// on a 32-bit type would be undefined there and would typically lose the high half silently. +TEST(TestTritPack, WidestLegalGroupRoundTrips) +{ + using P = PackedTrits<1, 32>; + unsigned char src[P::tritCount]; + for (unsigned long long i = 0; i < P::tritCount; i++) + { + src[i] = 2; + } + + P packed; + packed.pack(src); + EXPECT_EQ(packed.word[0], 0xAAAAAAAAAAAAAAAAull); // every trit 0b10 + + unsigned char back[P::tritCount]; + packed.unpack(back); + for (unsigned long long i = 0; i < P::tritCount; i++) + { + ASSERT_EQ(back[i], 2) << "trit " << i; + } +} + +// pack() masks instead of validating. A byte the scorer should never have written is truncated to +// its low two bits and stays inside its own trit - it does not shift the ones after it, which is +// the property that keeps one bad byte from corrupting a whole row. +TEST(TestTritPack, OutOfRangeByteCannotDisturbItsNeighbours) +{ + PackedTrits<1, 4> packed; + const unsigned char src[4] = { 4, 1, 7, 2 }; // 4 -> 0, 7 -> 3 + packed.pack(src); + + unsigned char back[4]; + packed.unpack(back); + + EXPECT_EQ(back[0], 0); + EXPECT_EQ(back[1], 1); + EXPECT_EQ(back[2], 3); + EXPECT_EQ(back[3], 2); +}