Repository navigation
policy: Count data in a dead conditional branch - #400
jasonsopko wants to merge 13 commits into
Conversation
|
Why allow dead branches at all? No one is using this method. We could just make a dead branch filter, maybe even have it on by default, then when rdts expires, this would supersede it (for the tapscript part)? |
|
It does not allow them. Under the defaults, any byte the detector counts inside a dead branch rejects the transaction as It reaches tapscript already. On nobody using it: one transaction so far, the one in the description. Generalizing the trigger rather than adding |
OLGA hides a payload in P2WSH outputs at the dust threshold, and the detector keys on its "stamp:" magic. The same layout with "ACME" has put over 100 transactions on chain since 957497, and changing the magic is free. Judge the bytes instead. A hash or x-only key is uniformly random, so one byte value eight times in a program, a run of six, or a taproot key off the curve marks it as data, along with its same-type same-value siblings. Three or more distinct P2WSH outputs sharing one value of 546 sat or less are data on their own: encoders pay every chunk the same, batch payouts on chain start at 600 sat, Lightning anchors are a pair and HTLC amounts differ. Small taproot runs are left to the curve check, since the ones on chain are ordinals postage with no data in the outputs. The all-zero burn hash is left alone. The detector sits behind -rejectfakeoutputs, default on and reset by -corepolicy. Over the 3,018,541 transactions since the fork it newly rejects 45: 14 funding transactions of the fake-key and JXL encoders, 14 swap memos continued as ASCII in a P2WPKH hash, 11 ACME-style P2WSH payloads and 6 taproot outputs tagged DECODE WITNESS. Checked on Ubuntu 24.04 against 29.4.2: a gcc 13.3 -Werror build with wallet and ZMQ has no warnings, and all 147 ctest suites and the full functional suite pass, alone and merged with bitcoinknots#400, bitcoinknots#319 and bitcoinknots#422. The clang 18 ASan/UBSan build passes the unit suite and the policy functional tests.
f2e6893 to
e704b7a
Compare
|
Rebased on 29.4.2 and squashed. The rule is now behind |
|
The transaction above replays on regtest with the fork and RDTS active and the default policy on: stock 29.4.2 accepts it, this branch refuses it with Nothing on the branch has changed since e704b7a. It sits on the current @kwsantiago @MarcanoFilms review welcome. Edit: 2026-10-14 removed the link to the replay script, as in the description. The branch this comment describes has been replaced since; see the latest comment of mine below. |
|
AuthintegX raised a cost question in the Knots Discord:
Pushed 68e6db8:
Median time for one
The 390 KB rows no longer reach the analysis in a node, since anything over 1,650 bytes is skipped; they show the bound holds without the size gate. The worst left is the arithmetic chain at about 85 ns per byte, the same order as the interpreter on that script. Over every transaction from 961640 to 974573 (3,103,397), the counts are byte-identical before and after. Thanks to AuthintegX for raising it. Benchmark (built into test_bitcoin, run with DEADBRANCH_COST=1)// Local measurement, not for submission. Times CScript::DatacarrierBytes with
// and without dead-branch tracking on adversarial scripts of growing size, and
// the interpreter on the same script for comparison. Runs only with
// DEADBRANCH_COST=1 in the environment; prints one line per case.
#include <crypto/sha256.h>
#include <hash.h>
#include <script/interpreter.h>
#include <script/script.h>
#include <test/util/setup_common.h>
#include <boost/test/unit_test.hpp>
#include <algorithm>
#include <chrono>
#include <cstdio>
#include <cstdlib>
#include <functional>
#include <string>
#include <vector>
namespace {
using Clock = std::chrono::steady_clock;
double TimeUs(const std::function<void()>& f, int reps)
{
std::vector<double> runs;
for (int i{0}; i < reps; ++i) {
const auto t0{Clock::now()};
f();
runs.push_back(std::chrono::duration<double, std::micro>(Clock::now() - t0).count());
}
std::sort(runs.begin(), runs.end());
return runs[runs.size() / 2];
}
CScript Fill(size_t items)
{
CScript s;
for (size_t i{0}; i < items; ++i) s << OP_1;
return s;
}
// 999 known items, then <998> OP_ROLL until the size is reached: each roll moves 998 entries
CScript RollBomb(size_t size)
{
CScript s{Fill(999)};
while (s.size() + 4 <= size) s << CScriptNum{998} << OP_ROLL;
return s;
}
// Two different 520-byte values, then OP_2DUP OP_EQUAL OP_DROP: each compare reads 520 bytes
CScript EqualBomb(size_t size)
{
CScript s;
s << std::vector<unsigned char>(520, 0xaa) << std::vector<unsigned char>(520, 0xab);
while (s.size() + 3 <= size) s << OP_2DUP << OP_EQUAL << OP_DROP;
return s;
}
// OP_1 then OP_1ADD repeatedly: one new value per byte
CScript ArithBomb(size_t size)
{
CScript s;
s << OP_1;
while (s.size() + 1 <= size) s << OP_1ADD;
return s;
}
// 999 known items, then OP_2ROT OP_2SWAP OP_ROT repeatedly
CScript ShuffleBomb(size_t size)
{
CScript s{Fill(999)};
while (s.size() + 3 <= size) s << OP_2ROT << OP_2SWAP << OP_ROT;
return s;
}
// Known conditions nested as deep as the size allows, then closed
CScript NestBomb(size_t size)
{
CScript s;
size_t depth{0};
while (s.size() + 3 <= size) { s << OP_1 << OP_IF; ++depth; }
for (size_t i{0}; i < depth; ++i) s << OP_ENDIF;
return s;
}
// A dead branch full of pushes, the shape #400 exists for
CScript DeadBranchData(size_t size)
{
CScript s;
s << OP_1 << OP_NOTIF;
while (s.size() + 524 <= size) s << std::vector<unsigned char>(520, 0x42);
s << OP_ENDIF << OP_1;
return s;
}
} // namespace
BOOST_FIXTURE_TEST_SUITE(deadbranch_cost_tests, BasicTestingSetup)
BOOST_AUTO_TEST_CASE(deadbranch_cost)
{
if (!std::getenv("DEADBRANCH_COST")) return;
const std::vector<std::pair<std::string, std::function<CScript(size_t)>>> shapes{
{"roll", RollBomb}, {"equal", EqualBomb}, {"arith", ArithBomb},
{"shuffle", ShuffleBomb}, {"nest", NestBomb}, {"deadbranch", DeadBranchData},
};
std::printf("shape size_bytes scan_off_us scan_on_us interpreter_us sha256d_us\n");
for (const size_t size : {size_t{1650}, size_t{10000}, size_t{100000}, size_t{390000}}) {
std::vector<unsigned char> blob(size, 0x5a);
const double hash_us{TimeUs([&] { uint256 h{Hash(blob)}; (void)h; }, 21)};
for (const auto& [name, make] : shapes) {
const CScript script{make(size)};
const int reps{size > 100000 ? 7 : 21};
const double off{TimeUs([&] { (void)script.DatacarrierBytes(0, nullptr, false); }, reps)};
const double on{TimeUs([&] { (void)script.DatacarrierBytes(0, nullptr, true); }, reps)};
const double interp{TimeUs([&] {
std::vector<std::vector<unsigned char>> stack;
ScriptExecutionData execdata;
execdata.m_validation_weight_left_init = true;
execdata.m_validation_weight_left = 1 << 30;
ScriptError err;
(void)EvalScript(stack, script, SCRIPT_VERIFY_TAPROOT, BaseSignatureChecker{}, SigVersion::TAPSCRIPT, execdata, &err);
}, reps)};
std::printf("%s %zu %.1f %.1f %.1f %.1f\n", name.c_str(), size_t{script.size()}, off, on, interp, hash_us);
}
}
}
BOOST_AUTO_TEST_SUITE_END() |
|
Concept ACK. Data in a constant-guarded dead branch is the same thing the OP_FALSE OP_IF envelope already counts; OP_1 OP_NOTIF is its mirror. Policy only: all callers are mempool acceptance and fee weight, no consensus path touched, existing opt-outs apply. (edited to remove incorrect approach note) |
|
Concept ACK. |
… bytes The rule only recognised a key in a 33 or 65 byte push, so the same payload written as tapscript went free: a 1-of-15 costs 132 datacarrier bytes under P2WSH and nothing at all under a taproot leaf, for 32 bytes of publishable space per key rather than 31. A 32 byte push is not a key on sight, since nothing separates an x-only key from a hash until something spends it. Count it where a checksig-family opcode claims it instead, which leaves the preimage of a hashlock alone. OP_CHECKSIGADD is how tapscript writes an m-of-n, so credit its canonical shape as OP_CHECKMULTISIG's is already credited: a run of <pubkey> OP_CHECKSIG (<pubkey> OP_CHECKSIGADD)* <k> OP_NUMEQUAL credits k, whatever size its keys are. Without it every honest tapscript multisig past eleven keys would be charged for the keys it does not spend, which is the opposite of the point: a 20-of-20 would cost 297 bytes. The OP_CHECKSIG that opens the run has credited one of the k already, so only the remaining k-1 are added there. Padding the run, reordering it or naming a k it cannot satisfy forfeits the credit rather than earning one, so working around the rule raises the charge instead of lowering it. An opcode that leaves the stack as it found it does not hide the key beneath it from the opcode that spends it, so it does not buy the count off either.
Checks the taproot leaf against the P2WSH script that says the same thing, shape by shape, so the two are charged alike rather than only plausibly, including at the tolerance boundary: 1-of-11 is free and 1-of-12, 1-of-13 and 5-of-16 pay the same either way. Covers the hashlock shapes a 32 byte push must not be mistaken for a key in, the padded and inflated runs that forfeit the credit, the OP_CHECKSIG that opens a run crediting one of the k and not two, a k above 16 written as a minimal push, and the key stranded on the stack that neither rule reaches.
…rrier' into policy-dead-branch-data bitcoinknots#422 at eea2467 changes DatacarrierBytes too. This branch builds on it so the 29.4.3 assembly can merge it after k422.
The detector counts the OP_FALSE OP_IF envelope but not its mirror, OP_1 OP_NOTIF, or any other branch constants make unreachable. 23e8f946... at 969961 carried a 37 KB JPEG XL image that way, and nothing on the fence saw it. Follow the script as far as constants decide it, through stack moves, arithmetic, comparisons, no-ops and the altstack, and count every conditional part that can never run, including the dead part of an OP_ELSE. Anything read from the witness or a signature check is unknown and never counted, which is how spendable scripts branch. The rule sits behind -rejectdeadbranches, default on and reset by -corepolicy; with it off the detector is the old one byte for byte. Checked on Ubuntu 24.04 against 29.4.2: a gcc 13.3 -Werror build with wallet and ZMQ has no warnings, and all 147 ctest suites and the full functional suite pass, alone and merged with bitcoinknots#389, bitcoinknots#319 and bitcoinknots#422. The clang 18 ASan/UBSan build passes the unit suite and the policy functional tests. A fuzz target checked that with the switch off the result matches the original function, and that every part marked dead leaves the interpreter's verdict unchanged when an OP_RETURN is put in it, over 4.9 million inputs in 45 minutes. Over all 3,018,541 transactions since the fork it counts one: 23e8f946..., at 37,854 bytes.
FindDeadParts ran on every input before IsWitnessStandard checks the witness size, and OP_ROLL with a known depth moved up to 1,000 stack entries. A 390 KB script repeating <998> OP_ROLL took 506 ms to analyze against 1.7 ms for the plain scan, on a transaction the node then rejects at no cost to the sender. AuthintegX raised it in the Knots Discord. Skip inputs whose witness is over -maxscriptsize, which fail IsWitnessStandard right after, and charge each opcode one step and each OP_ROLL its depth, stopping at four steps per byte. Stopping marks nothing more, so it can only undercount, the same as a condition read from the witness. The roll case drops to 92 us at 1,650 bytes (was 882) and 9.7 ms at 390 KB. The worst left is an arithmetic chain at about 85 ns per byte, the same order as the interpreter on that script. Counts over all 3,103,397 transactions from 961640 to 974573 are unchanged.
gcc 11 at -O2 cannot see that OP_WITHIN reads lo only after Num() has set it, and the previous-releases CI job (DEBUG, -Werror) fails with "'lo' may be used uninitialized". Start every int64_t the walk fills through Num() at zero so no compiler has to prove it. No behavior change: each one is still read only where Num() returned true.
68e6db8 to
847d6a2
Compare
|
Pushed 847d6a2, three changes so the branch can go into 29.4.3:
Tested on the new head: -Werror build, ctest 128 of 128, functional suite without wallet 202 passed, 138 skipped, 1 failed: tool_cli_completion.py, which fails on anything rooted before the release regenerated the completions (#429 had the same) and passes once merged. Merged into the 29.4.3rc1 candidate (chrisguida/bitcoin a7d7d2e): no conflict, the result differs from the candidate by exactly this PR's diff, and on that tree ctest 131 of 131 and the functional suite 218 passed, 141 skipped, 0 failed. Spec line proposed on luke-jr/tmp#5. |
OLGA hides a payload in P2WSH outputs at the dust threshold, and the detector keys on its "stamp:" magic. The same layout with "ACME" has put over 100 transactions on chain since 957497, and changing the magic is free. Judge the bytes instead. A hash or x-only key is uniformly random, so one byte value eight times in a program, a run of six, or a taproot key off the curve marks it as data, along with its same-type same-value siblings. Three or more distinct P2WSH outputs sharing one value of 546 sat or less are data on their own: encoders pay every chunk the same, batch payouts on chain start at 600 sat, Lightning anchors are a pair and HTLC amounts differ. Small taproot runs are left to the curve check, since the ones on chain are ordinals postage with no data in the outputs. The all-zero burn hash is left alone. The detector sits behind -rejectfakeoutputs, default on and reset by -corepolicy. Over the 3,018,541 transactions since the fork it newly rejects 45: 14 funding transactions of the fake-key and JXL encoders, 14 swap memos continued as ASCII in a P2WPKH hash, 11 ACME-style P2WSH payloads and 6 taproot outputs tagged DECODE WITNESS. Checked on Ubuntu 24.04 against 29.4.2: a gcc 13.3 -Werror build with wallet and ZMQ has no warnings, and all 147 ctest suites and the full functional suite pass, alone and merged with bitcoinknots#400, bitcoinknots#319 and bitcoinknots#422. The clang 18 ASan/UBSan build passes the unit suite and the policy functional tests.
|
Fork CI on 847d6a2: the gcc 11 previous-releases job compiles clean, so the zero-init commit does what it was for. Nine jobs fail, each at tool_cli_completion.py, the base artifact above: eight stop there, since CI runs the suite with |
6 participants
What
Counts data in any conditional part that constants make unreachable, including the dead part of an
OP_ELSE, behind a new-rejectdeadbranches(default on, reset by-corepolicy). With it off the detector is the old one byte for byte.Why
23e8f9466cd895c21a041155dba377f1a77a46cec13dcb14c430fa9c75f8786a at 969961 carried a 37 KB JPEG XL image in
OP_1 OP_NOTIFbranches across 25 P2WSH inputs; onlyOP_FALSE OP_IFwas counted. The guard is worked out through stack moves, arithmetic, comparisons, no-ops and the altstack, soOP_1 OP_NOP OP_NOTIFand the like are counted too. Anything read from the witness or a signature check is unknown and never counted, which is how spendable scripts branch; the Lightning to_local, HTLC and anchor scripts count zero.How I tested
The -Werror build and the full unit and functional suites pass on this branch merged into the 29.4.3rc1 candidate, and on the branch alone apart from tool_cli_completion.py, which needs the regenerated completions the release adds. The branch carries #319 and #422. On the earlier head, ASan/UBSan was clean, and a fuzz target checked that with the switch off the result matches the original function, and that every part marked dead leaves the interpreter's verdict unchanged when an
OP_RETURNis put in it, over 4.9 million inputs in 45 minutes. Over the 3,018,541 transactions since the fork it counts one: the transaction above, at 37,854 bytes.Risk and rollback
Policy only, and it reads input scripts only.
-rejectdeadbranches=0turns it off.Edit: 2026-10-01 added a link to a regtest replay to Why; 2026-10-04 added paragraphs on contiguity and carving. 2026-10-14 removed them; the rule does not depend on them. How I tested now describes the head rebased for 29.4.3.