Conversation
runChainDeploy only observes a deploy command's exit code — a policy violation that leaves the shell step at exit 0 is invisible, and the one finding the step can produce (category: commands) carries no expect/observed evidence shape. Characterization test, not a fix; see policy-probe/docs/HARNESS_ARCHITECTURE.md.
Adds chainValidation.actors (named additional ephemeral signers) and
chainValidation.assertions ({id, actor?, action, expect}) to the recipe
schema, parsed and validated in specLoader.ts. action reuses the existing
ChainValidationDeployCommand shape rather than duplicating it.
Schema only: the execution/evaluation engine that runs an assertion and
turns a mismatch into a ValidationFinding is not implemented yet (next
change). docs/authoring-a-recipe.md says so explicitly.
9 new tests (chain-assertions-schema.test.mjs), full suite 206/206,
typecheck clean. See policy-probe/docs/DECISIONS.md ADR-0006.
Generalizes chainSigner.ts's provision/reuse/top-up/sweep logic (previously hardcoded to the one primary signer) so it works for any number of named chainValidation.actors too, each persisted to its own chain-actor-<name>.json and independently reused/swept. Threads chainActors through sessionRunner -> attemptLoop -> AttemptStageContext alongside the existing chainSigner. Adds an actor field to the chain_signer_provisioned/chain_signer_swept log events. Live-tested against real Hedera testnet (2 accounts created, funded, reused, and swept back) -- skips gracefully when HEDERA_OPERATOR_ID/ HEDERA_OPERATOR_KEY are not set. Full suite 208/208, typecheck clean.
Adds chainAssertionEvidence.ts: fetchTransactionResult (real consensus result, never a script's exit code), fetchHbarBalanceTinybars, fetchTokenBalance -- each returning a strict found/not-found/infra-error trichotomy so propagation lag and genuine infra failures are never conflated with each other or with a policy result. Deliberately scoped to exactly what postcondition evaluation needs, not a general Mirror Node client -- see the module comment on why this doesn't duplicate PR hedera-dev#39/hedera-dev#43. 13 unit tests against a local mock server exercise every branch of that trichotomy (200/404-then-200/all-404/500/malformed-JSON/connection-refused). 2 live tests round-trip a real HBAR transfer and a real balance query through the actual testnet Mirror Node -- skip gracefully without HEDERA_OPERATOR_ID/HEDERA_OPERATOR_KEY. Full suite 223/223.
…tions Wires the assertion execution engine into the real pipeline: runs each chainValidation.assertions[] entry's action with the resolved signer right after a successful chain deploy (before SMOKE boots a dev server), captures the real transaction id the action prints, queries its real Mirror Node result, and compares it to the declared expect deterministically -- never trusting the action's own exit code as the verdict. New ValidationFinding categories: chain-assertion (confirmed violation or a fixable config/script problem, fed to repair) and chain-assertion-infra (evidence couldn't be obtained -- Mirror Node lag/outage, or the action itself didn't complete). A batch that is ALL infra aborts the attempt (reusing the existing evaluation.infrastructureFailure mechanism) instead of spending a repair attempt on something no agent could fix -- found and fixed as a real gap during implementation. promptBuilder.ts classifies the new category into repair scope and strips -infra from the repair prompt, matching the eval/eval-infra precedent; both repair prompt templates now name the new category explicitly. Incorporates an independent review pass: - action command failing to complete (non-zero exit / timeout) is now chain-assertion-infra, not a policy-violation claim from ambiguous evidence (previously any missing transaction id was treated the same way regardless of why) - balanceDelta.equals is validated as a signed integer string at load time, so a recipe typo can no longer crash the run via an unguarded BigInt() - authoring-a-recipe.md updated -- it previously said execution wasn't implemented yet - a private-repo-path reference was removed from chainAssertionEvidence.ts's module comment (folded into that already-committed, unpushed commit) - one known, intentionally out-of-scope limitation documented in-code (executeCommand can reject on a spawn error, uncaught here, same pre-existing gap as runChainDeploy -- fixing it belongs in its own change) 18 new/changed tests (2 negative tests for the action-failure fix, 2 for the balanceDelta.equals fix), full suite 245/245 with real testnet credentials, including the live end-to-end test.
Discovered while building the ATS bond fixture: an action submitted through
an EVM JSON-RPC relay (ethers.js, Hardhat, viem -- Hashio on Hedera, and how
essentially every Solidity dApp built on Hedera signs transactions) produces
an EVM transaction hash, not the native Hedera SDK transaction id
(0.0.x@seconds.nanos) the evidence reader previously only recognized. Such
an action's real, successful evidence was being reported as "no parseable
transaction id" -- a false chain-assertion violation from a script that
had, in fact, worked correctly.
Adds extractEvmTransactionHash and fetchContractCallResult (Mirror Node's
/contracts/results/{transactionIdOrHash}, which returns the same
result vocabulary -- SUCCESS/CONTRACT_REVERT_EXECUTED/etc. -- as the native
/transactions/{id} endpoint, so downstream comparison logic is unchanged).
chainAssertions.ts now tries both id shapes and dispatches to the matching
fetch function.
Verified against two real, permanently-recorded testnet transactions from
building the ATS fixture: a successful bond deployment and a reverted
attempt with an invalid ISIN (decoded error_message present in both new
live tests, no operator credentials needed -- these are historical Mirror
Node facts, not new transactions). Plus 3 new mock-server unit tests and 3
pure extraction-function tests. Full suite 254/254.
…d revert reasons, retry Mirror Node fetch failures Three real defects found by an independent adversarial review, all fixed and tested: 1. CRITICAL: the repair/generate-loop prompt writer (attemptReporting.ts) had zero secret redaction, unlike the EVALUATE-stage prompt writer which redacted only the primary signer. Any leaked key reaching a finding's details (or any future source) had a clear path to a third-party LLM API call and a plaintext file, un-redactable after the fact. Fixed: every signer (primary + all named actors) is now redacted at this sink too, in addition to being redacted at the chain-assertion/chain-deploy source (chainSigner.ts::redactSignerSecrets, applied to both call sites). writePromptFile also now strips the bare (non-0x) hex form. 2. reasonContains only ever checked Mirror Node's coarse result status (CONTRACT_REVERT_EXECUTED), identical for every revert reason on a contract -- unable to distinguish *why* an EVM call reverted. Fixed: decodeStandardRevertReason() decodes a standard Solidity Error(string) revert into its human message; reasonContains checks that when present. Honestly documented limitation: a contract-specific custom error (confirmed live against this project's own ATS fixture, selector 0x796c1f0d) can't be decoded without that contract's ABI -- authoring-a-recipe.md now says so explicitly instead of implying universal support. 3. A thrown fetch() returned infra-error immediately while a 404 retried through the full poll window -- identical chain state could get a different verdict purely from which poll hit a one-off network blip. Fixed: fetch exceptions now retry the same as a 404, and the final error message reflects the most recent attempt. 13 new tests (revert-reason decoding x4, EVM-path reasonContains x2, fetch-exception retry x2, prompt-file redaction x3 in a new attempt-reporting-redaction.test.mjs, plus 2 more). Full suite 267/267.
Discovered a real, previously-undetected gap while planning a coupon
demo on the ATS bond fixture: balanceDelta only supported HBAR and native
HTS tokens, both read via Mirror Node's account/token-association data.
An ERC20/ERC1400-style Solidity token -- which is exactly what an Asset
Tokenization Studio bond, or any Solidity security-token contract, actually
is -- lives entirely as contract storage and has NO Mirror Node account/
token-association entry at all. Confirmed empirically: a real ATS bond
holder with a genuine, positive, on-chain balance shows zero token
associations -- fetchTokenBalance would silently read 0 for every such
holder, always, with no way to tell "never held any" from "this asset type
isn't visible to this endpoint."
Adds fetchContractTokenBalance: reads a contract's balance via its standard
ERC20 balanceOf(address), through Mirror Node's own read-only contract-call
simulation (/contracts/call) -- no external JSON-RPC relay (Hashio or
otherwise) needed, keeping the module's Mirror-Node-only dependency
footprint. Generalized pollForEvidence to support a POST body so this
reuses the exact same found/not-found/infra-error retry contract as every
other evidence function.
balanceDelta.asset gains a third shape, { contract: "0x..." }, alongside
"hbar" and { tokenId }. Schema validates the object has exactly one
recognized shape.
9 new tests (3 evidence unit tests, 2 live tests against the real ATS bond
-- one proving the HTS-invisibility claim, one proving the new function
reads Alice's real positive balance -- plus schema tests). Full suite
273/273.
Closes a limitation deliberately deferred and documented (not fixed) in
ADR-0007: executeCommand's returned promise can reject outright on a
child-process "error" event (ENOENT/EACCES/an unspawnable command) --
not just resolve with a non-zero exit code. Uncaught, this crashes the
whole attempt instead of producing a graceful finding, in both
chainAssertions.ts's action execution and the pre-existing
runChainDeploy in attemptStages.ts.
Verified the trigger is real, not theoretical: a nonexistent cwd reliably
causes this rejection even with shell:true (the shell itself can't be
spawned into a directory that doesn't exist) -- confirmed directly against
the built executeCommand before writing the fix.
Both call sites now catch the rejection and report it the same way an
exit code failure already is: chain-assertion-infra ("could not be
started") for the assertion path, commands ("could not be started") for
the deploy path -- never an uncaught crash, never a policy-violation
claim from something that never even ran.
2 new integration tests (one per call site) using the real nonexistent-cwd
trigger, not a mock. Full suite 275/275.
association for a confirmed zero balance
Real, previously-undiscovered gap found by exercising fetchTokenBalance
against a genuine, freshly-created HTS token for the first time in this
suite (every earlier test used either a mock, an operator's pre-existing
HBAR balance, or an ATS bond -- which is an EVM contract token, not HTS,
so it never touched this code path at all).
The bug: `entry?.balance ?? 0` made `extract` return a *defined* `0n`
whenever the token's entry was merely absent from
`/accounts/{id}`'s `balance.tokens[]` -- indistinguishable, to
pollForEvidence, from a confirmed real zero. Because Mirror Node's
account endpoint itself answers 200 immediately (the account already
exists), this meant a brand-new token's not-yet-indexed balance entry
was accepted as final on the very first poll, with zero retries.
Confirmed live: a manual poll loop found the same entry present and
correct within ~3s, while the harness's own fetchTokenBalance read 0n
with no retry at all.
Fix: `extract` now returns `undefined` (triggering the existing retry
loop) while the entry is absent, exactly like a 404. `pollForEvidence`
gains an optional `fallbackOnHealthyTimeout` value, returned as the
real, final `found` answer only if every response for the whole budget
was healthy (200, just missing the shape) -- never used after any
404/non-200/fetch failure, so genuine infra trouble is still reported
as infra-error, not silently coerced to zero.
New coverage:
- a deterministic mock regression test (entry absent for two polls,
present with a real non-zero balance on the third) locking in the
retry behavior without depending on real testnet timing.
- a real-testnet test: create a genuine HTS token, read the treasury's
balance immediately after the create receipt, assert it resolves to
the real minted supply -- the primitive fetchTokenBalance exists for,
now actually exercised against a real HTS token for the first time.
Full suite 277/277 on real testnet (was 275/275 before this change; the
2 new tests are the ones above).
A genuinely-zero native-HTS balance (e.g. an account with no association yet) can now take the full poll budget (default 20s) to resolve, since the fix in the previous commit can no longer treat an absent token entry as an instant, confirmed zero -- that was exactly the bug it closed. Correctness over latency is the right tradeoff (the alternative is silently misreporting a fresh association's real balance as zero again), but a recipe author hitting this for the first time deserves to know why, not just discover it. Documented which sample this mainly affects (BEFORE, not AFTER) and confirmed the contract-token path is unaffected.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CHAIN runs can confirm that a transaction landed, but recipes could not declare the expected on-chain behavior for a specific action. This change adds optional named actors and deterministic postcondition assertions so a recipe can require a successful transaction, a revert, or an exact balance delta.
The Harness resolves native Hedera transaction IDs and EVM hashes through Mirror Node, reports confirmed policy mismatches separately from evidence or execution failures, and feeds fixable findings into the existing repair flow. Actor keys are redacted from persisted reports and prompts. Recipes that omit the new fields keep their existing behavior.
Validation
Review guide
Start with
src/types.tsandsrc/specLoader.ts, then reviewsrc/validation/chainAssertionEvidence.tsandsrc/validation/chainAssertions.ts. The stage and signer integration is insrc/attemptStages.tsandsrc/validation/chainSigner.ts; focused coverage is in thetest/chain-*files.