Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 91 additions & 2 deletions docs/audit-evidence.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,10 +165,19 @@ one of them
| `com.fixtures.safe.weather-lookup` | 1.2.0 | **SAFE** | 90 | [`4f5632c3e1e805ee…`](https://stellar.expert/explorer/testnet/tx/4f5632c3e1e805ee7ac4ae595e8c237d43ea2fac807c5f81fcc322eb3814a030) | [`e0c587e1f4363958…`](https://stellar.expert/explorer/testnet/tx/e0c587e1f43639582b0774bc584b288e00cc791288a164d969e1752030bdce81) | [`fbc4c297ca5f3ad1…`](https://stellar.expert/explorer/testnet/tx/fbc4c297ca5f3ad1f117a8073a8c1cfdc86bf01eb27554f9667353b8c3f07670) |

`com.fixtures.safe.price-checker` landed as `DANGEROUS`, and **that is wrong**. The `wallet_op`
detector is negation-blind, so the sentence *"it never touches a wallet, never signs anything,
detector was negation-blind, so the sentence *"it never touches a wallet, never signs anything,
never moves funds"* is what condemned it. It was left visible on chain rather than quietly
dropped from the batch: this is our own fixture, not an accusation against anyone else, and a
false-positive rate that gets hidden helps no one. Tracked in STE-37.
false-positive rate that gets hidden helps no one.

**Update, STE-37 (15 September 2026): the scanner is fixed; the row above is not.** `wallet_op`
now fires only on a sentence that directs a move of assets, so `price-checker` audits `SAFE`
(score 90) offline, while all four poisoned fixtures stay `DANGEROUS`. Measured over the corpus:
0 false positives among 16 benign entries, 0 false negatives among 4 poisoned ones. The verdict
**on chain** is still `DANGEROUS` until the correction below lands. `submit_verdict` overwrites
the record for a `(skill_id, version)`, so the fix is one re-audit transaction; the original
`75055115…` stays in history as the record of what was published and when. The correction is in
STE-37's own scope (see [Correcting the chain after STE-37](#correcting-the-chain-after-ste-37-and-ste-39)).

## `content_hash` and `evidence_hash`

Expand Down Expand Up @@ -260,6 +269,39 @@ convenient version of the principle rather than the principle.
Consequence: the catalogue is 13 of 13, with **zero exclusions**, and the SOW D2 threshold of
"10+ real catalogue skills" is met with 13.

This section is kept as it was written. What follows is added beneath it rather than replacing
it: an audit registry has to be able to show that it publishes the answer it got, says in public
when it believes that answer is wrong, and then corrects it — in that order, with dates.

**Update, STE-37 (16–17 September 2026): the reading above was right, and the verdict is
corrected.** With `wallet_op` reading whether a sentence *directs* a move of assets rather than
whether transfer vocabulary appears anywhere, cctp audits `SAFE` (score 100): both findings
disappear, and all four poisoned fixtures stay `DANGEROUS`. The on-chain `DANGEROUS` is replaced
by a re-audit transaction rather than left standing beside a fix already made in code. The
`542468dc…` transaction above stays in history as what we published on 15 September.

The protocol's own documentation says the same thing the skill said. Circle's Stellar contract
reference (<https://developers.circle.com/cctp/references/stellar-contracts>, read 16 September
2026) describes `TokenMessengerMinter` as *"Burns USDC and emits a crosschain message for minting
on another domain"* — first-party, burn-and-mint phrasing for a protocol whose contracts are live
on Stellar:

| Circle contract | testnet | mainnet |
|---|---|---|
| `TokenMessengerMinter` | `CDNG7HXAPBWICI2E3AUBP3YZWZELJLYSB6F5CC7WLDTLTHVM74SLRTHP` | `CAE2G5Z77UP7GYPYGFOWFGW7C7J6I4YP2AFGSADRKQY62SYUFLPNFTXL` |
| `MessageTransmitter` | `CBJ6MTCKKZG73PMDZCJMSFRD7DQEMI4FKDH7CGDSV4W6FHCRBCQAVVJY` | `CACMENFFJPJMSDAJQLX4R7K3SFZIW2LJSE3R2UMLGSWHFHS353FVXAZV` |
| `CctpForwarder` | `CA66Q2WFBND6V4UEB7RD4SAXSVIWMD6RA4X3U32ELVFGXV5PJK4T4VSZ` | `CBZL2IH7F6BIDAA3WBNXYKIXSATJGMSW7K5P5MJ6STX5RXN47TZJDF5T` |

So the sentence the detector condemned was not a careless skill author; it is the standard way the
company that built the protocol describes it. That is the distinction the fixed detector exists to
make — *describing* a burn-and-mint protocol versus *instructing* an agent to move funds — and
Circle's sentence is pinned as a benign case in `tests/test_wallet_op_context.py`. (Checked, not
assumed: that exact sentence never fired even before the fix, because "burns" is not a transfer
verb. What fired was the skill's "Cross-Chain **Transfer** Protocol" and "**moves** USDC by
burning it", both also pinned. Circle's sentence is kept so a future verb list that adds
burn/mint cannot start accusing the protocol's own documentation.) See
[Correcting the chain after STE-37](#correcting-the-chain-after-ste-37-and-ste-39).

## The demo set: four skills, four states

Before this run the registry held **0 WARNING, 0 UNAUDITED and 0 stale versions**, so three of
Expand Down Expand Up @@ -547,3 +589,50 @@ notes, so the same audit produced different bytes depending on whether a key was
which ends the config dependence for good — and shifts the bytes one last time, so at the commit
that merges STE-39 all 25 reports need re-anchoring once more. That re-anchor rides on the
re-submission STE-37 already requires for `cctp` and `price-checker`.

### Correcting the chain after STE-37 and STE-39

One seed run does both jobs, because `submit_verdict` overwrites the record for a version and the
orchestrator re-submits whenever verdict, score **or** `evidence_hash` differ. It is rehearsed first
with `intake seed --dry-run`, which since STE-37 reads each version back from the registry and
names the write a real run would make, together with the hash it would anchor — the same bytes
`reports.publish` writes, computed without writing them.

Dry run against Registry v2 `CCZJN366SV57JEBZVXGYY3ZBLJNFV4IR5ILCAI3EMX2WDNQPEPQ4BRL2`, 16 September
2026, on `fix/37-wallet-op-negation` rebased on `dc0a0c6` (STE-39 merged), no LLM key, nothing
signed ([`evidence/ste-37-dry-run-reanchor-v2-2026-09-16.json`](evidence/ste-37-dry-run-reanchor-v2-2026-09-16.json)):

| entries | on chain now | after | write |
|---|---|---|---|
| `com.fixtures.safe.price-checker` 0.9.0 | DANGEROUS 10 | **SAFE 90** | `submit_verdict` (verdict, score, hash), then `mint_verified` |
| `org.stellar.skills.cross-chain.cctp` 2026.8.31 | DANGEROUS 10 | **SAFE 100** | `submit_verdict` (verdict, score, hash), then `mint_verified` |
| the other 23 audited entries | unchanged verdict and score | unchanged | `submit_verdict` (hash only — the STE-39 byte shift) |
| `com.fixtures.demo.changelog-writer` 2.0.0 (register-only) | UNAUDITED | UNAUDITED | none |

Before the run, all 25 committed `reports/` files still hash to what v2 holds, so nothing is
already out of step. The run itself happens only after STE-37 merges, from `main`, with `REGISTRY_CA`, `TOKENS_CA` and
the signer secrets loaded from the root `.env`:

```bash
cd pipeline
uv run --project . sterish intake seed --corpus corpus \
--label catalog --label safe --label poisoned --label demo --allow-dangerous \
--reports-dir ../reports --json-out ../docs/evidence/ste-37-reanchor-<date>.json
```

`--allow-dangerous` is needed for the poisoned and demo rows, which are ours and are meant to be
`DANGEROUS`. Since STE-37 it is not a blanket: it publishes a `DANGEROUS` verdict **only where the
corpus's own `expected_verdict` is `DANGEROUS`**, and every such row is marked
`dangerous_intended` with its reason in the run log (and listed at the end of the console output).
A `DANGEROUS` the corpus did not expect, e.g. a catalogue skill after a detector regression, stays
held with or without the flag.

**If you recorded an `evidence_hash` before 17 September, it will no longer match — and that is not
tampering.** This run replaces the `evidence_hash` of 23 entries whose skill bytes, verdict and score
did **not** change. The verdict document they anchor changed shape: STE-39 (merged 16 September
2026) took the model's advisory trail out of the hashed report, so the same audit now serialises
to different bytes. Nothing was re-judged for those 23. Their `content_hash` is untouched, and
the previous `evidence_hash` stays readable in the history of each `submit_verdict` transaction;
the report that hashes to it is the `reports/` file at the commit before this run. Only
`price-checker` and `cctp` change verdict. The rewritten `reports/` files are committed with the run log, and the API
redeployed so `/reports` serves the bytes the chain now anchors.
Loading
Loading