Canon declaration index from transaction scriptSigs

[cdonnachie]

Indexes Canon declarations — signed JSON documents a key publishes about refs it claims — revealed inside a transaction's scriptSig.

Based on feat/hashmark-index (#45), not main. It sits on the shared index wiring that PR introduces (set_indexer, the security allowlist, the BlockProcessor stub's index list). Retarget to main once #45 merges; it rebases cleanly. Reviewing against #45 shows just the one commit.

Why this can't be a metadata backfill

This is the only scanner in the tree that reads inputs rather than outputs, and nothing already in the DB can answer its queries: ElectrumX keeps UTXOs, history and headers, while input scripts survive only inside the reorg window (write_raw_block deletes past it).

So unlike the GCM/GMT/GA/GMH backfills, this one reads blocks back from the daemon. That makes its cost per-block while declarations are per-document and rare, which two things bound:

• DECLARATION_START_HEIGHT skips everything below the format's activation — a document cannot exist there.
• POST /declarations/scan/{txid} indexes one known transaction in two daemon calls.

Scanning from height 0 is a fallback, not the expected path.

sig_valid is tri-state, and that matters

Only schema-valid documents are indexed; the signature never gates indexing. sig_valid is a hint:

value	meaning
true / false	checked
null	UNCHECKED, not invalid

coincurve is an optional dependency, so a node without it must not be able to make a client discard valid declarations — anything collapsing null into false will do exactly that. Structurally bad signatures (non-base64, wrong length) are decided before any EC work and do report false. The message magic is Bitcoin-compatible, confirmed against Radiant-Core's strMessageMagic rather than assumed.

A row means "this document was revealed on chain at this height", never that its claim is true — anyone can sign a document about a ref they don't own, and the signature proves only who authored it. The endpoints return pointers and hints so Canon re-verifies and decides for itself.

Storage

Three prefixes in a fresh DC family, none a prefix of another:

DCd + doc_hash(32)                                  -> record, earliest reveal wins
DCr + ref(36) + height(4) + tx_index(2) + doc_hash  -> kind|action
DCs + signer_hash(16) + height(4) + tx_index(2) + doc_hash

Height leads both secondary key tails, so height-ascending ordering falls out of iteration with no sort. doc_hash is sha256 of the pushed body verbatim, sidestepping any canonical-JSON question. "Unique on doc_hash, earliest wins" is enforced on write: a re-reveal adds its by-ref/by-signer rows but never overwrites the original record.

Versions and revokes

v1 and v2 are both read. Unlike the HashMark record — where v2 moved the label between pushes, so a v1 parser would index a signature as a label — a declaration is JSON keyed by name, so a version that only adds fields stays readable. The parsed version rides on every row so consumers can still tell them apart, and a JSON true is rejected rather than passing as version 1 through Python's bool/int equivalence.

A document may declare, revoke, or both: declares may be empty when the top-level revokes array isn't, and a document doing neither is not indexed. A revoke carries no kind — it withdraws what was declared rather than restating it — so it's stored as empty and read back as null, not the literal string "None".

API

GET  /declarations/by-ref/{72hex}     -> { ref, rows: [...], next_cursor }
GET  /declarations/by-signer/{addr}   -> { signer, rows: [...], next_cursor }
GET  /declarations/stats
POST /declarations/scan/{txid}        -> targeted ingest (API key required)

Defaults off via DECLARATION_INDEX, given the rescan cost.

Tests

40 tests, including the schema check against a real production document, so the live ref, signer, ISO timestamps and action-defaulting are covered by real bytes rather than only synthetic ones: revoke-only and combined documents, the null-kind round trip, height-ascending ordering, earliest-reveal-wins, reorg unwind, the bounded backfill starting at the activation height, and the targeted scan.

pytest wasn't available locally, so these ran through a harness with cbor2/pylru stubbed — the REST routes have not been exercised end-to-end, and the sig_valid: true branch needs an environment with coincurve to cover.