Design: Migrate JSON API calls to flat `submit-and-wait-for-transaction` (impl: #38)

[gyorgybalazsi]

Design

Implementation issue: #38.
Branch (when work starts): feature/migrate-submit-and-wait-flat.
Approach: single mechanical-sweep PR (Approach A — see "Approach considered, rejected" at the bottom).

Goal

Migrate every Daml-submit call site in cbtc-lib off the deprecated POST /v2/commands/submit-and-wait-for-transaction-tree JSON Ledger API endpoint and onto the flat POST /v2/commands/submit-and-wait-for-transaction endpoint, before Canton 3.5.0 removes the tree endpoint.

Architectural invariant

The PR must preserve identical external behavior. Every public function in cbtc-lib returns the same Rust types, containing the same data, under the same error conditions, before and after the migration. The only observable difference is the network endpoint each call hits and the JSON-shape internal parsing path.

Mechanism

Bump the four canton-lib dependencies (ledger, keycloak, registry, common) from v0.4.0 to v0.5.0, then swap every ledger::submit::wait_for_transaction_tree(...) call for ledger::submit::wait_for_transaction(...) and update the response-parsing JSON path at each site. No new types are introduced in cbtc-lib; no canton-lib changes are made in this PR (canton-lib v0.5.0 ships the helper independently via DLC-link/canton-lib#13).

Scope

In scope.

• All 12 submit call sites listed in Migrate JSON API calls off deprecated submit-and-wait-for-transaction-tree #38, plus the parse_transfer_response helper in transfer.rs.
• README curl example for "Accept Transfer" (README.md:571).
• Two new integration-test steps (split, credentials) in examples/integration_test.rs.
• Bump Cargo.toml and Cargo.lock to canton-lib v0.5.0.

Out of scope.

• No new public functions or types in cbtc-lib's public API.
• No parsing-helper extraction (keep inline parsing loops as today).
• No canton-lib code changes (canton-lib v0.5.0 is built and tagged independently).
• No changes to flows that don't go through wait_for_transaction_tree (read-only contract queries, faucet, etc.).
• No CHANGELOG.md addition (cbtc-lib doesn't currently maintain one).
• No deprecation-window dual-path support (Canton 3.4.x keeps both endpoints; we flip atomically).

The canton-lib v0.5.0 helper

ledger::submit::wait_for_transaction(params: Params) takes the same Params struct as the deprecated wait_for_transaction_tree:

pub struct Params {
    pub ledger_host: String,
    pub access_token: String,
    pub request: Submission,
}

Every cbtc-lib call site changes only the function name — the argument expression is byte-for-byte identical.

Auto-built TransactionFormat. Submission already has a transaction_format: Option<TransactionFormat> field in v0.4.0 (crates/common/src/submission.rs:58 — verified identical in v0.4.0 and v0.5.0/main). The migration does not add the field; it was always there but unused by cbtc-lib. All 12 cbtc-lib sites build their Submission with ..Default::default(), so the field is None at every site without any code edit. When the helper sees None, it builds a default TransactionFormat that mirrors what the deprecated tree endpoint applied server-side:

• transactionShape: "TRANSACTION_SHAPE_LEDGER_EFFECTS"
• eventFormat.verbose: true
• eventFormat.filtersByParty: one wildcard {} entry per party in act_as ∪ read_as

All 12 sites pass non-empty act_as, so this default is safe everywhere. cbtc-lib does not construct any TransactionFormat itself.

Error surface. Result<String, String> — same signature as wait_for_transaction_tree. The helper has one new error path (act_as.is_empty() && transaction_format.is_none()) that is unreachable from every cbtc-lib call site.

Old function still exists. wait_for_transaction_tree remains in canton-lib v0.5.0 as #[deprecated(since = "0.5.0", note = "...")]. After this PR, no cbtc-lib code references it. Successful migration is verified by cargo build emitting zero deprecation warnings from cbtc-lib sources.

Shape change (wire-level, internal to cbtc-lib)

For context — this is what changes on the wire (and therefore inside cbtc-lib's parser loops). It does NOT change cbtc-lib's public outputs.

Today (tree endpoint response):

{
  "transactionTree": {
    "updateId": "...",
    "eventsById": {
      "0": { "CreatedTreeEvent":   { "value": { "templateId": "...", "contractId": "...", "createArgument": {...}, "createdEventBlob": "...", ... } } },
      "1": { "ExercisedTreeEvent": { "value": { "templateId": "...", "choice": "...", "exerciseResult": {...}, ... } } }
    }
  }
}

After migration (flat endpoint response, LEDGER_EFFECTS shape):

{
  "transaction": {
    "updateId": "...",
    "events": [
      { "CreatedEvent":   { "value": { "templateId": "...", "contractId": "...", "createArgument": {...}, "createdEventBlob": "...", "nodeId": 0, ... } } },
      { "ExercisedEvent": { "value": { "templateId": "...", "choice": "...", "exerciseResult": {...}, "nodeId": 1, "lastDescendantNodeId": 5, ... } } }
    ]
  }
}

The value.* payloads (templateId, contractId, createArgument, createdEventBlob, choice, exerciseResult, etc.) are byte-for-byte identical. The differences are:

1. Outer key: transactionTree → transaction.
2. Events container: eventsById object keyed by nodeId → events array.
3. Wrapper names: CreatedTreeEvent/ExercisedTreeEvent → CreatedEvent/ExercisedEvent.
4. New fields on each event (nodeId, exercises also carry lastDescendantNodeId) — none of which cbtc-lib reads. We ignore them.

Per-call-site migration recipe

The same four edits apply at every site, no exceptions.

Edit A: submit call name

- ledger::submit::wait_for_transaction_tree(ledger::submit::Params {
+ ledger::submit::wait_for_transaction(ledger::submit::Params {
      ledger_host: ...,
      access_token: ...,
      request: submission_request,
  })

Some sites import as submit:: instead of ledger::submit:: — preserve whichever alias the site uses.

Edit B: outer JSON path

- response["transactionTree"]["eventsById"].as_object()
+ response["transaction"]["events"].as_array()

And (only in parse_transfer_response):

- response["transactionTree"]["updateId"]
+ response["transaction"]["updateId"]

Edit C: iteration

- for (_key, event) in events_by_id {
+ for event in events {

Edit D: event wrapper names

- event.get("CreatedTreeEvent")
+ event.get("CreatedEvent")

- event.get("ExercisedTreeEvent")
+ event.get("ExercisedEvent")

Everything under value.* stays byte-for-byte identical. No further changes inside matched arms.

Call sites (13 hunks across 12 actual submits)

#	File:line	Kind	Parser changes?
1	src/mint_redeem/mint.rs:160	submit + parse CreatedEvent for :CBTC.DepositAccount:CBTCDepositAccount	yes
2	src/mint_redeem/redeem.rs:189	submit + parse CreatedEvent for :CBTC.WithdrawAccount:CBTCWithdrawAccount	yes
3	src/mint_redeem/redeem.rs:457	submit + reconstruct JsActiveContract from CreatedEvent fields	yes
4	src/accept.rs:122	submit-only (response discarded)	no
5	src/accept.rs:304	submit-only (batch loop)	no
6	src/cancel_offers.rs:123	submit-only	no
7	src/cancel_offers.rs:311	submit-only (batch loop)	no
8	src/transfer.rs:237	submit; parses via parse_transfer_response	no (helper does it)
9	src/transfer.rs:478	submit; parses via parse_transfer_response	no (helper does it)
10	src/transfer.rs:592 parse_transfer_response	helper; reads updateId + walks events for TransferFactory_Transfer	yes (the only updateId site)
11	src/consolidate.rs:246	submit + parse ExercisedEvent, read receiverHoldingCids	yes
12	src/split.rs:112	submit + parse ExercisedEvent, read receiverHoldingCids + senderChangeCids	yes
13	src/credentials.rs:411	submit + parse CreatedEvent for :Utility.Credential.V0.Credential:Credential	yes

Error handling

• Every public cbtc-lib function returns the same Result<T, String> shape, under the same error conditions, with the same error strings — except for two narrow drifts:
  • wait_for_transaction_tree → wait_for_transaction substring in submit-failure error strings. No cbtc-lib caller string-matches on this (verified via grep).
  • .ok_or(...) literals at parser sites change from "Failed to find eventsById in transaction" → "Failed to find events in transaction". These describe JSON-shape contract violations the server should never produce; they don't fire in practice.
• Domain-level "not found" messages (e.g. "No DepositAccount was created in the transaction") stay verbatim.
• The new helper's act_as.is_empty() error path is unreachable from cbtc-lib (every site passes act_as: vec![params.party.clone()] or equivalent). No defensive code added.
• No new Result types. No retries. No fallbacks. No migration-period dual-path logic.

Test additions

Two new steps appended to examples/integration_test.rs, in the existing run_step! style. The current 18-step / 21-step (with faucet) flow already exercises 8 of the 12 migrated sites; these add the two genuinely uncovered parsers.

Step S — Split sender holding

Exercises src/split.rs:112 — the only ExercisedEvent parser that reads both receiverHoldingCids and senderChangeCids from exerciseResult.

• Given the sender has at least one holding (print_skip and continue if not).
• When we call cbtc::split::submit(...) on one input, splitting into N=2 outputs whose values sum to the input.
• Then the result contains the expected new contract IDs, sender's balance is preserved (minus any fees), and cargo build emits zero warnings from the split parser.
• Where: after the existing consolidate step (last step), before the summary.

Step C — Accept free credential offer

Exercises src/credentials.rs:411 (UserService_AcceptFreeCredentialOffer choice via the public function cbtc::credentials::accept_credential_offer).

• Given RUN_CREDENTIAL_ACCEPT=1 env var is set AND the sender's UserService exposes a free credential offer.
• When we call cbtc::credentials::accept_credential_offer(...).
• Then the result is a UserCredential with non-empty contract_id; a follow-up list_credentials includes it.
• Where: after the existing list_credentials read step.
• Default behavior: env var unset → print_skip (because accepting creates persistent ledger contracts with no archive choice; defaulting off matches the existing project stance on test-state accumulation).

Step-count update

Current (examples/integration_test.rs:210):

let base_steps: usize = 18;
let total_steps = base_steps + if faucet_url.is_some() { 3 } else { 0 };

Updated:

let base_steps: usize = 20;   // 18 + split + credential
let total_steps = base_steps + if faucet_url.is_some() { 3 } else { 0 };

(If step C is decided to be "skip-by-default and not counted," use 19; pick at implementation time.)

Not added

• No unit tests with canned-JSON fixtures (explicitly chosen: integration-test extension over fixtures).
• No new test scaffolding, dev-dependencies, or mocking.
• No re-test of mint/redeem/transfer/accept/cancel/consolidate — already covered.

README update

Single block at README.md:571 (### Accept Transfer). Two changes:

1. Endpoint URL:

   - POST $LEDGER_HOST/v2/commands/submit-and-wait-for-transaction-tree
   + POST $LEDGER_HOST/v2/commands/submit-and-wait-for-transaction

2. Request body shape — wrap the current top-level fields in an outer commands object and add transactionFormat:

   - {
   -   "commands": [...],
   -   "commandId": "...",
   -   "actAs": ["..."],
   -   "disclosedContracts": [...]
   - }
   + {
   +   "commands": {
   +     "commands": [...],
   +     "commandId": "...",
   +     "actAs": ["..."],
   +     "disclosedContracts": [...]
   +   },
   +   "transactionFormat": {
   +     "transactionShape": "TRANSACTION_SHAPE_LEDGER_EFFECTS",
   +     "eventFormat": {
   +       "filtersByParty": { "<actAs party>": {} },
   +       "verbose": true
   +     }
   +   }
   + }

If any other curl examples in README.md reach into a sample response with transactionTree.eventsById, update those too. (Spec-writing will not re-grep here; implementation must.)

Cargo.toml + Cargo.lock

Four lines in Cargo.toml:

- ledger   = { git = "ssh://git@github.com/DLC-link/canton-lib", tag = "v0.4.0" }
- keycloak = { git = "ssh://git@github.com/DLC-link/canton-lib", tag = "v0.4.0" }
- registry = { git = "ssh://git@github.com/DLC-link/canton-lib", tag = "v0.4.0" }
- common   = { git = "ssh://git@github.com/DLC-link/canton-lib", tag = "v0.4.0" }
+ ledger   = { git = "ssh://git@github.com/DLC-link/canton-lib", tag = "v0.5.0" }
+ keycloak = { git = "ssh://git@github.com/DLC-link/canton-lib", tag = "v0.5.0" }
+ registry = { git = "ssh://git@github.com/DLC-link/canton-lib", tag = "v0.5.0" }
+ common   = { git = "ssh://git@github.com/DLC-link/canton-lib", tag = "v0.5.0" }

Cargo.lock updates via cargo build and is committed alongside the manifest.

All four crates bump together. Not just ledger. common::submission::Submission is referenced both from cbtc-lib's submission literals AND from ledger::submit::wait_for_transaction's Params. If ledger were v0.5.0 but common stayed v0.4.0, two distinct Submission types would be in scope and call-site argument types would fail to unify. Same logic for keycloak and registry if any of them re-export shared types.

Pin fallback if v0.5.0 isn't tagged when execution starts. Use rev = "f68dd6fc66711d37bddc88e3b999771314ff809a" (the canton-lib#13 merge commit) temporarily. The final commit on the migration branch must reference tag = "v0.5.0" — no rev pin in merged code.

Ship checklist

1. Confirm canton-lib v0.5.0 is tagged (or use the rev fallback above).
2. Pre-flight devnet check. Before opening the PR, confirm the target devnet runs a Canton build that supports POST /v2/commands/submit-and-wait-for-transaction. Easiest: hit the endpoint manually with a known-good submission (curl) or run any flow against devnet from a throwaway branch with only step 3 applied. If the endpoint returns 404/501, halt and escalate.
3. Bump four Cargo.toml lines (ledger, keycloak, registry, common — all together, see rationale above); run cargo build to regenerate Cargo.lock.
4. cargo check --all-targets — zero new errors. This isolates dep-bump breakage from migration breakage: if canton-lib v0.5.0 broke any other imported surface (active_contracts, ledger_end, keycloak::login, etc.) we find out before editing any submit site.
5. Apply the 4-edit recipe (above) to all 13 hunks. Suggested order, grouped by review story:
   • Submit-only (Edit A only): accept.rs:122, accept.rs:304, cancel_offers.rs:123, cancel_offers.rs:311.
   • Transfer flow (Edit A at sites + B/C/D in helper): transfer.rs:237, transfer.rs:478, transfer.rs:592 (parse_transfer_response).
   • Mint / redeem: mint.rs:160, redeem.rs:189, redeem.rs:457.
   • Consolidate / split / credentials: consolidate.rs:246, split.rs:112, credentials.rs:411.
     Each per-file edit can be committed independently. canton-lib v0.5.0 still exports wait_for_transaction_tree as #[deprecated], so the repo compiles (with deprecation warnings) at every intermediate state — only the final commit removes the last reference and turns the warning count to zero.
6. cargo build — must compile with zero deprecation warnings from cbtc-lib sources.
7. cargo clippy --all-targets -- -D warnings if the project uses it (verify at execution).
8. Update README.md:571 curl example.
9. Add the two new integration-test steps.
10. cargo check --example integration_test — clean.
11. cargo run --example integration_test against a devnet on a Canton build that supports the flat endpoint. Both new steps pass or cleanly skip.
12. Open a single PR: feat: migrate JSON API calls off deprecated submit-and-wait-for-transaction-tree (closes #38).

Risks

Risk	Mitigation
canton-lib v0.5.0 not yet tagged at execution time	Use rev pin against f68dd6f...; swap to tag before merge
Devnet still on a Canton build without the flat endpoint	Pre-flight check (ship-checklist step 2) catches this before any code change
Other curl examples in README.md show tree-shaped responses	Implementer re-greps README during step 8 for transactionTree/eventsById and updates any hits
A 13th submit call site exists that grep missed	Ship-checklist step 6 (cargo build with zero deprecation warnings from cbtc-lib) is itself the catch-all — any missed site still calls #[deprecated] wait_for_transaction_tree and triggers a warning

Approach considered, rejected

• Approach B: two PRs split by test coverage. PR 1 migrates the 8 currently-tested sites; PR 2 migrates the remaining 4 + adds the new test steps. Rejected because the parser changes are uniform — if one works, they all work — and the gap PR would emit deprecation warnings.
• Approach C: three PRs by repo state. Pure churn; PR 1 (deps bump alone) provides no standalone value.

Non-goals reiterated

• No canton-lib code changes.
• No public-API changes in cbtc-lib.
• No parsing-helper extraction.
• No CHANGELOG.md entry.
• No deprecation-window dual-path support.

[gyorgybalazsi]

Implementation PR opened: #41