Skip to content

[POC BDK] Introduce modular trait-based watch-only wallet architecture#950

Draft
moisesPompilio wants to merge 5 commits into
getfloresta:masterfrom
moisesPompilio:add-bdk
Draft

[POC BDK] Introduce modular trait-based watch-only wallet architecture#950
moisesPompilio wants to merge 5 commits into
getfloresta:masterfrom
moisesPompilio:add-bdk

Conversation

@moisesPompilio
Copy link
Copy Markdown
Collaborator

Description and Notes

This PR introduces a proof of concept for integrating BDK into Floresta by restructuring the watch-only wallet architecture into a more modular and specialized design.

The new architecture splits responsibilities across three main layers:

  • Service (Wallet): provides the high-level API and coordinates wallet operations
  • Repository: handles persistence and wallet data storage
  • Provider: handles descriptor-level wallet logic and blockchain-related information

It also adds shared models used for communication between layers, such as richer balance structures that expose more detailed information like trusted, used, and untrusted values.

All layers communicate through traits, which makes the architecture more flexible and extensible. This allows multiple implementations for the same layer, for example:

  • different repositories, such as SQLite or in-memory
  • different providers, such as a BDK-backed provider or a native Floresta provider
  • third-party implementations of custom providers or repositories

For testing, the idea is to build an integration-test architecture around the traits themselves. I already started this approach in the provider layer, and in the service layer I left some usage examples to illustrate the intended behavior. The goal is to validate that the expected behavior of the repository, provider, and service traits remains consistent regardless of the concrete implementation being used.

With this approach, implementation-specific code should not need to duplicate unit tests for the core behavior already defined by the traits. Instead, concrete implementations would only need tests for auxiliary or internal helper functions that are specific to that implementation, similar to what is already done in the provider layer. This helps ensure that, no matter which Floresta implementation is plugged in, the same rules and expectations apply across the board.

In addition, this PR includes a short documentation update in crates/floresta-watch-only/README.md to explain how the new design works.

Notes for reviewers:

  • This is a proof of concept aimed at making the watch-only architecture more modular and easier to extend.
  • The goal is to define stable trait-based contracts so that different implementations can follow the same expected behavior.
  • The service layer is responsible for orchestrating the lower layers, while the provider and repository focus on their own specialties.
  • this PR bumps Rust to 1.85.0, which is the minimum version supported by bdk

How to verify the changes you have done?

  • Verify the new wallet architecture and confirm that the service, repository, and provider layers are separated as intended.
  • Run the unit tests and make sure they pass successfully.
  • Review the unit test scenarios to confirm they cover the expected behavior of the implemented traits.
  • Run the integration tests, when available, to validate that the trait implementations.
  • Check that the provider tests cover descriptor management, transaction handling, balance calculations, and event processing as expected.

…anagement

The provider layer establishes a new abstraction for managing all descriptor-related
operations in the watch-only wallet. It centralizes responsibility for tracking wallet
state across descriptors, including address derivation, transaction indexing, and balance
calculations. The provider serves as the single source of truth for descriptor-scoped
information, isolating this logic from higher-level wallet coordination.

Key responsibilities:
- Manage descriptor persistence and retrieval
- Track generated and observed addresses per descriptor
- Index transactions associated with each descriptor
- Maintain UTXO sets and output tracking
- Calculate balances on a per-descriptor basis
- Process blockchain events (blocks, mempool) and emit descriptor-specific events

- Added `WalletProvider` trait defining the provider interface
- Introduced `WalletProviderEvent` enum for event-driven transaction notifications
- Defined `WalletProviderError` for comprehensive error handling
- Implemented feature-gated BDK provider backend via `bdk-provider` feature

test(provider): add comprehensive provider unit tests

Established test coverage for the provider interface, validating core descriptor and
transaction management operations.

Test scenarios:
- Descriptor lifecycle (persist, retrieve, list, deduplicate)
- Transaction indexing and querying by descriptor
- Balance calculations with confirmation requirements
- Address generation and management
- UTXO tracking and spend status filtering
- Mempool and block event processing
- Edge cases (empty wallets, nonexistent descriptors, duplicate operations)
- Script buffer management and local output tracking
…sistence

The repository layer establishes a higher-level persistence abstraction for the watch-only
wallet, centralizing all data storage operations. It manages wallet metadata, descriptor
configurations, transaction indexing, and script tracking—providing a clean interface
between the wallet service and the underlying database backend.

Core responsibilities:
- Persist wallet names and lifecycle management
- Store descriptors associated with each wallet with their metadata (active status, change flag, labels)
- Maintain descriptor configurations and derivation information
- Index and retrieve transactions for Electrum protocol support
- Track script buffers and derive addresses for transaction monitoring
- Provide auxiliary transaction data structures needed for Electrum responses

Implementation:
- Added SQLiteRepository backed by rusqlite with migration-based schema initialization
- Designed comprehensive WalletPersist trait defining the repository interface
- Implemented wallet CRUD operations supporting multi-wallet environments
- Created database schema with normalized tables for wallets, descriptors, transactions, and script buffers
- Established foreign key constraints for data integrity

Test coverage:
- Wallet creation, listing, and deletion operations
- Descriptor lifecycle (persist, retrieve, update, deduplication)
- Multi-wallet descriptor management and isolation
- Transaction indexing and querying
- Script buffer operations and state tracking
- Edge cases and error conditions

Dependencies:
- rusqlite: SQLite driver for Rust
- refinery: Database migration management
…n and lifecycle management

The wallet service establishes the coordination layer between the provider, repository, and
metadata layers, orchestrating wallet operations and managing the complete wallet lifecycle.
It serves as the primary interface for wallet clients (such as Electrum servers), translating
high-level operations into coordinated calls across the underlying layers while maintaining
consistent wallet state.

Architecture & Responsibilities:

*Provider Integration:*
- Queries descriptor-specific transaction data and balance information
- Retrieves address generation and UTXOs for each descriptor
- Processes blockchain events and emits descriptor-scoped notifications

*Repository Integration:*
- Persists wallet metadata (name, creation, deletion)
- Stores descriptor configurations with metadata (active flag, change flag, labels)
- Maintains transaction index for Electrum protocol responses
- Tracks script buffers and historical transaction data

*Metadata Management:*
- Maintains in-memory wallet state and descriptor registry
- Administers descriptor lifecycle (add, activate, deactivate, replace)
- Handles descriptor state transitions when adding new descriptors
- Enforces single active descriptor per category (external/change)

*Core Operations:*
- Wallet creation and loading from persistent storage
- Descriptor management with automatic deactivation of replaced descriptors
- Block and mempool transaction processing with event propagation
- Balance calculations aggregating across descriptors
- Transaction history and proof retrieval for Electrum clients
- Address generation delegated to active descriptors

Implementation:
- Implemented `Wallet` trait defining the complete service interface
- Multi-layered error handling with service-specific error types
- RwLock-based concurrency for thread-safe metadata access
- Deterministic descriptor ID generation via SHA256 hashing
- Add architecture overview with three-layer design explanation
- Include Mermaid class diagram showing trait relationships
- Add sequence diagram for block processing data flow
- Provide practical usage examples (wallet creation, descriptors, blocks, queries)
- Document feature flags (bdk-provider, sqlite) and combinations
- Detail error types, concurrency model, and development guidelines
- All content in English with clear code examples
@moisesPompilio
Copy link
Copy Markdown
Collaborator Author

In this case, what is in service.rs will basically replace everything from lib.rs, however I didn't put it there so the code can still compile in case someone wants to test something

@moisesPompilio moisesPompilio self-assigned this Apr 9, 2026
@moisesPompilio moisesPompilio added reliability Related to runtime reliability, stability and production readiness ecosystem support Enable interoperability, compatibility and practical integration with the broader Bitcoin ecosystem labels Apr 9, 2026
Copy link
Copy Markdown
Member

@Davidson-Souza Davidson-Souza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pretty much like the direction proposed here. Just a couple questions, specially about the interfaces (I might drop a few ones as I think more about it 🙃).

@csgui @jaoleal could you guys give a look at this. I would like to see a decision on this ASAP, so @moisesPompilio can strip the BDK part for now and get this going.

@jaoleal specifically, there's some changes here that I think will make your lives way better on the RPC side, and will help us implementing most of the rawtransaction category from Core. Please double check my assumptions here.

height: u32,
) -> Result<Vec<WalletProviderEvent>, WalletProviderError>;

fn get_transaction(&self, txid: &Txid) -> Result<Transaction, WalletProviderError>;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On get_* methods I usually like to use Result<Option<T>> because in several cases, "we can't find it" and "we can't get it" are two different code paths. The former means everything is working, but we really don't have that info. While the latter means something isn't working, therefore we shouldn't continue doing what we are doing, and possibly even try to recover/HCF.

We may find ourselves matching against a TransactionNotFound variant in several places because we can't distinguish between the two.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, in my case, I was used to the idea that when you send a request and the element isn’t there, that should indicate NotFound. But I agree with this approach of returning an Option, and letting the higher layers decide what to do with that information if it isn’t found.

Comment on lines +215 to +219
fn create_transaction(
&self,
ids: HashSet<String>,
address: &str,
) -> Result<(), WalletProviderError>;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking out loud: perhaps we could break the createtransaction side into another trait? Maybe one trait for address keeping and another for creation, then Provider ties them up?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. In this case, I did it that way just to have something simple for the user to send to an address, but this transaction-creation part will need to be split up more so we can allow more customization, like coin selection, sending to multiple addresses, or even sending to hex, which would be the output script.

Comment on lines +182 to +186
fn block_process(
&self,
block: &Block,
height: u32,
) -> Result<Vec<WalletProviderEvent>, WalletProviderError>;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would there be any usefulness in giving proofs/inputs here? Do we want to keep utreexo proofs? Can inputs be useful for wallet somehow? If we support Silent Payments, they sure are.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we can add more events. In this case, the event could expose the important information, while the upper layer would be responsible for building and storing anything it needs, such as proofs or any other metadata that the provider itself does not persist.

Comment on lines +234 to +238
fn get_txo(
&self,
outpoint: &OutPoint,
is_spent: Option<bool>,
) -> Result<Option<TxOut>, WalletProviderError>;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, should we really require our caller to know this?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can be removed. I only implemented it because there was a method like that in the wallet.

@jaoleal
Copy link
Copy Markdown
Member

jaoleal commented May 7, 2026

@jaoleal specifically, there's some changes here that I think will make your lives way better on the RPC side, and will help us implementing most of the rawtransaction category from Core. Please double check my assumptions here.

Sorry, since i had my setup ready here im dealing with my huge backlog... Ill dedicate this next saturday (may 9th) for this.

@jaoleal
Copy link
Copy Markdown
Member

jaoleal commented May 10, 2026

RFC: Direction for the watch-only wallet architecture

What I think #950 gets right

A few things worth calling out before digging in, because the questions below are about going further on the same direction, not reverting any of it:

  • Three layers, trait-backed. Service / Provider / Repository is a sensible decomposition. Each being a trait means we can swap providers, swap persistence backends, and stub things in tests.
  • BDK as the first concrete provider. Reusing bdk_wallet for descriptor logic is the right call — we don't want to be reimplementing keychain/derivation machinery.
  • Shared models module. Balance, LocalOutput, GetBalanceParams, LastProcessedBlock lifted out of any one layer.
  • README documenting intent. The class diagram and responsibility split in crates/floresta-watch-only/README.md are at the right altitude for a layered-architecture doc.

Architectural questions surfaced by #950

Six points where I think there's room to push the modularity further. Framing them as questions, not verdicts.

1. The Provider module imports BDK at its public surface

crates/floresta-watch-only/src/provider/mod.rs:

#[cfg(feature = "bdk-provider")]
use bdk_wallet::rusqlite::Connection;

The new_provider factory in the same file hardcodes BdkWalletProvider<Connection, KeyId>. This means the provider namespace itself knows BDK exists — a future non-BDK provider isn't a peer of bdk_provider.rs, it's something mod.rs would need to learn about.

Question: should the provider module only know the trait, with new_provider (and any factory) living inside bdk_provider.rs itself? That keeps the provider trait genuinely backend-agnostic.

2. Descriptor data is duplicated between Provider and Repository, with no atomicity

push_descriptor writes to two places:

let existing_descriptor = self
    .persister
    .exists_descriptor(Some(&descriptor.id), None)?;

if !existing_descriptor {
    self.get_provider_mut()?
        .persist_descriptor(&descriptor.id, &descriptor.descriptor)?;
}
self.persister.insert_or_update_descriptor(&descriptor)?;

There's no transaction crossing the two backends. The provider write is conditional on the descriptor being new (exists_descriptor check), but the repository write happens unconditionally — so even in the steady-state path where they should agree, a failure in insert_or_update_descriptor leaves BDK with the descriptor and the repository without it. Next loaddescriptors call won't list it; BDK will keep deriving addresses; state divergence survives restart.

Plus the data itself overlaps: DbDescriptor carries the descriptor string, is_active, is_change, label — but BDK also stores the descriptor string internally.

Question: who owns descriptor strings? My intuition is the Provider owns them (since it has BDK's persister anyway), and the Repository owns only what's exclusively a Floresta concept — Merkle proofs, the script_hash → script index, and wallet configuration metadata. That's a stricter boundary than the current PR draws.

3. load_wallet is single-wallet with a destructive swap

let wallet_metadata = WalletMetadata::new(...);
let mut metadata = self.get_metadata_mut_not_validated()?;
*metadata = wallet_metadata;

If a client calls loadwallet("alice") then loadwallet("bob"), the in-memory state for "alice" is gone. If a block arrives between the two calls, it gets processed against "bob" even when it carries txs for "alice".

Bitcoin Core has been multi-wallet for years (-wallet=..., loadwallet, unloadwallet, listwallets). If RPC parity is on the roadmap, multi-wallet becomes load-bearing.

Question: should the metadata layer hold all loaded wallets indexed by ID, with load_wallet adding to a set instead of replacing a slot?

4. WalletProvider is a 17-method super-trait

Different consumers need different slices:

  • the BlockConsumer integration needs block_process
  • the Electrum server needs get_local_output_by_script, get_balances, list_script_buff
  • the JSON-RPC loaddescriptors path needs persist_descriptor, new_address
  • a future fee/coin-selection consumer needs none of the above

Today everyone gets &dyn WalletProvider (or has to). Tests have to mock the entire surface.

Question: would interface segregation help here — separate BlockProcessor / BalanceQuery / AddressGenerator / DescriptorRegister traits with the current trait composing them as a super-trait?

5. The descriptor-ID policy is buried in the service

generate_id_for_descriptor (sha256 of the descriptor string) lives at the bottom of service.rs. But that hash is the key both the Provider and the Repository use — the Provider stores descriptors by it, the Repository's DbDescriptor indexes by it, the service derives it on every push_descriptor.

A shared policy living in only one of the three callers is implicit coupling.

Question: should this be a DescriptorId newtype in models.rs with a single canonical constructor, used everywhere?

6. Likely bug in BlockConsumer::on_block

Not an architectural question, just something I noticed:

fn on_block(&self, block: &Block, height: u32, ...) {
    // We only process block if the wallet is initialized,
    if self.persister.exists_descriptor(None, None).unwrap_or(false) {
        return;
    }
    self.process_block_inner(block, height)...
}

The comment says "process if initialized" but the code returns when a descriptor exists — so blocks would only be processed for uninitialized wallets. Either the comment is wrong or there's a missing !. Worth flagging independently of the larger discussion.

Proposed direction

If we accept those questions as worth answering, the shape that falls out looks roughly like this:

flowchart TD
    subgraph clients ["External clients"]
        direction LR
        B1["<b>[B1]</b> floresta-chain<br/><i>block source</i>"]
        B2["<b>[B2]</b> JSON-RPC server<br/><i>admin commands</i>"]
        B3["<b>[B3]</b> Electrum server<br/><i>wallet queries</i>"]
        B4["<b>[B4]</b> libfloresta<br/><i>embedded users</i>"]
    end

    subgraph orchestrator ["Orchestrator [O] — single writer, single gateway"]
        direction TB
        O1["<b>[O1]</b> Handler<br/><i>validates · builds events · notifies clients</i>"]
        O2["<b>[O2]</b> Event queue (FIFO, append-only)"]
        O3["<b>[O3]</b> Processor — single writer<br/><i>drains queue · talks to P + R</i>"]
        M["<b>[M]</b> WalletIndex — multi-wallet<br/><i>HashMap&lt;WalletId, WalletState&gt;</i><br/><i>routing: by_script_hash, by_descriptor</i><br/><i>owned exclusively by [O3] — no locks</i>"]

        O1 --> O2
        O2 --> O3
        O3 -.->|owns| M
    end

    subgraph provider ["Provider [P] — BDK-backed, multi-wallet"]
        direction TB
        P1["<b>[P1]</b> trait WalletProvider"]
        P2["<b>[P2]</b> BdkProvider<br/><i>HashMap&lt;WalletId, BdkWallet&gt;</i>"]
        P3["<b>[P3]</b> owned domain<br/><i>descriptors · tx graph · UTXOs · derivation</i>"]
    end

    subgraph repository ["Repository [R] — Floresta metadata only"]
        direction TB
        R1["<b>[R1]</b> trait WalletRepository"]
        R2["<b>[R2]</b> SqliteRepository<br/><i>no descriptor data</i>"]
        R3["<b>[R3]</b> owned domain<br/><i>WalletConfig · MerkleProof · script index</i>"]
    end

    B1 --> O1
    B2 --> O1
    B3 --> O1
    B4 --> O1

    O3 --> P1
    O3 --> R1
Loading

Plural clients, single gateway

The BlockConsumer integration is one of several entry points. But we also have the JSON-RPC server, the Electrum server, and library users embedding libfloresta directly. A single Wallet trait fronting all of them tends toward the god-trait shape.

We also need to consider future Client cases even if they are somewhat blurry. Bitcoin Core's multiprocess/IPC effort has been steadily landing PRs and will likely show up in Floresta sooner or later. Other clients that come to mind: silent payment scanning (which is itself essentially a watch-only wallet pattern), LSP on-chain monitoring, and ASP VTXO tracking — all multi-wallet by nature, all benefiting from a typed command surface rather than a per-integration glue layer. These last ones are shower thoughts — not asking for them to be designed in, just worth keeping the interfaces flexible enough not to preclude them.

Inverting it: one orchestrator, accepting typed commands from any client.

Orchestrator instead of Service

Renaming "Service" to "Orchestrator" because that's what the role is once it grows: not service-business-logic, but the gatekeeper that serializes writes, owns the in-memory index, and routes to the persistence backends.

Internally it splits into three pieces:

  • a Handler that validates incoming WalletCommands and notifies clients of results
  • a FIFO event queue between Handler and Processor (mpsc channel)
  • a Processor that drains the queue, holds exclusive ownership of the wallet index, and is the single writer to Provider and Repository

The single-writer property is what kills the current RwLock<Box<dyn WalletProvider>> + RwLock<WalletMetadata> pattern. If only one task ever writes, the locks aren't needed for write coordination — only for cross-task read access, and even those go away if reads use a published snapshot.

The shape is heavily inspired by how floresta-wire already orchestrates the node internally — a task draining a channel and dispatching to handlers. The trait extraction of that interface is being landed in #1035, where Davidson sketches the same super-trait + sub-traits decomposition that Q4 above proposes for WalletProvider.

Multi-wallet index inside the Processor

Renaming "Metadata" to "WalletIndex" because that's the role: a router from (WalletId, ScriptHash, DescriptorId) to the right per-wallet state. Because the Processor is the only writer, the index doesn't need its own lock. Multi-wallet falls out of having HashMap<WalletId, WalletState> instead of a single Option<active>.

Tighter boundary between Provider and Repository when the Provider holds its own Database, like BDK as it is the example here.

Provider owns: descriptors (the strings), tx graph, UTXOs, balance computation, derivation indices. Repository owns: wallet config (IDs and refs, not descriptor strings), Merkle proofs, the script_hash → ScriptBuf index for Electrum responses. No overlap. push_descriptor becomes provider.add_descriptor() followed by an index update — the Repository never sees the descriptor string.

Typed interfaces, no String keys

The current PR's external interfaces use String for wallet names, descriptor IDs, and labels, plus u32 for heights and bool for is_change. Replacing those with newtypes (WalletId, DescriptorId, BlockHeight, KeychainKind) costs almost nothing in code and pushes a class of mistakes to compile time.

Type catalog

The catalog isn't a final spec — most signatures have alternative shapes worth discussing (see open questions below). It's a starting point for what "bureaucratic interfaces" looks like in practice.

Identifier newtypes

Substitutes for String / u32 / bool at interface boundaries.

[T1] WalletId

pub struct WalletId(pub Uuid);

[T2] DescriptorId

pub struct DescriptorId(pub sha256::Hash);
// canonical: sha256 of normalized descriptor

[T3] BlockHeight

pub struct BlockHeight(pub u32);
// ord-aware, saturating arithmetic

[T4] KeychainKind

pub enum KeychainKind {
    External,
    Internal,
}

[T5] WalletName

pub struct WalletName(String);
// validated: 1..=64 chars [A-Za-z0-9_-]
// constructed via WalletName::new(s) -> Result<Self, _>

Public API — what clients submit

[Cmd] enum WalletCommand

pub enum WalletCommand {
    CreateWallet      { name: WalletName },
    LoadWallet        { id: WalletId },
    UnloadWallet      { id: WalletId },
    PushDescriptor    { wallet: WalletId, desc: ImportDescriptor },
    GetDescriptor     { wallet: WalletId, desc: DescriptorId },
    ProcessBlock      { block: Block, height: BlockHeight },
    ProcessMempoolTx  { txs: Vec<Transaction> },
    NewAddress        { wallet: WalletId, kind: KeychainKind },
    GetBalance        { wallet: WalletId, params: GetBalanceParams },
    GetHistory        { wallet: WalletId, script: ScriptHash },
    GetUtxo           { wallet: WalletId, outpoint: OutPoint },
}

[O1] trait Handler

pub trait Handler: Send + Sync {
    fn submit(&self, cmd: WalletCommand)
        -> impl Future<Output = Result<WalletResponse, WalletError>>;

    fn submit_batch(&self, cmds: &[WalletCommand])
        -> impl Future<Output = Result<Vec<WalletResponse>, WalletError>>;
}

pub enum WalletResponse {
    Ack,
    WalletCreated(WalletId),
    DescriptorAdded(DescriptorId),
    Address(Address),
    Balance(Balance),
    History(Vec<TxRecord>),
    Utxo(Option<TxOut>),
}

This connects with the Result<Option<T>> suggestion made earlier on the PR. At the JSON-RPC boundary, NotFound from the orchestrator translates naturally to Ok(None) — keeping the orchestrator's Result<WalletResponse, WalletError> strict while letting client-facing layers express absence as Option. The two suggestions compose.

Orchestrator internals — exclusive to the Processor

[O2] struct WalletEvent

pub struct WalletEvent {
    pub cmd: WalletCommand,
    pub reply: ResponseTx,
}

pub type ResponseTx = oneshot::Sender<Result<WalletResponse, WalletError>>;
pub type EventTx    = mpsc::Sender<WalletEvent>;
pub type EventRx    = mpsc::Receiver<WalletEvent>;

[O3] struct Processor

pub struct Processor {
    rx: EventRx,
    index: WalletIndex,
    provider: Box<dyn WalletProvider>,
    repo: Box<dyn WalletRepository>,
}

impl Processor {
    pub async fn run(self) {
        // drain rx, dispatch to provider/repo, reply via oneshot
    }
}

[M] struct WalletIndex + WalletState

pub struct WalletIndex {
    wallets:       HashMap<WalletId,     WalletState>,
    by_script:     HashMap<ScriptHash,   WalletId>,
    by_descriptor: HashMap<DescriptorId, WalletId>,
}

pub struct WalletState {
    pub id: WalletId,
    pub name: WalletName,
    pub active_external: Option<DescriptorId>,
    pub active_internal: Option<DescriptorId>,
    pub descriptors:     Vec<DescriptorId>,
}

Provider — owns descriptors, tx graph, UTXOs

[P1] trait WalletProvider

pub trait WalletProvider: Send + Sync {
    fn add_descriptor(&mut self, wallet: WalletId, desc: ParsedDescriptor)
        -> Result<DescriptorId, ProviderError>;

    fn process_block(&mut self, wallet: WalletId, block: &Block, height: BlockHeight)
        -> Result<Vec<ProviderEvent>, ProviderError>;

    fn balance(&self, wallet: WalletId, params: GetBalanceParams)
        -> Result<Balance, ProviderError>;

    fn next_address(&mut self, wallet: WalletId, kind: KeychainKind)
        -> Result<Address, ProviderError>;

    fn get_utxos(&self, wallet: WalletId, script: ScriptHash)
        -> Result<Vec<LocalOutput>, ProviderError>;
}

pub enum ProviderEvent {
    OwnedOutputFound { tx: Transaction, output: TxOut, position: u64 },
    TxConfirmed      { txid: Txid, position: u64 },
    TxInMempool      { tx: Transaction },
}

Repository — Floresta-only metadata

[R1] trait WalletRepository

pub trait WalletRepository: Send + Sync {
    fn upsert_wallet_config(&self, cfg: &WalletConfig)
        -> Result<(), RepoError>;
    fn get_wallet_config(&self, id: WalletId)
        -> Result<WalletConfig, RepoError>;
    fn list_wallets(&self)
        -> Result<Vec<WalletId>, RepoError>;
    fn delete_wallet(&self, id: WalletId)
        -> Result<(), RepoError>;

    fn upsert_merkle_proof(&self, txid: Txid, proof: MerkleProof)
        -> Result<(), RepoError>;
    fn get_merkle_proof(&self, txid: Txid)
        -> Result<MerkleProof, RepoError>;

    fn upsert_script(&self, script: ScriptBuf)
        -> Result<(), RepoError>;
    fn get_script(&self, hash: ScriptHash)
        -> Result<ScriptBuf, RepoError>;
}

[R3] struct WalletConfig

pub struct WalletConfig {
    pub id: WalletId,
    pub name: WalletName,
    pub descriptors: Vec<DescriptorRef>,
}

pub struct DescriptorRef {
    pub id: DescriptorId,
    pub label: Option<String>,
    pub is_change: bool,
}

[Err] error enums

pub enum WalletError {
    Provider(ProviderError),
    Repository(RepoError),
    NotLoaded(WalletId),
    InvalidName(String),
    QueueFull,
    LockPoisoned,
}

pub enum ProviderError { /* domain-specific variants */ }
pub enum RepoError    { /* persistence-specific variants */ }

Open questions

Things I don't have a settled view on. The whole point of posting this is to hear pushback:

  • async fn submit vs sync at the Handler boundary. Drafted async because the queue is a natural fit, but a sync submit returning a oneshot::Receiver<Result<...>> is also clean. Tokio everywhere already, so async isn't an imposition — but it's a real choice.

  • Box<dyn WalletProvider> vs generics in the Processor. Trait objects are simpler and let Processor carry uniform fields. Generics give us monomorphisation and let the compiler inline through the boundary, at the cost of compile time and binary size. I lean trait objects for this codebase but it's a real trade-off.

  • WalletId representation. Uuid is what I drafted — stable, opaque, doesn't carry user data. A validated String (the wallet name itself) is shorter and more debuggable but couples ID and name.

  • BDK 2.x maturity. [POC BDK] Introduce modular trait-based watch-only wallet architecture #950 bumps Rust to 1.85 to pull in bdk_wallet. How comfortable are we betting the descriptor logic on BDK at its current state? On multi-wallets setup the support still needs to land there, perhaps we could work with it in mind but with an implementation of ours.

  • Migration path. Two viable approaches:

Asks

If you've read this far, what would help most:

  1. Pushback on the six diagnostic points. If any of them are based on a misreading of [POC BDK] Introduce modular trait-based watch-only wallet architecture #950, I'd rather know now.
  2. Opinions on the open questions. Especially the migration-path one — that determines what the next PR looks like.
  3. Anything missing. The watch-only crate touches Electrum, JSON-RPC, the chain consumer, and florestad's startup. If the proposed direction breaks an integration I haven't accounted for, please flag it.

@moisesPompilio
Copy link
Copy Markdown
Collaborator Author

The new_provider factory in the same file hardcodes BdkWalletProvider<Connection, KeyId>. This means the provider namespace itself knows BDK exists — a future non-BDK provider isn't a peer of bdk_provider.rs, it's something mod.rs would need to learn about.

Question: should the provider module only know the trait, with new_provider (and any factory) living inside bdk_provider.rs itself? That keeps the provider trait genuinely backend-agnostic.

In Rust’s structure, since the trait is inside the module, it already knows about BDK. In this case, I put this function that instantiates the BDK provider there so that, if we have other providers in the future, they can also be added in the same place. That way, this becomes the standard, centralized place to get the function that instantiates the provider, since it always returns something that implements the trait.

In this way, the user doesn’t need to access the BDK module to get these things, and doesn’t even need to know it exists.

Question: who owns descriptor strings? My intuition is the Provider owns them (since it has BDK's persister anyway), and the Repository owns only what's exclusively a Floresta concept — Merkle proofs, the script_hash → script index, and wallet configuration metadata. That's a stricter boundary than the current PR draws.

In this case, I only put the string in the repository to make requests to some endpoints easier, so it wouldn’t need to query the provider. But in this case, they can just stay in the provider; the only thing the rest of the system knows is the descriptor hash, which will be its ID, and that way it can link everything to it.

Question: should the metadata layer hold all loaded wallets indexed by ID, with load_wallet adding to a set instead of replacing a slot?

In this case, loadwallet is for loading a wallet, while createwallet creates a wallet in Floresta. When a wallet is loaded, it is placed in the metadata, which only contains wallet information such as which descriptors it has, which one is change, and so on. It should not interfere with block or mempool processing.

As for block and mempool processing, that is done for all wallets simultaneously, because that is handled by the provider, and the provider doesn’t know which wallets exist. The only thing it knows is which descriptors it must track information for.

Question: would interface segregation help here — separate BlockProcessor / BalanceQuery / AddressGenerator / DescriptorRegister traits with the current trait composing them as a super-trait?

I think splitting the trait in the provider doesn’t make sense. However, it can exist in the service, which would be the higher-level API for floresta-watch-only.

In that case, the provider trait would be available for export in case someone wants to implement their own provider, but the service would be the component that uses the provider. That way, the service becomes the public API for using floresta-watch-only.

Then, if needed, we can split things into several traits later if that becomes necessary.

Question: should this be a DescriptorId newtype in models.rs with a single canonical constructor, used everywhere?

I don’t think this is necessary, because the descriptor ID should be generated only in the service layer. Everywhere else only needs to know that the unique descriptor ID is a string, and nothing more than that.

Not an architectural question, just something I noticed:

fn on_block(&self, block: &Block, height: u32, ...) {
    // We only process block if the wallet is initialized,
    if self.persister.exists_descriptor(None, None).unwrap_or(false) {
        return;
    }
    self.process_block_inner(block, height)...
}

The comment says "process if initialized" but the code returns when a descriptor exists — so blocks would only be processed for uninitialized wallets. Either the comment is wrong or there's a missing !. Worth flagging independently of the larger discussion.

This comment is poorly written, because blocks are only processed if we already have a wallet that contains a descriptor. It doesn’t make sense for the wallet to process blocks if there’s nothing to process.

@moisesPompilio
Copy link
Copy Markdown
Collaborator Author

Regarding multi_process and async in the wallet, this needs to be evaluated carefully, because in Bitcoin the wallet itself relies heavily on synchronous processing, especially for block processing.

We can’t put block processing in parallel in the wallet, for example, because whenever we discover a TXO, we don’t know when an input spending it will appear.

@moisesPompilio
Copy link
Copy Markdown
Collaborator Author

Multi-wallet index inside the Processor

Renaming "Metadata" to "WalletIndex" because that's the role: a router from (WalletId, ScriptHash, DescriptorId) to the right per-wallet state. Because the Processor is the only writer, the index doesn't need its own lock. Multi-wallet falls out of having HashMap<WalletId, WalletState> instead of a single Option<active>.

The wallet metadata keeps in memory which wallet the user is currently using, so it can’t be a list. Technically, it doesn’t persist data; it’s just a helper used when we need to ask something specific about a wallet. For example, if the user wants to generate an address, we need to know from which descriptor that address should be generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ecosystem support Enable interoperability, compatibility and practical integration with the broader Bitcoin ecosystem reliability Related to runtime reliability, stability and production readiness

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants