feat: pool read connections in MDSQLiteAdapter

[LucDeCaf]

Pooled read connections

Status

TODO

• Implement readConnection pool
• Add option to set max pool size
• Benchmark against old implementation (high number of watched queries)
  • Roughly equivalent for ReadPoolSize = 1
  • Much better for ReadPoolSize > 1
• Tests

Description

Currently, we use an exclusive lock to manage reading from the SQLite database. The lock is used to restrict access to the one singular SqliteConnection object we manage for reading. This means that only one read can occur against the database at any given time, which can become an issue when dealing with multiple watched queries, which might lock each other up despite all only needing read access to the database.

This PR replaces the old readConnection and readMutex objects with a readPool channel. The MDSQLiteAdapter now maintains a "queue" of SqliteConnection objects which it distributes access to via a bounded channel. The pool size can be set via MDSQLiteOptions.ReadPoolSize (currently internal), and defaults to 5 read connections.

Potential improvements

UNPLANNED

• Dynamic pool sizing

  • Scale up or down the size of the read pool when load changes.

• GetPersistentConnection method

  • Returns a SqliteConnection object setup for reading data, mainly for integrations with existing tools that require an object implementing IDbConnection.

Alternative solutions

Built-in ADO.NET connection pool

ADO.NET provides a built-in connection pool that allows multiple SqliteConnection objects with the same connection string to automatically reuse underlying connections. There's a potential use case here; however, since we call LoadExtension on each connection we use (as well as set up a number of PRAGMAs for both read and write connections), I'm unsure of how practical this would be ITO keeping connections setup correctly.

Edit: Upon testing, it does seem to be considerably slower. Will stick with a manual connection pool for now.

[Chriztiaan]

I am tempted to suggest wrapping this in a class to help with readability for other devs (like me 😉):

public class ConnectionPool<T>
{
    /// <summary>Creates a bounded pool with the given capacity.</summary>
    public ConnectionPool(int size);

    // xxxx
    Claim() // or some other word for taking requesting connection

    // xxx
    Release() // Or some other word for giving a connection back to the pool

    /// <summary>
    /// Borrows a connection, executes the function, and returns it to the pool.
    /// Blocks asynchronously if no connections are available.
    /// </summary>
    public async Task<TResult> Borrow<TResult>(Func<T, Task<TResult>> fn);

    /// <summary>
    /// Drains all connections from the pool, executes the callback,
    /// then returns them. Blocks all other borrowers while running.
    /// </summary>
    public async Task WithAll(Func<List<T>, Task> callback);
}

Borrow could for example be used in readLock's implementation instead:

public async Task<T> ReadLock<T>(Func<IDBGetConnection, Task<T>> fn, DBLockOptions? options = null)
{
    await initialized;
    return await readPool.Borrow(conn => fn(conn));
}

Basically just making some of the reading easier. Some of the signatures could have better names though.

[LucDeCaf]

Can do 👍

[LucDeCaf]

I added an MDSQLiteConnectionPool class which wraps the channel.