Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,12 +37,16 @@ state libraries start to strain:
to persistent storage on a miss, then cache the covered index range for next
time. Cache fills, write-through mutations, and transactions are serialized
per HybridDB instance so async selectors and actions do not overlap against
the in-memory cache tier.
the in-memory cache tier. Drivers explicitly report whether selector readonly
transactions are supported; enabled drivers use `beginTx("readonly")` for
scoped reuse. With an IndexedDB primary, that readonly transaction starts
only when the persistent tier is actually read.
- **Synchronous on the frontend.** Against the in-memory driver, selectors and
actions execute **synchronously** (no `await`, no microtask hop), so a click
updates the store and the UI in the same tick. `useAsyncSelector` keeps this
fast path when a run completes from memory, then promotes to async only if a
command yields a promise.
command yields a promise. Its async React API returns a React Query-style
object with `data`, `status`, `error`, fetching flags, and `refetch()`.
- **JavaScript selectors and actions.** Selectors and actions are ordinary JS: loops,
conditionals, function calls. You get fast indexed lookups underneath, not a
query language to learn.
Expand Down
19 changes: 10 additions & 9 deletions packages/hyperdb-demo/src/BenchmarkApp.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -60,15 +60,16 @@ export function BenchmarkApp() {
isWorking,
setIsWorking,
} = benchmarkState;
const dashboard =
useAsyncSelector({
selector: getDashboardSnapshot,
args: {
taskLimit,
projectLimit,
selectedProjectId: benchmarkState.selectedProjectId,
},
}) ?? EMPTY_DASHBOARD_SNAPSHOT;
const { data: dashboard } = useAsyncSelector({
selector: getDashboardSnapshot,
args: {
taskLimit,
projectLimit,
selectedProjectId: benchmarkState.selectedProjectId,
},
placeholderData: (previousDashboard) =>
previousDashboard ?? EMPTY_DASHBOARD_SNAPSHOT,
});

const storeMode = getStoredMode();
const persistence = usePersistence();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,6 @@ initCachedSelector(db, projectTasks, { projectId: "p1" }, { gcTime: 30_000 });
initCachedSelector(db, projectTasks, { projectId: "p1" }, { gcTime: 0 });
```

In React, pass `gcTime` to [`useSyncSelector`](/integrations/react/).

## Memoization controls

Selectors take a `memoization` option:
Expand Down
63 changes: 51 additions & 12 deletions packages/hyperdb-doc/src/content/docs/integrations/react.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,32 +66,71 @@ function Tasks({ projectId }: { projectId: string }) {

Options:

| Option | Description |
| -------------- | ------------------------------------------------------------------------------------------------ |
| `selector` | The selector to run |
| `args` | Its arguments (also the cache key) |
| `defaultValue` | Value returned before the first result / when disabled |
| `enabled` | Set `false` to skip running; returns `defaultValue` |
| `gcTime` | Override the cache [garbage-collection time](/database/selectors-reactivity/#garbage-collection) |
| Option | Description |
| -------------- | ------------------------------------------------------ |
| `selector` | The selector to run |
| `args` | Its arguments (also the cache key) |
| `defaultValue` | Value returned before the first result / when disabled |
| `enabled` | Set `false` to skip running; returns `defaultValue` |

### `useAsyncSelector`

For asynchronous drivers (IndexedDB, async SQLite). Same shape, but the result
arrives asynchronously, so it returns `defaultValue` (or `undefined`) until the
first run resolves, and re-runs on relevant changes.
For asynchronous drivers (IndexedDB, async SQLite). It accepts the same
`selector` and `args` identity as `useSyncSelector`, but returns a
React Query-style result object so loading, error, and manual refetch states are
explicit.

Each run starts synchronously. If the selector completes from memory or cache,
the result is applied in the same tick; if a command yields a promise, that run
continues asynchronously.

```tsx
const tasks = useAsyncSelector({
const {
data: tasks = [],
error,
isFetching,
isLoading,
isError,
refetch,
status,
} = useAsyncSelector({
selector: projectTasks,
args: { projectId },
defaultValue: [],
});
```

Options:

| Option | Description |
| ---------------------- | -------------------------------------------------------------------------- |
| `selector` | The selector to run |
| `args` | Its arguments (also the reactive identity) |
| `enabled` | Set `false` to skip automatic runs; call `refetch()` to run manually |
| `defaultValue` | Compatibility alias for placeholder data before the first resolved run |
| `initialData` | Initial successful data for the result |
| `initialDataUpdatedAt` | Timestamp for `initialData` |
| `placeholderData` | Temporary data while the selector is still pending |
| `subscribed` | Set `false` to avoid automatic runs and DB subscriptions for this instance |
| `throwOnError` | Throw render-phase errors to an error boundary when `true` or a predicate |

Returns:

| Field | Description |
| ----------------------------------------------------- | ------------------------------------------------------------------------------- |
| `data` | Last successful selector result, placeholder data, initial data, or `undefined` |
| `status` | `"pending"`, `"success"`, or `"error"` |
| `fetchStatus` | `"fetching"` or `"idle"` (`"paused"` is reserved for query compatibility) |
| `error` | Last selector error, or `null` |
| `dataUpdatedAt` / `errorUpdatedAt` | Timestamps for the last success or error |
| `isPending` / `isSuccess` / `isError` | Status booleans |
| `isFetching` / `isLoading` / `isRefetching` | Fetching booleans, matching React Query naming |
| `isLoadingError` / `isRefetchError` | Distinguish first-load failures from refresh failures |
| `isPlaceholderData` / `isStale` / `isEnabled` | Extra query-state booleans |
| `failureCount` / `failureReason` / `errorUpdateCount` | Failure counters and reason |
| `promise` | Promise for the current run's data |
| `refetch(options)` | Manually rerun the selector; pass `{ throwOnError: true }` to reject on error |

## Writing

### `useDispatch` / `useAsyncDispatch`
Expand Down Expand Up @@ -141,7 +180,7 @@ const handleClick = () => {
| ------------------------ | ------------------------------ | ---------------------------- |
| `useDB()` | the `SubscribableDB` | accessing the DB directly |
| `useSyncSelector(opts)` | the selector result | reactive read, sync drivers |
| `useAsyncSelector(opts)` | the result or default | reactive read, async drivers |
| `useAsyncSelector(opts)` | React Query-style result | reactive read, async drivers |
| `useDispatch()` | `(action) => TReturn` | write, sync drivers |
| `useAsyncDispatch()` | `(action) => Promise<TReturn>` | write, async drivers |
| `useSelect()` | `(gen) => TReturn` | one-off read, sync drivers |
Expand Down
8 changes: 7 additions & 1 deletion packages/hyperdb-doc/src/content/docs/runtime/db.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,9 @@ later reads. Empty misses are cached too. Limited B-tree reads cache the covered
prefix or suffix when the runtime can prove the returned rows are enough to
answer the same limited query from memory. With an IndexedDB primary, this means
no readonly IndexedDB transaction is opened until the selector actually falls
through to the persisted tier.
through to the persisted tier. If the persistent tier is read, readonly
transaction reuse stays scoped to that selector run, so concurrent selector runs
do not share one IndexedDB transaction.

```ts
import { DB, HybridDB, SubscribableDB, execAsync } from "@will-be-done/hyperdb";
Expand All @@ -138,6 +140,10 @@ Writes go to both tiers in the same operation. That means cached rows stay
current immediately, while uncached ranges still load lazily on first access.
Transactions open transactions against both tiers; scan coverage discovered
inside a transaction is published to the outer cache only after commit.
Drivers explicitly report whether selector-scoped readonly transactions are
supported. When they are, HyperDB uses `beginTx("readonly")`; HybridDB keeps
that context lazy until a selector misses the cache and reads the persistent
tier.

HybridDB serializes cache fills, write-through mutations, coverage updates, and
transaction lifetimes per instance. This keeps async selector misses and actions
Expand Down
9 changes: 6 additions & 3 deletions packages/hyperdb-doc/src/content/docs/runtime/drivers.md
Original file line number Diff line number Diff line change
Expand Up @@ -311,9 +311,12 @@ await asyncDispatch(

The IndexedDB driver uses the same storage encoding and sort-key ordering as
the SQLite driver, so data and index semantics are consistent across the two
persistent backends. Selector reads use readonly IndexedDB transactions; when
multiple scans happen while the browser keeps a readonly transaction active, the
driver reuses it instead of opening one transaction per scan.
persistent backends. IndexedDB reports selector readonly transaction support,
so selector reads use `beginTx("readonly")`; when multiple scans happen inside
one selector run while the browser keeps a readonly transaction active, the
driver reuses it instead of opening one transaction per scan. Concurrent
selector runs get separate readonly transactions, and an inactive readonly
transaction is reopened once for the current scan.

## Sync vs. async, in practice

Expand Down
4 changes: 4 additions & 0 deletions packages/hyperdb-doc/src/content/docs/start/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,10 @@ export function App() {
The list re-renders automatically whenever a `createTask` (or any mutation
touching the queried range) commits.

For async drivers, use `useAsyncSelector` instead. It keeps the same
`selector`/`args` input and returns a React Query-style object with `data`,
`status`, `error`, fetching flags, and `refetch()`.

## Where to next

- [Schemas](/database/schemas/): tables, validators, tagged unions.
Expand Down
3 changes: 2 additions & 1 deletion packages/hyperdb-doc/summary.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,8 @@ check the matching docs below and also check the root `README.md`.
- `src/content/docs/integrations/react.md`: React integration guide. Covers
`DBProvider`, `useDB`, `useSyncSelector`, `useAsyncSelector`, `useDispatch`,
`useAsyncDispatch`, `useSelect`, `useAsyncSelect`, selector options, default
values, `enabled`, `gcTime`, and the full hook reference table.
values, `enabled`, the React Query-style async selector result, and the full
hook reference table.
- `src/content/docs/integrations/devtools.md`: Devtool and tracing guide. Covers
adding `HyperDBDevtools`, devtool tabs and trace inspection, component props,
embedded panel option, trace contents, cache-hit traces, `HybridDB` source
Expand Down
3 changes: 2 additions & 1 deletion packages/hyperdb/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,8 @@ state libraries start to strain:
actions execute **synchronously** (no `await`, no microtask hop), so a click
updates the store and the UI in the same tick. `useAsyncSelector` keeps this
fast path when a run completes from memory, then promotes to async only if a
command yields a promise.
command yields a promise. Its async React API returns a React Query-style
object with `data`, `status`, `error`, fetching flags, and `refetch()`.
- **JavaScript selectors and actions.** Selectors and actions are ordinary JS: loops,
conditionals, function calls. You get fast indexed lookups underneath, not a
query language to learn.
Expand Down
38 changes: 35 additions & 3 deletions packages/hyperdb/src/hyperdb/commands/selector/selector.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
/* eslint-disable @typescript-eslint/no-explicit-any */
import type { SubscribableDB, Op } from "../../runtime/subscribable-db";
import { execAsync, execMaybeAsync, execSync } from "../../core/executor";
import type { DBCmd } from "../async";
import type { HyperDB } from "../../core/contracts";
import { deepFreeze } from "../../deep-freeze";
import type { Row } from "../../core/primitives";
Expand Down Expand Up @@ -348,6 +349,25 @@ type RunSelectorOptions = Pick<CommandRunnerOptions, "ops" | "childMemo">;
const makeVisited = (options: RunSelectorOptions): ChildVisited | undefined =>
options.childMemo ? new Map() : undefined;

function* runCommandGeneratorWithReadonlyTransaction<TReturn>(
db: HyperDB,
gen: Generator<unknown, TReturn, unknown>,
options: CommandRunnerOptions,
): Generator<DBCmd, TReturn, unknown> {
const tx = db.canUseReadonlyTransactionsForSelectors()
? yield* db.beginTx("readonly")
: undefined;
const runnerDB = tx ?? db;

try {
return yield* runCommandGenerator(runnerDB, gen, options);
} finally {
if (tx) {
yield* tx.rollback();
}
}
}

export function runSelector<TReturn>(
db: HyperDB,
gen: () => Generator<unknown, TReturn, unknown>,
Expand All @@ -358,7 +378,11 @@ export function runSelector<TReturn>(

const visited = makeVisited(options);
const result = execSync(
runCommandGenerator(db, gen(), { ...options, selectRangeCmds, visited }),
runCommandGeneratorWithReadonlyTransaction(db, gen(), {
...options,
selectRangeCmds,
visited,
}),
);
if (options.childMemo && visited) {
pruneChildMemo(options.childMemo, visited);
Expand All @@ -376,7 +400,11 @@ export async function runSelectorAsync<TReturn>(

const visited = makeVisited(options);
const result = await execAsync(
runCommandGenerator(db, gen(), { ...options, selectRangeCmds, visited }),
runCommandGeneratorWithReadonlyTransaction(db, gen(), {
...options,
selectRangeCmds,
visited,
}),
);
if (options.childMemo && visited) {
pruneChildMemo(options.childMemo, visited);
Expand All @@ -394,7 +422,11 @@ export function runSelectorMaybeAsync<TReturn>(

const visited = makeVisited(options);
const result = execMaybeAsync(
runCommandGenerator(db, gen(), { ...options, selectRangeCmds, visited }),
runCommandGeneratorWithReadonlyTransaction(db, gen(), {
...options,
selectRangeCmds,
visited,
}),
);

if (result instanceof Promise) {
Expand Down
4 changes: 3 additions & 1 deletion packages/hyperdb/src/hyperdb/core/contracts.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import type { DBCmd } from "../commands/async";
import type { DBTransactionMode } from "./driver";
import type { CodecOptions } from "../storage/codec";
import type {
ExtractIndexes,
Expand Down Expand Up @@ -36,8 +37,9 @@ export interface HyperDB {
getDBName?(): string | undefined;
getTracer?(): HyperDBTracerOption | undefined;
getOptions?(): CodecOptions;
canUseReadonlyTransactionsForSelectors(): boolean;

beginTx(): Generator<DBCmd, HyperDBTx>;
beginTx(mode?: DBTransactionMode): Generator<DBCmd, HyperDBTx>;
// eslint-disable-next-line @typescript-eslint/no-explicit-any
loadTables(tables: TableDefinition<any, any>[]): Generator<DBCmd, void>;
}
Expand Down
5 changes: 4 additions & 1 deletion packages/hyperdb/src/hyperdb/core/driver.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ import type { DBCmd } from "../commands/async";
import type { TableDefinition } from "../schema/table";
import type { Row, SelectOptions, WhereClause } from "./primitives";

export type DBTransactionMode = "readonly" | "readwrite";

export type BaseDBDriverOperations = {
intervalScan(
table: string,
Expand All @@ -17,7 +19,8 @@ export type BaseDBDriverOperations = {

export interface DBDriver extends BaseDBDriverOperations {
loadTables(table: TableDefinition<any, any>[]): Generator<DBCmd>;
beginTx(): Generator<DBCmd, DBDriverTX>;
beginTx(mode?: DBTransactionMode): Generator<DBCmd, DBDriverTX>;
canUseReadonlyTransactionsForSelectors(): boolean;
}

export interface DBDriverTX extends BaseDBDriverOperations {
Expand Down
Loading
Loading