Skip to content

perf: make prov fast without a persisted cache - #24

Merged
adammharris merged 22 commits into
mainfrom
agent/prov-speed-without-caching
Nov 2, 2026
Merged

adammharris merged 22 commits into
mainfrom
agent/prov-speed-without-caching

Conversation

@adammharris

Copy link
Copy Markdown
Member

prov keeps no persisted cache, and this makes it fast without one (DESIGN §5 now says so, and lists how). Every speedup comes from reading and parsing less, once, and in parallel. Nothing derived is written to disk.

Measured on two synthetic ~10k-document workspaces: flat is 100 sections × 100 documents, deep is three children per node, eight levels. Each figure is an average of 10–20 interleaved runs, warm cache, 11 cores. Under "now", the first figure is the default build and the one in brackets has --features mimalloc:

original now (mimalloc)
flat tree 166 ms 44 (39) ms
flat docs 186 ms 54 (52) ms
flat check 394 ms 88 (≈80) ms
flat search 366 ms 70 ms
flat mv 441 ms ~135 (122) ms
deep tree 181 ms 61 (53) ms
deep check 720 ms 128 ms
deep search 399 ms ~105 ms

Parse less

  • Frontmatter is located from the lines its fences span, not by parsing the whole document; this uses twig 7.1's find_front.
  • A body with no [[ or ]( isn't parsed for links.
  • Search folds ASCII text without Unicode decomposition.

Parse and read once, per operation

  • The read memo shares Arc<Loaded> values instead of copying them. Graph::read is new, and load keeps its signature.
  • Each body's links, and each document's links classified for the walk (graph::prepare), are remembered beside the document.
  • Each directory is listed once per operation. A write forgets the listings of every directory above it.
  • The memo is keyed by a path's bytes, and the path-keyed maps use foldhash (prov_graph::hash).

Parallel, with the parallel feature

  • The census walk and the tree descent read the tree ahead one level at a time. Each level's documents are read, parsed and have their links prepared on rayon's pool, and the next level's directories are listed there too.
  • Reads go to the pool where the storage offers ReadStorage::shared_reader (new in fs-transaction 0.4.2). Batches under 16 documents are read in place.
  • The walk itself is unchanged and finds those documents in the memo, so its order and findings don't depend on the feature. Output is byte-identical to before on prov's own docs and both benchmark trees.
  • A large memo is freed on the pool rather than before returning.

Smaller

  • Hash sets replace linear and ordered lookups in the walk and in check.
  • tree, docs and grouped views use buffered output.

Release-facing

Diaryx doesn't enable parallel today, so it gets the single-threaded wins only.

Local CI, all seven jobs: fmt, clippy, test, guide transcripts, prov check, package isolation and MSRV. clippy and tests also pass with parallel off.

The walk now reads each document's spanning children before resolving
them, and parses their frontmatter and body links in parallel into the
read memo. The walk itself is unchanged and meets them as memo hits. A
body's links are remembered beside the document they were found in, so
the census and the inbound index stop parsing the same prose twice.
… shares its reads

The prefetch reads and parses each child in one task on rayon's pool when
ReadStorage::shared_reader hands over a reader, and the tree descent
prefetches its children too, now that it can read them together. Needs
fs-transaction with shared_reader (diaryx-org/fs-transaction
agent/shared-reads); the workspace takes it from a local patch until that
is released.
find_front parses only what locating the frontmatter takes, which is what
prov's fenced_prefix did on its side, so the workaround goes. The body
scan asks nodes_of_kind for the six kinds it sorts rather than copying out
every node. Needs twig 7.1; the workspace takes it from a local patch until
that is released.
Graph::read hands back an Arc<Loaded>, the document's text and parse as
one read found them, and the memo stores that Arc: a second read in one
operation is a reference count, where load copied both. A prefetch moves
what it built into the memo instead of copying it in. load keeps its
signature, as an owned copy of read, and the walk, the tree descent, the
check passes, the inbound index and the views' documents and corpus read
shared.
Off by default: mimalloc is C, and building prov from source needs a Rust
toolchain and nothing else. The release passes cargo-features: mimalloc,
which needs diaryx-org/.github's reusable-homebrew to take that input.
The prefetch read one parent's children per batch, which on a tree three
children wide is a batch of three: rayon woke every worker for it, and a
deep journal walked slower than with no prefetch at all. The walk and the
tree descent now prefetch once, before they start, a level per batch, so
the batches are large whatever the tree's shape. A batch under sixteen is
read in place, and the prefetch stops when the memo's budget is spent.
…ets in bulk

missing_containment and orphans asked an ordered set whether each document
on disk was reached, a dozen component-by-component path comparisons each;
they ask a hash view of it now. reachable_set and documents_among build
their ordered sets from a vector sorted by bytes, which is nearly Path's
order, instead of inserting each path.
The read memo, the walk's queue and visited set, the shadow probe, the
reachable views check asks, and the views' per-document maps are keyed by
path and asked once per link per document. prov_graph::hash::{FastMap,
FastSet} are std's collections with foldhash's seeded hasher.
stdout flushes at every newline, so tree, docs and a grouped view or query
were a write call per document. They write through a BufWriter now, and a
grouped view flushes before its failures go to stderr, so a terminal shows
the answer first as it did.
check listed most of the tree four times over: documents_among and orphans
each listed the reached directories, missing_containment and
nested_findings each walked the whole folder. The read memo now keeps a
directory's entries beside the name index exact_name already kept, and the
read-only scans list through it. A write forgets every listing above it,
since it may have created the directories a tree scan lists. A mutation's
own listing stays fresh.
Path's Hash walks the path a component at a time, a hasher call per
component, and these are asked once per link per document. The paths are
normalized, so their bytes agree with Path's equality; a path spelled two
ways costs at worst a second read.
…k's thread

The census walk runs on one thread because its findings depend on its
order, but most of its per-document work did not: extracting the
frontmatter links, parsing each, classifying it (external, same-document,
foreign, malformed, local id, nominal, path) and resolving and normalizing
its path. The prefetch now does that on rayon's pool and the memo keeps it
beside the document, as it keeps the body's links; the walk is left the
registry and the shared directory listing. The inverse check and the tree
prefetch's next level take the same precomputed links. Nominal links still
go to the resolvers whole, so the title index is consulted as before.
The walk asks the directory listing of every link's target; the tree
prefetch now lists the directories its next level lives in through the
storage's shared reader, on rayon's pool, so the walk finds them in the
memo. Needs SharedRead::read_dir (fs-transaction agent/shared-reads).
twig 7.1 for meta::find_front and Document::nodes_of_kind; fs-transaction
0.4.2 for ReadStorage::shared_reader. The workspace no longer patches
either from a local checkout.
@adammharris
adammharris merged commit 1f882e0 into main Nov 2, 2026
8 checks passed
@adammharris
adammharris deleted the agent/prov-speed-without-caching branch November 2, 2026 19:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant