Per-branch perplexity accumulates only inside branch::accept_token() (model surprisal from the logits snapshot; sampling surprisal from last_candidates when token == last_token). The accumulators live in the store's metrics registry and are freed by release() — so the moment a branch is pruned, its perplexity is unrecoverable.
Why this bites
There is no harvest-before-prune affordance. A consumer that prunes eagerly (correct KV hygiene) and reads PPL later has nothing to read, and the natural guard it writes upstream is a silent sentinel. Live instance in lloyal-ai/hdk's agent pool:
ppl: a.branch.disposed ? 0 : a.branch.perplexity
Across 25 production traces, 30 of 33 agent records carry ppl: 0 — a number that reads as "perplexity 0" (impossibly perfect) rather than "not measured". The correct-hygiene path and the observability path are in direct conflict, and the conflict resolves silently.
Also worth stating in the docs: get_perplexity() returns Infinity for a live branch with no accepted tokens, while a disposed branch upstream typically reports 0 — the two "no data" cases surface as opposite extremes.
Smallest fix
Either of:
- a
harvest_metrics(handle) (or have release() return the final BranchMetricsState) so consumers can capture PPL at prune time, or
- document explicitly on
get_perplexity/get_sampling_perplexity that metrics die with the branch, and that consumers must read before prune — with a recommended sentinel that cannot be mistaken for a measurement (NaN/undefined, never 0).
Found during a full read of branch.hpp/metrics.hpp; one of five silent-failure edges filed together.
Per-branch perplexity accumulates only inside
branch::accept_token()(model surprisal from the logits snapshot; sampling surprisal fromlast_candidateswhentoken == last_token). The accumulators live in the store's metrics registry and are freed byrelease()— so the moment a branch is pruned, its perplexity is unrecoverable.Why this bites
There is no harvest-before-prune affordance. A consumer that prunes eagerly (correct KV hygiene) and reads PPL later has nothing to read, and the natural guard it writes upstream is a silent sentinel. Live instance in
lloyal-ai/hdk's agent pool:Across 25 production traces, 30 of 33 agent records carry
ppl: 0— a number that reads as "perplexity 0" (impossibly perfect) rather than "not measured". The correct-hygiene path and the observability path are in direct conflict, and the conflict resolves silently.Also worth stating in the docs:
get_perplexity()returns Infinity for a live branch with no accepted tokens, while a disposed branch upstream typically reports 0 — the two "no data" cases surface as opposite extremes.Smallest fix
Either of:
harvest_metrics(handle)(or haverelease()return the finalBranchMetricsState) so consumers can capture PPL at prune time, orget_perplexity/get_sampling_perplexitythat metrics die with the branch, and that consumers must read before prune — with a recommended sentinel that cannot be mistaken for a measurement (NaN/undefined, never 0).Found during a full read of
branch.hpp/metrics.hpp; one of five silent-failure edges filed together.