Skip to content

perf(analytics): measure and reduce graph-score warmup allocations #561

Closed

Description

@dborup

Summary

Measure and reduce bridge/coverage graph-score warmup cost without changing score semantics. This is a measured algorithm/allocation task, not permission to weaken readiness or expose zero scores as completed analytics.

Verified baseline: ff1acc02255b5c6ab852e4cf931f85dbe3043ef9.

Current behavior

main.go starts the bridge and usefulness-axes recomputers before ListenAndServe. Both starters perform their initial computation synchronously:

  • bridge_recomputer.go: recomputeBridgeScoresSafe(s) before starting the periodic loop.
  • usefulness_axes_recomputer.go: s.recomputeUsefulnessAxes() before starting its periodic loop.

Bridge scoring creates fresh string-keyed predecessor, distance, sigma and visited structures per source. Coverage repeatedly calls dijkstraFrom, which also allocates distance/visited maps and heap storage per source. Our shared adjacency helper is useful reuse, but it is not yet an indexed graph plus reusable bounded workspace.

The code therefore puts these warmups on the HTTP startup path and retains substantial repeated allocation opportunities. The amount of time and allocated bytes on the current fork still needs measurement.

Coordinate with existing work

#492 / PR #554 concerns opt-in pausing of unread steady-state snapshot recomputers. It intentionally retains the successful initial warmup, and explicitly leaves the separate bridge/usefulness-axes loops unchanged. It is not this algorithm/startup-cost change. Coordinate with that PR to preserve the applicable lifecycle and snapshot semantics; do not duplicate or silently replace it.

Measure before implementation

  1. Capture paired baseline results on the current fork: bridge and coverage separately, startup warmup elapsed time, ns/op, B/op, allocs/op, graph vertices/edges and environment.
  2. Include small deterministic edge-case graphs and a bounded representative graph such as 2,000 vertices / 10,000 edges. State whether reported memory is allocated bytes or retained heap.
  3. Use independent small-graph reference/oracle results to pin behavior before optimizing. The production implementation must not be its own oracle.
  4. Only choose an optimization once profiles/benchmarks confirm where current work is spent. Report before/after using identical inputs, alternating runs where practical and a declared noise policy.

Bounded implementation direction

  • Evaluate indexed adjacency and reusable bounded per-computation workspaces rather than rebuilding string-keyed maps and heap objects for every source.
  • Preserve weighted Brandes/Dijkstra behavior, normalization, duplicate-edge filtering, directedness as currently defined, floating-point/tie tolerance, disconnected components and zero/invalid-edge handling.
  • Keep state isolated between sources and recomputations; no stale distances, predecessors or cross-call buffers. Avoid an unbounded global workspace cache or new expensive work under PacketStore locks.
  • Preserve immutable snapshot publication and concurrent-reader safety.
  • The existing synchronous initial snapshot contract can remain unchanged. Moving warmup behind the listener would require a separate explicit readiness/API design, not an incidental shortcut in this optimization.
  • No schema changes, new operator configuration or unrelated analytics rewrites are planned. Any new tuning knob would need a separate rationale and future customizer plan.

Acceptance and tests

  • Publish current-fork baseline measurements before claiming a startup/performance improvement.
  • Match an independent oracle on small graphs, including disconnected nodes, duplicate edges, ties/equal shortest paths and supported weight edge cases.
  • Verify score normalization and deterministic results/tolerances across repeated computations and node/edge order permutations.
  • Add tests that catch stale reusable-workspace contents, buffer reuse across different graph sizes and concurrent computations under -race.
  • Add realistic before/after benchmarks with allocation figures and a bounded-work/memory justification. Do not accept an optimization that merely moves unreported allocation elsewhere.
  • Confirm the initial snapshot, recomputer lifecycle and perf(analytics): opt-in pause of unread snapshot recomputers #492 / PR perf(analytics): opt-in pause of unread snapshot recomputers #554 behavior remain intact.
  • Run focused tests, -race, go vet and the relevant server suite in isolation; no staging or production benchmark is needed for the initial work.

Source provenance

Source repository: Kpa-clawbot/CoreScope
Issue number: 2226

Historical measurements in that source used another revision, graph and host. They are motivation only, not a baseline for this fork. No new runtime benchmark or live-host measurement was performed while preparing this issue.

Activity

  1. dborup commented on Oct 25, 2026

    @dborup
    OwnerAuthor

    Fixed by #587 (merged as 5699b63e0d75aa0d49c9171c573767db41bd0211).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions