Skip to content

perf(analytics): reduce graph-score warmup allocations - #587

Merged
dborup merged 2 commits into
masterfrom
perf/561-graph-score-warmup
Oct 25, 2026
Merged

dborup merged 2 commits into
masterfrom
perf/561-graph-score-warmup

Conversation

@dborup-agent

Copy link
Copy Markdown
Collaborator

Relates to #561

Summary

Measures and reduces the warmup cost of the Bridge (weighted Brandes betweenness) and Coverage (harmonic reach) graph scores without changing score semantics. Both axes now index the weighted neighbor graph once per computation (sorted int32 vertex ids, CSR adjacency) and run every source over one reusable workspace of flat, vertex-indexed slices. Previously each source allocated fresh string-keyed maps and boxed heap entries.

The synchronous initial snapshot in StartBridgeScoreRecomputer / StartUsefulnessAxesRecomputer is unchanged: the warmup remains on the startup path, so zero scores are never served as finished analytics. It is simply ~5× cheaper now. Moving the warmup off the HTTP startup path would need a separate readiness/API design (as #561 says), so this PR does not do it.

Plan (two commits)

  1. test(analytics) — pin semantics + baseline. Adds an independent oracle, a golden before/after file captured from the current implementation, determinism/stale-state/concurrency tests, an allocation budget, a readiness pin, and seeded benchmarks. Red on master: allocation budget, bit-identical determinism, no-state-across-calls.
  2. perf(analytics) — indexed graph + reusable workspace. Implementation, plus workspace-internal tests (poisoned reuse across sources and graph sizes, CSR structure).

What changed

  • graph_weighted.go: indexedDistanceGraph (CSR) replaces weightedDistanceAdjacency. Same filtering: trim and lowercase, drop empty endpoints, self-loops, NaN/±Inf and w < 1e-9; distance 1/w; keep the cheapest duplicate. Vertex ids are in sorted-key order and rows are sorted by neighbor id. Also adds ssspWorkspace (dist / settled / typed min-heap with lazy deletion) and harmonicReach.
  • bridge_score.go: brandesWorkspace embeds the SSSP workspace and adds sigma / delta / stack, with predecessors stored in each vertex's own CSR row (predBuf, length 2E). The relaxation logic is a line-for-line port, including the 1e-12 tie tolerance, halving and max normalization.
  • coverage_score.go: one SSSP workspace across all sources; Σ 1/d is accumulated at settle time (source and unreachable vertices excluded, as before).
  • weightedDistanceAdjacency, dijkstraFrom and bridgePQ had no other callers and are removed.
  • No schema changes, no new config knobs (so no customizer follow-up), no writes, no map[string]interface{}.

Bounded work / memory. Each Compute* call owns its workspace: no global cache, no sync.Pool, no sharing across goroutines. The workspace is O(V + E): five vertex-length slices, predBuf of 2E, and a heap of ≤ 2E+1 entries. It is reset in O(V) per source, the same as the old per-source map initialization. No new work runs under PacketStore locks; the immutable atomic.Pointer snapshot publication is untouched.

Determinism (behavioral note). Before, vertex order came from Go map iteration, so the floating-point summation order — and the low bits of the scores — varied between runs. Now results are bit-identical across runs and edge-order permutations. Values agree with the old implementation within float rounding; see below.

Benchmarks

go test -run '^$' -bench GraphScore561 -benchmem. Test binaries for the master implementation and this branch were built from the same benchmark file and run alternately, 6 rounds each. Results are compared with benchstat: median ± 95% CI, Mann-Whitney U test. Noise policy: only differences with p < 0.05 are claimed. This host is noisy (time CIs up to ±70%), but every row has p = 0.002 with non-overlapping ranges. B/op is allocated bytes, not retained heap.

Environment: linux/amd64, 12th Gen Intel i7-1260P (4 vCPU), Go 1.27.1, single-threaded compute. Graphs: deterministic seeded synthetic, connected (spanning tree + random extra edges), weights in (0, 1]. The Warmup benchmark runs exactly what the two Start* recomputers do before ListenAndServe (recomputeBridgeScoresSafe + recomputeUsefulnessAxes, i.e. bridge + coverage + redundancy, including the bridgeEdgesFromGraph snapshot) over a real NeighborGraph.

Benchmark V / E time before time after Δ B/op before B/op after allocs before allocs after
Bridge 600 / 2 000 302 ms ± 14% 62.5 ms ± 11% −79% 145.1 MiB 318 KiB 1 458 313 64
Bridge 2 000 / 10 000 4.88 s ± 70% 0.869 s ± 20% −82% 1.89 GiB 1.30 MiB 17 089 080 91
Coverage 600 / 2 000 225 ms ± 18% 57.6 ms ± 16% −74% 105.5 MiB 283 KiB 1 108 512 50
Coverage 2 000 / 10 000 3.79 s ± 10% 0.797 s ± 28% −79% 1.40 GiB 1.16 MiB 13 119 081 75
Warmup (startup path) 600 / 2 000 639 ms ± 22% 122 ms ± 30% −81% 251 MiB 1.23 MiB 2 569 119 1 773
Warmup (startup path) 2 000 / 10 000 8.45 s ± 14% 1.69 s ± 38% −80% 3.29 GiB 5.53 MiB 30 213 610 8 553

The remaining ~8.5k warmup allocations at 2 000 / 10 000 come from bridgeEdgesFromGraph and the map-based Redundancy axis. Both are outside the scope of #561 and unchanged; they are reported here so no allocation is hidden.

Equivalence (before/after)

  • Independent oracle (graph_score_561_test.go): Floyd–Warshall all-pairs distances, path counts by a distance-ordered DP, and betweenness from the pair-dependency definition Σ σ_st(v)/σ_st. Production is never its own oracle. 17 fixtures: empty, single edge, line, triangle, star, diamond with an equal-path tie, weighted diamond, disconnected components, duplicates (either direction and case), invalid and boundary weights (0, negative, NaN, ±Inf, sub-epsilon including a sub-epsilon edge to a new vertex, exactly epsilon, w > 1), case/whitespace keys, a 4×4 grid, tie-weighted random graphs (power-of-two weights so ties are exact) and continuous random graphs up to 150 / 600. Tolerance: |Δ| ≤ 1e-9 on scores in [0, 1]. Hand-derived values pin the oracle itself.
  • Golden file captured from the pre-perf(analytics): measure and reduce graph-score warmup allocations #561 implementation (master 2e6fd9b0) on a fixed 400 / 2 000 graph (testdata/graph_scores_561_golden.json; do not regenerate). Measured: Bridge bit-identical on 400/400 vertices; Coverage max |Δ| = 2.2e-15 (summation order only). The test enforces 1e-9.
  • Real fixture data: master and branch servers on the same prepared e2e-fixture.db, /api/nodes compared per node. bridge_score and redundancy_score are identical; coverage_score max |Δ| = 3.3e-16; usefulness_score max |Δ| = 5.6e-17 (195 nodes).

Tests

Point Test Red on master? Mutants (all killed unless noted)
Bridge allocations O(V+E), < 1 alloc per source TestGraphScore561_AllocationBudget yes (1 458 310 > 300) per-source workspace
Coverage allocations TestGraphScore561_AllocationBudget yes (1 108 509 > 300) per-source workspace
Bridge semantics BridgeMatchesOracle, OracleSanity, GoldenBeforeAfter pin (green on both) drop tie path counting; credit the source
Coverage semantics and input contract CoverageMatchesOracle, IndexedGraphStructure pin sum d instead of 1/d; keep most expensive duplicate; accept w < 1e-9
Determinism across runs and permutations DeterministicAcrossRunsAndPermutations (continuous + tie-weighted) yes ids in encounter order (no sort)
No stale workspace state, across sources and graph sizes WorkspaceReuseIsStateless (poisoned after every source, shrink + grow, anchored to the oracle), NoStateAcrossCallsOrSizes NoState… yes (via nondeterminism) delta not reset; dist not reset; heap not cleared. predLen not reset is an equivalent mutant: predLen[w] is only read after the strict-improvement branch has set it to 1.
Concurrent computations ConcurrentComputations under -race pin shared global workspace → DATA RACE
Readiness unchanged StartInstallsScoresBeforeReturn pin bridge prewarm removed; bridge prewarm made async; axes prewarm removed

Commands run locally (all green):

  • cd cmd/server && go vet . && go test -count=1 ./...
  • go test -race -run 'GraphScore561|Bridge|Coverage|Usefulness|Redundancy' .
  • cd cmd/ingestor && go test -count=1 ./...
  • sh test-all.sh (270/270 files)
  • node test-frontend-helpers.js (725 passed)
  • bash scripts/check-xss-sinks.sh --diff origin/master (no frontend changes)
  • Core E2E (test-e2e-playwright.js, 154 passed / 3 skipped by design) against a local Go server on a scratch copy of e2e-fixture.db, prepared as in CI (freshen, inline seed, migrate, seed files).

Coordination with #492 / PR #554

PR #554 (merged) pauses unread snapshot recomputers and explicitly leaves the bridge and usefulness-axes loops unchanged. This PR does not touch any recomputer lifecycle, ticker or snapshot publication. It only changes the pure Compute* functions they call.

🤖 Generated with Claude Code

dborup and others added 2 commits October 25, 2026 06:45
…561)

Adds an independent Floyd-Warshall / pair-dependency oracle for the Bridge
(weighted Brandes betweenness) and Coverage (harmonic reach) scores, a
golden before/after file captured from the current map-based
implementation, determinism / stale-state / concurrency tests, an O(V+E)
allocation budget, a readiness pin for both Start* recomputers, and
seeded benchmarks at 600/2000 and 2000/10000.

Red on master: the allocation budget (1.46M / 1.11M allocs at 600/2000),
bit-identical results across runs and edge permutations (map iteration
order perturbs float summation), and the no-state-across-calls check.

Relates to #561

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ace (#561)

Bridge (Brandes) and Coverage (harmonic reach) now index the weighted
neighbor graph once per computation (sorted int32 vertex ids, CSR rows
sorted by neighbor, cheapest duplicate kept) and run every source over one
workspace of flat vertex-indexed slices with a typed min-heap and
CSR-row predecessor storage, instead of allocating string-keyed maps and
boxing heap entries per source. Workspaces are per call: no global cache,
no sharing between goroutines, O(V+E) memory per in-flight computation.

Semantics are unchanged (same input filtering, 1/weight distances,
1e-12 Brandes tie tolerance, halving and max normalization; Coverage still
skips the source and unreachable vertices). Vertex numbering no longer
depends on map iteration, so results are now bit-identical across runs
and edge permutations. The synchronous initial snapshot in both Start*
recomputers is untouched.

weightedDistanceAdjacency, dijkstraFrom and bridgePQ had no other callers
and are removed.

Paired benchmarks (2000 vertices / 10000 edges, n=6 alternating):
Bridge 4.88s -> 0.87s, 1.93GiB -> 1.3MiB, 17.1M -> 91 allocs;
Coverage 3.79s -> 0.80s, 1.43GiB -> 1.2MiB, 13.1M -> 75 allocs;
startup warmup (bridge + coverage + redundancy) 8.45s -> 1.69s.

Tests: workspace reuse across sources and graph sizes with poisoning,
indexed-graph structure, tie-weighted permutation determinism, and an
allocation budget below one allocation per source.

Relates to #561

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dborup-agent

Copy link
Copy Markdown
Collaborator Author

Rapport — CS-pve-agent2 PR#587 #561 — head a62a2aa

Status: Draft ready for review. All requirements are covered by tests and mutants, and CI is green on every job with no reruns. Not merged and not marked ready.

Evidence legend: [T] verified by a local test or benchmark run · [A] analysis or reasoning, not executed · [K] checked by command or CI (grep, diff, CI job result).

Requirements

# Requirement Test(s) Red on master → green on head Mutant(s) and result Evidence
1 Measure first: time, B/op, allocs at a realistic size BenchmarkGraphScore561_{Bridge,Coverage,Warmup} at 600/2000 and 2000/10000; baseline committed first (071c7942) n/a (measurement) n/a [T] master and head binaries run alternately, n=6, benchstat p=0.002 on every row
2 Reduce Bridge warmup allocations (indexed graph, reused per-source workspace) TestGraphScore561_AllocationBudget (budget V/2 = 300; any per-source allocation fails it) red: 1 458 310 allocs → green: 64 per-source workspace → killed (16 013 allocs) [T]
3 Reduce Coverage warmup allocations TestGraphScore561_AllocationBudget red: 1 108 509 → green: 50 per-source workspace → killed (7 613 allocs) [T]
4 Bridge semantics unchanged BridgeMatchesOracle (independent Floyd–Warshall + pair-dependency oracle, 17 fixtures), OracleSanity, GoldenBeforeAfter pin: green on both by design drop tie path counting → killed; credit the source → killed [T] golden: 400/400 bit-identical; real fixture /api/nodes max |Δ| 0
5 Coverage semantics and input contract unchanged CoverageMatchesOracle, IndexedGraphStructure, GoldenBeforeAfter pin sum d instead of 1/d → killed; keep most expensive duplicate → killed; accept w < 1e-9 → killed [T] golden max |Δ| 2.2e-15; real fixture max |Δ| 3.3e-16 (tolerance 1e-9)
6 Deterministic across runs and edge permutations DeterministicAcrossRunsAndPermutations (continuous + tie-weighted graphs, bitwise) red (map iteration order) → green ids in encounter order (no sort) → killed [T]
7 No stale workspace state across sources, calls or graph sizes WorkspaceReuseIsStateless (poisoned after every source, shrink + grow, anchored to the oracle), NoStateAcrossCallsOrSizes NoState…: red (via nondeterminism) → green; WorkspaceReuse… is new-code-only delta not reset → killed; dist not reset → killed; heap not cleared → killed; predLen not reset → survived, equivalent mutant [T] for the kills; [A] predLen is only read after the strict-improvement branch has set it to 1
8 Concurrent computations safe ConcurrentComputations under -race pin shared global workspace → killed (DATA RACE) [T]
9 Readiness not weakened (zero scores never exposed as finished) StartInstallsScoresBeforeReturn pin: the synchronous prewarm is unchanged bridge prewarm removed → killed; bridge prewarm made async → killed; axes prewarm removed → killed [T]; [A] warmup intentionally kept on the startup path (issue: moving it needs a separate readiness design)

Benchmarks (2000 vertices / 10000 edges, median of 6)

Benchmark before after B/op before → after allocs before → after
Bridge 4.88 s 0.869 s 1.89 GiB → 1.30 MiB 17 089 080 → 91
Coverage 3.79 s 0.797 s 1.40 GiB → 1.16 MiB 13 119 081 → 75
Warmup (startup path: bridge + coverage + redundancy) 8.45 s 1.69 s 3.29 GiB → 5.53 MiB 30 213 610 → 8 553

[T] B/op is allocated bytes, not retained heap. The 600/2000 rows are in the PR description.

Standing constraints

Constraint Result Evidence
cmd/server stays read-only Only pure compute functions and tests changed; no DB access added [K] diff review; [T] readonly_invariant_test.go passes in the full suite
No new map[string]interface{} outside tests 0 added lines [K] git diff origin/master | grep '^+' | grep -c 'map\[string\]interface{}' → 0
No hardcoded colors No frontend or CSS changes [K]
scripts/check-xss-sinks.sh --diff origin/master clean "no public/**/*.{js,html} changes to scan", exit 0 [K]
Fork guards unchanged (9 in deploy.yml, 1 in release-fast-path.yml) 9 and 1; no diff under .github/ [K] grep -c "github.repository == 'Kpa-clawbot/CoreScope'"
No staging or production access None used; all measurements local [A]

Local test runs (head a62a2aaa)

  • [T] cmd/server: go vet . clean; go test -count=1 ./... ok
  • [T] go test -race -run 'GraphScore561|Bridge|Coverage|Usefulness|Redundancy' . ok
  • [T] cmd/ingestor: go test -count=1 ./... ok
  • [T] sh test-all.sh: 270/270 files passed
  • [T] node test-frontend-helpers.js: 725 passed, 0 failed
  • [T] Core E2E test-e2e-playwright.js against a local Go server on a scratch copy of e2e-fixture.db (prepared as in CI: freshen, inline seed, migrate, seed files): 154 passed, 3 skipped by design. The server was stopped by pid/port afterwards.
  • [T] Master and head servers on the same prepared fixture, /api/nodes compared over 195 nodes: bridge and redundancy identical; coverage max |Δ| 3.3e-16; usefulness max |Δ| 5.6e-17

CI per job (run 40172139883, head a62a2aaa)

Job Result
Go Build & Test pass
Go Ingestor Tests pass
Go Libraries & CLI pass
Go Server Tests (1/3) pass
Go Server Tests (2/3) pass
Go Server Tests (3/3) pass
JavaScript Unit Tests & Lint pass
Script & Container Unit Checks pass
Playwright E2E Tests (1/3) pass
Playwright E2E Tests (2/3) pass
Playwright E2E Tests (3/3) pass
Verify E2E Shards & Coverage pass
Verify Go Test Assignments pass
Validate Docker Image pass
Build & Publish Docker Image pass
Publish Badges & Summary / Release Artifacts / Deploy Staging skipped (fork guards / not a push)

[K] No reruns were needed; the known flaky test #271 did not fail.

Remaining items

  • [A] About 8.5k warmup allocations remain at 2000/10000. They come from bridgeEdgesFromGraph and the map-based Redundancy axis, which are outside the scope of perf(analytics): measure and reduce graph-score warmup allocations #561 and unchanged. A follow-up could reuse the indexed graph for Redundancy too.
  • [A] Wall time is still dominated by the O(V·(E + V log V)) all-sources shortest paths (~1.7 s warmup at 2000/10000 on this host). Moving the warmup off the startup path, or parallelizing it across sources, would each need a separate readiness or CPU-budget decision. Neither is done here.
  • [A] Behavior note: scores are now bit-deterministic. Previously the low float bits varied between runs because vertex order came from map iteration.
  • [T] Equivalent mutant (predLen reset) documented above; the reset is kept as defense in depth for the "fully re-initialized workspace" contract.
  • [K] scripts/check-xss-sinks.sh needs bash; sh fails with a syntax error. This is pre-existing and not changed here.

@dborup-agent

Copy link
Copy Markdown
Collaborator Author

Review — CS-MacBook PR#587 — head a62a2aa

Dom: APPROVE with nits

Evidence legend: [T] reproduced by a local test, benchmark or mutant run · [A] analysis or code reading, not executed · [K] checked by command or CI (grep, diff, CI job result).

Trees used: PR head a62a2aaa; the merged tree git merge-tree --write-tree origin/master a62a2aaa = 6afba08b on top of origin/master bb91292a; and the PR's first commit 071c7942 (the old implementation plus the new tests) as the "before" tree. All were extracted with git archive into scratch. The head was checked with git ls-remote before and after the review and was unchanged.

Findings

# Severity Finding Evidence
1 Low (should fix) TestGraphScore561_ConcurrentComputations deadlocks instead of failing when the implementation regresses. errs is a channel with a buffer of 64. The workers send one message per mismatched key (up to 8 × 3 × ~96 sends), and the channel is only drained after wg.Wait(). With more than 64 mismatches every worker blocks, and the test hangs until the go test timeout. Under my mutants M1, M3 and M6 the package panicked with test timed out … running tests: TestGraphScore561_ConcurrentComputations (10 min at the default timeout, 2 min with -timeout 120s). Other failing tests are still reported first, so a regression is not hidden, but a CI shard would burn its whole timeout. Fix: stop each worker at its first mismatch, collect errors under a mutex, or drain errs concurrently. [T]
2 Low (nit) The 1e-12 absolute tie tolerance in Brandes relaxation is not pinned by any test. Mutant M4 (math.Abs(alt-dist[w]) <= 1e-12 → alt == dist[w]) survives the full GraphScore561|Bridge|Coverage|Usefulness set. The tie fixtures use power-of-two weights, so their ties are exact and an exact comparison is enough. #561 explicitly lists "floating-point/tie tolerance" as behavior to preserve. The port is line-for-line, so the behavior is unchanged [A], but nothing would catch a future change. Suggested fix: add a near-tie fixture where two shortest paths differ only by rounding (e.g. costs 0.1 + 0.2 vs 0.3), with σ split between them. This gap already exists on master and was not introduced here. [T] M4 survived · [A]
3 Info The equivalent-mutant rationale for predLen (report item 7) is incomplete. It says predLen[w] "is only read after the strict-improvement branch has set it to 1". The back-propagation also reads predLen[src], which the strict branch never sets for the source. The mutant is still equivalent, because src is the first vertex settled, so it is the last element processed in reverse-stack order, and the stale delta[v] writes it causes are discarded before the next source. The conclusion holds; only the stated reason is narrower than the real one. Keeping the reset is correct. [A]
4 Info Toolchain note: the absolute "before" Warmup alloc counts differ from the PR (600/2000: 2 153 180 vs 2 569 119; 2000/10000: 25.1 M vs 30.2 M). This is most likely the runtime/map differences between Go 1.26.3 (here) and Go 1.27.1 (the author's machine). The "after" counts match the PR exactly, and the conclusion is unchanged. [T]

No blocking issues found.

Answers to the review points

Scores identical before/after on a fixed graph

  • [T] On the golden 400/2000 graph I re-ran the old implementation (tree 071c7942) 3 times and the new one 3 times, comparing bitwise against testdata/graph_scores_561_golden.json:
    • Bridge: 0/400 values differ in every run, old and new.
    • Coverage: max |Δ| is 1.9e-15 to 2.7e-15 for the old implementation, which also varies between its own runs (map order). For the new one it is a stable 2.2e-15 across runs.
    • So the new results lie inside the old implementation's own run-to-run noise. The test tolerance of 1e-9 is documented in the test header.
  • [T] The golden file matches the old implementation (GoldenBeforeAfter is green on 071c7942). It is therefore not self-generated by the new code.
  • [T] Real fixture: I ran master and merged-tree servers side by side on identically prepared copies of e2e-fixture.db and compared /api/nodes on 195 scored nodes:
    • bridge_score, redundancy_score and traffic_share_score: max |Δ| = 0.
    • coverage_score: max |Δ| = 3.3e-16.
    • usefulness_score: max |Δ| = 5.6e-17.
    • Null patterns are identical. This matches the author's numbers.
  • [A] Code reading confirms the port:
    • Filtering (trim/lowercase, empty, self-loop, NaN/±Inf, < 1e-9) is unchanged. Vertex ids are assigned only after validation, so the vertex set is unchanged.
    • Cheapest-duplicate selection: sort by (from, to, cost), then keep the first of each run.
    • Relaxation, including the tie branch and the lack of a settled check on w, is identical. Back-propagation, halving and max normalization are identical.
    • Coverage still sums 1/d at settle time and excludes src.
    • The typed heap's sift-up/sift-down tie behavior matches container/heap.
    • predBuf cannot overflow a row: predecessors are distinct settled neighbors, and the count resets on every strict improvement.

Readiness not weakened

  • [K] The PR touches only bridge_score.go, coverage_score.go, graph_weighted.go and test/testdata files. main.go, bridge_recomputer.go, usefulness_axes_recomputer.go and the snapshot publication are untouched, so the startup order is unchanged and there is no new interaction with the readiness gates.
  • [T] TestGraphScore561_StartInstallsScoresBeforeReturn is green, and my mutant M5 (axes prewarm made go s.recomputeUsefulnessAxes()) is killed by it.

Benchmark reproduction

  • [T] Before = test binary built from 071c7942; after = test binary built from the merged tree. They ran alternately, 4 rounds each, on darwin/arm64 (Apple M5) with Go 1.26.3. Values are medians, with ranges in brackets; ranges do not overlap on any row.
Benchmark V / E time before time after Δ B/op before → after allocs before → after
Bridge 600 / 2 000 200.5 ms [198–216] 33.5 ms [33–34] −83% 145.1 MiB → 0.31 MiB 1 458 324 → 64
Bridge 2 000 / 10 000 2.92 s [2.90–2.98] 0.480 s [0.472–0.486] −84% 1.89 GiB → 1.29 MiB 17 089 149 → 91
Coverage 600 / 2 000 142.5 ms [141–143] 32.3 ms [32–33] −77% 105.5 MiB → 0.28 MiB 1 108 519 → 50
Coverage 2 000 / 10 000 2.09 s [2.09–2.15] 0.446 s [0.444–0.449] −79% 1.40 GiB → 1.16 MiB 13 119 126 → 75
Warmup 600 / 2 000 289.9 ms [288–291] 65.9 ms [66–66] −77% 245.2 MiB → 1.23 MiB 2 153 180 → 1 773
Warmup 2 000 / 10 000 4.23 s [4.22–4.32] 0.924 s [0.920–0.925] −78% 3.12 GiB → 5.53 MiB 25 135 406 → 8 552

The relative improvements match the PR (−74…−82%). The "after" alloc counts are identical to the PR's. B/op is allocated bytes, not retained heap.

-race

  • [T] go test -race -count=1 -run 'GraphScore561|Bridge|Coverage|Usefulness|Redundancy' . on the merged tree: ok.
  • [A] Each Compute* call owns its workspace. There is no global cache and no sync.Pool.

Acceptance criteria → red before / green after

  • [T] On 071c7942 (old implementation + new tests), three tests fail: DeterministicAcrossRunsAndPermutations, NoStateAcrossCallsOrSizes and AllocationBudget. All three are green on head and on the merged tree.
  • The remaining criteria are pins that are green on both trees by design. These are oracle equivalence, normalization, concurrency and readiness. That is legitimate for a behavior-preserving refactor, and my mutants show the pins are live: see below.
  • WorkspaceReuseIsStateless and IndexedGraphStructure test new internals and can only exist with the new code.
  • Gaps: findings 1 and 2.

No behavior change beyond the goal

  • [T]/[A] The only behavior change is that results are now bit-deterministic. The old low-bit jitter came from map iteration and is documented in the PR.

Standing constraints

Constraint Result Evidence
cmd/server stays read-only Pure compute plus tests; no DB/SQL lines in the diff [K]
No new map[string]interface{} outside tests 0 added (git diff origin/master...head) [K]
No hardcoded colors No frontend/CSS changes [K]
scripts/check-xss-sinks.sh --diff origin/master "no public/**/*.{js,html} changes to scan", exit 0 (run with bash in a scratch clone at head) [K]
Fork guards deploy.yml 9, release-fast-path.yml 1; no .github/ changes in the PR [K]
No closing keywords PR body uses "Relates to #561"; nothing matched in the title, body or commit messages [K]
Commit author Both commits are author and committer dborup <kontakt@meshview.dk> [K]

Tests run (merged tree 6afba08b on origin/master bb91292a)

  • [T] cmd/server: go vet . clean; go test -count=1 ./... ok (75 s)
  • [T] cmd/server -race, focused set as above: ok
  • [T] cmd/ingestor: go test -count=1 ./... ok
  • [T] sh test-all.sh: 271 passed, 0 failed (271 files)
  • [T] node test-frontend-helpers.js: 725 passed, 0 failed
  • [T] E2E against a local merged-tree Go server on port 13900. The fixture was a scratch copy of e2e-fixture.db, prepared as in CI: freshen, the inline seed SQL from deploy.yml, corescope-migrate, then seeds 2073, 199, 245 and 122.
    • test-e2e-playwright.js: 154/157 passed, 3 skipped.
    • test-issue-236-analytics-sorting-e2e.js: PASS.
    • The PR adds no E2E of its own.
    • The server was stopped by its PID, found via lsof on the port; the port was confirmed free afterwards.
  • [K] CI run 40172139883 at head: all 15 executed jobs pass, including Go Server 1–3, Go Ingestor, Go Libraries & CLI, JS Unit & Lint, Script & Container, Playwright E2E 1–3, Verify E2E Shards & Coverage, Verify Go Test Assignments, Validate Docker Image and Build & Publish. Badges, Release and Deploy Staging were skipped as expected. The known flaky tests flaky: #226 Hash Stats adopters sort E2E — server sorts multiByteNodes on packets only, ties come out in random order #256 and flaky: TestResolvedPathBackfill_WriteHoldUnderBudget_188 — wall-clock write-hold budget fails under CI load #267 did not appear. Go Server shard durations are in line with the two latest master runs.

Reviewer mutants (merged tree; filter GraphScore561|Bridge|Coverage|Usefulness)

# Mutant Result
M1 ssspWorkspace.reset no longer clears settled killed: 4 legacy Coverage tests, BridgeMatchesOracle, CoverageMatchesOracle, GoldenBeforeAfter, NoStateAcrossCallsOrSizes; then the finding 1 hang
M2 CSR build keeps duplicate arcs (dedup disabled) killed: BridgeMatchesOracle, IndexedGraphStructure
M3 back-propagation drops the 1.0 + term killed: HandleNodesSurface, legacy Bridge tests, BridgeMatchesOracle, GoldenBeforeAfter, NoStateAcrossCallsOrSizes; then the finding 1 hang
M4 tie test abs(alt-dist) <= 1e-12 → exact == survived (finding 2)
M5 axes initial compute made asynchronous killed: StartInstallsScoresBeforeReturn
M6 distHeap.pop picks the larger right child killed: BridgeMatchesOracle/random-ties-b, CoverageMatchesOracle/grid-4x4, GoldenBeforeAfter; then the finding 1 hang

Not verified

  • I did not rerun the author's own mutant list; I ran my six above instead.
  • No measurement on linux/amd64. My numbers are darwin/arm64 with Go 1.26.3, not the CI toolchain.
  • The real-fixture comparison covers only the small E2E fixture (195 scored nodes, mostly zero redundancy); I did not compare on a production-sized graph. No staging or production access was used.
  • I did not run the E2E suites unrelated to node scores.
  • I did not check retained heap; all memory figures are allocated bytes.
  • I did not browser-validate visually. There is no UI change; the E2E runs exercise the nodes and analytics pages.

@dborup
dborup marked this pull request as ready for review October 25, 2026 08:42
@dborup
dborup merged commit 5699b63 into master Oct 25, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants