Skip to content

feat: add native vs Python neighbor-sampler timing charts - #6

Merged
sathariels merged 2 commits into
mainfrom
cursor/phase-2-timing-charts-726a
Sep 13, 2026
Merged

sathariels merged 2 commits into
mainfrom
cursor/phase-2-timing-charts-726a

Conversation

@sathariels

@sathariels sathariels commented Sep 13, 2026 •

Copy link
Copy Markdown
Owner

Summary

Opens the Phase 2 timing-chart slice: a reproducible C++ native vs Python reference sample_neighbors benchmark with measured stored evidence.

  • python/sagerec_sampler_benchmark.py builds a deterministic synthetic CSR, verifies ADR-005 without-replacement parity, then times graph_sampler.BipartiteCSR.sample_neighbors against sagerec_reference_sampler with warm-up and multiple measured repetitions.
  • Headline latency, throughput, and speedup use the median of measured repetitions (mean/min/max are also recorded). Speedup is reference_median_seconds / native_median_seconds from measured seconds only.
  • Provenance includes graph size, node-query workload, k, replacement policy, seed, build type, compiler, Python, CPU, and the timing statistic.
  • scripts/run_sampler_timing.py writes:
    • results/sampler_timing.json
    • results/sampler_timing.md
    • results/sampler_timing.svg
  • Default source is synthetic so CI never downloads MovieLens. --source movielens-100k uses existing train-only processed/ artifacts and never downloads.
  • Unittests cover parity-before-timing (timer is not called on mismatch) and the required JSON schema. CTest python_binding_smoke discovers the new tests.

Docs no longer mark Phase 2 timing charts as closed.

Does not invent ML quality metrics, retcon MF/GraphSAGE JSON, add MovieLens 1M, GCN, PyG NeighborLoader, multi-seed leaderboards, or Rust.

Measured locally (Release, this agent environment)

Copied from the committed JSON (synthetic 512×1024 graph, degree 32, k=10, 4000 queries, seed 7, 3 warm-up + 9 measured reps, GNU 13.3.0, Python 3.12.3):

Sampler Median seconds Median queries/s
native graph_sampler 0.003911 1,022,638
Python reference 0.434863 9,198

Median speedup (reference / native): 111.177×. One-machine sampler evidence, not a production latency or SOTA claim.

How to verify

cmake -S cpp -B build -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=g++
cmake --build build --config Release
ctest --test-dir build --output-on-failure --build-config Release
PYTHONPATH=build:python python3 -m unittest python/tests/test_sampler_benchmark.py -v
PYTHONPATH=build:python python3 scripts/run_sampler_timing.py

Locally: Release CTest 17/17 passed, including python_binding_smoke. Default CI must stay on the synthetic path.

Open in Web Open in Cursor 

cursoragent and others added 2 commits September 13, 2026 21:49
Open the Phase 2 timing-chart slice with a synthetic-graph benchmark
that verifies ADR-005 parity before measuring graph_sampler against
the Python reference sampler, plus schema tests and docs.

Co-authored-by: Nithilan Kumaran <sathariels@users.noreply.github.com>
Commit Release-build JSON, markdown, and SVG from the synthetic
ADR-005 workload. Infer compiler id/version when CMake omits them,
and chart median throughput so both bars stay readable.

Co-authored-by: Nithilan Kumaran <sathariels@users.noreply.github.com>
@sathariels
sathariels marked this pull request as ready for review September 13, 2026 21:53
@sathariels
sathariels merged commit a9900cd into main Sep 13, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants