Repository navigation
feat: add multi-seed MovieLens 100K MF vs GraphSAGE leaderboard - #8
Merged
Merged
Conversation
Record ADR-007 and aggregate implicit MF and native-backed GraphSAGE across seeds 7, 11, 13, 17, 19 with mean±sample std. Reuse the Phase 5 PairScorer protocol and keep historical single-seed result files unchanged. Co-authored-by: Nithilan Kumaran <sathariels@users.noreply.github.com>
Co-authored-by: Nithilan Kumaran <sathariels@users.noreply.github.com>
Co-authored-by: Nithilan Kumaran <sathariels@users.noreply.github.com>
Commit JSON, markdown, and error-bar chart from a real ADR-007 run on seeds 7, 11, 13, 17, and 19. Phase 5 single-seed files are unchanged. Co-authored-by: Nithilan Kumaran <sathariels@users.noreply.github.com>
sathariels
marked this pull request as ready for review
September 15, 2026 20:29
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Opens the ADR-007 multi-seed MovieLens 100K leaderboard: implicit MF vs GraphSAGE on the same ADR-003 protocol, eligible users, candidate construction, and Recall@10 / NDCG@10, across seeds 7, 11, 13, 17, 19.
PairScorer+sagerec_metrics, GraphSAGE via nativegraph_sampler/sagerec_minibatch+ PyGSAGEConv).python/sagerec_multiseed.pywrites per-seed rows plus mean ± sample std (n-1) (median stored too) and an error-bar chart.scripts/run_multiseed_gnn_vs_mf.py(not run in default CI).results/mf_movielens_100k.json,results/graphsage_movielens_100k.json,results/gnn_vs_mf_*) stay single-seed provenance and were not overwritten.Measured artifacts from this environment are committed:
results/multiseed_gnn_vs_mf_movielens_100k.jsonresults/multiseed_gnn_vs_mf_movielens_100k.mdresults/multiseed_gnn_vs_mf_movielens_100k.svgCopied from that JSON (943 eligible users):
Seed-7 MF matches the historical Phase 5 MF file. Seed-7 GraphSAGE is a new SAGEConv measurement and is not mixed with the Phase 5 in-repo mean-layer number.
Fairness
Shared eval protocol only. MF remains 1 SGD epoch; GraphSAGE remains 3 Adam epochs. NeighborLoader is still forbidden on the primary path. This is not a SOTA claim.
Tests
Schema/aggregation unit tests, historical-file labels, committed-file math check, and a tiny synthetic two-seed smoke. Default CI does not download MovieLens or run the 5-seed 100K job.
Release CTest is green locally (
17/17, includingpython_binding_smoke).