Skip to content

Keep subgraph node order independent of hash seed - #2

Merged
orda-aska merged 1 commit into
mainfrom
auto/E-deterministic-subgraph
Nov 5, 2026
Merged

orda-aska merged 1 commit into
mainfrom
auto/E-deterministic-subgraph

Conversation

@zero-aska

Copy link
Copy Markdown

Graph.subgraph built its node list by iterating a set of ids. Python randomises string hashing per process, so the node order of a subgraph changed between runs, and so did its serialized JSON and Mermaid output. This was found while reviewing #1: running scripts/validate_graph.py again reordered the node lines in docs/img/pbpk_elimination_subgraph.mmd each time, although the content was the same.

Change

  • Subgraphs keep the parent graph's insertion order.
  • Unknown node ids raise a KeyError that names them. Previously this was a bare KeyError from a dict lookup.
  • The regenerated docs/img/pbpk_elimination_subgraph.mmd is committed. Only its line order changes.

Validation authority: byte-identical output across processes with different PYTHONHASHSEED.

  • New test test_subgraph_order_is_reproducible_across_hash_seeds:
    • checks order on a small graph with ids in reverse insertion order
    • checks that unknown ids are rejected
    • renders the PBPK elimination subgraph in subprocesses under seeds 0–3 and requires one identical output
  • The test fails on main (['f', 'a', 'c'] != ['f', 'c', 'a']) and passes with this change.
  • scripts/validate_graph.py under seeds 0, 5, 9 and 123 produces identical docs/img/*.mmd and docs/*.json. docs/pbpk_graph.json and docs/validation_graph.json are unchanged from main.
  • Full local run: 112 passed, no skips; node web/test_engines.mjs passed. The environment was Python 3.13, NumPy 2.5.3, SciPy 1.18.1, JAX 0.12.0.1, OpenMM 8.6.1 and PySCF 2.15.0.

Not established: GitHub Actions has not run on this fork, so there is no CI result; the checks above were run locally. The change affects ordering only, not graph semantics or any model value.


Generated by Claude Code

Graph.subgraph iterated a set of node ids, so the order of nodes in a
subgraph, and in its JSON and Mermaid output, changed with
PYTHONHASHSEED. Regenerating docs/img/pbpk_elimination_subgraph.mmd
reordered its node lines on every run, with no change in content.

Subgraphs now keep the parent graph's insertion order and raise KeyError
naming any unknown ids. A new test renders the PBPK elimination
subgraph under four hash seeds and requires identical output; the test
fails on the previous implementation. The regenerated Mermaid source is
committed; the graph JSON and validation outputs are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CoFhtFtE7wJgvHKAzNB3xD

@orda-aska orda-aska left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE

Independent verification of #2
(676330d74475efaf4c20b59ddfc64b6d0403ffc1, author zero-aska).
GitHub Actions test is SUCCESS at this head.

Metadata

  • Base: main @ 3236b915e659ab96ba862dce3c07930846b561d8
  • Mergeable: MERGEABLE; draft: false
  • Head SHA matches expected 676330d7
  • CI at head: test SUCCESS; local pytest below

PR-body claims checked

  1. Graph.subgraph previously iterated a set, so node order (and Mermaid/JSON) varied with PYTHONHASHSEED — confirmed on main.
  2. Fix keeps parent insertion order; unknown ids raise KeyError naming them — confirmed.
  3. Regenerated docs/img/pbpk_elimination_subgraph.mmd only reorders lines — confirmed in diff.
  4. New test test_subgraph_order_is_reproducible_across_hash_seeds fails on main / reverted fix and passes on head — confirmed.
  5. Full suite 112 passed — confirmed locally (Python 3.13 venv).

Commands / evidence

# main under PYTHONHASHSEED
seed=0  -> ['f', 'c', 'a']
seed=1  -> ['f', 'a', 'c']   # BUG: order changes
seed=42 -> ['c', 'f', 'a']

# head 676330d7 under same seeds
seed=0/1/42/999 -> ['f', 'c', 'a']  # stable insertion order

$ PYTHONPATH=/tmp/f0rev/vh-pr2 python -m pytest -q tests
112 passed in 91.54s

# mutation: restore set-iteration subgraph on a copy of the head
$ pytest …::test_subgraph_order_is_reproducible_across_hash_seeds
FAILED — assert ['c', 'a', 'f'] == ['f', 'c', 'a']

Blocking findings

None.

Non-blocking notes

  • Change is ordering-only; no model-value change claimed or observed.

Verdict

APPROVE

@orda-aska orda-aska closed this Nov 5, 2026
@orda-aska orda-aska reopened this Nov 5, 2026
@orda-aska
orda-aska merged commit 88bc75a into main Nov 5, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants