fix(layout): make Smart Layout refinement deterministic and never worse

[tomasz-zajac-oss]

Summary

Smart Layout's simulated-annealing stage was making results worse more often than better. In a before/after run on the bundled sample model and synthetic landscapes, the refined layout scored worse than the best raw ELK candidate in 10 of 14 runs, often by overlapping boxes to shave off crossings. The same diagram also got a different layout on every click, and a collapsed container with related children made all 9 ELK candidates throw (Referenced shape does not exist).

This PR fixes those and makes the output reproducible.

• Visible graph only. Layout engines (ELK, Radical, Smart) get only the nodes that are drawn. Relations of hidden children are re-attached to the nearest visible ancestor, the same way deriveRFEdges does it (projectToVisibleGraph in geometry.ts). The store builds layout input with the render path's rules (view-collapsed, per-view collapsed/expanded, hidden relations). It also stops overwriting a collapsed node's stored size with the collapsed size. This also fixes Tree Layout, which crashed on collapsed containers the same way.
• No overlaps. The annealer rejects any move that creates or worsens sibling overlap. Every candidate and phase is scored after fitting parents and separating boxes (layoutFinalize.ts), so the score describes what gets rendered. Compound padding is shared with the store's fitParentToChildren.
• Never worse. The refined layout competes with the best raw candidate and with the current layout. If the current layout still wins, nothing is moved and the user is told so (keptCurrent).
• New annealing engine (annealing.ts).
  • Deterministic: the PRNG is seeded from the diagram's structure, and budgets are counted in estimated work, not milliseconds. Deadlines remain only as a safety cap.
  • Step size (px) is separated from temperature, which is calibrated from sampled uphill moves.
  • The aestheticBoost discontinuity is removed; it used to make removing the last crossing cost energy.
  • Proxy energy is incremental: a move only re-tests the pairs it affects. It is verified against a full recount in tests.
• Radical layout.
  • Compounds are laid out recursively by parentId. Before, nested systems, databases and queues got no position: 16 of 20 nodes in the bench fixture.
  • Cycles are broken from the entry points before longest-path layering. Before, A↔B pushed the system users enter below the one calling back into it.
  • External systems are sized from their children.
• Bench fix. saThroughput.bench.mjs imported a no-longer-exported function.

Results

Composite cost is the Smart Layout score (lower is better). "Before" is the mean of 3 runs of origin/main, re-scored with this branch's scorer. "After" is identical on every run.

Input	Before	After
Sample: Core Banking	118	2
Sample: Requirements Traceability	94	8
Sample: Payments Domain	47	13
Sample: whole model	17 335	16 118
Bench fixture (fintech)	377	131
Synthetic landscape, 95 nodes	80 249	24 841
Sample: Governance	701 (91 / 119 / 1893)	174
Sample: Payment Flow	3	5

• Across 16 inputs, 14 improved on average. Governance is worse than the old pipeline's good runs; its remaining cost is 3 crossings that need moves the no-overlap constraint rules out. Payment Flow went from 3 to 5.
• Overlapping sibling pairs: before up to 22, after 0 on every input and run.
• Time: small views take 20–650 ms (was 0.2–1.4 s); large ones are the same or faster.

Known issues / follow-ups

• Main-thread block (not introduced here). The ELK candidate phase still blocks the main thread for about 3.4 s on the whole sample model. Almost all of it is minimizeCrossings (300–400 ms × 10 candidates), not ELK itself. Moving it into the worker is the natural next step.
• Unexplained stall in benchmarking. While benchmarking, the candidate phase twice took about 15 minutes instead of 3.4 s on the whole model. It did not reproduce in 12 further runs, and the CPU-bound annealing in those same runs was unaffected. I suspect background-process timer throttling in the sandbox, but it isn't proven, so it's worth running Smart Layout on the whole sample model in the app.
• Seed sensitivity. One seed is one draw: the same graph with renamed relation ids scores 131 vs 211. runSmartLayoutSAPhase already takes a seed, so a "try another arrangement" action would be small.

Test plan

• npm run typecheck
• npm test: 428 passed, 15 of them new. 6 of the new tests fail on origin/main: nested compounds, cycle entry order, expanded externals, determinism, keep-current, collapsed container.
• Incremental energy matches a full recount after 500 random moves.
• Before/after benchmark on the sample model views and synthetic graphs (3 runs each).
• Run Smart Layout, Tree Layout and Radical Layout in the app on the sample model, including a view with collapsed containers (not done: no app automation available).

🤖 Generated with Claude Code