Refactor JIT cache layer with JITCallbackSet - #93
Conversation
Refactor the JIT compilation pipeline to use a JITCallbackSet dataclass that groups the five PETSc callback lists (residual, bcs, jacobian, bd_residual, bd_jacobian) into a single structured container. This addresses the root cause of the cache-collision bug (PR #92) at an architectural level: the flat tuple hash that lost callback role information is replaced by a structured signature that preserves which slot each expression belongs to. Changes: - Add JITCallbackSet dataclass with flat(), signature(), map(), counts - Extract _structural_expand() as a module-level function (was inline) - Refactor getext() to accept JITCallbackSet (with backward compat) - Refactor _createext() to accept JITCallbackSet - Update all 6 call sites: 3 solvers (Scalar, Vector, Stokes) and 3 integrals (Integral, Integral._evaluate_integral, BdIntegral) - Include PR #92 regression tests (spherical shell cache collision) Incorporates the fix from PR #92 (gthyagi) which identified the bug and added the regression tests. Test results: 374 passed, 7 skipped, 1 xfailed (level_1 suite) Underworld development team with AI support from Claude Code
|
This is just some tidying prompted by @gthyagi 's bug fix. Hardening the JIT infrastructure is always and improvement. |
There was a problem hiding this comment.
Pull request overview
Refactors the JIT extension cache interface to prevent callback-role collisions by introducing a structured callback container (JITCallbackSet) and updating all callers (integrals, solvers, and tests) to use it.
Changes:
- Introduces
JITCallbackSetand refactorsgetext()/_createext()to accept structured callbacks (fixing role-collision cache bugs by construction). - Extracts cache-key preparation into
prepare_for_cache_key()and uses role-preserving signatures for hashing. - Updates solver/integral call sites and adjusts tests, including spherical-shell regression coverage for boundary/volume cache collisions.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
src/underworld3/utilities/_jitextension.py |
Adds JITCallbackSet, factors cache-key preparation, refactors getext()/_createext() to use structured callbacks and role-preserving signatures. |
src/underworld3/cython/petsc_maths.pyx |
Updates Integral / CellWiseIntegral / BdIntegral to call getext() with JITCallbackSet. |
src/underworld3/cython/petsc_generic_snes_solvers.pyx |
Updates SNES solvers to pass structured callbacks into getext(). |
tests/test_0004_pointwise_fns.py |
Updates direct getext() test calls to use JITCallbackSet. |
tests/test_0502_boundary_integrals.py |
Adds/updates regression tests covering spherical-shell boundary/volume integral cache-collision scenarios. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| residual: tuple = () | ||
| bcs: tuple = () | ||
| jacobian: tuple = () | ||
| bd_residual: tuple = () | ||
| bd_jacobian: tuple = () |
There was a problem hiding this comment.
JITCallbackSet claims slots are tuples and getext() relies on tuple concatenation + hashing (signature()). If a caller passes a list (likely, since the old getext API accepted lists), flat()/signature() will raise (list+tuple TypeError or unhashable list). Consider coercing each slot to tuple (and treating None as empty) in post_init using object.setattr so the container is always immutable + hashable.
| @@ -1,11 +1,11 @@ | |||
| from typing import List | |||
| from typing import List, Optional, Tuple | |||
There was a problem hiding this comment.
The typing imports include List and Tuple, but this module no longer uses them (Optional is still used). Consider removing unused imports to avoid confusion and keep linters clean.
| from typing import List, Optional, Tuple | |
| from typing import Optional |
- Add __post_init__ to coerce list/None inputs to tuples, ensuring immutability and hashability (prevents TypeError in flat()/signature()) - Remove unused List, Tuple imports from typing Addresses Copilot review comments on PR #93. Underworld development team with AI support from Claude Code
|
Copilot fixes adopted. I think we can merge this as this passes all CI tests and it is really quite a simple task. |
Summary
Builds on PR #92 (@gthyagi) which fixed the Integral/BdIntegral JIT cache collision bug. This PR refactors the fix into a reusable architectural pattern that simplifies the JIT interface for all callers.
What PR #92 identified
When
Integral(fn=1.0)andBdIntegral(fn=1.0)ran on the same mesh, the JIT cache key was a flat tuple of all expressions — losing which callback role each expression belonged to. The second call reused the first's cached module, wiring the wrong PETSc callback slot and returning0.0.What this PR does
Introduces
JITCallbackSet, a frozen dataclass that groups the five PETSc callback lists (residual, bcs, jacobian, bd_residual, bd_jacobian) into a single structured container.Before (PR #92 fix): The cache key was made structured by manually building parallel tuples for each role inside
getext(). Callers still passed 5 positional lists:After: Callers declare intent by name, with a clean 3-argument interface:
The structured cache key is now a data structure invariant — you can't construct a callback set without declaring roles, so the collision bug cannot recur.
Changes
JITCallbackSetdataclass withflat(),signature(),map(),countsprepare_for_cache_key()— extracted as a module-level function (was an inline closure). Two-phase expression preparation: constants → placeholders, then unwrap to pure SymPy. Reusable by any code needing structural expansion for hashing.getext(mesh, callbacks, primary_field_list, ...)— clean 3-positional-arg signature_createext(name, mesh, callbacks, primary_field_list, ...)— takesJITCallbackSetdirectly, usescallbacks.countsinstead oflen(fns_residual)etc.Files changed
src/underworld3/utilities/_jitextension.pyJITCallbackSet,prepare_for_cache_key(); refactorgetext()and_createext()src/underworld3/cython/petsc_maths.pyxJITCallbackSetsrc/underworld3/cython/petsc_generic_snes_solvers.pyxJITCallbackSettests/test_0004_pointwise_fns.pygetext()callstests/test_0502_boundary_integrals.pyTest plan
Underworld development team with AI support from Claude Code