ext/PinaxTestExt.jl:279-295 recovers the real got/want/tol by pattern-matching the isapprox expression that Test already evaluated. The comment says it plainly: "Measured on Julia 1.12."
The shape it depends on:
Test.Pass.data must be an ExprTest.Fail.data must be a String that Meta.parses
Both are wrapped in guards that return nothing, and _check_from (src/testset.jl:504-512) turns nothing into got=1.0, want=1.0, tol=0.5 -- the generic indicator.
The failure mode
If a future Julia changes either shape, every check in every report becomes a 1/0 indicator with margin 0.0. The suite still passes, the report still renders, the verdict is still PASS, and the delta/tol that is the entire product is gone -- with no warning, and no way for a reader to tell it apart from "this suite has no isapprox tests".
Project.toml declares julia = "1.12" with no upper bound.
Why this is the wrong dependency to take silently
These are undocumented internals of the Test stdlib, taken by a rendering package. That is defensible only if the failure is loud.
Fix
Assert the shape once, at capture-install time, and @warn (or error) if it does not hold -- instead of degrading every check individually and invisibly. Consider an upper bound on the julia compat until the shape is verified on the next minor.
Reported by review; the mechanism was read, the degradation was not executed against a modified Julia.
ext/PinaxTestExt.jl:279-295recovers the realgot/want/tolby pattern-matching theisapproxexpression thatTestalready evaluated. The comment says it plainly: "Measured on Julia 1.12."The shape it depends on:
Test.Pass.datamust be anExprTest.Fail.datamust be aStringthatMeta.parsesBoth are wrapped in guards that return
nothing, and_check_from(src/testset.jl:504-512) turnsnothingintogot=1.0, want=1.0, tol=0.5-- the generic indicator.The failure mode
If a future Julia changes either shape, every check in every report becomes a 1/0 indicator with margin 0.0. The suite still passes, the report still renders, the verdict is still
PASS, and thedelta/tolthat is the entire product is gone -- with no warning, and no way for a reader to tell it apart from "this suite has noisapproxtests".Project.tomldeclaresjulia = "1.12"with no upper bound.Why this is the wrong dependency to take silently
These are undocumented internals of the
Teststdlib, taken by a rendering package. That is defensible only if the failure is loud.Fix
Assert the shape once, at capture-install time, and
@warn(or error) if it does not hold -- instead of degrading every check individually and invisibly. Consider an upper bound on thejuliacompat until the shape is verified on the next minor.Reported by review; the mechanism was read, the degradation was not executed against a modified Julia.