Epsilon value is left relaxed by a number of tests in various suites

[YYDan]

Description

Some test suites reset the float epsilon and others leave it relaxed, which could mean subsequent tests pass when they should not.

Suites that leave epsilon modified (the bug)

Suite	Leaking facts	Left at
BasicMathTestSuite	11	0.001 / 0.0001 / 0 / -0.001 / int64
ResourceCameraTestSuite	8 (camera_matrix_test #1–#8)	0.01
ResourceSequenceTestSuite	9 (sequence.*, sequenceInstance.*)	0.001 / 0.01
BasicMathTestSuite leaking facts: lengthdir_x_test, lengthdir_y_test, point_direction_test, point_distance_test, point_distance_3d_test, math_get_epsilon_test #1/#2, math_set_epsilon_test #1–#4.

The math_get/set_epsilon facts are arguably fine — mutating epsilon is their subject — but they should still restore on exit.

The other five (lengthdir_, point_) raise epsilon for legitimate float tolerance and simply never put it back.

Suites that handle it correctly

Suite	Facts	Pattern
BasicBufferTestSuite	6	0.01 at start  0.00001 at end — true set/restore
BasicMatrixTestSuite	14	0.01 at start  0.00001 at end — true set/restore

The middle category — defensive, not restoring

BasicDateTimeTestSuite (7 facts) and 19 facts in BasicMathTestSuite (tan_test, arccos_test, ln_test, power_test, etc.) call math_set_epsilon(0.00001) at the start and never raise it. Comment on those lines: "Set an explicit epsilon to account for different default epsilons in YYC and VM."

So they don't leak, but they aren't restoring either — they're defending themselves against inherited state. That's the load-bearing accident here: enough facts self-baseline that the leak mostly gets papered over.

Blast radius

No setUp/tearDown hooks in any of the six suites — nothing resets epsilon between facts or between suites.

Registration order is in projects/xUnit/objects/objRunner/Create_0.gml:

• BasicMathTestSuite gets away with it — its last fact is tan_test, which self-baselines to 0.00001. Pure luck of file ordering. Facts within the suite that follow a leaker still inherit it.
• ResourceCameraTestSuite followed by ResourceEventsTestSuite, inherits 0.01.
• ResourceSequenceTestSuite followed by ResourceSpriteTestSuite, inherits 0.001/0.01.

0.01 is a very generous tolerance for anything comparing floats downstream — tests could be passing that shouldn't be.

epsilon_audit.py

Expected Change

A framework-level epsilon reset in the suite/fact teardown rather than 44 hand-written restore calls, since the manual pattern is exactly what's been drifting.

Script Name

See above

Fact/Assert Name

See above

How reliably does this issue happen?

Always

[YYDan]

Phases 1-6 increased the blast radius of the epsilon-leak bug, as expected under the agreed approach of replicating the existing defensive math_set_epsilon() calls into each new fact rather than blocking on the fix.

Facts in BasicMathTestSuite that leave epsilon at a non-default value have gone from 7 to 32:

Test	Before	After	Left at
lengthdir_x_test	1	3	0.001
lengthdir_y_test	1	2	0.001
point_direction_test	1	7	0.0001
point_distance_3d_test	1	7	0.0001
point_distance_test	1	7	0.0001
math_get_epsilon_test	1	2	0 / test-computed
math_set_epsilon_test	1	4	-0.001 / 0 / 0.05 / int64

No new leak sites were introduced — the same calls, now replicated across self-contained facts. The epsilon fix cleans all 32 up in one pass.

[YYDan]

Fixed — framework-level baseline instead of hand-written restores

Resolved as the issue's Expected Change described: a framework-level reset rather than per-fact restore calls.

Fresh audit

The counts recorded higher up on this issue were pre-#329, so fresh numbers are given below.

Category	Facts	Calls
A. Leaking (last value set ≠ 0.00001)	32	53
B. Set and restore	0	0
C. Defensive (single call, already the default)	260	260
D. No epsilon call	334	0
Total	626	313

The fix

00a57bc — reset the float epsilon at the start of every test (17 insertions, 0 deletions)

Test.gml gains a macro and a static:

#macro test_default_epsilon 0.00001

static resetGlobalState = function() {
    math_set_epsilon(test_default_epsilon);
}

resetGlobalState() is called as the first statement of preRunFunc in both Test.gml and TestAsync.gml. Those two call sites cover every test type in the hierarchy (Fact, Theory, TestSuite, TestFrameworkRun).

Three deliberate choices:

• Start of run, not teardown. postRunFunc is skipped when a test bails or expires — exactly the cases where a relaxed epsilon is most likely to be left behind. Resetting on entry guarantees a clean baseline regardless of how the previous test died.
• Before callStartHook(), so the existing (currently unassigned) startHook can still deliberately override the baseline.
• A general resetGlobalState() rather than an epsilon-specific call, so any future global that tests are permitted to modify has an obvious home.

bef5a6d — remove redundant defensive epsilon calls (585 deletions, 0 insertions)

With the baseline guaranteed, all 260 category-C calls are dead code. Removed, along with their trailing blank lines and the 65 surviving // Set an explicit epsilon to account for different default epsilons in YYC and VM. comments. The diff is deletions only.

The 32 category-A facts were left untouched: they relax tolerance for their own asserts and can no longer leak, so they need no restore. Same for math_get/set_epsilon_test, where mutating epsilon is the subject under test.

Side effects

• ResourceCameraTestSuite (8 facts at 0.01) and ResourceSequenceTestSuite (9 facts at 0.001/0.01) are also fixed by the same change, without either file being edited.
• BasicMatrixTestSuite (28 calls) and BasicBufferTestSuite (12) now carry redundant restore calls. These were the two suites doing it correctly by hand. Harmless but no longer necessary — worth a separate tidy-up issue rather than widening this branch.

Code verification

Check	Before	After
Facts	626	626
Fact names	—	byte-identical
Asserts	703	703
Brace / paren / bracket balance	0 / 0 / 0
Duplicate fact names	0	0
Category C facts	260	0
Real math_set_epsilon calls	313	53

Results after running TestFramework

All seems fine to me after this change, as we did not suddenly get lots of tests start failing due to epsilons being wrong.

[INFO]: TestFramework ended
[INFO]: {"failed":1.0,"passed":2336.0,"skipped":6.0,"expired":6.0}
[INFO]: FAILED: 0.0
[INFO]: EXPIRED: 0.0
[INFO]: EXPIRED: 0.0
[INFO]: EXPIRED: 0.0
[INFO]: EXPIRED: 0.0
[INFO]: EXPIRED: 0.0
[INFO]: EXPIRED: 0.0

No idea why one check says a single test fails, but then the other does not - something to investigate as part of another issue.