cxxtest baselines fail under msvc 192 (VS2019): floating-point divergence from VS2022

[gagelarsen]

Summary

Building xmsmesher for msvc 192 (VS2019) fails the cxxtest suite — 21 of 214 tests — while the same source passes under msvc 194 (VS2022) in CI. The failures are floating-point comparison mismatches against checked-in baseline files, not build or link errors.

Found while putting xmsmesher 6.0.8 onto the VS2019 bridge (aquaveo-vs2019). Every other library in the suite — xmscore, xmsgrid, xmsinterp, xmsextractor — built 14/14 clean; xmsmesher is the only one that is this FP-sensitive.

Evidence

Divergence appears around the 9th significant digit:

ND 27  8.5108203108082   31.7579582765     <- baseline (generated with VS2022 / msvc 194)
ND 27  8.5108203218325   31.75795805983    <- produced by VS2019 / msvc 192

Failing assertions are in:

• xmsmesher/meshing/MeMultiPolyTo2dm.cpp:333 and :572 — ~18 Files different on line N failures
• xmsmesher/tutorial/TutMeshing.cpp:284 — 3 failures

Which configurations

The split is exactly along the testing option. Everything that only compiles and links succeeds; only the four configurations that run the suite fail.

result	configurations
FAIL (4)	Debug-dynamic-testing, Debug-static-testing, Release-dynamic-testing, Release-static-testing
ok (10)	both wchar_t variants × Debug/Release × static/dynamic, plus Release-dynamic-pybind for py3.10 and py3.13

So the msvc 192 binaries themselves are fine — it is the baseline comparison that does not hold across compilers.

Why the existing FP workaround does not cover this

build.toml already carries an FP-determinism guard:

# Disable FP contraction (FMA fusion) for consistent floating-point results across
# architectures. GCC on ARM emits fmadd at -O2+ which breaks exact arithmetic
# predicates and produces non-deterministic geometry results.
if (NOT MSVC)
    add_compile_options(-ffp-contract=off)
endif()

It is scoped if (NOT MSVC) because MSVC does not contract by default. That reasoning holds for x86_64-vs-ARM under GCC, but it does not address msvc 192 vs msvc 194 producing different results from the same source — which is what this issue is about.

Current state

xmsmesher 6.0.8 is published to aquaveo-vs2019 with 10 of 14 package ids. The four testing=True variants are absent for msvc 192. Nothing in the XMS dependency DAG consumes xmsmesher's testing package, so this does not block any other library; it means xmsmesher's own test variants cannot be built for VS2019.

aquaveo-stable is unaffected and carries the full msvc 194 / gcc / clang set.

Options

1. Relax the comparison tolerance in MeMultiPolyTo2dm.cpp and TutMeshing.cpp so the assertions compare with an epsilon rather than exact text. Correct long-term; needs someone who knows the acceptable numerical tolerance for these meshes.
2. Pin FP behavior for MSVC (/fp:strict, or extending the contraction guard to MSVC) so both compilers agree. Most principled, but it would change VS2022 output and therefore require regenerating baselines anyway.
3. Accept the gap and keep skipping the testing variants on VS2019 via --filter '{"options": {"testing": false}}'. This is the current state.

Option 3 is what is in place. 1 or 2 wants an owner who knows the meshing numerics.

Note: unrelated tooling bug

xmsconan vs2019 build exited with code 0 despite 4 failed configurations — the summary table reported succeeded 9 / failed 4 but the process return code was 0. A script trusting the exit code would treat the run as clean and publish a partial set silently. Filed separately against xmsconan; noting here because it is how this nearly went unnoticed.

[gagelarsen]

Narrowed by 6.1.0: msvc 192 is the sole outlier, and it is not a stale baseline

Releasing 6.1.0 produced evidence that materially narrows this issue. Two things are now ruled out.

1. The three CI toolchains agree with each other — byte for byte

While fixing case3_base.2dm in #86 I compared the case3_out.2dm produced by each CI toolchain against the others:

comparison	result
msvc 194 vs gcc 13	IDENTICAL
msvc 194 vs apple-clang	IDENTICAL
gcc 13 vs apple-clang	IDENTICAL

So this is not a general "mesh baselines are not portable across compilers" problem. Three quite different toolchains produce bit-identical mesh output. msvc 192 is the only outlier.

2. It is not explained by a stale baseline

case3_base.2dm was genuinely stale — both master's copy and the feature branch's differed from what all three CI toolchains produce, and it was regenerated in 6.1.0.

That correction did not change the msvc 192 result:

release	msvc 192 result
6.0.8 (stale baseline)	21 failed of 214
6.1.0 (corrected baseline)	21 failed of 231

Identical failure count, with the 17 new raster tests all passing. Current distribution:

17  MeMultiPolyTo2dm.cpp:333
 1  MeMultiPolyTo2dm.cpp:572
 3  TutMeshing.cpp:312

Consequence

The failures are a real msvc 192 code-generation divergence, not baseline rot and not a cross-platform property of the tests. That is a much more tractable problem than the one originally described here — it means a fix only has to reconcile one compiler with the other three, rather than making the baselines toolchain-agnostic in general.

Negative result: lowering the printed precision does not work

Tried on internalFeatures/case3. precision is a field width, not a count of significant digits, and STRstd falls back to scientific notation when a value will not fit:

width	x (~2.63e6)	y (~6.1e5)
10 (current)	2632423.87 ✅	614430.008
9	2.63e+06 ❌ destroyed	614430.01 ✅
8	2.6e+06 ❌	6.1e+05 ❌

This test's x and y magnitudes differ by ~4×, so any width that rounds y enough also discards x entirely. That is why case100–case103 can use widths of 6 and 7 while this test cannot. Do not re-try this approach here.

Current state

xmsmesher 6.1.0 publishes 10 of 14 package ids to aquaveo-vs2019; the four testing=True configurations are absent for msvc 192. Nothing in the XMS dependency DAG consumes xmsmesher's testing package, so no other library is blocked. aquaveo-stable carries the full msvc 194 / gcc / clang set at the same recipe revision.

[gagelarsen]

Recurred on 6.1.1 — and the failure set is unchanged

Publishing xmsmesher 6.1.1 (the dependency bump to xmsgrid 9.1.0 / xmsinterp 7.0.11) to aquaveo-vs2019 hit this again, with the same split as 6.0.8: 10 of 14 configurations published, the four testing ones failed.

Missing package ids on aquaveo-vs2019 for 6.1.1#3670dda5d176740708dd8632b2d13c75:

configuration	status
Debug-dynamic-testing	not published
Debug-static-testing	not published
Release-dynamic-testing	not published
Release-static-testing	not published

The 10 that did publish are the 8 non-testing variants (wchar_t builtin/typedef × Debug/Release × static/dynamic) plus both pybind builds (py3.10, py3.13). Recipe revision is identical to aquaveo-stable, so only binary coverage differs.

Worth recording: this was checked against the alternative explanation

6.1.1 bumps xmsgrid to 9.1.0, which contains a behavior change — the fix for issue 0015785 narrows GmMultiPolyIntersector output where an entry had no matching exit. That makes "the mesher baselines now fail" ambiguous: it could be this FP issue, or it could be the new intersection behavior reaching mesher's baselines.

It is not the intersection change. Rebuilding 6.1.0 (old pins: xmsgrid 9.0.11) for msvc 192 and diffing the failing test sets:

6.1.0 (xmsgrid 9.0.11): 21 failing tests
6.1.1 (xmsgrid 9.1.0):  21 failing tests
diff: identical

Same 21 tests, same assertions — 17 at MeMultiPolyTo2dm.cpp:333, 1 at :572, 3 in TutMeshing.cpp:312. The testCase4 divergence is still the documented signature (8.5108203108082 vs 8.5108203218325, ~9th significant digit).

Note the line number in TutMeshing moved from :284 to :312 since this issue was filed — that is source churn between 6.0.8 and 6.1.1, not a new failure.

Comparing the set rather than the count was deliberate: a count-only check is what previously let a new structural failure hide inside an unchanged total.

Also worth noting: msvc 194 CI passed all 9 jobs on this same commit, so the divergence remains strictly toolchain-specific.