Fix integer precision loss in mysave for large meshes#23
Open
AmitSubhash wants to merge 4 commits into
Open
Conversation
…t-product-errors I'M A GOD FOR FIXING THIS THIS QUICK HAHAHAHAHA
The %g format specifier defaults to 6 significant digits, which silently truncates node indices above ~1e6 when writing .elem files. For example, node index 1932581 is written as 1.93258e+06 and read back as 1932580. This causes "invalid triangulation" errors in PlotMeshSurface when the mesh has more than ~1 million nodes, because element indices reference node positions that no longer exist after the round-trip through save_mesh/load_mesh. The fix detects integer-valued data and uses %d instead of %g, while preserving %g for floating-point data like node coordinates.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
mysave.muses%gformat to write all data, but%gdefaults to 6 significant digits1932581becomes1.93258e+06=1932580)save_mesh/load_meshround-trip, elements reference non-existent node indicestriangulationerrors inPlotMeshSurfaceand potentially corrupted light modelsRoot cause
%gwith 6 significant digits cannot represent integers above ~999,999 exactly. Element files (.elem) contain node indices that must be exact integers.Fix
Detect integer-valued data and use
%dformat instead of%g. Floating-point data (e.g. node coordinates in.nodefiles) continues to use%g.Reproduction
NirfastMesh_Regionsave_meshthenload_meshmax(mesh.elements(:)) > size(mesh.nodes, 1)-- should be false, but is true due to precision lossPlotMeshSurfacefails with "The input triangulation is invalid"Test plan
.elemfiles use integer format (no scientific notation) after fix.nodefiles still use floating-point formatload_meshround-trip preserves exact element indices for meshes with >1M nodes