Render Module & Repo Tidy Up

.gitignore

@@ -7,6 +7,7 @@
 pyvale-env/
 .venv/
 .pytest_cache/
+temp/
 
 # Output files
 *.npy
@@ -19,9 +20,11 @@ pyvale-env/
 *.blend
 *.csv
 *.json
+*.bmp
 pyvale-output/
 pyvale-input/
 test-output/
+render-fails/
 test-input/
 
 # MOOSE / gmsh
@@ -32,10 +35,13 @@ test-input/
 # Keep these as they are used for examples
 !src/pyvale/data/*.tiff
 !src/pyvale/data/*.e
+!src/pyvale/data/**/
+!src/pyvale/data/**
 !reports/images/*.png
 !reports/images/*.svg
 !images/*.png
 !images/*.svg
+!images/*.yaml
 !docs/source/_static/*.png
 
 # Keep these as they are gold output for regression tests
@@ -50,13 +56,16 @@ test-input/
 !tests/dataio/txt_gold/*.npy
 !tests/blender/2D_gold/*.npy
 !tests/blender/3D_gold/*.npy
+!tests/render/gold_riley/*.npy
+!tests/render/gold_blender/*.npy
 
 # Image deformation test cases
 *.pkl
 *.dill
 imdefcase*_*/
 deformed_images/
 src/deformed_images/
+src/pyvale/examples/render/out-riley-py
 
 # Cython diagnostics
 src/pyvale/cython/*.html
@@ -69,11 +78,14 @@ src/pyvale/cython/*.html
 # restructured text files for API documentation.
 pyvale.rst
 pyvale.*.rst
+!docs/source/pyvale.render.uvtools.rst
 
-# files generated by sphinx-gallery extension
-docs/source/examples/*/
-# keep the gallery layouts
+# Files generated by sphinx-gallery. Keep the hand-written navigation files.
+docs/source/examples/**
+!docs/source/examples/examples.rst
 !docs/source/examples/examples_*.rst
+docs/source/sg_execution_times.rst
+docs/source/sg_execution_times.rst
 
 # doxygen folder generated when compiling docs
 docs/source/doxygen
@@ -259,10 +271,17 @@ cpp/Makefile
 #.idea/
 
 
-# files generated by sphinx-gallery extension
-docs/source/examples/
-
 # doxygen folder generated when compiling docs
 docs/source/doxygen
 
 .DS_Store
+
+# Legacy temporary output from strain tests.
+/tests/strain/test/
+
+# Example scratch output written relative to the example source directories
+# (sphinx-gallery and manual example runs); never commit these.
+src/pyvale/examples/**/pyvale-output/
+src/pyvale/examples/**/out-riley-py/
+src/pyvale/examples/**/deformed_images/
+src/pyvale/examples/**/tmp_dic/

CONTRIBUTING.md

@@ -80,7 +80,7 @@ git switch -c branchname
 
 ### Developer Guidance
 
-Much of Pyvale is written in Python. See our [Pyvale Developer Guide](https://github.com/Computer-Aided-Validation-Laboratory/pyvale/blob/main/designspec/README.md) for guidance on how to write Python code that best aligns with Pyvale's code values.
+Much of Pyvale is written in Python. See our [Pyvale Developer Guide](https://github.com/Computer-Aided-Validation-Laboratory/pyvale/blob/main/dev/README.md) for guidance on how to write Python code that best aligns with Pyvale's code values.
 
 ### Docstrings
 

dev/ABBREVIATIONS.md

@@ -0,0 +1,91 @@
+# Pyvale Common Abbreviations
+
+A glossary of abbreviations used across the `pyvale` source code, grouped by
+usage. Keep new abbreviations consistent with these forms where possible.
+
+## Loop Iterators
+
+Double-letter iterators are the numpy-style convention sanctioned in
+`dev/README.md`. The letters hint at what is being iterated over:
+
+- `ii`, `jj`, `kk`, `ll`: generic first, second, third, fourth loop indices;
+  `kk` is also the key half of key/value pairs (`for kk, vv in ...`)
+- `vv`: value half of key/value pairs (`for kk, vv in sens_vars`)
+- `mm`: mapper/key half of function-table pairs (`for kk, mm in sens_funcs`)
+- `ff`: frame index (`for ff in range(num_frames)`)
+- `ee`: element index (`for ee in range(elem_coords.shape[0])`)
+- `ss`: sensor or set entry (`for ss in sens_data_dict`); note this collides
+  with exodus *side sets* terminology, so prefer it only for sensors
+- `bb`: element-variable block tuples (`(field, block)` pairs)
+- `pp`: process/frame record dictionaries in parallel loading loops
+- `dd`, `tt`, `nn`: occasional data/time/node indices in local scopes
+
+## Sensors & Simulation
+
+- `sens`: sensor(s); the most common abbreviation in the codebase
+  (`sens_data`, `SensDesc`, `sens_pos`)
+- `sim`: simulation (`SimData`, `sim_case_*`, `SimLoaderByTime`)
+- `exp`: experiment or experimental data (`ExpData`, `ExpLoadOpts`,
+  `expsim`)
+- `disp`: displacement, never display (`disp_x`, `field_disp_keys`)
+- `coords`: nodal coordinates, shape `(nodes, spatial_dims)`
+- `dims`: dimensionality or the simulation-dimension dictionary
+  (`get_sim_dims`); also `num_spat_dims`
+- `temp`: temperature in thermomechanical simulations, but temporary in
+  `temp_mask`/`temp_dir`; ambiguous, prefer spelling out one of the two in
+  new code
+- `calc`: calculate (`calc_first_surface_metric`)
+- `gen`: generator (`AnalyticSimDataGen`, `gen_gold_measurements`)
+- `ref`: reference, usually the undeformed reference image/frame/config
+- `seed`, `rng`: random-number seed and `np.random.default_rng` generator
+- `glob`: exodus *global* variables (`glob_vars`), alongside `node_vars`,
+  `elem_vars`, and `side_sets`
+
+## Cameras & Rendering
+
+- `cam`: camera; `cam0`/`cam1` are stereo camera pairs
+- `roi`: region of interest (`roi_cent_world`)
+- `px`: pixel (`mm_per_px`); spelled `pixels_` in field names
+  (`pixels_num`, `pixels_size`, `pixels_count`)
+- `leng`: length (`leng_per_px`, field-of-view extents)
+- `pos`: position (`pos_world`)
+- `rot`: rotation, normally a scipy `Rotation` (`rot_world`)
+- `fov`: field of view (`field_of_view`)
+- `psf`: point-spread function (`EPSFType`, `GaussianPSF`)
+- `trans`: transformation (`trans_mat`, `tensor_trans`); the `mat` fragment
+  appears only inside `trans_mat` - materials are always spelled out
+- `sub_sample` vs `subsample`: historical inconsistency - the 3D `Camera`
+  uses `sub_sample`, while `Camera2D` and `PxInt2DOpts` use `subsample`
+
+## DIC & Images
+
+- `dic`: digital image correlation (`pyvale.dic`, `DIC2D`)
+- `img`/`image`: images; field names spell out `image`
+- `mask`: validity/specimen mask applied to image pixels
+
+## Data Structures & Options
+
+- `opts`: options dataclass instances (`ImageDefOpts`, `VisOpts*`,
+  `SimLoadOpts`)
+- `num`: number-of counter prefix (`num_frames`, `num_spat_dims`)
+- `arr`: array
+- `err`: sensor error models (`ErrChain`, error calculators); raised
+  exceptions use `error`/`exception` instead
+- `chain`: error-model chain applied to synthetic sensor data (`err_chain`,
+  `set_error_chain`)
+- `vars`: exodus variable groups (`node_vars`, `elem_vars`, `glob_vars`)
+- `config`: renderer/backend configuration objects, spelled out unlike
+  `opts`
+
+## Files & Libraries
+
+- `np`: numpy; `plt`: matplotlib.pyplot; `pv`: pyvista; `nc`: netCDF4
+- Format tokens in names: `csv`, `yaml`, `tiff`, `bmp`, `npy`, and exodus
+  `.e` files
+
+## Naming Conventions
+
+- `E` prefix: enumerations (`EMeshType`, `ELightType`, `EElementType`)
+- `I` prefix: abstract-base-class interfaces (`ISensor`, `IRenderer3D`,
+  `IImageWarp2D`)
+- `_` leading underscore: private/internal members and modules

dev/MESHCONVENTION.md

@@ -0,0 +1,75 @@
+# Mesh Convention
+
+Pyvale uses one finite-element mesh convention across DataIO, SensorSim, and
+the render APIs. The common implementation is in
+`pyvale.dataio.meshconv`.
+
+## Required representation
+
+- `coords` has one node per row. Three columns are `[x, y, z]`; planar meshes
+  use `z = 0`.
+- Connectivity has one element per row and contains zero-based node indices.
+- Every connectivity index refers to a row in `coords`.
+- Element connectivity is row-major. Legacy transposed and one-based tables
+  are detected and normalised by `enforce_mesh_convention`.
+
+## Mesh topology
+
+`SimData.mesh_type` explicitly records the topology as `EMeshType.VOL` or
+`EMeshType.SURF`. `SimData.__post_init__` infers it whenever both coordinates
+and connectivity are supplied. The Exodus and CSV/array loaders call
+`refresh_mesh_type()` after loading their fields, so their returned objects
+carry the same value.
+
+Use `is_volume_mesh(sim_data)` where code needs to distinguish a volume mesh
+from a surface mesh. It resolves the ambiguous four- and eight-node tables by
+their signed cell volume (TET4 versus QUAD4, and HEX8 versus QUAD8). A
+`SimData` containing both surface and volume connectivity tables is rejected:
+it cannot have one unambiguous `EMeshType`.
+
+## Winding and handedness
+
+For a planar face, list corner nodes counter-clockwise when viewing its
+outward/visible side. The right-hand rule then gives the face normal: curl the
+fingers from the first edge to the second, and the thumb points outward.
+
+For a volume element, corner order must give a positive signed volume. Surface
+extraction orients each exposed face outward from its parent element.
+
+`check_mesh_convention(mesh)` reports zero-based indexing, row-major layout,
+valid indices, counter-clockwise winding, and right-handed geometry.
+`enforce_mesh_convention(mesh)` normalises supported input and should be
+idempotent.
+
+## Element node order
+
+The first nodes are always the corner nodes. Higher-order edge nodes follow
+the perimeter in the same direction as the corners.
+
+| Element | Corner order | Extra-node order |
+| --- | --- | --- |
+| TRI3 | `0, 1, 2` | — |
+| TRI6 | `0, 1, 2` | edges `01, 12, 20` |
+| QUAD4 | `0, 1, 2, 3` | — |
+| QUAD8 | `0, 1, 2, 3` | edges `01, 12, 23, 30` |
+| QUAD9 | QUAD8 order | QUAD8 edges, then centre |
+| TET4 | `0, 1, 2, 3` | — |
+| TET10 | TET4 corners | canonical tetrahedral edge nodes |
+| HEX8 | `0, 1, 2, 3, 4, 5, 6, 7` | — |
+| HEX20 | HEX8 corners | canonical hexahedral edge nodes |
+| HEX27 | HEX20 order | face centres, then volume centre |
+
+TET14 is packaged as a cube fixture but is not yet supported by the common
+convention checker or surface extractor.
+
+## Verification
+
+The cube fixtures provide regression coverage for TET4, TET10, HEX8, HEX20,
+and HEX27:
+
+```bash
+python -m pytest tests/dataio/meshtools_test.py -k cube
+```
+
+The suite verifies raw legacy layout, successful enforcement, idempotence, and
+the local outward orientation of extracted closed surfaces.

designspec/README.md → dev/README.md

@@ -23,7 +23,14 @@ We have non-software engineers and scientists working on the project so these gu
 - Prioritise an easy to remember and intuitive user API and performant code under the hood.
 - Work in your own 'feature' branch, merge into 'dev' - don't push to main (it should be protected and yell at you)!
 - Follow the PEP8 style guide: https://peps.python.org/pep-0008/
+- Format your code so it is readable, use an 80 character line length and put blank lines around logical groups of statements
 - Use descriptive variable names, no single letter variables (double letters for iterators in numpy style are ok) single letter variables for indices / iterators are ok.
+- Abbreviations are ok in variable names as long as they are not ambiguous for examples `calc` for `calculate`.
+- Functions should have a verb as the first word in the function name that indicates what the function actually does.
+- Avoid using magic numbers in code. If you need to use magic numbers, make them a named module constant with a descriptive name and add a comment when the name is not self-explanatory.
+- Keep comprehensions to one line with one `for` loop and at most one function call. Split comprehensions containing filters, nested loops, nested comprehensions or multiple function calls into explicit statements and loops.
+- Keep `if` conditions to at most two lines and avoid nested function calls in conditions. Calculate complex predicates in clearly named intermediate statements before the `if`.
+- Don't nest `def` statements.
 - Use major function first variable names: e.g. `FieldScalar`, `FieldVector` and `FieldTensor` instead of `ScalarField`, `VectorField` and `TensorField`.
 - Type hint everything: e.g. `def add_ints(a: int, b: int) -> int:`. This makes your code easier to understand and you have the possibility of compiling things if you need.
 - `pylint` is a slow linter but will help you if you have type hinted everything. `Ruff` is another good option, it is faster but doesn't pick up type hints as well.
@@ -33,7 +40,7 @@ We have non-software engineers and scientists working on the project so these gu
 - `numpy` and `scipy` are your friend - avoid for/while loops. Push everything you can down into C. Unless you are writing Cython then loops are great!
 - Minimise dependencies as much as possible.
 - Avoid decorators unless absolutely necessary (`@dataclass`,  `@abstractmethod` and `@staticmethod` are examples that are ok)
-- Don't use `@property` to hide complicated variable initialisation behind the `.` notation - in fact just avoid `@property` altogether and just use a `@dataclass` for data only classes.
+- Don't use `@property`. It is normally used to hide complicated variable initialisation behind the `.` notation - just avoid `@property` altogether and just use a `@dataclass` for data only classes.
 - No inheritance unless it is a purely abstract interface (python abstract base class `ABC`) - use composition / dependency injection. See this [video](https://www.youtube.com/watch?v=hxGOiiR9ZKg&t=3s) and thie [video](https://www.youtube.com/watch?v=J1f5b4vcxCQ&t=2s).
 - Only use one layer of abstraction - don't inherit from multiple interfaces and don't use mix-ins.
 - For interfaces (abstract base classes) prefix the name of the class with a capital `I` e.g. `ISensor`