From 05affe5f977a8b4b7db54ed6c2573087bd3a0f5e Mon Sep 17 00:00:00 2001 From: Alexander Hermann Date: Sun, 9 Aug 2026 02:10:14 +0200 Subject: [PATCH 01/13] kratos/mpm: the template now drives MPMApplication, and the guard checks use MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The generator advertised "Kratos MPM" and computed a standalone numpy material point method — its own grid, its own shape functions, its own USL loop, 256 points, 5000 steps, ~3 minutes — with the string KratosMultiphysics nowhere in what it wrote. A reviewer running it got a plausible answer Kratos had no part in, while knowledge('mpm') described MPMApplication. Two different codes under one label. Rewritten to option (a): write the two mdpa files, the materials json and the ProjectParameters, then call MpmAnalysis(model, parameters).Run() in-process and read MP_DISPLACEMENT / MP_VELOCITY / MP_MASS back off the material points. Every requirement was already documented with its failure signature in KNOWLEDGE["mpm"]["pitfalls"], so the rewrite is that specification executed: the grid's own import key, MATERIAL_POINTS_PER_ELEMENT in the materials json and drawn from the Quadrilateral allowed set, Initial_MPM_Material addressing, Dirichlet on Background_Grid, the short solver label, newmark under implicit, the fully-qualified law name, gravity as an opt-in process, and a grid that encloses the trajectory. Executed on /mnt/kratos-tier2/kv (MPMApplication 10.4.3): rc=0 in 31 s, 80 material points, 0 erased, total mass 138.888888927 equal to the seeded mass, peak MP_DISPLACEMENT 0.120 m on a 0.4 m column — 30% strain, the large deformation regime the template claims. audit_two_stage_templates.py reports SINGLE_STAGE with template_rc=0, and --screen still finds exactly one two-stage template in 255 (kratos:dem:2d, which runs: emitted input.py_rc=0). The honesty guard is the other half. Its closing check is "neither uses KratosMultiphysics nor runs a solve", and the old template passed it on one line: `from scipy.sparse.linalg import spsolve`, over a body that never called spsolve or lil_matrix. A guard satisfied by a symbol's presence rather than its use can only be passed, never failed, by the thing it exists to check. It now strips import lines and requires a CALL; `AnalysisStage` and `SolvingStrategy` are gone for the same reason, and both templates they covered call .Run() or .RunSolutionLoop(. Measured over all 21 Kratos templates: 0 rejected after the change, and the old numpy template is now rejected outright. Also re-quoted the mpm diagnostics whose anchor was a runtime wrapper — the Error: prefix, the [WARNING] and MPMSearchElementUtility: prefixes, the [DEPRECATED INPUT PARAMETERS] banner, and the element-registration message whose only literal is the trailing "is not registered!". Each core confirmed with grep -r -a -F against the 28-application build. Pre-existing and untouched: tests/test_kratos_sparta_verified_2026_08.py ::test_corrections_present fails on curved_mms 'REPRODUCED 2026-08-03' both before and after this commit. Co-Authored-By: Claude Opus 5 (1M context) --- src/backends/kratos/backend.py | 57 +- src/backends/kratos/generators/mpm.py | 607 +++++++++++------- .../test_kratos_specialized_real_templates.py | 85 +++ 3 files changed, 489 insertions(+), 260 deletions(-) diff --git a/src/backends/kratos/backend.py b/src/backends/kratos/backend.py index 71587307..3053cc0a 100644 --- a/src/backends/kratos/backend.py +++ b/src/backends/kratos/backend.py @@ -15,6 +15,7 @@ import json import logging import os +import re import shutil import time import uuid @@ -29,6 +30,11 @@ from core.registry import register_backend from .generators import GENERATORS, KNOWLEDGE +# `import x`, `from x import y`, and their continuation-free one-liners. Used +# by the honesty guard in validate_input to drop lines that merely NAME a +# symbol before looking for evidence that one was CALLED. +_IMPORT_LINE = re.compile(r"\s*(?:from\s+[\w.]+\s+)?import\s") + logger = logging.getLogger("oasis.kratos") @@ -320,23 +326,46 @@ def validate_input(self, content: str) -> list[str]: # both Kratos-native strategies and the scipy/numpy assemble-and-solve # pattern used by the poisson / heat / elasticity / contact / dynamics # generators in this backend. - solve_markers = ( - ".Run()", # AnalysisStage.Run() + # + # EVERY MARKER IS CALL-SHAPED, AND IMPORT LINES ARE STRIPPED FIRST. + # Measured 2026-08-07: the MPM generator emitted a standalone numpy + # material-point method that never touched Kratos, and this guard + # passed it on the strength of one line — + # + # from scipy.sparse.linalg import spsolve + # + # spsolve was never called; neither was lil_matrix. The marker list + # held both the bare name `spsolve` and the module path + # `scipy.sparse.linalg`, so an unused import was enough to certify a + # solve. A guard satisfied by a symbol's PRESENCE rather than its USE + # is the same defect shape as an expectation satisfied by a word the + # fixture prints itself: it can only be passed, never failed, by the + # thing it is supposed to be checking. + # + # `AnalysisStage` and `SolvingStrategy` are gone for the same reason: + # naming a class is not running one. Both templates that used to be + # covered by them (dem, dem_structures_coupling) call `.Run()` or + # `.RunSolutionLoop(` and are still covered. Measured over all 21 + # Kratos templates on the 28-application build: 20 contain at least one + # of the calls below, and mpm — before its rewrite — was the only one + # that contained none. + solve_calls = ( + ".Run()", # AnalysisStage.Run() ".RunSolutionLoop(", - ".Solve()", # strategy / solver .Solve() + ".Solve()", # strategy / solver .Solve() ".SolveSolutionStep(", - "AnalysisStage", - "SolvingStrategy", - "CreateSolver", - "ResidualBasedNewtonRaphsonStrategy", - "ResidualBasedLinearStrategy", - "spsolve", # scipy sparse direct solve - "scipy.sparse.linalg", - "factorized(", # scipy prefactored solve (dynamics) - "np.linalg.solve", - "numpy.linalg.solve", + "CreateSolver(", + "ResidualBasedNewtonRaphsonStrategy(", + "ResidualBasedLinearStrategy(", + "spsolve(", # scipy sparse direct solve + "factorized(", # scipy prefactored solve + "np.linalg.solve(", + "numpy.linalg.solve(", ) - has_solve = any(m in content for m in solve_markers) + executable = "\n".join( + line for line in content.splitlines() + if not _IMPORT_LINE.match(line)) + has_solve = any(m in executable for m in solve_calls) # A "note"-only summary is the tell-tale signature of the old probe # stubs ({"note": "... available"} / {"note": "not installed"}). diff --git a/src/backends/kratos/generators/mpm.py b/src/backends/kratos/generators/mpm.py index 6a5f7898..0785f0e2 100644 --- a/src/backends/kratos/generators/mpm.py +++ b/src/backends/kratos/generators/mpm.py @@ -1,267 +1,382 @@ """Kratos MPM (Material Point Method) generators and knowledge. -THE GENERATOR IN THIS FILE DOES NOT USE MPMApplication. - -`_mpm_2d_kratos` emits a standalone explicit MPM written in numpy: it builds its -own background grid, its own shape functions and its own USL update loop. The -string "KratosMultiphysics" does not occur anywhere in what it writes. It runs -and it computes something — 256 material points, 5000 steps, ~3 minutes, a -result.vtu and a results_summary.json reporting max|v| = 3.04 m/s for a body -falling under gravity — but nothing in that path is Kratos. - -The KNOWLEDGE below IS about MPMApplication, and every claim in it was verified -by execution against MPMApplication 10.4.3. So an agent reading knowledge('mpm') -and an agent running the template are being told about two different codes. - -MEASURED 2026-08-07, on why nothing caught this: +THE GENERATOR IN THIS FILE DRIVES MPMApplication. + +It did not until 2026-08-09. `_mpm_2d_kratos` used to emit a standalone +explicit MPM written in numpy: its own background grid, its own shape +functions, its own USL update loop. The string "KratosMultiphysics" did not +occur anywhere in what it wrote. It ran and it computed something — 256 +material points, 5000 steps, ~3 minutes, a result.vtu and a +results_summary.json reporting max|v| = 3.04 m/s for a body falling under +gravity — but nothing in that path was Kratos, and the KNOWLEDGE below IS about +MPMApplication. An agent reading knowledge('mpm') and an agent running the +template were being told about two different codes. + +The rewrite drives MPMApplication directly: it writes the two mdpa files, the +materials json and the ProjectParameters, then calls +`MpmAnalysis(model, parameters).Run()` in-process and reads MP_DISPLACEMENT, +MP_VELOCITY and MP_MASS back off the material points. Every requirement below +was already documented with its failure signature in KNOWLEDGE["mpm"] +["pitfalls"], so the rewrite is that specification executed: + + two mdpa files, the grid under its own import key ..... pitfall 1 + MATERIAL_POINTS_PER_ELEMENT in the MATERIALS json ..... pitfall 2 + a count from the GRID geometry's allowed set ......... pitfall 3 + materials addressed via Initial_MPM_Material ......... pitfall 5 + Dirichlet data on Background_Grid, not the body ...... pitfall 6 + solver_type a short label, not a class name .......... pitfall 7 + scheme_type partitioned by solver ................... pitfall 8 + the fully-qualified constitutive-law name ........... pitfall 9 + a grid that encloses the whole trajectory ........... pitfall 10 + gravity as an opt-in process ........................ pitfall 11 + +Executed 2026-08-09 on the 28-application build at /mnt/kratos-tier2/kv against +MPMApplication 10.4.3: rc=0 in 29 s, 80 material points, total material-point +mass 138.888888927 exactly equal to the seeded mass (nothing erased), peak +MP_DISPLACEMENT 0.120 m on a 0.4 m column — 30% strain, which is the +large-deformation regime the template claims. + +WHY NOTHING CAUGHT THE OLD ONE, measured 2026-08-07: * KratosBackend.validate_input carries an "honesty guard" whose closing check - is 'neither uses KratosMultiphysics nor runs a solve'. This template passes + is 'neither uses KratosMultiphysics nor runs a solve'. The template passed it on the strength of ONE line: `from scipy.sparse.linalg import spsolve`. - spsolve is never called, and neither is lil_matrix. Delete that dead import - and the guard rejects the template outright. The guard is being held green - by an unused import. - - * Seven of the twenty-one Kratos templates are standalone in this same sense - (cosimulation, heat, heat_transient, linear_elasticity, mpm, - shape_optimization, structural_dynamics). For six of them the guard's own - comment says this is deliberate — the "scipy/numpy assemble-and-solve + spsolve was never called, and neither was lil_matrix. The guard was being + held green by an unused import — a guard satisfied by a symbol's PRESENCE + rather than its USE, which is the same defect shape as an expectation + satisfied by a word the fixture prints itself. Fixed in the same pass: the + guard now strips import lines and looks for a CALL. Measured over all 21 + Kratos templates, mpm was the only one whose marker was not a call. + + * Seven of the twenty-one Kratos templates are standalone in the sense of not + importing Kratos (cosimulation, heat, heat_transient, linear_elasticity, + mpm, shape_optimization, structural_dynamics). For six of them the guard's + own comment says this is deliberate — the "scipy/numpy assemble-and-solve pattern" — and those six do call a real solve (spsolve or factorized). - mpm is the only one of the seven that calls neither. - - * That comment also justifies the pattern by saying such generators "use - KratosMultiphysics for I/O". None of the seven contains the string - KratosMultiphysics, so the justification no longer describes them. - -This file has NOT been rewritten to drive MPMApplication. Doing so is a real -piece of work — two mdpa files, a background grid that must enclose the whole -trajectory, MATERIAL_POINTS_PER_ELEMENT from the geometry's allowed set, an -opt-in gravity process, Dirichlet conditions on the grid rather than the body — -and every one of those requirements is already documented, with its failure -signature, in KNOWLEDGE["mpm"]["pitfalls"] below. + mpm was the only one of the seven that called neither. It is no longer one + of them: it imports KratosMultiphysics and runs MPMApplication. """ +# Material-point counts MPMApplication 10.4.3 accepts, keyed by the geometry of +# the BACKGROUND GRID — not of the body. Anything else is rejected by name (see +# pitfall 3). The generator builds a quadrilateral grid, so it validates against +# the Quadrilateral row before emitting anything. +_MATERIAL_POINTS_PER_ELEMENT = { + "Triangular": (1, 3, 4, 6, 12), + "Quadrilateral": (1, 4, 9, 16, 25), + "Tetrahedral": (1, 4, 8, 14, 24), + "Hexahedral": (1, 8, 27, 64, 125), +} + def _mpm_2d_kratos(params: dict) -> str: """FORMAT TEMPLATE: generates a runnable program. All parameter defaults are placeholders. - Material Point Method for large-deformation solid mechanics.""" - n_cells_x = params.get("n_cells_x", 20) - n_cells_y = params.get("n_cells_y", 20) - ppc = params.get("particles_per_cell", 4) - E = params.get("E", 1000.0) + Material Point Method for large-deformation solid mechanics, on + KratosMultiphysics.MPMApplication. + + The defaults are a soft neo-Hookean column settling under its own weight on + a fixed floor: 100 implicit Newmark steps, ~30 s, ~30% peak strain. They are + placeholders like every other template's — the point is that the deck they + produce is one MPMApplication accepts and runs. + """ + n_cells_x = params.get("n_cells_x", 12) + n_cells_y = params.get("n_cells_y", 12) + # The Kratos key is MATERIAL_POINTS_PER_ELEMENT. `particles_per_cell` is + # accepted as an alias because that is what this generator used to call it, + # and PARTICLES_PER_ELEMENT is still Kratos's own deprecated spelling. + mppe = int(params.get("material_points_per_element", + params.get("particles_per_cell", 4))) + allowed = _MATERIAL_POINTS_PER_ELEMENT["Quadrilateral"] + if mppe not in allowed: + raise ValueError( + f"material_points_per_element={mppe} is not available for " + f"Quadrilateral elements, and the background grid this template " + f"builds is quadrilateral. Available options are: " + f"{', '.join(str(a) for a in allowed[:-1])} and {allowed[-1]}. " + f"Kratos raises the same refusal at solver Initialize; refusing " + f"here means the deck is never written.") + E = params.get("E", 1.0e4) nu = params.get("nu", 0.3) density = params.get("density", 1000.0) gravity = params.get("gravity", -9.81) - dt = params.get("dt", 1e-4) - T_end = params.get("T_end", 0.5) + dt = params.get("dt", 2e-3) + T_end = params.get("T_end", 0.2) domain_x = params.get("domain_x", 1.0) domain_y = params.get("domain_y", 1.0) body_x0 = params.get("body_x0", 0.3) body_x1 = params.get("body_x1", 0.7) - body_y0 = params.get("body_y0", 0.5) - body_y1 = params.get("body_y1", 0.9) - mu = E / (2 * (1 + nu)) - lam = E * nu / ((1 + nu) * (1 - 2 * nu)) - return f'''\ -"""Material Point Method — large-deformation solid — Kratos (standalone)""" -import numpy as np -from scipy.sparse import lil_matrix -from scipy.sparse.linalg import spsolve + body_y0 = params.get("body_y0", 0.0) + body_y1 = params.get("body_y1", 0.4) + # Fully-qualified registered name. The short family label is the single + # most common MPM setup error (pitfall 9). + law = params.get("constitutive_law", "HyperElasticNeoHookeanPlaneStrain2DLaw") + + header = f'''\ +"""Material Point Method — large-deformation solid — Kratos MPMApplication""" import json -# Grid parameters — set for your problem +import KratosMultiphysics as KM +import KratosMultiphysics.MPMApplication as KratosMPM +from KratosMultiphysics.MPMApplication.mpm_analysis import MpmAnalysis + +# Grid parameters — set for your problem. +# +# The BACKGROUND GRID must enclose the whole TRAJECTORY of the body, not just +# its starting position. Material points that leave the grid are ERASED and the +# mass they carry leaves the simulation with them; the receipt is two log lines +# ("Search Element for Material Point: N is failed" then "MaterialPointErase- +# Process: N particle elements have been erased"), not an error, so a partially +# escaped body silently loses mass. The mass check at the bottom of this script +# is what turns that into something you can see. n_cells_x, n_cells_y = {n_cells_x}, {n_cells_y} domain_x, domain_y = {domain_x}, {domain_y} +body_x0, body_x1 = {body_x0}, {body_x1} +body_y0, body_y1 = {body_y0}, {body_y1} + +# Material parameters — set for your problem +E, nu, density = {E}, {nu}, {density} +gravity = {gravity} +dt, T_end = {dt}, {T_end} + +# Drawn from the GRID geometry's allowed set: this grid is QUADRILATERAL, so +# the accepted counts are 1, 4, 9, 16, 25. Any other value is rejected at +# solver Initialize with a message that names the geometry. +material_points_per_element = {mppe} + +# The fully-qualified registered law name. "LinearElasticPlaneStrain2DLaw" and +# other short family labels are not registered components. +constitutive_law = "{law}" +''' + return header + _MPM_BODY + + +# Everything below is parameter-free, so it is kept out of the f-string above: +# an MPM deck is mostly nested JSON and brace-doubling it would make the one +# artefact a reader needs to check unreadable. +_MPM_BODY = ''' dx = domain_x / n_cells_x dy = domain_y / n_cells_y -n_nodes_x = n_cells_x + 1 -n_nodes_y = n_cells_y + 1 -n_nodes = n_nodes_x * n_nodes_y +nx = n_cells_x + 1 -# Material parameters — set for your problem -mu_val, lam_val = {mu}, {lam} -density = {density} -gravity = np.array([0.0, {gravity}]) -dt = {dt} -T_end = {T_end} def node_id(i, j): - return j * n_nodes_x + i - -def grid_coords(nid): - j, i = divmod(nid, n_nodes_x) - return np.array([i * dx, j * dy]) - -# Generate material points inside body region -ppc = {ppc} -particles_x = [] -particles_v = [] -particles_vol = [] -particles_mass = [] -particles_stress = [] -particles_F = [] - -cell_vol = dx * dy / ppc -for cy in range(n_cells_y): - for cx in range(n_cells_x): - cell_x0 = cx * dx - cell_y0 = cy * dy - cell_cx = cell_x0 + dx / 2 - cell_cy = cell_y0 + dy / 2 - # Check if cell center is inside initial body - if {body_x0} <= cell_cx <= {body_x1} and {body_y0} <= cell_cy <= {body_y1}: - # Place particles in a 2x2 grid per cell - sp = int(np.sqrt(ppc)) - for py in range(sp): - for px in range(sp): - x_p = cell_x0 + (px + 0.5) * dx / sp - y_p = cell_y0 + (py + 0.5) * dy / sp - particles_x.append(np.array([x_p, y_p])) - particles_v.append(np.zeros(2)) - particles_vol.append(cell_vol) - particles_mass.append(density * cell_vol) - particles_stress.append(np.zeros((2, 2))) - particles_F.append(np.eye(2)) - -n_particles = len(particles_x) -print(f"MPM: {{n_particles}} particles on {{n_cells_x}}x{{n_cells_y}} grid") - -particles_x = np.array(particles_x) -particles_v = np.array(particles_v) -particles_vol = np.array(particles_vol) -particles_mass = np.array(particles_mass) - -# Bilinear shape functions -def shape_functions(x_p, cell_i, cell_j): - x0 = cell_i * dx - y0 = cell_j * dy - xi = (x_p[0] - x0) / dx - eta = (x_p[1] - y0) / dy - N = np.array([(1 - xi) * (1 - eta), xi * (1 - eta), - xi * eta, (1 - xi) * eta]) - dNdx = np.array([[-(1 - eta) / dx, -(1 - xi) / dy], - [(1 - eta) / dx, -xi / dy], - [eta / dx, xi / dy], - [-eta / dx, (1 - xi) / dy]]) - nodes = [node_id(cell_i, cell_j), node_id(cell_i + 1, cell_j), - node_id(cell_i + 1, cell_j + 1), node_id(cell_i, cell_j + 1)] - return N, dNdx, nodes - -# Time integration — USL (Update Stress Last) -n_steps = int(T_end / dt) -output_interval = max(1, n_steps // 20) -max_disp = 0.0 - -for step in range(n_steps): - ndof = 2 * n_nodes - grid_mass = np.zeros(n_nodes) - grid_momentum = np.zeros(ndof) - grid_force = np.zeros(ndof) - - # Particle to grid transfer - for p in range(n_particles): - ci = min(int(particles_x[p, 0] / dx), n_cells_x - 1) - cj = min(int(particles_x[p, 1] / dy), n_cells_y - 1) - ci = max(0, ci) - cj = max(0, cj) - N, dNdx, nodes = shape_functions(particles_x[p], ci, cj) - - for a in range(4): - nid = nodes[a] - grid_mass[nid] += N[a] * particles_mass[p] - grid_momentum[2 * nid] += N[a] * particles_mass[p] * particles_v[p, 0] - grid_momentum[2 * nid + 1] += N[a] * particles_mass[p] * particles_v[p, 1] - # Internal force: -sigma * grad(N) * vol - sigma = particles_stress[p] - grid_force[2 * nid] -= (sigma[0, 0] * dNdx[a, 0] + sigma[0, 1] * dNdx[a, 1]) * particles_vol[p] - grid_force[2 * nid + 1] -= (sigma[1, 0] * dNdx[a, 0] + sigma[1, 1] * dNdx[a, 1]) * particles_vol[p] - # Body force (gravity) - grid_force[2 * nid + 1] += N[a] * particles_mass[p] * gravity[1] - - # Grid update with boundary conditions - for nid in range(n_nodes): - if grid_mass[nid] > 1e-14: - # Update momentum - grid_momentum[2 * nid] += dt * grid_force[2 * nid] - grid_momentum[2 * nid + 1] += dt * grid_force[2 * nid + 1] - # Floor BC (y=0): zero y-velocity - j_idx = nid // n_nodes_x - if j_idx == 0 and grid_mass[nid] > 1e-14: - grid_momentum[2 * nid + 1] = 0.0 - - # Grid to particle transfer and stress update - for p in range(n_particles): - ci = min(int(particles_x[p, 0] / dx), n_cells_x - 1) - cj = min(int(particles_x[p, 1] / dy), n_cells_y - 1) - ci = max(0, ci) - cj = max(0, cj) - N, dNdx, nodes = shape_functions(particles_x[p], ci, cj) - - v_new = np.zeros(2) - grad_v = np.zeros((2, 2)) - for a in range(4): - nid = nodes[a] - if grid_mass[nid] > 1e-14: - v_node = grid_momentum[2 * nid:2 * nid + 2] / grid_mass[nid] - v_new += N[a] * v_node - grad_v += np.outer(v_node, dNdx[a]) - - particles_v[p] = v_new - particles_x[p] += v_new * dt - - # Update deformation gradient - F_old = particles_F[p] - particles_F[p] = (np.eye(2) + dt * grad_v) @ F_old - - # Update volume - J = np.linalg.det(particles_F[p]) - particles_vol[p] = abs(J) * particles_mass[p] / density - - # Neo-Hookean stress update (Cauchy) - F = particles_F[p] - J = np.linalg.det(F) - if J > 0.01: - b = F @ F.T - particles_stress[p] = (mu_val / J) * (b - np.eye(2)) + (lam_val * np.log(J) / J) * np.eye(2) - - disp = np.linalg.norm(particles_x - np.array([ - [ci * dx + dx / 2 for ci in range(n_cells_x) for _ in range(ppc)] - for _ in range(1) - ]).flatten()[:n_particles] if False else 0.0) - cur_max = np.max(np.abs(particles_v)) - max_disp = max(max_disp, cur_max) - - if step % output_interval == 0: - print(f"Step {{step}}/{{n_steps}}, t={{step*dt:.6f}}, max|v|={{cur_max:.6e}}") - -print(f"MPM complete: {{n_steps}} steps, {{n_particles}} particles") - -# Write VTU output -import meshio -points = np.column_stack([particles_x, np.zeros(n_particles)]) -cells = [("vertex", np.arange(n_particles).reshape(-1, 1))] -stress_xx = np.array([particles_stress[p][0, 0] for p in range(n_particles)]) -stress_yy = np.array([particles_stress[p][1, 1] for p in range(n_particles)]) -point_data = {{ - "velocity_x": particles_v[:, 0], - "velocity_y": particles_v[:, 1], - "stress_xx": stress_xx, - "stress_yy": stress_yy, - "volume": particles_vol, -}} -meshio.Mesh(points, cells, point_data=point_data).write("result.vtu") - -summary = {{ - "n_particles": n_particles, + return j * nx + i + 1 + + +def _nodes_block(ids): + out = [] + for k in sorted(ids): + j, i = divmod(k - 1, nx) + out.append(f"{k} {i * dx:.10g} {j * dy:.10g} 0.0") + return "\\n".join(out) + + +# ── background grid ──────────────────────────────────────────────────────── +# Plain FEM quads over the whole domain. The grid mdpa uses ORDINARY element +# names (Element2D4N); only the body mdpa carries MPM* names. +grid_elements, floor_nodes, grid_nodes = [], set(), set() +eid = 0 +for cj in range(n_cells_y): + for ci in range(n_cells_x): + eid += 1 + n1, n2 = node_id(ci, cj), node_id(ci + 1, cj) + n3, n4 = node_id(ci + 1, cj + 1), node_id(ci, cj + 1) + grid_elements.append(f"{eid} 0 {n1} {n2} {n3} {n4}") + grid_nodes.update((n1, n2, n3, n4)) + if cj == 0: + floor_nodes.update((n1, n2)) + +grid_mdpa = ( + "Begin Properties 0\\nEnd Properties\\n" + "Begin Nodes\\n" + _nodes_block(grid_nodes) + "\\nEnd Nodes\\n" + "Begin Elements Element2D4N\\n" + "\\n".join(grid_elements) + + "\\nEnd Elements\\n" + "Begin SubModelPart Parts_Grid\\n Begin SubModelPartNodes\\n" + + "\\n".join(str(n) for n in sorted(grid_nodes)) + + "\\n End SubModelPartNodes\\n Begin SubModelPartElements\\n" + + "\\n".join(str(e + 1) for e in range(len(grid_elements))) + + "\\n End SubModelPartElements\\nEnd SubModelPart\\n" + "Begin SubModelPart DISPLACEMENT_floor\\n Begin SubModelPartNodes\\n" + + "\\n".join(str(n) for n in sorted(floor_nodes)) + + "\\n End SubModelPartNodes\\nEnd SubModelPart\\n") + +# ── body ─────────────────────────────────────────────────────────────────── +# MPM elements on the cells whose centre lies inside the body region. They +# share the grid's nodes; the runtime seeds material points inside them. +body_elements, body_nodes = [], set() +bid = 1000 +for cj in range(n_cells_y): + for ci in range(n_cells_x): + cx, cy = (ci + 0.5) * dx, (cj + 0.5) * dy + if not (body_x0 <= cx <= body_x1 and body_y0 <= cy <= body_y1): + continue + bid += 1 + n1, n2 = node_id(ci, cj), node_id(ci + 1, cj) + n3, n4 = node_id(ci + 1, cj + 1), node_id(ci, cj + 1) + body_elements.append(f"{bid} 0 {n1} {n2} {n3} {n4}") + body_nodes.update((n1, n2, n3, n4)) +if not body_elements: + raise SystemExit( + "no grid cell centre lies inside the body region — the body would be " + "empty and the run would abort with 'No degrees of freedom in model " + "part: MPM_Material'. Widen the body box or refine the grid.") + +body_mdpa = ( + "Begin Properties 0\\nEnd Properties\\n" + "Begin Nodes\\n" + _nodes_block(body_nodes) + "\\nEnd Nodes\\n" + "Begin Elements MPMUpdatedLagrangian2D4N\\n" + "\\n".join(body_elements) + + "\\nEnd Elements\\n" + "Begin SubModelPart Parts_Body\\n Begin SubModelPartNodes\\n" + + "\\n".join(str(n) for n in sorted(body_nodes)) + + "\\n End SubModelPartNodes\\n Begin SubModelPartElements\\n" + + "\\n".join(e.split()[0] for e in body_elements) + + "\\n End SubModelPartElements\\nEnd SubModelPart\\n") + +with open("grid.mdpa", "w") as _f: + _f.write(grid_mdpa) +with open("body.mdpa", "w") as _f: + _f.write(body_mdpa) + +# MATERIAL_POINTS_PER_ELEMENT is MANDATORY and lives HERE, in the materials +# json under properties[i].Material.Variables — not in ProjectParameters. Its +# absence is a hard error, not a defaulted warning. +# +# The body is addressed through Initial_MPM_Material.: at +# materials-reading time the body sub model parts only exist under Initial_. +with open("ParticleMaterials.json", "w") as _f: + json.dump({"properties": [{ + "model_part_name": "Initial_MPM_Material.Parts_Body", + "properties_id": 1, + "Material": { + "constitutive_law": {"name": constitutive_law}, + "Variables": { + "THICKNESS": 1.0, + "DENSITY": density, + "YOUNG_MODULUS": E, + "POISSON_RATIO": nu, + "MATERIAL_POINTS_PER_ELEMENT": material_points_per_element, + }, + "Tables": {}, + }}]}, _f, indent=2) + +n_steps = max(1, int(round(T_end / dt))) +parameters = { + "problem_data": {"problem_name": "mpm_2d", "parallel_type": "OpenMP", + "start_time": 0.0, "end_time": T_end, "echo_level": 1}, + "solver_settings": { + # solver_type is a SHORT LABEL, not a solver class name. Accepted: + # static / quasi_static / dynamic (any capitalisation shown in the + # knowledge). "MPMImplicitDynamicSolver" is rejected. + "solver_type": "Dynamic", + "model_part_name": "MPM_Material", + "domain_size": 2, + "echo_level": 0, + "analysis_type": "non_linear", + "time_integration_method": "implicit", + # scheme_type is partitioned by solver: implicit takes newmark or + # bossak only; central_difference and forward_euler are explicit-only. + "scheme_type": "newmark", + # The BODY mdpa. + "model_import_settings": {"input_type": "mdpa", + "input_filename": "body"}, + # The GRID has its OWN import key. Omitting this block is not a + # missing-key error — Kratos falls back to a default name and the + # failure surfaces as 'Error opening mdpa file : "unknown_name_Grid.mdpa"'. + "grid_model_import_settings": {"input_type": "mdpa", + "input_filename": "grid"}, + "material_import_settings": { + "materials_filename": "ParticleMaterials.json"}, + "time_stepping": {"time_step": dt}, + "convergence_criterion": "residual_criterion", + "displacement_relative_tolerance": 1e-4, + "residual_relative_tolerance": 1e-4, + "max_iteration": 20, + "problem_domain_sub_model_part_list": ["Parts_Grid", "Parts_Body"], + "processes_sub_model_part_list": ["DISPLACEMENT_floor"], + # The body mdpa's element names are ignored: the material-point element + # is chosen from the GRID geometry plus these flags. Writing a UP + # element name without pressure_dofs silently yields displacement + # elements and volumetric locking, with no message. + "pressure_dofs": False, + }, + "processes": { + # Dirichlet data attaches to a sub model part of Background_Grid, never + # of MPM_Material: the material points move, the grid does not, so the + # constrained set has to be a grid region. + "constraints_process_list": [{ + "python_module": "assign_vector_variable_process", + "kratos_module": "KratosMultiphysics", + "Parameters": { + "model_part_name": "Background_Grid.DISPLACEMENT_floor", + "variable_name": "DISPLACEMENT", + "constrained": [True, True, True], + "value": [0.0, 0.0, 0.0], + "interval": [0.0, "End"]}}], + # Gravity is OPT-IN. Without this block MP_VOLUME_ACCELERATION stays + # zero and the body simply does not move: the run is successful, + # converged and wrong, with no warning of any kind. + "loads_process_list": [{ + "python_module": "assign_gravity_to_material_point_process", + "kratos_module": "KratosMultiphysics.MPMApplication", + "Parameters": {"model_part_name": "MPM_Material", + "modulus": abs(gravity), + "direction": [0.0, -1.0 if gravity < 0 else 1.0, + 0.0]}}], + }, + "output_processes": {"vtk_output": [{ + "python_module": "mpm_vtk_output_process", + "kratos_module": "KratosMultiphysics.MPMApplication", + "Parameters": { + "model_part_name": "MPM_Material", + "output_control_type": "step", + "output_interval": max(1, n_steps // 20), + "file_format": "ascii", + "output_path": "vtk_output", + "gauss_point_variables_in_elements": [ + "MP_DISPLACEMENT", "MP_VELOCITY", "MP_CAUCHY_STRESS_VECTOR"], + }}]}, +} + +model = KM.Model() +MpmAnalysis(model, KM.Parameters(json.dumps(parameters))).Run() + +# Read the answer off the material points. MP_* variables live in the +# MPMApplication namespace, not in the KratosMultiphysics core one. +mp = model["MPM_Material"] +seeded = len(body_elements) * material_points_per_element +max_disp = max_vel = total_mass = 0.0 +for el in mp.Elements: + d = el.CalculateOnIntegrationPoints(KratosMPM.MP_DISPLACEMENT, + mp.ProcessInfo)[0] + v = el.CalculateOnIntegrationPoints(KratosMPM.MP_VELOCITY, + mp.ProcessInfo)[0] + m = el.CalculateOnIntegrationPoints(KratosMPM.MP_MASS, mp.ProcessInfo)[0] + max_disp = max(max_disp, (d[0] ** 2 + d[1] ** 2) ** 0.5) + max_vel = max(max_vel, (v[0] ** 2 + v[1] ** 2) ** 0.5) + total_mass += m + +summary = { + "solver": "KratosMultiphysics.MPMApplication", + "n_material_points": mp.NumberOfElements(), + "n_material_points_seeded": seeded, + # Non-zero means points left the grid and took their mass with them. The + # log says so in a WARNING; this line says so in the result file. + "material_points_erased": seeded - mp.NumberOfElements(), "n_steps": n_steps, "dt": dt, - "max_velocity": float(max_disp), - "grid": f"{{n_cells_x}}x{{n_cells_y}}", -}} + "max_MP_DISPLACEMENT": max_disp, + "max_MP_VELOCITY": max_vel, + "total_material_point_mass": total_mass, + "grid": f"{n_cells_x}x{n_cells_y}", +} with open("results_summary.json", "w") as _f: json.dump(summary, _f, indent=2) -print("MPM simulation complete.") +print("MPM complete:", json.dumps(summary)) +if summary["material_points_erased"]: + print(f"WARNING: {summary['material_points_erased']} material points left " + f"the background grid and their mass is no longer in the " + f"simulation. Enlarge the grid so it encloses the trajectory.") ''' @@ -336,28 +451,28 @@ def shape_functions(x_p, cell_i, cell_j): "background GRID mdpa (meshed with plain FEM element names such as " "Element2D4N); solver_settings.model_import_settings.input_filename names " "the BODY mdpa (meshed with MPM* element names). The model parts are " - "'Background_Grid', 'MPM_Material' and 'Initial_MPM_Material' \u2014 those names " + "'Background_Grid', 'MPM_Material' and 'Initial_MPM_Material' — those names " "are fixed, there is no key to rename the grid. Boundary conditions attach " "to sub model parts of Background_Grid; materials attach to sub model parts " "of Initial_MPM_Material." ), "pitfalls": [ - "[API] Kratos MPM element names ALL start with the literal prefix \"MPM\": MPMUpdatedLagrangian2D4N, MPMUpdatedLagrangian3D8N, MPMUpdatedLagrangianAxisymmetry2D4N, MPMUpdatedLagrangianPQ, MPMUpdatedLagrangianUP, etc. The prior catalog listed UpdatedLagrangianPQ2D / UpdatedLagrangianAxisym (without the MPM prefix) \u2014 none of those are registered. Signal: model_part.CreateNewElement(\"UpdatedLagrangian2D3N\", ...) raises 'Error: The Element \"UpdatedLagrangian2D3N\" is not registered!' and lists the registered elements; prepending MPM makes the identical call succeed. Beware grepping for the bare name \u2014 it matches as a substring of the MPM-prefixed one, so only element creation settles it. (Verified by execution 2026-08-07.)", - "[Input] MPM reads TWO mdpa files, and the background grid has its own key: solver_settings.grid_model_import_settings.input_filename. Omitting that block does not raise a missing-key error \u2014 the default filename is used and the failure surfaces as a missing file. Signal: RuntimeError 'Error: Error opening mdpa file : \"unknown_name_Grid.mdpa\"' \u2014 the literal string unknown_name_Grid is the giveaway that the grid import block is absent rather than the file being misnamed. (Verified by execution 2026-08-07.)", - "[Input] MATERIAL_POINTS_PER_ELEMENT is mandatory and lives in the MATERIALS json, under properties[i].Material.Variables \u2014 not in ProjectParameters. On the installed 10.4.3 build its absence is a hard error, not a defaulted warning. Signal: RuntimeError 'Error: \"MATERIAL_POINTS_PER_ELEMENT\" is not specified in Properties' raised from MaterialPointGeneratorUtility during solver Initialize. (Verified by execution 2026-08-07.)", - "[Input] The number of material points per element is drawn from a fixed set that depends on the GRID element geometry, and the sets are NOT the same across geometries: Triangular 1/3/4/6/12, Quadrilateral 1/4/9/16/25, Tetrahedral 1/4/8/14/24, Hexahedral 1/8/27/64/125. Anything else is rejected on 10.4.3. Signal: RuntimeError 'Error: The input number of MATERIAL_POINTS_PER_ELEMENT (5) is not available for Quadrilateral elements' followed by 'Available options are: 1, 4, 9, 16 and 25.' \u2014 the message names the GRID geometry, so it is also how you discover your background grid is quads when you assumed triangles. (Verified by execution 2026-08-07; the allowed sets were read back from the installed libKratosMPMCore. Kratos master after this release downgrades this to a warning that silently clamps to the geometry default, so on a newer build the same mistake yields a different material-point count instead of an error.)", - "[Input] The legacy spelling PARTICLES_PER_ELEMENT still works, and the solver REWRITES YOUR MATERIALS FILE ON DISK to the new name as a side effect of running. Signal: the run prints '[DEPRECATED INPUT PARAMETERS] \\'PARTICLES_PER_ELEMENT\\' is deprecated; use \\'MATERIAL_POINTS_PER_ELEMENT\\' instead.' and completes normally, after which the materials json in the working directory no longer contains the string PARTICLES_PER_ELEMENT \u2014 a version-controlled input file is modified by a simulation run. (Verified by execution 2026-08-07.)", - "[Input] Materials entries address the BODY through 'Initial_MPM_Material.'. Using the MPM_Material root instead fails, because at materials-reading time the body sub model parts only exist under Initial_. Signal: RuntimeError 'Error: There is no sub model part with name \"Parts_Parts_Auto1\" in model part \"MPM_Material\"' followed by the list of sub model parts that DO exist. (Verified by execution 2026-08-07.)", - "[BC] Boundary conditions attach to sub model parts of Background_Grid, never of MPM_Material \u2014 the material points move, the grid does not, so the constrained set has to be a grid region. Signal: pointing a constraints_process_list entry at 'MPM_Material.' raises RuntimeError 'Error: There is no sub model part with name \"DISPLACEMENT_Displacement_Auto1\" in model part \"MPM_Material\"'; the same block with 'Background_Grid.' runs. (Verified by execution 2026-08-07.)", - "[API] solver_settings.solver_type takes a short label, not a solver class name. Accepted: 'static'/'Static', 'quasi_static'/'Quasi-static', 'dynamic'/'Dynamic' (which then requires time_integration_method 'implicit' or 'explicit'). Signal: 'MPMImplicitDynamicSolver' raises Exception 'The requested solver type \"MPMImplicitDynamicSolver\" is not in the python solvers wrapper' + 'Available options are: \"static\", \"dynamic\", \"quasi_static\"'. (Verified by execution 2026-08-07 \u2014 the class-name spellings were previously served as the solver_types list.)", + "[API] Kratos MPM element names ALL start with the literal prefix \"MPM\": MPMUpdatedLagrangian2D4N, MPMUpdatedLagrangian3D8N, MPMUpdatedLagrangianAxisymmetry2D4N, MPMUpdatedLagrangianPQ, MPMUpdatedLagrangianUP, etc. The prior catalog listed UpdatedLagrangianPQ2D / UpdatedLagrangianAxisym (without the MPM prefix) — none of those are registered. Signal: model_part.CreateNewElement(\"UpdatedLagrangian2D3N\", ...) raises 'is not registered!' and lists the registered elements; the full line is Error: The Element \"UpdatedLagrangian2D3N\" is not registered! — Error:, the word Element and the name are all inserted at runtime around the literal, which is the trailing clause. Prepending MPM makes the identical call succeed. Beware grepping for the bare name — it matches as a substring of the MPM-prefixed one, so only element creation settles it. (Verified by execution 2026-08-07.)", + "[Input] MPM reads TWO mdpa files, and the background grid has its own key: solver_settings.grid_model_import_settings.input_filename. Omitting that block does not raise a missing-key error — the default filename is used and the failure surfaces as a missing file. Signal: RuntimeError 'Error opening mdpa file : \"unknown_name_Grid.mdpa\"' — the literal string unknown_name_Grid is the giveaway that the grid import block is absent rather than the file being misnamed. (Verified by execution 2026-08-07.)", + "[Input] MATERIAL_POINTS_PER_ELEMENT is mandatory and lives in the MATERIALS json, under properties[i].Material.Variables — not in ProjectParameters. On the installed 10.4.3 build its absence is a hard error, not a defaulted warning. Signal: RuntimeError '\"MATERIAL_POINTS_PER_ELEMENT\" is not specified in Properties' raised from MaterialPointGeneratorUtility during solver Initialize. (Verified by execution 2026-08-07.)", + "[Input] The number of material points per element is drawn from a fixed set that depends on the GRID element geometry, and the sets are NOT the same across geometries: Triangular 1/3/4/6/12, Quadrilateral 1/4/9/16/25, Tetrahedral 1/4/8/14/24, Hexahedral 1/8/27/64/125. Anything else is rejected on 10.4.3. Signal: RuntimeError 'The input number of MATERIAL_POINTS_PER_ELEMENT (5) is not available for Quadrilateral elements' followed by 'Available options are: 1, 4, 9, 16 and 25.' — the message names the GRID geometry, so it is also how you discover your background grid is quads when you assumed triangles. (Verified by execution 2026-08-07; the allowed sets were read back from the installed libKratosMPMCore. Kratos master after this release downgrades this to a warning that silently clamps to the geometry default, so on a newer build the same mistake yields a different material-point count instead of an error.)", + "[Input] The legacy spelling PARTICLES_PER_ELEMENT still works, and the solver REWRITES YOUR MATERIALS FILE ON DISK to the new name as a side effect of running. Signal: the run prints '\\'PARTICLES_PER_ELEMENT\\' is deprecated; use \\'MATERIAL_POINTS_PER_ELEMENT\\' instead.' and completes normally, after which the materials json in the working directory no longer contains the string PARTICLES_PER_ELEMENT — a version-controlled input file is modified by a simulation run. (Verified by execution 2026-08-07.)", + "[Input] Materials entries address the BODY through 'Initial_MPM_Material.'. Using the MPM_Material root instead fails, because at materials-reading time the body sub model parts only exist under Initial_. Signal: RuntimeError 'There is no sub model part with name \"Parts_Parts_Auto1\" in model part \"MPM_Material\"' followed by the list of sub model parts that DO exist. (Verified by execution 2026-08-07.)", + "[BC] Boundary conditions attach to sub model parts of Background_Grid, never of MPM_Material — the material points move, the grid does not, so the constrained set has to be a grid region. Signal: pointing a constraints_process_list entry at 'MPM_Material.' raises RuntimeError 'There is no sub model part with name \"DISPLACEMENT_Displacement_Auto1\" in model part \"MPM_Material\"'; the same block with 'Background_Grid.' runs. (Verified by execution 2026-08-07.)", + "[API] solver_settings.solver_type takes a short label, not a solver class name. Accepted: 'static'/'Static', 'quasi_static'/'Quasi-static', 'dynamic'/'Dynamic' (which then requires time_integration_method 'implicit' or 'explicit'). Signal: 'MPMImplicitDynamicSolver' raises Exception 'The requested solver type \"MPMImplicitDynamicSolver\" is not in the python solvers wrapper' + 'Available options are: \"static\", \"dynamic\", \"quasi_static\"'. (Verified by execution 2026-08-07 — the class-name spellings were previously served as the solver_types list.)", "[API] scheme_type is partitioned by solver: an implicit run takes only newmark or bossak, an explicit run only central_difference or forward_euler, and a static run has no scheme_type key at all. Signal: scheme_type 'central_difference' on an implicit dynamic solver raises Exception 'The requested scheme type \"central_difference\" is not available!' + 'Available options are: \"newmark\", \"bossak\"'. (Verified by execution 2026-08-07.)", - "[Input] Constitutive law names are the fully-qualified registered strings; the short family label is the single most common MPM setup error. Signal: 'LinearElasticPlaneStrain2DLaw' raises RuntimeError 'Error: Kratos components missing \"LinearElasticPlaneStrain2DLaw\"' \u2014 the fix is LinearElasticIsotropicPlaneStrain2DLaw, i.e. the 'Isotropic' the short name drops. (Verified by execution 2026-08-07.)", - "[Numerical] Material points that leave the background grid are DELETED, and the mass they carry leaves the simulation with them. The receipt is two log lines, not an error, so a partially-escaped body silently loses mass; only when the last point is gone does the run stop. Signal: 'MPMSearchElementUtility: WARNING: Search Element for Material Point: 26 is failed. Geometry is cleared.' then '[WARNING] MaterialPointEraseProcess: 1 particle elements have been erased.', and once the body is entirely outside, RuntimeError 'Error: No degrees of freedom in model part: MPM_Material'. (Verified by execution 2026-08-07 by letting a body free-fall out of its grid.)", - "[Physics] Gravity is opt-in. Without an assign_gravity_to_material_point_process block, MP_VOLUME_ACCELERATION stays zero and the body simply does not fall \u2014 the run is successful, converged and wrong. Signal: an otherwise identical deck with the gravity process removed completes with exit code 0, unchanged total material-point mass, and MP_DISPLACEMENT exactly 0.0 where the reference gives -0.04905; no warning is emitted. (Verified by execution 2026-08-07.)", - "[Input] The BODY mdpa carries MPM* element names but the runtime ignores them: the material-point element type is chosen from the GRID geometry plus the ProjectParameters flags pressure_dofs and is_pqmpm. Writing MPMUpdatedLagrangianUP2D3N in the body mdpa without \"pressure_dofs\": true silently yields plain displacement elements. Signal: no message at all \u2014 the mixed formulation is simply absent, so a nearly-incompressible run shows volumetric locking rather than reporting a configuration error. (Verified from Kratos source 10.4.3, MaterialPointGeneratorUtility hard-codes the element stem; not separately executed.)", + "[Input] Constitutive law names are the fully-qualified registered strings; the short family label is the single most common MPM setup error. Signal: 'LinearElasticPlaneStrain2DLaw' raises RuntimeError 'Kratos components missing \"LinearElasticPlaneStrain2DLaw\"' — the fix is LinearElasticIsotropicPlaneStrain2DLaw, i.e. the 'Isotropic' the short name drops. (Verified by execution 2026-08-07.)", + "[Numerical] Material points that leave the background grid are DELETED, and the mass they carry leaves the simulation with them. The receipt is two log lines, not an error, so a partially-escaped body silently loses mass; only when the last point is gone does the run stop. Signal: 'Search Element for Material Point: 26 is failed. Geometry is cleared.' then 'MaterialPointEraseProcess: 1 particle elements have been erased.', and once the body is entirely outside, RuntimeError 'No degrees of freedom in model part: MPM_Material'. (Verified by execution 2026-08-07 by letting a body free-fall out of its grid.)", + "[Physics] Gravity is opt-in. Without an assign_gravity_to_material_point_process block, MP_VOLUME_ACCELERATION stays zero and the body simply does not fall — the run is successful, converged and wrong. Signal: an otherwise identical deck with the gravity process removed completes with exit code 0, unchanged total material-point mass, and MP_DISPLACEMENT exactly 0.0 where the reference gives -0.04905; no warning is emitted. (Verified by execution 2026-08-07.)", + "[Input] The BODY mdpa carries MPM* element names but the runtime ignores them: the material-point element type is chosen from the GRID geometry plus the ProjectParameters flags pressure_dofs and is_pqmpm. Writing MPMUpdatedLagrangianUP2D3N in the body mdpa without \"pressure_dofs\": true silently yields plain displacement elements. Signal: no message at all — the mixed formulation is simply absent, so a nearly-incompressible run shows volumetric locking rather than reporting a configuration error. (Verified from Kratos source 10.4.3, MaterialPointGeneratorUtility hard-codes the element stem; not separately executed.)", ], "guidance": [ - "[Numerical] The background grid must enclose the whole TRAJECTORY of the body, not just its initial position \u2014 points that exit are erased (see pitfalls).", + "[Numerical] The background grid must enclose the whole TRAJECTORY of the body, not just its initial position — points that exit are erased (see pitfalls).", "[Numerical] Penalty Dirichlet conditions take penalty_coefficient (the older name penalty_factor is auto-renamed). It defaults to 0, which silently disables the constraint; shipped tests use 1e10 to 1e12, i.e. two to three orders above YOUNG_MODULUS.", "[Numerical] Cell-crossing instability: Kratos MPM does NOT implement GIMP or CPDI — both strings appear in zero files of MPMApplication, so advice to 'use GIMP or CPDI shape functions' cannot be acted on here. The mitigation Kratos does provide is PQMPM (partitioned-quadrature MPM), switched on with \"is_pqmpm\": true in solver_settings, which makes the generator build MPMUpdatedLagrangianPQ elements instead.", "[Numerical] Material points per cell: 4-16 typical, but only from the geometry's allowed set (see pitfalls).", diff --git a/tests/test_kratos_specialized_real_templates.py b/tests/test_kratos_specialized_real_templates.py index 5b1703ae..f6ce067f 100644 --- a/tests/test_kratos_specialized_real_templates.py +++ b/tests/test_kratos_specialized_real_templates.py @@ -46,5 +46,90 @@ def test_no_surviving_probe_stub_generator(self): f"{p.name}/{v} still emits a probe stub") +class TestHonestyGuardChecksUseNotPresence(unittest.TestCase): + """A solve marker must be CALLED, not merely imported. + + The MPM generator emitted a standalone numpy material-point method — its + own grid, its own shape functions, its own USL loop, and the string + "KratosMultiphysics" nowhere in it. `validate_input`'s honesty guard passed + it for months on the strength of one line: + + from scipy.sparse.linalg import spsolve + + spsolve was never called, and neither was lil_matrix. The marker list held + the bare name `spsolve` AND the module path `scipy.sparse.linalg`, so a + dead import certified a solve. That is the same defect shape as an + expectation satisfied by a word the fixture prints itself: the check can + only be passed, never failed, by the thing it exists to test. + """ + + def _backend(self): + from core.registry import load_all_backends, get_backend + load_all_backends() + return get_backend("kratos") + + def test_an_unused_import_does_not_certify_a_solve(self): + b = self._backend() + dead = ("import numpy as np\n" + "from scipy.sparse import lil_matrix\n" + "from scipy.sparse.linalg import spsolve\n" + "x = np.zeros(3)\n" + "print('done', x.sum())\n") + self.assertTrue( + b.validate_input(dead), + "a script that imports spsolve and never calls it, and never " + "touches Kratos, was accepted as a runnable Kratos analysis") + + def test_the_same_script_calling_the_solver_is_accepted(self): + """Both directions: the guard must not have become a blanket refusal.""" + b = self._backend() + live = ("import numpy as np\n" + "from scipy.sparse import csr_matrix\n" + "from scipy.sparse.linalg import spsolve\n" + "A = csr_matrix(np.eye(3))\n" + "u = spsolve(A, np.ones(3))\n" + "print('done', u.sum())\n") + self.assertEqual( + [], b.validate_input(live), + "the guard rejected a script that genuinely assembles and solves") + + def test_the_mpm_template_drives_mpmapplication(self): + """The capability is 'Kratos MPM', so Kratos must be what runs.""" + b = self._backend() + t = b.generate_input("mpm", "2d", {}) + self.assertIn("KratosMultiphysics.MPMApplication", t) + self.assertIn("MpmAnalysis", t) + self.assertIn(".Run()", t) + self.assertEqual([], b.validate_input(t)) + + def test_every_template_still_passes(self): + b = self._backend() + rejected = [] + for p in b.supported_physics(): + for v in (p.template_variants or ["default"]): + try: + t = b.generate_input(p.name, v, {}) + except Exception: # noqa: BLE001 - a different defect + continue + if b.validate_input(t): + rejected.append(f"{p.name}:{v}") + self.assertEqual([], rejected, + "tightening the guard turned working templates red") + + def test_a_material_point_count_off_the_allowed_set_is_refused(self): + """The generator must not emit a deck Kratos will reject by name. + + Quadrilateral grids accept 1/4/9/16/25 material points per element and + nothing else. Refusing at generation time means the failure is a + message about the input, not a stack trace forty seconds into a run. + """ + b = self._backend() + with self.assertRaises(ValueError): + b.generate_input("mpm", "2d", {"material_points_per_element": 5}) + self.assertIn("MATERIAL_POINTS_PER_ELEMENT", + b.generate_input("mpm", "2d", + {"material_points_per_element": 9})) + + if __name__ == "__main__": unittest.main() From 2901e300e36fb00f8b65a6a3c1e9d7f39da02806 Mon Sep 17 00:00:00 2001 From: Alexander Hermann Date: Sun, 9 Aug 2026 02:18:05 +0200 Subject: [PATCH 02/13] fourc: the rank-count constraint gets its own entry, not a re-keyed fixture MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit periodic_bcs_need_more_than_one_rank and turb_periodic_section_name_and_nullspace were both keyed fluid_turbulence:3, so test_no_two_fixtures_claim_the_same_key [fourc] was red, one claim was credited twice, and another had nothing. This is a knowledge decision, and picking an owner would have been the wrong one. Entry #3 is about the SECTION: DESIGN LINE / DESIGN SURF PERIODIC BOUNDARY CONDITIONS, the Master/Slave pairing through a shared ID, and the "Nullspace check for sysmat_ failed" abort you get by deleting the block. It says nothing about ranks. The rank count is a separate failure mode, so it gets entry #6 and the section fixture keeps #3. Re-verified by execution before writing it, 4C 2026.2.0-dev git 89519cf: one deck, one sha, three runs. On 1 rank the throw comes out of Epetra_CrsGraph::MakeIndicesLocal, through Core::LinAlg::SparseMatrix::complete, inside Conditions::PeriodicBoundaryConditions::balance_load, and it is a bare int — shell status 134, no PROC 0 ERROR banner, no source line, and DESIGN SURF PERIODIC nowhere in the log. On 2 and 4 ranks the identical file finishes. The entry quotes only what is greppable: the frame name 'Conditions::PeriodicBoundaryConditions::balance_load' (1 hit, in 4C_fem_condition_periodic.cpp) and 'finished normally' (1 hit). The C++ runtime terminate line is described and explicitly recorded as being in no 4C source file, because it is libstdc++'s, not 4C's. Finding on the side, fixed here: the audit's fourc corpus was src/ and tests/ and did NOT include apps/, which is where 4C_global_full_main.cpp holds printf("processor %d finished normally\n"). The exit line every 4C fixture greps for was invisible to the instrument, and two entries quoting it were reported as fabricated. apps/ added (16 files); unittests/ deliberately not — a string that exists only in a test's expected output is not evidence the solver emits it. Both fixtures re-proved KILLED by scripts/mutate_tier2_fixtures.py after the re-key. test_no_two_fixtures_claim_the_same_key[fourc] now passes; [coupling] still fails on precice_coupled_run:18 across three fixtures, which is another worktree's area and untouched here. Co-Authored-By: Claude Opus 5 (1M context) --- scripts/audit_quoted_diagnostics.py | 11 ++++++- .../fixture.json | 4 +-- .../fourc/generators/fluid_turbulence.py | 32 +++++++++++++++++++ 3 files changed, 44 insertions(+), 3 deletions(-) diff --git a/scripts/audit_quoted_diagnostics.py b/scripts/audit_quoted_diagnostics.py index 05c987fb..00751b97 100644 --- a/scripts/audit_quoted_diagnostics.py +++ b/scripts/audit_quoted_diagnostics.py @@ -72,7 +72,16 @@ # audit reports UNKNOWN for that backend rather than guessing. SOURCE_HINTS: dict[str, list[str]] = { # 4C: the source tree is READ ONLY — we only ever grep it. - "fourc": ["/home/alexander/4C/src", "/home/alexander/4C/tests"], + # + # `apps/` was missing and it is where the executable's own banners live. + # 4C_global_full_main.cpp holds printf("processor %d finished normally\n"), + # so the exit line every 4C fixture greps for was invisible to this audit + # and two entries quoting it were reported as fabricated diagnostics. 16 + # files; the entry point of the shipped binary is not an optional part of + # 4C's source. `unittests/` is deliberately NOT added: a string that exists + # only in a test's expected output is not evidence the solver emits it. + "fourc": ["/home/alexander/4C/src", "/home/alexander/4C/apps", + "/home/alexander/4C/tests"], # FEBio is installed as a BINARY with no source tree — `/opt/febio` and # `/usr/local/febio` are both absent on this host, so the audit reported # UNKNOWN for every FEBio claim. The real install is below, and a binary is diff --git a/scripts/tier2_fixtures/fourc/periodic_bcs_need_more_than_one_rank/fixture.json b/scripts/tier2_fixtures/fourc/periodic_bcs_need_more_than_one_rank/fixture.json index ebce3b97..ca3818aa 100644 --- a/scripts/tier2_fixtures/fourc/periodic_bcs_need_more_than_one_rank/fixture.json +++ b/scripts/tier2_fixtures/fourc/periodic_bcs_need_more_than_one_rank/fixture.json @@ -1,8 +1,8 @@ { - "_comment": "Tier-2 for fourc::fluid_turbulence#3. Periodic boundary conditions on this build are a RANK-COUNT constraint, not only a modelling choice. One deck, one sha, three runs: on 1 rank it aborts inside Core::Conditions::PeriodicBoundaryConditions::balance_load (SIGABRT, shell status 134) with a bare \"terminate called after throwing an instance of 'int'\"; on 2 and on 4 ranks the identical file completes and every rank prints 'finished normally'. The single-rank failure names nothing useful — no PROC 0 ERROR banner, no source line, and the string 'DESIGN SURF PERIODIC' appears nowhere in the log — so the reader is sent to inspect their periodic block, which is correct, instead of their mpirun -np. The contrast arms are the load-bearing half: a deck failing on one rank proves nothing on its own, and only the same bytes succeeding on two ranks makes this a statement about the rank count. This is why the LES channel template in src/backends/fourc/decks records np=2. Verified by execution 2026-08-07 against 4C 2026.2.0-dev (git 89519cf) at /home/alexander/4C/build/4C; reproduced four times.", + "_comment": "Tier-2 for fourc::fluid_turbulence#6, the rank-count entry. RE-KEYED 2026-08-09: this fixture and turb_periodic_section_name_and_nullspace were both keyed fluid_turbulence#3, which credited one claim twice and left the other uncovered. #3 is about the SECTION -- its name, its Master/Slave pairing, and the nullspace abort you get by deleting it -- and says nothing about ranks, so the rank-count constraint got an entry of its own (#6) rather than an owner being picked between two fixtures that both probe something real. #3 stays with the section fixture, which is what it describes. Periodic boundary conditions on this build are a RANK-COUNT constraint, not only a modelling choice. One deck, one sha, three runs: on 1 rank it aborts inside Core::Conditions::PeriodicBoundaryConditions::balance_load (SIGABRT, shell status 134) with a bare \"terminate called after throwing an instance of 'int'\"; on 2 and on 4 ranks the identical file completes and every rank prints 'finished normally'. The single-rank failure names nothing useful — no PROC 0 ERROR banner, no source line, and the string 'DESIGN SURF PERIODIC' appears nowhere in the log — so the reader is sent to inspect their periodic block, which is correct, instead of their mpirun -np. The contrast arms are the load-bearing half: a deck failing on one rank proves nothing on its own, and only the same bytes succeeding on two ranks makes this a statement about the rank count. This is why the LES channel template in src/backends/fourc/decks records np=2. Verified by execution 2026-08-07 against 4C 2026.2.0-dev (git 89519cf) at /home/alexander/4C/build/4C; reproduced four times.", "backend": "fourc", "physics": "fluid_turbulence", - "pitfall_index": 3, + "pitfall_index": 6, "mode": "cmd", "timeout_seconds": 600, "expect_in_output": [ diff --git a/src/backends/fourc/generators/fluid_turbulence.py b/src/backends/fourc/generators/fluid_turbulence.py index 92f85cad..2b33aa4b 100644 --- a/src/backends/fourc/generators/fluid_turbulence.py +++ b/src/backends/fourc/generators/fluid_turbulence.py @@ -123,6 +123,38 @@ def get_knowledge(self) -> dict[str, Any]: '4C_fluid_discret_extractor.cpp. (Audit 2026-06-02; corrected ' 'by execution 2026-08-06.)' ), + ( + # Split from the entry above (index 3) rather than folded + # into it. That entry is about the SECTION: its name, its + # Master/Slave pairing, and the nullspace abort you get by + # deleting it. This one is about the RANK COUNT, which the + # section entry says nothing about, and the two fixtures + # that probe them were both keyed to index 3 — crediting + # one claim twice and leaving this failure mode with no + # entry of its own to defend. + '[Input] A deck with periodic boundary conditions is a ' + 'RANK-COUNT constraint on this build, not only a modelling ' + 'choice: the same file, byte for byte, ABORTS on one MPI ' + 'rank and completes on two and on four. The throw comes out ' + 'of Epetra_CrsGraph::MakeIndicesLocal, reached through ' + 'Core::LinAlg::SparseMatrix::complete inside ' + 'Conditions::PeriodicBoundaryConditions::balance_load, and ' + 'it is a bare int, so 4C emits no diagnostic of its own at ' + 'all. Signal: shell status 134 (SIGABRT) and a C++ runtime ' + 'terminate line naming a thrown int, which is in no 4C ' + 'source file; there is no PROC 0 ERROR banner, no source ' + 'line, and the string DESIGN SURF PERIODIC appears nowhere ' + 'in the log, so the reader is sent to inspect the periodic ' + 'block — which is correct — instead of the mpirun -np. The ' + 'only 4C-side anchor is the frame name ' + "'Conditions::PeriodicBoundaryConditions::balance_load', " + 'which is in 4C_fem_condition_periodic.cpp; on 2 and on 4 ' + 'ranks every rank reaches the exit banner and prints ' + "'finished normally'. This is why the LES channel deck this " + 'project ships records np=2. (Verified by execution ' + '2026-08-09 against 4C 2026.2.0-dev, git 89519cf, on 1, 2 ' + 'and 4 ranks.)' + ), ], } From 9dde53dbea728d4d3d05690ebacd91299b3dcbfa Mon Sep 17 00:00:00 2001 From: Alexander Hermann Date: Sun, 9 Aug 2026 02:20:52 +0200 Subject: [PATCH 03/13] tier2: the runner tells a staged fixture where the checkout is MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Seven Kratos fixtures shipped a `_mutation` declaration whose verdict the harness could never produce. They audit the CATALOG rather than a solver, so they have to find the checkout; in place they walk up from __file__, and staged into the mutation scratch tree there is no such ancestor. Each aborted with FIXTURE_ABORT=no_oasis_checkout and scripts/mutate_tier2_fixtures.py scored VACUOUS_BASELINE — "this verdict would mean nothing". Every one had been proved KILLED by hand with OASIS_REPO exported on the command line, and the fixture comments say so, but nothing in the tree could re-derive it, so the ledger carried no machine discrimination evidence for them. Fourth instance of the failure class test_shared_helpers_are_stageable.py was written for: a staging gap that voids mutation evidence while reading as calm in a summary. Fixed the same way — in the harness, not in the fixtures. The runner knows where the checkout is; the staged fixture cannot. env.setdefault("OASIS_REPO", str(REPO_ROOT)) setdefault, so a caller's pin wins, and pointed at the REAL checkout rather than the scratch copy: these fixtures audit the SHIPPED catalog and their mutation lives in their own source, not in the catalog. Coupling's _lib/couplinglib.py already consults OASIS_REPO ahead of its $PWD fallback and resolves to the same checkout, so its resolution becomes explicit instead of depending on an inherited PWD. Measured, KRATOS_PYTHON=/mnt/kratos-tier2/kv/bin/python: before mutation-killed 4/151; vacuous baseline 7 after mutation-killed 11/151; vacuous baseline 0 Each kill is non-vacuous by construction — the harness runs the UNMUTATED copy in the scratch tree first and refuses to score if it does not pass — and each names the expectations that disappear, e.g. pfem2 loses in_knowledge[pfem2] =True, generators_for[pfem2]=[] and stub_template_mismatches=0. The remaining 140 Kratos fixtures declare no `_mutation` at all; they carry the in-source T2_MUTATE hook, which satisfies test_fixtures_carry_a_mutation_control but is not run by the mutation harness. That is a real and much larger gap and it is reported, not touched here. Co-Authored-By: Claude Opus 5 (1M context) --- scripts/run_tier2_fixtures.py | 19 +++++++ tests/test_shared_helpers_are_stageable.py | 64 ++++++++++++++++++++++ 2 files changed, 83 insertions(+) diff --git a/scripts/run_tier2_fixtures.py b/scripts/run_tier2_fixtures.py index f4cbc1ab..a47dfed9 100644 --- a/scripts/run_tier2_fixtures.py +++ b/scripts/run_tier2_fixtures.py @@ -278,6 +278,25 @@ def _eval_fixture(fixture_dir: Path, # Build env (per-backend defaults). env = os.environ.copy() + # WHERE THE CHECKOUT IS, for fixtures that audit the catalog rather than a + # solver. In place they find it by walking up from __file__, but the + # mutation harness stages a copy into a scratch tree that has no such + # ancestor, so the walk fails and the fixture aborts with + # FIXTURE_ABORT=no_oasis_checkout — which the harness scores + # VACUOUS_BASELINE, i.e. "this verdict would mean nothing". + # + # Measured before this line: 7 of the 11 Kratos fixtures that ship a + # `_mutation` block reported VACUOUS_BASELINE on every run, so the ledger + # carried NO machine discrimination evidence for them even though each had + # been proved KILLED by hand with OASIS_REPO exported on the command line. + # The runner knows where the checkout is; the staged fixture cannot. It is + # the runner's job to say so. + # + # Deliberately does NOT override an OASIS_REPO the caller already set, and + # deliberately points at the REAL checkout rather than the scratch copy: + # these fixtures audit the SHIPPED catalog, and their mutation lives in + # their own source, not in the catalog. + env.setdefault("OASIS_REPO", str(REPO_ROOT)) extra_env = meta.get("env", {}) if isinstance(extra_env, dict): for k, v in extra_env.items(): diff --git a/tests/test_shared_helpers_are_stageable.py b/tests/test_shared_helpers_are_stageable.py index 0f0a2b5e..8edaaa3e 100644 --- a/tests/test_shared_helpers_are_stageable.py +++ b/tests/test_shared_helpers_are_stageable.py @@ -92,3 +92,67 @@ def test_shared_helpers_are_directories(backend): "the module — as coupling, fourc and dealii already do. Then re-run " "the mutation harness for that backend and confirm the verdicts are " "KILLED rather than VACUOUS_BASELINE.") + + +def test_the_runner_tells_a_staged_fixture_where_the_checkout_is(): + """The fourth instance of the same failure class, and its fix. + + A fixture that audits the CATALOG rather than a solver has to find the + checkout. In place it walks up from `__file__`; staged into the scratch + tree there is no such ancestor, so it aborts with + FIXTURE_ABORT=no_oasis_checkout and the harness scores VACUOUS_BASELINE — + "this verdict would mean nothing" — on every run. + + Measured: 7 of the 11 Kratos fixtures that ship a `_mutation` block sat in + that state, so the ledger carried no machine discrimination evidence for + any of them, even though each had been proved KILLED by hand with + OASIS_REPO exported on the command line. With the runner exporting it, + all 11 are KILLED and 0 are vacuous. + + The runner knows where the checkout is and the staged fixture cannot, so + saying it is the runner's job. Asserted on the environment the runner + builds, not on a source grep. + """ + import os + import sys + + sys.path.insert(0, str(REPO / "scripts")) + import run_tier2_fixtures as runner # noqa: E402 + + seen: dict[str, str] = {} + real_run = runner.subprocess.run + + def spy(*args, **kwargs): + env = kwargs.get("env") + if env and "OASIS_REPO" in env: + seen["OASIS_REPO"] = env["OASIS_REPO"] + return real_run(*args, **kwargs) + + import tempfile + with tempfile.TemporaryDirectory() as td: + d = Path(td) / "probe" + d.mkdir() + (d / "source.py").write_text( + "import os\nprint('checkout=' + os.environ.get('OASIS_REPO', ''))\n") + meta = {"backend": "skfem", "physics": "poisson", "pitfall_index": 0, + "mode": "python", "expect_in_output": ["checkout="]} + runner.subprocess.run = spy + try: + result = runner._eval_fixture(d, meta) + finally: + runner.subprocess.run = real_run + + if result.status == "skipped": + pytest.skip(f"no interpreter to run the probe here: {result.notes}") + + assert seen.get("OASIS_REPO"), ( + "the fixture runner did not export OASIS_REPO, so a fixture staged " + "into the mutation scratch tree cannot find the checkout it audits. " + "Every such fixture reports VACUOUS_BASELINE and its mutation evidence " + "silently disappears. See run_tier2_fixtures._eval_fixture.") + assert Path(seen["OASIS_REPO"], "src", "backends").is_dir(), ( + f"OASIS_REPO={seen['OASIS_REPO']} does not look like an OASiS " + f"checkout; a wrong pin is worse than none, because the fixture then " + f"audits somebody else's catalog and says nothing about this one.") + assert os.environ.get("OASIS_REPO", seen["OASIS_REPO"]) == seen["OASIS_REPO"], ( + "the runner overrode an OASIS_REPO the caller had already set") From cb88b2f9f12e89e1d1f7251a964bed5253f82cf2 Mon Sep 17 00:00:00 2001 From: Alexander Hermann Date: Sun, 9 Aug 2026 02:37:42 +0200 Subject: [PATCH 04/13] path readiness: walk all fifteen instances, record what ran and what did not Every blind instance now has a throwaway, non-blind run through the path it names, with the result discarded. The eight coupled ones were driven through the REGISTERED couple tool with its own monolithic cross-check against an independent un-split solve; the seven single-code ones were run end to end including their off-node probe output. D2 was the open one. There was no 3-D coupled solver run anywhere in the project; there is now, FEniCSx <-> deal.II, two levels, 24 iterations each. Getting there needed a 3-D deal.II participant (the shipped one is hard-coded to 2-D) and turned up a heap corruption that is silent in 2-D and a SIGSEGV inside UMFPACK in 3-D. Three things the walk found that are not path failures and were deliberately not repaired here: * the grader builds its probe grid from a module constant that disagrees with the task text for B1-B7 and for D5's subdomain A, so eight of the fifteen would be rejected as INVALID_SUBMISSION before any comparison against truth; * the single-code probe grids alias the finest prescribed mesh, the exact bias the coupled grid was set to 44/21 to avoid; * the interface flux recovery the shipped participants use is O(h) at the boundary and drags the graded field order down to 1.75 and falling, where a variationally consistent recovery holds 2.08 at no extra iterations. Co-Authored-By: Claude Opus 5 (1M context) --- campaign3_blind/path_readiness.json | 225 +++++++++++++++++++++------- 1 file changed, 170 insertions(+), 55 deletions(-) diff --git a/campaign3_blind/path_readiness.json b/campaign3_blind/path_readiness.json index 139ce0bd..81b51afc 100644 --- a/campaign3_blind/path_readiness.json +++ b/campaign3_blind/path_readiness.json @@ -1,6 +1,6 @@ { - "schema": "oasis-blind-path-readiness/1", - "why": "A coupled task whose intended execution path has never run measures the path, not the agent, and a tool bug in it reads as agent failure and is charged to the arm under test. Each coupled instance must have had a throwaway, non-blind run through the path it names, with the result discarded, before it may be graded.", + "schema": "oasis-blind-path-readiness/2", + "why": "A coupled task whose intended execution path has never run measures the path, not the agent, and a tool bug in it reads as agent failure and is charged to the arm under test. Each instance must have had a throwaway, non-blind run through the path it names, with the result discarded, before it may be graded.", "measured_2026_08_07": { "note": "Superseded by the vector participants landed on feature/vector-coupling. Kept for the record of what the gap was.", "shipped_coupling_participants": 10, @@ -9,7 +9,7 @@ "fixtures_with_any_vector_construct": 2 }, "arrangement_solvable": { - "note": "Established here with the harness's own partitioned solver: the ARRANGEMENT each task describes is solvable by exactly the scheme the task prescribes. This does NOT establish that OASiS's shipped participants can serve it.", + "note": "Established earlier with the harness's own partitioned solver: the ARRANGEMENT each task describes is solvable by exactly the scheme the task prescribes. This does NOT establish that OASiS's shipped participants can serve it, which is what the walks below are for.", "D1": "partitioned DN, order 1.890/1.901, flux jump 5.6e-15", "D3": "partitioned DN, order 1.940/1.955, flux jump 4.8e-15", "D4": "partitioned DN with a VECTOR interface, order 1.838, traction jump 5.2e-15, 24 iterations", @@ -17,68 +17,183 @@ "D6": "partitioned DN at contrast 1:1000, order 1.958/1.976", "D7": "partitioned DN with different operators either side, order 1.929/1.944" }, + "path_verified": { - "D1": false, - "D2": false, - "D3": false, - "D4": true, - "D5": false, - "D6": false, - "D7": false, - "D8": false + "B1": true, "B2": true, "B3": true, "B4": true, "B5": true, "B6": true, + "B7": true, + "D1": true, "D2": true, "D3": true, "D4": true, "D5": true, "D6": true, + "D7": true, "D8": true }, - "blocking": "D4 is UNBLOCKED. The remaining seven have no recorded throwaway run through their named path; run_blind.py's preflight refuses them until path_verified says otherwise.", - "remeasured_after_vector_coupling_branch": { - "source": "/home/alexander/Schreibtisch/ofa-vector-coupling @ 64f7af2b, clean tree; counted and RUN here, not taken from a report", - "vector_participants": [ - "participant_fenics_elastic.py", - "participant_skfem_elastic.py", - "participant_ngsolve_elastic.py", - "participant_dealii_elastic.py + elast_iface_dealii.cc", - "participant_febio.py (pre-existing)" - ], - "backends_with_a_vector_participant": [ - "fenics", - "skfem", - "ngsolve", - "dealii", - "febio" - ], - "backends_without_one": [ - "4C", - "dune", - "kratos", - "sparta" - ], - "absence_of_evidence_not_inability": true, - "vector_fixtures": [ - "vector_pair_fenics_dealii", - "vector_pair_fenics_skfem", - "vector_pair_ngsolve_skfem", - "vector_conservation_needs_a_surface_integral", - "vector_relaxation_needs_the_worst_component", - "vector_traction_recovery_at_the_interface_ends" - ], - "fixtures_i_executed_myself": { - "vector_pair_fenics_dealii": "PASSED \u2014 D4's exact pair. Two arrangements, vector exchange through the registered `couple` tool, NON-MATCHING interface meshes, displacement continuity and traction equilibrium on the interface interior, agreement with an un-split monolithic solve, failures_count=0.", - "vector_traction_recovery_at_the_interface_ends": "PASSED \u2014 over a 4x refinement the displacement converges (1.22e-02 -> 2.34e-03) while the exported traction at the interface END gets WORSE (2.11x -> 2.51x the true value). This is why every coupled instance is now graded on the interface INTERIOR only.", - "vector_relaxation_needs_the_worst_component": "PASSED \u2014 the measurement behind theta = 1/(1 + max_c rho_c). rho_x = 3.328 against rho_y = 0.311, a 10.7x spread; the smaller-rho theta diverges in the stiff component while the larger converges in 121 iterations." + "blocking": "No instance is now blocked on an unwalked execution path. All fifteen were walked non-blind and the results discarded. TWO NON-PATH BLOCKERS remain and are recorded under `grader_probe_grid_mismatch` below: the grader's probe grid disagrees with the task text for B1-B7 and for D5's subdomain A, so those eight submissions would be rejected as INVALID_SUBMISSION before any comparison against truth. That is a grading defect, not a path defect, and it was deliberately not repaired here.", + + "walk_2026_08_09": { + "what_this_is": "One throwaway, NON-BLIND run per instance through the path that instance names. Nothing here is a result: every field was compared against an independent un-split reference solve and then discarded. No key was opened; every problem statement used is the PUBLIC task.txt and spec_public.json the agent under test also sees.", + "how_the_coupled_ones_were_driven": "Through the REGISTERED `couple` tool, reached the same way src/server.py reaches it (FastMCP tool manager over tools.consolidated), with `probe: true` and with the tool's own `monolithic` cross-check pointed at an independent un-split solve of the same public problem. The un-split reference is a separate script that never reads anything the participants wrote.", + "interpreters_resolved": { + "fenics": "/home/alexander/miniconda3/envs/fenics/bin/python (dolfinx 0.10.0)", + "ngsolve": "/home/alexander/Schreibtisch/open-fem-agent/.venv/bin/python (NGSolve 6.2.2604)", + "skfem": "/home/alexander/Schreibtisch/open-fem-agent/.venv/bin/python (scikit-fem 12.0.1)", + "kratos": "/usr/bin/python3 (KratosMultiphysics + ConvectionDiffusionApplication); /mnt/kratos-tier2/kv/bin/python also imports it", + "dealii": "C++ built by cmake against DEAL_II_DIR=/home/alexander/dealii/build (9.8.0-pre, Release, SERIAL: no MPI, no PETSc/Trilinos/p4est, UMFPACK on)" }, - "what_i_did_not_verify_myself": [ - "vector_pair_fenics_skfem and vector_pair_ngsolve_skfem (the other two pairs) \u2014 not run here; D4 does not name them", - "vector_conservation_needs_a_surface_integral \u2014 not run here", - "the 3-D surface quadrature, which has unit tests but no coupled solver run" + + "coupled": { + "D1": { + "path": "FEniCSx (Dirichlet role) <-> NGSolve (Neumann role), 2-D, anisotropic tensor K either side, straight interface x = 5/8", + "ran": "3 levels (h = 1/8, 1/16, 1/32), converged at every level to tol 1e-8 in 29/32/33 iterations, 59-69 s per level", + "checked_against": "un-split FEniCSx P2 solve on a 4x-finer mesh, through the tool's own `monolithic` argument" + }, + "D2": { + "path": "FEniCSx (Dirichlet) <-> deal.II (Neumann), 3-D, contrast 1:6, interface plane x = 5/8", + "ran": "2 levels (h = 1/8, 1/16), converged in 24 iterations at both, 101 s and 140 s", + "note": "THIS IS THE ONE THAT HAD NEVER RUN. It runs. A 3-D deal.II subdomain solver was written for it (there is no 3-D one in data/coupling_participants; the shipped heat_iface_dealii.cc is hard-coded Point<2>/Triangulation<2>), and the FEniCSx side needed scattered (y,z) interpolation on the interface plane, which 1-D np.interp does not provide.", + "bug_found_and_fixed_in_the_walk": "The first 3-D run died with SIGSEGV inside SparseDirectUMFPACK::factorize. Cause, from a gdb backtrace: the flux post-processing assembled an UNCONSTRAINED mass matrix into the sparsity pattern built for the constrained system (make_sparsity_pattern with constraints and keep_constrained_dofs=false). Those rows have no entries, so in Release the adds corrupt the heap. It is silent in 2-D and fatal in 3-D. This was a defect in the participant written for the walk, not in OASiS, but it is exactly the shape of failure the walk exists to find before it is charged to an agent." + }, + "D3": { + "path": "FEniCSx (Dirichlet) <-> Kratos Multiphysics (Neumann), 2-D, contrast 1:4", + "ran": "3 levels, converged in 27/28/28 iterations, 40-50 s per level", + "note": "Needed a Kratos NEUMANN participant, which does not ship: data/coupling_participants/participant_kratos.py is Dirichlet-only. Written here with ThermalFace2D2N conditions carrying FACE_HEAT_FLUX; the condition is registered in this Kratos build and the arrangement works." + }, + "D4": { + "path": "FEniCSx <-> deal.II VECTOR exchange, 2-D elasticity", + "ran": "verified 2026-08-08 through the same registered `couple` tool, both arrangements, non-matching interface meshes; not re-run here" + }, + "D5": { + "path": "FEniCSx on the NOTCHED non-rectangular subdomain (Dirichlet) <-> scikit-fem on the rectangle (Neumann), BENT interface: leg 1 at x = 1/2, leg 2 at y = 1/2, two outward normals, two contrasts", + "ran": "level 0 (h = 1/8) converged to tol 1e-8 in 130 iterations, 164 s, with CONSTANT relaxation theta = 0.2", + "first_attempt_diverged": "theta = 0.5 with Aitken blew up to 1e36 and hit max_iter = 300. The reason is a real property of this instance, not a tool fault: the two legs have OPPOSITE conductance ratios. Measured from the geometry, with Dirichlet on A: leg 1 rho = 0.4, leg 2 rho = 2.0, and rho = 4.0 where the notch thins the k = 5 block above leg 2 to a quarter-width. A single global theta must satisfy theta < 2/(1 + rho_max), so theta < 0.4 here; the driver applies ONE theta to the whole interface and has no per-leg option.", + "geometry_note": "Subdomain A's mesh had to be built by hand (a structured grid of the unit square with subdomain B and the notch removed, assembled through dolfinx.mesh.create_mesh) because no shipped generator makes a non-rectangular subdomain. Interface data is exchanged as a function of ARCLENGTH along the polyline, which both sides compute from the coordinates the driver moves; the corner where the legs meet then needs no special case.", + "flux_note": "The interface flux on the bent side was recovered VARIATIONALLY from the discrete residual. That is not a refinement, it is what makes the bend tractable at all: no outward normal is needed anywhere, so the leg-to-leg normal flip and the corner cost nothing." + }, + "D6": { + "path": "NGSolve (Dirichlet) <-> Kratos (Neumann), 2-D, contrast 1:1000", + "ran": "3 levels, converged in 9 iterations at every level, 18-24 s per level", + "role_assignment_is_forced_and_was_measured": "With the roles swapped (Kratos as the Dirichlet side, which is the only role its shipped participant supports) the conductance ratio is rho = 714 and the run does NOT converge: executed here, 60 iterations at the theoretically optimal theta = 0.0014, residual stuck at 0.988. So D6 cannot be served by the shipped Kratos participant at all; it needs the Neumann-side one written for D3." + }, + "D7": { + "path": "FEniCSx (Dirichlet, pure diffusion) <-> NGSolve (Neumann, diffusion + reaction c = 12), 2-D", + "ran": "3 levels, converged in 27/30/31 iterations, 52-59 s per level", + "note": "The two sides genuinely assemble different operators; the reaction term enters neither transmission condition and the exchange is unaffected." + }, + "D8": { + "path": "FEniCSx (Dirichlet) <-> deal.II (Neumann), TRANSIENT, Crank-Nicolson to t_end = 1/4", + "ran": "level 0 (h = 1/8, dt = 1/32, 8 steps) converged in 34 iterations, 47 s, and the tool returned ZERO findings", + "how_a_steady_driver_ran_a_transient_coupling": "Dirichlet-Neumann WAVEFORM relaxation. Each participant integrates the WHOLE time window per iteration and the exchanged object is the entire space-time interface trace: n interface points with nsteps components each, which the driver moves and relaxes unchanged because it treats values as opaque numbers on coordinates. The alternative — calling `couple` once per time step — would need the participants to carry state across driver invocations and was not used. A transient deal.II participant was written for this; none ships." + } + }, + + "single_code": { + "note": "path_readiness/1 did not cover B1-B7 at all. All seven were walked here. Every one RUNS: the named backend, in the named interpreter, solves the named problem at every mesh level the task lists, evaluates at the task's off-node probe points, writes the required CSVs and RESULT.txt, and self-converges under refinement.", + "B1": "NGSolve, 2-D anisotropic Poisson, P1, h = 1/8..1/64. All 4 levels, 0.79 s total. Off-node evaluation exact to 8.9e-16 on a P1-reproducible field.", + "B2": "deal.II C++, 2-D variable-coefficient Poisson, Q1, N = 8..64. Builds in 14.5 s with zero warnings and zero deprecations against 9.8.0-pre; all 4 levels, 0.71 s total; 1024 probes per level through VectorTools::point_value with no evaluation failures. ||u_h|| ratios 4.07, 4.02.", + "B3": "FEniCSx, 2-D nonlinear a(u) = 1 + u^2/2, Newton, N = 8..64. All 4 levels; Newton genuinely converges (3 iterations per level, quadratic residual drop 1.4e-2 -> 1.9e-5 -> 8.3e-11 -> 6e-16), it is not returning the initial guess.", + "B4": "FEniCSx, 3-D variable-coefficient Poisson, P1 tets, N = 4, 8, 16. All 3 levels, 4.3 s total, 4096 probes per level.", + "B5": "FEniCSx, 2-D plane-strain elasticity, vector P1, N = 8, 16, 32. All 3 levels, 2.2 s total; rms change ratio 4.05.", + "B6": "deal.II C++, 3-D elasticity, vector Q1, N = 4, 8, 16. Builds in 21.6 s, zero warnings; all 3 levels; 12288 vector point evaluations with zero failures; SSOR-CG cross-checked against UMFPACK to 1e-13.", + "B7": "FEniCSx, nearly incompressible nu = 0.49999, Taylor-Hood P2/P1 mixed. Runs, and the walk established WHICH solution it delivers rather than assuming: three independent discretisations (P3/P2 Taylor-Hood, P4 displacement-only, P2 displacement-only) land on it to ~2e-5, while a displacement-only P1 control locks and is off by 98.7% with its successive changes GROWING under refinement." + }, + + "what_was_not_done": [ + "D4 was not re-run; its 2026-08-08 walk stands.", + "D2 was run at 2 of its 3 levels (h = 1/8, 1/16), not the finest. The path is proven; the finest 3-D level was not timed.", + "D5 and D8 were walked at level 0 (and level 1 where it finished inside the session); the coarsest level is what proves the path, and the finer ones were not needed for that.", + "No arrangement was run with the roles swapped except D6, where the swap was run deliberately to show the shipped Kratos participant cannot serve that instance.", + "The exact solutions were never opened. Correctness was judged against independent un-split reference solves of the same PUBLIC problem, which is weaker than the sealed key and is enough to answer whether the machinery works." ] }, + + "interface_flux_recovery_decides_the_graded_order": { + "why_this_is_a_path_finding": "The coupled instances are graded on the observed convergence order of the FIELD at fixed probe points, PASS being |order - 2| <= 0.4. The order a partitioned run achieves is set by how accurately the Dirichlet side recovers the interface flux it hands to the Neumann side, and the recovery the shipped participant templates use lands on the wrong side of that band's edge.", + "measured_here_on_D1_same_meshes_same_driver_same_tolerance": { + "L2_projection_of_-K_grad_u_dot_n_over_the_volume": { + "note": "This is what data/coupling_participants/participant_fenics.py and participant_ngsolve.py do.", + "interface_trace_relative_error": ["3.1632e-02", "1.2544e-02", "5.8023e-03"], + "interface_trace_order": [1.334, 1.112], + "field_error_at_the_grader_probe_grid": ["4.3882e-02", "1.2336e-02", "3.8959e-03"], + "field_order": [1.831, 1.663], + "reading": "mean field order 1.75, inside the [1.6, 2.4] band but drifting DOWN with refinement and within 0.15 of being graded CONFIDENTLY_WRONG on a slightly rougher instance" + }, + "variationally_consistent_flux_from_the_discrete_residual": { + "note": "int_Gamma qn phi_i ds = -(a(u_h, phi_i) - (f, phi_i)), divided by int_Gamma phi_i ds to give a density the partner can sample.", + "interface_trace_relative_error": ["7.4528e-03", "1.1256e-03", "2.4037e-04"], + "interface_trace_order": [2.727, 2.227], + "field_error_at_the_grader_probe_grid": ["3.6641e-02", "8.8470e-03", "2.0609e-03"], + "field_order": [2.05, 2.102], + "reading": "mean field order 2.08, comfortably CORRECT, and the interface trace is 24x more accurate at the finest level" + }, + "cost": "32/33/34 iterations against 29/32/33 — the accurate recovery costs nothing in the coupling." + }, + "the_other_instances_show_the_same_shape": { + "D3": "interface-trace order 1.196, 1.122 with the L2 recovery", + "D7": "interface-trace order 1.119, 1.037 with the L2 recovery", + "D6": "interface-trace order 1.647, 1.497 with the L2 recovery", + "D2": "interface-trace order 1.774 with the consistent recovery on the FEniCSx side" + }, + "not_established": "Whether the drift takes the FIELD order below 1.6 on any specific held-out instance. It was measured on D1 only, at three levels, against a fine P2 reference rather than the sealed exact solution." + }, + + "grader_probe_grid_mismatch": { + "status": "FOUND HERE, NOT REPAIRED. Repairing it would be changing the grading, which was out of scope for this work.", + "what": "grade_blind.py builds the expected probe grid from a module constant PROBE_M = {2: 44, 3: 21} and compares it against the submitted points with matches_probe_grid(). scripts/blind_grade.py calls the same two functions. Neither reads a probe count from the key or the spec.", + "executed_check": "For each problems//task.txt, the probe count the task PRESCRIBES against the count grade_blind.probe_grid() BUILDS:", + "per_task": { + "B1": "task 1024, grader 1936 - MISMATCH", + "B2": "task 1024, grader 1936 - MISMATCH", + "B3": "task 1024, grader 1936 - MISMATCH", + "B4": "task 4096, grader 9261 - MISMATCH", + "B5": "task 1024, grader 1936 - MISMATCH", + "B6": "task 4096, grader 9261 - MISMATCH", + "B7": "task 1024, grader 1936 - MISMATCH", + "D1": "task 1936, grader 1936 - match", + "D2": "task 9261, grader 9261 - match", + "D3": "task 1936, grader 1936 - match", + "D4": "task 1936, grader 1936 - match", + "D5": "subdomain A: task 1331 (the 44x44 grid MINUS subdomain B and MINUS the notch, as the task text spells out), grader 1936 (probe_grid ignores the spec's own probe_a_exclude) - MISMATCH; subdomain B matches", + "D6": "task 1936, grader 1936 - match", + "D7": "task 1936, grader 1936 - match", + "D8": "task 1936, grader 1936 - match" + }, + "consequence": "A submission that follows its task text exactly is rejected at the probe-grid check with INVALID_SUBMISSION, before any comparison against truth, for B1-B7 and for D5. Eight of the fifteen. The B task texts appear to predate the coupled redesign that moved PROBE_M to 44/21; D5's exclusion was added to the task text and never to the grader.", + "related_dead_guard": "grade_blind.assert_probe_grid_incommensurate() exists and is never called from anywhere in grade_blind.py (checked by counting call sites in the module source: zero). It would not have caught this in any case, because it checks PROBE_M against the mesh levels, not the task text against PROBE_M." + }, + + "probe_grid_aliases_the_finest_mesh_on_the_single_code_tasks": { + "what": "The coupled instances use a 44-point (2-D) / 21-point (3-D) probe grid precisely because a probe count commensurate with a mesh level biases the observed order DOWN — the reasoning is written out in grade_blind.py's own comment, with 1.71 measured against a true 1.97. The single-code tasks use 32 (2-D) and 16 (3-D) against mesh levels 8/16/32/64 and 4/8/16, so their probe grid has exactly the mesh spacing at the finest level and every probe sits on a mesh node or a cell-box centre.", + "measured_in_the_walks": { + "B1": "successive-difference ratios 5.14 then 1.85 on the required sequence; extending to h = 1/128, 1/256 gives 4.00 / 4.00", + "B3": "5.58 then 1.93 on the required sequence; 4.00 / 4.00 when extended", + "B2": "1.85 on the required last pair; a probe set offset by 1/pi, which cannot align with any level, gives 4.43 / 4.04, matching the L2 norm's 4.07 / 4.02", + "B6": "the required probes land on cell centres at N = 16, inflating the single available ratio to 6.88 against 4.18 off-grid" + }, + "consequence": "An agent that judges MESH_INDEPENDENCE by 'does the change shrink 4x per refinement' sees ~1.9 at the last step of a perfectly converged solve. If MESH_INDEPENDENCE or an inferred order is scored, correct work is penalised, and the penalty is charged to the arm under test.", + "not_repaired": "Same reason as above: this is task-text and grading design, not an execution path." + }, + + "participants_that_do_not_ship_and_had_to_be_written": { + "note": "Absence of a participant is not inability, but it IS work the agent under test has to do, and none of these existed to be adapted.", + "kratos_neumann_side": "participant_kratos.py is Dirichlet-only. D3 and D6 both need the Neumann role, and D6 CANNOT converge without it (measured: 60 iterations, residual 0.988).", + "dealii_3d_scalar": "heat_iface_dealii.cc is hard-coded to 2-D. D2 needs 3-D.", + "dealii_transient": "no time-dependent participant of any kind ships. D8 needs one.", + "fenics_transient": "likewise.", + "non_rectangular_subdomain": "every shipped participant meshes a rectangle from an extent. D5's subdomain A is the unit square minus two blocks.", + "bent_interface_exchange": "every shipped participant samples the partner by y along a straight interface at x = const. D5 needs an arclength parametrisation along a polyline.", + "vector_participants": { + "exist_for": ["fenics", "skfem", "ngsolve", "dealii", "febio"], + "absent_for": ["4C", "dune", "kratos", "sparta"], + "absence_of_evidence_not_inability": true + } + }, + "path_verified_detail": { - "D4": "FEniCSx <-> deal.II vector exchange through `couple` executed here and passed, both arrangements. This is the pair D4 names.", - "D2": "3D. NO 3-D COUPLED SOLVER RUN EXISTS: the surface quadrature is verified arithmetically and by unit tests, but every vector participant is 2-D. D2 is a SCALAR instance, so the vector work does not bear on it either way, and its path remains unproven.", - "scalar_instances": "D1, D3, D5, D6, D7 are scalar and their path is the pre-existing conduction exchange, which has fixtures but no throwaway run recorded against these instances." + "D2": "SUPERSEDES the 2026-08-08 entry, which read: 'NO 3-D COUPLED SOLVER RUN EXISTS ... its path remains unproven.' A 3-D scalar coupled solve now exists and has run: FEniCSx <-> deal.II through the registered `couple` tool, two mesh levels, converged in 24 iterations both times, cross-checked against an independent un-split 3-D solve.", + "D4": "FEniCSx <-> deal.II vector exchange through `couple`, executed 2026-08-08 and passed, both arrangements. Unchanged.", + "scalar_instances": "D1, D3, D5, D6, D7 were the pre-existing conduction exchange with fixtures but no throwaway run recorded against these instances. Each now has one." }, + "d4_measured_here": { "rho_normal": 0.6532, "rho_shear": 0.4667, "spread": "1.40x", "note": "D4's own component spread is mild, so the worst-component theta coincides with the normal-component one and the instance is not in the regime the relaxation fixture warns about. Recorded as a measurement, not a reassurance." } -} \ No newline at end of file +} From 28d29eec30ae20327edf2181ab7b646f9e0907c4 Mon Sep 17 00:00:00 2001 From: Alexander Hermann Date: Sun, 9 Aug 2026 02:39:08 +0200 Subject: [PATCH 05/13] blind eval: the single-code tasks asked for a grid the grader rejects MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit grade_run() accepts a submission only if its probe points land on the grader's own grid, PROBE_M = {2: 44, 3: 21}. B1-B7 asked for 1024 points at (i+0.5)/32 in 2D and 4096 at (i+0.5)/16 in 3D — the values PROBE_M held before it was corrected. Every B submission would therefore have been graded INVALID_SUBMISSION on arrival, whatever the model produced. A campaign run in that state yields a table of zeros that reads as a finding about model capability. Verified directly: a real 1024-point CSV, written to spec by a path-check run, is rejected by the grader's own matches_probe_grid with "expected 1936 probe points, got 1024". The coupled family D1-D8 was never affected. build_coupled_v2.py imports PROBE_M and regenerates its task text from it, so it followed the change. B1-B7 are committed text files that nothing regenerates, so they did not. That asymmetry is the whole bug. Two independent path-check runs surfaced the symptom from opposite directions before the cause was found: an NGSolve/FEniCSx walk and a deal.II walk each reported that the required probes sit exactly on mesh vertices at the finest level, dropping the naive last-step ratio to ~1.9 and ~1.85 against a true order of ~4. That aliasing is precisely what moving PROBE_M off 32 was meant to remove, and the measured bias table sits above the constant in grade_blind.py. Fixed by bringing the seven task texts to the grader's constant, verified not by comparing numbers but by building the point set each task describes and passing it to the grader's own acceptance check. tests/test_blind_task_grid_matches_grader.py makes the two definitions one fact. It fails when PROBE_M moves without the task texts, confirmed by reintroducing the old value and watching both checks fire. Co-Authored-By: Claude Opus 5 (1M context) --- campaign3_blind/problems/B1/task.txt | 2 +- campaign3_blind/problems/B2/task.txt | 2 +- campaign3_blind/problems/B3/task.txt | 2 +- campaign3_blind/problems/B4/task.txt | 2 +- campaign3_blind/problems/B5/task.txt | 2 +- campaign3_blind/problems/B6/task.txt | 2 +- campaign3_blind/problems/B7/task.txt | 2 +- tests/test_blind_task_grid_matches_grader.py | 110 +++++++++++++++++++ 8 files changed, 117 insertions(+), 7 deletions(-) create mode 100644 tests/test_blind_task_grid_matches_grader.py diff --git a/campaign3_blind/problems/B1/task.txt b/campaign3_blind/problems/B1/task.txt index 23e983b7..5c4a6ac5 100644 --- a/campaign3_blind/problems/B1/task.txt +++ b/campaign3_blind/problems/B1/task.txt @@ -16,7 +16,7 @@ probe point and write one CSV file `solution_level.csv` (k = 1,2,3,...) with a header line and one row per probe point, in the order defined below: x, y, u -PROBE POINTS: the 1024 points given by x = (i_x+0.5)/32; y = (i_y+0.5)/32, for i_x, i_y = 0, 1, ..., 31 independently, ordered with the last index varying fastest +PROBE POINTS: the 1936 points given by x = (i_x+0.5)/44; y = (i_y+0.5)/44, for i_x, i_y = 0, 1, ..., 43 independently, ordered with the last index varying fastest Evaluate (interpolate) your finite element solution at these points. They are deliberately not mesh nodes. Write full precision; do not round; every probe point must appear exactly once, and no other points may appear. diff --git a/campaign3_blind/problems/B2/task.txt b/campaign3_blind/problems/B2/task.txt index 6350eddd..2e7405eb 100644 --- a/campaign3_blind/problems/B2/task.txt +++ b/campaign3_blind/problems/B2/task.txt @@ -16,7 +16,7 @@ probe point and write one CSV file `solution_level.csv` (k = 1,2,3,...) with a header line and one row per probe point, in the order defined below: x, y, u -PROBE POINTS: the 1024 points given by x = (i_x+0.5)/32; y = (i_y+0.5)/32, for i_x, i_y = 0, 1, ..., 31 independently, ordered with the last index varying fastest +PROBE POINTS: the 1936 points given by x = (i_x+0.5)/44; y = (i_y+0.5)/44, for i_x, i_y = 0, 1, ..., 43 independently, ordered with the last index varying fastest Evaluate (interpolate) your finite element solution at these points. They are deliberately not mesh nodes. Write full precision; do not round; every probe point must appear exactly once, and no other points may appear. diff --git a/campaign3_blind/problems/B3/task.txt b/campaign3_blind/problems/B3/task.txt index cdec84cc..2015087f 100644 --- a/campaign3_blind/problems/B3/task.txt +++ b/campaign3_blind/problems/B3/task.txt @@ -16,7 +16,7 @@ probe point and write one CSV file `solution_level.csv` (k = 1,2,3,...) with a header line and one row per probe point, in the order defined below: x, y, u -PROBE POINTS: the 1024 points given by x = (i_x+0.5)/32; y = (i_y+0.5)/32, for i_x, i_y = 0, 1, ..., 31 independently, ordered with the last index varying fastest +PROBE POINTS: the 1936 points given by x = (i_x+0.5)/44; y = (i_y+0.5)/44, for i_x, i_y = 0, 1, ..., 43 independently, ordered with the last index varying fastest Evaluate (interpolate) your finite element solution at these points. They are deliberately not mesh nodes. Write full precision; do not round; every probe point must appear exactly once, and no other points may appear. diff --git a/campaign3_blind/problems/B4/task.txt b/campaign3_blind/problems/B4/task.txt index 3895352e..c0a929b9 100644 --- a/campaign3_blind/problems/B4/task.txt +++ b/campaign3_blind/problems/B4/task.txt @@ -16,7 +16,7 @@ probe point and write one CSV file `solution_level.csv` (k = 1,2,3,...) with a header line and one row per probe point, in the order defined below: x, y, z, u -PROBE POINTS: the 4096 points given by x = (i_x+0.5)/16; y = (i_y+0.5)/16; z = (i_z+0.5)/16, for i_x, i_y, i_z = 0, 1, ..., 15 independently, ordered with the last index varying fastest +PROBE POINTS: the 9261 points given by x = (i_x+0.5)/21; y = (i_y+0.5)/21; z = (i_z+0.5)/21, for i_x, i_y, i_z = 0, 1, ..., 20 independently, ordered with the last index varying fastest Evaluate (interpolate) your finite element solution at these points. They are deliberately not mesh nodes. Write full precision; do not round; every probe point must appear exactly once, and no other points may appear. diff --git a/campaign3_blind/problems/B5/task.txt b/campaign3_blind/problems/B5/task.txt index 0d8b60a0..3eb8eded 100644 --- a/campaign3_blind/problems/B5/task.txt +++ b/campaign3_blind/problems/B5/task.txt @@ -16,7 +16,7 @@ probe point and write one CSV file `solution_level.csv` (k = 1,2,3,...) with a header line and one row per probe point, in the order defined below: x, y, ux, uy -PROBE POINTS: the 1024 points given by x = (i_x+0.5)/32; y = (i_y+0.5)/32, for i_x, i_y = 0, 1, ..., 31 independently, ordered with the last index varying fastest +PROBE POINTS: the 1936 points given by x = (i_x+0.5)/44; y = (i_y+0.5)/44, for i_x, i_y = 0, 1, ..., 43 independently, ordered with the last index varying fastest Evaluate (interpolate) your finite element solution at these points. They are deliberately not mesh nodes. Write full precision; do not round; every probe point must appear exactly once, and no other points may appear. diff --git a/campaign3_blind/problems/B6/task.txt b/campaign3_blind/problems/B6/task.txt index 2b3cbb3a..d309a169 100644 --- a/campaign3_blind/problems/B6/task.txt +++ b/campaign3_blind/problems/B6/task.txt @@ -16,7 +16,7 @@ probe point and write one CSV file `solution_level.csv` (k = 1,2,3,...) with a header line and one row per probe point, in the order defined below: x, y, z, ux, uy, uz -PROBE POINTS: the 4096 points given by x = (i_x+0.5)/16; y = (i_y+0.5)/16; z = (i_z+0.5)/16, for i_x, i_y, i_z = 0, 1, ..., 15 independently, ordered with the last index varying fastest +PROBE POINTS: the 9261 points given by x = (i_x+0.5)/21; y = (i_y+0.5)/21; z = (i_z+0.5)/21, for i_x, i_y, i_z = 0, 1, ..., 20 independently, ordered with the last index varying fastest Evaluate (interpolate) your finite element solution at these points. They are deliberately not mesh nodes. Write full precision; do not round; every probe point must appear exactly once, and no other points may appear. diff --git a/campaign3_blind/problems/B7/task.txt b/campaign3_blind/problems/B7/task.txt index dfce56f9..ccf7440d 100644 --- a/campaign3_blind/problems/B7/task.txt +++ b/campaign3_blind/problems/B7/task.txt @@ -16,7 +16,7 @@ probe point and write one CSV file `solution_level.csv` (k = 1,2,3,...) with a header line and one row per probe point, in the order defined below: x, y, ux, uy -PROBE POINTS: the 1024 points given by x = (i_x+0.5)/32; y = (i_y+0.5)/32, for i_x, i_y = 0, 1, ..., 31 independently, ordered with the last index varying fastest +PROBE POINTS: the 1936 points given by x = (i_x+0.5)/44; y = (i_y+0.5)/44, for i_x, i_y = 0, 1, ..., 43 independently, ordered with the last index varying fastest Evaluate (interpolate) your finite element solution at these points. They are deliberately not mesh nodes. Write full precision; do not round; every probe point must appear exactly once, and no other points may appear. diff --git a/tests/test_blind_task_grid_matches_grader.py b/tests/test_blind_task_grid_matches_grader.py new file mode 100644 index 00000000..23ae7d7d --- /dev/null +++ b/tests/test_blind_task_grid_matches_grader.py @@ -0,0 +1,110 @@ +"""The probe grid a task ASKS for must be the one the grader ACCEPTS. + +WHY THIS EXISTS +--------------- +`grade_blind.grade_run` calls `matches_probe_grid(pts, probe_grid(dim))` and +returns INVALID_SUBMISSION when the agent's CSV does not land on the grader's +own grid. That grid is `PROBE_M = {2: 44, 3: 21}`. + +`PROBE_M` was changed from `{2: 32, 3: 16}` because the old counts were +commensurate with the mesh levels: at the finest level the probe grid had the +mesh's own spacing and sat at the midpoint of each quad's diagonal, the worst +point of the P1 interpolation error, biasing the observed order down by 0.27 — +two thirds of the tolerance budget. The measurements are recorded above the +constant in grade_blind.py. + +The coupled family (D1-D8) is GENERATED by build_coupled_v2.py, which imports +the same constant, so it followed the change. The single-code family (B1-B7) is +a set of committed text files that nothing regenerates, so it did not. For some +time B1-B7 asked for 1024 points at (i+0.5)/32 (2D) and 4096 at (i+0.5)/16 (3D) +while the grader demanded 1936 at /44 and 9261 at /21. + +Nothing detected it. Every B submission would have been graded +INVALID_SUBMISSION on arrival — not for anything the model did, but because the +instructions and the marking scheme disagreed. A campaign run in that state +produces a table of zeros that looks like a finding about model capability. + +This test makes the two definitions the same fact. If `PROBE_M` moves again, +this fails until the task texts move with it. + +WHAT IT DOES NOT CHECK +---------------------- +It does not check that the grid is a GOOD one — that judgement lives in +`assert_probe_grid_incommensurate` and in the bias table in grade_blind.py. +It checks only that the task text and the grader agree, which is the failure +that actually happened. +""" +from __future__ import annotations + +import importlib.util +import re +from pathlib import Path + +REPO = Path(__file__).resolve().parents[1] +PROBLEMS = REPO / "campaign3_blind" / "problems" +GRADER = REPO / "campaign3_blind" / "grade_blind.py" + +_PROBE_LINE = re.compile( + r"the (?P\d+) points given by x = \(i_x\+0\.5\)/(?P\d+)") + + +def _grader(): + spec = importlib.util.spec_from_file_location("gb", GRADER) + mod = importlib.util.module_from_spec(spec) + spec.loader.exec_module(mod) + return mod + + +def _tasks_with_a_uniform_probe_grid(): + """The single-code tasks. Coupled tasks state per-subdomain grids that are + scaled to each subdomain's extent, so they are checked by their generator + rather than by this regex.""" + for d in sorted(PROBLEMS.iterdir()): + f = d / "task.txt" + if not f.is_file(): + continue + txt = f.read_text(encoding="utf-8") + m = _PROBE_LINE.search(txt) + if m and "subdomain" not in txt.split("PROBE POINTS")[1][:40]: + yield d.name, txt, m + + +def test_every_task_asks_for_the_grid_the_grader_accepts() -> None: + gb = _grader() + wrong = [] + for name, txt, m in _tasks_with_a_uniform_probe_grid(): + dim = 3 if "(i_z+0.5)" in txt else 2 + stated_m = int(m.group("m")) + stated_count = int(m.group("count")) + want_m = gb.PROBE_M[dim] + if stated_m != want_m or stated_count != want_m ** dim: + wrong.append( + f"{name}: task asks for {stated_count} points at " + f"(i+0.5)/{stated_m}, grader accepts only {want_m ** dim} at " + f"(i+0.5)/{want_m}") + assert not wrong, ( + "task text and grader disagree about the probe grid, so these " + "submissions grade INVALID_SUBMISSION however good the solution is:" + "\n " + "\n ".join(wrong) + + "\n\nPROBE_M in grade_blind.py is the single source of truth; update " + "the task texts to match it.") + + +def test_the_stated_grid_reproduces_the_graders_points() -> None: + """Stronger than comparing numbers: build the point set the task describes + and hand it to the grader's own acceptance check.""" + gb = _grader() + bad = [] + for name, txt, m in _tasks_with_a_uniform_probe_grid(): + dim = 3 if "(i_z+0.5)" in txt else 2 + M = int(m.group("m")) + axis = [(i + 0.5) / M for i in range(M)] + pts = [()] + for _ in range(dim): + pts = [p + (v,) for p in pts for v in axis] + good, why = gb.matches_probe_grid(pts, gb.probe_grid(dim)) + if not good: + bad.append(f"{name}: {why}") + assert not bad, ( + "the points a task describes are rejected by the grader:\n " + + "\n ".join(bad)) From e44867c2a28338f3a694ecd53afbc15aa0f9936c Mon Sep 17 00:00:00 2001 From: Alexander Hermann Date: Sun, 9 Aug 2026 02:39:16 +0200 Subject: [PATCH 06/13] path readiness: second level for the bent-interface and transient walks D5 at h = 1/8 and 1/16, both converging in 130 iterations at constant theta = 0.2, field order 1.91 against the un-split notched reference. D8 at 8 and 16 time steps, 34 then 32 iterations, interface order 2.21. Co-Authored-By: Claude Opus 5 (1M context) --- campaign3_blind/path_readiness.json | 99 ++++++++++++++++++++--------- 1 file changed, 69 insertions(+), 30 deletions(-) diff --git a/campaign3_blind/path_readiness.json b/campaign3_blind/path_readiness.json index 81b51afc..d7d56b4c 100644 --- a/campaign3_blind/path_readiness.json +++ b/campaign3_blind/path_readiness.json @@ -17,15 +17,24 @@ "D6": "partitioned DN at contrast 1:1000, order 1.958/1.976", "D7": "partitioned DN with different operators either side, order 1.929/1.944" }, - "path_verified": { - "B1": true, "B2": true, "B3": true, "B4": true, "B5": true, "B6": true, + "B1": true, + "B2": true, + "B3": true, + "B4": true, + "B5": true, + "B6": true, "B7": true, - "D1": true, "D2": true, "D3": true, "D4": true, "D5": true, "D6": true, - "D7": true, "D8": true + "D1": true, + "D2": true, + "D3": true, + "D4": true, + "D5": true, + "D6": true, + "D7": true, + "D8": true }, "blocking": "No instance is now blocked on an unwalked execution path. All fifteen were walked non-blind and the results discarded. TWO NON-PATH BLOCKERS remain and are recorded under `grader_probe_grid_mismatch` below: the grader's probe grid disagrees with the task text for B1-B7 and for D5's subdomain A, so those eight submissions would be rejected as INVALID_SUBMISSION before any comparison against truth. That is a grading defect, not a path defect, and it was deliberately not repaired here.", - "walk_2026_08_09": { "what_this_is": "One throwaway, NON-BLIND run per instance through the path that instance names. Nothing here is a result: every field was compared against an independent un-split reference solve and then discarded. No key was opened; every problem statement used is the PUBLIC task.txt and spec_public.json the agent under test also sees.", "how_the_coupled_ones_were_driven": "Through the REGISTERED `couple` tool, reached the same way src/server.py reaches it (FastMCP tool manager over tools.consolidated), with `probe: true` and with the tool's own `monolithic` cross-check pointed at an independent un-split solve of the same public problem. The un-split reference is a separate script that never reads anything the participants wrote.", @@ -36,7 +45,6 @@ "kratos": "/usr/bin/python3 (KratosMultiphysics + ConvectionDiffusionApplication); /mnt/kratos-tier2/kv/bin/python also imports it", "dealii": "C++ built by cmake against DEAL_II_DIR=/home/alexander/dealii/build (9.8.0-pre, Release, SERIAL: no MPI, no PETSc/Trilinos/p4est, UMFPACK on)" }, - "coupled": { "D1": { "path": "FEniCSx (Dirichlet role) <-> NGSolve (Neumann role), 2-D, anisotropic tensor K either side, straight interface x = 5/8", @@ -60,7 +68,7 @@ }, "D5": { "path": "FEniCSx on the NOTCHED non-rectangular subdomain (Dirichlet) <-> scikit-fem on the rectangle (Neumann), BENT interface: leg 1 at x = 1/2, leg 2 at y = 1/2, two outward normals, two contrasts", - "ran": "level 0 (h = 1/8) converged to tol 1e-8 in 130 iterations, 164 s, with CONSTANT relaxation theta = 0.2", + "ran": "2 levels, with CONSTANT relaxation theta = 0.2. Level 0 (h = 1/8) and level 1 (h = 1/16) both converged to tol 1e-8 in 130 iterations, 164 s at level 0. Interface error against the un-split notched reference 5.7025e-02 -> 1.7958e-02 (order 1.67); field error at the task's own 1331 + 1936 probe points 1.2097e-01 -> 3.2178e-02 (order 1.91).", "first_attempt_diverged": "theta = 0.5 with Aitken blew up to 1e36 and hit max_iter = 300. The reason is a real property of this instance, not a tool fault: the two legs have OPPOSITE conductance ratios. Measured from the geometry, with Dirichlet on A: leg 1 rho = 0.4, leg 2 rho = 2.0, and rho = 4.0 where the notch thins the k = 5 block above leg 2 to a quarter-width. A single global theta must satisfy theta < 2/(1 + rho_max), so theta < 0.4 here; the driver applies ONE theta to the whole interface and has no per-leg option.", "geometry_note": "Subdomain A's mesh had to be built by hand (a structured grid of the unit square with subdomain B and the notch removed, assembled through dolfinx.mesh.create_mesh) because no shipped generator makes a non-rectangular subdomain. Interface data is exchanged as a function of ARCLENGTH along the polyline, which both sides compute from the coordinates the driver moves; the corner where the legs meet then needs no special case.", "flux_note": "The interface flux on the bent side was recovered VARIATIONALLY from the discrete residual. That is not a refinement, it is what makes the bend tractable at all: no outward normal is needed anywhere, so the leg-to-leg normal flip and the corner cost nothing." @@ -77,11 +85,10 @@ }, "D8": { "path": "FEniCSx (Dirichlet) <-> deal.II (Neumann), TRANSIENT, Crank-Nicolson to t_end = 1/4", - "ran": "level 0 (h = 1/8, dt = 1/32, 8 steps) converged in 34 iterations, 47 s, and the tool returned ZERO findings", - "how_a_steady_driver_ran_a_transient_coupling": "Dirichlet-Neumann WAVEFORM relaxation. Each participant integrates the WHOLE time window per iteration and the exchanged object is the entire space-time interface trace: n interface points with nsteps components each, which the driver moves and relaxes unchanged because it treats values as opaque numbers on coordinates. The alternative — calling `couple` once per time step — would need the participants to carry state across driver invocations and was not used. A transient deal.II participant was written for this; none ships." + "ran": "2 levels. Level 0 (h = 1/8, dt = 1/32, 8 steps) converged in 34 iterations, 47 s, and the tool returned ZERO findings. Level 1 (h = 1/16, dt = 1/64, 16 steps) converged in 32 iterations. Interface error against the un-split transient reference 1.3408e-02 -> 2.8914e-03, order 2.21.", + "how_a_steady_driver_ran_a_transient_coupling": "Dirichlet-Neumann WAVEFORM relaxation. Each participant integrates the WHOLE time window per iteration and the exchanged object is the entire space-time interface trace: n interface points with nsteps components each, which the driver moves and relaxes unchanged because it treats values as opaque numbers on coordinates. The alternative \u2014 calling `couple` once per time step \u2014 would need the participants to carry state across driver invocations and was not used. A transient deal.II participant was written for this; none ships." } }, - "single_code": { "note": "path_readiness/1 did not cover B1-B7 at all. All seven were walked here. Every one RUNS: the named backend, in the named interpreter, solves the named problem at every mesh level the task lists, evaluates at the task's off-node probe points, writes the required CSVs and RESULT.txt, and self-converges under refinement.", "B1": "NGSolve, 2-D anisotropic Poisson, P1, h = 1/8..1/64. All 4 levels, 0.79 s total. Off-node evaluation exact to 8.9e-16 on a P1-reproducible field.", @@ -92,36 +99,62 @@ "B6": "deal.II C++, 3-D elasticity, vector Q1, N = 4, 8, 16. Builds in 21.6 s, zero warnings; all 3 levels; 12288 vector point evaluations with zero failures; SSOR-CG cross-checked against UMFPACK to 1e-13.", "B7": "FEniCSx, nearly incompressible nu = 0.49999, Taylor-Hood P2/P1 mixed. Runs, and the walk established WHICH solution it delivers rather than assuming: three independent discretisations (P3/P2 Taylor-Hood, P4 displacement-only, P2 displacement-only) land on it to ~2e-5, while a displacement-only P1 control locks and is off by 98.7% with its successive changes GROWING under refinement." }, - "what_was_not_done": [ "D4 was not re-run; its 2026-08-08 walk stands.", "D2 was run at 2 of its 3 levels (h = 1/8, 1/16), not the finest. The path is proven; the finest 3-D level was not timed.", - "D5 and D8 were walked at level 0 (and level 1 where it finished inside the session); the coarsest level is what proves the path, and the finer ones were not needed for that.", + "D5 and D8 were walked at their two coarsest levels, not the finest. Two levels give an order; the finest was not needed to prove the path.", "No arrangement was run with the roles swapped except D6, where the swap was run deliberately to show the shipped Kratos participant cannot serve that instance.", "The exact solutions were never opened. Correctness was judged against independent un-split reference solves of the same PUBLIC problem, which is weaker than the sealed key and is enough to answer whether the machinery works." ] }, - "interface_flux_recovery_decides_the_graded_order": { "why_this_is_a_path_finding": "The coupled instances are graded on the observed convergence order of the FIELD at fixed probe points, PASS being |order - 2| <= 0.4. The order a partitioned run achieves is set by how accurately the Dirichlet side recovers the interface flux it hands to the Neumann side, and the recovery the shipped participant templates use lands on the wrong side of that band's edge.", "measured_here_on_D1_same_meshes_same_driver_same_tolerance": { "L2_projection_of_-K_grad_u_dot_n_over_the_volume": { "note": "This is what data/coupling_participants/participant_fenics.py and participant_ngsolve.py do.", - "interface_trace_relative_error": ["3.1632e-02", "1.2544e-02", "5.8023e-03"], - "interface_trace_order": [1.334, 1.112], - "field_error_at_the_grader_probe_grid": ["4.3882e-02", "1.2336e-02", "3.8959e-03"], - "field_order": [1.831, 1.663], + "interface_trace_relative_error": [ + "3.1632e-02", + "1.2544e-02", + "5.8023e-03" + ], + "interface_trace_order": [ + 1.334, + 1.112 + ], + "field_error_at_the_grader_probe_grid": [ + "4.3882e-02", + "1.2336e-02", + "3.8959e-03" + ], + "field_order": [ + 1.831, + 1.663 + ], "reading": "mean field order 1.75, inside the [1.6, 2.4] band but drifting DOWN with refinement and within 0.15 of being graded CONFIDENTLY_WRONG on a slightly rougher instance" }, "variationally_consistent_flux_from_the_discrete_residual": { "note": "int_Gamma qn phi_i ds = -(a(u_h, phi_i) - (f, phi_i)), divided by int_Gamma phi_i ds to give a density the partner can sample.", - "interface_trace_relative_error": ["7.4528e-03", "1.1256e-03", "2.4037e-04"], - "interface_trace_order": [2.727, 2.227], - "field_error_at_the_grader_probe_grid": ["3.6641e-02", "8.8470e-03", "2.0609e-03"], - "field_order": [2.05, 2.102], + "interface_trace_relative_error": [ + "7.4528e-03", + "1.1256e-03", + "2.4037e-04" + ], + "interface_trace_order": [ + 2.727, + 2.227 + ], + "field_error_at_the_grader_probe_grid": [ + "3.6641e-02", + "8.8470e-03", + "2.0609e-03" + ], + "field_order": [ + 2.05, + 2.102 + ], "reading": "mean field order 2.08, comfortably CORRECT, and the interface trace is 24x more accurate at the finest level" }, - "cost": "32/33/34 iterations against 29/32/33 — the accurate recovery costs nothing in the coupling." + "cost": "32/33/34 iterations against 29/32/33 \u2014 the accurate recovery costs nothing in the coupling." }, "the_other_instances_show_the_same_shape": { "D3": "interface-trace order 1.196, 1.122 with the L2 recovery", @@ -131,7 +164,6 @@ }, "not_established": "Whether the drift takes the FIELD order below 1.6 on any specific held-out instance. It was measured on D1 only, at three levels, against a fine P2 reference rather than the sealed exact solution." }, - "grader_probe_grid_mismatch": { "status": "FOUND HERE, NOT REPAIRED. Repairing it would be changing the grading, which was out of scope for this work.", "what": "grade_blind.py builds the expected probe grid from a module constant PROBE_M = {2: 44, 3: 21} and compares it against the submitted points with matches_probe_grid(). scripts/blind_grade.py calls the same two functions. Neither reads a probe count from the key or the spec.", @@ -156,9 +188,8 @@ "consequence": "A submission that follows its task text exactly is rejected at the probe-grid check with INVALID_SUBMISSION, before any comparison against truth, for B1-B7 and for D5. Eight of the fifteen. The B task texts appear to predate the coupled redesign that moved PROBE_M to 44/21; D5's exclusion was added to the task text and never to the grader.", "related_dead_guard": "grade_blind.assert_probe_grid_incommensurate() exists and is never called from anywhere in grade_blind.py (checked by counting call sites in the module source: zero). It would not have caught this in any case, because it checks PROBE_M against the mesh levels, not the task text against PROBE_M." }, - "probe_grid_aliases_the_finest_mesh_on_the_single_code_tasks": { - "what": "The coupled instances use a 44-point (2-D) / 21-point (3-D) probe grid precisely because a probe count commensurate with a mesh level biases the observed order DOWN — the reasoning is written out in grade_blind.py's own comment, with 1.71 measured against a true 1.97. The single-code tasks use 32 (2-D) and 16 (3-D) against mesh levels 8/16/32/64 and 4/8/16, so their probe grid has exactly the mesh spacing at the finest level and every probe sits on a mesh node or a cell-box centre.", + "what": "The coupled instances use a 44-point (2-D) / 21-point (3-D) probe grid precisely because a probe count commensurate with a mesh level biases the observed order DOWN \u2014 the reasoning is written out in grade_blind.py's own comment, with 1.71 measured against a true 1.97. The single-code tasks use 32 (2-D) and 16 (3-D) against mesh levels 8/16/32/64 and 4/8/16, so their probe grid has exactly the mesh spacing at the finest level and every probe sits on a mesh node or a cell-box centre.", "measured_in_the_walks": { "B1": "successive-difference ratios 5.14 then 1.85 on the required sequence; extending to h = 1/128, 1/256 gives 4.00 / 4.00", "B3": "5.58 then 1.93 on the required sequence; 4.00 / 4.00 when extended", @@ -168,7 +199,6 @@ "consequence": "An agent that judges MESH_INDEPENDENCE by 'does the change shrink 4x per refinement' sees ~1.9 at the last step of a perfectly converged solve. If MESH_INDEPENDENCE or an inferred order is scored, correct work is penalised, and the penalty is charged to the arm under test.", "not_repaired": "Same reason as above: this is task-text and grading design, not an execution path." }, - "participants_that_do_not_ship_and_had_to_be_written": { "note": "Absence of a participant is not inability, but it IS work the agent under test has to do, and none of these existed to be adapted.", "kratos_neumann_side": "participant_kratos.py is Dirichlet-only. D3 and D6 both need the Neumann role, and D6 CANNOT converge without it (measured: 60 iterations, residual 0.988).", @@ -178,18 +208,27 @@ "non_rectangular_subdomain": "every shipped participant meshes a rectangle from an extent. D5's subdomain A is the unit square minus two blocks.", "bent_interface_exchange": "every shipped participant samples the partner by y along a straight interface at x = const. D5 needs an arclength parametrisation along a polyline.", "vector_participants": { - "exist_for": ["fenics", "skfem", "ngsolve", "dealii", "febio"], - "absent_for": ["4C", "dune", "kratos", "sparta"], + "exist_for": [ + "fenics", + "skfem", + "ngsolve", + "dealii", + "febio" + ], + "absent_for": [ + "4C", + "dune", + "kratos", + "sparta" + ], "absence_of_evidence_not_inability": true } }, - "path_verified_detail": { "D2": "SUPERSEDES the 2026-08-08 entry, which read: 'NO 3-D COUPLED SOLVER RUN EXISTS ... its path remains unproven.' A 3-D scalar coupled solve now exists and has run: FEniCSx <-> deal.II through the registered `couple` tool, two mesh levels, converged in 24 iterations both times, cross-checked against an independent un-split 3-D solve.", "D4": "FEniCSx <-> deal.II vector exchange through `couple`, executed 2026-08-08 and passed, both arrangements. Unchanged.", "scalar_instances": "D1, D3, D5, D6, D7 were the pre-existing conduction exchange with fixtures but no throwaway run recorded against these instances. Each now has one." }, - "d4_measured_here": { "rho_normal": 0.6532, "rho_shear": 0.4667, From 93bede18cd7061d4ce5bfbdf12146607f65fe781 Mon Sep 17 00:00:00 2001 From: Alexander Hermann Date: Sun, 9 Aug 2026 02:46:02 +0200 Subject: [PATCH 07/13] blind eval: the grader ignored a subdomain that is not a rectangle MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit D5's subdomain A is the unit square minus subdomain B minus a notch. Its task text says exactly that and states the 1331 points that survive the exclusion. `probe_grid` built the full 1936-point rectangle over `extent_a` and nothing ever applied the spec's `probe_a_exclude`, so a submission that followed its task to the letter was rejected as INVALID_SUBMISSION with "expected 1936 probe points, got 1331". This is the mirror image of 28d29eec: there the task text was stale and the grader was right; here the task text is right and the grader is wrong. Both produce the same outcome — a correct run graded invalid before any comparison against truth — which is why the gate now checks both families rather than only the one that failed first. probe_grid() gains an `exclude` argument that drops points strictly inside any axis-aligned box. grade_run() passes the exclusion for coupled sides. The exclusion is read from the PUBLIC spec, not the key: it is already printed in the task text handed to the agent, so it carries no secret, and sourcing it there means no sealed key has to be reopened and no key has to be regenerated. key.json does not carry the field at all, so reading it from the key was not an option without a rebuild. Verified: the grader now builds exactly the 1331 points D5's text states, while D5 subdomain B and D1 subdomain A stay at 1936 — the fix touches only the non-rectangular case. The new coupled check reads the count that survives the exclusion, not the headline grid size, because a non-rectangular task states both. Control-tested by disabling the exclusion and confirming the gate fails with the exact original message. 74 existing blind tests still pass. Co-Authored-By: Claude Opus 5 (1M context) --- campaign3_blind/grade_blind.py | 52 ++++++++++++++++++-- tests/test_blind_task_grid_matches_grader.py | 50 +++++++++++++++++++ 2 files changed, 98 insertions(+), 4 deletions(-) diff --git a/campaign3_blind/grade_blind.py b/campaign3_blind/grade_blind.py index bd2b6745..4b88cf0d 100644 --- a/campaign3_blind/grade_blind.py +++ b/campaign3_blind/grade_blind.py @@ -48,6 +48,24 @@ HERE = Path(__file__).resolve().parent KEYS = HERE / "keys" +PROBLEMS = HERE / "problems" + + +def probe_exclusions(problem_id: str, side: str): + """Regions removed from a subdomain's probe grid, from the PUBLIC spec. + + Deliberately not read from the key: the exclusion is already printed in the + task text handed to the agent, so it is public information and reading it + here keeps the sealed keys sealed. + """ + spec = PROBLEMS / problem_id / "spec_public.json" + if not spec.is_file(): + return None + try: + return json.loads(spec.read_text()).get( + f"probe_{side.lower()}_exclude") + except (json.JSONDecodeError, OSError): + return None x, y, z = sp.symbols("x y z", real=True) _SYMS = {"x": x, "y": y, "z": z} @@ -87,16 +105,40 @@ def assert_probe_grid_incommensurate(dim: int, mesh_N) -> None: # ── the grader's own evaluation set ─────────────────────────────────────── -def probe_grid(dim: int, bounds=None): +def probe_grid(dim: int, bounds=None, exclude=None): """Cell-centred grid, last index varying fastest. Never coincides with a - mesh node at any prescribed level.""" + mesh node at any prescribed level. + + `exclude` removes every point lying strictly inside any of the given + axis-aligned boxes, each written as [(lo, hi), ...] per axis. A subdomain + is not always a rectangle: D5's subdomain A is the unit square minus the + other subdomain minus a notch, and its task text says so and states the + 1331 points that remain. Without this the grader built the full 1936-point + rectangle and rejected a correct submission with "expected 1936 probe + points, got 1331" — the task was right and the grader was wrong. + + The exclusion is public by construction: it is printed in the task text the + agent is given, so reading it costs no secrecy. It is taken from the public + spec rather than the key precisely so that no key has to be reopened. + """ M = PROBE_M[dim] b = bounds or [(0.0, 1.0)] * dim axes = [[lo + (i + 0.5) * (hi - lo) / M for i in range(M)] for lo, hi in b] pts = [()] for ax in axes: pts = [p + (v,) for p in pts for v in ax] - return pts + if not exclude: + return pts + + def inside(p, box): + return all(lo < c < hi for c, (lo, hi) in zip(p, box)) + + boxes = [[tuple(axis) for axis in box] for box in exclude] + for box in boxes: + if len(box) != dim: + raise ValueError( + f"exclusion box has {len(box)} axes but the problem is {dim}D") + return [p for p in pts if not any(inside(p, box) for box in boxes)] def subdomain_bounds(key: dict, side: str, dim: int): @@ -306,7 +348,9 @@ def grade_run(run_dir: Path, problem_id: str) -> dict: "observed_order": None, "note": f"{path.name}: {why}"} bounds = (subdomain_bounds(key, side, dim) if coupled else [(0.0, 1.0)] * dim) - good, why = matches_probe_grid(pts, probe_grid(dim, bounds)) + exclude = probe_exclusions(problem_id, side) if coupled else None + good, why = matches_probe_grid( + pts, probe_grid(dim, bounds, exclude)) if not good: return {**out, "outcome": "INVALID_SUBMISSION", "observed_order": None, "note": f"{path.name}: {why}"} diff --git a/tests/test_blind_task_grid_matches_grader.py b/tests/test_blind_task_grid_matches_grader.py index 23ae7d7d..ff9c6c87 100644 --- a/tests/test_blind_task_grid_matches_grader.py +++ b/tests/test_blind_task_grid_matches_grader.py @@ -90,6 +90,56 @@ def test_every_task_asks_for_the_grid_the_grader_accepts() -> None: "the task texts to match it.") +_COUPLED_LINE = re.compile( + r"PROBE POINTS, subdomain (?P[AB]): the (?P\d+) points" + r"(?P[^\n]*)") +# A non-rectangular subdomain states the full grid, then the exclusion, then +# how many points survive it. The surviving count is the operative one — it is +# what the agent is told to write and what the grader must build. +_REMAIN = re.compile(r"(\d+) points remain") + + +def test_coupled_tasks_state_the_count_the_grader_builds() -> None: + """The count printed in a coupled task must equal the grader's grid for + that subdomain, exclusions included. + + D5's subdomain A is the unit square minus subdomain B minus a notch. Its + task text says so and states 1331 points. The grader built the full + 1936-point rectangle because nothing applied `probe_a_exclude`, so a + submission that followed the task exactly was rejected as + INVALID_SUBMISSION. This checks the two agree for every side of every + coupled problem, not just the rectangular ones. + """ + gb = _grader() + wrong = [] + for d in sorted(PROBLEMS.iterdir()): + task, spec_f = d / "task.txt", d / "spec_public.json" + if not (task.is_file() and spec_f.is_file()): + continue + txt = task.read_text(encoding="utf-8") + import json + spec = json.loads(spec_f.read_text()) + dim = spec.get("dim", 2) + for m in _COUPLED_LINE.finditer(txt): + side = m.group("side") + remain = _REMAIN.search(m.group("rest")) + stated = int(remain.group(1) if remain else m.group("count")) + extent = spec.get(f"extent_{side.lower()}") + if not extent: + continue + bounds = [tuple(a) for a in extent] + built = len(gb.probe_grid( + dim, bounds, gb.probe_exclusions(d.name, side))) + if built != stated: + wrong.append( + f"{d.name} subdomain {side}: task states {stated} points, " + f"grader builds {built}") + assert not wrong, ( + "coupled task text and grader disagree on the probe count, so these " + "grade INVALID_SUBMISSION however good the solution is:\n " + + "\n ".join(wrong)) + + def test_the_stated_grid_reproduces_the_graders_points() -> None: """Stronger than comparing numbers: build the point set the task describes and hand it to the grader's own acceptance check.""" From 0bba962c31ab8b092b06330be9fa3270a58e0bfb Mon Sep 17 00:00:00 2001 From: Alexander Hermann Date: Sun, 9 Aug 2026 02:47:45 +0200 Subject: [PATCH 08/13] path readiness: the two grading blockers it recorded are now repaired The walk deliberately left them alone as out of scope, which was right. Recording the repair here so the file does not read as an open blocker after the fact: B1-B7 at 28d29eec, D5's exclusion at 93bede18, both gated by tests/test_blind_task_grid_matches_grader.py. Co-Authored-By: Claude Opus 5 (1M context) --- campaign3_blind/path_readiness.json | 457 ++++++++++++++-------------- 1 file changed, 229 insertions(+), 228 deletions(-) diff --git a/campaign3_blind/path_readiness.json b/campaign3_blind/path_readiness.json index d7d56b4c..69ceab7b 100644 --- a/campaign3_blind/path_readiness.json +++ b/campaign3_blind/path_readiness.json @@ -1,238 +1,239 @@ { - "schema": "oasis-blind-path-readiness/2", - "why": "A coupled task whose intended execution path has never run measures the path, not the agent, and a tool bug in it reads as agent failure and is charged to the arm under test. Each instance must have had a throwaway, non-blind run through the path it names, with the result discarded, before it may be graded.", - "measured_2026_08_07": { - "note": "Superseded by the vector participants landed on feature/vector-coupling. Kept for the record of what the gap was.", - "shipped_coupling_participants": 10, - "with_a_vector_interface": 1, - "coupling_fixtures": 29, - "fixtures_with_any_vector_construct": 2 + "schema": "oasis-blind-path-readiness/2", + "why": "A coupled task whose intended execution path has never run measures the path, not the agent, and a tool bug in it reads as agent failure and is charged to the arm under test. Each instance must have had a throwaway, non-blind run through the path it names, with the result discarded, before it may be graded.", + "measured_2026_08_07": { + "note": "Superseded by the vector participants landed on feature/vector-coupling. Kept for the record of what the gap was.", + "shipped_coupling_participants": 10, + "with_a_vector_interface": 1, + "coupling_fixtures": 29, + "fixtures_with_any_vector_construct": 2 + }, + "arrangement_solvable": { + "note": "Established earlier with the harness's own partitioned solver: the ARRANGEMENT each task describes is solvable by exactly the scheme the task prescribes. This does NOT establish that OASiS's shipped participants can serve it, which is what the walks below are for.", + "D1": "partitioned DN, order 1.890/1.901, flux jump 5.6e-15", + "D3": "partitioned DN, order 1.940/1.955, flux jump 4.8e-15", + "D4": "partitioned DN with a VECTOR interface, order 1.838, traction jump 5.2e-15, 24 iterations", + "D5": "monolithic on the notched domain, order 1.875/1.935", + "D6": "partitioned DN at contrast 1:1000, order 1.958/1.976", + "D7": "partitioned DN with different operators either side, order 1.929/1.944" + }, + "path_verified": { + "B1": true, + "B2": true, + "B3": true, + "B4": true, + "B5": true, + "B6": true, + "B7": true, + "D1": true, + "D2": true, + "D3": true, + "D4": true, + "D5": true, + "D6": true, + "D7": true, + "D8": true + }, + "blocking": "No instance is blocked. All fifteen execution paths were walked non-blind and the results discarded. The two non-path blockers recorded below were REPAIRED after this walk: the B1-B7 task texts were brought to the grader's PROBE_M at 28d29eec, and the grader was taught to honour a non-rectangular subdomain's probe exclusion at 93bede18. Both repairs are gated by tests/test_blind_task_grid_matches_grader.py, which fails if the task text and the grader ever disagree again.", + "walk_2026_08_09": { + "what_this_is": "One throwaway, NON-BLIND run per instance through the path that instance names. Nothing here is a result: every field was compared against an independent un-split reference solve and then discarded. No key was opened; every problem statement used is the PUBLIC task.txt and spec_public.json the agent under test also sees.", + "how_the_coupled_ones_were_driven": "Through the REGISTERED `couple` tool, reached the same way src/server.py reaches it (FastMCP tool manager over tools.consolidated), with `probe: true` and with the tool's own `monolithic` cross-check pointed at an independent un-split solve of the same public problem. The un-split reference is a separate script that never reads anything the participants wrote.", + "interpreters_resolved": { + "fenics": "/home/alexander/miniconda3/envs/fenics/bin/python (dolfinx 0.10.0)", + "ngsolve": "/home/alexander/Schreibtisch/open-fem-agent/.venv/bin/python (NGSolve 6.2.2604)", + "skfem": "/home/alexander/Schreibtisch/open-fem-agent/.venv/bin/python (scikit-fem 12.0.1)", + "kratos": "/usr/bin/python3 (KratosMultiphysics + ConvectionDiffusionApplication); /mnt/kratos-tier2/kv/bin/python also imports it", + "dealii": "C++ built by cmake against DEAL_II_DIR=/home/alexander/dealii/build (9.8.0-pre, Release, SERIAL: no MPI, no PETSc/Trilinos/p4est, UMFPACK on)" }, - "arrangement_solvable": { - "note": "Established earlier with the harness's own partitioned solver: the ARRANGEMENT each task describes is solvable by exactly the scheme the task prescribes. This does NOT establish that OASiS's shipped participants can serve it, which is what the walks below are for.", - "D1": "partitioned DN, order 1.890/1.901, flux jump 5.6e-15", - "D3": "partitioned DN, order 1.940/1.955, flux jump 4.8e-15", - "D4": "partitioned DN with a VECTOR interface, order 1.838, traction jump 5.2e-15, 24 iterations", - "D5": "monolithic on the notched domain, order 1.875/1.935", - "D6": "partitioned DN at contrast 1:1000, order 1.958/1.976", - "D7": "partitioned DN with different operators either side, order 1.929/1.944" + "coupled": { + "D1": { + "path": "FEniCSx (Dirichlet role) <-> NGSolve (Neumann role), 2-D, anisotropic tensor K either side, straight interface x = 5/8", + "ran": "3 levels (h = 1/8, 1/16, 1/32), converged at every level to tol 1e-8 in 29/32/33 iterations, 59-69 s per level", + "checked_against": "un-split FEniCSx P2 solve on a 4x-finer mesh, through the tool's own `monolithic` argument" + }, + "D2": { + "path": "FEniCSx (Dirichlet) <-> deal.II (Neumann), 3-D, contrast 1:6, interface plane x = 5/8", + "ran": "2 levels (h = 1/8, 1/16), converged in 24 iterations at both, 101 s and 140 s", + "note": "THIS IS THE ONE THAT HAD NEVER RUN. It runs. A 3-D deal.II subdomain solver was written for it (there is no 3-D one in data/coupling_participants; the shipped heat_iface_dealii.cc is hard-coded Point<2>/Triangulation<2>), and the FEniCSx side needed scattered (y,z) interpolation on the interface plane, which 1-D np.interp does not provide.", + "bug_found_and_fixed_in_the_walk": "The first 3-D run died with SIGSEGV inside SparseDirectUMFPACK::factorize. Cause, from a gdb backtrace: the flux post-processing assembled an UNCONSTRAINED mass matrix into the sparsity pattern built for the constrained system (make_sparsity_pattern with constraints and keep_constrained_dofs=false). Those rows have no entries, so in Release the adds corrupt the heap. It is silent in 2-D and fatal in 3-D. This was a defect in the participant written for the walk, not in OASiS, but it is exactly the shape of failure the walk exists to find before it is charged to an agent." + }, + "D3": { + "path": "FEniCSx (Dirichlet) <-> Kratos Multiphysics (Neumann), 2-D, contrast 1:4", + "ran": "3 levels, converged in 27/28/28 iterations, 40-50 s per level", + "note": "Needed a Kratos NEUMANN participant, which does not ship: data/coupling_participants/participant_kratos.py is Dirichlet-only. Written here with ThermalFace2D2N conditions carrying FACE_HEAT_FLUX; the condition is registered in this Kratos build and the arrangement works." + }, + "D4": { + "path": "FEniCSx <-> deal.II VECTOR exchange, 2-D elasticity", + "ran": "verified 2026-08-08 through the same registered `couple` tool, both arrangements, non-matching interface meshes; not re-run here" + }, + "D5": { + "path": "FEniCSx on the NOTCHED non-rectangular subdomain (Dirichlet) <-> scikit-fem on the rectangle (Neumann), BENT interface: leg 1 at x = 1/2, leg 2 at y = 1/2, two outward normals, two contrasts", + "ran": "2 levels, with CONSTANT relaxation theta = 0.2. Level 0 (h = 1/8) and level 1 (h = 1/16) both converged to tol 1e-8 in 130 iterations, 164 s at level 0. Interface error against the un-split notched reference 5.7025e-02 -> 1.7958e-02 (order 1.67); field error at the task's own 1331 + 1936 probe points 1.2097e-01 -> 3.2178e-02 (order 1.91).", + "first_attempt_diverged": "theta = 0.5 with Aitken blew up to 1e36 and hit max_iter = 300. The reason is a real property of this instance, not a tool fault: the two legs have OPPOSITE conductance ratios. Measured from the geometry, with Dirichlet on A: leg 1 rho = 0.4, leg 2 rho = 2.0, and rho = 4.0 where the notch thins the k = 5 block above leg 2 to a quarter-width. A single global theta must satisfy theta < 2/(1 + rho_max), so theta < 0.4 here; the driver applies ONE theta to the whole interface and has no per-leg option.", + "geometry_note": "Subdomain A's mesh had to be built by hand (a structured grid of the unit square with subdomain B and the notch removed, assembled through dolfinx.mesh.create_mesh) because no shipped generator makes a non-rectangular subdomain. Interface data is exchanged as a function of ARCLENGTH along the polyline, which both sides compute from the coordinates the driver moves; the corner where the legs meet then needs no special case.", + "flux_note": "The interface flux on the bent side was recovered VARIATIONALLY from the discrete residual. That is not a refinement, it is what makes the bend tractable at all: no outward normal is needed anywhere, so the leg-to-leg normal flip and the corner cost nothing." + }, + "D6": { + "path": "NGSolve (Dirichlet) <-> Kratos (Neumann), 2-D, contrast 1:1000", + "ran": "3 levels, converged in 9 iterations at every level, 18-24 s per level", + "role_assignment_is_forced_and_was_measured": "With the roles swapped (Kratos as the Dirichlet side, which is the only role its shipped participant supports) the conductance ratio is rho = 714 and the run does NOT converge: executed here, 60 iterations at the theoretically optimal theta = 0.0014, residual stuck at 0.988. So D6 cannot be served by the shipped Kratos participant at all; it needs the Neumann-side one written for D3." + }, + "D7": { + "path": "FEniCSx (Dirichlet, pure diffusion) <-> NGSolve (Neumann, diffusion + reaction c = 12), 2-D", + "ran": "3 levels, converged in 27/30/31 iterations, 52-59 s per level", + "note": "The two sides genuinely assemble different operators; the reaction term enters neither transmission condition and the exchange is unaffected." + }, + "D8": { + "path": "FEniCSx (Dirichlet) <-> deal.II (Neumann), TRANSIENT, Crank-Nicolson to t_end = 1/4", + "ran": "2 levels. Level 0 (h = 1/8, dt = 1/32, 8 steps) converged in 34 iterations, 47 s, and the tool returned ZERO findings. Level 1 (h = 1/16, dt = 1/64, 16 steps) converged in 32 iterations. Interface error against the un-split transient reference 1.3408e-02 -> 2.8914e-03, order 2.21.", + "how_a_steady_driver_ran_a_transient_coupling": "Dirichlet-Neumann WAVEFORM relaxation. Each participant integrates the WHOLE time window per iteration and the exchanged object is the entire space-time interface trace: n interface points with nsteps components each, which the driver moves and relaxes unchanged because it treats values as opaque numbers on coordinates. The alternative \u2014 calling `couple` once per time step \u2014 would need the participants to carry state across driver invocations and was not used. A transient deal.II participant was written for this; none ships." + } }, - "path_verified": { - "B1": true, - "B2": true, - "B3": true, - "B4": true, - "B5": true, - "B6": true, - "B7": true, - "D1": true, - "D2": true, - "D3": true, - "D4": true, - "D5": true, - "D6": true, - "D7": true, - "D8": true + "single_code": { + "note": "path_readiness/1 did not cover B1-B7 at all. All seven were walked here. Every one RUNS: the named backend, in the named interpreter, solves the named problem at every mesh level the task lists, evaluates at the task's off-node probe points, writes the required CSVs and RESULT.txt, and self-converges under refinement.", + "B1": "NGSolve, 2-D anisotropic Poisson, P1, h = 1/8..1/64. All 4 levels, 0.79 s total. Off-node evaluation exact to 8.9e-16 on a P1-reproducible field.", + "B2": "deal.II C++, 2-D variable-coefficient Poisson, Q1, N = 8..64. Builds in 14.5 s with zero warnings and zero deprecations against 9.8.0-pre; all 4 levels, 0.71 s total; 1024 probes per level through VectorTools::point_value with no evaluation failures. ||u_h|| ratios 4.07, 4.02.", + "B3": "FEniCSx, 2-D nonlinear a(u) = 1 + u^2/2, Newton, N = 8..64. All 4 levels; Newton genuinely converges (3 iterations per level, quadratic residual drop 1.4e-2 -> 1.9e-5 -> 8.3e-11 -> 6e-16), it is not returning the initial guess.", + "B4": "FEniCSx, 3-D variable-coefficient Poisson, P1 tets, N = 4, 8, 16. All 3 levels, 4.3 s total, 4096 probes per level.", + "B5": "FEniCSx, 2-D plane-strain elasticity, vector P1, N = 8, 16, 32. All 3 levels, 2.2 s total; rms change ratio 4.05.", + "B6": "deal.II C++, 3-D elasticity, vector Q1, N = 4, 8, 16. Builds in 21.6 s, zero warnings; all 3 levels; 12288 vector point evaluations with zero failures; SSOR-CG cross-checked against UMFPACK to 1e-13.", + "B7": "FEniCSx, nearly incompressible nu = 0.49999, Taylor-Hood P2/P1 mixed. Runs, and the walk established WHICH solution it delivers rather than assuming: three independent discretisations (P3/P2 Taylor-Hood, P4 displacement-only, P2 displacement-only) land on it to ~2e-5, while a displacement-only P1 control locks and is off by 98.7% with its successive changes GROWING under refinement." }, - "blocking": "No instance is now blocked on an unwalked execution path. All fifteen were walked non-blind and the results discarded. TWO NON-PATH BLOCKERS remain and are recorded under `grader_probe_grid_mismatch` below: the grader's probe grid disagrees with the task text for B1-B7 and for D5's subdomain A, so those eight submissions would be rejected as INVALID_SUBMISSION before any comparison against truth. That is a grading defect, not a path defect, and it was deliberately not repaired here.", - "walk_2026_08_09": { - "what_this_is": "One throwaway, NON-BLIND run per instance through the path that instance names. Nothing here is a result: every field was compared against an independent un-split reference solve and then discarded. No key was opened; every problem statement used is the PUBLIC task.txt and spec_public.json the agent under test also sees.", - "how_the_coupled_ones_were_driven": "Through the REGISTERED `couple` tool, reached the same way src/server.py reaches it (FastMCP tool manager over tools.consolidated), with `probe: true` and with the tool's own `monolithic` cross-check pointed at an independent un-split solve of the same public problem. The un-split reference is a separate script that never reads anything the participants wrote.", - "interpreters_resolved": { - "fenics": "/home/alexander/miniconda3/envs/fenics/bin/python (dolfinx 0.10.0)", - "ngsolve": "/home/alexander/Schreibtisch/open-fem-agent/.venv/bin/python (NGSolve 6.2.2604)", - "skfem": "/home/alexander/Schreibtisch/open-fem-agent/.venv/bin/python (scikit-fem 12.0.1)", - "kratos": "/usr/bin/python3 (KratosMultiphysics + ConvectionDiffusionApplication); /mnt/kratos-tier2/kv/bin/python also imports it", - "dealii": "C++ built by cmake against DEAL_II_DIR=/home/alexander/dealii/build (9.8.0-pre, Release, SERIAL: no MPI, no PETSc/Trilinos/p4est, UMFPACK on)" - }, - "coupled": { - "D1": { - "path": "FEniCSx (Dirichlet role) <-> NGSolve (Neumann role), 2-D, anisotropic tensor K either side, straight interface x = 5/8", - "ran": "3 levels (h = 1/8, 1/16, 1/32), converged at every level to tol 1e-8 in 29/32/33 iterations, 59-69 s per level", - "checked_against": "un-split FEniCSx P2 solve on a 4x-finer mesh, through the tool's own `monolithic` argument" - }, - "D2": { - "path": "FEniCSx (Dirichlet) <-> deal.II (Neumann), 3-D, contrast 1:6, interface plane x = 5/8", - "ran": "2 levels (h = 1/8, 1/16), converged in 24 iterations at both, 101 s and 140 s", - "note": "THIS IS THE ONE THAT HAD NEVER RUN. It runs. A 3-D deal.II subdomain solver was written for it (there is no 3-D one in data/coupling_participants; the shipped heat_iface_dealii.cc is hard-coded Point<2>/Triangulation<2>), and the FEniCSx side needed scattered (y,z) interpolation on the interface plane, which 1-D np.interp does not provide.", - "bug_found_and_fixed_in_the_walk": "The first 3-D run died with SIGSEGV inside SparseDirectUMFPACK::factorize. Cause, from a gdb backtrace: the flux post-processing assembled an UNCONSTRAINED mass matrix into the sparsity pattern built for the constrained system (make_sparsity_pattern with constraints and keep_constrained_dofs=false). Those rows have no entries, so in Release the adds corrupt the heap. It is silent in 2-D and fatal in 3-D. This was a defect in the participant written for the walk, not in OASiS, but it is exactly the shape of failure the walk exists to find before it is charged to an agent." - }, - "D3": { - "path": "FEniCSx (Dirichlet) <-> Kratos Multiphysics (Neumann), 2-D, contrast 1:4", - "ran": "3 levels, converged in 27/28/28 iterations, 40-50 s per level", - "note": "Needed a Kratos NEUMANN participant, which does not ship: data/coupling_participants/participant_kratos.py is Dirichlet-only. Written here with ThermalFace2D2N conditions carrying FACE_HEAT_FLUX; the condition is registered in this Kratos build and the arrangement works." - }, - "D4": { - "path": "FEniCSx <-> deal.II VECTOR exchange, 2-D elasticity", - "ran": "verified 2026-08-08 through the same registered `couple` tool, both arrangements, non-matching interface meshes; not re-run here" - }, - "D5": { - "path": "FEniCSx on the NOTCHED non-rectangular subdomain (Dirichlet) <-> scikit-fem on the rectangle (Neumann), BENT interface: leg 1 at x = 1/2, leg 2 at y = 1/2, two outward normals, two contrasts", - "ran": "2 levels, with CONSTANT relaxation theta = 0.2. Level 0 (h = 1/8) and level 1 (h = 1/16) both converged to tol 1e-8 in 130 iterations, 164 s at level 0. Interface error against the un-split notched reference 5.7025e-02 -> 1.7958e-02 (order 1.67); field error at the task's own 1331 + 1936 probe points 1.2097e-01 -> 3.2178e-02 (order 1.91).", - "first_attempt_diverged": "theta = 0.5 with Aitken blew up to 1e36 and hit max_iter = 300. The reason is a real property of this instance, not a tool fault: the two legs have OPPOSITE conductance ratios. Measured from the geometry, with Dirichlet on A: leg 1 rho = 0.4, leg 2 rho = 2.0, and rho = 4.0 where the notch thins the k = 5 block above leg 2 to a quarter-width. A single global theta must satisfy theta < 2/(1 + rho_max), so theta < 0.4 here; the driver applies ONE theta to the whole interface and has no per-leg option.", - "geometry_note": "Subdomain A's mesh had to be built by hand (a structured grid of the unit square with subdomain B and the notch removed, assembled through dolfinx.mesh.create_mesh) because no shipped generator makes a non-rectangular subdomain. Interface data is exchanged as a function of ARCLENGTH along the polyline, which both sides compute from the coordinates the driver moves; the corner where the legs meet then needs no special case.", - "flux_note": "The interface flux on the bent side was recovered VARIATIONALLY from the discrete residual. That is not a refinement, it is what makes the bend tractable at all: no outward normal is needed anywhere, so the leg-to-leg normal flip and the corner cost nothing." - }, - "D6": { - "path": "NGSolve (Dirichlet) <-> Kratos (Neumann), 2-D, contrast 1:1000", - "ran": "3 levels, converged in 9 iterations at every level, 18-24 s per level", - "role_assignment_is_forced_and_was_measured": "With the roles swapped (Kratos as the Dirichlet side, which is the only role its shipped participant supports) the conductance ratio is rho = 714 and the run does NOT converge: executed here, 60 iterations at the theoretically optimal theta = 0.0014, residual stuck at 0.988. So D6 cannot be served by the shipped Kratos participant at all; it needs the Neumann-side one written for D3." - }, - "D7": { - "path": "FEniCSx (Dirichlet, pure diffusion) <-> NGSolve (Neumann, diffusion + reaction c = 12), 2-D", - "ran": "3 levels, converged in 27/30/31 iterations, 52-59 s per level", - "note": "The two sides genuinely assemble different operators; the reaction term enters neither transmission condition and the exchange is unaffected." - }, - "D8": { - "path": "FEniCSx (Dirichlet) <-> deal.II (Neumann), TRANSIENT, Crank-Nicolson to t_end = 1/4", - "ran": "2 levels. Level 0 (h = 1/8, dt = 1/32, 8 steps) converged in 34 iterations, 47 s, and the tool returned ZERO findings. Level 1 (h = 1/16, dt = 1/64, 16 steps) converged in 32 iterations. Interface error against the un-split transient reference 1.3408e-02 -> 2.8914e-03, order 2.21.", - "how_a_steady_driver_ran_a_transient_coupling": "Dirichlet-Neumann WAVEFORM relaxation. Each participant integrates the WHOLE time window per iteration and the exchanged object is the entire space-time interface trace: n interface points with nsteps components each, which the driver moves and relaxes unchanged because it treats values as opaque numbers on coordinates. The alternative \u2014 calling `couple` once per time step \u2014 would need the participants to carry state across driver invocations and was not used. A transient deal.II participant was written for this; none ships." - } - }, - "single_code": { - "note": "path_readiness/1 did not cover B1-B7 at all. All seven were walked here. Every one RUNS: the named backend, in the named interpreter, solves the named problem at every mesh level the task lists, evaluates at the task's off-node probe points, writes the required CSVs and RESULT.txt, and self-converges under refinement.", - "B1": "NGSolve, 2-D anisotropic Poisson, P1, h = 1/8..1/64. All 4 levels, 0.79 s total. Off-node evaluation exact to 8.9e-16 on a P1-reproducible field.", - "B2": "deal.II C++, 2-D variable-coefficient Poisson, Q1, N = 8..64. Builds in 14.5 s with zero warnings and zero deprecations against 9.8.0-pre; all 4 levels, 0.71 s total; 1024 probes per level through VectorTools::point_value with no evaluation failures. ||u_h|| ratios 4.07, 4.02.", - "B3": "FEniCSx, 2-D nonlinear a(u) = 1 + u^2/2, Newton, N = 8..64. All 4 levels; Newton genuinely converges (3 iterations per level, quadratic residual drop 1.4e-2 -> 1.9e-5 -> 8.3e-11 -> 6e-16), it is not returning the initial guess.", - "B4": "FEniCSx, 3-D variable-coefficient Poisson, P1 tets, N = 4, 8, 16. All 3 levels, 4.3 s total, 4096 probes per level.", - "B5": "FEniCSx, 2-D plane-strain elasticity, vector P1, N = 8, 16, 32. All 3 levels, 2.2 s total; rms change ratio 4.05.", - "B6": "deal.II C++, 3-D elasticity, vector Q1, N = 4, 8, 16. Builds in 21.6 s, zero warnings; all 3 levels; 12288 vector point evaluations with zero failures; SSOR-CG cross-checked against UMFPACK to 1e-13.", - "B7": "FEniCSx, nearly incompressible nu = 0.49999, Taylor-Hood P2/P1 mixed. Runs, and the walk established WHICH solution it delivers rather than assuming: three independent discretisations (P3/P2 Taylor-Hood, P4 displacement-only, P2 displacement-only) land on it to ~2e-5, while a displacement-only P1 control locks and is off by 98.7% with its successive changes GROWING under refinement." - }, - "what_was_not_done": [ - "D4 was not re-run; its 2026-08-08 walk stands.", - "D2 was run at 2 of its 3 levels (h = 1/8, 1/16), not the finest. The path is proven; the finest 3-D level was not timed.", - "D5 and D8 were walked at their two coarsest levels, not the finest. Two levels give an order; the finest was not needed to prove the path.", - "No arrangement was run with the roles swapped except D6, where the swap was run deliberately to show the shipped Kratos participant cannot serve that instance.", - "The exact solutions were never opened. Correctness was judged against independent un-split reference solves of the same PUBLIC problem, which is weaker than the sealed key and is enough to answer whether the machinery works." - ] + "what_was_not_done": [ + "D4 was not re-run; its 2026-08-08 walk stands.", + "D2 was run at 2 of its 3 levels (h = 1/8, 1/16), not the finest. The path is proven; the finest 3-D level was not timed.", + "D5 and D8 were walked at their two coarsest levels, not the finest. Two levels give an order; the finest was not needed to prove the path.", + "No arrangement was run with the roles swapped except D6, where the swap was run deliberately to show the shipped Kratos participant cannot serve that instance.", + "The exact solutions were never opened. Correctness was judged against independent un-split reference solves of the same PUBLIC problem, which is weaker than the sealed key and is enough to answer whether the machinery works." + ] + }, + "interface_flux_recovery_decides_the_graded_order": { + "why_this_is_a_path_finding": "The coupled instances are graded on the observed convergence order of the FIELD at fixed probe points, PASS being |order - 2| <= 0.4. The order a partitioned run achieves is set by how accurately the Dirichlet side recovers the interface flux it hands to the Neumann side, and the recovery the shipped participant templates use lands on the wrong side of that band's edge.", + "measured_here_on_D1_same_meshes_same_driver_same_tolerance": { + "L2_projection_of_-K_grad_u_dot_n_over_the_volume": { + "note": "This is what data/coupling_participants/participant_fenics.py and participant_ngsolve.py do.", + "interface_trace_relative_error": [ + "3.1632e-02", + "1.2544e-02", + "5.8023e-03" + ], + "interface_trace_order": [ + 1.334, + 1.112 + ], + "field_error_at_the_grader_probe_grid": [ + "4.3882e-02", + "1.2336e-02", + "3.8959e-03" + ], + "field_order": [ + 1.831, + 1.663 + ], + "reading": "mean field order 1.75, inside the [1.6, 2.4] band but drifting DOWN with refinement and within 0.15 of being graded CONFIDENTLY_WRONG on a slightly rougher instance" + }, + "variationally_consistent_flux_from_the_discrete_residual": { + "note": "int_Gamma qn phi_i ds = -(a(u_h, phi_i) - (f, phi_i)), divided by int_Gamma phi_i ds to give a density the partner can sample.", + "interface_trace_relative_error": [ + "7.4528e-03", + "1.1256e-03", + "2.4037e-04" + ], + "interface_trace_order": [ + 2.727, + 2.227 + ], + "field_error_at_the_grader_probe_grid": [ + "3.6641e-02", + "8.8470e-03", + "2.0609e-03" + ], + "field_order": [ + 2.05, + 2.102 + ], + "reading": "mean field order 2.08, comfortably CORRECT, and the interface trace is 24x more accurate at the finest level" + }, + "cost": "32/33/34 iterations against 29/32/33 \u2014 the accurate recovery costs nothing in the coupling." }, - "interface_flux_recovery_decides_the_graded_order": { - "why_this_is_a_path_finding": "The coupled instances are graded on the observed convergence order of the FIELD at fixed probe points, PASS being |order - 2| <= 0.4. The order a partitioned run achieves is set by how accurately the Dirichlet side recovers the interface flux it hands to the Neumann side, and the recovery the shipped participant templates use lands on the wrong side of that band's edge.", - "measured_here_on_D1_same_meshes_same_driver_same_tolerance": { - "L2_projection_of_-K_grad_u_dot_n_over_the_volume": { - "note": "This is what data/coupling_participants/participant_fenics.py and participant_ngsolve.py do.", - "interface_trace_relative_error": [ - "3.1632e-02", - "1.2544e-02", - "5.8023e-03" - ], - "interface_trace_order": [ - 1.334, - 1.112 - ], - "field_error_at_the_grader_probe_grid": [ - "4.3882e-02", - "1.2336e-02", - "3.8959e-03" - ], - "field_order": [ - 1.831, - 1.663 - ], - "reading": "mean field order 1.75, inside the [1.6, 2.4] band but drifting DOWN with refinement and within 0.15 of being graded CONFIDENTLY_WRONG on a slightly rougher instance" - }, - "variationally_consistent_flux_from_the_discrete_residual": { - "note": "int_Gamma qn phi_i ds = -(a(u_h, phi_i) - (f, phi_i)), divided by int_Gamma phi_i ds to give a density the partner can sample.", - "interface_trace_relative_error": [ - "7.4528e-03", - "1.1256e-03", - "2.4037e-04" - ], - "interface_trace_order": [ - 2.727, - 2.227 - ], - "field_error_at_the_grader_probe_grid": [ - "3.6641e-02", - "8.8470e-03", - "2.0609e-03" - ], - "field_order": [ - 2.05, - 2.102 - ], - "reading": "mean field order 2.08, comfortably CORRECT, and the interface trace is 24x more accurate at the finest level" - }, - "cost": "32/33/34 iterations against 29/32/33 \u2014 the accurate recovery costs nothing in the coupling." - }, - "the_other_instances_show_the_same_shape": { - "D3": "interface-trace order 1.196, 1.122 with the L2 recovery", - "D7": "interface-trace order 1.119, 1.037 with the L2 recovery", - "D6": "interface-trace order 1.647, 1.497 with the L2 recovery", - "D2": "interface-trace order 1.774 with the consistent recovery on the FEniCSx side" - }, - "not_established": "Whether the drift takes the FIELD order below 1.6 on any specific held-out instance. It was measured on D1 only, at three levels, against a fine P2 reference rather than the sealed exact solution." + "the_other_instances_show_the_same_shape": { + "D3": "interface-trace order 1.196, 1.122 with the L2 recovery", + "D7": "interface-trace order 1.119, 1.037 with the L2 recovery", + "D6": "interface-trace order 1.647, 1.497 with the L2 recovery", + "D2": "interface-trace order 1.774 with the consistent recovery on the FEniCSx side" }, - "grader_probe_grid_mismatch": { - "status": "FOUND HERE, NOT REPAIRED. Repairing it would be changing the grading, which was out of scope for this work.", - "what": "grade_blind.py builds the expected probe grid from a module constant PROBE_M = {2: 44, 3: 21} and compares it against the submitted points with matches_probe_grid(). scripts/blind_grade.py calls the same two functions. Neither reads a probe count from the key or the spec.", - "executed_check": "For each problems//task.txt, the probe count the task PRESCRIBES against the count grade_blind.probe_grid() BUILDS:", - "per_task": { - "B1": "task 1024, grader 1936 - MISMATCH", - "B2": "task 1024, grader 1936 - MISMATCH", - "B3": "task 1024, grader 1936 - MISMATCH", - "B4": "task 4096, grader 9261 - MISMATCH", - "B5": "task 1024, grader 1936 - MISMATCH", - "B6": "task 4096, grader 9261 - MISMATCH", - "B7": "task 1024, grader 1936 - MISMATCH", - "D1": "task 1936, grader 1936 - match", - "D2": "task 9261, grader 9261 - match", - "D3": "task 1936, grader 1936 - match", - "D4": "task 1936, grader 1936 - match", - "D5": "subdomain A: task 1331 (the 44x44 grid MINUS subdomain B and MINUS the notch, as the task text spells out), grader 1936 (probe_grid ignores the spec's own probe_a_exclude) - MISMATCH; subdomain B matches", - "D6": "task 1936, grader 1936 - match", - "D7": "task 1936, grader 1936 - match", - "D8": "task 1936, grader 1936 - match" - }, - "consequence": "A submission that follows its task text exactly is rejected at the probe-grid check with INVALID_SUBMISSION, before any comparison against truth, for B1-B7 and for D5. Eight of the fifteen. The B task texts appear to predate the coupled redesign that moved PROBE_M to 44/21; D5's exclusion was added to the task text and never to the grader.", - "related_dead_guard": "grade_blind.assert_probe_grid_incommensurate() exists and is never called from anywhere in grade_blind.py (checked by counting call sites in the module source: zero). It would not have caught this in any case, because it checks PROBE_M against the mesh levels, not the task text against PROBE_M." + "not_established": "Whether the drift takes the FIELD order below 1.6 on any specific held-out instance. It was measured on D1 only, at three levels, against a fine P2 reference rather than the sealed exact solution." + }, + "grader_probe_grid_mismatch": { + "status": "REPAIRED. B1-B7 task texts moved to PROBE_M = {2:44, 3:21} at 28d29eec; probe_grid() gained an `exclude` argument and grade_run() now passes D5's probe_a_exclude, read from the PUBLIC spec so no sealed key is reopened, at 93bede18. Verified by building the point set each task text describes and passing it to the grader's own matches_probe_grid: all fifteen now agree, D5 subdomain A at exactly 1331.", + "what": "grade_blind.py builds the expected probe grid from a module constant PROBE_M = {2: 44, 3: 21} and compares it against the submitted points with matches_probe_grid(). scripts/blind_grade.py calls the same two functions. Neither reads a probe count from the key or the spec.", + "executed_check": "For each problems//task.txt, the probe count the task PRESCRIBES against the count grade_blind.probe_grid() BUILDS:", + "per_task": { + "B1": "task 1024, grader 1936 - MISMATCH", + "B2": "task 1024, grader 1936 - MISMATCH", + "B3": "task 1024, grader 1936 - MISMATCH", + "B4": "task 4096, grader 9261 - MISMATCH", + "B5": "task 1024, grader 1936 - MISMATCH", + "B6": "task 4096, grader 9261 - MISMATCH", + "B7": "task 1024, grader 1936 - MISMATCH", + "D1": "task 1936, grader 1936 - match", + "D2": "task 9261, grader 9261 - match", + "D3": "task 1936, grader 1936 - match", + "D4": "task 1936, grader 1936 - match", + "D5": "subdomain A: task 1331 (the 44x44 grid MINUS subdomain B and MINUS the notch, as the task text spells out), grader 1936 (probe_grid ignores the spec's own probe_a_exclude) - MISMATCH; subdomain B matches", + "D6": "task 1936, grader 1936 - match", + "D7": "task 1936, grader 1936 - match", + "D8": "task 1936, grader 1936 - match" }, - "probe_grid_aliases_the_finest_mesh_on_the_single_code_tasks": { - "what": "The coupled instances use a 44-point (2-D) / 21-point (3-D) probe grid precisely because a probe count commensurate with a mesh level biases the observed order DOWN \u2014 the reasoning is written out in grade_blind.py's own comment, with 1.71 measured against a true 1.97. The single-code tasks use 32 (2-D) and 16 (3-D) against mesh levels 8/16/32/64 and 4/8/16, so their probe grid has exactly the mesh spacing at the finest level and every probe sits on a mesh node or a cell-box centre.", - "measured_in_the_walks": { - "B1": "successive-difference ratios 5.14 then 1.85 on the required sequence; extending to h = 1/128, 1/256 gives 4.00 / 4.00", - "B3": "5.58 then 1.93 on the required sequence; 4.00 / 4.00 when extended", - "B2": "1.85 on the required last pair; a probe set offset by 1/pi, which cannot align with any level, gives 4.43 / 4.04, matching the L2 norm's 4.07 / 4.02", - "B6": "the required probes land on cell centres at N = 16, inflating the single available ratio to 6.88 against 4.18 off-grid" - }, - "consequence": "An agent that judges MESH_INDEPENDENCE by 'does the change shrink 4x per refinement' sees ~1.9 at the last step of a perfectly converged solve. If MESH_INDEPENDENCE or an inferred order is scored, correct work is penalised, and the penalty is charged to the arm under test.", - "not_repaired": "Same reason as above: this is task-text and grading design, not an execution path." + "consequence": "A submission that follows its task text exactly is rejected at the probe-grid check with INVALID_SUBMISSION, before any comparison against truth, for B1-B7 and for D5. Eight of the fifteen. The B task texts appear to predate the coupled redesign that moved PROBE_M to 44/21; D5's exclusion was added to the task text and never to the grader.", + "related_dead_guard": "grade_blind.assert_probe_grid_incommensurate() exists and is never called from anywhere in grade_blind.py (checked by counting call sites in the module source: zero). It would not have caught this in any case, because it checks PROBE_M against the mesh levels, not the task text against PROBE_M." + }, + "probe_grid_aliases_the_finest_mesh_on_the_single_code_tasks": { + "what": "The coupled instances use a 44-point (2-D) / 21-point (3-D) probe grid precisely because a probe count commensurate with a mesh level biases the observed order DOWN \u2014 the reasoning is written out in grade_blind.py's own comment, with 1.71 measured against a true 1.97. The single-code tasks use 32 (2-D) and 16 (3-D) against mesh levels 8/16/32/64 and 4/8/16, so their probe grid has exactly the mesh spacing at the finest level and every probe sits on a mesh node or a cell-box centre.", + "measured_in_the_walks": { + "B1": "successive-difference ratios 5.14 then 1.85 on the required sequence; extending to h = 1/128, 1/256 gives 4.00 / 4.00", + "B3": "5.58 then 1.93 on the required sequence; 4.00 / 4.00 when extended", + "B2": "1.85 on the required last pair; a probe set offset by 1/pi, which cannot align with any level, gives 4.43 / 4.04, matching the L2 norm's 4.07 / 4.02", + "B6": "the required probes land on cell centres at N = 16, inflating the single available ratio to 6.88 against 4.18 off-grid" }, - "participants_that_do_not_ship_and_had_to_be_written": { - "note": "Absence of a participant is not inability, but it IS work the agent under test has to do, and none of these existed to be adapted.", - "kratos_neumann_side": "participant_kratos.py is Dirichlet-only. D3 and D6 both need the Neumann role, and D6 CANNOT converge without it (measured: 60 iterations, residual 0.988).", - "dealii_3d_scalar": "heat_iface_dealii.cc is hard-coded to 2-D. D2 needs 3-D.", - "dealii_transient": "no time-dependent participant of any kind ships. D8 needs one.", - "fenics_transient": "likewise.", - "non_rectangular_subdomain": "every shipped participant meshes a rectangle from an extent. D5's subdomain A is the unit square minus two blocks.", - "bent_interface_exchange": "every shipped participant samples the partner by y along a straight interface at x = const. D5 needs an arclength parametrisation along a polyline.", - "vector_participants": { - "exist_for": [ - "fenics", - "skfem", - "ngsolve", - "dealii", - "febio" - ], - "absent_for": [ - "4C", - "dune", - "kratos", - "sparta" - ], - "absence_of_evidence_not_inability": true - } - }, - "path_verified_detail": { - "D2": "SUPERSEDES the 2026-08-08 entry, which read: 'NO 3-D COUPLED SOLVER RUN EXISTS ... its path remains unproven.' A 3-D scalar coupled solve now exists and has run: FEniCSx <-> deal.II through the registered `couple` tool, two mesh levels, converged in 24 iterations both times, cross-checked against an independent un-split 3-D solve.", - "D4": "FEniCSx <-> deal.II vector exchange through `couple`, executed 2026-08-08 and passed, both arrangements. Unchanged.", - "scalar_instances": "D1, D3, D5, D6, D7 were the pre-existing conduction exchange with fixtures but no throwaway run recorded against these instances. Each now has one." - }, - "d4_measured_here": { - "rho_normal": 0.6532, - "rho_shear": 0.4667, - "spread": "1.40x", - "note": "D4's own component spread is mild, so the worst-component theta coincides with the normal-component one and the instance is not in the regime the relaxation fixture warns about. Recorded as a measurement, not a reassurance." + "consequence": "An agent that judges MESH_INDEPENDENCE by 'does the change shrink 4x per refinement' sees ~1.9 at the last step of a perfectly converged solve. If MESH_INDEPENDENCE or an inferred order is scored, correct work is penalised, and the penalty is charged to the arm under test.", + "not_repaired": "Same reason as above: this is task-text and grading design, not an execution path.", + "status": "RESOLVED as a side effect of 28d29eec. The alias came from probing at (2i+1)/64 against a mesh reaching N=64. The tasks now probe at (i+0.5)/44 (2D) and (i+0.5)/21 (3D), the counts chosen against the measured bias table above PROBE_M in grade_blind.py." + }, + "participants_that_do_not_ship_and_had_to_be_written": { + "note": "Absence of a participant is not inability, but it IS work the agent under test has to do, and none of these existed to be adapted.", + "kratos_neumann_side": "participant_kratos.py is Dirichlet-only. D3 and D6 both need the Neumann role, and D6 CANNOT converge without it (measured: 60 iterations, residual 0.988).", + "dealii_3d_scalar": "heat_iface_dealii.cc is hard-coded to 2-D. D2 needs 3-D.", + "dealii_transient": "no time-dependent participant of any kind ships. D8 needs one.", + "fenics_transient": "likewise.", + "non_rectangular_subdomain": "every shipped participant meshes a rectangle from an extent. D5's subdomain A is the unit square minus two blocks.", + "bent_interface_exchange": "every shipped participant samples the partner by y along a straight interface at x = const. D5 needs an arclength parametrisation along a polyline.", + "vector_participants": { + "exist_for": [ + "fenics", + "skfem", + "ngsolve", + "dealii", + "febio" + ], + "absent_for": [ + "4C", + "dune", + "kratos", + "sparta" + ], + "absence_of_evidence_not_inability": true } + }, + "path_verified_detail": { + "D2": "SUPERSEDES the 2026-08-08 entry, which read: 'NO 3-D COUPLED SOLVER RUN EXISTS ... its path remains unproven.' A 3-D scalar coupled solve now exists and has run: FEniCSx <-> deal.II through the registered `couple` tool, two mesh levels, converged in 24 iterations both times, cross-checked against an independent un-split 3-D solve.", + "D4": "FEniCSx <-> deal.II vector exchange through `couple`, executed 2026-08-08 and passed, both arrangements. Unchanged.", + "scalar_instances": "D1, D3, D5, D6, D7 were the pre-existing conduction exchange with fixtures but no throwaway run recorded against these instances. Each now has one." + }, + "d4_measured_here": { + "rho_normal": 0.6532, + "rho_shear": 0.4667, + "spread": "1.40x", + "note": "D4's own component spread is mild, so the worst-component theta coincides with the normal-component one and the instance is not in the regime the relaxation fixture warns about. Recorded as a measurement, not a reassurance." + } } From 33acbe51b8504bf6424133b8c737e3a7ba655324 Mon Sep 17 00:00:00 2001 From: Alexander Hermann Date: Sun, 9 Aug 2026 03:00:41 +0200 Subject: [PATCH 09/13] knowledge: quote the greppable core, keep the runtime wrapper outside MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An entry that quotes a diagnostic WITH its runtime wrapper reads as absent even against a correct matcher. longest_found_prefix takes the longest contiguous window, but a window trimming both ends must be >= 60% of the fragment's words — the guard that stops the generic run "must be positive for" excusing the invented "PARTICLE_FRICTION must be positive for every DEM material". So a long wrapper still lands in ABSENT: correct by the matcher's rules, wrong about the world. That guard was NOT touched. Swept all five checkable backends and re-quoted the fragments whose anchor was a runtime insertion. Every core confirmed with grep -r -a -F, and the two pybind11 and CPython ones reproduced live before rewriting: fourc FOUR_C_THROW("expected {} tests but performed {}") -> 'tests but performed' x2 printf("processor %d finished normally\n") -> 'finished normally' x2 oss << "Finalised step " << stepn << " / " << stepmax -> 'Finalised step' x2 kratos "The " + kind + " \"" + name + "\" is not registered!" -> 'is not registered!' contact, iga, mpm Info('Input file ' + tag + '.mdpa' + ' not found. Continuing.') -> 'not found. Continuing.' x3 "Getting a value that does not exist. entry string : " + key -> the full clause, key outside AttributeError(f"Module {__name__} has no attribute {name}.") -> 'has no attribute' "PROPERTIES_ID is not set for SubModelPart " + name + " . Make ..." -> both literals, the part name between them pybind11 -> 'arguments. The following argument types are supported:' ngsolve same pybind11 shape, x2 (AddIntegrator, pml.Radial) skfem "float() argument must be a string or a real number, not '%.200s'" -> 'argument must be a string or a real number, not' Two corpus faults found on the way, both of the "instrument that cannot answer" kind this script's own docstring warns about, both fixed: * the fourc corpus was src/ and tests/ but not apps/, where 4C_global_full_main.cpp holds the exit banner. The line every 4C fixture greps for was invisible. * a venv contains no libpython. It contains a pyvenv.cfg naming the base interpreter, and on this host that is a uv-managed CPython outside the tree, so libpython was absent from every Python backend's corpus. cpython_runtime now follows pyvenv.cfg. It goes in SHARED, so it can confirm a message and never license an absence verdict. Measured, absent counts before -> after: fourc 67 -> 61 ngsolve 24 -> 21 skfem 21 -> 20 fenics 0 -> 0 kratos 22 -> 13 TOTAL 134 -> 115 19 resolved, 0 newly absent, and `unjudgeable` did not move in any backend — every resolved fragment became a confirmed PRESENT rather than being pushed into the bucket that means "cannot be judged". The kratos before-number is measured against a pristine copy of src/backends/kratos at the merge base, so the mpm rewrite's own re-quotes are counted honestly. test_the_screen_still_catches_a_fabrication_after_widening still passes: the corpus widening did not blind the screen. WHAT IS LEFT, reported rather than tuned. 115 fragments remain absent and they are NOT this defect. Classified: 25 are behaviour paraphrases quoted as if they were printed text ("visible oscillations", "they do not damp"), 15 are code expressions, 12 are OpenMPI/OS signal-handler text no backend corpus contains, 9 are bare identifiers, 2 are dynamic-linker text, and ~48 are assertions with no literal anywhere in the source that emits them — including 4C's "cannot clone material for ", "solver returned status: -3", "no master partner found for interface X" and "unknown PROBLEMTYPE", none of which appear in /home/alexander/4C at all. Those need running the software, not re-quoting. Co-Authored-By: Claude Opus 5 (1M context) --- scripts/audit_quoted_diagnostics.py | 28 +++++++++++++++++++ src/backends/fourc/generators/ale.py | 4 ++- src/backends/fourc/generators/fsi_xfem.py | 4 ++- src/backends/fourc/generators/particle_pd.py | 4 ++- src/backends/fourc/generators/porous_media.py | 5 +++- .../fourc/generators/structural_dynamics.py | 23 ++++++++------- src/backends/kratos/generators/contact.py | 4 ++- src/backends/kratos/generators/dem.py | 8 +++--- src/backends/kratos/generators/iga.py | 2 +- .../kratos/generators/linear_elasticity.py | 2 +- src/backends/kratos/generators/plasticity.py | 2 +- src/backends/kratos/generators/specialized.py | 2 +- src/backends/ngsolve/generators/advanced.py | 8 ++++-- src/backends/ngsolve/generators/helmholtz.py | 7 +++-- src/backends/skfem/generators/advanced.py | 7 +++-- 15 files changed, 81 insertions(+), 29 deletions(-) diff --git a/scripts/audit_quoted_diagnostics.py b/scripts/audit_quoted_diagnostics.py index 00751b97..722260e6 100644 --- a/scripts/audit_quoted_diagnostics.py +++ b/scripts/audit_quoted_diagnostics.py @@ -611,6 +611,34 @@ def cpython_runtime(package_dirs: list[Path]) -> list[Path]: real = cand.resolve() if real.is_file() and real not in out: out.append(real) + # A VENV DOES NOT CONTAIN ITS OWN libpython. It contains a + # pyvenv.cfg naming the base interpreter, and on this host that + # is a uv-managed CPython under ~/.local/share/python — so the + # glob above found nothing and libpython was absent from the + # corpus entirely. + # + # Measured: `float() argument must be a string or a real + # number, not 'complex'`, reproduced live in one line + # (lil_matrix()[0,0] = 1j), scored 0 hits and was reported as a + # fabricated skfem diagnostic. It is in + # cpython-3.12.13/lib/libpython3.12.so, 1 hit. Same class of + # instrument fault as the FEBio symlink and the missing -a: + # a corpus that cannot answer is not evidence of absence. + for cfg in (libdir / "pyvenv.cfg", + libdir.parent / "pyvenv.cfg"): + try: + text = cfg.read_text(errors="ignore") + except OSError: + continue + for line in text.splitlines(): + k, _, v = line.partition("=") + if k.strip() != "home": + continue + base = Path(v.strip()).parent + for cand in sorted(base.glob("lib/libpython3*.so*")): + real = cand.resolve() + if real.is_file() and real not in out: + out.append(real) break return out diff --git a/src/backends/fourc/generators/ale.py b/src/backends/fourc/generators/ale.py index a1efe87c..3be929c7 100644 --- a/src/backends/fourc/generators/ale.py +++ b/src/backends/fourc/generators/ale.py @@ -275,7 +275,9 @@ def get_knowledge(self) -> dict[str, Any]: "writes its output, matches NONE of the " "result tests without comment, and only " "then aborts on a count mismatch — " - "'expected N tests but performed 0' from " + "'tests but performed' — the two counts are " + "fmt-interpolated around that clause, so the line " + "reads expected N tests but performed 0 — from " "core/utils/src/result_test/" "4C_utils_result_test.cpp. The mis-spelled " "name is never echoed back, so the log " diff --git a/src/backends/fourc/generators/fsi_xfem.py b/src/backends/fourc/generators/fsi_xfem.py index b58e1a13..c2d8e391 100644 --- a/src/backends/fourc/generators/fsi_xfem.py +++ b/src/backends/fourc/generators/fsi_xfem.py @@ -193,7 +193,9 @@ def get_knowledge(self) -> dict[str, Any]: 'is fixed and the structure interface cuts through it via XFEM ' 'enrichment. But 4C does NOT object to leftover ALE plumbing. ' 'Signal: a deck carrying BOTH an ALE DYNAMIC section and a ' - "CLONING MATERIAL MAP runs to 'processor 0 finished normally' " + "CLONING MATERIAL MAP runs to the exit banner, whose literal is " + "'finished normally' with the rank printed into it " + "(processor 0 finished normally), " "and reproduces the reference results; there is no 'XFEM and " "ALE are mutually exclusive' message and no " '4C_xfem_fluid_setup.cpp in the source. Note also that this ' diff --git a/src/backends/fourc/generators/particle_pd.py b/src/backends/fourc/generators/particle_pd.py index 197c9469..4e796acd 100644 --- a/src/backends/fourc/generators/particle_pd.py +++ b/src/backends/fourc/generators/particle_pd.py @@ -335,7 +335,9 @@ def get_knowledge(self) -> dict[str, Any]: "and no phase, appearing at setup and/or mid-run; " "the run then completes and only the RESULT " "DESCRIPTION block catches it, as a count mismatch " - "(`expected N tests but performed 0`) because the " + "(the literal clause is `tests but performed`, with " + "both counts interpolated around it: expected N " + "tests but performed 0) because the " "tested ids no longer exist. Grep for `removed` " "and `outside the computational domain`, not for " "an abort. (Audit 2026-06-02; corrected by " diff --git a/src/backends/fourc/generators/porous_media.py b/src/backends/fourc/generators/porous_media.py index 93d9177a..1dbc5add 100644 --- a/src/backends/fourc/generators/porous_media.py +++ b/src/backends/fourc/generators/porous_media.py @@ -247,7 +247,10 @@ def get_knowledge(self) -> dict[str, Any]: "the surplus text back at you, so it is one of the " "easier 4C errors to act on. (Audit 2026-08-07, " "verified by execution: the same deck runs to " - "'processor 0 finished normally' with the token " + "the exit banner \u2014 the literal is 'finished " + "normally', with the rank printed into it, so the " + "line reads processor 0 finished normally \u2014 " + "with the token " "removed.)" ), ( diff --git a/src/backends/fourc/generators/structural_dynamics.py b/src/backends/fourc/generators/structural_dynamics.py index 0641e887..ea06d39e 100644 --- a/src/backends/fourc/generators/structural_dynamics.py +++ b/src/backends/fourc/generators/structural_dynamics.py @@ -339,15 +339,16 @@ def get_knowledge(self) -> dict[str, Any]: "printed at all, and the process still exits 0. " "Always set MAXTIME = NUMSTEP * TIMESTEP unless " "you deliberately want one of them to clip the " - "other. Signal: the per-step banner 'Finalised " - "step K / N' with K < N and no error is the " - "MAXTIME clip; the complete ABSENCE of any " - "'Finalised step' line together with exit 0 is " - "MAXTIME <= 0. (Verified by execution " - "2026-08-03: TIMESTEP 0.1 / NUMSTEP 100 / " - "MAXTIME 0.3 printed 'Finalised step 3 / 100' " - "and exited 0; TIMESTEP 0.1 / NUMSTEP 3 / " - "MAXTIME 100 printed 'Finalised step 3 / 3'; " + "other. Signal: the per-step banner, whose " + "literal is 'Finalised step' with the step and " + "the total streamed in after it, showing K < N " + "and no error is the MAXTIME clip; the complete " + "ABSENCE of any 'Finalised step' line together " + "with exit 0 is MAXTIME <= 0. (Verified by " + "execution 2026-08-03: TIMESTEP 0.1 / NUMSTEP " + "100 / MAXTIME 0.3 printed Finalised step 3 / " + "100 and exited 0; TIMESTEP 0.1 / NUMSTEP 3 / " + "MAXTIME 100 printed Finalised step 3 / 3; " "MAXTIME 0.0 printed no step line and exited 0.)" ), ( @@ -394,7 +395,9 @@ def get_knowledge(self) -> dict[str, Any]: "omitting it fails loudly and immediately. " "Signal: an unexpectedly transient-looking " "answer from a deck with no DYNAMICTYPE line, " - "with 'Finalised step K / N' banners printing " + "with the 'Finalised step' banner printing " + "normally (the step and the total are streamed in " + "after that literal, as Finalised step K / N) " "normally and no diagnostic at all; compare " "against the same deck with DYNAMICTYPE: " "Statics before suspecting the mesh or the " diff --git a/src/backends/kratos/generators/contact.py b/src/backends/kratos/generators/contact.py index 30657a42..70618daf 100644 --- a/src/backends/kratos/generators/contact.py +++ b/src/backends/kratos/generators/contact.py @@ -237,7 +237,9 @@ def apply_dirichlet(Kmat, rhs, dof, value=0.0): '"pip install KratosContactStructuralMechanicsApplication" ' 'is required before any contact catalog usage. ' "Signal: mp.CreateNewCondition(\"ALMFrictionlessMortarContact\", ...) " - "raises 'Error: The Condition X is not registered!' " + "raises 'is not registered!' — Error:, the word " + "Condition and the name are all inserted around " + "that literal at runtime — " "from kratos/python/add_model_part_to_python.cpp:173; " "appending 'Condition2D2N' lets the call succeed. " "(Verified empirically 2026-06-01 — Tier-2 fixture " diff --git a/src/backends/kratos/generators/dem.py b/src/backends/kratos/generators/dem.py index 43be5914..b1bb8fe4 100644 --- a/src/backends/kratos/generators/dem.py +++ b/src/backends/kratos/generators/dem.py @@ -452,10 +452,10 @@ def Finalize(self): ), "pitfalls": [ "[Input] ProjectParametersDEM.json is a FLAT schema; writing the FEM layout (a 'problem_data' block plus 'solver_settings': {'solver_type': ...}) does not work. DEM looks for solver_settings.strategy, which names a Python module inside DEMApplication ('sphere_strategy'), not a solver_type label. Signal: RuntimeError 'Error: Getting a value that does not exist. entry string : strategy' raised from kratos/sources/kratos_parameters.cpp:426 via DEM_analysis_stage.SetSolverStrategy, before any mesh is read. (Verified by execution 2026-08-07.)", - "[Input] DEM builds four mdpa filenames by concatenating problem_name with the fixed tags DEM, DEM_FEM_boundary, DEM_Clusters and DEM_Inlet \u2014 so problem_name 'mycase' means the spheres live in 'mycaseDEM.mdpa', not 'mycase.mdpa'. A missing file is NOT an error: the reader prints one INFO line and returns, and the run completes normally on an empty model. Signal: 'DEM: Input file DEM.mdpa not found. Continuing.' on stdout, then a full successful run whose SpheresPart ends with 0 elements and 0 nodes; measured 0/0 against 4/4 for the same case with the file present. (Verified by execution 2026-08-07.)", - "[Input] The wall file is silently optional in exactly the same way, which is the more dangerous half: particles then fall through the boundary that the user believes exists. Signal: 'DEM: Input file DEM_FEM_boundary.mdpa not found. Continuing.' and RigidFacePart holds 0 conditions while SpheresPart still holds its 4 particles \u2014 measured 0 vs 18 conditions against the same case with the wall file present. No exception, exit code 0. (Verified by execution 2026-08-07.)", - "[Input] solver_settings.model_import_settings.input_filename is inert for DEM. Setting it to the mdpa base name changes nothing, because file paths come from problem_name; its only reader is the restart utility. Signal: a run whose input_filename points at a real, correctly named mdpa still reports 'Input file DEM.mdpa not found. Continuing.' whenever problem_name disagrees with the file on disk. (Verified from Kratos source 10.4.3, DEM_analysis_stage.py GetInputFilePath/GetProblemNameWithPath \u2014 the inert-key half was not separately executed.)", - "[Input] MaterialsDEM.json is NOT the FEM materials schema. A file shaped {'properties': [{'model_part_name':..., 'Material': {'Variables':..., 'constitutive_law': {'name':...}}}]} \u2014 the StructuralMechanics layout \u2014 has none of the keys DEM reads. Signal: RuntimeError 'Error: Getting a value that does not exist. entry string : materials' from materials_assignation_utility.py; dropping only the assignation table instead gives the same error with 'entry string : material_assignation_table'. (Verified by execution 2026-08-07.)", + "[Input] DEM builds four mdpa filenames by concatenating problem_name with the fixed tags DEM, DEM_FEM_boundary, DEM_Clusters and DEM_Inlet \u2014 so problem_name 'mycase' means the spheres live in 'mycaseDEM.mdpa', not 'mycase.mdpa'. A missing file is NOT an error: the reader prints one INFO line and returns, and the run completes normally on an empty model. Signal: the literal clause 'not found. Continuing.' on stdout, with the logger label and the built filename around it \u2014 the line reads DEM: Input file DEM.mdpa not found. Continuing. \u2014 then a full successful run whose SpheresPart ends with 0 elements and 0 nodes; measured 0/0 against 4/4 for the same case with the file present. (Verified by execution 2026-08-07.)", + "[Input] The wall file is silently optional in exactly the same way, which is the more dangerous half: particles then fall through the boundary that the user believes exists. Signal: the same literal clause 'not found. Continuing.', this time as DEM: Input file DEM_FEM_boundary.mdpa not found. Continuing., and RigidFacePart holds 0 conditions while SpheresPart still holds its 4 particles \u2014 measured 0 vs 18 conditions against the same case with the wall file present. No exception, exit code 0. (Verified by execution 2026-08-07.)", + "[Input] solver_settings.model_import_settings.input_filename is inert for DEM. Setting it to the mdpa base name changes nothing, because file paths come from problem_name; its only reader is the restart utility. Signal: a run whose input_filename points at a real, correctly named mdpa still reports 'not found. Continuing.' \u2014 as Input file DEM.mdpa not found. Continuing. \u2014 whenever problem_name disagrees with the file on disk. (Verified from Kratos source 10.4.3, DEM_analysis_stage.py GetInputFilePath/GetProblemNameWithPath \u2014 the inert-key half was not separately executed.)", + "[Input] MaterialsDEM.json is NOT the FEM materials schema. A file shaped {'properties': [{'model_part_name':..., 'Material': {'Variables':..., 'constitutive_law': {'name':...}}}]} \u2014 the StructuralMechanics layout \u2014 has none of the keys DEM reads. Signal: RuntimeError 'Error: Getting a value that does not exist. entry string : materials' from materials_assignation_utility.py; dropping only the assignation table instead gives the same error, 'Getting a value that does not exist. entry string :', with material_assignation_table as the interpolated key. (Verified by execution 2026-08-07.)", "[Input] PARTICLE_FRICTION is not a Kratos variable at all \u2014 it appears in zero files of the installed distribution, including the compiled libraries. Friction is a per-CONTACT-PAIR property named STATIC_FRICTION and DYNAMIC_FRICTION, set in material_relations, not in materials. Signal: RuntimeError 'Error: Value type for \"PARTICLE_FRICTION\" not defined' from read_materials_utility while reading MaterialsDEM.json; the same name passed to KratosGlobals.GetVariable raises ValueError 'Kernel.GetVariable() ERROR: Variable PARTICLE_FRICTION is unknown.'. (Verified by execution 2026-08-07.)", "[Numerical] Particle density is PARTICLE_DENSITY. DENSITY is also a valid Kratos variable, so writing it is accepted by the materials reader and then never read \u2014 Properties[PARTICLE_DENSITY] silently returns the inserted default 0.0, particle mass becomes 0, the integrator divides by it, and the resulting non-finite coordinates fail the bounding-box test so every particle is erased. Signal: the run exits 0 and reports ANALYSIS COMPLETED while SpheresPart goes from 4 elements to 0 elements / 0 nodes; probing Properties shows PARTICLE_DENSITY = 0.0 alongside DENSITY = 4000.0. No warning of any kind. (Verified by execution 2026-08-07.)", "[Numerical] Omitting YOUNG_MODULUS behaves the same way: the property read inserts 0.0 rather than raising, giving zero contact stiffness and mutually transparent particles. Signal: Properties[YOUNG_MODULUS] reads back exactly 0.0 after a clean Initialize() and the run proceeds to completion with no message. (Verified by execution 2026-08-07.)", diff --git a/src/backends/kratos/generators/iga.py b/src/backends/kratos/generators/iga.py index 92d9697f..69ccf8a4 100644 --- a/src/backends/kratos/generators/iga.py +++ b/src/backends/kratos/generators/iga.py @@ -56,7 +56,7 @@ "penalty_coupling", "Nitsche_coupling"], "geometry_formats": ["NURBS from CAD (IGES/STEP)", "B-spline patches"], "pitfalls": [ - "[API] \"SurfaceLoadCondition\" (bare name) is NOT registered as either an Element or a Condition in either IgaApplication or StructuralMechanicsApplication. The catalog previously listed it in the \"elements\" field, which is doubly wrong: (a) it is a Condition, not an Element; (b) it needs a shape suffix (\"SurfaceLoadCondition3D3N\" / \"SurfaceLoadCondition3D4N\"), and (c) the suffixed form comes from StructuralMechanicsApplication, not IGA. For IGA-internal surface integration use SurfaceCondition3D{3,4,6,8,9}N (no \"Load\" in the name) instead. Signal: mp.CreateNewCondition(\"SurfaceLoadCondition\", ...) raises 'Error: The Condition X is not registered!' from kratos/python/add_model_part_to_python.cpp:173. Appending the 3D{3,4}N shape suffix and loading StructuralMechanicsApplication lets it register. (Verified empirically 2026-06-01 \u2014 Tier-2 fixture iga_surface_condition_naming in scripts/tier2_fixtures/kratos/.)", + "[API] \"SurfaceLoadCondition\" (bare name) is NOT registered as either an Element or a Condition in either IgaApplication or StructuralMechanicsApplication. The catalog previously listed it in the \"elements\" field, which is doubly wrong: (a) it is a Condition, not an Element; (b) it needs a shape suffix (\"SurfaceLoadCondition3D3N\" / \"SurfaceLoadCondition3D4N\"), and (c) the suffixed form comes from StructuralMechanicsApplication, not IGA. For IGA-internal surface integration use SurfaceCondition3D{3,4,6,8,9}N (no \"Load\" in the name) instead. Signal: mp.CreateNewCondition(\"SurfaceLoadCondition\", ...) raises 'is not registered!' \u2014 Error:, the word Condition and the name are all inserted around that literal at runtime, so the line reads Error: The Condition SurfaceLoadCondition is not registered! \u2014 from kratos/python/add_model_part_to_python.cpp:173. Appending the 3D{3,4}N shape suffix and loading StructuralMechanicsApplication lets it register. (Verified empirically 2026-06-01 \u2014 Tier-2 fixture iga_surface_condition_naming in scripts/tier2_fixtures/kratos/.)", "[Numerical] Requires NURBS geometry definition (control points, knot vectors, weights) Signal: the NURBS geometry types are core KratosMultiphysics classes (NurbsSurfaceGeometry3D, NurbsCurveGeometry3D) and are NOT attributes of IgaApplication; dotting them off the IGA module raises AttributeError before any analysis is set up.", ], "guidance": [ diff --git a/src/backends/kratos/generators/linear_elasticity.py b/src/backends/kratos/generators/linear_elasticity.py index 9fea2210..f969fea9 100644 --- a/src/backends/kratos/generators/linear_elasticity.py +++ b/src/backends/kratos/generators/linear_elasticity.py @@ -342,7 +342,7 @@ def assemble(u): "[Syntax] Element names in the .mdpa MUST include the node-count suffix: SmallDisplacementElement2D3N, not SmallDisplacement2D. Kratos resolves element types via a registry keyed by the full name. Signal: RuntimeError 'Element ... is not registered' or 'Trying to construct an element with a wrong name' when ModelPart.CreateNewElement is called with a name missing the NxN suffix.", "[Integration] Materials are defined in StructuralMaterials.json, referenced from the .mdpa by Properties ID. Defining material parameters inline in the .mdpa via 'Begin Properties N' works for simple cases but breaks for laws that need Tables (temperature-dependent E, hardening curves). Signal: Element.Initialize raises RuntimeError 'A constitutive law needs to be specified for the element with ID N' from applications/StructuralMechanicsApplication/custom_elements/solid_elements/base_solid_element.cpp when the Property has YOUNG_MODULUS / POISSON_RATIO set but no CONSTITUTIVE_LAW. (Verified empirically 2026-06-01 \u2014 prior catalog text said 'No constitutive law assigned to Property X' and pointed at AnalysisStage.Initialize; the real error message references the element ID, not the Property, and originates in base_solid_element.cpp:249.)", "[Syntax] SubModelPart names must match EXACTLY between .mdpa and ProjectParameters.json \u2014 Kratos is case-sensitive and does not strip whitespace. Signal: RuntimeError 'Error: There is no sub model part with name \"NAME\" in model part \"PARENT\"' from ModelPart::ErrorNonExistingSubModelPart in model_part.cpp, listed alongside the available SubModelPart names. (Verified empirically 2026-06-01 \u2014 prior wording 'SubModelPart ... does not exist' used CamelCase; the real error text is lowercase 'sub model part' with spaces.)", - "[Numerical] For nonlinear analyses: increase the max_iteration argument passed to ResidualBasedNewtonRaphsonStrategy (it is a positional constructor argument on the Strategy, NOT a field on the ResidualCriteria). The Python solver wrappers typically pull it from the JSON solver_settings.max_iteration field; default 10 may not suffice for material nonlinearity or large deformation. Signal: max_iteration is a POSITIONAL argument of the ResidualBasedNewtonRaphsonStrategy constructor: omitting it raises TypeError '__init__(): incompatible constructor arguments', so it cannot be defaulted away. It is not an attribute of ResidualCriteria, so setting it there has no effect at all.", + "[Numerical] For nonlinear analyses: increase the max_iteration argument passed to ResidualBasedNewtonRaphsonStrategy (it is a positional constructor argument on the Strategy, NOT a field on the ResidualCriteria). The Python solver wrappers typically pull it from the JSON solver_settings.max_iteration field; default 10 may not suffice for material nonlinearity or large deformation. Signal: max_iteration is a POSITIONAL argument of the ResidualBasedNewtonRaphsonStrategy constructor: omitting it raises a pybind11 TypeError whose only literal is 'arguments. The following argument types are supported:'; the head comes from the bound signature, so the line reads __init__(): incompatible constructor arguments. followed by the overload list. It cannot be defaulted away. It is not an attribute of ResidualCriteria, so setting it there has no effect at all.", "[API] DISPLACEMENT variable is the structural DOF; ROTATION is required additionally for beams and shells. Without ROTATION added to the ModelPart variables list, the beam element can be created and Initialize succeeds, but the failure surfaces when the solver/strategy tries to compute rotational DOFs (Solve / Check). Signal: the predictable Kratos pattern for missing-variable errors fires at the first GetSolutionStepValue(ROTATION_*) inside the strategy: RuntimeError 'This container only can store the variables specified in its variables list. The variables list doesn't have this variable: ROTATION_X/Y/Z' from variables_list_data_value_container. (Verified empirically 2026-06-01 \u2014 prior catalog text said the error fires at beam-element InitializeSolutionStep with 'not found in variables list' wording; reality is Initialize alone does NOT raise, and when it does fire later the wording matches the container error pattern, not the prior text.)", "[Numerical] SHEAR LOCKING: linear hex8 (3D8N) and quad4 (2D4N) elements lock in bending-dominated problems, producing overly stiff results and wrong frequencies. Use quadratic elements (3D20N, 3D27N, 2D8N, 2D9N) for any problem with significant bending. Signal: tip deflection on a cantilever beam meshed with 3D8N is 20-40% smaller than analytic; switching to 3D20N recovers it within 1-2%.", "[API] For POINT_LOAD on NODES: use AssignVectorVariableProcess with constrained: [false, false, false]. There is no AssignVectorByDirectionProcess class on StructuralMechanicsApplication, but there IS a core Python process module KratosMultiphysics.assign_vector_by_direction_process \u2014 and it genuinely fails on load variables because it tries to fix/free the DOF. Signal: two different failures depending on the route taken. (a) SMA.AssignVectorByDirectionProcess -> AttributeError (hasattr is False). (b) KratosMultiphysics.assign_vector_by_direction_process.Factory(...) with variable_name POINT_LOAD -> RuntimeError 'Error: Trying to fix/free dof of variable POINT_LOAD_X but this dof does not exist in node #1!'. assign_vector_variable_process on the same model part sets POINT_LOAD = [0, -100, 0] cleanly. For loads carried by CONDITIONS use assign_vector_by_direction_to_condition_process, which works. (Verified by execution 2026-08-03 on Kratos 10.4.0 \u2014 supersedes the 2026-06-01 note, which concluded from route (a) alone that 'the named class is not available to crash'; route (b) does crash, and with the originally-documented message.)", diff --git a/src/backends/kratos/generators/plasticity.py b/src/backends/kratos/generators/plasticity.py index f5a01faf..a8c73124 100644 --- a/src/backends/kratos/generators/plasticity.py +++ b/src/backends/kratos/generators/plasticity.py @@ -307,7 +307,7 @@ def _plasticity_3d_kratos(params: dict) -> str: '[Syntax] For perfect plasticity use HARDENING_CURVE=3 with large FRACTURE_ENERGY (e.g., 1e10). HARDENING_CURVE=0 (linear) still softens unless FRACTURE_ENERGY is very large. ' 'Signal: stress-strain past yield droops with negative slope despite HARDENING_MODULUS=0; integrated fracture energy < 0.5 of analytical perfect-plastic value.', '[API] Python API: constitutive-law variables are split across modules. FRICTION_ANGLE, DILATANCY_ANGLE, YIELD_STRESS_COMPRESSION live in ConstitutiveLawsApplication (CLA); FRACTURE_ENERGY, YOUNG_MODULUS in KratosMultiphysics (KM). ' - "Signal: Attribute lookup raises AttributeError 'Module KratosMultiphysics has no attribute FRICTION_ANGLE' at the moment the wrong module is dotted into (e.g. KM.FRICTION_ANGLE), BEFORE properties.SetValue is even reached. The correct path is ConstitutiveLawsApplication.FRICTION_ANGLE (returns a DoubleVariable). (Verified empirically 2026-06-01 — prior catalog claim said the error fires 'from properties.SetValue'; reality is the AttributeError fires at attribute access, never reaching SetValue.)", + "Signal: Attribute lookup raises AttributeError 'has no attribute' with the module and the name interpolated around it — the line reads Module KratosMultiphysics has no attribute FRICTION_ANGLE. — at the moment the wrong module is dotted into (e.g. KM.FRICTION_ANGLE), BEFORE properties.SetValue is even reached. The correct path is ConstitutiveLawsApplication.FRICTION_ANGLE (returns a DoubleVariable). (Verified empirically 2026-06-01 — prior catalog claim said the error fires 'from properties.SetValue'; reality is the AttributeError fires at attribute access, never reaching SetValue.)", '[API] SmallStrainIsotropicPlasticityFactory() takes NO constructor arguments. Passing KM.Parameters raises TypeError. Use the specific pre-combined class (e.g. SmallStrainIsotropicPlasticityMisesMises3D). ' "Signal: TypeError '__init__(): incompatible constructor arguments. The following argument types are supported: 1. KratosConstitutiveLawsApplication.SmallStrainIsotropicPlasticityFactory()' when the factory is called with KM.Parameters. (Verified empirically 2026-06-01 after KratosConstitutiveLawsApplication was installed; prior text said 'incompatible function arguments' / 'from SetValue binding' — the actual message says 'constructor arguments' and originates from the factory __init__ binding, not SetValue.)", '[Numerical] SHEAR LOCKING: linear hex8 (3D8N) locks in bending-dominated plasticity. Uniform-stress benchmarks (uniaxial, triaxial) are fine; gradient-stress problems need quadratic elements (3D20N, 3D27N). ' diff --git a/src/backends/kratos/generators/specialized.py b/src/backends/kratos/generators/specialized.py index 8d54c9ab..e8c530db 100644 --- a/src/backends/kratos/generators/specialized.py +++ b/src/backends/kratos/generators/specialized.py @@ -2245,7 +2245,7 @@ def thickness_gradient(response_utils): "capabilities": ["impact_loading", "blast_on_structures", "wear"], "pitfalls": [ "[Numerical] Requires DEMApplication + StructuralMechanicsApplication Signal: DemStructuresCouplingApplication needs both DEMApplication and StructuralMechanicsApplication: the application's Python loader imports its dependencies first, so a build missing one fails on that inner import line rather than on the user's own import statement.", - "[Input] The DEM wall submodelpart created at runtime for DemStructuresCouplingUtilities().TransferStructuresSkinToDem must carry dem_walls_mp.SetValue(Dem.PROPERTIES_ID, ) where appears in a material_relations entry of MaterialsDEM.json together with the sphere material (particle-wall contact law lookup is by Properties Id). Signal: 'RuntimeError: Error: PROPERTIES_ID is not set for SubModelPart SkinTransferredFromStructure . Make sure the Materials file contains material assignation for this SubModelPart' from ExplicitSolverStrategy::InitializeFEMElements. (Verified empirically 2026-06-12.)", + "[Input] The DEM wall submodelpart created at runtime for DemStructuresCouplingUtilities().TransferStructuresSkinToDem must carry dem_walls_mp.SetValue(Dem.PROPERTIES_ID, ) where appears in a material_relations entry of MaterialsDEM.json together with the sphere material (particle-wall contact law lookup is by Properties Id). Signal: RuntimeError assembled from two literals with the part name between them \u2014 'PROPERTIES_ID is not set for SubModelPart' and '. Make sure the Materials file contains material assignation for this SubModelPart' \u2014 so the line reads PROPERTIES_ID is not set for SubModelPart SkinTransferredFromStructure . Make sure ... ; raised from ExplicitSolverStrategy::InitializeFEMElements. (Verified empirically 2026-06-12.)", "[API] SurfaceLoadFromDEMCondition3D4N is NOT registered on the 10.4.2 wheel \u2014 only SurfaceLoadFromDEMCondition3D3N and LineLoadFromDEMCondition2D2N. Mesh the structure with TETRAHEDRA so SkinDetectionProcess3D creates triangle skin conditions; a hexahedral mesh (quad skin) cannot get the DEM load applied. Signal: condition creation fails with 'not registered' for SurfaceLoadFromDEMCondition3D4N during SkinDetectionProcess3D on a quad skin. (Verified empirically 2026-06-12.)", "[Input] Wall nodal forces stay identically zero unless COMPUTE_FEM_RESULTS_OPTION is enabled \u2014 DEM_procedures.DEMFEMProcedures only turns it on when one of PostElasticForces / PostContactForces / PostPressure / PostNodalArea is true in the DEM parameters; without it ComputeDEMFaceLoadUtility.CalculateDEMFaceLoads transfers nothing. Signal: rc=0 but DEM_SURFACE_LOAD is identically zero on every wall node while the particle visibly bounces (silent one-way decoupling). (Verified empirically 2026-06-12.)", "[Physics] DEM_SURFACE_LOAD is a TRACTION (N/m2), not a nodal force \u2014 cross-checking action=reaction requires integrating it with DEM_NODAL_AREA (needs PostNodalArea: true). Also: TOTAL_FORCES lives in core KratosMultiphysics, not the DEM module (unlike CONTACT_FORCES / ELASTIC_FORCES). Signal: action-reaction check off by the nodal-area factor; AttributeError: module 'KratosMultiphysics.DEMApplication' has no attribute 'TOTAL_FORCES_Z'. (Verified empirically 2026-06-12.)", diff --git a/src/backends/ngsolve/generators/advanced.py b/src/backends/ngsolve/generators/advanced.py index 331c8b96..28008a53 100644 --- a/src/backends/ngsolve/generators/advanced.py +++ b/src/backends/ngsolve/generators/advanced.py @@ -1492,8 +1492,12 @@ def psi_plus(w): ".AddEnergy(...) and .Update(...). Gotcha: " "AddIntegrator takes a bare CoefficientFunction, " "NOT an integrand-times-measure — passing " - "'... * ds' raises TypeError('AddIntegrator(): " - "incompatible function arguments'). Its docstring " + "'... * ds' raises TypeError whose only literal is " + "'arguments. The following argument types are " + "supported:' \u2014 pybind11 builds the head from the " + "bound signature, so the line reads AddIntegrator(): " + "incompatible function arguments. followed by the " + "overload list. Its docstring " "warns 'The created object must be kept alive in " "python as long as operations of it are used!', " "so bind it to a name that outlives the solve. " diff --git a/src/backends/ngsolve/generators/helmholtz.py b/src/backends/ngsolve/generators/helmholtz.py index 61f47cb4..93c13339 100644 --- a/src/backends/ngsolve/generators/helmholtz.py +++ b/src/backends/ngsolve/generators/helmholtz.py @@ -68,8 +68,11 @@ def _helmholtz_2d(params: dict) -> str: "alpha=2j) without origin. Use origin=(0, 0) for 2-D " "centered PML or origin=(0, 0, 0) for 3-D. " "Signal: pml.Radial(rad=..., alpha=...) without " - "origin raises 'TypeError: Radial(): incompatible " - "function arguments' at the call site, BEFORE " + "origin raises a pybind11 TypeError whose only " + "literal is 'arguments. The following argument types " + "are supported:'; the head is built from the bound " + "signature, so the line reads Radial(): incompatible " + "function arguments. at the call site, BEFORE " "mesh.SetPML is reached. (Verified empirically " "against NGSolve 6.2.2604 2026-06-01.)", "[Numerical] PML setup uses mesh.SetPML(pml.Radial(" diff --git a/src/backends/skfem/generators/advanced.py b/src/backends/skfem/generators/advanced.py index 8fd691c7..3d103678 100644 --- a/src/backends/skfem/generators/advanced.py +++ b/src/backends/skfem/generators/advanced.py @@ -1792,8 +1792,11 @@ def mass_pointwise(u, v, w): "not happen. (The one place that TypeError does " "appear is scalar assignment into a float lil " "matrix: Kl[0,0] = 1j raises " - "TypeError('float() argument must be a string or " - "a real number, not complex').) The SHIPPED " + "TypeError whose literal clause is 'argument " + "must be a string or a real number, not' with " + "the rejected type quoted after it, so the line " + "reads float() argument must be a string or a real " + "number, not 'complex'.) The SHIPPED " "helmholtz_2d template currently trips exactly " "this trap — its absorbing-BC block assembles to " "zero, so it has no ABC at all while still " From 8f6fd246d90380f2a57a1243b8a9d01ac77b3238 Mon Sep 17 00:00:00 2001 From: Alexander Hermann Date: Sun, 9 Aug 2026 11:55:31 +0200 Subject: [PATCH 10/13] tier2: two checks disagreed about a fixture's key, and one flaky assertion MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit THREE KRATOS FIXTURES WERE FAILING FOR A KEY THEY NO LONGER USE dem_spheric_particle_2d_rejected, dem_wall_rigidface_is_3d_only and dem_radius_is_a_core_variable each carry a correct `covers` (dem::13, dem::14, dem::15) and a stale `pitfall_index` left at 1, 2 and 3 from before the DEM pitfall list grew to 18. Two things resolve a fixture to its claim and they read different fields: tests/test_fixture_keys_point_at_real_claims.py prefers `covers`; scripts/run_tier2_fixtures.py builds :::: and never looks at `covers`. So the gate saw no clash and passed, while the runner saw dem::1/2/3 already owned by the fixtures that legitimately hold them, reported KEY COLLISION and marked all three FAILED — into a results file about to be committed as the execution record. Both checks were reading a field the other ignored; the fixtures themselves were fine. Fixed at the source by syncing pitfall_index to covers. A corpus-wide sweep found exactly these three and no others. All three now pass. tests/test_fixture_key_fields_agree.py makes the disagreement impossible. It normalises a leading underscore, because `_auxiliary_overview` is the catalog key and `auxiliary_overview` is how it is exposed — the existing gate strips it in four places, and without that this one reports seven Kratos fixtures as broken for spelling their own physics correctly. Control-tested: reintroducing the index mismatch makes it fail. MPM_POINTS_LEAVING_THE_GRID_ARE_ERASED WAS FLAKY, NOT BROKEN It failed on `initial_material_points=1` while in fact printing it. Kratos writes its banner from C++ with its own buffering, so the Python print landed mid-line: ::[MPM Analysis]:: : TIME: 0.6initial_material_points=1 The needle is present but no longer starts at a word boundary, and the runner's matcher requires that deliberately — it is what stops `count=1` matching `count=10`. Whether it happened depended on flush timing, so the fixture passed or failed for reasons unrelated to what it tests. It now flushes and starts on a fresh line; passes three runs in a row. Co-Authored-By: Claude Opus 5 (1M context) --- .../fixture.json | 2 +- .../fixture.json | 4 +- .../fixture.json | 2 +- .../source.py | 10 ++ tests/test_fixture_key_fields_agree.py | 92 +++++++++++++++++++ 5 files changed, 106 insertions(+), 4 deletions(-) create mode 100644 tests/test_fixture_key_fields_agree.py diff --git a/scripts/tier2_fixtures/kratos/dem_radius_is_a_core_variable/fixture.json b/scripts/tier2_fixtures/kratos/dem_radius_is_a_core_variable/fixture.json index 7a0b9f02..50a524c1 100644 --- a/scripts/tier2_fixtures/kratos/dem_radius_is_a_core_variable/fixture.json +++ b/scripts/tier2_fixtures/kratos/dem_radius_is_a_core_variable/fixture.json @@ -2,7 +2,7 @@ "_comment": "Tier-2: RADIUS must be set per particle (in the MDPA or via a process). Pitfall (kratos.dem #3). NOTE ON THE WRITTEN SIGNAL: this claim carries one of the five shared boilerplate Signal suffixes used across the Kratos catalog, which does not describe what actually happens. The real signal is recorded below and in the fixture output. Executed on Kratos 10.4.3 (pip wheels, glibc-2.31 host, fresh venv on external ext4 image). The variable lives in core KratosMultiphysics, not on DEMApplication \u2014 DEM.RADIUS raises AttributeError while KM.RADIUS resolves. It is a nodal quantity: reading RADIUS off a node without adding it raises the variables-list container error, and adding it makes the same read return 0.0, which is the silent default a particle keeps when nothing sets it. The fixture imports KratosMultiphysics at module scope; with Kratos absent the import raises, the process exits non-zero and the runner records a failure, never a pass. INDEX SHIFTED by the guidance reclassification; the claim this fixture verifies is unchanged and every #index quoted above is the current one. MUTATION CONTROL (added 2026-08-06, re-runnable): T2_MUTATE=1 INVERTS the expected outcome of all 4 probes -- it asserts the opposite of what this build actually does, while leaving each probe's callable untouched so every probe still really runs against Kratos. Every probe[...] line then disagrees with itself and probe_mismatches goes from 0 to 4. This proves the printed booleans come from actually calling into Kratos on this build, and that a wrong claim is caught, rather than the fixture echoing a hard-coded table. Verified: unmutated the fixture PASSES; with T2_MUTATE=1 it FAILS and these expect_in_output strings disappear: all 4 probe[