Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/docs/developer/build-lifecycle.md
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,8 @@ The count lines (`C18 survey ...`, `C18 cache [...]`, `built N stations`, the `Q
unchanged, and so is every gate that stops or withholds: a `WITHHOLD`, a `GATE FAIL`, a `SKIP` and a
duplicate-id `ERROR` still name their own file or station.

Two export shapes look like frame mixing and are not drops. A rotation block that reads 0 only at periods whose transfer function is the EMPTY sentinel is one frame: exporters write 0 where they have no estimate, so the gate reads rotation angles only at periods that carry data (the Delamerian shape, 8 or 41 degrees on every estimate). A spectra ROTSPEC exactly a half-turn from the coil azimuths (ROTSPEC=180 with HX written 360, the Bollards Lagoon shape) leaves the impedance unchanged, so the station serves in the azimuth frame with the tipper sign ambiguity noted.

`stations_dropped` is the ledger every drop lands in, whatever refused it: a convention gate, a
missing coordinate or period, or the reader. Each row carries the source `file` beside the `station`
and the `reason`, because the only action a drop row can lead to is opening that file and `station`
Expand Down
50 changes: 46 additions & 4 deletions engine/extract/_conventions.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,15 +30,23 @@
* Arm C - per-period rotation WITHIN a station (PAX class: per-period ZROT/TROT, or per-block
SPECTRA ROTSPEC): REFUSE the station, exactly like a convention-gate refusal — a single served
curve from period-varying frames is misleading-by-construction; absence is honester. The reason
names the per-period rotation and the fix ("re-export in a single coherent frame").
names the per-period rotation and the fix ("re-export in a single coherent frame"). An angle
stated at a period whose impedance (for ZROT) or tipper (for TROT) is the EMPTY sentinel is not
a frame: exporters write 0 there, so only the angles at data-bearing periods decide whether a
station is uniform (the Delamerian shape, 8 or 41 degrees on every estimate and 0 on the empties).
* rotation UNKNOWABLE (sentinel/missing angles at data-bearing periods, reader/text disagreement,
ROTSPEC-vs-azimuth conflict, RHOROT declared rotated while the Z frame is undeclared) -> FAIL:
ROTSPEC-vs-azimuth conflict other than a half-turn, RHOROT declared rotated while the Z frame is
undeclared) -> FAIL:
the station is skipped loudly (fail-closed) - never served in an unresolvable frame.
* no declaration at all, or azimuth metadata too inconsistent to be evidence (e.g. the harness
Tasmania files' HX AZM=180/HY AZM=90 non-orthogonal placeholders) -> serve with the frame
facts recorded; Gate 2 still checks the convention. Azimuths on the impedance branch are
ACQUISITION metadata, not the stored-tensor frame — the >ZROT declaration wins when present
(USArray: physical sensor azimuths ±19° with ZROT=0 = processed-to-zero, served as-is).
* a spectra ROTSPEC exactly a half-turn from the HMEAS azimuths (ROTSPEC=180 with HX written 360,
the Bollards Lagoon shape) is not a conflict: negating both horizontal axes leaves every
cross-power, and so the impedance, unchanged; the station serves in the azimuth frame with the
tipper's sign ambiguity noted.
The de-rotation MATH (Z0(i) = R(-θi) Z(i) R(-θi)^T, T0(i) = T(i) R(-θi)^T, R(β) = [[cosβ, sinβ],
[-sinβ, cosβ]]) is RETAINED below for DIAGNOSTICS only — no serve-path caller invokes it (v3). It is
pinned by the synthetic round-trips and the AusLAMP-SA custodian-twin proof for future diagnostic use.
Expand Down Expand Up @@ -156,6 +164,8 @@ def parse_frame_evidence(text: str) -> dict:
# bears on no disposition.)
return {
"branch": "spectra" if spectra else "mt",
"z_empty": _empty_mask(text, ("ZXXR", "ZXYR", "ZYXR", "ZYYR")),
"t_empty": _empty_mask(text, ("TXR", "TYR", "TXR.EXP", "TYR.EXP")),
"zrot": _block_values(text, "ZROT"),
"rhorot": _block_values(text, "RHOROT"),
"trot": (_block_values(text, "TROT") if _block_values(text, "TROT") is not None
Expand All @@ -169,6 +179,26 @@ def parse_frame_evidence(text: str) -> dict:
}


def _empty_mask(text: str, labels) -> Optional[list]:
"""Per-period True where EVERY present block among `labels` carries the missing-data sentinel
(|v| > ROT_FILL_MAX, the ~1e32 EMPTY convention) at that period; None when no block among
`labels` is present or the present blocks disagree in length. Exporters write a rotation of 0
at periods that carry no estimate, so a rotation block is only read at periods that carry data."""
cols = [v for v in (_block_values(text, lab) for lab in labels) if v is not None]
if not cols or any(len(c) != len(cols[0]) for c in cols):
return None
return [all((not math.isfinite(c[i])) or abs(c[i]) > ROT_FILL_MAX for c in cols)
for i in range(len(cols[0]))]


def _angles_at_data(vals, empty):
"""`vals` with the entries at empty periods dropped, when `empty` aligns with it; otherwise
`vals` unchanged. A rotation stated where there is nothing to rotate is not a frame."""
if not vals or not empty or len(empty) != len(vals):
return list(vals or [])
return [v for v, e in zip(vals, empty) if not e]


def _mask_sentinels(vals):
"""Replace missing-data sentinels (|v| > ROT_FILL_MAX, the community ~1e32 convention) with
None, preserving positions. None in -> None out."""
Expand Down Expand Up @@ -356,6 +386,18 @@ def _done(recorded_deg=None) -> FrameDisposition:
if abs(abs(_norm_angle(rs_th)) - abs(_norm_angle(az))) <= AZIMUTH_TOL_DEG:
# The Black Hill case: |HMEAS-implied| == |ROTSPEC| -> ONE rotation, not two.
theta = _norm_angle(rs_th)
elif abs(((rs_th - az) % 360.0) - 180.0) <= AZIMUTH_TOL_DEG:
# A half-turn between the two declarations (the Bollards Lagoon case: ROTSPEC=180
# with HX written as 360). Rotating both horizontal axes by 180 degrees negates
# every channel, and a cross-power of two negated channels is unchanged, so the
# impedance is the same in either frame; the azimuth frame is served. The tipper
# is a vector and does change sign under a half-turn, so that ambiguity is noted.
theta = _norm_angle(az)
notes.append(f"frame: spectra ROTSPEC={rs_th:g} is a half-turn from the HMEAS "
f"azimuths (HX={ev['azm_hx']}, HY={ev['azm_hy']}); a half-turn leaves "
f"the impedance unchanged, so it is served in the azimuth frame "
f"({theta:g} deg); the tipper sign carries the half-turn ambiguity "
f"and is served as stored")
else:
return _fail(f"SPECTRA frame declarations conflict: ROTSPEC={rs_th:g} but the HMEAS "
f"azimuths imply {az:g} (HX={ev['azm_hx']}, HY={ev['azm_hy']}) — the "
Expand Down Expand Up @@ -386,7 +428,7 @@ def _done(recorded_deg=None) -> FrameDisposition:
return _fail("ZROT carries a missing-data sentinel (~1e32) at periods that HAVE "
"impedance data — the frame of those estimates is unknowable; fix: "
"supply real per-period rotation angles or zero (the declared zero reference).")
u = _uniq_eps(zr)
u = _uniq_eps(_angles_at_data(zr, ev.get("z_empty")))
# cross-check what the reader itself recorded (mt_metadata nulls sentinels to 0)
if rot_mtm is not None and len(u) >= 1:
mu = _uniq_eps([float(v) for v in rot_mtm])
Expand Down Expand Up @@ -440,7 +482,7 @@ def _done(recorded_deg=None) -> FrameDisposition:
# rides in facts["evidence"]["trot"] as a recorded fact; nothing is rotated. ----
if has_tipper and ev["trot"] is not None:
tr = _mask_sentinels(ev["trot"])
tu = _uniq_eps(tr)
tu = _uniq_eps(_angles_at_data(tr, ev.get("t_empty")))
tnz = [a for a in tu if abs(a) > ROT_ZERO_EPS_DEG]
if any(v is None for v in tr) and tnz:
return _fail("TROT mixes missing-data sentinels with nonzero angles — the tipper "
Expand Down
81 changes: 81 additions & 0 deletions engine/tests/test_convention_gates.py
Original file line number Diff line number Diff line change
Expand Up @@ -758,3 +758,84 @@ def test_conventions_header_states_the_same_rotation_sign_as_the_derotation_docs
"the impedance quadrants are named with their signed bounds")
assert re.search(r"applies R\(-θ\)\)", src), "the de-rotation docstring states R(-θ)"
assert "R(θi) Z(i)" not in src and "(180..-90°)" not in src, "no unsigned copy of either statement may stand"


# ---------------------------------------------------------------------------------------------
# A rotation stated at a period that carries no estimate is not a frame (the Delamerian shape:
# EMpower writes TROT 0 where the tipper is EMPTY and 8 or 41 where it exists), and a spectra
# ROTSPEC that is a half-turn from the coil azimuths leaves the impedance unchanged (Bollards Lagoon).
# ---------------------------------------------------------------------------------------------
_EMPTY = 1.0e32


def _blank_periods(text, labels, idx):
for lab in labels:
vals = _read_block(text, lab)
for i in idx:
vals[i] = _EMPTY
text = _write_block(text, lab, vals)
return text


def test_trot_zero_only_at_empty_tipper_periods_is_one_frame(tmp_path):
"""FAILS IF: a station whose TROT reads 0 only at periods where every tipper block carries the
EMPTY sentinel is refused as per-period. The exporter writes 0 where it has no estimate; every
real estimate sits at one angle, so the tipper frame is uniform and the station serves."""
n = _N_VULCAN
idx = list(range(3, n, 7))
text = _vulcan_with_tipper(0.0, 8.0)
trot = [8.0] * n
for i in idx:
trot[i] = 0.0
text = _write_block(text, "TROT", trot)
text = _blank_periods(text, ("TXR.EXP", "TXI.EXP", "TYR.EXP", "TYI.EXP"), idx)
parsed = _parse(tmp_path, "trot_empty.edi", text)
assert "skip" not in parsed, f"a zero written where the tipper is EMPTY is not a frame: {parsed.get('skip')}"


def test_trot_two_angles_on_data_periods_still_refused(tmp_path):
"""FAILS IF: the empty-period rule lets a real per-period tipper frame through. The same two
angles, but the tipper carries data at both, must still be refused."""
n = _N_VULCAN
idx = list(range(3, n, 7))
text = _vulcan_with_tipper(0.0, 8.0)
trot = [8.0] * n
for i in idx:
trot[i] = 0.0
text = _write_block(text, "TROT", trot)
parsed = _parse(tmp_path, "trot_mixed.edi", text)
assert "skip" in parsed and "per-period TROT" in parsed["skip"]["reason"]


def test_zrot_zero_only_at_empty_impedance_periods_is_one_frame(tmp_path):
"""FAILS IF: a ZROT of 0 written only at periods where every impedance block is EMPTY makes the
station per-period. The estimates all sit at one angle, which is served as stored."""
n = _N_VULCAN
idx = [0, 1, n - 1]
text = _vulcan_rotated(8.0)
zrot = [8.0] * n
for i in idx:
zrot[i] = 0.0
text = _write_block(text, "ZROT", zrot)
text = _blank_periods(text, ("ZXXR", "ZXXI", "ZXYR", "ZXYI", "ZYXR", "ZYXI", "ZYYR", "ZYYI"), idx)
parsed = _parse(tmp_path, "zrot_empty.edi", text)
assert "skip" not in parsed, f"a zero written where the impedance is EMPTY is not a frame: {parsed.get('skip')}"
assert parsed["frame"]["frame_served"] == "declared-azimuth"
assert parsed["frame"]["declared_azimuth_deg"] == 8.0


def test_spectra_rotspec_half_turn_from_azimuths_serves_in_azimuth_frame(tmp_path):
"""FAILS IF: a spectra file with ROTSPEC=180 and HX written as 360 is refused as a conflict.
A half-turn negates every channel and a cross-power of two negated channels is unchanged, so
the impedance is the same in either frame; it serves in the azimuth frame with the half-turn
noted. A quarter-turn conflict (ROTSPEC=90 against azimuth 0) must still refuse."""
text = PHOENIX.read_text(encoding="latin-1")
half = re.sub(r"ROTSPEC=\S+", "ROTSPEC=180", text)
half = re.sub(r"(CHTYPE=HX[^\n]*?AZM=)0\b", r"\g<1>360", half)
assert "AZM=360" in half
parsed = _parse(tmp_path, "halfturn.edi", half)
assert "skip" not in parsed, f"a half-turn is not a conflict: {parsed.get('skip')}"
assert parsed["frame"]["frame_served"] == "declared-zero"
quarter = re.sub(r"ROTSPEC=\S+", "ROTSPEC=90", text)
parsed_q = _parse(tmp_path, "quarter.edi", quarter)
assert "skip" in parsed_q and "conflict" in parsed_q["skip"]["reason"]
Loading