Skip to content

UoA reader updates - #6

Merged
kujaku11 merged 18 commits into
IAGA-DVI-DataStandards:developfrom
bvkay:uoa-update
Sep 11, 2026
Merged

kujaku11 merged 18 commits into
IAGA-DVI-DataStandards:developfrom
bvkay:uoa-update

Conversation

@bvkay

@bvkay bvkay commented Aug 24, 2026 •

Copy link
Copy Markdown
Contributor

Fixes 05

Updates with:

  1. Reader Registration: for Lemi-423, EDL PR6-24 and legacy orange box.

  2. Filter Gain Direction: gains are stored counts to physical in pr624.py and lemi423.py. MTH5 divides by the total gain when removing a response, so they must be stored physical to recorded.

  3. Filters Marked Not Applied: pr624.py:760 writes applied=False. Aurora only divides out stages flagged applied, so the sensor and interface responses stay in the data.

  4. AppliedFilter Field Name: the same call passes sequence=sequence. AppliedFilter takes stage, not sequence, so every stage was written as None.

  5. LEMI-120 Sensitivity: updated to 400 mV/nT.

  6. LEMI-423 Reader Cannot Run: four errors before it returns, with MTime called positionally at three sites, data_type = "MTBB" where the enum takes BBMT, ch_metadata.filter.name from the pre-1.0 mt_metadata API, and ChannelTS and RunTS used but never imported.

  7. Orange Box Format: records are 21 bytes, not 18; the first header field is a sample count, not a sample rate; and the rate comes from the filter point as 10e6 / (512 * filterpoint).

  8. AppleDouble Sidecars: Collection.get_files matches ._name sidecars, which share the data extension but hold resource fork bytes.

  9. miniSEED PR6-24: added miniseed support for EDL PR6-24, as AusLAMP Victoria was recorded in this format.

  10. B423 Collection Summaries: building a Lemi-423 collection read every file in full. read_summary now memmaps the time columns and takes min/max, the same start/end the old sorted parse reported, so summaries come from the 1024-byte header plus two columns.

  11. LEMI-424 Speed and the 16-Column Variant: read_metadata tail-seeks instead of parsing the whole file, position columns map unique values only, and multi-file reads join once; the n_samples estimate divides by the first line's width instead of a hard-coded 152. Instruments with gps/clock problems write a 16-column file, time + fields + battery, no GPS block; which the reader could not open at all. The reader now sniffs the first line's field count; position properties return None when there is no GPS block.

  12. Repeated Parsing in Phoenix and miniSEED: recmeta.json was re-parsed for every channel reader; now cached per (path, mtime). EDL miniSEED reads pass format="MSEED" so obspy skips its format-detection cascade.

@bvkay
bvkay requested a review from kujaku11 August 24, 2026 09:15

@kujaku11 kujaku11 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bvkay This is great, thanks for these updates.

@kujaku11
kujaku11 changed the base branch from main to develop September 11, 2026 21:49
@kujaku11
kujaku11 merged commit 4580b78 into IAGA-DVI-DataStandards:develop Sep 11, 2026
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

UoA + Lemi-423 readers cannot produce calibrated data

2 participants