Drop element/daqreader_ndr/daqmetadatareader migrators; preserve superclasses entry fields#140
Merged
Merged
Conversation
…rclasses entry fields
did-schema reverted the V_delta renames for these three property
blocks (element.name/type, daqreader_ndr.ndr_reader_string +
ndi_daqreader_ndr_class, daqmetadatareader.ndi_daqmetadatareader_class)
so v1 bodies now satisfy the V_delta schema unchanged. The per-class
migrators became no-ops; deleting them lets the dispatcher's identity
fallback handle these classes.
Companion fix in universalRenames.normaliseSuperclasses: the previous
implementation rebuilt the superclasses struct array as
struct('class_name', names) which silently DROPPED the v1 `definition`
field. did.database/validate_doc_vs_schema then read
{superclasses.definition} as empty and the
DID:Database:ValidationSuperClasses assertion fired on the very next
write of any migrated body. The validator and the recursive
superclass-schema lookup at line 1255 both rely on `definition` being
present, so preserving it (rather than teaching every reader to fall
back to class_name) is the smaller, safer change. The new
normaliseSuperclasses keeps every input field, derives/normalises
class_name as before, and rebuilds a homogeneous struct array across
the union of fields.
Tests:
- removed three rename-asserting tests (Daqreader/Daqmetadatareader/Element)
plus the corresponding end-to-end dispatcher tests;
- added three identity-passthrough tests showing v1 bodies for these
classes survive the dispatcher with their v1 field names intact.
The element_epoch migrator is kept: that one performs a real
structural rewrite (parallel epoch_clock + t0_t1 -> clocks array of
records), which the user explicitly elected to keep over the
per-block-revert path.
V_delta (and v1) declare element.direct as integer, but the PRED
corpus and historical writes in the larger public corpora emit it
as a JSON boolean ("direct": true/false). After jsondecode that
arrives as MATLAB `logical`, and the V_delta validator rejects it
because `isnumeric(logical)` is false:
Field "element.direct" must be integer.
That accounted for all ~5,365 element-doc quarantines in the
20211116 / B / Dab / JH discovery reports and the 2 strict
quarantines that flip testCorpusPRED to fail.
Add a per-class element migrator that canonicalises direct to
numeric integer: logical -> double, char/string '0'/'1'/'true'/
'false' -> 0/1 (defensive), already-numeric passes through. Other
element fields (name, type, reference, ndi_element_class) keep
their v1 shape since V_delta declares them identically.
Update the existing identity-passthrough element test to assert
the integer coercion, and add a logical->integer regression test.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Companion to did-schema PR #53 (V_delta property-block rename revert) and NDI-matlab PR #803 (gate flip). did-schema reverts the V_delta renames for
element.{name,type},daqreader_ndr.ndr_reader_string+ndi_daqreader_ndr_class, anddaqmetadatareader.ndi_daqmetadatareader_class. v1 bodies now satisfy the V_delta schema unchanged, so the per-class migrators become no-ops.Changes:
src/did/+did2/+convert/+migrators/{element,daqreader_ndr,daqmetadatareader}.m. The dispatcher's identity fallback handles them.did2.convert.universalRenames.normaliseSuperclasses: the previous implementation rebuilt the superclasses struct array asstruct('class_name', names)which silently DROPPED the v1definitionpath.did.database/validate_doc_vs_schema:1206then read{superclasses.definition}as empty and theDID:Database:ValidationSuperClassesassertion fired on the next write of any migrated body. The new implementation preserves every input field, derives/normalisesclass_name, and rebuilds a homogeneous struct array across the union of fields.The
element_epochsuperclass migrator (parallelepoch_clock+t0_t1→clocksarray-of-records) is kept; the user elected to preserve that structural change and update the single NDI-matlab reader site instead of reverting it.Merge order
Test plan
Generated by Claude Code
Generated by Claude Code