Fix columnMappings keys to dcid: predicate form (#102)#117
Conversation
712220b to
130e1a1
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #117 +/- ##
==========================================
+ Coverage 91.06% 93.35% +2.29%
==========================================
Files 37 38 +1
Lines 1801 2048 +247
==========================================
+ Hits 1640 1912 +272
+ Misses 161 136 -25 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
130e1a1 to
a0e3dc1
Compare
jm-rivera
left a comment
There was a problem hiding this comment.
Thanks @tillywoodfield! it may be worth fixing a small linked issue. Since columnMappings is consumed by the importer, which requires the dcid prefixed keys, we need the serialisation to always work with the alias mapping.
To solve it, please just add by_alias=True to the model_dump call in config_to_dict(). You may have to update a related test that may not be expecting the dcid prefix on the keys.
a0e3dc1 to
9d20ba0
Compare
jm-rivera
left a comment
There was a problem hiding this comment.
Thanks @tillywoodfield !
Closes #102.
What
ColumnMappingsserialized its fields under short Python names (entity,variable,date, …), but the DCP importer keys observations bydcid:-prefixed predicate names. Any other key is treated as a custom dimension, so every exportedconfig.jsonfailed the import:How
Update
ColumnMappingsto accept either form on input and always emit thedcid:key on output.variabledcid:variableMeasuredentitydcid:observationAboutdatedcid:observationDatevaluedcid:valueunitdcid:unitmeasurementMethoddcid:measurementMethodobservationPerioddcid:observationPeriodscalingFactorscalingFactor(unchanged — see below)scalingFactor
Left as the short
scalingFactorkey because the importer has nodcid:scalingFactorroute yet.Tests
test_column_mappings_emit_dcid_keys— asserts the exact emitted key strings and thatscalingFactorstays short (guards against alias typos).test_column_mappings_accepts_dcid_keys_on_input— read-back guard.Co-authored with Claude Code