feat: populate Rows for SS04 and SS10 validation errors#69
Merged
Conversation
The SS04 (codelist mismatch) and SS10 (cube constraint) error generators previously hard-coded `'Rows': None`, leaving consumers without the per-row context that SS02/SS05/SS06/SS07/SS11 already provide. `create_error_SS10_SS04` now takes the dataframe and filters rows where the offending value appears, populating `Rows` with the same record-dict shape as the other row-level error codes. Both call sites in `process_errors_by_column` are updated to pass `data`. This is a public API change to the function signature. Bump 2.6.8 -> 2.6.9.
The errors_test_2.json reference now expects Rows to contain the offending record dict, matching the new behaviour of create_error_SS10_SS04. This is the test that flagged the API-shape change in CI.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rowsfield with the dataframe records holding each offending value, matching the behaviour already provided by SS02/SS05/SS06/SS07/SS11.create_error_SS10_SS04gains a leadingdata(DataFrame) parameter; both call sites inprocess_errors_by_columnare updated. Public API change — external callers passing positional args need to insertdatabeforevalues.2.6.8→2.6.9.Motivation
Until now, when a CSV contained a wrong codelist value (e.g.
INV_ADV_TY=TNLSCODwhereTNLSCODis not in the codelist), the resultingSS04warning told you what the bad value was but not which rows contained it. Downstream consumers had to re-scan the dataframe themselves with brittle message parsing.After this change, the warning carries the same shape as SS07:
{ "Code": "SS04", "Component": "INV_ADV_TY", "Rows": [{ "FREQ": "M", "INV_ADV_TY": "TNLSCOD", "OBS_VALUE": "230", ... }, ...], "Message": "Wrong value TNLSCOD for dimension INV_ADV_TY" }Verified on a downstream consumer: one bad value (
TNLSCOD) yields one SS04 warning whoseRowslists all 80 dataframe records whereINV_ADV_TY == "TNLSCOD". Per-record dict shape is identical to what SS07 produces.Files changed
sdmxthon/parsers/data_validations.py— function signature + body, two call sitessdmxthon/__version__.py—2.6.8→2.6.9Changelog.rst— new entry under 2.6.9Test plan
Rowsis a list of dicts, one entry per matching dataframe rowRowscorrectly (no regression in the unmodified code paths)