Skip to content

Recover CensusAPI.load() metadata from name + long data - #116

Merged
jinskeep-morpc merged 2 commits into
mainfrom
feature/load-recover-metadata
Jun 16, 2026
Merged

jinskeep-morpc merged 2 commits into
mainfrom
feature/load-recover-metadata

Conversation

@jinskeep-morpc

Copy link
Copy Markdown
Owner

Summary

CensusAPI.load() previously depended on a private _morpc block written into the resource descriptor. This removes that dependency: save()/create_resource() no longer write _morpc, and load() recovers all six constructor arguments from the canonical dataset name and the long-format data.

Recovery map (_recover_metadata(name, long)):

Arg Source
survey, year uniform survey / reference_period columns
variables base codes re-suffixed from the value-type columns (new _recover_variable_codes); only when the name carries -select-variables
group _group_code_from_variable on the data, kept only when the name ends with the matching -{code}
sumlevel GeoIDFQ.parse(...).sumlevel (geoidfq primary), kept only when the name carries the hierarchy token
scope the SCOPES key remaining after the survey/year/sumlevel/group/variable markers are stripped

The name's presence/absence of the sumlevel token and group segment is the source of truth for distinguishing the three modes (group-only / variables-only / both) — the long data alone cannot, since all three can produce identical rows. Anchored parsing recovers the other parts from the data first, so the survey's embedded dashes don't make the name parse ambiguous.

Behavior change

load() no longer raises on resources lacking a _morpc block. It raises ValueError only when the name doesn't match the data's survey/year or the residual token isn't a known scope.

Tests

  • Replaced test_resource_without_morpc_raises with test_resource_has_no_morpc_block (asserts save() no longer emits _morpc and the resource still loads).
  • Added recovery tests for all three modes: test_recover_group_only_no_sumlevel, test_recover_variables_only, test_recover_group_and_variables.
  • Full suite: 334 passed.

🤖 Generated with Claude Code

jinskeep-morpc and others added 2 commits June 16, 2026 10:16
Drop the private _morpc descriptor that load() depended on. save() now
writes only standard frictionless fields, and load() recovers all six
constructor arguments via the new _recover_metadata(name, long):

- survey/year from the uniform survey/reference_period columns
- variables from base codes re-suffixed off the value-type columns
  (new _recover_variable_codes), gated on the -select-variables marker
- group from the variable codes, gated on the matching -{code} segment
- sumlevel from the geoidfq summary level, gated on the name's hierarchy
  token (the name is the source of truth for whether sumlevel was given)
- scope from the SCOPES key remaining after the other markers are stripped

load() no longer raises on a missing _morpc block; it raises ValueError
only when the name/data are inconsistent with a known scope.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@jinskeep-morpc
jinskeep-morpc merged commit ddc9bd2 into main Jun 16, 2026
7 checks passed
@jinskeep-morpc
jinskeep-morpc deleted the feature/load-recover-metadata branch June 16, 2026 14:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant