Fix hierarchical geoinfo for sumlevels that don't nest in the scope

[jinskeep-morpc]

Problem

CensusAPI(Endpoint('dec/pl', 2020), 'region15', sumlevel=SumLevel('155'), variables=['P1_001N']) (place-county parts) failed with UnboundLocalError: json. Underneath, it was an HTTP 400, hidden by morpc.req.get_json_safely; that's fixed in morpc 0.7.3.

155 requires state and place, and place doesn't nest under county, so the code falls back to geoinfo_for_hierarchical_geos(). That function had three bugs:

1. The last required geography was never split into separate requests (the explode ran only if len(in_scope) < len(requires)). The final request sent all 1,265 Ohio places, comma-joined, in one in=place:… clause, and the API allows only one place.
2. Every place in the state was fetched once per scope county (15 times for region15), because that hierarchy can't filter by county. Fixing (1) alone would mean about 19,000 requests.
3. No filtering to the scope, so the parts of scope places that lie in other counties would be returned.

Change (morpc_census/geos.py)

• For each missing requirement, first try one pseudo query from the scope (e.g. 050$1600000, which gives 191 places for region15). If that isn't available, fall back to the existing per-row for/in lookup.
• Always split the last level into separate requests, and drop duplicate parent combinations.
• Keep only results inside the scope. Results are filtered on the scope's fields that are part of the sumlevel's GEOIDFQ but not required by the API (e.g. county).

Region15 155 now takes about 191 requests per call (about 100 s).

Limitation: the place list comes from current (2024) geoinfo, so places that no longer exist aren't returned. For example, Hidden Lakes CDP (16 people in 2020) is missing.

Testing

• New tests/test_geos_hierarchical.py with 3 tests: scope filtering, one request per place, and the fallback path. All 3 fail before the change.
• Full suite: 340 passed.
• Live test, dec/pl 2000/2010/2020 for region15 155:
  • 209 / 202 / 165 parts.
  • Columbus's parts sum to its place total (905,748 in 2020).
  • Every part/place mismatch except Hidden Lakes is explained by parts in counties outside the region.

CHANGELOG (0.6.3) and reference/dev_notes.md are updated.

🤖 Generated with Claude Code