Varnasr · Varnasr · Sep 23, 2026 · Sep 22, 2026
@@ -7,20 +7,63 @@ on:
     branches: [main]
 
 jobs:
-  validate:
+  test:
     runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.11", "3.12"]
     steps:
       - uses: actions/checkout@v6
 
-      - name: Set up Python
+      - name: Set up Python ${{ matrix.python-version }}
         uses: actions/setup-python@v6
         with:
-          python-version: "3.11"
+          python-version: ${{ matrix.python-version }}
+          cache: pip
 
       - name: Install dependencies
-        run: |
-          pip install -r requirements.txt
+        run: pip install -r requirements-dev.txt
 
-      - name: List available generators
+      # This used to be the entire job. `--list` imports no generator, so all 36
+      # modules were unexecuted by CI and a broken one would have gone unnoticed.
+      - name: Registry lists cleanly
         run: python generate.py --list
 
+      - name: Test suite
+        run: python -m pytest tests/ -q
+
+  end_to_end:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v6
+      - uses: actions/setup-python@v6
+        with:
+          python-version: "3.11"
+          cache: pip
+      - run: pip install -r requirements.txt
+
+      # Writing the files is a separate failure surface from generating the
+      # frames: a column dtype that a frame holds happily can still fail to
+      # serialise, and parquet is stricter than csv about mixed types.
+      - name: Generate every dataset to CSV
+        run: python generate.py --rows 500 --output /tmp/out-csv
+
+      - name: Generate every dataset to Parquet
+        run: python generate.py --rows 500 --format parquet --output /tmp/out-parquet
+
+      - name: Every registered dataset produced a file
+        run: |
+          python - <<'PY'
+          import pathlib, sys
+          from generate import GENERATORS
+          missing = []
+          for fmt, d in (("csv", "/tmp/out-csv"), ("parquet", "/tmp/out-parquet")):
+              for name in GENERATORS:
+                  p = pathlib.Path(d) / f"{name}.{fmt}"
+                  if not p.exists() or p.stat().st_size == 0:
+                      missing.append(str(p))
+          if missing:
+              sys.exit("missing or empty outputs:\n  " + "\n  ".join(missing))
+          print(f"PASS - {len(GENERATORS)} datasets written in both formats")
+          PY
@@ -3,6 +3,7 @@ output/
 *.parquet
 __pycache__/
 *.pyc
+.pytest_cache/
 .venv/
 .env
 *.egg-info/

@@ -0,0 +1,105 @@
+# devdata-practice
+
+Synthetic datasets for practising development economics analysis. Thirty-six
+generators, pure Python, no service dependencies, nothing fetched at runtime.
+
+## Commands
+
+```bash
+pip install -r requirements-dev.txt
+python generate.py --list                  # registry
+python generate.py rct_experiment          # one dataset to ./output
+python generate.py --rows 500              # all of them, small
+python -m pytest tests/ -q                 # ~2 min, mostly the degeneracy guard
+```
+
+## Layout
+
+`generate.py` holds `GENERATORS`, the registry mapping a dataset name to its
+module, its size keyword and a description. `generators/` holds one module per
+dataset, each exporting `generate(**kwargs) -> DataFrame`. `generators/utils.py`
+holds the shared geography, missingness and id helpers. `TRUTH.md` is the answer
+key. `tests/` is the guard.
+
+A module on disk that is not in `GENERATORS` is invisible to every user; one in
+`GENERATORS` that is not on disk crashes. `tests/test_generators_run.py` compares
+the two, so neither can happen quietly.
+
+## The bug this repository is shaped around
+
+`rng.binomial(1, 0.55)` **without a size argument returns a scalar**, which numpy
+broadcasts across the whole column. Nothing raises, the row count is right, the
+file writes, and the variable is a constant. It was present at nineteen call
+sites across nine generators and produced fifteen constant columns, including
+four asset variables in `targeting`, a proxy-means-test dataset whose asset
+predictors did not vary.
+
+It appears in two shapes:
+
+```python
+owns_radio = rng.binomial(1, 0.55)                          # every row: the same draw
+months_displaced = np.where(displaced, rng.exponential(14), 0)   # every displaced row: the same
+```
+
+The second shape is the dangerous one, because the column then holds *two*
+values rather than one and so survives any "is this column constant" check. It
+is why `tests/test_no_degenerate_columns.py` tests for perfect binary aliasing
+as well as for constants.
+
+The discriminator when reading code: **is the probability a scalar or an array?**
+`barrier_cost` in `girls_education` was always fine because its probability is
+`0.35 - 0.10 * receives_scholarship`, an array, so numpy returned a vector. Its
+neighbours on the surrounding lines took scalar probabilities and were broken.
+
+A scalar draw is correct when it is a *parameter* rather than a column: the
+per-arm compliance rate in `rct_experiment`, the per-country intercept in
+`panel_data`, anything inside a per-row loop. Do not add `size=n` to those. The
+test to apply is whether the result becomes a column.
+
+## Watch out for
+
+- **Do not loosen a tolerance in `tests/` to make a failure go away.** The
+  numbers in `TRUTH.md` are what a learner checks their answer against. If a test
+  fails, either the generator changed or `TRUTH.md` is now lying.
+- **The ITT in `rct_experiment` is not a fixed number.** Take-up is drawn
+  `U(0.65, 0.85)` per arm per run, so the ITT moved between +0.089 and +0.203
+  across five seeds. Only the complier effect (Wald: ITT ÷ take-up) is a stable
+  target, and only against pure-control villages.
+- **`spillover_risk` has a village level under it.** Villages are randomised
+  into treatment (70%) or pure control (30%); the flag marks control units inside
+  a treatment village. Before that existed, randomisation was stratified within
+  district, every district therefore contained treated units, and the flag was an
+  exact alias of the control dummy. If you flatten the design back to individual
+  randomisation, the flag becomes useless again.
+- **PHQ-9 is a sum of nine 0–3 items, not a transformed latent.** The previous
+  `9 * logistic(...) * 3` construction was bounded and had no item-level variance:
+  the floor sat at 2 and `depression_severe` was identically zero in every draw.
+  If you touch the item severities, re-check the bands in `TRUTH.md`.
+- **Requirements are pinned on purpose.** `pandas>=2.0` would pick up pandas 3,
+  where text columns carry a `str` dtype rather than `object`, so any check keyed
+  on `dtype == object` skips every text column without saying so. Raise the pins
+  deliberately and together, then re-run the suite.
+- **Nothing here is real data.** Country names and GDP per capita in
+  `generators/utils.py` are plausible orders of magnitude, not current figures.
+  Do not cite them, and do not describe any generator as reproducing a survey.
+
+## Adding a generator
+
+1. Write `generators/<name>.py` exporting `generate(n_x=..., seed=...) -> DataFrame`.
+2. Register it in `GENERATORS` in `generate.py` with its size keyword and a
+   one-line description.
+3. Run `python -m pytest tests/ -q`. The degeneracy guard runs against every
+   registered generator automatically, so you do not add a test for it.
+4. If the generator encodes a parameter someone is meant to recover, put it in
+   `TRUTH.md` and assert it in `tests/`.
+
+## Testing
+
+`.github/workflows/ci.yml` runs the suite on 3.11 and 3.12, plus an end-to-end
+job that writes every dataset to CSV and Parquet and checks the files are
+non-empty. Serialisation is a separate failure surface from generation: parquet
+is stricter than csv about mixed types, so a frame that builds happily can still
+fail to write.
+
+CI used to run `python generate.py --list` and nothing else. That imports no
+generator, so all thirty-six modules were unexecuted by any automated check.
@@ -2,21 +2,33 @@
 
 [![Website](https://img.shields.io/badge/Docs-varnasr.github.io%2Fdevdata--practice-blue)](https://varnasr.github.io/devdata-practice/)
 [![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
-[![Python 3.9+](https://img.shields.io/badge/Python-3.9%2B-blue)](https://www.python.org/)
+[![Python 3.11+](https://img.shields.io/badge/Python-3.11%2B-blue)](https://www.python.org/)
 [![GitHub Last Commit](https://img.shields.io/github/last-commit/Varnasr/devdata-practice)](https://github.com/Varnasr/devdata-practice/commits/main)
 [![Part of ImpactMojo](https://img.shields.io/badge/Part%20of-ImpactMojo-orange)](https://www.impactmojo.in)
 
 **Realistic, large-scale practice datasets for development economics — 36 generators, 840,000+ rows.**
 
-Built for researchers, students, and practitioners who need real-feeling data modelled on DHS, NFHS, ASER, and other major development survey frameworks.
+Built for researchers, students and practitioners who need data shaped like a
+real survey: the same variables, the same awkward missingness, the same
+structure, without waiting on a data request.
 
 **Full documentation:** [varnasr.github.io/devdata-practice](https://varnasr.github.io/devdata-practice/)
 
 ---
 
 ## About
 
-DevData Practice generates synthetic datasets that closely mirror the structure, variable distributions, and statistical properties of real development sector surveys. The data is designed for:
+DevData Practice generates synthetic datasets built to the *shape* of development
+sector surveys: the variables a DHS or LSMS instrument collects, laid out the way
+it lays them out, with realistic missingness, partial compliance and attrition.
+
+It does not reproduce any survey's distributions, and the figures it produces are
+not estimates of anything. A stunting rate here is a number the code was told to
+produce, not a measurement. If you need to check a pipeline against published
+figures, use a real recode and a published table. `TRUTH.md` says exactly which
+parameters are in the data and which estimand recovers each one.
+
+The data is designed for:
 
 - **Learning** — practice data analysis, MEL, and econometrics without needing access to restricted datasets
 - **Teaching** — ready-made datasets for classroom exercises, workshops, and tutorials
@@ -47,7 +59,8 @@ python generate.py --list
 python generate.py rct_experiment labor_market household_survey
 ```
 
-Generated files are saved to the `data/` directory as CSV files.
+Generated files are saved to `./output` as CSV files. Use `--output` to change
+the directory and `--format parquet` to change the format.
 
 ---
 
@@ -107,13 +120,55 @@ devdata-practice/
 ## Requirements
 
 ```
-pandas>=1.5.0
-numpy>=1.23.0
-scipy>=1.9.0
-faker>=15.0.0
+numpy==2.4.6
+pandas==3.0.6
+scipy==1.17.1
+pyarrow==25.0.1
 ```
 
-Python 3.9 or higher.
+Python 3.11 or higher. Versions are pinned exactly, not floored: see `CLAUDE.md`
+for why. `requirements-dev.txt` adds pytest.
+
+An earlier version of this section listed `faker>=15.0.0`. Nothing in the
+repository imports it.
+
+---
+
+## Checking your answer
+
+Every generator encodes parameters on purpose. `TRUTH.md` records what they are
+and which estimand recovers each one, so an exercise can be marked rather than
+guessed at.
+
+The one worth reading before you use `rct_experiment`: its intention-to-treat
+effect is **not** a fixed number. Take-up is drawn `U(0.65, 0.85)` for each arm on
+each run, so the ITT moved between +0.089 and +0.203 log points across five seeds.
+Only the complier effect is stable, and only when computed against the
+pure-control villages (`spillover_risk == 0`). Controls inside a treatment village
+receive a +3% spillover; using them as the comparison biases every ITT toward
+zero by about 0.011 log points, which is the lesson the design exists to teach.
+
+## Testing
+
+```bash
+pip install -r requirements-dev.txt
+python -m pytest tests/ -q          # 233 tests, about two minutes
+```
+
+CI runs the suite on Python 3.11 and 3.12, then writes every dataset to CSV and
+Parquet and checks the files are non-empty.
+
+The largest part of the suite is a guard against degenerate columns, and it exists
+because of a specific defect. `rng.binomial(1, 0.55)` without a size argument
+returns a *scalar*, which numpy broadcasts across the whole column: nothing
+raises, the row count is right, the file writes, and the variable is a constant.
+Nineteen call sites across nine generators were affected. Among the fifteen
+constant columns were four asset variables in `targeting`, a proxy-means-test
+dataset whose asset predictors did not vary, and the three dropout barriers in
+`girls_education`, all identically zero.
+
+It was invisible because CI ran `python generate.py --list`, which imports no
+generator at all. Thirty-six modules were unexecuted by any automated check.
 
 ---
 

@@ -0,0 +1,85 @@
+# Ground truth
+
+A practice dataset you cannot check your answer against is a worked example with
+the answer torn off. This file records the parameters the generators actually
+use, which estimand recovers each one, and how close you should expect to get.
+Everything here is asserted by `tests/`, so it cannot quietly drift away from the
+code.
+
+Two warnings before the numbers.
+
+**Not every true parameter is a stable target.** In `rct_experiment` the take-up
+rate is drawn `U(0.65, 0.85)` afresh for each arm on each run. The intention-to-treat
+effect is therefore a different number for every seed. It ranged from +0.089 to
++0.203 log points across five seeds at n = 60,000. Only the complier effect is
+stable. If you are checking your work against a fixed number, check the LATE.
+
+**Recovering a parameter is not the same as the parameter being there.** Several
+of these are recoverable only under the identifying assumption the exercise is
+meant to teach. The spillover figure below is recoverable because the design has
+pure-control villages; drop them and it is not identified at all.
+
+## rct_experiment
+
+Outcome is `log(endline_consumption_usd) - log(baseline_consumption_usd)`.
+
+| Quantity | True value | How to recover it |
+|---|---|---|
+| Complier effect, cash_transfer | `log(1.15) + h` = **+0.1893** | Wald: ITT ÷ take-up, against pure controls |
+| Complier effect, cash_plus_training | `log(1.22) + h` = **+0.2484** | same |
+| Complier effect, training_only | `log(1.08) + h` = **+0.1265** | same |
+| Spillover on untreated neighbours | `log(1.03)` = **+0.0296** | exposed controls minus pure controls |
+| Common time trend | `log(1.03)` = +0.0296 | absorbed by any control group; not separately identified |
+
+`h = log(1 + 0.04 × P(female) + 0.06 × P(below-median baseline)) = +0.0496` is the
+average heterogeneity loading among compliers. It is part of the complier effect
+because the heterogeneity terms multiply only for units with `actually_treated == 1`.
+If you compare against `log(1.15) = +0.1398` alone you will appear to over-recover
+by about five log points, and the gap is this term, not an error.
+
+Observed across seeds 1, 42, 123, 777 and 2026 at n = 60,000: Wald means of
++0.1873, +0.2480 and +0.1248, all within 0.002 of theory. `tests/test_rct_truth.py`
+asserts a tolerance of 0.01.
+
+**The control group you choose changes the answer.** Villages are randomised into
+treatment (70%) or pure control (30%), then individuals within treatment villages
+are randomised across arms. Control units inside a treatment village receive the
++3% spillover. Using all controls as the comparison therefore biases every ITT
+toward zero by roughly 0.011 log points. Using `spillover_risk == 0` controls only
+is the clean contrast.
+
+**`spillover_risk` used to be unusable.** Before this was fixed it equalled 1 for
+every control and 0 for every treated unit, an exact alias of the treatment dummy,
+because randomisation was stratified within district so every district always
+contained treated units. Any regression including both dropped a collinear term.
+`tests/test_no_degenerate_columns.py` now fails if any column collapses that way.
+
+## Calibration targets
+
+These are design targets for a synthetic teaching dataset, not estimates from a
+named survey. They are asserted in `tests/test_calibration.py` so a future change
+to a generator cannot silently move them.
+
+| Generator | Quantity | Target |
+|---|---|---|
+| `public_health` | PHQ-9 ≥ 10 (moderate or worse) | 12–18% |
+| `public_health` | PHQ-9 ≥ 20 (severe) | 0.8–2.5% |
+| `public_health` | PHQ-9 in 0–4 (minimal) | 45–65% |
+| `public_health` | PHQ-9 support | reaches both 0 and the upper 20s |
+| `girls_education` | mean distance to school, rural | 3.5–4.5 km |
+| `girls_education` | mean distance to school, urban | 1.2–1.8 km |
+| `rct_experiment` | take-up in each treatment arm | 0.60–0.90 |
+| `targeting` | asset ownership (radio, mobile, bicycle) | matches its stated rate ± 0.05 |
+
+PHQ-9 is built the way the instrument is: nine items each scored 0–3 and summed.
+The previous version mapped a single latent normal through `9 × logistic(...) × 3`,
+which is bounded and has no item-level variance, so the floor sat at 2 and
+`depression_severe` was identically zero in every draw.
+
+## What is deliberately *not* true here
+
+- No generator reproduces a real survey. Country names and GDP per capita in
+  `generators/utils.py` are plausible orders of magnitude, not current figures,
+  and should not be cited.
+- Nothing here is a benchmark. If you need to check a pipeline against published
+  numbers, use a real recode with a published table.