docs: convert plot descriptions to plt.figtext + strip tqdm noise from notebooks

[jonathanhhb]

Summary

Two bundled improvements, both aimed at making notebook content clean and RAG-friendly. Rebased onto main (was originally stacked on #204's branch, which is now closed — this PR is now standalone).

1. Plot descriptions in plt.figtext form (originally #240)

The 70 plot descriptions authored in #204 are converted from standalone ### Reading the … markdown cells into plt.figtext(...) captions inside the plot-generating code cells. Content compressed to the takeaway sentence per description.

Why: the markdown form created its own RAG chunks that competed for retrieval on prompts like "build this SIR model" — semantically close ("simulate + plot") but no API content, so retrieval picked them up and displaced model-building code. plt.figtext embeds the description inside the code cell's Python source; the concat pipeline emits that cell as a fenced python block, and the ingest H4 splitter (laser-mcp #37) + fence-first character splitter keep the string fused with the surrounding plotting code.

Alpha suite evidence (jenner-generic-mcp, N=3 per arm):

Arm	Runs	Mean / 20	Δ vs no-descriptions
No descriptions (baseline)	17, 16, 18	17.00	—
Fat markdown (#204 as-authored)	15, 13, 15	14.33	−2.67
Slim markdown (H3 + takeaway sentence)	14, 17, 18	16.33	−0.67
This PR (figtext form)	17, 16, 18	17.00	±0.00

Re-tested against latest main (post-#236 concat improvements, post-laser-mcp-#37 H4 splitter) — got 17, 18, 20 → mean 18.33 on the fresh corpus, with run 3 hitting a clean-sweep 20/20. Everything compounds cleanly.

2. tqdm progress-bar noise cleanup (new)

The tutorial notebooks call model.run() with a tqdm progress display; nbconvert captures those mid-run snapshots ("N,NNN agents in M node(s): 37%|... it/s]") into cell outputs. That noise was leaking into three places:

1. Committed notebooks (docs/tutorials/notebooks/*.ipynb) — carried 381 tqdm lines across 14 of 19 files.
2. CI executed_nbs artifact — captured whatever nbconvert saw. MkDocs Deploy and Build Combined Doc both consume this artifact, so both were displaying / ingesting noise.
3. Historically fixed at concat time (concat_mkdocs: broaden tqdm strip to catch all tqdm variants #229's regex in docs/concat_mkdocs.py) — but that only fixed the RAG corpus, not the docs site or the artifact.

New: docs/tqdm_strip.py owns the shared regex + a small library + CLI.

• Wired into docs/execute_notebooks.py — the CI artifact now ships clean by construction, so the docs site (via mkdocs-ghp's overlay) and RAG corpus (via Build Combined Doc's concat) both benefit without further changes on their side.
• docs/concat_mkdocs.py imports the shared regex — keeps its own filter as a defensive second pass but no longer duplicates the pattern.
• Applied to all 19 committed notebooks: 381 lines removed across 14 files.
• New pytest gate tests/test_notebooks_no_tqdm_crud.py fails the existing check matrix entry if any committed notebook has tqdm output. No new workflow — plugs into the existing github-actions.yml check job.

Files changed

• tools/apply_figtext_captions.py — new tool for the figtext transformation
• tools/plot_descriptions/ — 70 description .md files + config.json manifest
• docs/tqdm_strip.py — new shared module + CLI
• docs/execute_notebooks.py — post-execute strip step
• docs/concat_mkdocs.py — imports shared regex
• tests/test_notebooks_no_tqdm_crud.py — new CI gate
• 18 tutorial notebooks — figtext calls added to plot code cells + tqdm crud stripped from outputs

Closes

Supersedes and replaces #204 (already closed).

[jonathanhhb]

Alpha suite re-test: rebased #240 vs prior baselines

Ran the 20-prompt alpha suite N=3 against a freshly-built corpus from this branch (cherry-picked onto latest main, so it now composes with #236 concat improvements and post-#37 laser-mcp ingest). Corpus fingerprint: 515 sections / 991 chunks (identical to the earlier figtext test).

Per-iteration:

Run	passed	a1	a2	a3	never	Fails
1	17/20	5	8	4	3	p06, p09, p16
2	18/20	6	8	4	2	p05, p09
3	20/20	8	9	3	0	—
mean	18.33	6.33	8.33	3.67	1.67	—

Vs baselines this session:

Arm	Mean passed	Mean a1
No descriptions (without204)	17.00	9.00
Figtext (earlier test)	17.00	9.00
Figtext rebased (this test)	18.33	6.33

Run 3 was a clean sweep (20/20 first time this session). The +1.33 improvement over the earlier figtext test likely reflects the RAG-quality improvements that landed since — laser-generic #236 (concat heading demote + oversize warning) and laser-mcp #37 (H4 splitter in the MarkdownHeaderTextSplitter) are both live now, and both compound with the figtext form's chunk-fusion property.

Green light to merge from an alpha-suite standpoint.

[Copilot]

Pull request overview

This PR improves the cleanliness and retrieval quality of the tutorial notebook corpus by (1) embedding plot descriptions directly into plotting code via plt.figtext(...) and (2) stripping tqdm progress-bar noise from committed notebooks and CI-executed notebook artifacts via a shared docs/tqdm_strip.py utility.

Changes:

• Added a shared tqdm-stripping module + CLI (docs/tqdm_strip.py) and integrated it into notebook execution and concat tooling.
• Added a CI gate to prevent tqdm output from being committed into docs/tutorials/notebooks/*.ipynb.
• Updated tutorial notebooks to remove tqdm output noise and add plt.figtext(...) captions near plots.

Reviewed changes

Copilot reviewed 23 out of 23 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
tools/apply_figtext_captions.py	New script to inject plt.figtext(...) captions into notebook code cells based on the plot description manifest.
docs/tqdm_strip.py	New shared regex + library + CLI to remove tqdm progress-bar snapshot lines from notebook outputs.
docs/execute_notebooks.py	Runs strip_notebook_file(...) post-nbconvert so CI executed_nbs artifacts ship clean.
docs/concat_mkdocs.py	Imports shared tqdm regex from docs/tqdm_strip.py instead of duplicating it.
tests/test_notebooks_no_tqdm_crud.py	New pytest gate ensuring committed tutorial notebooks contain no tqdm progress output lines.
docs/tutorials/notebooks/SEI_and_SEIS_implementations.ipynb	Removes tqdm output noise; adds plt.figtext(...) plot captions.
docs/tutorials/notebooks/seasonality.ipynb	Removes tqdm output noise; adds plot captions (noted helper/call-site caption placement issues in review).
docs/tutorials/notebooks/ri_exploration.ipynb	Removes tqdm output noise; adds plt.figtext(...) plot captions.
docs/tutorials/notebooks/mortality.ipynb	Removes tqdm output noise; adds plt.figtext(...) plot captions.
docs/tutorials/notebooks/grid_examples.ipynb	Removes tqdm output noise; adds plt.figtext(...) plot captions.
docs/tutorials/notebooks/distributions.ipynb	Adds plt.figtext(...) plot captions to distribution sampler plots.
docs/tutorials/notebooks/constant_pop.ipynb	Removes tqdm output noise; adds plt.figtext(...) plot caption.
docs/tutorials/notebooks/births.ipynb	Removes tqdm output noise; adds plt.figtext(...) plot captions.
docs/tutorials/notebooks/10_England_and_Wales.ipynb	Removes tqdm output noise; adds plt.figtext(...) plot captions; fixes a spelling issue (“vicinity”).
docs/tutorials/notebooks/09_rabies_diffusion_1D.ipynb	Reduces tqdm output noise; adds plt.figtext(...) plot captions.
docs/tutorials/notebooks/08_2patch_SIR_wbirths_correlation.ipynb	Adds plt.figtext(...) plot caption to the coupling/correlation plot cell.
docs/tutorials/notebooks/07_SIR_CCS.ipynb	Adds plt.figtext(...) plot captions in CCS-related plot cells.
docs/tutorials/notebooks/06_SIR_wbirths_natural_periodicity.ipynb	Adds plt.figtext(...) plot captions for periodicity plots.
docs/tutorials/notebooks/05_SIR_wbirths_age_distribution.ipynb	Removes tqdm output noise; adds plot labeling + plt.figtext(...) captions.
docs/tutorials/notebooks/04_SIR_nobirths_outbreak_size.ipynb	Removes tqdm output noise; adds plt.figtext(...) plot captions.
docs/tutorials/notebooks/03_SIS_nobirths_logistic_growth.ipynb	Removes tqdm output noise; adds plt.figtext(...) plot captions and plot formatting tweaks.
docs/tutorials/notebooks/02_SI_wbirths_logistic_growth.ipynb	Removes tqdm output noise; adds plt.figtext(...) plot captions and consolidates fit plots into subplots.
docs/tutorials/notebooks/01_SI_nobirths_logistic_growth.ipynb	Removes tqdm output noise; adds plt.figtext(...) plot captions and consolidates beta-fit plots into subplots.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[jonathanhhb]

Fixed in 93d7eb8. inject_before_show now checks isinstance(source, list) on the way in and preserves that shape on the way out — string in, string out; list in, list out. Removes the diff churn on re-runs against string-form cells.

[jonathanhhb]

Legitimate rendering bug — same root cause across all six comments on this file. Fixed in 93d7eb8.

Root cause

The apply_figtext_captions tool injects plt.figtext(...) just before the last plt.show() in the target code cell. In cells 5 (plot_model) and 23 (plot_si_model), that last plt.show() sits inside the helper function — so:

1. Every call to plot_model / plot_si_model used the same caption (wrong content for subsequent calls).
2. Cells 9/14/19/27 tried to compensate by placing plt.figtext AFTER the plot_model(...) call, but the helper had already called plt.show() internally — so the caption attached to a fresh empty figure and didn't render on the intended plot at all.

Fix

Gave each helper a caption=None parameter, rendering figtext inside the helper right before plt.show() when caption is passed. Each call site now passes the appropriate caption via kwarg; the trailing standalone plt.figtext lines were removed. Six cells updated (5, 9, 14, 19, 23, 27).

Also fixed a subtle escape bug I hit in the first attempt: my regex captured the source-form of the caption string (escaped backslashes) and re-repr'd it, doubling the escapes and breaking LaTeX math ($\\beta(t)$ instead of $\beta(t)$). Second pass uses ast.literal_eval to get the interpreted Python value before repr.

[jonathanhhb]

Same root cause as the first plot_model/plot_si_model comment on this file — fixed in 93d7eb8 via the shared caption=None parameter on the helper. See the reply on that thread for the full mechanism.

[jonathanhhb]

Same root cause as the first plot_model/plot_si_model comment on this file — fixed in 93d7eb8 via the shared caption=None parameter on the helper. See the reply on that thread for the full mechanism.

[jonathanhhb]

Same root cause as the first plot_model/plot_si_model comment on this file — fixed in 93d7eb8 via the shared caption=None parameter on the helper. See the reply on that thread for the full mechanism.

[jonathanhhb]

Same root cause as the first plot_model/plot_si_model comment on this file — fixed in 93d7eb8 via the shared caption=None parameter on the helper. See the reply on that thread for the full mechanism.

[jonathanhhb]

Same root cause as the first plot_model/plot_si_model comment on this file — fixed in 93d7eb8 via the shared caption=None parameter on the helper. See the reply on that thread for the full mechanism.