Python: DSPy adapter backed by DSRs (dsrs-dspy, PyO3)

[darinkishore]

What

Adds crates/dspy-py — a PyO3 extension + pure-Python shim that makes DSRs's BAML-style schema rendering and JSONish parsing available to DSPy as a drop-in ChatAdapter replacement (dsrs_dspy.DSRSBAMLAdapter).

Since DSRs isn't published, the package installs straight from git — no PyPI involved, wheel builds from source via maturin:

uv add "dsrs-dspy @ git+https://github.com/krypticmouse/DSRs@main#subdirectory=crates/dspy-py"

import dspy
from dsrs_dspy import DSRSBAMLAdapter

dspy.configure(lm=dspy.LM("openai/gpt-5.2"), adapter=DSRSBAMLAdapter())

Why

• BAML's compact schema rendering + JSONish's error-tolerant parsing measurably beat stock ChatAdapter formatting on structured outputs (this adapter ran in production on a financial-workbook classification pipeline).
• Parsing tolerates real LLM output: trailing commas, single quotes, Python literals, [[ ## field ##]] marker whitespace variants. No silent fallback re-call — failures surface as AdapterParseError.
• Gives DSRs a Python on-ramp: DSPy users get the Rust rendering/parsing without leaving Python.

Commits

• style: cargo fmt — fmt --check was red on main (one hunk in test_predict_lm_override.rs).
• feat(jsonish): Python literals — True/False/None map to true/false/null in the fixing parser (LLMs emit Pythonish JSON constantly; includes test).
• feat(bamltype): export SchemaRegistry — dspy-py compiles JSON Schema from DSPy/pydantic signatures directly, so it needs the registry schema_builder uses internally.
• feat(dspy-py): the crate — abi3-py39 (one wheel per platform covers CPython ≥ 3.9), module-name = "dsrs_dspy._dsrs_dspy" so one wheel ships both the Python package and the native module.
• ci,docs — new python-adapter job builds the wheel and runs the Python tests against a real DSPy install; miri excludes dspy-py (can't interpret embedded CPython); README section.

Testing

• cargo test --all-features — full workspace green (73 test binaries), including dspy-py's 8 Rust tests: unit + property (random schema fuzzing with JSONish noise, description propagation, and a matrix driven by real pydantic.TypeAdapter(...).json_schema() output).
• crates/dspy-py/tests/python/ — 5 end-to-end tests against released DSPy 3.2.1: render, noisy parse, pydantic default restoration, missing-field error, chat message formatting. No network, no LM calls.
• Verified the exact user install path (git+…#subdirectory=crates/dspy-py) from the committed state, then imported and exercised the adapter.
• cargo clippy -p dspy-py --all-targets --all-features — clean (remaining warnings are pre-existing in vendored crates).

Notes for reviewers

• cargo build at the workspace root now compiles dspy-py, which needs a python3 on PATH for pyo3's build script (present on GH runners and dev machines; if it ever annoys, default-members can gate it out of default builds).
• The adapter touches DSPy-internal surface (ChatAdapter, dspy.adapters.utils.parse_value); the python-adapter CI job exists precisely to catch DSPy-side drift.

🤖 Generated with Claude Code

[darinkishore]

CI status note: MIRI red is pre-existing on main, not from this PR. The failure is fastant (via dspy-rs's tracing deps) executing cpuid inline asm in a static ctor, which miri cannot interpret — the latest main run fails identically on 5bb65ca. This PR's dspy-py crate is already excluded from miri.

Meanwhile Clippy Lint was also red on main (fmt drift in test_predict_lm_override.rs) and is green on this PR thanks to the fmt commit. Build, Test, and the new Python Adapter job are all green.