Repository navigation
Implement the design: rules, engine, both directions, MCP server - #1
Merged
Merged
Conversation
- uv-managed `dualect` package (Python 3.11+) with ruff, mypy, pytest dev group - `dualect` CLI: verify and `rules list|new` work; scan, context and translate exit 3 with "not implemented yet" - JSON Schema for rule.yaml plus a validator for folder conventions (id matches folder, paired cases, status vs files, related ids exist) - rule scaffolder that writes TODO markers the test suite rejects - verify harness: runs .m through oct2py and .py in-process, compares every variable with numpy.allclose; index/ignore directives; logical vs numeric is a mismatch; a pair that compares nothing fails - Octave-dependent tests marked and skipped when Octave is absent - GitHub Actions workflow installing Octave from the Ubuntu apt package - design.md section 12: open questions recorded as decisions; verify semantics Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
All rules listed under milestone 2 in docs/design.md section 11 (the list there names 26), each with rule.yaml (why/pitfalls), status guided, and 37 hand-written .m/.py case pairs in total: - syntax: comments, line-continuation, command-syntax - indexing: one-based, ranges, end-in-index, logical, grow-on-assign, colon-all - operators: matrix-multiply, transpose, power, not-equal, short-circuit - control-flow: for-columns, while, if-elseif, switch - functions: definition, multiple-returns, nargin - builtins: zeros-ones-eye, linspace, size-numel-length, sum-mean-max, find Every pair is executed in Octave and Python by a parametrized test and compared with numpy.allclose. Edge cases target the pitfalls (column-major mask order, linspace with one point, zeros(n) being square, argument copy semantics, captured values in anonymous functions, NaN in max). Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Parse .m files with tree-sitter-matlab, lower them into the shared IR, infer
shapes, and tag every statement with the constructs it uses. `dualect scan`
lists the rules a file needs (with locations) and names every construct no
rule covers; `dualect context` prints exactly those rules (rule.yaml plus up
to two cases) as LLM-ready context.
- dualect/source.py: the grammar knows only the MATLAB dialect and silently
misreads `do`/`until` and `n++`, so Octave-only syntax (# comments, #{ #},
!=, !x, endfunction & co, +=, ++, do ... until) is rewritten in front of
it, outside strings and comments, without moving lines; each use is
recorded so --dialect matlab can reject it.
- dualect/lower.py: tree -> IR, with name resolution (index vs call) and
operator chains rebuilt by precedence, because the grammar nests -2^2,
a == b & c and a.^b' wrongly. Unparsable text becomes parse:error.
- dualect/ir.py: the direction-neutral IR (index base/order and range
inclusivity are explicit fields).
- dualect/infer.py: flow-sensitive shape/type facts, loops to a fixed
point, local functions specialized on their call sites.
- dualect/constructs.py + rule.yaml `match`: rules declare the construct
tags they cover; the validator rejects unknown or doubly-claimed tags.
- Nine new rules the scanner showed every file needs (array literals,
strings, arithmetic, division, augmented assignment, element-wise math,
constants, isempty, display), each with cases verified against Octave.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
`dualect translate file.m -o file.py [--strict] [--dialect ...] [--no-verify]` runs the whole pipeline: parse, scan, plan each statement, translate it with its rules' translate.py, emit with imports and ruff formatting, verify by running both sides, and report every statement with its rules, the status used and whether the file verified. - dualect/emit.py: a statement is translated only when every construct it uses has a mechanical rule; otherwise (or when a translator declines a form it cannot translate correctly) it becomes a TODO(dualect) hole that raises NotImplementedError and is reported. The engine owns what all rules share: copy-on-write (.copy() where an alias is later modified in place) and position variables stored 0-based when they are only read in subscripts. - dualect/pipeline.py, pyformat.py, translators.py: assembly, ruff formatting, the verdict on the output's first line (verified / UNVERIFIED: why), exit codes 0/1/4, and loading of per-rule translators. - 30 of 35 rules are now mechanical: each translate.py reproduces every case's hand-written .py (layer 2, comments ignored) and the translation verifies against Octave. Guided: multiple-returns, nargin, grow-on-assign, display, command-syntax (reasons in docs/design.md section 11). - Hand-written cases changed to the equally idiomatic statement-by-statement form: while (x = 1, was 1.0), power (x**-1.0 on an int array), logical (z = x; z = z[~(z < 0)]), colon-all (C = A.copy(); C[:] = ...reshape). - Never silently wrong: int-dtype stores of fractions are declined, max/min of data not known NaN-free use nan-skipping forms, shape guesses carry a "# dualect: assumed" comment, and a loop no longer hides that an array assigned inside it was undefined before (assign:grow). - corpus/: ode_rk4.m translates completely and verifies; beam_deflection.m is 29/30 statements (fprintf is guided) and signal_filter.m 29/33 (filter() has no rule); both are reported, not hidden. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Adversarial probes (tests/test_adversarial.py) found translations that ran but computed something else; each is now correct or a named hole: - Lambdas are annotated with the join of every call's argument facts, not the last call's; a row/column mix becomes a vector of unknown orientation. - Products and element-wise operations on such a vector are declined instead of an assumed `@` that is wrong for one of the calls. - Arithmetic on two logical operands converts to numbers (NumPy kept bool + bool as a logical OR, bool * bool as an AND). - Integer overflow: `.^` with a large or unknown exponent and `2 .^ n` are computed in floats; prod of integers uses dtype=float. - mod/rem by a known zero are declined, by an unknown divisor assumed. - `for c = A` copies A.T when the body writes into c; a loop over a range known to be empty is declined. - `2.^x` was read as the number `2.` and a matrix power; the normalizer now keeps the dotted operator whole. - Assumption comments no longer leak from a body onto the enclosing header. - Horizontal concatenation of columns is np.column_stack (new case). mypy now covers rules/**/translate.py (tools/mypy_rules.py), in CI too. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
- dualect/pyinfer.py: facts over the Python ast (1-D/2-D, dtype, NaN-free,
non-negative, may-be-a-view) that lifting depends on.
- dualect/pyconstructs.py: py:-prefixed construct tags; unknown Python
constructs are reported as unsupported by tag.
- dualect/lift.py: per-statement planning and holes as in the other
direction; rules lift Python nodes into the IR with .m semantics; the
engine owns comments, assignment and NumPy aliasing.
- dualect/memit.py: prints the IR as .m code (Octave 8 / R2016b safe).
- One rule folder serves both directions: rule.yaml gains
from_python: {status, match}; translate.py gains lift() and friends.
Status is tracked per direction: 29 rules mechanical from Python, 3 guided,
3 with no Python construct.
- translate/scan/context pick the direction from the source suffix;
`rules list` shows both statuses.
- Declined, never imitated: writes through views/aliases, writes into array
arguments, module variables read in a def, late-binding lambdas, names
that hide functions, int/bool store conversions, bool arithmetic, NaN in
max/argmax/maximum/clip, np.round halves, linspace(a, b, 1), ~ on ints.
- tests/test_reverse.py: every case .py lifts or reports its holes, complete
ones verify against their .py; the corpus round-trips .m -> .py -> .m.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
- dualect/mcp_server.py: `dualect mcp` serves scan, get_rule, translate and verify over stdio on the official mcp SDK (2.x MCPServer); it never calls an LLM API. Tests call the tools in-process. - dualect/promotion.py, `dualect rules check`: runs every rule's translator on its cases in each direction, checks the layer-2 match and execution, and fails when a mechanical rule does not qualify or a guided one does. From Python, layer 2 is a structural IR comparison, reported but not required. Wired into tests/test_milestone6.py (67 rule-directions pass). - dualect/llmstxt.py, `dualect llms-txt`: llms.txt with one line per rule; a test fails when the committed file is stale. - Runner fix found by the check: change directory before clearing the Octave workspace, so a deleted temporary folder cannot break later runs. - README, CLAUDE.md and docs/design.md updated for both directions. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2 of 3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements milestones 1–6 of
docs/design.md.What is in here
scan,context,translate(both directions,--strict,--dialect),verify,rules list|new|check,llms-txt,mcp..m/.pycase pairs, each executed in both languages..m→ Python: 30 mechanical, 5 guided. Python →.m: 29 mechanical, 3 guided, 3 with no Python construct.translate.py, verification by execution, per-statement report.forcolumn,2.^x), with regression tests.ode_rk4.mtranslates fully and verifies in both directions;beam_deflection.mandsignal_filter.mare incomplete (fprintf,filter()have no rule) and are reported as unverified.llms.txt.Known gaps
x .^ 2on integers stays integer and can overflow for large values.forkptydeprecation warnings from the MCP worker thread.Test plan
uv run pytest(689 passed locally, Octave 11)uv run ruff check .anduv run ruff format --check .uv run mypy dualectanduv run python tools/mypy_rules.py🤖 Generated with Claude Code