Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 12 additions & 7 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,21 @@ architecture, the rule format and the milestones.

## Current state

Milestones 1-6 of docs/design.md are done. Both directions work end to end:
`.m` -> Python (dualect/lower.py, infer.py, emit.py; 30 of 35 rules mechanical)
and Python -> `.m` (dualect/pyinfer.py, lift.py, memit.py; 29 mechanical), one
Milestones 1-6 of docs/design.md are done, plus a coverage pass (section 11,
item 7). Both directions work end to end:
`.m` -> Python (dualect/lower.py, infer.py, emit.py; 58 of 61 rules mechanical)
and Python -> `.m` (dualect/pyinfer.py, lift.py, memit.py; 54 mechanical), one
rule folder per construct serving both (`from_python:` in rule.yaml, `lift()` in
translate.py, status per direction). `dualect rules check` enforces that every
translate.py, status per direction; inference facts for a built-in come from
its rule's `infer_call`/`pyinfer_call`). Still guided: functions/nargout,
io/display, builtins/tic-toc. `dualect rules check` enforces that every
status is earned; `dualect mcp` serves scan/get_rule/translate/verify;
`llms.txt` is generated (`dualect llms-txt -o llms.txt`) and must stay current.
Adversarial probes live in tests/test_adversarial.py and tests/test_reverse.py.
Next: more rules (the guided ones, `cumsum`, `filter`, printing), cell arrays
and structs.
Verify compares printed output when a script carries `% dualect: stdout`
(translations that print add it) and random values by shape (`shape r`).
Adversarial probes live in tests/test_adversarial*.py and tests/test_reverse.py.
All six corpus scripts are tested; three translate completely both ways.
Next: cell arrays and structs, `cond`/`ind2sub`-style gaps the corpus shows.
The project site is `docs/index.html` (GitHub Pages from `/docs`, TagBrush.css);
keep its numbers in step with README.md and run TagCheck after editing it.

Expand Down
62 changes: 39 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ the same way.

Site: https://kovirlabs.github.io/dualect/ · Design: [docs/design.md](docs/design.md)

> **Status: early (0.1.0).** 35 rules. `.m` → Python: 30 mechanical, 5 guided.
> Python → `.m`: 29 mechanical, 3 guided. Details under [Status and limits](#status-and-limits).
> **Status: early (0.1.0).** 61 rules. `.m` → Python: 58 mechanical, 3 guided.
> Python → `.m`: 54 mechanical, 1 guided. Details under [Status and limits](#status-and-limits).

## Example

Expand Down Expand Up @@ -118,7 +118,7 @@ The `.m` side may use Octave syntax (`#` comments, `!=`, `++`, `+=`,

`translate` prints one line per statement: line and column, kind, the status
used, whether it was verified, and the rules it used. Under a statement it could
not translate, it prints why, for example `needs call:filter (no rule)` or
not translate, it prints why, for example `needs call:cond (no rule)` or
`translator declined: max of a value whose shape is unknown or along a singleton dimension`. Then come a
summary (`30 statements: 29 mechanical, 1 guided, 0 unsupported`), the
per-variable comparison when verification ran, and a `verified:` line.
Expand All @@ -128,7 +128,9 @@ An untranslated statement becomes a hole in the output: in Python, a
`raise NotImplementedError(...)`; in `.m`, a `% TODO(dualect)` comment followed
by `error(...)`. The output's first line then says `UNVERIFIED` and why.
`# dualect: assumed ...` marks an assumption the engine could not prove (for
example a real `sqrt` argument).
example a real `sqrt` argument). A translation that prints carries
`# dualect: stdout`, so verification also compares the printed text exactly;
`# dualect: shape r` compares random values by shape only.

| Exit code | `translate` | `verify` |
|---|---|---|
Expand Down Expand Up @@ -201,26 +203,40 @@ Desktop, ...) looks like:

## Status and limits

From `uv run dualect rules list`: 35 rules with 49 case pairs.

- `.m` → Python: 30 mechanical; guided: `functions/multiple-returns`,
`functions/nargin`, `indexing/grow-on-assign`, `io/display`,
`syntax/command-syntax`.
- Python → `.m`: 29 mechanical; guided: `functions/nargin`,
`indexing/grow-on-assign`, `io/display`; 3 rules cover `.m`-only constructs.

The [`corpus/`](corpus) scripts, `.m` → Python:

| Script | Statements | Translated | Verified |
From `uv run dualect rules list`: 61 rules with 88 case pairs.

- `.m` → Python: 58 mechanical; guided: `functions/nargout` (a body that
depends on how many outputs were requested), `io/display` (a missing `;`:
the display format depends on the value) and `builtins/tic-toc` (timing).
- Python → `.m`: 54 mechanical; guided: `builtins/tic-toc`; 6 rules cover
`.m`-only constructs.

Built-ins with rules include printing (`printf`, `fprintf`, `sprintf`, `disp`,
`num2str`), `filter` (as `scipy.signal.lfilter`), reductions and statistics
(`sum mean max min prod cumsum cumprod diff any all std var median`), ordering
(`sort` with its index output, `unique`, `flip`), shape (`reshape repmat cat
horzcat vertcat meshgrid`), linear algebra (`norm dot cross kron inv det trace
rank`, `\` for square and tall systems), `interp1`, `polyfit`, `polyval`,
`trapz`, `rand`/`randn` (compared by shape only: no two runs share values) and
the element-wise math functions; `llms.txt` lists them all.
[NumPy for MATLAB users](https://numpy.org/doc/stable/user/numpy-for-matlab-users.html)
served as the checklist. Forms a rule cannot translate exactly are declined
by name (for example `%d` of a fraction, `unique` with several outputs,
`interp1` with `'spline'`).

The [`corpus/`](corpus) scripts:

| Script | `.m` → Python | Verified | Back to `.m` |
|---|---|---|---|
| `ode_rk4.m` | 23 | 23 | yes |
| `beam_deflection.m` | 30 | 29 (`fprintf` is guided) | no |
| `signal_filter.m` | 33 | 29 (`filter` has no rule; two `max` calls depend on it) | no |

The Python that dualect writes for `ode_rk4.m` translates back to `.m` and verifies.

Not supported yet: cell arrays, structs, plotting, random numbers, file I/O and
most built-ins beyond the rules listed in `llms.txt`. Out of scope for now:
| `ode_rk4.m` | 23/23 | yes | 21/21, verified |
| `beam_deflection.m` | 30/30 | yes | 27/27, verified |
| `signal_filter.m` | 33/33 | yes | 29/29, verified |
| `least_squares_fit.m` | 26/27 (`cond` has no rule) | no | holes carried over |
| `data_statistics.m` | 27/28 (`round` of an expression is declined) | no | holes carried over |
| `grid_heat.m` | 20/26 (`trapz` along a dimension, `ind2sub`, shapes lost after a slice) | no | holes carried over |

Not supported yet: cell arrays, structs, plotting, file I/O and built-ins
beyond the rules listed in `llms.txt`. Out of scope for now:
`classdef`, GUIs, Simulink, MEX and toolboxes. Verification shows the two
programs agree when run as written, not on every possible input.

Expand Down
35 changes: 35 additions & 0 deletions corpus/data_statistics.m
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
% Summary statistics of a data matrix: one column per sensor, one row per
% sample (deterministic data, so both sides see the same numbers).
n = 12;
k = (1:n)';
D = [20 + 2*sin(k), 15 + 0.5*k, 30 - cos(3*k).^2, mod(7*k, 5)];

col_mean = mean(D);
col_std = std(D); % N-1 normalisation
col_var = var(D, 1); % N normalisation
col_median = median(D);
[col_max, row_of_max] = max(D);
[col_min, row_of_min] = min(D);
spread = col_max - col_min;

% Standardise each column (implicit expansion), then find outliers.
Z = (D - mean(D)) ./ std(D);
outlier = abs(Z) > 1.5;
n_outliers = sum(outlier(:));
any_outlier = any(outlier);
all_positive = all(D > 0);

% Running totals and ranks of the second sensor.
running = cumsum(D(:, 2));
[sorted2, order2] = sort(D(:, 2), 'descend');
top3 = order2(1:3);

% Rounded values and how many distinct levels the last sensor takes.
levels = unique(D(:, 4));
n_levels = numel(levels);
D_round = round(D * 10) / 10;

for j = 1:size(D, 2)
fprintf('sensor %d: mean %8.3f std %7.3f median %8.3f\n', ...
j, col_mean(j), col_std(j), col_median(j));
end
33 changes: 33 additions & 0 deletions corpus/grid_heat.m
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
% Steady temperature on a unit square: a hot top edge, cold elsewhere,
% solved by Jacobi iteration on a grid, then integrated with trapz.
nx = 21;
ny = 11;
x = linspace(0, 1, nx);
y = linspace(0, 1, ny);
[X, Y] = meshgrid(x, y); % ny-by-nx: rows follow y, columns follow x

T = zeros(ny, nx);
T(end, :) = 100; % the top edge (y = 1) is hot

for iter = 1:400
T_old = T;
T(2:end-1, 2:end-1) = 0.25 * (T_old(1:end-2, 2:end-1) + T_old(3:end, 2:end-1) ...
+ T_old(2:end-1, 1:end-2) + T_old(2:end-1, 3:end));
end
change = max(max(abs(T - T_old)));

% Mean temperature over the square, by the trapezoidal rule in both directions.
inner = trapz(x, T, 2); % along x, for each row
mean_T = trapz(y, inner);

% The centre line, its gradient, and the hottest interior point.
centre = T(:, (nx + 1) / 2);
dT = diff(centre) / (y(2) - y(1));
interior = T(2:end-1, 2:end-1);
[t_max, idx] = max(interior(:));
[row, col] = ind2sub(size(interior), idx);

% A smooth reference field on the same grid, and its distance from T.
F = 100 * sinh(pi * Y) .* sin(pi * X) / sinh(pi);
gap = norm(T - F, 'fro') / sqrt(numel(T));
profile = reshape(T(:, 1:2:end), 1, []);
32 changes: 32 additions & 0 deletions corpus/least_squares_fit.m
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
% Fit a quadratic to noisy measurements, three ways: polyfit, the normal
% equations, and a QR-free backslash solve; compare and summarise.
t = linspace(0, 2, 21);
noise = 0.05 * sin(37 * t) .* cos(11 * t); % deterministic "noise"
y = 1.5 - 0.8 * t + 0.6 * t.^2 + noise;

% 1) polyfit / polyval
p = polyfit(t, y, 2);
y_fit = polyval(p, t);

% 2) the same model by least squares on a design matrix
A = [ones(numel(t), 1), t', (t.^2)'];
c = A \ y';
c_normal = (A' * A) \ (A' * y');

% Residuals and goodness of fit
r = y - y_fit;
ss_res = sum(r.^2);
ss_tot = sum((y - mean(y)).^2);
r_squared = 1 - ss_res / ss_tot;
rms_err = sqrt(mean(r.^2));
coef_diff = max(abs(fliplr(p) - c'));
cond_A = cond(A);

% Prediction at a few new points, and where the fitted curve is lowest
t_new = [2.5 3.0];
y_new = polyval(p, t_new);
[y_low, k_low] = min(y_fit);
t_low = t(k_low);

fprintf('fit: y = %.4f + %.4f t + %.4f t^2\n', p(3), p(2), p(1));
fprintf('R^2 = %.6f, rms = %.3e\n', r_squared, rms_err);
42 changes: 40 additions & 2 deletions docs/design.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# dualect — design

Status: 2026-11-13. This document is the plan; section 11 tracks what is built.
Status: 2026-11-14. This document is the plan; section 11 tracks what is built.

## 1. Problem

Expand Down Expand Up @@ -266,7 +266,21 @@ join of the argument facts at its call sites in the file.
(the source leaves the variable as a `1x0` array). The normalizer turns
`2.^x` into `2 .^x`: the grammar read it as the number `2.` and a matrix
power. Horizontal concatenation of columns and matrices is
`np.column_stack`.
`np.column_stack`. Squares overflow too (an int64 holding 3037000500 has a
square beyond 2**63), so every exponent of 2 or more on an integer array,
or on an element read out of one (`np.int64`), is a float (`x**2.0`), and
an integer matrix power works in floats (`A.astype(float)`).
- **Built-ins teach the analysis themselves.** A rule's `translate.py` may
define `infer_call(node, args, nargout)` and `infer_outputs(...)` (what a
built-in returns, as `Info`), and `pyinfer_call(tag, node, args, keywords,
receiver)` for the Python direction; `dualect/infer.py` and
`dualect/pyinfer.py` ask the rule covering a call before their own tables.
So a new built-in is still one folder, facts included. A call made only for
its effect (`fprintf(...)`) may need several statements (a loop that
recycles a format): its rule's `translate_call_statement` provides them.
A store the scanner tagged `assign:grow` goes to indexing/grow-on-assign's
`translate_store`, and `a, b = <built-in>(...)` from Python to the claiming
rule's `lift_multi`.
- **Output.** ruff formats the file (its default style, so the result does not
depend on the caller's configuration). The first line states the verdict:
`verified against the .m source`, or `UNVERIFIED: <reason>` (holes, no
Expand Down Expand Up @@ -473,6 +487,23 @@ good seed for `builtins/*` rules (link, don't copy).
directory, failing every later script; the runner now changes directory
before clearing the workspace.

7. **Coverage** (done): 61 rules, 88 case pairs. `.m` -> Python 58
mechanical, 3 guided (`functions/nargout`, split out of
multiple-returns; `io/display`, now only a missing `;`; `builtins/tic-toc`);
Python -> `.m` 54 mechanical, 1 guided, 6 `.m`-only. New: `io/printf`
(formats as f-strings, recycling carried out statically or as a loop),
`builtins/filter` (`scipy.signal.lfilter`, SciPy added as a dependency),
`types/double`, 22 built-in rules chosen from NumPy's "NumPy for MATLAB
users" page, and per-rule inference hooks (section 5.4). Promoted to
mechanical: `functions/multiple-returns`, `functions/nargin` (both ways),
`indexing/grow-on-assign` (both ways) and `syntax/command-syntax`
(display-only commands dropped with a comment, `clear a b` is `del`).
Verification compares printed output on request and random values by
shape (section 12.1). All six corpus scripts run in the tests;
`beam_deflection`, `signal_filter` and `ode_rk4` translate completely and
verify both ways; `least_squares_fit` (26/27), `data_statistics` (27/28)
and `grid_heat` (20/26) report their holes by name.

## 12. Decisions

Settled by the owner on 2026-11-12 (these were the open questions of the first draft):
Expand Down Expand Up @@ -511,6 +542,13 @@ How `dualect verify a.m a.py` and the rule-case tests decide two scripts agree:
`.m`, 0-based in Python, compared as `m - 1`) and `% dualect: ignore k`
(e.g. a loop variable whose final value legitimately differs). Unknown
directives are errors, so a typo cannot silently weaken a check.
- Printing is compared only on request, `% dualect: stdout`: then the text
both scripts print must be identical, character for character (the `.m`
side is captured with `evalc`, which keeps trailing spaces and a last line
without a newline). Most scripts print incidentally, so it is opt-in; a
translation that prints (rule `io/printf`) writes the directive into its
output itself. `% dualect: shape r` compares `r` by shape and
logical-ness only, for random values (`rand`) that no two runs share.
- `.m` case scripts that define functions start with `1;` and define them before
use: Octave does not accept functions at the end of a script.
- A case file must not be named like a function it calls (`round.m`): the
Expand Down
Loading
Loading