oxmera 0.5.2 — the autograd round - #102
Merged
Merged
Conversation
…s, typed refusals - A tape is a linked structure and its derived drop recursed once per node, overflowing the stack and ABORTING the process at ~20k operations in debug and ~65k in release. An abort cannot be caught, so an unrolled RNN simply killed the program. AutogradMeta now dismantles the graph with an explicit worklist; a 200,000-node tape builds and drops cleanly (#96). - matmul's single-row path skipped the multiply when the left operand was 0.0, so 0 * inf never happened there while the four-row block produced NaN: one call returned NaN for some rows and 0.0 for others of the same data. Both paths do the same arithmetic now (#98). - max/min reductions propagate NaN. f32::max returns the non-NaN operand, so a NaN that sum propagates and argmax refuses disappeared through the op on the common path through pooling and attention (#97). - try_zeros/try_ones/try_full allocate with try_reserve_exact instead of vec![v; n], which aborts the process when the allocator refuses — the one thing a `try_` constructor must never do. eye checks n * n instead of wrapping into an index-out-of-bounds. slice rejects a reversed range rather than saturating it to an empty view (#99). - trace reports trace, not the diag it happens to call. A negative index is reported as negative instead of being clamped to "index [0]", an index that is in bounds. matmul's rank error states the real constraint: rank-4 and rank-5 both work, so "supported ranks are 2 and 3" was wrong (#100). - cholesky's rustdoc records the gradient convention (symmetric perturbations, so an elementwise gradcheck does not apply) and corrects the claim that the backward pass runs in f64 — it takes f32 inputs (#95). Signed-off-by: Vyncint Ng <115854244+vyncint@users.noreply.github.com>
… init - pad_dim validates `dim` before the no-op shortcut. It panicked with a raw index-out-of-bounds on an out-of-range dim, and accepted the same dim silently when before == after == 0; its sibling `cat` has always returned a typed error for this (#99). - kaiming_uniform/xavier_uniform survive a zero fan. 6.0 / 0 is infinite and `rand` refuses a non-finite range by panicking; a layer with no fan has nothing to spread, so the bound collapses to zero (#99). - check_param_dtype's rustdoc names the error it actually returns, InvalidArgument, rather than DTypeMismatch (#101). Signed-off-by: Vyncint Ng <115854244+vyncint@users.noreply.github.com>
…0.5.2 - oxmera-tensor's crate doc and register_backend still taught load-time registration. 0.5.0 removed the pre-main constructor and updated oxmera-runtime; this is the crate a user actually reads for the registry, and following it produced the BackendUnavailable the docs called impossible (#101). - The README called reshape zero-copy unconditionally; it is a view only when the input is contiguous and copies otherwise. - The top-level --help omitted --json and --seed, which the subcommands' own --help documents and which both work. - Every crate declares [package.metadata.docs.rs], and oxmera-metal names an Apple target: docs.rs builds on Linux, where that crate is an empty stub, so its page documented one function and oxmera::metal led nowhere. The workspace version is 0.5.2 and the CHANGELOG cuts the release. Signed-off-by: Vyncint Ng <115854244+vyncint@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
oxmera 0.5.2 — the autograd round
A second deep test of the published crates, reaching the autograd surface the
first round never got to. Every fix is reproduced by a test that fails without
it. No public API changed, so
semver-checkspasses as a patch.The two that could kill a program
its derived drop recursed once per node: ~20k operations in debug, ~65k in
release, then
fatal runtime error: stack overflow, aborting. An abortcannot be caught.
AutogradMetanow dismantles the graph with an explicitworklist — a 200,000-node tape builds and drops cleanly.
try_zerosaborted on an allocation the allocator refused.vec![v; n]aborts; thetry_prefix exists precisely so the caller canhandle this. Now
try_reserve_exact.Wrong numbers
matmuldisagreed with itself. The single-row path skipped themultiply when the left operand was
0.0, so0 × ∞never happened therewhile the four-row block produced
NaN. One call returned[NaN × 8, 0.0, 0.0]for uniform input.max/minsilently discarded NaN.f32::maxreturns the non-NaNoperand, so a NaN that
sumpropagates andargmaxrefuses vanishedthrough the op on the common path in pooling and attention.
Refusals and messages
eye(n)wrappedn * n;pad_dimpanicked on a baddimandaccepted it silently when padding was zero (its sibling
catalwaysreported it);
kaiming_uniformpanicked insiderandon a zero fan;sliceturned a reversed range into an empty view.tracereported an error namingdiag, which the caller neverwrote;
index_selectreported a negative index asindex [0], an indexthat is valid;
matmulclaimed "supported ranks are 2 and 3" while rank-4and rank-5 work.
oxmera-tensor's rustdoc still taught load-time registration(0.5.0 removed it and updated only
oxmera-runtime);check_param_dtypepromised the wrong error; the README called
reshapezero-copyunconditionally; top-level
--helpomitted--jsonand--seed; no cratehad docs.rs metadata, so
oxmera-metaldocumented as its Linux stub.#95 — documented, not changed
The
cholesky/logdet/detgradient looked wrong against an elementwisefinite difference: exactly 0.5× below the diagonal and nonzero above it, and
the crate's own
gradcheckrejected all three.I started to "fix" it to a one-sided gradient — and the suite caught me. The
test at
gradcheck.rsassertsd logdet/dA = A⁻¹for a symmetricA,checked independently against
eigh. The symmetrized gradient is adeliberate, verified contract, matching PyTorch; a one-sided gradient
would silently break anyone optimising over symmetric matrices, whose updates
must stay symmetric.
So the defect was that this was written down nowhere.
Tensor::choleskynowstates the convention, why an elementwise
gradcheckcannot apply to thesethree ops, and what to do instead. It also corrects a second claim in the same
paragraph: the backward pass takes
f32inputs, so anf64gradient carriesf32precision — it saidf64.Closes #95
Closes #96
Closes #97
Closes #98
Closes #99
Closes #100
Closes #101