oxmera 0.5.1 — ten defects found by deep-testing the published 0.5.0 - #94
Merged
Merged
Conversation
- `maximum`/`minimum` split a tie's gradient evenly instead of handing all of it to one operand. Every composite built on them was wrong at a tie: BCEWithLogitsLoss is `max(x, 0) - x*t + ...`, so at a logit of exactly 0 it returned `-t` rather than `sigmoid(0) - t`, and one SGD step on a balanced batch raised the loss. This is the convention LIMITATIONS.md already documented. `relu'(0) = 0` is untouched — relu has its own VJP and never went through `maximum` (#84). - `Tensor::from_storage` computes its bounds with checked arithmetic. The 0.5.0 check overflowed isize: it panicked inside a Result-returning constructor in debug and accepted an out-of-bounds layout in release, where the first read then indexed far past the buffer. Any overflow is now "cannot be proven in bounds", which is a typed refusal (#85). - Batched matmul returns the empty tensor when the output has a zero-size dimension instead of handing rayon a chunk size of zero and panicking; the rank-2 path already did. A zero inner dimension still produces the zero-filled result it should (#86). - `mean` on an f64 tensor divides in f64. Scaling the sum by an f32 reciprocal gave every f64 mean f32 precision: mean([1, 2, 3]) came back 2.0000000596046448 (#87). Signed-off-by: Vyncint Ng <115854244+vyncint@users.noreply.github.com>
…a group - Adam and AdamW correct bias with each parameter's own update count. A single global counter mis-scaled the first update of any parameter whose gradient arrived late: two parameters given identical first gradients moved by 0.100 and 0.074. 0.5.0 made skipping grad-less parameters the normal path, which makes a late first gradient routine (#88). - Per-parameter state grows to match its group. `ParamGroup::params` is public and reachable through `groups_mut()`, so adding a parameter between steps is a supported thing to do; indexing the state blindly panicked with an index out of bounds on the next step. A parameter with no slot yet simply has no history, which is what `None` already meant (#89). Signed-off-by: Vyncint Ng <115854244+vyncint@users.noreply.github.com>
…yped - `serialize::load` decodes and validates every parameter before writing any of them. Writing as it walked left a failed load half-applied: the parameters already visited held checkpoint values and the rest held their own, so a caller that handled the error kept training a model from neither source (#90). - `serialize::save` rejects a duplicate parameter name with a typed InvalidArgument naming it, instead of letting safetensors panic on the repeated key. `named_parameters` is written by hand in every downstream Module, so a duplicate is ordinary caller error (#91). Signed-off-by: Vyncint Ng <115854244+vyncint@users.noreply.github.com>
LICENSE-MIT and LICENSE-APACHE lived only at the workspace root, so cargo never packaged them: all twelve 0.5.0 tarballs carried the licence claim `license = "MIT OR Apache-2.0"` without the licence text, which both licences require a redistribution to include. Both files now sit in every crate, and the release workflow refuses to publish a crate whose package listing is missing either one, so this cannot regress silently (#92). Signed-off-by: Vyncint Ng <115854244+vyncint@users.noreply.github.com>
- README and docs/LIMITATIONS.md described the `ctor` pre-`main` constructor that 0.5.0 deleted and restated 0.4.0's "seven crates, 47 -> 40" for `--no-default-features`. Measured with `cargo tree -e normal`: three crates leave (cudarc, libloading, oxmera-cuda) and it is 43 -> 40. - SECURITY.md said "Nothing is released yet" for a project with twelve crates published; docs/STABILITY.md cites that file for the supported- versions policy, so the stale line was load-bearing. - The README's optim row named `SGD` and `RMSprop`; the types are `Sgd` and `RmsProp`, as docs/STABILITY.md already said (#93). The workspace version is 0.5.1 and the CHANGELOG cuts the release. Signed-off-by: Vyncint Ng <115854244+vyncint@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
oxmera 0.5.1 — the defects deep-testing the published 0.5.0 found
0.5.0 was tested as a repository. This release is what happened when it was
tested as a published artifact — a downstream project depending on
oxmera = "0.5.0"from crates.io. Ten defects, each reproduced by runningcode, each now carrying a regression test. No public API changed, so
semver-checkspasses as a patch.Three of these are defects in 0.5.0's own new work, which is the honest
argument for testing the artifact rather than the tree.
Wrong numbers
maximum/minimumgave a tie's whole gradient to one operand (maximum/minimum give a tie's whole gradient to one operand, so BCEWithLogitsLoss is wrong at a logit of 0 #84).BCEWithLogitsLossismax(x, 0) - x·t + …, so at a logit of exactly0.0it returned-tinstead ofsigmoid(0) - t. On a balanced batch thetrue bias gradient is
0, oxmera reported-0.5, and one SGD step raisedthe loss from 0.6931472 to 0.724077. A central finite difference of
oxmera's own forward disagreed with its own autograd. Ties now split
evenly — the convention
docs/LIMITATIONS.mdalready documented.relu'(0) = 0is untouched:reluhas its own VJP.meanon anf64tensor was scaled by anf32reciprocal (mean on an f64 tensor is scaled by an f32 reciprocal, so f64 means carry f32 precision #87), soevery f64 mean carried f32 precision:
mean([1,2,3]) = 2.0000000596046448.parameter whose gradient arrived late had its first update mis-scaled —
identical first gradients produced updates of 0.100 and 0.074. 0.5.0 made
skipping grad-less parameters normal, which makes this routine.
Panics where a typed error or a value was promised
Tensor::from_storage's bounds check overflowedisize(Tensor::from_storage's bounds check overflows isize: panics in debug, accepts out-of-bounds layouts in release #85) — aregression in the check 0.5.0 added for Tensor::from_storage accepts a layout with negative strides and the first read panics #50. It panicked inside a
Result-returning constructor in debug and accepted out-of-boundslayouts in release, where the first read indexed past the buffer. Now
checked arithmetic throughout.
matmulpanicked on a zero-size output dimension (Batched matmul panics with "chunk_size must not be zero" on a zero-size dimension #86) whilethe rank-2 path returned the empty tensor.
groups_mut()panicked on the next step (Optimizer state is keyed by position in ParamGroup::params, so mutating a group through groups_mut() panics #89).serialize::savepanicked inside safetensors on duplicate parameternames (serialize::save panics inside safetensors when a Module reports duplicate parameter names #91).
Data integrity and packaging
serialize::loadwas not atomic (serialize::load is not atomic: a failed load leaves the module half-overwritten #90): a load that failed partway leftthe module holding checkpoint values for the parameters already visited and
original values for the rest — a model from neither source, behind a typed
error.
LICENSE-*lived onlyat the workspace root, so all twelve 0.5.0 tarballs shipped the licence
claim without the licence. Both files are now in every crate and the
release workflow refuses to publish one that omits them.
ctorpre-mainconstructor 0.5.0 deleted, 0.4.0's "seven crates, 47 → 40"(measured: three crates, 43 → 40),
SECURITY.md's "Nothing is releasedyet", and a README optim row naming
SGD/RMSpropinstead ofSgd/RmsProp.Closes #84
Closes #85
Closes #86
Closes #87
Closes #88
Closes #89
Closes #90
Closes #91
Closes #92
Closes #93
Not in this release
The deep-test run hit the org's monthly agent budget partway through, so the
autograddimension never ran and 46 reported findings were left unverifiedrather than disproven. Promising leads still to check:
eigh's absoluteconvergence threshold, its f32-computed symmetry guard on f64 input, whether
sum/meanare the compensated summation the CHANGELOG claims, andserialize::loadmoving parameters back to the CPU afterModule::to_device.