Skip to content

Clone on Linear, Conv2d and Embedding shares parameters with the original #60

Description

@vyncint

Summary

Param is an Arc<RwLock<Tensor>> handle and the layers derive Clone, so a cloned layer trains together with its original. LayerNorm, BatchNorm2d, Dropout and Sequential are not Clone, so the surface is also inconsistent.

Where (main at 853a224)

  • crates/oxmera-nn/src/param.rs:12-15#[derive(Debug, Clone)] pub struct Param { inner: Arc<RwLock<Tensor>> }
  • crates/oxmera-nn/src/linear.rs:11-15, conv.rs:14, embedding.rs:11#[derive(Debug, Clone)] on the layer.

Reproduction

let a = Linear::new(2, 2, 1);
let b = a.clone();
a.weight().set(Tensor::zeros([2, 2]));
assert!(b.weight().value().to_vec_f32()?.iter().all(|&x| x == 0.0)); // holds: b changed too

Fix

Either remove Clone from the layers (a declared break in the 0.x window) and offer fn deep_clone(&self) -> Self / Param::detached_copy(), or keep Clone and document the aliasing on every layer and on Param. Weight tying should be a visible choice, not a side effect of .clone().

Done when

Cloning a layer either copies its parameters or is impossible; the chosen behaviour is documented on Param and tested.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions