Reverse mode returns zero gradient on closure-captured aliased mutable buffer (workaround: copy)

[ChrisRackauckas-Claude]

Summary

Enzyme.gradient(set_runtime_activity(Reverse), Const(loss), x) returns an all-zero gradient when:

1. A mutable struct instance holding a Vector{Float64} is captured by a closure.
2. The loss constructs a new instance of the same struct whose Vector field aliases the captured buffer (i.e. it stores the same Vector object, no copy).
3. The loss mutates that buffer through the new struct from the active input.
4. The loss reads sum(buffer).

Manually copying the buffer before mutation (so the alias is broken) gives the correct gradient that matches FiniteDiff to >8 sig figs. Plain Reverse (no runtime activity) raises EnzymeRuntimeActivityError on the aliasing version; set_runtime_activity(Reverse) runs through and silently returns zeros.

This was reduced from a SciML / ModelingToolkit MTKParameters case where the closure captures iprob.p, a repack callback returns a new MTKParameters whose caches::Tuple{Vector{Float64}} field aliases iprob.p.caches, and solve! then mutates p_new.caches[1]. The standalone version below has no SciML dependencies.

MWE

using Enzyme, FiniteDiff

mutable struct Holder
    v::Vector{Float64}
end

const captured = Holder([0.0, 0.0, 0.0])

function loss_alias(t::Vector{Float64})
    h = Holder(captured.v)           # NEW struct, .v aliases captured.v
    for i in eachindex(h.v)
        h.v[i] = t[i]^2
    end
    return sum(h.v)
end

function loss_copy(t::Vector{Float64})
    h = Holder(copy(captured.v))     # break the alias
    for i in eachindex(h.v)
        h.v[i] = t[i]^2
    end
    return sum(h.v)
end

t0 = [1.0, 2.0, 3.0]
mode = set_runtime_activity(Reverse)

@show FiniteDiff.finite_difference_gradient(loss_alias, t0)
@show FiniteDiff.finite_difference_gradient(loss_copy,  t0)
@show Enzyme.gradient(mode, Const(loss_alias), t0)
@show Enzyme.gradient(mode, Const(loss_copy),  t0)

Output

loss_alias(t0)  = 14.0
loss_copy(t0)   = 14.0
FD  grad (alias) = [2.0000000000471077, 3.9999999999475415, 5.999999999994649]
FD  grad (copy)  = [2.0000000000471077, 3.9999999999475415, 5.999999999994649]
Enz grad (alias) = ([0.0, 0.0, 0.0],)        # WRONG
Enz grad (copy)  = ([2.0, 4.0, 6.0],)        # correct, matches FD

With plain Reverse (no runtime activity), the loss_alias call instead raises:

EnzymeRuntimeActivityError: Detected potential need for runtime activity.
... Failure within method: getproperty(::Holder, ::Symbol) ...

so the issue manifests as either a hard error (plain Reverse) or a silently wrong zero gradient (set_runtime_activity(Reverse)).

Expected

Enzyme.gradient(mode, Const(loss_alias), t0) == [2.0, 4.0, 6.0] (same as the loss_copy version, same as FiniteDiff). The captured Holder is Const from Enzyme's perspective, but the buffer inside it is being treated as the active storage for the gradient computation in this call, and Enzyme should follow the alias and accumulate into it.

Workaround

Manually copy the buffer before storing it in the new struct, so the new struct does not alias any closure-captured storage.

Versions

• Julia: 1.12.6
• Enzyme: v0.13.150
• FiniteDiff: v2.31.0
• OS: Linux x86_64

[ChrisRackauckas-Claude]

Followup test isolating the root cause:

Variant A: Enzyme.gradient(set_runtime_activity(Reverse), Const(loss), t0)        → [0, 0, 0]   BUG
Variant B: autodiff(set_runtime_activity(Reverse), Const(f_explicit),
                    Active, Duplicated(t0), Duplicated(captured, shadow))         → [2, 4, 6]   CORRECT
Variant C: autodiff(set_runtime_activity(Reverse), Const(f_explicit),
                    Active, Duplicated(t0), Const(captured))                      → [0, 0, 0]   BUG

The captured mutable buffer needs an explicit Duplicated annotation for the alias-write path to work. With Const(captured) (or equivalently when it's just closure-captured under a Const loss), no shadow buffer is allocated, and the writes through the aliased field have nowhere to record derivative info.

This is arguably WAI ('annotate captured mutable state as Duplicated'), except that set_runtime_activity(Reverse) is presumably designed to detect and handle exactly this pattern at runtime — and it doesn't. The naive Enzyme.gradient(set_runtime_activity(Reverse), Const(loss), x) API has no way for the user to communicate captured-mutable-via-closure state.

So this can be filed as either:

1. A correctness bug: set_runtime_activity should handle closure-captured-mutable-aliased-into-new-struct correctly without explicit Duplicated.
2. A documentation/feature-request bug: top-level Enzyme.gradient should error (loudly) or have an API for marking captured state, instead of silently returning zero.

Real-world manifestation: in SciML/ModelingToolkit, every closure that captures a problem object and modifies its caches via the standard SciMLStructures.canonicalize + solve workflow hits this. Users cannot easily annotate the captured problem as Duplicated (it's deep inside the SCC/init machinery). PR SciML/ModelingToolkit.jl#4557 worked around by allocating fresh caches buffers in repack — but the user/maintainer feedback is that the alias is intended MTK semantics and the real fix should be here in Enzyme.

[wsmoses]

You can't call Duplicated(t0), Duplicated takes two arguments?

[ChrisRackauckas-Claude]

Closing — this is expected Enzyme behavior, not a bug.

Const(closure_with_mutable_captures) correctly tells Enzyme not to allocate a shadow for the closure's captured state. The user is responsible for annotating captured mutable state as Duplicated (via autodiff with explicit shadow args) when they want derivatives to flow through it. My MWE was a misuse of the API: I was annotating a closure that captures and writes through mutable state as Const.

The downstream SciML problem (desauty SCC init Enzyme.gradient returning zero) is being fixed user-side in the SciMLSensitivity test by breaking the cache alias explicitly via ConstructionBase.setproperties before remake — no Enzyme-side change required. Apologies for the noise.

[ChrisRackauckas]

Yeah sorry this one escaped lol. I found the real remaining issue, and it's with the test definition. I think I got it all working now? Give me a few hours and I'll know for sure.