Inconsistent Outputs with DoubleGrad #40
Description
While trying to reproduce the DoubleGrad results, I encountered a behavior I wanted to clarify. Specifically, I noticed that when running the DoubleGrad visualization multiple times on the same image-text pair, without changing any settings or inputs, the outputs vary noticeably between runs. I'm sharing results from three separate runs on the same input. I'm using the pre-trained MDETR model. My understanding was that second-order gradients should be deterministic for a fixed model and input?
For context, I'm doing some work on how specific objects in an image contribute to particular parts of a caption, this variability is making interpretation difficult.
Sorry if I am missing something, but it would be helpful to clarify these questions:
1. Is this variability expected with current formula of DoubleGrad? 2. Is there a recommended way to stabilize results for consistent attribution?