It'd be nice to have a function that exactly gets the diagonal mixed gradient of a PFB, i.e. for a given real-valued function $T(\mathbf{x},y;\mathbf{c})$ for given parameters $\mathbf{c}$, it'd be nice to get the value of $\nabla_\mathbf{c}\partial_y T(\mathbf{x},y;\mathbf{c})$.
In practice, this is possible currently, but it's somewhat annoying (and I'm not really sure how stable it is). In Julia code, you have to do something like this
T = # ...
samples = #...
logdet_eval = LogDeterminant(T, samples)
logdet_grad = LogDeterminantCoeffGrad(T, samples)
mixed_grad = logdet_grad
for j in 1:size(samples,2)
mixed_grad[:,j] *= exp(logdet_eval[j])
endThis comes from the fact that this mixed gradient is given by
$$\nabla_\mathbf{c}\partial_yT(\mathbf{x},y;\mathbf{c}) = \exp(\log\partial_y T(\mathbf{x},y;\mathbf{c})))\nabla_\mathbf{c}\log\partial_y T(\mathbf{x},y;\mathbf{c})$$
The exponential worries me and it seems like a waste to evaluate two functions when you only need one.
It'd be nice to have a function that exactly gets the diagonal mixed gradient of a PFB, i.e. for a given real-valued function$T(\mathbf{x},y;\mathbf{c})$ for given parameters $\mathbf{c}$ , it'd be nice to get the value of $\nabla_\mathbf{c}\partial_y T(\mathbf{x},y;\mathbf{c})$ .
In practice, this is possible currently, but it's somewhat annoying (and I'm not really sure how stable it is). In Julia code, you have to do something like this
This comes from the fact that this mixed gradient is given by
$$\nabla_\mathbf{c}\partial_yT(\mathbf{x},y;\mathbf{c}) = \exp(\log\partial_y T(\mathbf{x},y;\mathbf{c})))\nabla_\mathbf{c}\log\partial_y T(\mathbf{x},y;\mathbf{c})$$
The exponential worries me and it seems like a waste to evaluate two functions when you only need one.