Code examples and use cases

First order extensions with a ResNet

Diagonal Gauss-Newton Second order optimizer

Gradient of backpropagated quantities

Hutchinson Trace Estimation

Differentially Private SGD.

Matrix-free second-order optimization