Code examples and use cases

Extension hook example

First order extensions with a ResNet

Diagonal Gauss-Newton Second order optimizer

Custom module example

Saving memory in convolutions

Gradient of backpropagated quantities

Hutchinson Trace Estimation

Differentially Private SGD.

Matrix-free second-order optimization