Automatic Differentiation with Dual Numbers

tl;dr: Given any model that maps inputs to outputs, Dual Numbers can be used to calculate the exact derivative of any output with respect to any input without requiring the user to calculate these derivatives. An implementation of a Dual Number system only requires to define a data type or class with methods for every function and operator used inside the model. What is Automatic Differentiation? Automatic Differentiation (AD) is a series of methods to calculate the derivative of outputs of a model with respect to its inputs.