Introduction to Differentiable Programming

Derivatives are at the heart of scientific programming. From the Jacobian matrices used to solve nonlinear systems to the gradient vectors used for optimisation methods, from the backpropagation operation in machine learning to the data assimilation methods used in weather forecasting, all of these techniques rely on derivative information. Differentiable programming (also known as automatic/algorithmic differentiation (AD)) provides a suite of tools for users to compute derivatives of quantities in their code without any manual encoding.

In Session 1, we will learn about the history and mathematical background of differentiable programming and consider examples using the Autograd AD tool. In Session 2, we will learn about more advanced topics and consider examples using the JAX differentiable modelling framework.

📓 Notebooks

Main course notebooks:

• 🔗 Session 1: Introducton to Differentiable Programming with Autograd
• 🔗 Session 2: Further Differentiable Programming with JAX

Mini-project notebooks:

• 🔗 MNIST: Mini-project with a neural network trained on the MNIST dataset
• 🔗 Lorenz ’96: Mini-project with the Lorenz ’96 model
• 🔗 Shallow water: Mini-project with the linear shallow water equations