Multithread handling in CoDiPack

Speaker: Max Sagebaum, University Kaiserslautern-Landau RPTU

Abstract

A program that is thread-safe contains no race conditions on variable writes. If a variable is read by multiple threads, then the last update of the variable needs to be synchronized. Reverse AD for threaded programs will have writes on the adjoint of variables that are read and reads on the adjoint of variables that are written. The adjoint reads are safe since we assume that all writes are thread-safe. The problem lies with the adjoint writes. Since a variable can be accessed by multiple threads, there can be multiple writes on the adjoint at the same time. The safe strategy is an adjoint vector using atomic variables, but studies have shown that this has a large impact on the adjoint performance.

We want to present an algorithm that can detect if a variable is accessed by multiple threads and use this information to perform the adjoint write either in an atomic fashion or a non-atomic one.