Coupling Machine Learning to Fortran using the FTorch Library

Jack Atkinson

Principal Research Software Engineer
ICCS - University of Cambridge

Joe Wallwork

Senior Research Software Engineer
ICCS - University of Cambridge

The ICCS Team and Collaborators

2026-07-15

Precursors

Slides and Materials

The material for this workshop, including a link to the slides to follow on your own device, can be found at: github.com/Cambridge-ICCS/FTorch-workshop

Licensing

Except where otherwise noted, these presentation materials are licensed under the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) License.

Vectors and icons by SVG Repo under CC0(1.0) or FontAwesome under SIL OFL 1.1

Preparation

Codespaces

In this tutorial, we will be using GitHub Codespaces to run the exercises. If you are not familiar with Codespaces, please refer to the Codespaces documentation for more information.

1. Navigate to the FTorch-workshop repo page repository and click on the green “Code” button. NOTE: Make sure you open a Codespace for FTorch-workshop, not FTorch.
2. Select “Codespaces” and then “Create codespace on main”.
3. Wait for the codespace to be created and opened in your browser. You will be put in a VSCode environment with a terminal at the bottom.

Execute the following to start the FTorch build process:¹

source build_FTorch.sh

Note that after running this there will be a Python virtual environment set up and activated for you.

1. NOTE: running ./build_FTorch.sh is insufficcient as this will not export environment variables to the current shell session.

Motivation

Machine Learning in Science

We typically think of Deep Learning as an end-to-end process;
a black box with an input and an output¹.

Who’s that Pokémon?

\[\begin{bmatrix}\vdots\\a_{23}\\a_{24}\\a_{25}\\a_{26}\\a_{27}\\\vdots\\\end{bmatrix}=\begin{bmatrix}\vdots\\0\\0\\1\\0\\0\\\vdots\\\end{bmatrix}\] It’s Pikachu!

Neural Net by 3Blue1Brown under fair dealing.
Pikachu © The Pokemon Company, used under fair dealing.

1. And some models like GraphCast, FourCastNet, and Pangu operate on this principle

Machine Learning in Science

Neural Net by 3Blue1Brown under fair dealing.
Pikachu © The Pokemon Company, used under fair dealing.

Challenges

• Reproducibility
  • Ensure net functions the same in-situ
• Re-usability
  • Make ML parameterisations available to many models
  • Facilitate easy re-training/adaptation
• Language Interoperation

Language interoperation

Many large scientific models are written in Fortran (or C, or C++).
Much machine learning is conducted in Python.

Mathematical Bridge by cmglee used under CC BY-SA 3.0
PyTorch, the PyTorch logo and any related marks are trademarks of The Linux Foundation.”

Efficiency

We consider 2 types:

Computational

Developer

Both affect ‘time-to-science’.

How it Works

Torch

PyTorch

• an open-source deep-learning framework
• developed by Meta AI, now part of the Linux Foundation
• written in C++ with a Python interface
• port of Torch (ATen), but also includes Caffe2 etc.

libtorch

• A (Pythonic-ish) C++ interface to the underlying code.
• Ability to save and read PyTorch models (and weights) through TorchScript
• Fortran can bind to this using the iso_c_binding module (intrinsic since 2003).
• Utilising shared memory (on CPU) reduces data transfer overheads.

Torch and PyTorch logos under Creative Commons

FTorch

Python
env

Python
runtime

xkcd #1987 by Randall Munroe, used under CC BY-NC 2.5

Further Information

For more details on the development see slides and recording from a recent talk here: jackatkinson.net/slides/Oxford-FTorch/

Or, for the lastest on what’s new in FTorch, see jackatkinson.net/slides/CESM-Jun-26

Learning Objectives

The key learning objective from this workshop could be simply summarised as:
Provide the ability to couple PyTorch models to Fortran using FTorch.

However, more specifically we aim to:

• discuss Torch and introduce the libtorch library,
• introduce FTorch and its aims and benefits,
• teach users the full pipeline of taking a PyTorch model and coupling it to a Fortran code,
• highlight best practices and efficient use when doing the above, and
• (if time allows) introduce the automatic differentiation aspects of FTorch.

The FTorch library

Obtaining FTorch

FTorch is available from GitHub:
github.com/Cambridge-ICCS/FTorch

With supporting documentation at:
cambridge-iccs.github.io/FTorch/

For the purposes of this workshop using GitHub codespaces FTorch is built when sourcing the build_FTorch.sh script.

This downloads the FTorch source from GitHub, installs libtorch through PyTorch, and then uses CMake to build FTorch and link it to libtorch.

If you want to build locally you can inspect this script for more details and consult the comprehensive online documentation.

FTorch source code

The source code for FTorch is contained in the src/ directory. This contains:

• ctorch.cpp - Bindings to the libtorch C++ API.
• ctorch.h - C header file to bind to from Fortran.
• ftorch.f90 - the umbrella module that uses submodules containing the Fortran routines we will be calling (ftorch_devices.F90, ftorch_types.f90, ftorch_tensor.f90, ftorch_model.f90, ftorch_optim.f90).

These are compiled, linked together, and installed using CMake.

• Simplify build process for users.
• Accomodate different machines and setups.
• Configured in the CMakeLists.txt file.

FTorch_utils

FTorch includes a pip-installable Python library ftorch_utils that installs the handy pt2ts command-line utility for converting Python models to TorchScript.

It also installs PyTorch and other dependencies as required by the FTorch examples suite.

If installing in codespaces using build_FTorch.sh these will be installed into a Python virtual environment for you.

Time for the practical

Exercise 0 – Hello Fortran and PyTorch!

Question: What are people’s experience levels with and use cases of Fortran and PyTorch?

Navigate to exercises/exercise_00/ where you will see both Python and Fortran files.

Python

hello_pytorch.py defines a net SimpleNet that takes an input vector of length 5 and multiplies it by 2.

Note:

• the nn.Module class
• the forward() method

Running:

python hello_pytorch.py

should produce the output:

Input is  tensor([0., 1., 2., 3., 4.]).
Output is tensor([0., 2., 4., 6., 8.]).

Compilers and CMake

First we will check that we have a Fortran compiler and CMake installed:

Running:

gfortran --version
cmake --version

Should produce output similar to:

GNU Fortran (Homebrew GCC 14.1.0_1) 14.1.0
Copyright (C) 2024 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE.

cmake version 3.31.0
CMake suite maintained and supported by Kitware (kitware.com/cmake).

and not:

bash: command not found: gfortran
bash: command not found: cmake

Check that CMake version >= 3.18 and gcc >= 9 as required for FTorch.

Fortran

The file hello_fortran.f90 contains a program to take an input array and call a subroutine to multiply it by two before printing the result.

The subroutine is contained in a separate module math_mod.f90, however.

Fortran

Now we will compile the code using CMake.

First create a build directory and generate build scripts using CMake:

mkdir build
cd build
cmake ..

We then use CMake again to build the executable:

cmake --build .

Run the compiled executable:

./hello_fortran

should produce the output

 Hello, World!
 Input:     0.00000000        1.00000000        2.00000000        3.00000000        4.00000000
 Output:    0.00000000        2.00000000        4.00000000        6.00000000        8.00000000

CMake handles the compilation and linking of both Fortran files.

Exercises

Exercise 1

Now that we have FTorch installed on the system we can move to writing code that uses it, needing only to link to our installation at compile and runtime.

We will start of with a basic example showing how to couple code in Exercise 1:

1. Design and train a PyTorch model.
2. Save PyTorch model to TorchScript.
3. Write Fortran using FTorch to call saved model.
4. Compile and run code, linking to FTorch.

PyTorch

Examine exercises/exercise_01/simplenet.py.

This contains a contrived PyTorch model with a single nn.Linear layer that will multiply the input by two.

With our virtual environment active we can test this by running the code with¹:

python3 simplenet.py

Input:  tensor([0., 1., 2., 3., 4.])
Output: tensor([0., 2., 4., 6., 8.])

1. This will run the if __name__ == "__main__": section at the bottom of the file.

Offline training with FTorch

Saving to TorchScript

To use the net from Fortran we need to save our PyTorch net as TorchScript in a .pt file.

FTorch comes with a handy utility pt2ts to help with this, which is installed alongside the Python components of the library.

We run simplenet.py to produce a PyTorch model file (.pt file with the model architecture and weights), then use the pt2ts command to convert it to TorchScript ready for use from Fortran.

Saving to TorchScript

We already ran

python3 simplenet.py

Run

ls

to check that the pytorch_simplenet_input_tensor_cpu.pt file was created.

Then convert to TorchScript using the pt2ts command:

pt2ts SimpleNet \
  --model_definition_file simplenet.py \
  --input_model_file pytorch_simplenet_model_cpu.pt \
  --output_model_file torchscript_simplenet_model_cpu.pt

Now run

ls

to check that the torchscript_simplenet_input_tensor_cpu.pt file was created.

The --test flag runs a basic check that the TorchScript model produces the same output as the PyTorch model after saving it.

Offline training with FTorch

Calling from Fortran

We are now in a state to use our saved TorchScript model from within Fortran.

exercises/exercise_01/simplenet_fortran.f90 contains a skeleton code with a Fortran arrays to hold input data for the net, and the results returned.

We will modify it to create the neccessary data structures and load and call the net.

• Import the ftorch module.
• Create torch_tensors and a torch_model to hold the data and net.
• Map Fortran data from arrays to the torch_tensors.
• Load the model and print its parameters.
• Call torch_model_forward to run the net.

See the solution file simplenet_fortran_sol.f90 for an ideal code.
For more information on the subroutines and API see the online API documentation

Building the code

Once we have modified the Fortran we need to compile the code and link it to FTorch.

This can be done from the exercise_01/ subdirectory using CMake as follows:

mkdir build
cd build
cmake ..
cmake --build .

To run the code we can use the generated executable:

./simplenet_fortran

 Model parameters:
_fwd_seq.0.weight:
  2  0  0  0  0
  0  2  0  0  0
  0  0  2  0  0
  0  0  0  2  0
  0  0  0  0  2
 [ CPUFloatType{5,5} ]
 Model output:
   0.00000000       2.00000000       4.00000000       6.00000000       8.00000000

The path to the FTorch installation will be picked up automatically in Codespaces so doesn’t need to be specified. If building locally you may need to add FTorch to the LD_LIBRARY_PATH.

Exercise 2: A more realistic example

In exercise 1 we used a contrived model, SimpleNet, with a 1-D input.

In this exercise we take a step up using a real-world example: classifying images with a pre-trained ResNet-18 model from TorchVision.

We will reinforce what we learnt in exercise 1 whilst using multidimensional data and briefly exploting batched data as an extension. As part of this we will also discuss data layout differences between Python (row-major) and Fortran (column-major).

Exercise 2

Navigate to exercises/exercise_02/.

You will find:

• generate_input_batch.py – preprocesses image(s) to a binary data file.
• resnet_infer_python.py – Run resnet from Python.
• resnet_infer_fortran.f90 – template with ! TODO comments.
• resnet_infer_fortran_sol.f90 – complete solution.
• data/ – example images and ImageNet labels.

Stage 1: Basic inference

1. Generate input:

   python generate_input_batch.py

   takes images in data/, and pre-processes them ready for input to the ResNet model.

2. Convert to TorchScript:

   pt2ts resnet18 --model_weights IMAGENET1K_V1 --output_model_file torchscript_resnet18_model_cpu.pt

   downloads a model definition and weights from TorchVision and saves it to TorchScript.

3. Verify inference works in Python:

   python resnet_infer_python.py

   Image 0: Samoyed, probability 0.8846

4. Fortran: Build using CMake, completing the TODOs in resnet_infer_fortran.f90, then run:

   ./resnet_infer_fortran torchscript_resnet18_model_cpu.pt

The Fortran TODOs

The Fortran arrays are already set up for you to read data from file:

in_data(batch_size, 3, 224, 224)
out_data(batch_size, 1000)

You need to:

1. Declare torch_model and torch_tensor variables.
2. Create tensors from Fortran arrays with torch_tensor_from_array.
3. Load the model with torch_model_load.
   • Note that the input filename string is already available as the variable model_file.
4. Run inference with torch_model_forward.

Stage 2: Add batching

Now extend from single-image to multi-image inference:

1. Generate a larger batch by editing generate_input_batch.py to include dog2.jpg (and any other images you download). Re-run the script.

2. Run with batch_size 2:

   ./resnet_infer_fortran torchscript_resnet18_model_cpu.pt 2

3. Verify the results in Python:

   python3 resnet_infer_python.py --batch_size 2

Data layout discussion

Python (C) and Fortran store arrays in opposite memory order:

• Python: row-major, last index varies fastest.
• Fortran: column-major, first index varies fastest.

This is why we transpose the data in Python before saving when generating the batch input:

np_input = np_input.transpose().flatten()

transpose() reverses axes from [B, C, H, W] to [W, H, C, B].

This means that when read into memory by Fortran they will appear structured as we expect. FTorch then uses Torch’s strided memory access to account for differences in layout meaning the arrays can appear the same to the user in both Fortran and Python.

See the online docs for more detail.

Exercise 3: Larger code considerations

What we have considered so far is a simple contrived example designed to teach the basics.

However, in reality the codes we will be using are more complex, and full of terrors.

• We will be calling the net repeatedly over the course of many iterations.
• Reading in the net and weights from file is expensive.
  • Don’t do this at every step!

Exercise 3: Larger code considerations

In exercise 3 we will look at an example of how to ideally structure a slightly more complex code setup. For those familiar with climate models this may be nothing new. We make use of the traditional separation into subroutines for:

1. Initialisation
2. Updating
3. Finalisation

Exercise 3

Navigate to the exercises/exercise_03/ directory.

Here you will see two code directories, good/ and bad/.

Both perform the same operation:

• Running the simplenet from example 1 10,000 times.
• Increment the input vector at each step.
• Accumulate the sum of the output vector.

Exercise 3

The exercise is the same for both folders:

• Run the pt2ts command to save the net (as in exercise 1).
• Inspect the code to see how it works:
  • Both have a main program in simplenet_fortran.f90.
  • Both have FTorch code extracted to a module fortran_ml_mod.f90.
    • bad is in a single routine.
    • good is split into init, iter, and finalise.
• Modify the Makefile to link to FTorch and build the codes.
• Time the codes and observe the difference.

Exercise 3 - solution

For a complete version of the example, see the Looping example.

Exercise 4: automatic differentiation

PyTorch has a powerful automatic differentiation engine called autograd that can be used to compute derivatives of expressions involving tensors.

We recently exposed this functionality in FTorch, allowing you to do this in Fortran, too. This is a key step facilitating online training in FTorch.

In this exercise, we walk through differentiating the mathematical expression \[Q = 3 (a^3 - b^2/3)\] using both PyTorch and FTorch.

Online training with FTorch

Multiple inputs and outputs

Supply as an array of tensors, innit.

• For more details see the MultiIO Example.

GPU Acceleration

• FTorch automatically has access to GPU acceleration through the PyTorch backend.
• When running pt2ts, save the model on GPU.
  • Guidance provided in the file.
• When creating torch_tensors set the device to torch_kCUDA instead of torch_kCPU.¹
• To target a specific device supply the device_index argument.
• CPU-GPU cannot avoid data transfer. Use MPI_GATHER() to reduce.
• Use torch_tensor_to to transfer tensors between devices (and data types).
• For more details see:
  • the online documentation
  • the GPU Example

1. We can also use AMD HIP, Intel XPU and AppleSilicon MPS.

Thanks for Listening

For more information please speak to us afterwards, or drop us a message.

 Jack Atkinson

 jwa34[AT]cam.ac.uk

 jatkinson1000

 Joe Wallwork

 jw2423[AT]cam.ac.uk

 jwallwork23

Thanks to the rest of the FTorch team and contributors.

The ICCS received support from

FTorch has been supported by