FTorch test suite

Testing

FTorch's test suite includes unit tests of components, and integration tests based on a subset of the worked examples.

• Building Tests
• Running Tests
  • Unit Tests
  • Integration Tests
• Adding Tests

Building

To enable FTorch's test suite, set CMAKE_BUILD_TESTS=TRUE during the build. To run the unit tests, you will need to install pFUnit and provide its install location to the CMAKE_PREFIX_PATH or as the environment variable PFUNIT_DIR:

# Update `CMAKE_PREFIX_PATH` explicitly with pFUnit install directory
cmake -DCMAKE_PREFIX_PATH="</path/to/pFUnit/build/installed/PFUNIT-VERSION>" \
    -DCMAKE_BUILD_TESTS=True -S </path/to/FTorch> -B </path/to/FTorch/build>

or

# Using an environment variable with pFUnit install directory
export PFUNIT_DIR=</path/to/pFUnit/build/installed/PFUNIT-VERSION>
cmake -DCMAKE_BUILD_TESTS=TRUE -S </path/to/FTorch> -B </path/to/FTorch/build>

Note that pFUnit includes the version number in the install directory name, so for version 4.12 that path will need to be specified as /path/to/pFUnit/build/installed/PFUNIT-4.12, for example.

Building with tests enabled will automatically install any Python dependencies for the examples, so should be executed from within a virtual environment.¹ If this is not the case it will fail with appropriate warnings.

Note that, whilst example 5_Looping is built if CMAKE_BUILD_TESTS=TRUE is specified, it is not run as part of the integration test suite because it demonstrates 'good' versus 'bad' practice, as opposed to functionality.

Running

Ensure that the Python virtual environment used when building is active and then run ctest from the build directory to execute all tests. Use ctest arguments for greater control over the testing configuration.

This will produce a report on which of the requested tests passed, and which, if any, failed for your build.

Unit tests

Unit tests may be executed in the following ways:

1. To run just the unit tests (tests whose names start with 'unittest') use ctest -R unittest.
2. To run a specific unit test use ctest -R unittest_tensor_constructors_destructors, for example.

Integration tests

Integration tests may be executed in the following ways:

1. To run just the integration tests (tests whose names start with 'example') use ctest -R example
2. To run a specific integration test use ctest -R example2, for example.
   Alternatively navigate to the corresponding example in ${BUILD_DIR}/examples and call ctest.

Adding Tests

New components should come with unit tests written using the pFUnit framework.

• New unit tests should be added to the test/unit/ directory and start with unittest_.
• New tests need including in the CMakeLists.txt in that directory in order to be built as part of the test suite.

New functionalities should come with integration tests in the form of worked examples.

• These should take the form of a new example in the examples/ directory.
• In addition to a CMakeLists.txt to build the example code there should also be a section at the end setting up running of the example using CTest.
  • Integration test names should start with example_
  • New examples will also need including in examples/CMakeLists.txt
• Ensure the documentation on worked examples is updated accordingly.