NA-MIC-kit-curriculum/Testing-Based Programming/How to add Tests in CMake

This is a basic tutorial on how to set up testing in your project.

We start with a very simple case here, and progressively will add more interesting features

Hello World

We start by creating a minimal project with only

• main.cxx
• CMakeLists.txt

main.cxx

Here is the main file

 #include <iostream>
 #include <stdlib.h>
 int main( int argc, char * argv [] )
 {
   const unsigned int N = atoi( argv[1] );
   for( unsigned int i=0; i < N; i++ )
    {
     std::cout << "Hello World ! " << i << std::endl;
     }
   return 0;
 }

CMakeLists.txt

The key commands for testing are

• INCLUDE(CTest)
• ADD_TEST(testname executable arg1 arg2 arg3 ... argN)

Here is the minimal CMakeLists.txt file

 CMAKE_MINIMUM_REQUIRED(VERSION 2.6) 
 PROJECT(TestingTutorial)
 INCLUDE(CTest)
 ADD_EXECUTABLE(TestMain main.cxx)
 ADD_TEST(test1   TestMain  5)
 ADD_TEST(test2   TestMain  8)

Building

Create source directory

Put the two files

• main.cxx
• CMakeLists.txt

in a directory for source code. We will refer to this directory as the "SOURCE_DIR" hereafter.

Create binary directory

Create a directory for compiling the code. We will refer to this directory as the "BINARY_DIR" hereafter.

Configure, Build and Test

Then do the following

   cd ${BINARY_DIR}
   cmake ${SOURCE_DIR}
   make
   make test

at this point you should see an output similar to

 Running tests...
 Test project /home/ibanez/src/test/CMake/bin
     Start 1: test1
 1/2 Test #1: test1 ............................   Passed    0.00 sec
     Start 2: test2
 2/2 Test #2: test2 ............................   Passed    0.00 sec
 ..
 100% tests passed, 0 tests failed out of 2
 ..
 Total Test time (real) =   0.01 sec

Testing Options

when you type

  make test

you are running "ctest" behind the scenes. CTest is an executable that makes part of the CMake distribution.

Here are some useful ctest commands:

• Help

   ctest -h

• List the tests without running them

   ctest -N

• Verbose mode

   ctest -V

• Very verbose mode

   ctest -VV

• List the tests and their command without running them

   ctest -VN

• Run a set of tests that are selected based on a regular expression

   ctest -R segmentation

• Run a set of test selected by a numerical range (in this case, tests 5 to 10 at increments of 1)

   ctest -I 5,10,1