Testing and CI

Unit tests

Note

This section is under construction.

On-device tests

theCore testing facilities provides way to create on-device and system tests in modular and reusable manner. theCore uses the Unity test framework to create test cases.

On-device tests are located in the tests top-level directory.

Basic entities

A test case is a code that performs validation of the particular theCore component or module.

Each test, or test case, resides in separate directory under tests/cases dir. To be recognizable by theCore testing facilities the test case must contain a definition file with case_defs.cmake name.

Test case definition file must (at least) define CASE_SOURCES CMake variable. CASE_SOURCES must contain list of test case source files.

See [the unity_demo test case](cases/unity_demo) as an example.

You may check tests/cases/unity_demo as an example.

Targets and MCUs

A target is a board where tests should be executed.

Multiple boards can be built around single MCU type. MCU types are placed in the tests/mcus directory. Each MCU must contain mcu_defs.cmake and mcu_defs.hpp files. Those definition files can contain arbitrary data specific to the concrete microcontroller.

The TM4C MCU definition file (tests/mcus/tm4c123gh6pm/mcu_defs.cmake) provided here as an example:

1
2
3
4
5
6
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.

set(CONFIG_PLATFORM tm4c)
set(CONFIG_PLATFORM_DEVICE TM4C123GH6PM)

As shown, CMake file sets the platform associated with that MCU (CONFIG_PLATFORM) and the MCU model (CONFIG_PLATFORM_DEVICE).

Targets are residing in tests/targets directory and can reuse MCU definitions in its own definition files: target_defs.cmake and target_defs.hpp.

Targets must also declare set of test suites that they can handle (more on this in the Suites section).

Good example of the test target is a STM32 discovery target. It can be found in targets/stm32f4discovery_simple directory.

Suites

Every target can run limited set of possible tests. Limitations might be caused by lack of ROM, RAM or absence of the required hardware. Sets of tests that can be executed on the target called suite.

Suites are specific to the target, thus placed under <target_name>/suites directory. Suites are compiled into executables, thus every suite must contain ordinary CMakeLists.txt usable with theCore.

Suite does not need to contain any source files except the init code (usually placed in suite_init.cpp) that runs when board starts. Suite launchers are autogenerated by scripts/gen_suite.py script.

See Unity demo suite on stm32f4 discovery target placed under targets/stm32f4discovery_simple/suites/unity_demo as an example.

Building theCore suites

Existing test suites can be compiled by the following procedure (do not forget to complete Getting started guide to obtain Nix environment):

mkdir tests_build
cd tests_build
cmake ../tests
make

Paths to suite binaries are recorded in suite_list.txt file in the build directory:

$ cat suite_list.txt
platform_bat,tivac_tm4,/tmp/theCore/tests/build/tivac_tm4/platform_bat.gnu//build/platform_bat,/tmp/build3/tivac_tm4/platform_bat.gnu//build/platform_bat.bin
platform_bat,stm32f4discovery_simple,/tmp/theCore/tests/build/stm32f4discovery_simple/platform_bat.gnu//build/platform_bat,/tmp/build3/stm32f4discovery_simple/platform_bat.gnu//build/platform_bat.bin
platform_bat,stm32f4discovery_simple,/tmp/theCore/tests/build/stm32f4discovery_simple/platform_bat.clang//build/platform_bat,/tmp/build3/stm32f4discovery_simple/platform_bat.clang//build/platform_bat.bin
unity_demo,stm32f4discovery_simple,/tmp/theCore/tests/build/stm32f4discovery_simple/unity_demo.gnu//build/unity_demo,/tmp/build3/stm32f4discovery_simple/unity_demo.gnu//build/unity_demo.bin
unity_demo,tivac_tm4,/tmp/theCore/tests/build/tivac_tm4/unity_demo.gnu//build/unity_demo,/tmp/build3/tivac_tm4/unity_demo.gnu//build/unity_demo.bin

Launching suites

Suite binaries obtained in the Building theCore suites can be flashed into the MCU using either manual approach (using appropriate programmer) or using scripts/flasher.py script.

The flasher script tries to flash suite binaries to connected boards using suite_list.txt and the description of connected devices in a form of JSON.

Such JSON must be filled manually with respect to board connected. Example of the description file (/scripts/connected_devices_example.json) demonstrated below:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
{
  "stm32f4discovery_simple": {
    "debugger": {
      "name": "openocd",
      "flash_params": "init; reset halt; wait_halt; flash write_image erase {elf_path}; reset run; shutdown",
      "conf_file" : "stm32f4discovery.cfg"
    }
  },

  "tivac_tm4": {
    "debugger": {
      "name": "openocd",
      "flash_params": "init; reset halt; wait_halt; flash write_image erase {elf_path}; reset run; shutdown",
      "conf_file" : "ek-lm4f120xl.cfg"
    }
  }
}

Relation to the examples

Examples described in Examples section and on-device tests sometimes contains overlapping functionality but have different purpose.

Test cases and suites are not so well documented and indicative to be threatened as good examples.

Travis

Note

This section is under construction.