Testing and CI
==============

Unit tests
----------

.. note:: This section is under construction.

On-device tests
---------------

theCore testing facilities provides way to create on-device and system tests
in modular and reusable manner.
theCore uses `the Unity test framework`_ to create test cases.

On-device tests are located in the ``tests`` top-level directory.

Basic entities
~~~~~~~~~~~~~~

A test case is a code that performs validation of the particular theCore
component or module.

Each test, or test case, resides in separate directory under ``tests/cases`` dir.
To be recognizable by theCore testing facilities the test case
must contain a definition file with ``case_defs.cmake`` name.

Test case definition file must (at least) define ``CASE_SOURCES`` CMake variable.
``CASE_SOURCES`` must contain list of test case source files.

See [the unity_demo test case](cases/unity_demo) as an example.

You may check ``tests/cases/unity_demo`` as an example.

Targets and MCUs
~~~~~~~~~~~~~~~~

A target is a board where tests should be executed.

Multiple boards can be built around single MCU type.
MCU types are placed in the ``tests/mcus`` directory.
Each MCU must contain ``mcu_defs.cmake`` and ``mcu_defs.hpp`` files.
Those definition files can contain arbitrary data specific to the concrete microcontroller.

The TM4C MCU definition file (``tests/mcus/tm4c123gh6pm/mcu_defs.cmake``)
provided here as an example:

.. literalinclude:: ../../../tests/mcus/tm4c123gh6pm/mcu_defs.cmake
   :language: cmake
   :linenos:

As shown, CMake file sets the platform associated with that MCU (``CONFIG_PLATFORM``)
and the MCU model (``CONFIG_PLATFORM_DEVICE``).

Targets are residing in ``tests/targets`` directory and can reuse
MCU definitions in its own definition files: ``target_defs.cmake``
and ``target_defs.hpp``.

Targets must also declare set of test suites that they can handle
(more on this in the Suites_ section).

Good example of the test target is a  STM32 discovery target.
It can be found in ``targets/stm32f4discovery_simple`` directory.

Suites
~~~~~~
Every target can run `limited` set of possible tests.
Limitations might be caused by lack of ROM, RAM or absence of the required hardware.
Sets of tests that can be executed on the target called **suite**.

Suites are specific to the target, thus placed under ``<target_name>/suites`` directory.
Suites are compiled into executables, thus every suite must contain ordinary
``CMakeLists.txt`` usable with theCore.

Suite does not need to contain any source files except the init code
(usually placed in ``suite_init.cpp``) that runs when board starts.
Suite launchers are autogenerated by ``scripts/gen_suite.py`` script.

See Unity demo suite on stm32f4 discovery target placed under
``targets/stm32f4discovery_simple/suites/unity_demo`` as an example.

Building theCore suites
~~~~~~~~~~~~~~~~~~~~~~~

Existing test suites can be compiled by the following procedure
(do not forget to complete :ref:`theCore_GettingStarted` guide to obtain Nix environment)::

  mkdir tests_build
  cd tests_build
  cmake ../tests
  make

Paths to suite binaries are recorded in `suite_list.txt` file in the build directory::

  $ cat suite_list.txt
  platform_bat,tivac_tm4,/tmp/theCore/tests/build/tivac_tm4/platform_bat.gnu//build/platform_bat,/tmp/build3/tivac_tm4/platform_bat.gnu//build/platform_bat.bin
  platform_bat,stm32f4discovery_simple,/tmp/theCore/tests/build/stm32f4discovery_simple/platform_bat.gnu//build/platform_bat,/tmp/build3/stm32f4discovery_simple/platform_bat.gnu//build/platform_bat.bin
  platform_bat,stm32f4discovery_simple,/tmp/theCore/tests/build/stm32f4discovery_simple/platform_bat.clang//build/platform_bat,/tmp/build3/stm32f4discovery_simple/platform_bat.clang//build/platform_bat.bin
  unity_demo,stm32f4discovery_simple,/tmp/theCore/tests/build/stm32f4discovery_simple/unity_demo.gnu//build/unity_demo,/tmp/build3/stm32f4discovery_simple/unity_demo.gnu//build/unity_demo.bin
  unity_demo,tivac_tm4,/tmp/theCore/tests/build/tivac_tm4/unity_demo.gnu//build/unity_demo,/tmp/build3/tivac_tm4/unity_demo.gnu//build/unity_demo.bin

Launching suites
~~~~~~~~~~~~~~~~

Suite binaries obtained in the `Building theCore suites`_ can be flashed
into the MCU using either manual approach (using appropriate programmer)
or using ``scripts/flasher.py`` script.

The flasher script tries to flash suite binaries to connected boards
using ``suite_list.txt`` and the description of connected devices
in a form of JSON.

Such JSON must be filled manually with respect to board connected.
Example of the description file (``/scripts/connected_devices_example.json``)
demonstrated below:

.. literalinclude:: ../../../scripts/connected_devices_example.json
   :language: json
   :linenos:

Relation to the examples
~~~~~~~~~~~~~~~~~~~~~~~~

Examples described in :ref:`theCore_Examples` section and on-device tests
sometimes contains overlapping functionality but have different purpose.

Test cases and suites are not so well documented and indicative to be threatened
as good examples.

Travis
------

.. note:: This section is under construction.


.. _the Unity test framework: http://www.throwtheswitch.org/unity/