<commit_before># Description
This document is a simple cookbook introducing testing facilities available for mbed 3.0. From this document you will learn how to:
* Create simple test(s) for your software encapsulated in yotta package. To read full yotta documentation go [here](http://docs.yottabuild.org/reference/commands.html).
* Create more complicated test cases where additional mocking / test case supervision is required
* Write mocks (we call them **host tests** in mbed) running on test host (your PC / Mac).

This document provides examples for all three methods of testing.

* Create [unit tests](http://en.wikipedia.org/wiki/Unit_testing) with [mbed compatible CppUTest library](https://github.com/ARMmbed/mbed-cpputest-private).
  * Information about CppUTest library can be found [here](https://cpputest.github.io/manual.html).

Both test tools and host test scripts are written in Python 2.7. This means minimum knowledge about Python development is required.

## Test process
Currently using test tools you can develop test cases which run on a single hardware platform and can interact with host computer via serial port (series of ```print```s and ```sscanf``` calls). Host computer communicates with mbed device using serial port connection (duplex) to drive test scenarions.

Test tools offer [host test](https://github.com/ARMmbed/mbed-host-tests) functionality. A host test is a supervising Python script executed on host computer (your Mac or PC) and used to mock certain functionality or feature like TCP servers required to test your code. Host tests are part of separate Python module (mbed-host-tests).
Each host test should be as generic as possible so many test cases running on mbed platforms can reuse it. For example simple TCP or UDP echo server can be used to test different network APIs also for mbed 2.0.

Test tools are designed to:
* Let you can concentrate on test development not test framework building and
* (re)use existing host tests or 
* create new host test to cover new functional / interoperability domain of your tests.

### Test suite (mbed-greentea)
Mbed 3.0 test suite called mbed-greentea is a Python 2.7 application which:
* Build your tests inside your yotta package using ```yotta build``` command.
  * List all tests in \build\<target_name>\test.
* For each built's test case in your yotta module, mbed-greentea executes in order below actions:
  * Flash hardware (target) with test binary.
  * Reset hardware (target) via serial port (send break command).
  * Listen on target's serial port and scans for prints from target.
  * Execute proper host test which will supervise your test case and
  * pass test result back to test suite (for reporting).

### Limitations
It is impossible to develop (with current test tools) test cases or test scenarios where few mbed devices cooperate with each other. For example it is impossible to write test cases for mesh networks or tests where two mbed devices communicate with each other via sockets / Bluetooth etc.

## yotta package
Place your test case under a sub-directory of ```\test``` directory located in your yotta package:
Note: For details please check ['The test Directory' section of yotta documentation](http://docs.yottabuild.org/tutorial/testing.html).
```
\your-yotta-pacjage-dir
  \source
    ...
  \test
    \test_case_1
      source.cpp
      test-case-1.cpp
    \test_case_2
      test-case-2.cpp
    ...
```
All tests in ```\test```'s sub-directories will be build by yotta and test case binaries will be stored in ```\build\<target-name>\test``` directory where ```<target-name>``` is yotta target used to build tests.

It is your responsibility to provide test cases in ```\test``` directory so they will build to target specific binaries.
It is test suite's (mbed-greentea) responsibility to flash all test binaries and perform test supervision by calling host test from mbed-host-tests package.

## yotta's testDependencies
The [testDependencies](http://docs.yottabuild.org/reference/module.html#testDependencies) section can be used to list modules that are only depended on by tests. They will not normally be installed if your module is installed as a dependency of another.

In our case we need to add testDependencies to ```mbed-sdk-private``` so we can include test related macros from ```mbed/test_env.h``` header file. Test macros will be explained in following sections of this document.

# Dependencies
* [yotta](https://github.com/ARMmbed/yotta) - yotta is a tool that we're building at mbed.
* [mbed-greentea](https://github.com/ARMmbed/mbed-greentea-private) - test suite for mbed 3.0, codename 'greentea'. The mbed test suite is a collection of tools that enable automated testing on mbed platforms.<commit_msg>Update QUICKSTART.md<commit_after># Description
This document is a simple cookbook introducing testing facilities available for mbed 3.0. From this document you will learn how to:
* Create simple test(s) for your software encapsulated in yotta package. To read full yotta documentation go [here](http://docs.yottabuild.org/reference/commands.html).
* Create more complicated test cases where additional mocking / test case supervision is required
* Write mocks (we call them **host tests** in mbed) running on test host (your PC / Mac).

This document provides examples for all three methods of testing.

* Create [unit tests](http://en.wikipedia.org/wiki/Unit_testing) with [mbed compatible CppUTest library](https://github.com/ARMmbed/mbed-cpputest-private).
  * Information about CppUTest library can be found [here](https://cpputest.github.io/manual.html).

Both test tools and host test scripts are written in Python 2.7. This means minimum knowledge about Python development is required.

## Test process
Currently using test tools you can develop test cases which run on a single hardware platform and can interact with host computer via serial port (series of ```print```s and ```sscanf``` calls). Host computer communicates with mbed device using serial port connection (duplex) to drive test scenarions.

Test tools offer [host test](https://github.com/ARMmbed/mbed-host-tests) functionality. A host test is a supervising Python script executed on host computer (your Mac or PC) and used to mock certain functionality or feature like TCP servers required to test your code. Host tests are part of separate Python module (mbed-host-tests).
Each host test should be as generic as possible so many test cases running on mbed platforms can reuse it. For example simple TCP or UDP echo server can be used to test different network APIs also for mbed 2.0.

Test tools are designed to:
* Let you can concentrate on test development not test framework building and
* (re)use existing host tests or 
* create new host test to cover new functional / interoperability domain of your tests.

### Test suite (mbed-greentea)
Mbed 3.0 test suite called mbed-greentea is a Python 2.7 application which:
* Build your tests inside your yotta package using ```yotta build``` command.
  * List all tests in \build\<target_name>\test.
* For each built's test case in your yotta module, mbed-greentea executes in order below actions:
  * Flash hardware (target) with test binary.
  * Reset hardware (target) via serial port (send break command).
  * Listen on target's serial port and scans for prints from target.
  * Execute proper host test which will supervise your test case and
  * pass test result back to test suite (for reporting).

### Limitations
1. It is impossible to develop (with current test tools) test cases or test scenarios where few mbed devices cooperate with each other. For example it is impossible to write test cases for mesh networks or tests where two mbed devices communicate with each other via sockets / Bluetooth etc.
2. mbed-greentea doesn't support resource locking (mutually exclusive access to hardware platform) so running few instances of mbed-greentea for the same target may cause undefined behaviour. Please, in case of multiple test suite instances use control resource access manually. When using Continuous Integration system you can configure dependencies for CI jobs and by that control access to each device under test.
3. You can't mix unit tests written using CppUTest library with 'hellow rold' tests. Reason is CppUTest library defines its own ```main()``` function which will be linked with every test. Currently yotta package can't be configured to link each test separately with different test library.

## yotta package
Place your test case under a sub-directory of ```\test``` directory located in your yotta package:
Note: For details please check ['The test Directory' section of yotta documentation](http://docs.yottabuild.org/tutorial/testing.html).
```
\your-yotta-pacjage-dir
  \source
    ...
  \test
    \test_case_1
      source.cpp
      test-case-1.cpp
    \test_case_2
      test-case-2.cpp
    ...
```
All tests in ```\test```'s sub-directories will be build by yotta and test case binaries will be stored in ```\build\<target-name>\test``` directory where ```<target-name>``` is yotta target used to build tests.

It is your responsibility to provide test cases in ```\test``` directory so they will build to target specific binaries.
It is test suite's (mbed-greentea) responsibility to flash all test binaries and perform test supervision by calling host test from mbed-host-tests package.

## yotta's testDependencies
The [testDependencies](http://docs.yottabuild.org/reference/module.html#testDependencies) section can be used to list modules that are only depended on by tests. They will not normally be installed if your module is installed as a dependency of another.

In our case we need to add testDependencies to ```mbed-sdk-private``` so we can include test related macros from ```mbed/test_env.h``` header file. Test macros will be explained in following sections of this document.

# Dependencies
* [yotta](https://github.com/ARMmbed/yotta) - yotta is a tool that we're building at mbed.
* [mbed-greentea](https://github.com/ARMmbed/mbed-greentea-private) - test suite for mbed 3.0, codename 'greentea'. The mbed test suite is a collection of tools that enable automated testing on mbed platforms.
{'fraction_non_alphanumeric': 0.07300127214013113, 'fraction_numerical': 0.0034249926607300128, 'mean_word_length': 4.387453874538745, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 10, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4848437', 'n_tokens_mistral': 2682, 'n_tokens_neox': 2576, 'n_words': 1377}