<commit_before>.. _test-framework:

Test Framework
###############

The Zephyr Test Framework (Ztest) provides a simple testing framework intended
to be used during development.  It provides basic assertion macros and a generic
test structure.

The framework can be used in two ways, either as a generic framework for
integration testing, or for unit testing specific modules.

To enable the latest APIs of Ztest simply set :kconfig:option:`CONFIG_ZTEST_NEW_API=y`. The legacy
APIs will soon be deprecated and eventually removed.

Creating a test suite
*********************

Using Ztest to create a test suite is as easy as calling the :c:macro:`ZTEST_SUITE`. The macro
accepts the following arguments:

* ``suite_name`` - The name of the suite. This name must be unique within a single binary.
* :c:type:`ztest_suite_predicate_t` - An optional predicate function to allow choosing when the
  test will run. The predicate will get a pointer to the global state passed in through
  :c:func:`ztest_run_all` and should return a boolean to decide if the suite should run.
* :c:type:`ztest_suite_setup_t` - An optional setup function which returns a test fixture. This
  will be called and run once per test suite run.
* :c:type:`ztest_suite_before_t` - An optional before function which will run before every single
  test in this suite.
* :c:type:`ztest_suite_after_t` - An optional after function which will run after every single
  test in this suite.
* :c:type:`ztest_suite_teardown_t` - An optional teardown function which will run at the end of
  all the tests in the suite.

Below is an example of a test suite using a predicate::

    #include <zephyr/ztest.h>
    #include "test_state.h"

    static bool predicate(const void *global_state)
    {
      return ((const struct test_state*)global_state)->x == 5;
    }

    ZTEST_SUITE(alternating_suite, predicate, NULL, NULL, NULL, NULL);

Adding tests to a suite
***********************

There are 4 macros used to add a test to a suite, they are:

* :c:macro:`ZTEST` ``(suite_name, test_name)`` - Which can be used to add a test by ``test_name`` to a
  given suite by ``suite_name``.
* :c:macro:`ZTEST_USER` ``(suite_name, test_name)`` - Which behaves the same as :c:macro:`ZTEST`, only
  that when :kconfig:option:`CONFIG_USERSPACE` is enabled, then the test will be run in a userspace
  thread.
* :c:macro:`ZTEST_F` ``(suite_name, test_name)`` - Which behaves the same as :c:macro:`ZTEST`, only
  that the test function will already include a variable named ``fixture`` with the type
  ``<suite_name>_fixture``.
* :c:macro:`ZTEST_USER_F` ``(suite_name, test_name)`` - Which combines the fixture feature of
  :c:macro:`ZTEST_F` with the userspace threading for the test.

Test fixtures
=============

Test fixtures can be used to help simplify repeated test setup operations. In many cases, tests in
the same suite will require some initial setup followed by some form of reset between each test.
This is achieved via fixtures in the following way::

    #include <zephyr/ztest.h>

    struct my_suite_fixture {
      size_t max_size;
      size_t size;
      uint8_t buff[1];
    };

    static void *my_suite_setup(void)
    {
      /* Allocate the fixture with 256 byte buffer */
      struct my_suite_fixture *fixture = k_malloc(sizeof(struct my_suite_fixture) + 255);

      zassume_not_null(fixture, NULL);
      fixture->max_size = 256;

      return fixture;
    }

    static void my_suite_before(void *f)
    {
      struct my_suite_fixture *fixture = (struct my_suite_fixture *)f;
      memset(fixture->buff, 0, fixture->max_size);
      fixture->size = 0;
    }

    static void my_suite_teardown(void *f)
    {
      k_free(f);
    }

    ZTEST_SUITE(my_suite, NULL, my_suite_setup, my_suite_before, NULL, my_suite_teardown);

    ZTEST_F(my_suite, test_feature_x)
    {
      zassert_equal(0, fixture->size);
      zassert_equal(256, fixture->max_size);
    }


Advanced features
*****************

Test result expectations
========================

Some tests were made to be broken. In cases where the test is expected to fail or skip due to the
nature of the code, it's possible to annotate the test as such. For example::

  .. code-block:: C

  #include <zephyr/ztest.h>

  ZTEST_SUITE(my_suite, NULL, NULL, NULL, NULL, NULL);

  ZTEST_EXPECT_FAIL(my_suite, test_fail)
  ZTEST(my_suite, test_fail)
  {
    /** This will fail the test */
    zassert_true(false, NULL);
  }

  ZTEST_EXPECT_SKIP(my_suite, test_fail)
  ZTEST(my_suite, test_skip)
  {
    /** This will skip the test */
    zassume_true(false, NULL);
  }

In this example, the above tests should be marked as failed and skipped respectively. Instead,
Ztest will mark both as passed due to the expectation.

Test rules
==========

Test rules are a way to run the same logic for every test and every suite. There are a lot of cases
where you might want to reset some state for every test in the binary (regardless of which suite is
currently running). As an example, this could be to reset mocks, reset emulators, flush the UART,
etc.

  .. code-block:: C

    #include <zephyr/fff.h>
    #include <zephyr/ztest.h>

    #include "test_mocks.h"

    DEFINE_FFF_GLOBALS;

    DEFINE_FAKE_VOID_FUN(my_weak_func);

    static void fff_reset_rule_before(const struct ztest_unit_test *test, void *fixture)
    {
      ARG_UNUSED(test);
      ARG_UNUSED(fixture);

      RESET_FAKE(my_weak_func);
    }<commit_msg>doc: Remove double code block anchor<commit_after>.. _test-framework:

Test Framework
###############

The Zephyr Test Framework (Ztest) provides a simple testing framework intended
to be used during development.  It provides basic assertion macros and a generic
test structure.

The framework can be used in two ways, either as a generic framework for
integration testing, or for unit testing specific modules.

To enable the latest APIs of Ztest simply set :kconfig:option:`CONFIG_ZTEST_NEW_API=y`. The legacy
APIs will soon be deprecated and eventually removed.

Creating a test suite
*********************

Using Ztest to create a test suite is as easy as calling the :c:macro:`ZTEST_SUITE`. The macro
accepts the following arguments:

* ``suite_name`` - The name of the suite. This name must be unique within a single binary.
* :c:type:`ztest_suite_predicate_t` - An optional predicate function to allow choosing when the
  test will run. The predicate will get a pointer to the global state passed in through
  :c:func:`ztest_run_all` and should return a boolean to decide if the suite should run.
* :c:type:`ztest_suite_setup_t` - An optional setup function which returns a test fixture. This
  will be called and run once per test suite run.
* :c:type:`ztest_suite_before_t` - An optional before function which will run before every single
  test in this suite.
* :c:type:`ztest_suite_after_t` - An optional after function which will run after every single
  test in this suite.
* :c:type:`ztest_suite_teardown_t` - An optional teardown function which will run at the end of
  all the tests in the suite.

Below is an example of a test suite using a predicate::

    #include <zephyr/ztest.h>
    #include "test_state.h"

    static bool predicate(const void *global_state)
    {
      return ((const struct test_state*)global_state)->x == 5;
    }

    ZTEST_SUITE(alternating_suite, predicate, NULL, NULL, NULL, NULL);

Adding tests to a suite
***********************

There are 4 macros used to add a test to a suite, they are:

* :c:macro:`ZTEST` ``(suite_name, test_name)`` - Which can be used to add a test by ``test_name`` to a
  given suite by ``suite_name``.
* :c:macro:`ZTEST_USER` ``(suite_name, test_name)`` - Which behaves the same as :c:macro:`ZTEST`, only
  that when :kconfig:option:`CONFIG_USERSPACE` is enabled, then the test will be run in a userspace
  thread.
* :c:macro:`ZTEST_F` ``(suite_name, test_name)`` - Which behaves the same as :c:macro:`ZTEST`, only
  that the test function will already include a variable named ``fixture`` with the type
  ``<suite_name>_fixture``.
* :c:macro:`ZTEST_USER_F` ``(suite_name, test_name)`` - Which combines the fixture feature of
  :c:macro:`ZTEST_F` with the userspace threading for the test.

Test fixtures
=============

Test fixtures can be used to help simplify repeated test setup operations. In many cases, tests in
the same suite will require some initial setup followed by some form of reset between each test.
This is achieved via fixtures in the following way::

    #include <zephyr/ztest.h>

    struct my_suite_fixture {
      size_t max_size;
      size_t size;
      uint8_t buff[1];
    };

    static void *my_suite_setup(void)
    {
      /* Allocate the fixture with 256 byte buffer */
      struct my_suite_fixture *fixture = k_malloc(sizeof(struct my_suite_fixture) + 255);

      zassume_not_null(fixture, NULL);
      fixture->max_size = 256;

      return fixture;
    }

    static void my_suite_before(void *f)
    {
      struct my_suite_fixture *fixture = (struct my_suite_fixture *)f;
      memset(fixture->buff, 0, fixture->max_size);
      fixture->size = 0;
    }

    static void my_suite_teardown(void *f)
    {
      k_free(f);
    }

    ZTEST_SUITE(my_suite, NULL, my_suite_setup, my_suite_before, NULL, my_suite_teardown);

    ZTEST_F(my_suite, test_feature_x)
    {
      zassert_equal(0, fixture->size);
      zassert_equal(256, fixture->max_size);
    }


Advanced features
*****************

Test result expectations
========================

Some tests were made to be broken. In cases where the test is expected to fail or skip due to the
nature of the code, it's possible to annotate the test as such. For example:

  .. code-block:: C

    #include <zephyr/ztest.h>

    ZTEST_SUITE(my_suite, NULL, NULL, NULL, NULL, NULL);

    ZTEST_EXPECT_FAIL(my_suite, test_fail)
    ZTEST(my_suite, test_fail)
    {
      /** This will fail the test */
      zassert_true(false, NULL);
    }

    ZTEST_EXPECT_SKIP(my_suite, test_fail)
    ZTEST(my_suite, test_skip)
    {
      /** This will skip the test */
      zassume_true(false, NULL);
    }

In this example, the above tests should be marked as failed and skipped respectively. Instead,
Ztest will mark both as passed due to the expectation.

Test rules
==========

Test rules are a way to run the same logic for every test and every suite. There are a lot of cases
where you might want to reset some state for every test in the binary (regardless of which suite is
currently running). As an example, this could be to reset mocks, reset emulators, flush the UART,
etc.

  .. code-block:: C

    #include <zephyr/fff.h>
    #include <zephyr/ztest.h>

    #include "test_mocks.h"

    DEFINE_FFF_GLOBALS;

    DEFINE_FAKE_VOID_FUN(my_weak_func);

    static void fff_reset_rule_before(const struct ztest_unit_test *test, void *fixture)
    {
      ARG_UNUSED(test);
      ARG_UNUSED(fixture);

      RESET_FAKE(my_weak_func);
    }
{'fraction_non_alphanumeric': 0.10148107515085025, 'fraction_numerical': 0.003474126897056135, 'mean_word_length': 3.8574600355239785, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 29, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3364835', 'n_tokens_mistral': 3487, 'n_tokens_neox': 3323, 'n_words': 1289}