.. _omnigraph_versioning_testing:

Testing Version Upgrades
########################

In order to properly support backward compatibility we need tests in our own code that verify that any code that has not
been fully deprecated continues to be accessible and, if possible, functions as intended.

In an ideal world we would have tests that have full code coverage and when we deprecate some code we would move the
tests exercising that old functionality to the deprecated section and new tests would be added to exercise the new code.

Practically speaking this isn't possible so there will be a compromise between thoroughness and test time.

These are some of the areas to consider when designing tests for versioning:
- Use the ETM (Extension Test Matrix) to verify our new extension versions against existing users of our extensions
- For Python code, create "button tests" where every import in our API surface is checked for type and existence
- For C++ code create "ABI tests" whose only job is to exercise every function in the ABI
- For Python bindings create some combination of the above two that verify every function in the ABI

Creating these tests will require some test scaffolding that allows us to confirm coverage. Ideally this would take the
form of a code-coverage tool so that we don't have to write our own as that is prone to omissions.

.. note::

    Performance testing is an important part of compatibility however it will be handled in a separate effort.
