.. _omnigraph_versioning_python:

Python Support Versioning
#########################

Python has the simultaneous strength and weakness of introspection, where users can access anything that appears in
the Python scripts, making all of the Python deliverables a potential part of the API. This is not a reasonable amount
of code to support so part of the Python versioning involves defining the API "surface" that versioning support will
be provided for.

Python Bindings Support
***********************

Python bindings can be versioned in parallel with the ABI since for the most part they all correspond to ABI functions.
There are exceptions where bindings functions are made as variations of ABI functions to make them more Pythonic, or
to provide missing functionality like creation of Python class wrappers. These should be versioned in exactly the same
way as the ABI functions so in any given extension version the set of bindings will be consistent.

Defining The Python API Surface
*******************************
To support proper versioning we first have to be able to define what constitutes "a version" of a set of Python
scripts. The easiest way to do this is to use the import system to define the specific set of Python objects that
will be supported as part of a version in the way that many popular packages such as *numpy* and *pandas* do.

.. code-block:: python

    import omni.graph.core as og

The officially supported Python API "surface" is everything that can be imported in this way. There also might be
some submodules that have separate import support in the same way you can use both *import os* and *import os.path*.
The extension will provide the set of support imports in its help information.

Hiding Implementation Definitions
=================================
Python programmers have an understanding that elements with a leading underscore or double underscore ("dunder") are
not meant to be public so this can be used to signal this fact to anyone looking through the import structure.
This is done at a high level using a module structure that looks like this:

.. code-block:: text

    omni.my.extension/
    └── python/
        ├── __init__.py
        ├── _impl/
        |   extension.py
        |   my_script.py
        └── tests/
            └── test_my_extension.py

With this structure then anyone trying to import anything that's not officially supported would be faced with
something like this:

.. code-block:: python

    import omni.my.extension._impl.my_script
    my_script.my_script()

It's still possible to do this kind of thing but any Python programmer will realize there is something wrong with that.
The method for accessing this same function in an officially supported way would be this:

.. code-block:: python

    import omni.my.extension as me
    me.my_script()

Inside the *__init__.py* file the officially supported API surface is all imported from wherever it happens to be
defined so that it can be accessed in this way.

.. code-block:: python
    :caption: __init__.py

    """Define the API surface for omni.my.extension

    To access the API for this extension use this import:
        import omni.my.extension as me

    To access the testing support for this extension use this import:
        import omni.my.extension.tests as met
    """
    from ._impl.extension import PublicExtension  # Necessary for automatic extension initialization
    from ._impl.my_script import my_script

.. code-block:: python
    :caption: tests/__init__.py

    """Define the API surface for omni.my.extension.tests

    To access the testing support for this extension use this import:
        import omni.my.extension.tests as met
    """
    from ._impl.test_support import load_test_file

.. note::

    For future examples the comments will be elided for clarity but the API should always be commented.

Adding Version Support
**********************
After the soft deprecation period is over the deprecated features will move out of the main imports and into a new
location that is version-specific. This will allow continue support for deprecated functions without cluttering up
the active code.

.. warning::

    This is one approach to making a version-specific API layer, to be vetted.

A deprecated version should be presented to the user as a fully intact entity so that it can be used pretty much as
it was before deprecation. For example if I have upgraded my extension from version 1, which contained a function
*old_function()* to version 2, which deprecates that function and adds *new_function()* then the module structure will
be altered to this:

.. code-block:: text

    omni.my.extension/
    └── python/
        ├── __init__.py
        ├── _impl/
        |   extension.py
        |   my_script.py
        └── _1/
            __init__.py
            my_script.py

Only the new function will be exposed in the main API definition:

.. code-block:: python
    :caption: __init__.py

    from ._impl.extension import PublicExtension
    from ._impl.my_script import new_function

.. code-block:: python
    :caption: _1/__init__.py

    from .my_script import old_function

A-La Carte Access
=================
The most flexible method of access is to use a mix-and-match approach to pick up the exact versions of everything you
want. This is intentionally difficult to do as it's not a usage pattern we want to encourage, more of a back door that
allows users to revert to previous behaviour in very specific places.

.. code-block:: python
    :caption: user_code.py

    import omni.my.extension as me
    import omni.my.extension._1 as me1

    if some_condition:
        me.new_function()
    else:
        me1.old_function()

Always-Latest Access
====================
The most desired type of access is to use the latest API only so that when a new version is added your code will
already be using it. So long as we keep the top level *__init__.py* updated to define the latest API this is also the
easiest to use:

.. code-block:: python
    :caption: user_code.py

    import omni.my.extension as me

    me.new_function()

Fixed Version Access
====================
If users wish to lock to a specific version so that they can rely on a stable base they can lock to a specific version.
This will only affect them at some point in the future when something in that version is hard-deprecated.

.. code-block:: python
    :caption: user_code.py

    # Note the leading underscore on the version, hinting that it should not be accessed directly
    import omni.my.extension._1 as me

    me.old_function()

Module Constant Gated Access
============================
We also want to support a hybrid approach where the users can enable the new interface if they want in order to check
whether they need to modify any code for compatibility, while not being locked into it if any work required cannot
happen all at once.

A Pythonic approach to this problem is to modify the interface definition to have a module-level constant that can be
set to different values in order to dynamically flip between different versions using the same import statement.

.. code-block:: python
    :caption: __init__.py

    import os
    __version = os.getenv("omni.my.extension", 2)

    if __version == 1:
        from ._1 import *
    elif __version == 2:
        from ._impl.my_script import new_function
    else:
        raise VersionError(f"Environment variable omni.my.extension must be in [1, 2] - got {__version})

.. note::

    This is intentionally not something that can change at runtime as that would have unpredictable effects. Something more
    sophisticated like a sequence of **unload module** --> **edit environment variable** --> **reload module** would be
    required to support that.

Deprecation Of Python Objects
*****************************
In order to follow the :ref:`omnigraph_deprecation_approach` for Python objects there has to be different treatment
for different types of Python objects so as to provide the deprecation information to the user in an informative but
non-intrusive manner.

The soft deprecation stage is especially import in Python as the things that can be deprecated can be small and change
at a relatively high frequency.

To make it easier to deprecate Python code a number of decorators have been created which can be accessed through
this import:

.. code-block:: python

    import omni.graph.tools.deprecate as ogd
    help(ogd)

You can see the details of how to deprecate everything in the :ref:`Python deprecation documentation<omnigraph_deprecation>`.
