.. _omnigraph_versioning_nodes:

Node Type Versioning
####################

Although it will probably not happen too frequently, the implementation of node types will occasionally change in such a
way that makes them incompatible with existing uses. In extreme cases the best course of action might be to create an
entirely new node type with the new functionality. For less intrusive changes these are the types of modifications
for which OmniGraph supports backward compatibility.

- :ref:`omnigraph_versioning_nodes_abi`
- :ref:`omnigraph_versioning_nodes_renaming`
- :ref:`omnigraph_versioning_nodes_removing`
- :ref:`omnigraph_versioning_nodes_modifying`
- :ref:`omnigraph_versioning_nodes_collisions`

.. hint::

    Most node type version changes will result in a corresponding minor version bump of the extension in which they live.

.. _omnigraph_versioning_nodes_abi:

ABI Versioning
**************

ABI versioning for node types should follow the basic Carbonite guidelines. `INodeType` is a Carbonite interface, so its
backward compatibiliy is assured by following the update guidelines for it described in :ref:`omnigraph_versioning_carbonite`.

.. _omnigraph_versioning_nodes_renaming:

Renaming A Node Type
********************

From time to time users may wish to change the name of their node without changing the node itself, perhaps because they
came across a better name, to fix a misspelling, or to prevent conflicts with an existing name. The functionality to do
this is currently hardcoded in the Node.cpp file but should be surfaced to the ABI. Here’s a potential addition to the
INodeType interface to make it happen:

.. code-block:: cpp

    /**
    * Create an alternate name for a node type, most often used to support renaming a node type.
    * When the node type is deregistered the alternate name will be automatically removed as well.
    *
    * @param[in] alternateName Secondary name that can be used interchangeably with nodeTypeName
    * @param[in] nodeTypeName Name of the node type for which an alternate is to be created
    * @return True if the alternate name was successfully set up, false if the node type name could
    *         not be found or the alternateName was already in use (both issue a warning)
    */
    bool(CARB_ABI* createNodeTypeAlias)(char const* alternateName, char const* nodeTypeName);


The alternate name(s) will be specified in the .ogn file so that the node definition can control them directly.

.. code-block:: json

    {
        "MySpiffyNode":
        {
            "$comment": "...other stuff...",
            "alias": "MySpifyNode"
        }
    }

.. warning::
    This feature is not yet implemented. Move to user-side documentation once it is.

.. _omnigraph_versioning_nodes_removing:

Removing A Node Type
********************

From time to time a user may wish to withdraw support for a node, perhaps because they have a more functional replacement,
or the function it provided is no longer relevant. Support will be added to support our soft deprecation path whereby
the node first starts issuing deprecation warnings when it is instantiated and then later will be removed entirely.

The second phase just involves deletion of the node from the extension, which still retains some measure of backward
compatibility as the user can always enable an earlier version of the extension to bring it back. The first phase will
be supported in a standard way with this ABI function on INodeType:

.. code-block:: cpp

    /**
     * Mark a node type as being deprecated with a message for the user on action they should take.
     *
     * @param[in] nodeObj Node to which this function applies
     * @param[in] deprecationMsg Message to the user describing what to do before the deprecation happens
     */
    void(CARB_ABI* deprecate)(NodeObj& nodeObj, char const* deprecationMsg);

    /**
     * Return the deprecation message for an node.
     * 
     * @param[in] nodeObj node to which this function applies
     * @return String containing the node deprecation message (nullptr if the node is not deprecated)
     */
    char const*(CARB_ABI* deprecationMessage)(NodeObj const& nodeObj);

The soft deprecation is enabled through a new .ogn keyword:

.. code-block:: json

    {
        "MySpiffyNode":
        {
            "$comment": "...other stuff...",
            "deprecate": "This node will be removed in the next release"
        }
    }

.. warning::
    This feature is not yet implemented. Move to user-side documentation once it is.

.. _omnigraph_versioning_nodes_modifying:

Modifying Attributes
********************

Currently the `updateNodeVersion` method can be overridden on the node type implementation to support arbitrary changes
to attributes or internal node state, however it happens after the node initialization and therefore cannot support
things like moving connections. We've already seen several examples of desired upgrades, including the ability to have
a soft deprecation where old attributes remain in place, so this mechanism needs to be extended to be more robust.

The changes to a node type that we will have a built-in upgrade path for are:
- Renaming an attribute
- Soft deprecation of an attribute

Things that are not planned to be handled directly include:
- Removal of an attribute
- Replacement of an attribute with another attribute or attributes

.. _omnigraph_renaming_attributes:

Renaming Attributes
===================
Renaming an attribute can only be done if the attribute keeps all of the defining properties and only the name changes.
This includes the port type, data type, default value, and memory type. The metadata, such as UI name, hidden state,
etc. can be changed if required without affecting the attribute renaming.

This is what the attribute renaming would look like in the .ogn file, where an input named "x" changes its name to
"red".

.. code-block:: json
    :caption: Before the renaming

    {
        "MyNode": {
            "version": 1,
            "description": "This is my node",
            "inputs": {
                "x": {
                    "type": "float",
                    "description": "The first value",
                    "uiName": "First Value"
                }
            }
        }
    }

.. code-block:: json
    :caption: After the renaming
    :emphasize-lines: 3,7,9,10

    {
        "MyNode": {
            "version": 2,
            "description": "This is my node",
            "inputs": {
                "red": {
                    "oldName": ["x", 1],
                    "type": "float",
                    "description": "The red value",
                    "uiName": "Red Value"
                }
            }
        }
    }

The **oldName** keyword takes a pair of values consisting of the old attribute name and the last node version before
it was renamed. There can be multiple pairs on the off chance that you rename the same attribute more than once.
Notice how the node type version number was incremented as well, to tell OmniGraph that this implementation is
different from the previous version.

In order to support this there must be an addition to the `IAttribute` interface in the ABI that can remember this
information for processing. This is what it might look like:

.. code-block:: cpp

    /**
     * Define an old name by which an attribute was known.
     *
     * @param[in] attributeObj Attribute to which this function applies
     * @param[in] oldName Old name by which the attribute was known
     * @param[in] lastVersion Last node type version in which the old name was valid
     */
    void(CARB_ABI* renamedFrom)(AttributeObj& attributeObj, char const* oldName, int lastVersion);

.. warning::
    This feature is not yet implemented. Move to user-side documentation once it is.

Soft Deprecation Of Attributes
==============================
In order to respect our :ref:`omnigraph_soft_deprecation`, when an attribute is to be removed it must be flagged
as deprecated in its .ogn file, along with a message telling users what they must do to prepare for the loss of the
attribute, if possible. The node type version does not yet need to be bumped as nothing has really changed for it at
this point.

.. code-block:: json
    :emphasize-lines: 3,7,9,10

    {
        "MyNode": {
            "version": 1,
            "description": "This is my node",
            "inputs": {
                "x": {
                    "type": "float",
                    "description": "The first value",
                    "uiName": "First Value",
                    "deprecated": "Use 'offset' instead."
                },
                "offset": {
                    "type": "float",
                    "description": "This will be added to all values."
                }
            }
        }
    }

This flag is set on the Attribute using `IInternal::deprecateAttribute()` but there should be no need to call
this method outside of the node generator tool. `IAttribute::isDeprecated()` can be called to retrieve the flag and
`IAttribute::deprecationMessage()` will return the associated message. 

When actual deprecation occurs then the appropriate action will take the place of the deprecation message - renaming,
removing, or replacing the attribute.

Modifying Node Behaviour
========================
In addition to changes in attributes, there may also be a change in attribute interpretation or of computational
algorithm. A simple example might be the switch to a more numerically stable integration method, or the addition of
an extra parameter that can switch computation type.

For any change that affects how the node is computed you must bump the node type version number. A mismatch in version
number for a node being created and the current node implementation will trigger a callback to the `updateNodeVersion`
function, which your node can override if there is some cleanup you can do that will help maintain expected operation
of the node.

.. _omnigraph_versioning_nodes_collisions:

Generated Code Rearrangement
****************************

As the number of nodes and variety of use cases has grown some weaknesses in the structure of the generated code
directories has been found. This section will describe the problems to be solved without yet any proposed solutions.

Node Collisions
===============
The generated OgnXXDatabase.h files are all stored in a single directory `_build/ogn/include`, which opens the
possibility of name collisions as well as keeping the contents of an extension self-contained.

Global Documentation
====================
The generated OgnXX.rst files are all stored in a single directory `_build/ogn/docs`, which while allowing creation of
an index that lists all available nodes opens the possibility of collisions as above, and does not tie the documentation
of individual nodes to the extensions that implement them.

This problem is even more apparent with external/downstream extensions which aren't included in the Kit build and which
therefore will not appear in documentation anywhere.

There is also a lack of ability for a user to jump from a node type definition or node instantiation to the
documentation for that node type, which would be helpful in understanding how to use a node type. Bits and pieces
appear throughout the UI (descriptions, attribute types, etc. as tooltips) but this is not the same as a single source
of everything related to the node type.

Python Builds
=============
The Python node types can be built at runtime so while it's helpful to have the prebuilt OgnXXDatabase.py files
available it's not strictly necessary. A user should be able to include a Python node type in an extension simply by
linking the directory to which it belongs to their build module directory, or even in their extension's directory if
they are using the local Python extension approach.

Getting in the way of this are first the lack of versioning support in the generated code, especially for local
extensions, so that if a user loads their extension using multiple versions of the Kit SDK the generated code will
only match one of them. Ideally they would like to support both and the generated code would be a cache, similar to
how packman works. The Warp team has implemented something like this already for their own code generator.

The Symlinked ogn Directory
===========================
Also a problem in the Python generation is how the extension is expecting an `/ogn/` directory in their Python module
that has a `nodes/` subdirectory symlinked back to the source for their nodes. The latter is to support hot reload,
however the method of doing that breaks the standard method in the build system. If, as above, the generated files
appeared elsewhere then there would be no need for the `/ogn/` directory and any `nodes/` subdirectory could be
linked directly by the build for hot reload support.

The ogn/tests Directory
=======================
Similarly to the above the generated tests appear in the `/ogn/tests` subdirectory and the .usda files used for the
tests appear in the `/ogn/tests/usd` subdirectory below that. These directories may also vary based on the version of
the code generator and the version of the node type. They may also be generated at runtime rather than at build time,
making the current mechanism for locating and registering the tests ineffective. Similarly, if outdated versions exist
then they should not be registered and run, so the whole test registration mechanism may need rethinking.

Supporting Multiple Node Type Versions
======================================
At the moment although there is ABI support for creating multiple versions of the same node type at the same time
there is no API support, so that is a necessary addition to make before the code generator can fully support them.

Setting that aside, for the code generation there is currently no support for multiple versions of the same node type.
The paths and object names are hardcoded to just the node type and/or class name without a version number attached to
it. This would also suffer from the same file collisions cited above, exacerbated by the fact that the names for
multiple versions would currently be exactly the same.

Other Possible Node Type Changes
********************************

This is a catch-all to mention any types of node modifications that won't have intrinsic support. They are listed here
to make note of the fact that they have been considered, and if they occur often may merit more direct support.

- Node type consolidation. e.g. we had a vertical stack node type and a horizontal node stack type and they were
  replaced by a single stack node type with an attribute defining the direction.
- Node language conversion. A node that was implemented in Python that is changed to be implemented in C++. Ideally
  both might be available to the user as the Python one is easier to debug but the C++ one is faster.
- Device targeting. A node might wish to be implemented on both the CPU and the GPU so that an intelligent scheduler
  can choose where to execute the node at runtime. Or a node type implementation may just switch from one to the other
  between versions for other reasons.
