.. _ogn_node_conversion:

Tutorial - OmniGraph .ogn Conversion
====================================

This is a walkthrough of the conversion of an OmniGraph node written under the original DataModel and ABI architectures
to the .ogn format, which uses the new DataModel architecture.

.. note:: If you do not have build support for .ogn files yet, refer first to the :ref:`ogn_build_conversion` document.

The test node used will be the **Time** node, in the `omni.graph.core` extension. In order to fully convert
the file a series of steps must be followed.

.. note:: bash/gitbash commands are used to illustrate actions where required. Any other equivalent method is fine.
          For the commands the current directory is assumed to be ``source/extensions/omni.graph.core/``.

.. note:: This tutorial is focussed on conversion of C++ nodes. Other types will be handled in separate tutorials.

Before the node can be converted the build has to first be set up to

.. contents:: Node Conversion Steps

Identify attribute names
------------------------

Before modifying any code the .ogn file should be created, in the location described by :ref:`ogn_build_conversion`.
The key components of that file are the node type name, description, version, and the list of attributes, their types,
descriptions, and defaults.

The node type name for most existing nodes is found in the ``getNodeType()`` method.
By looking at the ``initializeType()`` method in the current node it's easy to determine the name of all of the
attributes.

.. code-block:: c++
    :linenos:
    :emphasize-lines: 5,11-15

    struct TimeComputeNode
    {
        static const char* getNodeType()
        {
            return "Time";
        }

    static void initializeType(const NodeTypeObj& nodeTypeObj)
    {
        const INodeType* iNodeType = nodeTypeObj.iNodeType;
        iNodeType->addOutput(nodeTypeObj, "output:time", "double", true, nullptr, nullptr);
        iNodeType->addOutput(nodeTypeObj, "output:frame", "double", true, nullptr, nullptr);
        iNodeType->addOutput(nodeTypeObj, "output:fps", "double", true, nullptr, nullptr);
        iNodeType->addOutput(nodeTypeObj, "output:elapsedTime", "double", true, nullptr, nullptr);
        iNodeType->addOutput(nodeTypeObj, "output:timeSinceStart", "double", true, nullptr, nullptr);
    }

From this little bit of information the skeleton of the file ``OgnTime.ogn`` can be put in place. In this case we
see that all of the attributes are outputs of type `double` so that dictates the basic contents. It's a bit of an
unusual node in that almost every node will have both inputs and outputs. For the purposes of conversion the principles
are the same, you just put the attributes into the `inputs` subsection instead of `outputs`.

.. note::

    Not all attributes might be added to the node in the above way, relying on USD contents to set up the missing
    attributes. You can find any missing attributes by examining the USD file, which will contain the attribute's
    type and any non-standard default values.

.. code-block:: json

    {
        "Time" :
        {
            "version" : 1,
            "description" : [""],
            "outputs" :
            {
                "time":
                {
                    "description": [""],
                    "type": "double"
                },
                "frame":
                {
                    "description": [""],
                    "type": "double"
                },
                "fps":
                {
                    "description": [""],
                    "type": "double"
                },
                "elapsedTime":
                {
                    "description": [""],
                    "type": "double"
                },
                "timeSinceStart":
                {
                    "description": [""],
                    "type": "double"
                }
            }
        }
    }

A key point to note here is that attributes generated from a .ogn file are automatically namespaced; **inputs:** for
input attributes and **outputs:** for output attributes, so existing USD files or scripts that use the above attribute
names may have to be changed.

Also, in order to keep node names unique, the name of the extension is prepended to the name of the node. So for all
scripts and USD files you will need to perform this renaming:

+----------------+------------------------+
| Old Name       | New Name               |
+================+========================+
| Time           | omni.graph.core.Time   |
+----------------+------------------------+
| time           | outputs:time           |
+----------------+------------------------+
| frame          | outputs:frame          |
+----------------+------------------------+
| fps            | outputs:fps            |
+----------------+------------------------+
| elapsedTime    | outputs:elapsedTime    |
+----------------+------------------------+
| timeSinceStart | outputs:timeSinceStart |
+----------------+------------------------+

The descriptions of the attributes and the node may be found in the node's comments, or may just be understood by
the node writer. They can be filled in directly.


.. code-block:: c++
    :linenos:
    :emphasize-lines: 5-9

    static bool compute(const GraphContextObj& contextObj, const NodeObj& node)
    {
        const GraphContext::Globals globals = contextObj.context->getGlobals();

        setAttributeValue(contextObj, nodeObj, "output:time", (double)globals.currentTime);
        setAttributeValue(contextObj, nodeObj, "output:frame", (double)globals.frameTime);
        setAttributeValue(contextObj, nodeObj, "output:fps", (double)globals.fps);
        setAttributeValue(contextObj, nodeObj, "output:elapsedTime", (double)globals.elapsedTime);
        setAttributeValue(contextObj, nodeObj, "output:timeSinceStart", (double)globals.timeSinceStart);

        return true;
    }

Details of the C++ types available and their corresponding .ogn types can be found in :ref:`ogn_attribute_types`,
which we can use to deduce the types of this node's attributes. (Certain types are modified due to them being different
between the previous and current implementations of the DataModel; they should be relatively obvious on inspection of
the .ogn types document.)

Potential default values can also be deduced from the node algorithm. In this case zero values for every attribute
seems reasonable. When no default values are specified in the .ogn file zeroes are assumed so they do not need to be
explicitly added.

On inspection it seems like some of the attributes can have a minimum value set. In particular, `fps`, `elapsedTime`,
and `timeSinceStart` don't seem like they could be negative values so a minimum of 0.0 is appropriate. Although the .ogn
file will not yet enforce limits at runtime that is the intention in the future so these limits should be recorded.

This is all the information required to complete the .ogn file:

.. code-block:: json

    {
        "Time" :
        {
            "version" : 1,
            "description" : ["System built-in node to return various time information from kit."],
            "outputs" :
            {
                "time":
                {
                    "description": ["Current global time, in seconds"],
                    "type": "double"
                },
                "frame":
                {
                    "description": ["Current global frame time, in seconds"],
                    "type": "double"
                },
                "fps":
                {
                    "description": ["Current evaluation rate, in frames per second"],
                    "type": "double",
                    "minimum": 0.0
                },
                "elapsedTime":
                {
                    "description": ["Elapsed time in the current evaluation, in seconds"],
                    "type": "double",
                    "minimum": 0.0
                },
                "timeSinceStart":
                {
                    "description": ["Elapsed time since the start of the session, in seconds"],
                    "type": "double",
                    "minimum": 0.0
                }
            }
        }
    }

Paring down the includes
------------------------

The generated OGN header file will include everything it needs in order to access the DataModel so the redundant
includes can be removed from the source file **OgnTime.cpp**. The simplest approach is to remove all but the one
necessary inclusion of the database file and add back in any that are missing. In this case only one more file
is necessary, to access the global time information.

.. code-block:: c++
    :linenos:
    :emphasize-lines: 1

    #include <OgnTimeDatabase.h>
    #include "GraphContext.h"


Set Up Node Type
----------------
For consistency the class should be named the same as the file, and can be an actual *class* type since there are
no direct ties to the C ABI to it. The node type is set up by the generated code so this starting clause:

.. code-block:: c++

    struct TimeComputeNode
    {
        static const char* getNodeType()
        {
            return "Time";
        }

can be replaced with the single declaration:

.. code-block:: c++

    class OgnTime
    {
    public:


Remove Unnecessary ABI Setup
----------------------------
The generated code handles interfacing with the OmniGraph ABI so the function to get that interface can be deleted:

.. code-block:: c++
    :linenos:
    :emphasize-lines: 1-11

    INodeType getNodeInterfaceTime()
    {
        INodeType iface = {};

        iface.getNodeType = TimeComputeNode::getNodeType;
        iface.compute = TimeComputeNode::compute;
        iface.initialize = TimeComputeNode::initialize;
        iface.release = TimeComputeNode::release;

        return iface;
    }

It also handles the registration and deregistration of the node type interfaces so that code can be deleted, shown
in the file *omni.graph.core/plugins/PluginInterface.cpp* here:

.. code-block:: c++
    :linenos:
    :emphasize-lines: 1-4,10-21,26-33

    namespace
    {
    std::vector<const NodeTypeRegistration*> pluginNodeTypeRegistrations;
    }

    const IToken* Token::iToken = nullptr;

    CARB_EXPORT void carbOnPluginStartup()
    {
        auto iComputeGraph = carb::getFramework()->acquireInterface<IGraphRegistry>();
        if (!iComputeGraph)
        {
            CARB_LOG_ERROR("omni.graph.expression failed to register nodes due to missing interface");
            return;
        }

        std::for_each(pluginNodeTypeRegistrations.begin(), pluginNodeTypeRegistrations.end(),
                    [&iComputeGraph](const NodeTypeRegistration* nodeTypeRegistration) {
                        iComputeGraph->registerNodeType(
                            nodeTypeRegistration->nodeTypeInterface(), nodeTypeRegistration->nodeTypeVersion());
                    });
    }

    CARB_EXPORT void carbOnPluginShutdown()
    {
        auto iComputeGraph = carb::getFramework()->acquireInterface<IGraphRegistry>();
        if (!iComputeGraph)
            return;

        std::for_each(pluginNodeTypeRegistrations.begin(), pluginNodeTypeRegistrations.end(),
                    [&iComputeGraph](const NodeTypeRegistration* nodeTypeRegistration) {
                        iComputeGraph->unregisterNodeType(nodeTypeRegistration->nodeTypeInterface().getNodeType());
                    });
    }

The extension must also create the OGN node registration support so that all of its nodes are registered and
deregistered at the proper times. To do so, inclusion of the file *omni/graph/core/OgnHelpers.h* must be added,
and the two relevant macros it defines must be inserted. *DECLARE_OGN_NODES()* appears at file-static level, which
is easiest after the *CARB_PLUGIN* macros, *INITIALIZE_OGN_NODES()* is added in the *carbOnPluginStartup()* method,
and *RELEASE_OGN_NODES()* is added in the *carbOnPluginShutdown()* method.

.. code-block:: c++
    :linenos:
    :emphasize-lines: 1,28,38,43

    #include <omni/graph/core/OgnHelpers.h>
    #include <omni/graph/core/iComputeGraph.h>

    const struct carb::PluginImplDesc kPluginImpl = { "omni.graph.core.plugin", "OmnIGraph Core", "NVIDIA",
                                                      carb::PluginHotReload::eEnabled, "dev" };

    CARB_PLUGIN_IMPL(kPluginImpl,
                    omni::graph::core::ComputeGraph,
                    omni::graph::core::IAttribute,
                    omni::graph::core::IAttributeData,
                    omni::graph::core::IAttributeType,
                    omni::graph::core::IBundle,
                    omni::graph::core::IGraph,
                    omni::graph::core::IGraphContext,
                    omni::graph::core::IGraphRegistry,
                    omni::graph::core::INode,
                    omni::graph::core::IScheduleNode,
                    omni::graph::core::IDataStealingPrototype,
                    omni::graph::core::IGatherPrototype)

    CARB_PLUGIN_IMPL_DEPS(carb::flatcache::FlatCache,
                        carb::flatcache::IPath,
                        carb::flatcache::IToken,
                        carb::graphics::Graphics,
                        carb::filesystem::IFileSystem,
                        omni::gpucompute::GpuCompute,
                        carb::settings::ISettings)
    DECLARE_OGN_NODES()

    // carbonite interface for this plugin (may contain multiple compute nodes)
    void fillInterface(omni::kit::ITime& iface)
    {
        iface = {};
    }

    CARB_EXPORT void carbOnPluginStartup()
    {
        INITIALIZE_OGN_NODES()
    }

    CARB_EXPORT void carbOnPluginShutdown()
    {
        RELEASE_OGN_NODES()
    }

.. note::

    The functions *carbOnPluginStartup()* and *carbOnPluginShutdown()* are required by the extension manager, even if
    they are empty.

Modify Compute Signature
------------------------
The *compute* method required by the generated code has a different signature than the ABI. It is passed a single
parameter, which is a reference to the node's database interface object. Any access required to the former ABI
parameters for other reasons can be accomplished by going through the database object, hence this:

.. code-block:: c++

    static bool compute(const GraphContextObj& contextObj, const NodeObj& nodeObj)
    {
        const IGraphContext* const iContext = contextObj.iContext;
        const INode* const iNode = nodeObj.iNode;

becomes this:

.. code-block:: c++

    static bool compute(OgnTimeDatabase& db)
    {

Refactor Input Attribute Value access
-------------------------------------
Nodes are currently using direct access to the DataModel, either old or new, to get the values of their input
attributes. This is the primary function provided by the generated database class so that access must be
refactored to make use of it.

Often what you'll see in your compute method, when it contains input attributes, are calls like this:

.. code-block:: c++

    float noiseRatio, frequency;

    noiseRatio = getDataR<float>(contextObj, iNode->getAttribute(nodeObj, "noiseRatio"));
    frequency = getDataR<float>(contextObj, iNode->getAttribute(nodeObj, "frequency"));

Although it's not necessary, it can help improve clarity if the values extracted from the database are placed into
local variables for more natural access. You can look up the exact data types returned for a given attribute type
in :ref:`ogn_user_guide`, though for most simple situations you can just use the *auto* declaration. The
highlighted comments are not included in the final code, they are just there for explanation:

.. code-block:: c++
    :linenos:
    :emphasize-lines: 1-2,4-7

    // Inputs always appear in the db.inputs nested structure, and are named for the name of the attribute
    // as it appears in the .ogn file. (Any colons used for subnamespaces are converted to underscore for the name.)
    const auto& noiseRatio = db.inputs.noiseRatio();
    // Actual data type of the above is "const float&" but it's easiest to use plain "const auto&". It also maintains
    // consistency with more complex types.
    // The original variable names are preserved here but it's a good idea to use the attribute name as the
    // variable name, for consistency. (e.g. use "noiseFrequency" instead of "frequency")
    const auto& frequency = db.inputs.noiseFrequency();

Refactor Setting Output Attribute Values
----------------------------------------
This snippet of code shows a similar conversion for the time node's setting of output values, whose original code
was seen above:

.. code-block:: c++
    :linenos:

    const GraphContext::Globals globals = db.abi_context().context->getGlobals();

    db.outputs.time() = (double)globals.currentTime;
    db.outputs.frame() = (double)globals.frameTime;
    db.outputs.fps() = (double)globals.fps;
    db.outputs.elapsedTime() = (double)globals.elapsedTime;
    db.outputs.timeSinceStart() = (double)globals.timeSinceStart;

Adjust The Algorithm Data Types If Necessary
--------------------------------------------
Most existing nodes are using the USD data types as that is what the old DataModel supported (e.g. GfVec3f). If you've
already examined :ref:`ogn_user_guide` you'll note that the new DataModel uses these types by default. If you
are using anything else you would have to cast the values being returns as in :ref:`ogn_tutorial_tupleData`, or
modify the type definitions used by a file as in :ref:`ogn_type_definition_overrides`.


Add Tests
---------
This particular node uses global data and so is not suitable for adding localized test data for the automatically
generated tests. See :ref:`ogn_test_data` for more information on how to add tests of this nature. For this node we would
want to add scripted tests that do things like confirm that elapsedTime increases during evaluation, and that the time
values are changing during an animation playback.

Modify Attribute Names In Scripts Or USD Files
----------------------------------------------
Previously, attribute names had no well-defined structure and so there may be existing scripts or USD files (.usda in
particular) that reference those names which, under OGN rules may have changed. In extreme cases when content is in
USD binary format or in customer's hands where it can't be changed you may have to implemented the node's
*updateNodeVersion()* method to handle the renaming.

Here's an example of how this node looks in a USD file before conversion:

.. code-block:: usda

    def ComputeNode "timeNode"
    {
        custom token node:type = "Time"
        custom int node:typeVersion = 1
        double output:time
    }

To fix this we rename the existing attributes and node type, and add in the missing ones for completeness:

.. code-block:: usda
    :linenos:
    :emphasize-lines: 3,5-9

    def ComputeNode "timeNode"
    {
        custom token node:type = "omni.graph.core.Time"
        custom int node:typeVersion = 1
        double outputs:time
        double outputs:frame
        double outputs:fps
        double outputs:elapsedTime
        double outputs:timeSinceStart
    }

And this is a place where attributes were being accessed by a test script:

.. code-block:: python

    time_node = graph.get_node("/defaultPrim/timeNode")
    time_attr = time_node.get_attribute("output:time")

which is also corrected with a simple renaming:

.. code-block:: python
    :linenos:
    :emphasize-lines: 2

    time_node = graph.get_node("/defaultPrim/timeNode")
    time_attr = time_node.get_attribute("outputs:time")


Final Result - The Converted Node
---------------------------------
If all has gone well that should be all you need to do to fully convert your node to the .ogn format.
Adding the macro call ``REGISTER_OGN_NODE()`` at the end of the file ensures the node is properly registered when
the extension loads. Note the position of the namespaces.

Here is the final version of the Time node:

.. code-block:: c++
    :linenos:

    // Copyright (c) 2018-2021, NVIDIA CORPORATION.  All rights reserved.
    //
    // NVIDIA CORPORATION and its licensors retain all intellectual property
    // and proprietary rights in and to this software, related documentation
    // and any modifications thereto.  Any use, reproduction, disclosure or
    // distribution of this software and related documentation without an express
    // license agreement from NVIDIA CORPORATION is strictly prohibited.
    //
    #include <OgnTimeDatabase.h>
    #include "GraphContext.h"

    namespace omni {
    namespace graph {
    namespace core {

    class OgnTime
    {
    public:
        static bool compute(OgnTimeDatabase& db)
        {
            const GraphContext::Globals globals = db.abi_context().context->getGlobals();

            db.outputs.time() = (double)globals.currentTime;
            db.outputs.frame() = (double)globals.frameTime;
            db.outputs.fps() = (double)globals.fps;
            db.outputs.elapsedTime() = (double)globals.elapsedTime;
            db.outputs.timeSinceStart() = (double)globals.timeSinceStart;

            return true;
        }
    };

    REGISTER_OGN_NODE()

    } // namespace core
    } // namespace graph
    } // namespace omni