OmniGraph And USD
#################

USD is used as a source of data for OmniGraph. Both can be described as a set of nodes, attributes, and connections,
each with their own set of strengths and restrictions.

USD is declarative - it has no system for procedurally generating attribute values (such as expression binding,
keyframing, etc). Timesamples can give different values when requested at different times, however they too are
declarative - they are fixed values at fixed times and are unaffected by any configuration of the USD stage outside of
composition. OmniGraph adds a procedural engine to Omniverse on top of USD that does generate computed attribute values.
OmniGraph is not constrained by the data values in its definition. It can create entirely new values at runtime based
on the computation algorithms in its nodes.

A USD stage can contain several layers forming "opinions" on what its declared values should be, which are composed
together to form the final result. OmniGraph only operates on the composed USD stage in general, although it is
possible for special purpose nodes to access the USD layer data through the USD APIs.

.. _omnigraph_fabric_sync:

In order to store and manage data efficiently, OmniGraph uses a data storage component called *Fabric*, which is a
part of Omniverse. After the USD stage is composed, USD prim data is read into *Fabric*, where it can be efficiently
accessed by OmniGraph for its computations. When appropriate, OmniGraph directs *Fabric* to synchronize with USD and the
computed data becomes available to the USD stage.

.. mermaid::

    sequenceDiagram
        autonumber
        USD->>Fabric: Populate Node Data
        loop Computation
            Fabric->>OmniGraph: Read Data For Computation
            OmniGraph->>Fabric: Update With Results Of Computation
        end
        Fabric-->>USD: Sync Computed Data

The nature of the computation being performed by OmniGraph will determine the interval for syncing computed data back
to USD. If, for example, the graph is deforming a mesh for passing on to some external stress analysis software then the
resulting mesh must be synced to USD on each playback frame
so that each version of the deformed mesh is available for analysis. On the other hand if the graph is performing
a large set of Machine Learning calculations for the purposes of training a model then it only needs to send the
results back to USD when the training is complete and the resulting model needs to be stored.

.. important::

    While some nodes within OmniGraph access USD data, and most have a direct correspondence with a USD prim, neither
    of these is required for OmniGraph to work properly. If an OmniGraph node is only required temporarily to perform
    some intermediate calculation, for example, it might only be known to *Fabric* and never appear in the USD stage.

.. note::

    To avoid awkward wording the generic term "graph" will be used herein when discussing the component of the
    OmniGraph subsystem that represents a graph to be evaluated. OmniGraph as a whole comprises the collection
    of components required to perform runtime evaluation (graphs, evaluators, schedulers) and their interaction.

How The Graph Appears In USD
****************************
We'll start with a simple example of how a graph looks in the USD Ascii format and the progressively fill in the details
of exactly what each part of it represents.

The graph will contain two nodes connected together to perform the two-part computation of multiplying a value
by four using two nodes whose computation multiplies a value by two.

.. code-block:: usd

    def OmniGraph "Graph"
    {
        token evaluator:type = "push"
        token fabricCacheBacking = "Shared"
        int2 fileFormatVersion = (1, 4)
        token pipelineStage = "pipelineStageOnDemand"

        def OmniGraphNode "times_2_node_1"
        {
            token node:type = "omni.graph.nodes.Times2"
            int node:typeVersion = 1
            custom double inputs:a = 2.5
            custom double outputs:two_a = 5.0
        }

        def OmniGraphNode "times_2_node_2"
        {
            token node:type = "omni.graph.nodes.Times2"
            int node:typeVersion = 1
            custom double inputs:a = 0.0
            custom double inputs:a.connect = </Graph/times_2_node_1.outputs:two_a>
            custom double outputs:two_a = 10.0
        }
    }


The Graph
=========
The graph itself is the top level USD prim of type **OmniGraph**. It serves to define the parameters of the graph
definition in its attribute values, and as a scope for all of the nodes that are contained within the graph.

.. important::

    Nodes must all be contained within the scope of an **OmniGraph** prim, and no prims that are not of type
    **OmniGraph** or **OmniGraphNode** are allowed within that scope.

.. code-block:: usd
    :emphasize-lines: 3-6

    def OmniGraph "Graph"
    {
        token evaluator:type = "push"
        token fabricCacheBacking = "Shared"
        int2 fileFormatVersion = (1, 4)
        token pipelineStage = "pipelineStageOnDemand"

        def OmniGraphNode "times_2_node_1"
        {
            token node:type = "omni.graph.nodes.Times2"
            int node:typeVersion = 1
            custom double inputs:a = 2.5
            custom double outputs:two_a = 5.0
        }

        def OmniGraphNode "times_2_node_2"
        {
            token node:type = "omni.graph.nodes.Times2"
            int node:typeVersion = 1
            custom double inputs:a = 0.0
            custom double inputs:a.connect = </Graph/times_2_node_1.outputs:two_a>
            custom double outputs:two_a = 10.0
        }
    }

.. mermaid::

    classDiagram
        class Graph
        Graph: token evaluator.type
        Graph: token fabricCacheBacking
        Graph: int2 fileFormatVersion
        Graph: token pipelineStage

The predefined attributes in the OmniGraph prim come from the OmniGraph prim schema, generated using the standard
`USD Schema Definition API <https://graphics.pixar.com/usd/release/tut_generating_new_schema.html>`_. This is
the generated schema definition.

.. code-block:: usd

    class OmniGraph "OmniGraph" (
        doc = "Base class for OmniGraph primitives containing the graph settings."
    )
    {
        token evaluationMode = "Automatic" (
            allowedTokens = ["Automatic", "Standalone", "Instanced"]
            doc = """Token that describes under what circumstances the graph should be evaluated. \r
                    Automatic mode will evaluate the graph in Standalone mode when \r
                    no Prims having a relationship to it exist. Otherwise it will be \r
                    evaluated in Instanced mode.\r
                    Standalone mode evaluates the graph with itself as the graph\r
                    target, and ignores any Prims having relationships to the graph Prim. \r
                    Use this mode with graphs that are expected to run only once per frame\r
                    and use explicit paths to Prims on the stage. \r
                    Instanced graphs will only evaluate the graph when it is part of a \r
                    relationship from an OmniGraphAPI interface. Each Prim with a  \r
                    relationship to the graph will cause a unique evaluation with the \r
                    graph target set to the path of the Prim having the relationship. \r
                    Use this mode when the graph represents an asset or template, and \r
                    parameterizes Prim paths through the use of the GraphTarget node,\r
                    variables, or similar mechanisms. """
        )
        token evaluator:type = "push" (
            allowedTokens = ["dirty_push", "push", "pull", "execution", "execution_pull"]
            doc = "Type name for the evaluator used by the graph."
        )
        token fabricCacheBacking = "Shared" (
            allowedTokens = ["Shared", "StageWithHistory", "StageWithoutHistory"]
            doc = "Token that identifies the type of FabricCache backing used by the graph."
        )
        int2 fileFormatVersion (
            doc = """A pair of integers consisting of the major and minor versions of the\r
                    file format under which the graph was saved."""
        )
        token pipelineStage = "pipelineStageSimulation" (
            allowedTokens = ["pipelineStageSimulation", "pipelineStagePreRender", "pipelineStagePostRender", "pipelineStageOnDemand"]
            doc = """Optional token which is used to indicate the role of a vector\r
                    valued field. This can drive the data type in which fields\r
                    are made available in a renderer or whether the vector values\r
                    are to be transformed."""
        )
    }

.. note::

    This schema is still in development and will most likely change in the upcoming releases.

The Nodes
=========
A graph is more than just a graph prim on its own. Within its scope are an interconnected set of nodes with attribute
values that are a visual representation of the computation algorithm of the graph. Each of these nodes are represented
in USD as prims with the schema type **OmniGraphNode**.

.. code-block:: usd
    :emphasize-lines: 8-14,16-23

    def OmniGraph "Graph"
    {
        token evaluator:type = "push"
        token fabricCacheBacking = "Shared"
        int2 fileFormatVersion = (1, 4)
        token pipelineStage = "pipelineStageOnDemand"

        def OmniGraphNode "times_2_node_1"
        {
            token node:type = "omni.graph.nodes.Times2"
            int node:typeVersion = 1
            custom double inputs:a = 2.5
            custom double outputs:two_a = 5.0
        }

        def OmniGraphNode "times_2_node_2"
        {
            token node:type = "omni.graph.nodes.Times2"
            int node:typeVersion = 1
            custom double inputs:a = 0.0
            custom double inputs:a.connect = </Graph/times_2_node_1.outputs:two_a>
            custom double outputs:two_a = 10.0
        }
    }

.. mermaid::

    flowchart LR
        subgraph Graph
            times_2_node_1 --> times_2_node_2
        end


The predefined attributes in the **OmniGraphNode** prim come from the **OmniGraphNode** prim schema, also defined
through the `USD Schema Definition API <https://graphics.pixar.com/usd/release/tut_generating_new_schema.html>`_.
This is the generated schema for the node's prim type.

.. code-block:: usd

    class OmniGraphNode "OmniGraphNode" (
        doc = "Base class for OmniGraph nodes."
    )
    {
        token node:type (
            doc = "Unique identifier for the name of the registered type for this node."
        )
        int node:typeVersion (
            doc = """Each node type is versioned for enabling backward compatibility. This value indicates which version of\r
                    the node type definition was used to create this node. By convention the version numbers start at 1."""
        )
    }

Connections
===========
Although OmniGraph connections between attributes are implemented and handled directly within OmniGraph they are
published within USD as *UsdAttribute* connections. Imagine a "Times2" node with one input and one output whose value is
computed as the input multiplied by two. Then you can connect two of these in a row to multiple the original input
by four. In USD this would look like:

.. code-block:: usd
    :emphasize-lines: 13,20-22

    def OmniGraph "Graph"
    {
        token evaluator:type = "push"
        token fabricCacheBacking = "Shared"
        int2 fileFormatVersion = (1, 4)
        token pipelineStage = "pipelineStageOnDemand"

        def OmniGraphNode "times_2_node_1"
        {
            token node:type = "omni.graph.nodes.Times2"
            int node:typeVersion = 1
            custom double inputs:a = 2.5
            custom double outputs:two_a = 5.0
        }

        def OmniGraphNode "times_2_node_2"
        {
            token node:type = "omni.graph.nodes.Times2"
            int node:typeVersion = 1
            custom double inputs:a = 0.0
            custom double inputs:a.connect = </Graph/times_2_node_1.outputs:two_a>
            custom double outputs:two_a = 10.0
        }
    }

.. mermaid::

    flowchart LR
        subgraph Graph
            times_2_node_1 --> times_2_node_2
        end

One thing worth mentioning here - in the second node */Graph/times_2_node_2* the value at *inputs:a*
defines a value of 0.0, which is used in the event that a connection is not authored, or the connection is not evaluated
(for example: outside an OmniGraph context), or an invalid connection is authored for the attribute.
Where a valid connection is authored, it will take precedence during OmniGraph evaluation. In this example this is
seen by the computation using the connected value of *5.0* as the multiplicand, not the authored value of *0.0*.


Graph Extras
============
The graph evaluation type is a hint to OmniGraph on how a set of nodes should be evaluated. One of the more common
variations is the type *execution*, which is usually referred to as an
:ref:`Action Graph<ogn_omni_graph_action_overview>`.


.. code-block:: usd
    :emphasize-lines: 3,9,14-16

    def OmniGraph "ActionGraph"
    {
        token evaluator:type = "execution"
        token fabricCacheBacking = "Shared"
        int2 fileFormatVersion = (1, 4)
        token pipelineStage = "pipelineStageOnDemand"

        def OmniGraphNode "DoNothingNode" (
            prepend apiSchemas = ["NodeGraphNodeAPI"]
        )
        {
            token node:type = "omni.graph.nodes.Noop"
            int node:typeVersion = 1
            uint inputs:execIn = 0
            uniform token ui:nodegraph:node:expansionState = "open"
            uniform float2 ui:nodegraph:node:pos = (-196.9863, 332.03223)
        }
    }

.. mermaid::

    flowchart LR
        subgraph ActionGraph
            DoNothingNode
        end

The key differentiator of an :ref:`Action Graph<ogn_omni_graph_action_overview>` is its use of
:ref:`execution attributes<omnigraph_usd_execution_attributes>` to trigger evaluation, rather than some external
event such as the update of a USD stage.

The additional two **ui:nodegraph** attributes shown in this example illustrate how extra information can be added
to the node and be stored with the prim definition. In this case the Action Graph editor has added the information
indicating the node's position and expansion state so that when the file is reloaded and the editor is opened the node
will appear in the same location it was last edited. The node graph editor has added the API schema *NodeGraphNodeAPI*
to help it in interpreting the extra data it has added. This is a standard Pixar USD schema that you can read more
about in the `USD documentation <https://graphics.pixar.com/usd/dev/api/class_usd_shade_node_graph.html>`_.


Custom Attributes
=================
Every node type can define its own set of custom attributes,
specified using the :ref:`.ogn format<ogn_reference_guide>`. These custom attributes are what an OmniGraph node uses
for its computation. Although the attributes are represented in USD, the actual values OmniGraph nodes use are
stored in *Fabric*. These are the values that need to be synchronized by OmniGraph as mentioned
:ref:`above<omnigraph_fabric_sync>`.

For the most part, every USD attribute type has a corresponding .ogn attribute type, though some may have some
different interpretations. The full :ref:`type correspondence<ogn_attribute_types>` shows how the types relate.

Here is how the definition of our "multiply by two" node might look in .ogn format:

.. code-block:: json

    {
        "Times2": {
            "version": 1,
            "description": "Multiplies a double value by 2.0",
            "inputs": {
                "a": {
                    "type": "double",
                    "description": "The number to be doubled"
                }
            },
            "outputs": {
                "two_a": {
                    "type": "double",
                    "description": "The input multiplied by 2.0"
                }
            }
        }

When the graph creates an instance of this node type then it will publish it in the equivalent USD format. The
data that can be recreated from the node type definition (i.e. the descriptions in this case) do not get published
into USD.

.. code-block:: usd
    :emphasize-lines: 12-13,20-22

    def OmniGraph "Graph"
    {
        token evaluator:type = "push"
        token fabricCacheBacking = "Shared"
        int2 fileFormatVersion = (1, 4)
        token pipelineStage = "pipelineStageOnDemand"

        def OmniGraphNode "times_2_node_1"
        {
            token node:type = "omni.graph.nodes.Times2"
            int node:typeVersion = 1
            custom double inputs:a = 2.5
            custom double outputs:two_a = 5.0
        }

        def OmniGraphNode "times_2_node_2"
        {
            token node:type = "omni.graph.nodes.Times2"
            int node:typeVersion = 1
            custom double inputs:a = 0.0
            custom double inputs:a.connect = </Graph/times_2_node_1.outputs:two_a>
            custom double outputs:two_a = 10.0
        }
    }


A few points of interest in this output:
- OmniGraph attributes are published as *custom* attributes
- The value of *node:typeVersion* must match the *"version"* value specified in the .ogn file
- The value of *node:type* is an extended version of the name key specified in the .ogn file, with the extension prepended
- Input attributes are put into the namespace *inputs:* and output attributes are put into the namespace *outputs:*

Special Attribute Types
=======================

There are some .ogn attribute types that do not have a direct equivalent in USD. They are implemented using existing
USD concepts, relying on OmniGraph to interpret them properly.

.. _omnigraph_usd_execution_attributes:

Execution Attributes
--------------------
Attributes of type *execution* are used by Action Graphs to trigger evaluation based on events. They are stored in
USD as unsigned integer values, with some custom data to indicate to OmniGraph that they are an execution type, not
a plain old value. Here is an example of the builtin **OnTick** node which has one such input.

.. code-block:: usd
    :emphasize-lines: 11-15

    def OmniGraphNode "on_tick"
    {
        custom uint inputs:framePeriod = 0
        custom bool inputs:onlyPlayback = 1
        custom token node:type = "omni.graph.action.OnTick"
        custom int node:typeVersion = 1
        custom double outputs:absoluteSimTime = 0
        custom double outputs:deltaSeconds = 0
        custom double outputs:frame = 0
        custom bool outputs:isPlaying = 0
        custom uint outputs:tick = 0 (
            customData = {
                bool isExecution = 1
            }
        )
        custom double outputs:time = 0
        custom double outputs:timeSinceStart = 0
    }

.. note::

    With execution attributes only the outputs require the **customData** values.

.. _omnigraph_usd_bundle_attributes:

Bundle Attributes
-----------------
A *bundle* attribute in OmniGraph is simply a collection of other attributes. They are published differently in USD
depending on whether they are inputs or outputs.

Input bundles cannot be specified directly; they take their attribute members from an input connection. For this reason
an input bundle attribute is represented as a USD *rel* type.

Output bundles are generated or passed along by the node's compute function, so they do not have a permanent
representation. Instead, a placeholder child prim is added to the node's prim in USD for it to populate when
computing.

Here is an example of a graph with two nodes, with an import of a cube prim to use for the extraction.

.. code-block:: usd
    :linenos:
    :emphasize-lines: 21-22,31-33,38,48,54-56

    def Xform "World"
    {
        def Mesh "Cube"
        {
            double3 xformOp:rotateXYZ = (0, 0, 0)
            double3 xformOp:scale = (1, 1, 1)
            double3 xformOp:translate = (0, 0, 0)
            string comment = "The rest of the cube data omitted for brevity"
        }

        def OmniGraph "PushGraph"
        {
            token evaluator:type = "push"
            token fabricCacheBacking = "Shared"
            int2 fileFormatVersion = (1, 4)
            token pipelineStage = "pipelineStageSimulation"

            def OmniGraphNode "read_prim_into_bundle"
            {
                custom bool inputs:computeBoundingBox = 0
                custom rel inputs:prim
                prepend rel inputs:prim = </World/Cube>
                custom token inputs:primPath = ""
                custom timecode inputs:usdTimecode = -1
                custom bool inputs:usePath = 0
                token node:type = "omni.graph.nodes.ReadPrimBundle"
                int node:typeVersion = 1
                custom uint64 state:target = 0
                custom timecode state:usdTimecode = -1

                def Output "outputs_primBundle"
                {
                }
            }

            def OmniGraphNode "extract_bundle"
            {
                custom rel inputs:bundle = </World/PushGraph/read_prim_into_bundle/outputs_primBundle>
                token node:type = "omni.graph.nodes.ExtractBundle"
                int node:typeVersion = 1
                custom int[] outputs:faceVertexCounts
                custom int[] outputs:faceVertexIndices
                custom normal3f[] outputs:normals
                custom point3f[] outputs:points
                custom float2[] outputs:primvars:st
                custom token outputs:sourcePrimPath = ""
                custom token outputs:subdivisionScheme = ""
                custom matrix4d outputs:worldMatrix = ( (0, 0, 0, 0), (0, 0, 0, 0), (0, 0, 0, 0), (0, 0, 0, 0) )
                custom double3 outputs:xformOp:rotateXYZ = (0, 0, 0)
                custom double3 outputs:xformOp:scale = (0, 0, 0)
                custom double3 outputs:xformOp:translate = (0, 0, 0)
                custom token[] outputs:xformOpOrder

                def Output "outputs_passThrough"
                {
                }
            }
        }
    }

.. mermaid::

    flowchart LR
        /World/Cube
        subgraph /World/PushGraph
            read_prim_into_bundle --> extract_bundle
        end
        /World/Cube -.-> read_prim_into_bundle

The interpretation of the five highlighted sections is:

1. The import node is given a relationship to the prim it will be importing
2. The bundle it is creating is given a placeholder child prim of type "Output"
3. The generated bundle is passed along to the extract_bundle node via a connection
4. The *worldMatrix* value was extracted from the bundle containing the cube's definition
5. The bundle being extracted is also presented as an output via the passThrough placeholder

Extended Attributes
-------------------
An extended attribute is one that can take on any number of different types when it is part of an OmniGraph.
For example it might be a double, float, or int value. The exact type will be determined by OmniGraph, either when a
connection is made or a value is set.

The actual attributes, however, are represented as token types in USD and it is only OmniGraph's interpretation that
provides them with their "real" type. Here is an example of a node having its *inputs:floatOrToken* attribute type
resolve to a *float* value at runtime since it has an incoming connection from a float type attribute.

.. code-block:: usd
    :emphasize-lines: 12-15,18

    def Xform "World"
    {
        def OmniGraph "ActionGraph"
        {
            token evaluator:type = "execution"
            token fabricCacheBacking = "Shared"
            int2 fileFormatVersion = (1, 4)
            token pipelineStage = "pipelineStageSimulation"

            def OmniGraphNode "tutorial_node_extended_attribute_types"
            {
                custom token inputs:flexible
                custom token inputs:floatOrToken
                custom token inputs:toNegate
                custom token inputs:tuple
                token node:type = "omni.graph.tutorials.ExtendedTypes"
                int node:typeVersion = 1
                prepend token inputs:floatOrToken.connect = </World/ActionGraph/constant_float.inputs:value>
                custom token outputs:doubledResult
                custom token outputs:flexible
                custom token outputs:negatedResult
                custom token outputs:tuple
            }

            def OmniGraphNode "constant_float"
            {
                custom float inputs:value = 0
                token node:type = "omni.graph.nodes.ConstantFloat"
                int node:typeVersion = 1
            }
        }
    }

.. mermaid::

    flowchart LR
        subgraph /World/PushGraph
            constant_float --> tutorial_node_extended_attribute_types
        end

.. _omnigraph_usd_path_attributes:

Path Attributes
---------------
The special OmniGraph attribute type *path* is used to represent USD paths, represented as token-type attributes in USD.
To OmniGraph they are equivalent to an *Sdf.Path* that points to a prim or attribute that may or may not exist on the
USD stage. When paths are modified, OmniGraph listens to the USD notices and attempts to fix up any path attributes
that may have changed locations. To USD though they are simply strings and no updates are attempted when OmniGraph is
not enabled and listening to the changes. See the attribute *inputs:primPath* in the example above for the
representation of a path attribute in USD.

Accessing USD Prims From Nodes
******************************
OmniGraph accesses USD scene data through special import and export nodes. Examples of each of these are the
:ref:`ReadPrimBundle<GENERATED - Documentation _ognomni.graph.nodes.ReadPrimBundle>` node for importing a USD prim
into an :ref:`OmniGraph bundle attribute<omnigraph_usd_bundle_attributes>` and a corresponding
:ref:`WritePrim<GENERATED - Documentation _ognomni.graph.nodes.WritePrim>` node for exporting a computed
bundle of attributes back onto a USD prim.

Both of these node types use the :ref:`path attribute<omnigraph_usd_path_attributes>` type to specify the location of
the prim on the USD stage that they will access.

Once the nodes are connected to the prim they are importing or exporting, usually via specifying the *SdfPath* to that
prim on the USD stage, they can be used the same as any other
OmniGraph node, passing the extracted USD data through the graph for computation.
See :ref:`above<omnigraph_usd_bundle_attributes>` for a detailed example.


The USD API
***********
OmniGraph listens the USD change notices, so it is able to update values when the USD API is used to modify them, or
by proxy when UI elements that use the USD API modify them. However this is not an efficient way to set values as
the notification process can be slow so it is best to avoid this if possible.

.. code-block:: python

    import omni.usd
    import omni.graph.core as og
    from pxr import Usd

    # Slow Method
    prim_node = omni.usd.get_context().get_stage().GetPrimAtPath("/World/ActionGraph/constant_float")
    attribute = prim_node.GetAttribute("inputs:value")
    attribute.Set(5.0)

    # Fast Method
    og.Controller("/World/ActionGraph/constant_float.inputs:value").set(5.0)
