.. _ogn_tutorial_cpuGpuData:


Tutorial 9 - Runtime CPU/GPU Decision
=====================================

The CPU/GPU data node creates various attributes for use in a CUDA-based GPU compute or a CPU-based compute, where the
decision of which to use is made at runtime rather than compile time. A few representative
types are used, though the list of potential attribute types is not exhaustive. See :ref:`ogn_attribute_types`
for the full list.

OgnTutorialCpuGpuData.ogn
-------------------------
The *ogn* file shows the implementation of a node named "omni.graph.tutorials.CpuGpuData", which has attributes whose memory
type is determined at runtime by the input named *isGPU*. The algorithm of the node is implemented in CUDA, but
in such a way that it can run on either the CPU or the GPU, depending on where the attribute data lives.

.. literalinclude:: ../../../../../source/extensions/omni.graph.tutorials/tutorials/tutorial9/OgnTutorialCpuGpuData.ogn
    :linenos:
    :language: json

OgnTutorialCpuGpuData.cpp
-------------------------
The *cpp* file contains the implementation of the compute method, which checks the value of the *isGPU* attribute
and then extracts the data of the specified type to pass to the algorithm in the .cu file.

.. literalinclude:: ../../../../../source/extensions/omni.graph.tutorials/tutorials/tutorial9/OgnTutorialCpuGpuData.cpp
    :linenos:
    :language: c++

OgnTutorialCpuGpuData_CUDA.cu
-----------------------------
The *cu* file contains the implementation of the deformation on the CPU and GPU using CUDA.

.. literalinclude:: ../../../../../source/extensions/omni.graph.tutorials/tutorials/tutorial9/OgnTutorialCpuGpuData_CUDA.cu
    :linenos:
    :language: c++

CPU/GPU Attribute Access
------------------------
Here is how the attribute values are returned from the database. Up until now the attribute name has sufficed as the
database member that accesses the value through its *operator()*. The addition of the runtime switch of memory
locations is facilitated by the addition of the *gpu()* and *cpu()* members.

+-------------------------+----------------+-------------------------+-----------------+----------------+
| CPU Function            | CPU Type       | GPU Function            | GPU Type        | CUDA Type      |
+=========================+================+=========================+=================+================+
| inputs.a.cpu()          | const float&   | inputs.a_t_gpu          | const float*    | const float*   |
+-------------------------+----------------+-------------------------+-----------------+----------------+
| inputs.b.cpu()          | const float&   | inputs.b.gpu()          | const float*    | const float*   |
+-------------------------+----------------+-------------------------+-----------------+----------------+
| outputs.sum.cpu()       | float&         | outputs.sum.gpu()       | float*          | float*         |
+-------------------------+----------------+-------------------------+-----------------+----------------+
| inputs.multiplier.cpu() | const GfVec3f& | inputs.multiplier.gpu() | const GfVec3f*  | const float3   |
+-------------------------+----------------+-------------------------+-----------------+----------------+
| inputs.points.cpu()     | const GfVec3f* | inputs.points.gpu()     | const GfVec3f** | const float3** |
+-------------------------+----------------+-------------------------+-----------------+----------------+
| outputs.points.cpu()    | GfVec3f*       | outputs.points.gpu()    | const GfVec3f** | float3**       |
+-------------------------+----------------+-------------------------+-----------------+----------------+

Type Information
++++++++++++++++

As there are three different potential types for each attribute when it varies location at runtime (CPU, CPU being
passed to GPU, and GPU) there are extra types introduced in order to handle each of them. The CUDA types are handled
as before, but on the CPU side there are extra types for the data being passed from the CPU to the GPU.

+----------------------+----------------+--------------------------+-----------------+
| CPU Type Method      | CPU Data Type  | GPU Type Method          | GPU Data Type   |
+======================+================+==========================+=================+
| inputs::a_t          | const float&   | inputs::a_t_gpu          | const float*    |
+----------------------+----------------+--------------------------+-----------------+
| inputs::b_t          | const float&   | inputs::b_t_gpu          | const float*    |
+----------------------+----------------+--------------------------+-----------------+
| outputs::sum_t       | float&         | outputs::sum_t_gpu       | float*          |
+----------------------+----------------+--------------------------+-----------------+
| inputs::multiplier_t | const GfVec3f& | inputs::multiplier_t_gpu | const GfVec3f*  |
+----------------------+----------------+--------------------------+-----------------+
| inputs::points_t     | const GfVec3f* | inputs::points_t_gpu     | const GfVec3f** |
+----------------------+----------------+--------------------------+-----------------+
| outputs::points_t    | GfVec3f*       | outputs::points_t_gpu    | const GfVec3f** |
+----------------------+----------------+--------------------------+-----------------+

On the C++ side the functions defined in the CUDA file are declared as:

.. code-block:: c++

    extern "C" void cpuGpuMultiplierCPU(outputs::points_t, inputs::multiplier_t, inputs::points_t, size_t);
    extern "C" void cpuGpuMultiplierGPU(outputs::points_t_gpu, inputs::multiplier_t_gpu, inputs::points_t_gpu, size_t);

The addition of the **_gpu** suffix mostly adds an extra layer of indirection to the values, since they exist in the
GPU memory namespace.  Care must be taken to call the correct version with the correctly extracted data:

.. code-block:: c++

    if (db.inputs.is_gpu())
    {
        cpuGpuMultiplierGPU(
            db.outputs.points.gpu(),
            db.inputs.multiplier.gpu(),
            db.inputs.points.gpu(),
            numberOfPoints
        );
    }
    else
    {
        // Note how array data is extracted in its raw form for passing to the function on the CUDA side.
        // This would be unnecessary if the implementation were entirely on the CPU side.
        cpuGpuMultiplierCPU(
            db.outputs.points.cpu().data(),
            db.inputs.multiplier.cpu(),
            db.inputs.points.cpu().data(),
            numberOfPoints
        );
    }

On the CUDA side the function definitions use the existing CUDA types, so their signatures are:

.. code-block:: c++

    extern "C" void cpuGpuMultiplierCPU(outputs::points_t, inputs::multiplier_t, inputs::points_t, size_t);
    extern "C" void cpuGpuMultiplierGPU(outputs::points_t, inputs::multiplier_t, inputs::points_t, size_t);
