# Viewport API

The ViewportAPI object is used to access and control aspects of the render, who will then push images into the backing texture.
It is accessed via the `ViewportWidget.viewport_api` property which returns a Python `weakref.proxy` object.
The use of a weakref is done in order to make sure that a Renderer and Texture aren't kept alive because client code
is keeping a reference to one of these objects.  That is: the lifetime of a Viewport is controlled by the creator of the Viewport and
cannot be extended beyond the usage they expect.

With our simple `omni.kit.viewport.stage_preview` example, we'll now run through some cases of directly controlling the render,
similar to what the Content context-menu does.

```python
from omni.kit.viewport.stage_preview.window import StagePreviewWindow
# Viewport widget can be setup to fill the frame with 'fill_frame', or to a constant resolution (x, y).
resolution = 'fill_frame' or (640, 480) # Will use 'fill_frame'

# Give the Window a unique name, the window-size, and forward the resolution to the Viewport itself
preview_window = StagePreviewWindow('My Stage Preview', usd_context_name='MyStagePreviewWindowContext',
                                    window_width=640, window_height=480, resolution=resolution)

# Open a USD file in the UsdContext that the preview_window is using
preview_window.open_stage('http://omniverse-content-production.s3-us-west-2.amazonaws.com/Samples/Astronaut/Astronaut.usd')

# Save an easy-to-use reference to the ViewportAPI object
viewport_api = preview_window.preview_viewport.viewport_api
```
  

## Camera

The camera being used to render the image is controlled with the `camera_path` property.
It will return and is settable with a an `Sdf.Path`.
The property will also accept a string, but it is slightly more efficient to set with an `Sdf.Path` if you have one available.

```python
print(viewport_api.camera_path)
# => '/Root/LongView'

# Change the camera to view through the implicit Top camera
viewport_api.camera_path = '/OmniverseKit_Front'

# Change the camera to view through the implicit Perspective camera
viewport_api.camera_path = '/OmniverseKit_Persp'
```
  

## Filling the canvas

Control whether the texture-resolution should be adjusted based on the parent objects ui-size with the `fill_frame` property.
It will return and is settable with a bool.

```python
print('viewport_api.fill_frame)
# => True

# Disable the automatic resolution
viewport_api.fill_frame = False

# And give it an explicit resolution
viewport_api.resolution = (640, 480)
```
  

## Resolution

Resolution of the underlying `ViewportTexture` can be queried and set via the `resolution` property.
It will return and is settable with a tuple representing the (x, y) resolution of the rendered image.

```python
print(viewport_api.resolution)
# => (640, 480)

# Change the render output resolution to a 512 x 512 texture 
viewport_api.resolution = (512, 512)
```
  

## SceneView
  

For simplicity, the ViewportAPI can push both view and projection matrices into an `omni.ui.scene.SceneView.model`.
The use of these methods are not required, rather provided a convenient way to
auto-manage their relationship (particularly for stage independent camera, resolution, and widget size changes),
while also providing room for future expansion where a `SceneView` may be locked 
to last delivered rendered frame versus the current implementation that pushes changes
into the Renderer and the `SceneView` simultaneously.

### add_scene_view
Add an `omni.ui.scene.SceneView` to the list of models that the Viewport will notify with changes to view and projection matrices.
The Viewport will only retain a weak-reference to the provided `SceneView` to avoid extending the lifetime of the `SceneView` itself.
A `remove_scene_view` is available if the need to dynamically change the list of models to be notified is necessary.
```python
scene_view = omni.ui.scene.SceneView()
# Pass the real object, as a weak-reference will be retained
viewport_api.add_scene_view(scene_view)
```
### remove_scene_view
Remove the `omni.ui.scene.SceneView` from the list of models that the Viewport will notify from changes to view and projection matrices.
Because the Viewport only retains a weak-reference to the `SceneView`, calling this explicitly isn't mandatory, but can be
used to dynamically change the list of `SceneView` objects whose model should be updated.
```python
viewport_api.remove_scene_view(scene_view)
```

## Additional Accessors
  

The `ViewportAPI` provides a few additional read-only accessors for convenience:

### usd_context_name
```python
viewport_api.usd_context_name => str
# Returns the name of the UsdContext the Viewport is bound to.
```
### usd_context
```python
viewport_api.usd_context => omni.usd.UsdContext
# Returns the actual UsdContext the Viewport is bound to, essentially `omni.usd.get_context(self.usd_context_name)`.
```
### stage
```python
viewport_api.stage => Usd.Stage
# Returns the `Usd.Stage` the ViewportAPI is bound to, essentially: `omni.usd.get_context(self.usd_context_name).get_stage()`.
```
### projection
```python
viewport_api.projection=> Gf.Matrix4d
# Returns the Gf.Matrix4d of the projection.  This may differ than the projection of the `Usd.Camera` based on how the Viewport is placed in the parent widget.
```
### transform
```python
viewport_api.transform => Gf.Matrix4d
# Return the Gf.Matrix4d of the transform for the `Usd.Camera` in world-space.
```
### view
```python
viewport_api.view => Gf.Matrix4d
# Return the Gf.Matrix4d of the inverse-transform for the `Usd.Camera` in world-space.
```
### time
```python
viewport_api.time => Usd.TimeCode
#  Return the Usd.TimeCode that the Viewport is using.
```  

## Space Mapping
   

The `ViewportAPI` natively uses NDC coordinates to interop with the payloads from `omni.ui.scene` gestures.
To understand where these coordinates are in 3D-space for the `ViewportTexture`, two properties will
can be used to get a `Gf.Matrix4d` to convert to and from NDC-space to World-space.

```python
origin = (0, 0, 0)
origin_screen = viewport_api.world_to_ndc.Transform(origin)
print('Origin is at {origin_screen}, in screen-space')

origin = viewport_api.ndc_to_world.Transform(origin_screen)
print('Converted back to world-space, origin is {origin}')
```

### world_to_ndc
```python
viewport_api.world_to_ndc => Gf.Matrix4d
# Returns a Gf.Matrix4d to move from World/Scene space into NDC space 
```
### ndc_to_world
```python
viewport_api.ndc_to_world => Gf.Matrix4d
# Returns a Gf.Matrix4d to move from NDC space into World/Scene space 
```
  

## World queries

While it is useful being able to move through these spaces is useful, for the most part what we'll be more interested
in using this to inspect the scene that is being rendered.  What that means is that we'll need to move from an NDC coordinate
into pixel-space of our rendered image.

A quick breakdown of the three spaces in use:

The native NDC space that works with `omni.ui.scene` gestures can be thought of as a cartesian space centered on the rendered image.
![](ndc_space_ui.png)

We can easily move to a 0.0 - 1.0 uv space on the rendered image.
![](texture_uv.png)

And finally expand that 0.0 - 1.0 uv space into pixel space.
![](texture_pixel.png)


Now that we know how to map between the ui and texture space, we can use it to convert between screen and 3D-space.
More importantly, we can use it to actually query the backing `ViewportTexture` about objects and locations in the scene.

### request_query
```python
viewport_api.request_query(pixel_coordinate: (x, y), query_completed: callable) => None
# Query a pixel co-ordinate with a callback to be executed when the query completes


# Get the mouse position from within an omni.ui.scene gesture
mouse_ndc = self.sender.gesture_payload.mouse

# Convert the ndc-mouse to pixel-space in our ViewportTexture
mouse, in_viewport = viewport_api.map_ndc_to_texture_pixel(mouse_ndc)

# We know immediately if the mapping is outside the ViewportTexture
if not in_viewport:
    return

# Inside the ViewportTexture, create a callback and query the 3D world
def query_completed(path, pos, *args):
    if not path:
        print('No object')
    else:
        print(f"Object '{path}' hit at {pos}")

viewport_api.request_query(mouse, query_completed)
```


## Asynchronous waiting

Some properties and API calls of the Viewport won't take effect immediately after control-flow has returned to the caller.
For example, setting the camera-path or resolution won't immediately deliver frames from that Camera at the new resolution.
There are two methods available to asynchrnously wait for the changes to make their way through the pipeline.

### wait_for_render_settings_change

Asynchronously wait until a render-settings change has been ansorbed by the backend.
This can be useful after changing the camera, resolution, or render-product.

```python
my_custom_product = '/Render/CutomRenderProduct'
viewport_api.render_product_path = my_custom_product
settings_changed = await viewport_api.wait_for_render_settings_change()
print('settings_changed', settings_changed)
```

### wait_for_rendered_frames

Asynchronously wait until a number of frames have been delivered.
Default behavior is to wait for deivery of 1 frame, but the function accepts an optional argument for an additional number of frames to wait on.

```python
await viewport_api.wait_for_rendered_frames(10)
print('Waited for 10 frames')
```


## Capturing Viewport frames

To access the state of the Viewport's next frame and any AOVs it may be rendering, you can schedule a capture with the `schedule_capture` method.
There are a variety of default Capturing delegates in the `omni.kit.widget.viewport.capture` that can handle writing to disk or accesing the AOV's buffer.

Because capture is asynchronous, it will likely not have occured when control-flow has returned to the caller of `schedule_capture`.
To wait until it has completed, the returned object has a `wait_for_result` method that can be used.

### schedule_capture
```python
viewport_api.schedule_capture(capture_delegate: Capture): -> Capture
```

```python
from omni.kit.widget.viewport.capture import FileCapture

capture = viewport_api.schedule_capture(FileCapture(image_path))
captured_aovs = await capture.wait_for_result()
if captured_aovs:
  print(f'AOV "{captured_aovs[0]}" was written to "{image_path}"')
else:
  print(f'No image was written to "{image_path}"')
```
