<commit_before>**TODO**

    - Terminology pass to make sure we're using the standard terms.

# Input System

MRTK's input system allows you to:
- Consume inputs from a variety of input devices in your scripts.
- Define abstract actions like "press" and associate them to different inputs.
- Setup pointers attached to input devices that apply focus to game objects and drive application logic via events.

This page provides an introduction to the most commonly used input functionality and the underlying concepts.

## Controllers and Input Actions

A controller in MRTK is an input device. Some of the controller types supported are:

- Hololens Gestures
- WMR Controller
- Xbox Controller
- Oculus Remote/Touch
- Vive Knuckles/Wand
- Generic Open VR Controller
- Mouse and touch screen

The *Registered Service Providers Profile* contains a service provider for each controller type in use. There you can add new providers or remove existing ones.

<img src="../../External/ReadMeImages/Input/RegisteredServiceProviders.png" width="256" align="center">

Each controller type has a number of *physical inputs*. Each of these is defined by an *Axis Type*, telling us the data type of the input value (Digital, Single Axis, Dual Axis, Six Dof, ...), and an *Input Type* (Button Press, Trigger, Thumb Stick, Spatial Pointer, ...) describing the origin of the input. Physical inputs are mapped to *input actions* via *interaction mappings* in *Controller Input Mapping Profile*.

<img src="../../External/ReadMeImages/Input/ControllerInputMapping.png" width="512" align="center">


*Input actions* are user abstractions over physical device inputs used to map application actions like "select" to different controller inputs (e.g. mouse click, controller button, finger tap, ...). They can be configured in the *Input Actions Profile* specifying a name for the action and the *Axis Type* of the physical inputs it can be mapped to. The input action mapped to a physical input, if any, is stored in *input events*.

<img src="../../External/ReadMeImages/Input/InputActions.png" width="256" align="center">

Controller inputs can be read subscribing to input events as described in the [Input Events](#input-events) section.

## Pointers

A pointer is basically a ray attached to a controller that gives *focus* to the game object it is pointing to. For a game object to be able to receive focus it must have a collider, so it can be hit by physics raycasts, and belong to one of the layers defined in the *Pointer Raycast Layer Masks* in the *Pointer Profile*.

Pointers are instanced automatically at runtime when a new controller is detected. The pointers that are created for each controller type are defined in the *Pointer Options* in the *Pointer Profile*. You can have more than one pointer attached to a controller; for example, with the default pointer profile, WMR controllers get both a line and a parabolic pointer for normal selection and teleportation respectively. Pointers communicate with each other to decide which one is active.

MRTK provides a set of pointer prefabs in *Assets/MixedRealityToolkit.SDK/Features/UX/Prefabs/Pointers*. You can use your own prefabs as long as they contain one of the pointer scripts in *Assets/MixedRealityToolkit.SDK/Features/UX/Scripts/Pointers* or any other script implementing [`IMixedRealityPointer`](xref:Microsoft.MixedReality.Toolkit.Input.IMixedRealityPointer).

<img src="../../External/ReadMeImages/Input/PointerProfile.png" width="256" align="center">

## Gestures

Gestures are input events based on human hands. There are two types of devices that raise gesture input events in MRTK:
- Windows Mixed Reality devices such as Hololens. This describes pinching motions ("Air Tap") and tap-and-hold gestures.

  For more information on Hololens gestures see the [Windows Mixed Reality Gestures documentation](https://docs.microsoft.com/en-us/windows/mixed-reality/gestures).

  [`WindowsMixedRealityDeviceManager`](xref:Microsoft.MixedReality.Toolkit.WindowsMixedReality.Input.WindowsMixedRealityDeviceManager) wraps the [Unity XR.WSA.Input.GestureRecognizer](https://docs.unity3d.com/ScriptReference/XR.WSA.Input.GestureRecognizer.html) to consume Unity's gesture events from Hololens devices.

- Touch screen devices.

  [`UnityTouchController`](xref:Microsoft.MixedReality.Toolkit.Input.UnityInput) wraps the [Unity Touch class](https://docs.unity3d.com/ScriptReference/Touch.html) that supports physical touch screens.

Both of these input sources use the _Gesture Settings_ profile to translate Unity's Touch and Gesture events respectively into MRTK's [Input Actions](#controllers-and-input-actions). This profile can be found under the _Input System Settings_ profile.

<img src="../../External/ReadMeImages/Input/GestureProfile.png" width="256" align="center">

## Speech

Speech commands are events raised when the user says a specific keyword. You can create commands in the *Speech Commands Profile*, found under the _Input System Settings_ profile, by specifying a keyword and, optionally, a key that will also raise the event and an input action associated with it. See [Speech and Dictation Events](#speech-and-dictation-events) for speech event details.

<img src="../../External/ReadMeImages/Input/SpeechCommands.png" width="256" align="center">

## Dictation



## Input Events

To receive input events in your script you must implement one of the *event handler interfaces*. By default you will only receive events when the script's game object or any of its children are focused by a pointer. If you want to receive events always, regardless of focus, register your game object as a *global listener* via *MixedRealityToolkit.InputSystem.Register* or derive your script from [`InputSystemGlobalListener`](xref:Microsoft.MixedReality.Toolkit.Input.InputSystemGlobalListener).

Next, we'll go over the different event types and their corresponding interfaces.

### Input Source Events

These events refer to the general state of a controller as opossed to its specific inputs. The following information is available:<commit_msg>Input introduction. WIP<commit_after>**TODO**

    - Terminology pass to make sure we're using the standard terms.

# Input System

## Introduction

The Input System in MRTK allows you to:
- Consume inputs from a variety of input sources, like 6 DOF controllers, articulated hands or speech, via input events.
- Define abstract actions, like *Select* or *Menu*, and associate them to different inputs.
- Setup pointers attached to controllers to drive UI components via focus and pointer events.

Inputs are produced by **Input Service Providers**. Each provider corresponds to a particular source of input: Open VR, Windows Mixed Reality (WMR), Unity Joystick, Windows Speech, etc. Providers are added to your project via the **Registered Service Providers Profile**  in the *Mixed Reality Toolkit* component and will produce input events automatically when the corresponding input sources are available, e.g. when a WMR controller is detected or a gamepad connected.

**Input Actions** are abstractions over raw inputs meant to help isolating application logic from the specific input sources producing an input. For example, it can be useful to define a *Select* action and map it to the left mouse button, a button in a gamepad and a trigger in a 6 DOF controller. You can then have your application logic listen for input events mapped to that action instead of having to be aware of all the different inputs that can produce it. Input Actions are defined in the **Input Actions Profile** and mapped to inputs in the **Controller Mapping Profile**, both found within the *Input System Profile* in the *Mixed Reality Toolkit* component.

**Controllers**

**Pointers**

## Controllers and Input Actions

A controller in MRTK is an input device. Some of the controller types supported are:

- Hololens Gestures
- WMR Controller
- Xbox Controller
- Oculus Remote/Touch
- Vive Knuckles/Wand
- Generic Open VR Controller
- Mouse and touch screen

The *Registered Service Providers Profile* contains a service provider for each controller type in use. There you can add new providers or remove existing ones.

<img src="../../External/ReadMeImages/Input/RegisteredServiceProviders.png" width="256" align="center">

Each controller type has a number of *physical inputs*. Each of these is defined by an *Axis Type*, telling us the data type of the input value (Digital, Single Axis, Dual Axis, Six Dof, ...), and an *Input Type* (Button Press, Trigger, Thumb Stick, Spatial Pointer, ...) describing the origin of the input. Physical inputs are mapped to *input actions* via *interaction mappings* in *Controller Input Mapping Profile*.

<img src="../../External/ReadMeImages/Input/ControllerInputMapping.png" width="512" align="center">


*Input actions* are user abstractions over physical device inputs used to map application actions like "select" to different controller inputs (e.g. mouse click, controller button, finger tap, ...). They can be configured in the *Input Actions Profile* specifying a name for the action and the *Axis Type* of the physical inputs it can be mapped to. The input action mapped to a physical input, if any, is stored in *input events*.

<img src="../../External/ReadMeImages/Input/InputActions.png" width="256" align="center">

Controller inputs can be read subscribing to input events as described in the [Input Events](#input-events) section.

## Pointers

A pointer is basically a ray attached to a controller that gives *focus* to the game object it is pointing to. For a game object to be able to receive focus it must have a collider, so it can be hit by physics raycasts, and belong to one of the layers defined in the *Pointer Raycast Layer Masks* in the *Pointer Profile*.

Pointers are instanced automatically at runtime when a new controller is detected. The pointers that are created for each controller type are defined in the *Pointer Options* in the *Pointer Profile*. You can have more than one pointer attached to a controller; for example, with the default pointer profile, WMR controllers get both a line and a parabolic pointer for normal selection and teleportation respectively. Pointers communicate with each other to decide which one is active.

MRTK provides a set of pointer prefabs in *Assets/MixedRealityToolkit.SDK/Features/UX/Prefabs/Pointers*. You can use your own prefabs as long as they contain one of the pointer scripts in *Assets/MixedRealityToolkit.SDK/Features/UX/Scripts/Pointers* or any other script implementing [`IMixedRealityPointer`](xref:Microsoft.MixedReality.Toolkit.Input.IMixedRealityPointer).

<img src="../../External/ReadMeImages/Input/PointerProfile.png" width="256" align="center">

## Gestures

Gestures are input events based on human hands. There are two types of devices that raise gesture input events in MRTK:
- Windows Mixed Reality devices such as Hololens. This describes pinching motions ("Air Tap") and tap-and-hold gestures.

  For more information on Hololens gestures see the [Windows Mixed Reality Gestures documentation](https://docs.microsoft.com/en-us/windows/mixed-reality/gestures).

  [`WindowsMixedRealityDeviceManager`](xref:Microsoft.MixedReality.Toolkit.WindowsMixedReality.Input.WindowsMixedRealityDeviceManager) wraps the [Unity XR.WSA.Input.GestureRecognizer](https://docs.unity3d.com/ScriptReference/XR.WSA.Input.GestureRecognizer.html) to consume Unity's gesture events from Hololens devices.

- Touch screen devices.

  [`UnityTouchController`](xref:Microsoft.MixedReality.Toolkit.Input.UnityInput) wraps the [Unity Touch class](https://docs.unity3d.com/ScriptReference/Touch.html) that supports physical touch screens.

Both of these input sources use the _Gesture Settings_ profile to translate Unity's Touch and Gesture events respectively into MRTK's [Input Actions](#controllers-and-input-actions). This profile can be found under the _Input System Settings_ profile.

<img src="../../External/ReadMeImages/Input/GestureProfile.png" width="256" align="center">

## Speech

Speech commands are events raised when the user says a specific keyword. You can create commands in the *Speech Commands Profile*, found under the _Input System Settings_ profile, by specifying a keyword and, optionally, a key that will also raise the event and an input action associated with it. See [Speech and Dictation Events](#speech-and-dictation-events) for speech event details.

<img src="../../External/ReadMeImages/Input/SpeechCommands.png" width="256" align="center">

## Dictation

The dication service allows users to record audio clips and, via Windows Speech services, obtain a transcription of the audio.


## Input Events

To receive input events in your script you must implement one of the *event handler interfaces*. By default you will only receive events when the script's game object or any of its children are focused by a pointer. If you want to receive events always, regardless of focus, register your game object as a *global listener* via *MixedRealityToolkit.InputSystem.Register* or derive your script from [`InputSystemGlobalListener`](xref:Microsoft.MixedReality.Toolkit.Input.InputSystemGlobalListener).

Next, we'll go over the different event types and their corresponding interfaces.

### Input Source Events

These events refer to the general state of a controller as opossed to its specific inputs. The following information is available:
{'fraction_non_alphanumeric': 0.07011426027600534, 'fraction_numerical': 0.0031161893456002373, 'mean_word_length': 4.940502423975319, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 15, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7200962', 'n_tokens_mistral': 3458, 'n_tokens_neox': 3271, 'n_words': 1715}