Geometry and reference spaces in WebXR - Web APIs

Geometry and reference spaces in WebXR
======================================

At a fundamental level, rendering of scenes for WebXR presentation in either augmented reality or virtual reality contexts is performed using WebGL, so the two APIs share much of the same design language. However, in order to provide the ability to present scenes in true 3D using XR headsets and other such equipment, WebXR has additional concepts that must be understood.

In this article, we introduce the ways in which WebXR expands upon the geometry of WebGL, and how the positions and orientations of objects—both physical and virtual—are described in relation to one another using spaces and, in particular, reference spaces.

The article Spatial tracking in WebXR builds upon the information provided here to cover how the physical position and orientation of the user's head, as well as potentially other parts of their body such as the hands, are mapped into the digital world, as well as how the relative positions of both physical and virtual objects are tracked as they move around, so that the scene can be properly rendered and composited.

Fundamentals of 3D geometry
---------------------------

While we'll examine here the required math operations used to compute the positions, orientations, and movement of objects in virtual space—plus the need to integrate the human viewer of the scene into the mix—a thorough introduction to geometry and the use of matrices and vectors to manage 3D representations of a scene is well beyond the scope of what can be accomplished in this article. You can learn more about the individual operations in Matrix math for the web.

### Units

Before discussing the details of the geometry of the 3D space used by WebXR, it's first useful to understand the units of measure that are applied to the 3D world.

#### Lengths and distances

WebGL measures all distances and lengths in **meters**. WebXR inherits this standard, as well as the fact that the world is a cube two meters wide, two meters tall, and two meters deep. Each of the three axes has a minimum value of -1.0 and a maximum of 1.0, with the center of the cube located at (0, 0, 0).

![Diagram showing a WebXR space whose X, Y, and Z coordinate axes each have a minimum value of -1 and a maximum of 1.](/en-US/docs/Web/API/WebXR_Device_API/Geometry/defaultspacedimensions.svg)

This eight cubic meter space encompasses the entire universe for the purposes of your code. Everything you draw must have its coordinates mapped to fit into this space, either explicitly within your code, or by using a transform to adjust the coordinates of all vertices. The most efficient way, of course, is to design your objects and code to use the same coordinate system as WebGL does.

The WebGL coordinates and lengths are transformed automatically at render time to the size of the viewport in which the scene is being rendered.

#### Angles

Angles are specified using **radians**. To convert degrees to radians, multiply the value in degrees by `π/180`. The following code snippet shows two simple functions, `degreesToRadians()` and `radiansToDegrees()`, which convert back and forth between the two units for measuring angles.

```js
const RADIANS\_PER\_DEGREE = Math.PI / 180.0;

let degreesToRadians = (deg) => deg \* RADIANS\_PER\_DEGREE;
let radiansToDegrees = (rad) => rad / RADIANS\_PER\_DEGREE;
```

#### Times and durations

**Note:** For security reasons, `DOMHighResTimeStamp` usually introduces a
 small amount of imprecision to the clock in order to prevent it from being used in fingerprinting and timing-based
 attacks.

All times and durations in WebXR are measured using the `DOMHighResTimeStamp` type, which is a double-precision floating-point value specifying the time in milliseconds relative to the starting time. Since the value is a floating-point number, it may be accurate to well better than the millisecond level, depending on the platform and hardware.

Time is primarily used to determine the amount of time that's elapsed since the scene's previous animation frame was drawn. As such the time is typically in alignment with the refresh rate of the display, or some fraction thereof if the frame rate needs to be constrained due to performance issues. This means that the time will usually step in 1/60th of a second intervals, assuming a 60 FPS frame rate. Doing the math, we find that this means that each frame will ideally be rendered 16.6667 milliseconds apart.

### Geometry operations with matrices

We offer a guide to matrix mathematics as it pertains to 3D geometry, including how matrices are used for the three primary transforms that need to be performed when rendering 3D scenes:

* **Translation** is the use of a matrix to shift the position of a point through the virtual space. This motion can be along any of the object's axes, or any combination of them.
* **Rotation** is the application of a matrix that rotates a point around the origin of the object's coordinate system.
* **Scaling** is the use of a matrix to alter the size of an object.

Note that when we say that a transform applies to a point, it also, by extension, can be applied to a *collection* of points. Since an object is represented by some number of polygons made up of a number of points in space, applying the same transform to every point that makes up the object will apply that same transform to the entire object overall. Transforms can also be applied to vectors, since vectors are described using a coordinate value to define the direction and magnitude of the vector.

On the origins of spaces
------------------------

A complete XR-enhanced scene—whether virtual or augmented—is a composite of anywhere from one to potentially dozens of frames of reference. Each object within the scene that needs to directly exchange position and orientation data with the WebXR system needs to be able to report that information in a way that can be understood and adapted as needed to be comprehensible by other objects within the scene.

In augmented reality (AR), this is because of the need to insert virtual objects into the real world, not only placing them correctly but also ensuring that they don't seem to wander around on their own as the user's perspective shifts. In virtual reality (VR), it's all about creating a sense of space in which the user's movements are precisely matched by the imagery presented on the virtual display, to prevent disjoints and disconnects that could cause discomfort or worse.

Thus it's all about creating a sense of space. From the perspective of an XR developer, designing the stage is the part that matters most to your users. Like an architect or a set designer, you have the power to create moods and experiences through a physical environment. How you structure that space will both depend on and influence how users can interact and explore it.

**Note:** A space will typically have foreground, mid-distance, and background elements. The right balance can create a unique presence and guide your user. The foreground includes objects and interfaces that you can interact with directly. The mid-distance includes objects you can interact with to some extent, or can approach in order to examine and engage with more closely. The background, on the other hand, is usually largely or entirely non-interactive, at least until and unless the user is able to approach it, bringing it into the mid-distance or foreground range.

In WebXR, the fundamental concept of a **space**—as in, a coordinate space in which a scene takes place—is represented by an instance of `XRSpace`. The space is used to make determinations about the relative positions and motion of objects and other entities (such as light sources and cameras) within the user's environment.

As mentioned earlier, any given 3D point consists of three components, each identifying the distance from the center of the space along one of the three axes.

This is the **native origin** of the space, corresponding to a specific physical location in the user's environment. Each space has its own native origin, which is tracked by the XR device's tracking system. This may be different from the **effective origin**, which is the origin point for the space's local coordinate system.

The directionality of the coordinate system can be seen in the following diagram:

![Diagram showing the coordinate system used by WebGL and WebXR.](/en-US/docs/Web/API/WebXR_Device_API/Geometry/webgl_coordinates.svg)

An `XRRigidTransform` called the **origin offset** is used to transform points from the space's own effective coordinate system to the XR device's native coordinate system. The origin offset is initially an identity transform, since typically the two origins are aligned when the space is first established. However, as changes in alignment accumulate over time the origin offset may change to compensate.

The position of a point in space relative to the origin is determined by determining its distance along each of the three spatial axes shown in the diagram above. The space's origin is the point (0, 0, 0), at the center of the space and at the zero position along each axis. Specifically, under the initial starting conditions, with the default orientation of the viewer upon the space:

* The **x-axis** extends horizontally from left to right away from the origin, with the *x* coordinate of +1.0 being located at the right edge of the world. Negative values of *x* extend toward the left from the origin, reaching a value of -1.0 at the left edge of the space.
* The **y-axis** is positive extending upward from the origin toward the top of the screen, reaching +1.0 at the top of the world space. Values of *y* less than 0 are found below the origin, extending toward the bottom of the screen and reaching -1.0 at the bottom of the world's space.
* The **z-axis** extends from the origin outward from the screen, reaching +1.0 at the closest point to the user in that direction. Negative values of *z* extend away from the user further into the screen, with the farthest away point in the world having a *z* of -1.0.

Every object is, at the simplest level, a set of polygons defined by points in 3D space and an offset transform, indicating how to move and rotate the object to position it at the desired point in space. If the offset transform is an identity matrix, the object is located at the origin point.

To be useful for spatial tracking and scene geometry, though, you need to be able to correlate the XR device's perceived position with the space's coordinate system. That's where reference spaces come in.

Reference spaces
----------------

Because of the variety of XR hardware available, coming in a wide variety of form factors from many developers, it's impractical and non-scalable to expect developers to have to directly communicate with the tracking technology being used. Instead, the WebXR Device API is designed to have developers plan their users' experiences and request an appropriate reference space that best represents those needs. This is done by asking the user agent for an **`XRReferenceSpace`** matching those needs.

An `XRReferenceSpace` object acts as a means to adapt one coordinate system reference frame to another. After putting on a headset, consider the virtual world around you to have a coordinate system in which your position is (0, 0, 0)—that is, you're at the center of everything. Doesn't that feel empowering? Forward, directly in front of your headset, is the -Z axis, with +Z behind you. X is positive to your right and negative to your left. Y is negative as you go downward and positive as you go upward. This indicates the position of the headset in space at the start of your use of the XR system, with the origin (0, 0, 0) being positioned basically at the bridge of your nose. This space is the **world space**.

Next, consider the XR controller you have in your left hand. It has the ability to report movement and its orientation, but it doesn't know anything about the position of the headset or, more crucially, its coordinate system. But the controller still needs a way to report its position to your app. Thus, it has its own coordinate system. This is a reference space which is provided to your app when input events occur. This reference space internally knows how to map the coordinates of the controller to the headset's coordinates, so WebXR can translate coordinates back and forth for you.

Once created, an `XRReferenceSpace` guarantees a certain level of support for motion and orientation tracking, and provides a mechanism for obtaining an `XRViewerPose` from which you can get a matrix which represents the position and facing direction of the space relative to the world space, if the space represents a viewer such as the user's headset, an observer's headset, or a virtual camera.

All of this is the browser's responsibility to handle, providing consistent behavior regardless of how capable each of the underlying reference spaces are. No matter how powerful or simple the individual XR device is, code written using WebXR will still work, within the limitations of the available hardware.

Regardless of the type of reference space you choose, its type is `XRReferenceSpace` or is a type derived from `XRReferenceSpace`. The currently available reference space types, are shown below.

`bounded-floor`

An `XRBoundedReferenceSpace` similar to the `local` type, except the user is not expected to move outside a predetermined boundary, given by the `boundsGeometry` in the returned object.

`local`

An `XRReferenceSpace` tracking space whose native origin is located near the viewer's position at the time the session was created. The exact position depends on the underlying platform and implementation. The user isn't expected to move much if at all beyond their starting position, and tracking is optimized for this use case. For devices with six degrees of freedom (6DoF) tracking, the `local` reference space tries to keep the origin stable relative to the environment.

`local-floor`

An `XRReferenceSpace` similar to the `local` type, except the starting position is placed in a safe location for the viewer to stand, where the value of the y axis is 0 at floor level. If that floor level isn't known, the user agent will estimate the floor level. If the estimated floor level is non-zero, the browser is expected to round it such a way as to avoid fingerprinting (likely to the nearest centimeter).

`unbounded`

An `XRReferenceSpace` tracking space which allows the user total freedom of movement, possibly over extremely long distances from their origin point. The viewer isn't tracked at all; tracking is optimized for stability around the user's current position, so the native origin may drift as needed to accommodate that need.

`viewer`

An `XRReferenceSpace` tracking space whose native origin tracks the viewer's position and orientation. This is used for environments in which the user can physically move around, and is supported by all instances of `XRSession`, both immersive and inline, though it's most useful for inline sessions. It's particularly useful when determining the distance between the viewer and an input, or when working with offset spaces. Otherwise, typically, one of the other reference space types will be used more often.

The remainder of this guide explores how to select the right reference space for your app's needs.

Defining spatial relationships with reference spaces
----------------------------------------------------

There are a number of commonly used ways to reference the positions and orientations of objects relative to their environment, as well as to constrain the environment itself. To that end, WebXR defines a set of standard spaces, called **reference spaces**, each of which supports a different technique for correlating its local space's reference frame coordinate system to the coordinate system of the space in which it exists.

However, regardless of which type of reference space is being used, you can use the same functions to convert coordinates from space to parent space.

### Selecting the reference space type

Straight off, let's state the simplest step in the process of deciding which reference type to use: the reference spaces you're most likely to use are `local`, `local-floor`, `unbounded`, or `bounded-floor`.

#### Floor level reference spaces

The reference space types with `-floor` in their names work just like the corresponding non-floor spaces, except that they attempt to automatically ensure that the viewer is positioned in a safe place at or near (but always above) ground level. This is the plane at which the `y` coordinate is always 0, unless a floor is otherwise established. These space types are *not* viable if the rooms have uneven floors or floors whose height above ground level vary, since they don't support the avatar's vertical position changing.

#### The primary reference space types

The `viewer` reference space corresponds to the viewer's position in space; it's used by the `XRViewerPose` returned by the `XRFrame` method `getViewerPose()`. It's not typically used directly otherwise. The only real exception is that you are likely to use the `viewer` reference space when performing the XR scene inline within web content.

The `local` reference space is typically used to describe a relatively small area, such as a single room. It is not only always available when using an immersive session mode (`immersive-vr` or `immersive-ar`), but is always included among the optional features when requesting a new session; thus, every session created by `navigator.xr.requestSession()` supports the `local` reference space type.

To represent a large area—potentially involving multiple rooms or beyond—you can use the `unbounded` reference space type, which specifies no constraints on the viewer's movement. If you wish to prevent the user from moving into certain areas, you must handle that yourself.

The `bounded-floor` reference space type doesn't have a corresponding one that isn't floor-bound. If the user's XR hardware permits them to move about their real-world space, and you are able to do so, it may be useful to use a `bounded-floor` reference space, which lets you specifically define the boundaries of the area in which passage is allowed and safe. See the article Using bounded reference spaces to learn more about the use of bounded reference spaces.

By using a reference space to describe the position and orientation of objects, WebXR is able to standardize the form of the data you use to describe these things, regardless of the underlying XR hardware. The reference space's configuration is then able to provide you with the view matrices and object poses needed to correctly render the contents of the space.

### Establishing the reference space

The topmost space—the one obtained by calling the `XRSession` method `requestReferenceSpace()`—describes the coordinate system used for the overall world space. Everything is fundamentally tied to this coordinate system, which represents the relationship between the user's equipment's position and the virtual world.

While you can use WebXR for everything from augmenting the world with annotations to 360° video playback to scientific simulations to virtual reality training systems or anything else you can imagine, let's take a 3D video game as an example of a typical WebXR application. Consider the model of a player's avatar standing in the game world's space. You position that avatar relative to the world space, using the coordinate system defined by the world's reference space.

To move the player to a new position, you could rewrite all of its coordinates or manually apply a transform each time they move, but there's an easier way, thanks to reference spaces and their ability to be created relative to one another. Create an `XRRigidTransform` object representing the new position and orientation of the player's avatar, then create a new reference space to represent the avatar's point of view at the new position using the `XRReferenceSpace` method `getOffsetReferenceSpace()`. This comes in especially handy when implementing support for using non-XR devices such as keyboards or mice to move the player's avatar through the world.

With the newly-created reference space, the avatar can remain at the same coordinates yet appear in the world to be located at (and be see the world from the perspective of) its new location. For a more detailed look at how to use reference spaces to manage the player's viewpoint, see the article

In the case of our game avatar example, it's rare for an avatar (or any other moving creature or machine) to be a simple blob sliding around the world. They usually have additional form, as well as internal movement, such as moving legs, arms that swing as they walk, a head that turns or bobs, weapons that move around, and so forth. Bring these to life using standard WebGL techniques and a positioning matrix or `XRRigidTransform` to shift the objects to the correct position relative to the effective origin.

### Device limitations on reference spaces

Some XR devices can't be made to support a given experience, despite the efforts the API goes to in order to make up for any missing capabilities. For example, there's no way for a basic headset such as a GearVR device to be made to work in an app that requires support for allowing the user to walk around the environment by tracking their real-world movements.

To support progressive enhancement—and thus broaden the availability of your app or site—you should choose a reference space that offers the lowest amount of functionality needed, or provide a fallback mechanism that detects failed attempts to obtain reference spaces and tries again with a less powerful alternative.

The compatibility issues that arise may be as fundamental as being unable to support `immersive-ar` mode (augmented reality sessions) on a VR-only headset, or may involve a request for one or more required options which cannot be met when attempting to create the XR session.

XR sessions are created using the `navigator.xr.requestSession()` method. One of its optional parameters is an object which you can use to specify required and/or optional features that the session must (or should ideally) support. Currently, the only supported options are strings identifying the standard reference spaces. Using these, you can ensure before your code even runs that you have access to a WebXR session that can support the reference space type you require or prefer.

**Note:** At this time, the reference space to use or to prefer is the only option available when creating an `XRSession`. In the future, it's likely that more options will become available.

Positioning and orienting objects
---------------------------------

All spatial (position, orientation, and movement) information exchanged between your app and the WebXR API is expressed in relation to a specific space at the time the frame is being rendered. Any further position and orientation management is between you and WebGL, though you do make use of the origin offset from the reference space in order to position the objects correctly in the 3D world.

When it's time to render an animation frame, the callback function specified when you called the WebXR session's `XRSession` object's `requestAnimationFrame()` method is invoked. The callback receives as one of its parameters a timestamp indicating the time at which the frame takes place, and should perform all rendering for the corresponding animation frame.

As the callback is repeatedly called with increasing time values, the callback generates a sequence of frames which are presented using the XR hardware, thereby showing a 3D scene to the user.

You can learn more about the animation process in the article Rendering and the WebXR frame animation callback.

For an example and a more detailed, code-level explanation of how to position, orient, and move objects in virtual space, see the article Movement, orientation, and motion.

See also
--------

* WebXR Device API
* WebGL: 2D and 3D rendering for the web
* Matrix math for the web


Viewpoints and viewers: Simulating cameras in WebXR - Web APIs

Viewpoints and viewers: Simulating cameras in WebXR
===================================================

The first and most important thing to understand when considering the code to manage point-of-view and cameras in your application is this: *WebXR does not have cameras*. There's no magic object provided by either the WebGL or the WebXR API that represents the viewer that you can rotate and move around to automatically change what's seen on the screen. In this guide we show how use WebGL to simulate camera movements without having a camera to move. These techniques can be used in any WebGL (or WebXR) project.

Animating 3D graphics is an area of software development that brings together multiple disciplines in computer science, mathematics, art, graphic design, kinematics, anatomy, physiology, physics, and cinematography. Since we don't have a real camera, we imagine one, reproducing the *effect* of having a camera, without actually having the ability to move the user around the scene.

There are a few articles about the fundamental math, geometry, and other concepts behind WebGL and WebXR which may be useful to read before or while reading this one, including:

* Explaining basic 3D theory
* Matrix math for the web
* WebGL model view projection
* Geometry and reference spaces in WebXR

*Ed. note: Most diagrams used in this article to show how the camera moves while performing standard movements were taken from an article on the FilmmakerIQ website; namely, from this image which is found all over the web. We assume due to their frequent reuse that they're available under a permissive license, ownership is not certain. We hope that it's freely usable; if not, and you're the owner, please let us know and we'll find or produce new diagrams. Or, if you're happy to let us continue to use the images, please let us know so we can credit you properly!*

Cameras and relative movement
-----------------------------

When a classic live-action movie is filmed, the actors are on a set and move about the set as they perform, with one or more cameras watching their moves. The cameras may be fixed in place, but they may also be set up to move around as well, tracking the movement of the performers, dollying in and out to achieve emotional impact, and so forth.

### Virtual cameras

In WebGL (and by extension, in WebXR), there is no camera object we can move and rotate, so we have to find a way to fake these movements. Since there is no camera, we have to find a way to fake it. Fortunately, physicists like Galileo, Newton, Lorentz, and Einstein have given us the **principle of relativity**, which states that the laws of physics have the same form in every frame of reference. That is, no matter where you're standing, the laws of physics work in the same way.

By extension, if you and another person are standing in an empty field of solid stone with nothing else visible as far as the eye can see, if you move three meters toward the other person, the result *looks the same* as if the other person had moved three meters toward you. There is no way for either of you to see the difference. A third party can tell the difference, but the two of you cannot. If you are a camera, you can achieve the same visual result both by moving the camera *or by moving everything around the camera*.

And that's our solution. Since we can't move the camera, we move the world around it. Our renderer needs to know where we imagine the camera to be, then alter the position of every visible object to simulate that position and orientation. Thus, instead of referring to an actual camera object, the term **camera** is used in WebGL and WebXR programming to refer to an object describing the position and viewing direction of a hypothetical viewer of the scene, whether there's an actual object present in 3D space or not.

### Points of view

Since the camera is a virtual object which, rather than necessarily representing a physical object in the virtual world, represents a viewer's position and viewing direction, it's useful to think about the kinds of situation that call for the use of a camera. Gaming-related situations are listed separately since they are often a special case specific to gaming, but any of these perspectives might apply to any 3D graphics scene.

#### Generalized cameras

In general, virtual cameras may or may not be incorporated into physical objects within the scene. Indeed, outside the scope of 3D gaming, the odds are far more likely that the camera will not correspond with an object that appears in the scene at all. Some examples of ways 3D cameras are used:

* When rendering animation—whether for filmmaking or for use within the context of a presentation or game—the virtual camera is used just like a real-world film camera. As much as possible, standard cinematographic techniques are used, since the viewer has likely grown up watching films using those techniques, and has subconscious expectations that a film or animation will follow those methods. Deviating from them can pull the viewer out of the moment.
* In business applications, the 3D camera is used to set the apparent size and perspective when rendering things such as graphs and charts.
* In mapping applications, the camera may either be placed directly over the scene, or might use various angles to show perspective. For 3D GPS solutions, the camera is positioned to show the area around the user, with the majority of the display showing the area ahead of the user's path of motion.
* When using WebGL to accelerate 2D graphics drawing, the camera is typically placed directly above the center of the scene with the distance and field of view set to allow the entire scene to be presented.
* When accelerating bitmapped graphics, the renderer would draw the 2D image into a WebGL texture's buffer, then redraw the texture to refresh the screen. This essentially uses the texture as a backbuffer for performing multiple buffering in your 2D graphics application.

#### Cameras in gaming

There are many kinds of games, and as such, there are several ways in which cameras might be used in games. Some common situations include:

* In a first-person game, the camera is located within the player's avatar's head, facing in the same direction as the avatar's eyes. This way, the view presented on the player's screen or headset is what their avatar would be seeing.
* In some third-person games, the camera is located a short distance behind the player's avatar or vehicle, showing them from behind as they move through the game world. This is used in a lot of multiplayer online role-playing games, certain shooter games, and so forth. Popular examples include *World of Warcraft*, *Tomb Raider*, and *Fortnite*. This category also includes games where the camera is placed just over the player's shoulder.
* Some 3D games offer the ability to change your point of view, such as to look out the various windows of an aircraft in a flight simulator, or to see the views from all the security cameras within the game level (a common feature of spy and stealth-based games). This ability is also used by games offering weapons with scopes, where the view isn't quite based off the head's position in the same way anymore.
* 3D games might also provide the ability for non-players to observe the action, either by positioning an invisible avatar of sorts or by choosing a fixed virtual camera to watch from.
* In advanced 3D games, a camera or camera-like object *could* be used to determine what a non-player character can see, relying on the same rendering and physics engine used by player characters for non-player characters.
* In single-screen 2D games, the camera is not associated directly with the player or any other character in the game, but is instead either fixed above or next to the game play area, or follows the action as the action moves around a scrolling game world. For example, a classic arcade game such as *Pac-Man* takes place on a fixed game map, so the camera remains fixed a set distance above the map, always pointed straight down at the game world.
* In a side-scrolling or top-scrolling game such as *Super Mario Bros.*, the camera moves along left and right (or up and down, or both) to ensure that the action remains visible even though the game level is much larger than the viewport.
### Positioning the camera

Since there are no standard camera objects in WebGL or WebXR, we need to simulate the camera ourselves. Before we can do so, and before we can then simulate the movement of the camera, let's actually take a look at the virtual camera and how it *can* move, at the most fundamental level. As in all things, the **position** of an object in space—even if that space if virtual—can be represented using three numbers that indicate its position relative to the origin, whose position is defined to be (0, 0, 0).

There's another aspect of the spatial relationship of an object to the origin in space left to be considered: **perspective**. Perspective, properly applied to the objects in a scene, can take a scene that would otherwise look as flat as a typical 2D screen and make it pop as if it were truly 3D. There are several kinds of perspective; those are defined and their mathematics explained in the article WebGL model view projection. Importantly, the effect of perspective on a vector can be represented by adding a fourth component to the vector: the perspective component, called `w`.

The value of `w` is applied by dividing each of the other three components by it to get the final position or vector; that is, for a coordinate given as (`x`, `y`, `z`, `w`), the point in the 3D space is actually (`x`/`w`, `y`/`w`, `z`/`w`, 1) or (`x`/`w`, `y`/`w`, `z`/`w`). If you're not using perspective, `w` is always 1. In that situation, the complete coordinates for an object located at (1, 0, 3) are (1, 0, 3, 1).

But the location isn't enough to describe an object in 3D space because an object's state in space isn't only about its location, but about its rotation or facing direction, otherwise known as its **orientation**. The orientation can be represented using a 3D vector, which is typically normalized so that its length is 1.0. For example, if the object is facing an object located at (3, 1, -2)—that is, three meters to the right, one meter up, and two meters away from the origin point—the result is:

[

3

1

-
2

]

\left [ \begin{matrix} 3 \ 1 \ -2 \end{matrix} \right ]

This can also be represented as an array:

```js
let directionVector = [3, 1, -2];
```

For the purposes of performing operations involving both the coordinates and the facing direction vector, the vector needs to include the `w` component. The value of `w` is always 0 for vectors, so the aforementioned vector can also be represented using `[3, 1, -2, 0]` or:

[

3

1

-
2

0

]

\left [ \begin{matrix} 3 \ 1 \ -2 \ 0 \end{matrix} \right ]

WebXR automatically normalizes vectors to have a length of 1 meter; however, you may find that it makes sense to do it yourself for various reasons, such as to improve performance of calculations by not having to repeatedly perform normalization.

Once you've determined the matrix representing the combination of movements you wish the camera to make, you need to reverse it, because you're not moving the camera. Since you're actually moving everything *except* the camera, take the inverse of the transform matrix to get an inverse transform matrix. This inverse matrix can then be applied to the objects in the world to alter their positions and orientations to simulate the desired camera position.

This is why the `XRRigidTransform` object used by WebXR to represent transforms includes an `inverse` property. The `inverse` property is another `XRRigidTransform` object which is the inverse of the parent transform. Since the `XRView` representing the view has a `transform` property which is an `XRRigidTransform` providing the camera view, you can get the model view matrix—the transform matrix needed to move the world to simulate the desired camera position—like this:

```js
let viewMatrix = view.transform.inverse.matrix;
```

If the library you're using accepts an `XRRigidTransform` object directly, you can instead get `view.transform.inverse`, rather than pulling out just the array representing the view matrix.

### Composing multiple transforms

If your camera needs to be performing multiple transforms simultaneously, such as zooming and panning at the same time, you can multiply the transform matrices together to compose them into a single matrix that applies both changes at once. See Multiplying two matrices in the article Matrix math for the web for a clear but readable function that does this or use your preferred matrix math library such as glMatrix to do the work.

It's crucial to remember that unlike typical arithmetic, where multiplication is commutative (that is, you get the same answer whether you multiply left to right or right to left), matrix multiplication *is not commutative!* This is because each transform affects the position of the object and possibly the very coordinate system itself, which can dramatically change the results of the next operation performed. So you need to be careful about the order in which you apply your transforms when building your composite transform (or directly applying transforms in sequence).

### Applying the transform

To apply the transform, you multiply the point or vector by the transform or composition of transforms.

This has been a very quick overview of the concepts of position in terms of physical location, orientation or facing direction, and perspective. For more detail on the subject, see the articles Geometry and reference spaces, WebGL model view projection, and Matrix math for the web.

Simulating classic cinematography
---------------------------------

Cinematography is the art of designing, planning, and executing camera movements to create the desired look and emotion for a scene in animation or film. There are a number of terms that are helpful to understand, primarily around camera movement, as these terms are used to describe designed viewpoint changes with the virtual camera. It's also entirely possible to perform more than one of these movements at the same time; for example, you can pan the camera while also zooming in on the scene.

Keep in mind that the majority of camera movements are described relative to the camera's reference space.

The format for storing matrices is generally as a flat array in column-major order; that is, the values from the matrix are written starting with the top-left corner and moving *down* to the bottom, then moving over to the right a row and repeating until all values are in the array.

Thus a matrix that looks like this:

[

a
1

a
5

a
9

a
13

a
2

a
6

a
10

a
14

a
3

a
7

a
11

a
15

a
4

a
8

a
12

a
16

]

\left [ \begin{matrix} a\_{1} & a\_{5} & a\_{9} & a\_{13} \ a\_{2} & a\_{6} & a\_{10} & a\_{14} \ a\_{3} & a\_{7} & a\_{11} & a\_{15} \ a\_{4} & a\_{8} & a\_{12} & a\_{16} \end{matrix} \right ]

Is represented in array form like this:

```js
let matrixArray = [
  a1, a2, a3, a4,
  a5, a6, a7, a8,
  a9, a10, a11, a12,
  a13, a14, a15, a16,
];
```

In this array, the leftmost column contains the entries `a1`, `a2`, `a3`, and `a4`. The topmost row contains the entries `a1`, `a5`, `a9`, and `a13`.

Keep in mind that most WebGL and WebXR programming is done using third-party libraries which expand upon the basic functionality of WebGL by adding routines that make it much easier to perform not only core matrix and other operations, but often also to simulate these standard cinematography techniques. You should strongly consider using one instead of directly using WebGL. This guide uses WebGL directly since it's useful to understand to some extent what goes on under the hood, and to aide in the development of libraries or to help you optimize code.

**Note:** Even though we use phrases like "move the camera," what we're really doing is moving the entire world around the camera. This affects the way certain values work, which will be noted as they come up below.

### Zooming

Among the best known camera effects is the **zoom**. Zooming is performed in a physical camera by altering the focal length of the lens; this is the distance between the center of the lens itself and the camera's light sensors. Thus, zooming doesn't actually involve moving the camera at all. Instead, a zoom shot changes the magnification of the camera over time to make the area of focus seem closer to or farther away from the viewer, without actually physically moving the camera. A slow move can bring a sense of movement, ease, or focus to a scene, while a rapid zoom can create a sense of anxiety, surprise, or tension.

Because a zoom does not move the camera's position, the resulting effect is unnatural. The human eye doesn't have a zoom lens on it. We make things smaller or larger by moving away from or toward them. In cinematography, that's called a dolly shot.

There are two techniques in 3D graphics that can create similar though not identical results, and whose methods apply more easily in different situations.

#### Zooming by adjusting field of view

You can do something more akin to a true "zoom" by altering the camera's **field of view** (**FOV**). The field of view is an angle defining the length of the arc on the entire viewable area surrounding the camera that should be visible at once. This is an effect of the focal length in a physical camera, so since there is no true camera, altering the FOV is a passable substitute.

Recall that the circumference of a circle is 2π⋅r radians (360°); as such, this is the theoretical maximum FOV. Realistically, though, not only do humans not see anywhere near that much, but viewing devices such as monitors and VR goggles tend to reduce the field of view even further. Human eyes typically have a horizontal field of view of around 135° (about 2.356 radians) and a vertical FOV of about 180° (π or around 3.142 radians).

Making the camera's FOV smaller reduces the arc that will be included in the viewport, thus enlarging that content when rendered to the view. There are differences between this and an optical zoom effect, but the result is generally close enough to get the job done.

The following function returns a projection perspective matrix that integrates the specified field of view angle as well as the given near and far clipping plane distances:

```js
function createPerspectiveMatrix(viewport, fovDegrees, nearClip, farClip) {
  const fovRadians = fovDegrees \* (Math.PI / 180.0);
  const aspectRatio = viewport.width / viewport.height;

  const transform = mat4.create();
  mat4.perspective(transform, fovRadians, aspectRatio, nearClip, farClip);
  return transform;
}
```

After converting the FOV angle, `fovDegrees`, from degrees to radians and computing the aspect ratio of the `XRViewport` specified by the `viewport` parameter, this function uses the glMatrix library's `mat4.perspective()` function to compute the perspective matrix.

The perspective matrix encapsulates the field of view (technically, this is the *vertical* field of view), aspect ratio, and the near and far clipping planes within the 4x4 matrix `transform`, which is then returned to the caller.

The near clipping plane is the distance in meters to a plane parallel to the display surface closer than which nothing gets drawn. Any vertices which lie on the same side of that plane as the camera are not drawn. Conversely, the far clipping plane is the distance in meters to a plane beyond which no vertices are drawn.

To zoom using a scaling factor or percentage, you can map 1x (100% of normal size) to the largest value of FOV you allow (which leads to the most content being visible), then map your maximum magnification to the maximum value of FOV you support and map corresponding values in between.

If you start each frame's rendering pass by computing the perspective matrix, you can then multiply into that matrix all the other transforms you need to apply in order to result in the frame's desired geometry. For example:

```js
const transform = createPerspectiveMatrix(viewport, 130, 1, 100);
const translateVec = vec3.fromValues(
  -trackDistance,
  -craneDistance,
  pushDistance,
);
mat4.translate(transform, transform, translateVec);
```

This starts with the perspective matrix representing a 130° vertical field of view, then applies a translation that moves the camera in a manner that includes track, crane, and push movements.

#### Scaling transforms

Unlike a true "zoom", **scaling** involves multiplying each of the `x`, `y`, and `z` coordinate values in a position or vertex by a scaling factor for that axis. These may or may not necessarily be identical for each axis, though the closest result you can get to a zoom effect would involve using the same value for each. This would need to be applied to every vertex in the scene—ideally in the vertex shader.

If you want to scale up by a factor of 2, you need to multiply each component by 2.0. To scale down by the same amount, multiply them by -2.0. In matrix terms, this is performed using a transform matrix with scaling factored into it, like this:

```js
let scaleTransform = [
  Sx, 0, 0, 0,
  0, Sy, 0, 0,
  0, 0, Sz, 0,
  0, 0, 0, 1
];
```

This matrix represents a transform that scales up or down by a factor indicated by `(Sx, Sy, Sz)`, where `Sx` indicates the scaling factor along the X axis, `Sy` the scaling factor along the Y axis, and `Sz` the factor for the Z axis. If any of these values differs from the others, the result will be stretching or contraction which is different in some dimensions compared to others.

If the same scaling factor is to be applied in every direction, you can create a simple function to generate the scaling transform matrix for you:

```js
function createScalingMatrix(f) {
  return [f, 0, 0, 0, 0, f, 0, 0, 0, 0, f, 0, 0, 0, 0, 1];
}
```

With the transform matrix in hand, we apply the transform `scaleTransform` to the vector (or vertex) `myVector`:

```js
let myVector = [2, 1, -3];
let scaleTransform = [2, 0, 0, 0, 0, 2, 0, 0, 0, 0, 2, 0, 0, 0, 0, 1];
vec4.transformMat4(myVector, myVector, scaleTransform);
```

Or, using scaling along every axis by the same factor using the `createScalingMatrix()` function shown above:

```js
let myVector = [2, 1, -3];
vec4.transformMat4(myVector, myVector, createScalingMatrix(2.0));
```

### Panning (Yawing left or right)

**Panning** or **yaw** is the rotation of the camera left to right or right to left, with its base otherwise fixed in place. The position of the camera in space does not change, only the direction in which it's looking. And that direction does not change other than horizontally. Panning is great for establishing a setting or providing a sense of scope in a vast space or on a vast object. Or just for looking left and right, like simulating the player turning their head in an immersive or VR scenario.

![A diagram showing a camera panning left or right](/en-US/docs/Web/API/WebXR_Device_API/Cameras/camera-pan.png)

To do this, then, we need to rotate around the Y axis, to simulate the left and right rotation of the camera. Using the glMatrix library we've used previously, this can be performed using the `rotateY()` method on the `mat4` class, which represents a standard 4x4 matrix. To rotate the viewpoint defined by the matrix `viewMatrix` by `panAngle` radians:

```js
mat4.rotateY(viewMatrix, viewMatrix, panAngle);
```

If `panAngle` is positive, this transform will pan the camera to the right; a negative value for `panAngle` will pan to the left.

### Tilting (Pitching up or down)

When you **tilt** or **pitch** the camera, you keep it fixed in space at the same coordinates while changing the direction in which it's facing vertically without altering the horizontal portion of its facing at all. It adjusts the direction it's pointing up and down. Tilting is good for capturing the scope of a tall object or scene, such as a forest or a mountain, but is also a popular way to introduce a character or locale of importance or which inspires awe. It's also of course useful for implementing support for a player looking up and down.

![A diagram showing a camera tilting up and down](/en-US/docs/Web/API/WebXR_Device_API/Cameras/camera-tilt.png)

Thus, tilting the camera can be achieved by rotating the camera around the X axis, so that it pivots to look up and down. This can be done using the appropriate method in your matrix math library, such as the `rotateX()` method in glMatrix's `mat4` class:

```js
mat4.rotateX(viewMatrix, viewMatrix, angle);
```

Positive values for `angle` will tilt the camera downward, while negative values of `angle` will tilt upward.

### Dollying (Moving in or out)

A **dolly** shot is one in which the entire camera is moved forward and backward. In classic filmmaking, this is typically done with the camera mounted on a track or on a moving vehicle. The resulting motion can create impressively smooth effects, especially if moving along with the person or object that's the focus of your shot.

![A diagram showing how a camera moves for a dolly shot](/en-US/docs/Web/API/WebXR_Device_API/Cameras/camera-dolly.png)

While a dolly shot and a zoom seem like they ought to look about the same, they don't. The fact that zooming alters the camera's focal length means that the spatial relationship between the target and its surroundings doesn't change even as the target gets larger or smaller in the frame. On the other hand, a dolly shot, by actually moving the camera, replicates the sense of physical movement, causing the relationships of objects in the scene to shift as you expect while moving past them as you go toward or away from the target of the shot.

To perform a dolly operation, translate the camera view forward and backward along the Z axis:

```js
mat4.translate(viewMatrix, viewMatrix, [0, 0, dollyDistance]);
```

Here, `[0, 0, dollyDistance]` is a vector wherein `dollyDistance` is the distance to dolly the camera. Since this works by moving the entire world around the camera, what really happens here is that the entire world moves along the Z axis by `dollyDistance` meters relative to the camera. If `dollyDistance` is positive, the world moves toward the user by that amount, resulting in the camera being closer to the scene. Contrariwise, negative values of `dollyDistance` move the world away from the user, causing the camera to appear to move backward from the target.

### Trucking (Moving left or right)

**Trucking** using a physical camera uses the same kind of rigging as dollying, but instead of moving the camera forward and backward, it moves from left to right or vice versa. The camera doesn't rotate at all, so the focus of the shot slowly glides off the screen. This can suggest concentration, time passing, or contemplation when attempting to establish emotion in a scene. It's also used frequently in "walk-and-talk" scenes, wherein the camera glides alongside the characters and they walk through the scene.

![A diagram showing how a camera trucks left and right](/en-US/docs/Web/API/WebXR_Device_API/Cameras/camera-truck.png)

To move the camera left and right, translate the view matrix along the X axis in the opposite direction from the desired camera movement:

```js
mat4.translate(viewMatrix, viewMatrix, [-truckDistance, 0, 0]);
```

Note the vector `[-truckDistance, 0, 0]`. This compensates for the fact that the truck operation works by moving the world rather than the camera. By moving the entire world in the opposite direction from the direction indicated by `truckDistance`, we achieve the effect of moving the camera the expected direction. This way, positive values of `truckDistance` will move the camera to the right (by moving the world to the left) and negative values of `truckDistance` will move the camera to the left by moving the world to the right.

### Pedestaling (Moving up or down)

A **pedestal** shot is one involving keeping the camera fixed horizontally relative to the floor, but moved straight up or down. Picture the camera on a pedestal (or pole) that gets taller or shorter. This is useful when tracking a subject that's getting taller or shorter, or is standing up or sitting down from a chair, or moving straight up and down.

![A diagram showing a camera moving up and down using a pedestal motion](/en-US/docs/Web/API/WebXR_Device_API/Cameras/camera-pedestal.png)

This is similar to a **crane** shot, which involves moving a camera attached to a crane up and down. To perform a pedestal or crane motion, translate the view along the Y axis in the opposite direction from the direction you want to move the camera:

```js
mat4.translate(viewMatrix, viewMatrix, [0, -pedestalDistance, 0]);
```

By negating the value of `pedestalDistance`, we compensate for the fact that we're actually moving the world rather than the camera. So positive values of `pedestalDistance` will move the camera up, while negative values will move it down.

### Canting (Rolling left and right)

**Cant** (or **roll**) is a rotation of the camera around its roll axis; that is, the camera remains fixed in space, and remains pointed at the same location, but rotates around so that the top of the camera is pointed in a different direction.

![A diagram showing a camera rolling left and right](/en-US/docs/Web/API/WebXR_Device_API/Cameras/camera-roll.png)

You can visualize this by holding your arm out in front of you with your hand open, palm down. Imagine that your hand is the camera and the back of your hand represents the top of the camera. Now rotate your hand so that the "camera" is upside-down. You have just canted your hand around the roll axis. In cinematography, cant can be used to simulate various types of unsteady motion such as waves or turbulence, but can also be used for dramatic effect.

To accomplish this rotation around the Z axis using glMatrix:

```js
mat4.rotateZ(viewMatrix, viewMatrix, cantAngle);
```

Combining movements
-------------------

You can perform multiple movements at once, such as zooming while panning, or tilting and canting at the same time.

### Translating along multiple axes

Translating along multiple axes is quite easy. Previously, we performed our translations like this:

```js
mat4.translate(viewMatrix, viewMatrix, [-truckDistance, 0, 0]);
mat4.translate(viewMatrix, viewMatrix, [0, -pedestalDistance, 0]);
mat4.translate(viewMatrix, viewMatrix, [0, 0, dollyDistance]);
```

The solution here is obvious. SInce the translation is expressed as a vector providing the distance to move along each axis, we can combine them like this:

```js
mat4.translate(viewMatrix, viewMatrix, [
  -truckDistance,
  -pedestalDistance,
  dollyDistance,
]);
```

This will shift the origin of the matrix `viewMatrix` by the specified amount along each axis.

### Rotating around multiple axes

You can also combine rotations around multiple axes into a single rotation around a quaternion representing a shared axis for the rotations. To perform the rotations separately, you use Euler angles (separate angles around each axis) to apply pitch, yaw, and roll like this:

```js
mat4.rotateX(viewMatrix, viewMatrix, pitchAngle);
mat4.rotateY(viewMatrix, viewMatrix, yawAngle);
mat4.rotateZ(viewMatrix, viewMatrix, rollAngle);
```

You can instead construct a quaternion representing a combined rotation axis from the Euler angles, then rotate the matrix using multiplication, like this:

```js
const axisQuat = quat.create();
const rotateMatrix = mat4.create();
quat.fromEuler(axisQuat, pitchAngle, yawAngle, rollAngle);
mat4.fromQuat(rotateMatrix, axisQuat);
mat4.multiply(viewMatrix, viewMatrix, rotateMatrix);
```

This converts the Euler angles for pitch, yaw, and roll into a quaternion representing all three rotations. This is then converted into a rotation transform matrix; then, finally, the view matrix is multiplied by the rotation transform to complete the rotations.

Representing 3D with WebXR
--------------------------

WebXR takes 3D graphics a step further, allowing them to be presented using special visual hardware such as goggles or a headset to create 3D graphics that appear to actually exist in three dimensions, potentially within the context of the real world (in the case of augmented reality).

In order to perceive depth, it's necessary to have two perspectives on the scene. By comparing the two views, it's possible to recognize the depth of objects and, by extension, the distance between the viewer and objects which are seen. This is why we have two eyes, spaced slightly apart. You can remind yourself of this fact by closing one eye at a time, switching back and forth between the two eyes. Notice how your left eye can see the left side of your nose but not the right, while your right eye sees the right side of your nose but not the left. That's just one of many differences that exist between what each of your eyes see.

Our brain receives two sets of data about light levels and wavelengths throughout our field of view—one from each eye. The brain uses this data to construct the scene in our minds, using the slight differences between the two perspectives to figure out depth and distance.

### Rendering the scene

An XR—shorthand that encompasses both virtual reality (VR) and augmented reality (AR)—headset presents 3D imagery to us by drawing two views of the scene, slightly offset from one another just like the views obtained by our two eyes. These views are then separately fed to each eye, in order to allow them to collect the data our brain needs in order to construct a 3D image in our minds.

To do this, WebXR asks your renderer to draw the scene twice for each frame of video—once for each eye. The two views are rendered into the same framebuffer, one on the left and one on the right. The XR device then uses screens and lenses to present the left half of the produced image to our left eye and the right half to our right eye.

For example, consider a device which uses a 2560x1440 pixel frame buffer. Dividing this into two parts—half for each eye—results in each eye's view being drawn at a resolution of 1280x1440 pixels. Here's what that looks like conceptually:

![Diagram showing how a framebuffer is divided between two eyes' viewpoints](/en-US/docs/Web/API/WebXR_Device_API/Cameras/twoviewsoneframebuffer.svg)

Your code tells the WebXR engine that you want to provide the next animation frame by calling the `XRSession` method `requestAnimationFrame()`, providing a callback function that renders a frame of animation. When the browser needs you to render the scene, it invokes the callback, providing as input parameters the current time and an `XRFrame` encapsulating the data needed to render the correct frame.

This information includes the `XRViewerPose` describing the position and facing direction of the viewer within the scene as well as a list of `XRView` objects, each representing one perspective on the scene. In current WebXR implementations, there will never be more than two entries in this list: one describing the position and viewing angle of the left eye and another doing the same for the right. You can tell which eye a given `XRView` represents by checking the value of its `eye` property, which is a string whose value is `left` or `right` (a third possible value, `none`, theoretically may be used to represent another point of view, but support for this is not entirely available in the current API).

### Example frame callback

A fairly basic (but typical) callback for rendering frames might look like this:

```js
function myAnimationFrameCallback(time, frame) {
  const adjustedRefSpace = applyPositionOffsets(xrReferenceSpace);
  const pose = frame.getViewerPose(adjustedRefSpace);

  animationFrameRequestID = frame.session.requestAnimationFrame(
    myAnimationFrameCallback,
  );

  if (pose) {
    const glLayer = frame.session.renderState.baseLayer;
    gl.bindFramebuffer(gl.FRAMEBUFFER, glLayer.framebuffer);
    CheckGLError("Binding the framebuffer");

    gl.clearColor(0, 0, 0, 1.0);
    gl.clearDepth(1.0);
    gl.clear(gl.COLOR\_BUFFER\_BIT | gl.DEPTH\_BUFFER\_BIT);
    CheckGLError("Clearing the framebuffer");

    const deltaTime = (time - lastFrameTime) \* 0.001;
    lastFrameTime = time;

    for (const view of pose.views) {
      const viewport = glLayer.getViewport(view);
      gl.viewport(viewport.x, viewport.y, viewport.width, viewport.height);
      CheckGLError(`Setting viewport for eye: ${view.eye}`);

      myRenderScene(gl, view, sceneData, deltaTime);
    }
  }
}
```

The callback begins by calling a custom function, `applyPositionOffsets()`, which takes a reference space and applies to its transform matrix any changes that need to be made to take into account things such as user inputs from devices not controlled by WebXR, such as the keyboard and mouse. The adjusted `XRReferenceSpace` returned by this function is then passed into the `XRFrame` method `getViewerPose()` to get the `XRViewerPose` representing the viewer's position and viewing angle.

Next, we go ahead and queue up the request to render the next frame of video, so we don't have to worry about doing it later, by calling `requestAnimationFrame()` again.

Now it's time to render the scene. If we did successfully obtain a pose, we get the `XRWebGLLayer` we need to use for rendering from the session's `renderState` object's `baseLayer` property. We bind this to WebGL's `gl.FRAMEBUFFER` target using the `WebGLRenderingContext` method `gl.bindFrameBuffer()`.

Then we clear the framebuffer to ensure we're starting with a known state, since our renderer will not be touching every pixel. We set the clear color to opaque black using `gl.clearColor()` and the value to clear the depth buffer to 1.0 by calling the `WebGLRenderingContext` method `gl.clearDepth()`. Then we call the `WebGLRenderingContext` method `gl.clear()`, which clears the framebuffer (since we include `gl.COLOR_BUFFER_BIT` in the mask parameter) and the depth buffer (because we include `gl.DEPTH_BUFFER_BIT`).

Then we determine how much time has elapsed since the previous frame was rendered by comparing the frame's desired render time with the time at which the last frame was drawn. Since this value is in milliseconds, we convert it to seconds by multiplying by 0.001 (or dividing by 1000).

Now we loop over the pose's views, as found in the `XRViewerPose` array, `views`. For each view, we ask the `XRWebGLLayer` for the appropriate viewport to use, configure the WebGL viewport to match by passing the position and size information into `gl.viewport()`. This constrains rendering so that we can only draw into the portion of the framebuffer that represents the image seen by the eye identified by `view.eye`.

With the constraints so established and everything else we need ready, we call a custom function, `myRenderScene()`, to actually perform the computations and WebGL rendering to render the frame. In this case, we're passing in the WebGL context, `gl`, the `XRView` `view`, a `sceneData` object (which contains things like the vertex and fragment shaders, vertex lists, textures, and so forth), and `deltaTime`, which indicates how much time has passed since the previous frame, so that we know how far to advance the animation.

When this function returns, the WebGL framebuffer being used by WebXR now has in it two copies of the scene, each occupying half the frame: one for the left eye, and one for the right eye. This makes its way through the XR software and drivers into the headset, where each half is shown to the appropriate eye.

See also
--------

* Geometry and reference spaces
* WebGL model view projection
* Matrix math for the web
* Movement, orientation, and motion: A WebXR example


WebXR permissions and security - Web APIs

WebXR permissions and security
==============================

The WebXR Device API has several areas of security to contend with, from establishing Permissions Policy to ensuring the user intends to use the mixed reality presentation before activating it. Among other things, you need to confirm access to device features such as the microphone and/or camera, get permission to use immersive VR mode (if applicable), and so forth. The variety of hardware and software involved in XR brings multiple APIs and technologies into play. In this guide, we'll cover how to ensure your app has the permissions it needs to provide a secure and private XR experience.

The WebXR Device API is subject to a number of permission and security controls. While not onerous, they are worth being aware of. These mostly revolve around the fully-immersive `immersive-vr` session mode, but there are things to be aware of when setting up an AR session, as well.

Immersive presentation of VR
----------------------------

First, any requests to activate the `immersive-vr` mode are rejected if the domain issuing the request does not have permission to enable an immersive session. This permission comes from the `xr-spatial-tracking` Permissions Policy.

Once that check is passed, the request to enter `immersive-vr` mode is allowed if all of the following are true:

* The `requestSession()` call was issued by code executing within the handler for a user event, or the from the startup code for a user-launched web application.
* The document is considered trustworthy, in that it is responsible and is both currently active and has focus.
* The user's intent to enter immersive VR mode is well understood; see User intent below for details.

If all of that is true, the promise returned by `requestSession()` is resolved, and the new `XRSession` object is passed into the fulfillment handler. Otherwise, an appropriate exception is thrown, such as `SecurityError` if the document doesn't have permission to enter immersive mode.

Inline presentation
-------------------

When you request an `XRSession` with the mode set to `inline`, and any features are required or requested, the browser will only allow the session to be created if the call to `requestSession()` was made by code which is executing expressly due to **user intent**.

Specifically:

* If the `requestSession()` call isn't coming from within the handler executed in response to a user event, and is not being issued while launching a web application, the request is denied and `false` is delivered to the promise's fulfillment handler.
* If the document making the request isn't the one which is responsible for the script, the request is denied.
* If the document making the request isn't trustworthy, the request is denied and `false` is returned through the promise's fulfillment routine. A trustworthy document is one which is both responsible and active, and which currently has focus.
* If the user's intent to open an inline XR presentation is not well understood, the request is denied. Understanding of the user's intent may be either implicit or explicit.

**Note:** Additional requirements may be put into effect due to the specific features requested by the options object when calling `requestSession()`.

User intent
-----------

**User intent** is the concept of whether or not an action being performed by code is being performed because of something the user intends to do or not. There are two kinds of user intent: **implicit** and **explicit**.

**Explicit user intent** (explicit user consent) is granted when the user has specifically and expressly been asked for permission to perform an action.

**Implicit user intent** (implicit user consent) is assumed if either of the following scenarios is the case:

* The user has interacted with the document in some way which has in turn caused your request to occur. For example, if you have an "Enter XR mode" button, and the user clicks it, calling `requestSession()` from the button's `click` event handler will permitted.
* If your code is executing during the launch of a web application, the runtime may consider the act of launching your web application to qualify as user intent.


Spaces and reference spaces: Spatial tracking in WebXR - Web APIs

Spaces and reference spaces: Spatial tracking in WebXR
======================================================

The WebXR APIs used for implementing augmented and virtual reality is designed specifically to provide the ability to insert a human into a virtual environment. To accomplish this, software needs the ability to not only track the locations, orientation, and movements of objects in the virtual world, but the user's location, orientation, and movement as well. But WebXR goes beyond that by adding the ability to track the location, orientation, and motion of the input devices which generate data used to determine the position and movement of individual parts of the viewer's body (with appropriate equipment).

The location and movement of the user's headset represent their head's position and orientation in the virtual environment. Hand controllers represent their hands in the same manner. Other hardware elements can be used similarly to represent other parts of the body, providing additional data to use when simulating the user's actions in their environment.

In this guide, we'll explore how WebXR uses **spaces** and, more specifically, **reference spaces**, to track the positions, orientations, and movements of objects and of the user's body in the virtual world.

**Note:** This article presumes that you are familiar with the concepts introduced in Geometry and reference spaces in WebXR: that is, the basics of 3D coordinate systems, as well as WebXR spaces, reference spaces, and how reference spaces are used to create local coordinate systems for individual objects or movable components within a scene.

Representing a position using a reference space
-----------------------------------------------

As covered in Defining spatial relationships with reference spaces, reference spaces establish a local coordinate system which is offset from another coordinate system that is itself defined by some other space. Thus, a reference space can be used to define the position and orientation of a point and by extension the entire object for which that point is the origin. While this is a little heavy-handed to be used for every single object in a scene, it can be very useful for a few specific objects to have their own coordinate system in this manner.

* **The world space**; the origin of this space is the origin of the WebGL coordinate system underlying the entire 3D canvas.
* The player, avatar, or camera; the origin of this space is used as the camera position for rendering the scene to be displayed to the user.
* The hand and/or hand controller; each of these represents one of the user's hands, either in the form of the hand itself or a controller (or both). The origin is generally the center of the user's fisted hand.
* **The target ray**; each controller or other handheld device may have a targeting ray associated with it, which is represented by a space whose origin is at the point on the controller at which the ray is emitted, and is oriented so that -Z extends in the direction of the target it's pointing at.

Because each of these is defined using a reference space as its foundation, the WebXR Device API can easily be used to convert among the coordinate systems, perform operations affecting these spaces and their corresponding objects, and so forth.

Describing a position relative to a space
-----------------------------------------

There are two scenarios in which you may need to describe a position and/or orientation relative to a space. The first is described above: applying a reference space to an offset (or vice versa, since the result is the same) to determine the transform matrix that represents the resulting location in the space's coordinate system.

### Poses

Once your reference space or spaces are established for the various key objects in the scene, you will have times when you need to describe another position relative to the origin of a particular reference space. This is done using poses. A pose, put, describes a position and orientation relative to the origin of the reference space from which it was created.

In WebXR, a pose is represented by an `XRPose` object, whose `transform` property is an `XRRigidTransform` defining the transform matrix that, applied to any coordinate, vector, or matrix in the original space, converts it to the target space. Thus a pose can be used to not only convert and determine positions, but also rotational information.

There is only one way to create an `XRPose`, and that's using the `getPose()` method on an animation frame as given using an `XRFrame` object. This means that most frequently, you'll be using poses within your frame rendering code, which is executed as a callback from the `XRFrame` method `requestAnimationFrame()`.

`getPose()` computes the position of an `XRReferenceSpace` relative to the origin of a specified `XRSpace` and then creates a pose representing the resulting position and orientation.

For example, if you wish to draw a hand controller's representation using the controller's `gripSpace`, you can get the pose needed like this:

```js
let controlPose = frame.getPose(inputSource.gripSpace, worldRefSpace);
```

This converts the position and orientation of the input's grip space to use the world's coordinate system, then generates the corresponding `XRPose`, storing it in `controlPose`. You can then apply `controlPose`'s `transform` to the vertices in the object model representing the controller to calculate the WebGL coordinates to use when rendering the controller's representation to the framebuffer.

### Viewer poses

A specialized type of pose, the **viewer pose**, is the pose that represents the perspective of the viewer of the scene. A viewer pose is represented by WebXR's `XRViewerPose` interface. When rendering a frame, you'll use the viewer pose to determine the position and facing direction of the viewer in order to place the virtual camera and render the scene.

The only way to obtain a pose that adapts positional information from one space to another is through the `XRFrame` object received by your frame rendering callback function specified when you called the `XRSession` method `requestAnimationFrame()`.

For example, given an `XRSession` whose reference space is `worldRefSpace`, the following line of code would request the first frame of animation to be scheduled:

```js
animationFrameRequestID = xrSession.requestAnimationFrame(myDrawFrame);
```

Then, the `myDrawFrame()` function—the callback executed when it's time to draw the frame—might be something like this:

```js
function myDrawFrame(currentFrameTime, frame) {
  let session = frame.session;
  let viewerPose = frame.getViewerPose(viewerRefSpace);

  animationFrameRequestID = session.requestAnimationFrame(myDrawFrame);

  if (viewerPose) {
    // render the frame
  }
}
```

The `frame` parameter is the `XRFrame` representing the animation frame information provided by WebXR. When called, this function begins by getting the `XRSession` from the frame object, then uses the frame's `getViewerPose()` method to compute the `XRViewerPose` for the viewer, given `viewerRefSpace`, which describes the current facing direction and position of the viewer.

The returned viewer pose, `viewerPose`, can in turn be used to compute positions and orientations to properly render the objects in the scene given the user's point of view.

Offsetting or moving reference spaces
-------------------------------------

While you can't change a reference space since both `XRReferenceSpace` and `XRBoundedReferenceSpace` are read-only, you can easily create new reference spaces by applying an offset transform to them. This is done by calling the reference space's `getOffsetReferenceSpace()` method.

### Offsetting positions within a reference space

The simplest case for using `getOffsetReferenceSpace()` is to transform a point or matrix within the context of the same space. For example, to create a new reference space that moves the reference space `aRefSpace` a half meter in each direction, you can do something like this:

```js
let halfMeterTransform = new XRRigidTransform({
  x: 0.5,
  y: 0.5,
  z: 0.5,
  w: 1.0,
});
aRefSpace = aRefSpace.getOffsetReferenceSpace(halfMeterTransform);
```

This replaces the existing reference space `aRefSpace` with one whose coordinates and orientation have had the transform `halfMeterTransform` applied. With no orientation data included in the transform, the orientation of `aRefSpace` is unaffected.

### Converting between WebXR session types

Another common reason you may need to convert positional information from one reference space to another is when it's necessary to change the session type from `inline` to `immersive-vr` or back. This commonly happens when your user interface provides a way to preview the scene inline within the context of a web page, with a button or other control to switch to immersive mode.

Since most users would prefer that you maintain the same viewer position and facing direction while performing this transition, you can use the `XRFrame` method `getViewerPose()` to obtain the current `XRViewerPose`, switch the session, then use the saved viewer pose to restore the viewer's position and facing.

```js
let viewerPose = frame.getViewerPose(worldReferenceSpace);
let newSession = navigator.xr.requestSession("immersive-vr", {
  requiredFeatures: "unbounded",
});
worldReferenceSpace = await newSession.requestReferenceSpace("unbounded");
viewerPose = worldReferenceSpace.getOffsetReferenceSpace(viewerPose.transform);
```

Here, the viewer pose is obtained, with its transform defined relative to `worldReferenceSpace`, the current session's global reference space. Then a new session is established and a reference space created to become the new world reference space.

Finally, the saved `viewerPose` is converted to the new world space's coordinate system by calling the new reference system's `getOffsetReferenceSpace()` method. With this in hand, we can resume rendering the scene as usual, with the viewer's perspective unaffected.

### Converting between bounded and unbounded spaces

Sometimes when your primary experience uses an unbounded space, you may have a need to temporarily transition the user's experience to a bounded space. For instance, it may be easier to implement the interactions with the objects in a single room in a house by switching to a bounded space representing the single room. This might make it easier to do things like implement support for customizing the walls, placing furniture on the floor, and so forth.

In cases like this, where you need to begin tracking using a different reference space than the one you've been using, you can use a combination of `getViewerPose()` and some matrix computations to transition everything in your scene to be based at the new reference frame's origin.

Because `getViewerPose()` is only available in an `XRFrame`, you will need to begin this process in your rendering callback, possibly using a worker to do the computations in order to reduce frame dropping.

```js
let previousViewerPose = null;

function myDrawFrame(currentFrameTime, frame) {
  let session = frame.session;
  let viewerPose = frame.getViewerPose(viewerRefSpace);

  animationFrameRequestID = session.requestAnimationFrame(myDrawFrame);

  if (viewerPose) {
    if (!previousViewerPose) {
      previousViewerPose = viewerPose;
    }

    let offsetMatrix = mat4.create();
    mat4.sub(
      offsetMatrix,
      previousViewerPose.transform.matrix,
      viewerPose.transform.matrix,
    );

    previousViewerPose = viewerPose;
  }
}
```

Continuity and recovery after tracking loss
-------------------------------------------

Sometimes, while the user is actively using their XR hardware with your app, the flow of data containing updates as to the user's position and orientation might be lost for a period of time. Not only does your app need to determine what to show the user during these periods, but it needs to cleanly recover when tracking resumes.

Once the XR hardware begins providing tracking information, it will continue to do so until the XR session is closed. This data is obtained during each frame by calling the `XRFrame` method `getViewerPose()` to get the position and facing direction of the viewer (defining what the user should see), and `getPose()` to get any other poses, such as the positions of the hand controllers and any other parts of the XR system.

### Detecting and functioning after tracking loss

If tracking fails, such as due to a temporary loss of connectivity between the headset and the user's device, the XR layer will continue to return poses, but those poses' `emulatedPosition` property will be `true`, indicating that the computation of the pose is based on a guess of the current position of the user.

Some XR hardware uses algorithms to compute the estimated position of the user based on the movement currently ongoing, while other hardware will report no movement at all, but with `emulatedPosition` set to `true`. In either case, you may wish to adjust your rendering to compensate for the loss, depending on your specific needs.

### When tracking resumes

You can detect when tracking has resumed after being lost when the user position jumps while at the same time the value of `emulatedPosition` changes from `true` to `false`. How you handle this depends on your application. If your app provides a way for the user to move through the virtual world without physically moving in the real world (a so-called **teleportation** mechanic), you can accept the new position and continue, allowing the jump from your previously-assumed position to be immediately corrected with the new position.

On the other hand, if your app involves the user physically moving in real space to move around through your virtual world, adopting the new tracking position and jumping there can be unsettling to the user, and should be avoided if possible. Instead, use the difference between the current position and the new tracking position to compute the new teleportation offset; that is, a transform applied to everything to adapt the position and orientation data from WebXR to your needs.

You can do this by creating a new reference space that incorporates into its effective origin the distance the viewer's position jumped since the previous frame, using the `XRReferenceSpace` method `getOffsetReferenceSpace()`.

### The reset event

***<<<--- this section probably has problems still; corrections are appreciated --->>>***

When a discontinuity or break in the native or effective origin of a reference space occurs, the user agent will send the `XRReferenceSpace` a `reset` event. This event indicates that a significant change to the origin's position has taken place relative to the user's environment.

A `reset` may occur because the XR hardware lost connectivity for a time, resulting in the user's movements not being properly tracked for a while. Upon recovery of tracking, a `reset` means tracking has been restored and the new position information represents the actual position information provided by the XR hardware, rather than cached or "best-guess" data.

Another reason `reset` may be sent is because the user has exited the bounds of a reference space and entered another reference space, or because the user has been transitioned programmatically from one reference space to another. Any time the bounds geometry of the user's space changes, a `reset` is sent.

`reset` is only used for significant jumps or transitions; minor things will just be absorbed automatically. The event is always sent to every affected reference space, including those created using `getOffsetReferenceSpace()`, so if you listen for `reset` events, you need to be sure you retain a strong reference to any spaces you're still using.

See also
--------

* Matrix math for the web
* Geometry and reference spaces in WebXR
* Rendering and the WebXR frame animation callback
* Viewpoints and viewers: Simulating cameras in WebXR
* Movement, orientation, and motion: A WebXR example
* Using bounded reference spaces