Digital video concepts - Web media technologies

Digital video concepts
======================

In this article, we explore important concepts that are useful to understand in order to fully grasp how to work with video on the web.

Encoding color
--------------

Representing the colors in an image or video requires several values for each pixel. What those values are depends on how you "split up" the color when converting it to numeric form. There are several color models, and video codec makes use of one or more of these to represent their pixels during the encoding process as well as after decoding the video frames.

### RGB

Most computer graphics models use the RGB color system, wherein some number of bits of data are used to represent each of the red, green, and blue components of the color of an individual pixel, and an image is comprised of a two-dimensional array of these pixels. There are two primary methods used to represent RGB samples: using integer components and using floating-point components. When using integer components, RGB color uses 8 bits each of red, green, and blue, as well as potentially 8 bits of alpha (the amount of transparency).

### YUV

Unlike RGB, the **YUV** (or **Y'UV**) color encoding system is based around how humans perceive a color image. Doing so allows color data to be represented using fewer total bits of space in a video stream.

The name "YUV" is generally used today to describe this color system, even though the term was originally used specifically for analog coding of color, while **YCbCr** was used for digital color. However, today both terms are used for digital color. The prime symbol, `'`, indicates that gamma compression is being used.

#### Contemplating human vision

We're able to represent an image this way because the human eye sees far more detail in greyscale than in color, thanks to biology. The human eye contains two kinds of photoreceptor (light-sensing) cells. The **rods** are highly-sensitive light intensity sensors. The **cones** are less sensitive but are able to detect color.

Because the eye has vastly more rods than cones (about 120 million rods to around 6 or 7 million cones), we see detail in greyscale, with color being far less detailed. In essence, our eyes are like a camera with two image sensor chips: one greyscale and one color. The greyscale sensor is 120 megapixels, while the color sensor is only about 7 megapixels. These sensors physically occupy the same surface, called the **retina**, at the back of the eye.

There are three types of cones, each of which responds to a particular wavelength band of incoming light, but also to the intensity of the light at that wavelength. Each type of cone, then, captures the relative response peaks at various wavelengths, and the brain uses this data to figure out the intensity and hue of the color of the light arriving at that part of the retina.

As our eyes capture a scene, they essentially take two photos at once: a finely-detailed greyscale image and a somewhat fuzzy, lower-resolution map of the colors of light received across the retina at the same time. The final color image is created by the brain, which essentially layers the colors onto the greyscale pixels.

#### YUV concepts

YUV works by defining a color space with three components:

Luma (Y')

The brightness of the pixel. Without the other two components, the luma of each pixel in the frame produces a greyscale representation of the image. In BT.709 (used for HDTV), for example, the luma value is the weighted sum of the gamma-corrected red, green, and blue components of the pixel, using the formula `Y' = 0.2126R' + 0.7152G' + 0.0722B'`. This weighting is performed because of the previously-mentioned ratio of green, red, and blue cones in the eye.

Blue-difference (U or Cb)

The blue-difference component of the chroma, or color, of the color sample. This value is computed by subtracting the luma from the gamma corrected blue value; that is, `U = B' - Y'`.

Red-difference (V or Cr)

The red-difference component of the chroma of the color sample. Computed by subtracting luma from the gamma corrected red value: `V = R' - Y'`.

#### Chroma subsampling

**Chroma subsampling** is the process of converting the color pixels into a greyscale image and a matrix of U and V values indicating the colors to map to those pixels.

Since the color data is being encoded at a lower resolution than the luma, when decoding the video to draw it to the screen each pixel's color is computed by calculating an appropriate color given the U and V values for the 4x2 block of 8 pixels in which the pixel resides. This decoding operation must mirror the method used during encoding, which is represented by three numbers separated by colons (":").

* The first number specifies the number of luminance samples per row encoded from the 4x2 pixel block. This value is essentially always 4, meaning "Use all of the luminance samples."
* The second number specifies how many chroma samples are used to represent the colors of the pixels in the first row.
* The third number indicates how many chroma samples are used to represent the colors of the pixels in the second row.

The table below shows examples of three chroma subsampling models.

|  | 4:2:0 | 4:2:2 | 4:4:4 |
| --- | --- | --- | --- |
| Luminance | Luminance of pixels in a 4:2 (8-pixel) block | Luminance of pixels in a 4:2 (8-pixel) block | Luminance of pixels in a 4:2 (8-pixel) block |
| Chroma (U and V) | A matrix of U and V values for a 4:2:0 decoding operation. The first row contains 2 chroma samples for U: a blue and a green. The first row's samples are duplicated in the second row for V. | A matrix of U and V values for a 4:2:2 decoding operation. The first row contains 2 chroma samples for U: a blue and a green. The second row contains 2 chroma samples for V: a pink and a yellow. | A matrix of U and V values for a 4:4:4 decoding operation. The first row contains 4 chroma samples for U: a light blue, a yellow, a green and a deep blue. The second row contains 4 chroma samples for V: a pink, a white, a red and a gray. |
| Decoded pixels | A 4:2 block of decoded pixels after a 4:2:0 decoding operation that applies the 2 samples of the chroma matrix to each row in the block of luminance data. The colors of the samples become darker when applied to gray shades, black when applied to solid black, and remain unchanged when applied to white. | A 4:2 block of decoded pixels after a 4:2:2 decoding operation that applies the 2 samples contained in each row of the chroma matrix to the corresponding rows in the block of luminance data. The colors of the samples become darker when applied to gray shades, black when applied to solid black and remain unchanged when applied to white. | A 4:2 block of decoded pixels after a 4:4:4 decoding operation that applies the 4 samples contained in each row of the chroma matrix to the corresponding rows in the block of luminance data. The colors of the samples become darker when applied to gray shades, black when applied to solid black and remain unchanged when applied to white. |

In all of these, all four pixels of each row in the block of luminance data are used in the decoded image. Looking at 4:2:0 chroma subsampling, in the first column, the value 2 indicates that the first row of chroma data has two samples in it; the 0 indicates that the second row has no chroma samples of its own, and that the first row's samples should be duplicated to the second row. In other words, we're taking color information from every other row of pixels in the source image and applying them to the greyscale image represented by the luma.

The second column shows 4:2:2 subsampling. The top row of four luma samples has two chroma samples applied to it, and the bottom row of luma has two samples applied to it as well. The eight pixels represented by this block, then, have four chroma samples shared among them.

The final example demonstrates 4:4:4 subsampling. There are four luma samples in each row and four chroma samples in each row as well, so that every pixel represented by the block is individually colored.

What if the chroma subsampling model is 4:0:0? That says to use every pixel of luma data, but that each row has 0 chroma samples applied to it. The resulting image, then, is comprised solely of the luminance data—a greyscale image.

#### YUV data representation

Because the image is represented using more detail in greyscale than in color, the values of Y', U, and V are not typically stored together, one sample per pixel, the way RGB images are stored in memory.

* Each frame of video is represented by a matrix of luma values, Y, with each pixel's color computed using the values in the U and V matrices. For example, in the AV1 codec, a record stores the encoded luma for a tile, and a second record contains the chroma data in the form of the U and V values.
* In broadcast television, the luma signal is the primary signal, while the chrominance data is sent on a subcarrier.
* The JPEG file type's compression works by first converting the image to the Y'UV color system, then compressing that data. Each of Y', Cb, and Cr are able to have values from 0 to 255.


Codecs used by WebRTC - Web media technologies

Codecs used by WebRTC
=====================

The WebRTC API makes it possible to construct websites and apps that let users communicate in real time, using audio and/or video as well as optional data and other information. To communicate, the two devices need to be able to agree upon a mutually-understood codec for each track so they can successfully communicate and present the shared media. This guide reviews the codecs that browsers are required to implement as well as other codecs that some or all browsers support for WebRTC.

Containerless media
-------------------

WebRTC uses bare `MediaStreamTrack` objects for each track being shared from one peer to another, without a container or even a `MediaStream` associated with the tracks. Which codecs can be within those tracks is not mandated by the WebRTC specification. However, RFC 7742 specifies that all WebRTC-compatible browsers must support VP8 and H.264's Constrained Baseline profile for video, and RFC 7874 specifies that browsers must support at least the Opus codec as well as G.711's PCMA and PCMU formats.

These two RFCs also lay out options that must be supported for each codec, as well as specific user comfort features such as echo cancellation. This guide reviews the codecs that browsers are required to implement as well as other codecs that some or all browsers support for WebRTC.

While compression is always a necessity when dealing with media on the web, it's of additional importance when videoconferencing in order to ensure that the participants are able to communicate without lag or interruptions. Of secondary importance is the need to keep the video and audio synchronized, so that the movements and any ancillary information (such as slides or a projection) are presented at the same time as the audio that corresponds.

General codec requirements
--------------------------

Before looking at codec-specific capabilities and requirements, there are a few overall requirements that must be met by *any* codec configuration used with WebRTC.

Unless the SDP specifically signals otherwise, the web browser receiving a WebRTC video stream must be able to handle video at 20 FPS at a minimum resolution of 320 pixels wide by 240 pixels tall. It's encouraged that video be encoded at a frame rate and size no lower than that, since that's essentially the lower bound of what WebRTC generally is expected to handle.

SDP supports a codec-independent way to specify preferred video resolutions (RFC 6236. This is done by sending an `a=imageattr` SDP attribute to indicate the maximum resolution that is acceptable. The sender is not required to support this mechanism, however, so you have to be prepared to receive media at a different resolution than you requested. Beyond this simple maximum resolution request, specific codecs may offer further ways to ask for specific media configurations.

Supported video codecs
----------------------

WebRTC establishes a baseline set of codecs which all compliant browsers are required to support. Some browsers may choose to allow other codecs as well.

Below are the video codecs which are *required* in any fully WebRTC-compliant browser, as well as the profiles which are required and the browsers which actually meet the requirement.

Mandatory video codecs| Codec name | Profile(s) | Browser compatibility |
| --- | --- | --- |
| VP8 | — | Chrome, Edge, Firefox, Safari (12.1+) |
| AVC / H.264 | Constrained Baseline (CB) | Chrome (52+), Edge, Firefox, Safari

 Firefox for Android 68 and later do not support AVC (H.264) anymore.
 This is due to a change in Google Play store requirements that prevent
 Firefox from downloading and installing the OpenH264 codec needed to
 handle H.264 in WebRTC connections. See
 this article on SUMO
 for details.
  |

For details on WebRTC-related considerations for each codec, see the sub-sections below by following the links on each codec's name.

Complete details of what video codecs and configurations WebRTC is required to support can be found in RFC 7742: WebRTC Video Processing and Codec Requirements. It's worth noting that the RFC covers a variety of video-related requirements, including color spaces (sRGB is the preferred, but not required, default color space), recommendations for webcam processing features (automatic focus, automatic white balance, automatic light level), and so on.

**Note:** These requirements are for web browsers and other fully-WebRTC compliant products. Non-WebRTC products that are able to communicate with WebRTC to some extent may or may not support these codecs, although they're encouraged to by the specification documents.

In addition to the mandatory codecs, some browsers support additional codecs as well. Those are listed in the following table.

Other video codecs| Codec name | Profile(s) | Browser compatibility |
| --- | --- | --- |
| VP9 | — | Chrome (48+), Firefox |

### VP8

VP8, which we describe in general in the main guide to video codecs used on the web, has some specific requirements that must be followed when using it to encode or decode a video track on a WebRTC connection.

Unless signaled otherwise, VP8 will use square pixels (that is, pixels with an aspect ratio of 1:1).

#### Other notes

The network payload format for sharing VP8 using RTP (such as when using WebRTC) is described in RFC 7741: RTP Payload Format for VP8 Video.

### AVC / H.264

Support for AVC's Constrained Baseline (CB) profile is required in all fully-compliant WebRTC implementations. CB is a subset of the main profile, and is specifically designed for low-complexity, low-delay applications such as mobile video and videoconferencing, as well as for platforms with lower performing video processing capabilities.

Our overview of AVC and its features can be found in the main video codec guide.

#### Special parameter support requirements

AVC offers a wide array of parameters for controlling optional values. In order to improve reliability of WebRTC media sharing across multiple platforms and browsers, it's required that WebRTC endpoints that support AVC handle certain parameters in specific ways. Sometimes this means a parameter must (or must not) be supported. Sometimes it means requiring a specific value for a parameter, or that a specific set of values be allowed. And sometimes the requirements are more intricate.

##### Parameters which are useful but not required

These parameters don't have to be supported by the WebRTC endpoint, and their use is not required either. Their use can improve the user experience in various ways, but don't have to be used. Indeed, some of these are pretty complicated to use.

`max-br`

If specified and supported by the software, the `max-br` parameter specifies the maximum video bit rate in units of 1,000 bps for VCL and 1,200 bps for NAL. You'll find details about this on page 47 of RFC 6184.

`max-cpb`

If specified and supported by the software, `max-cpb` specifies the maximum coded picture buffer size. This is a fairly complicated parameter whose unit size can vary. See page 45 of RFC 6184 for details.

`max-dpb`

If specified and supported, `max-dpb` indicates the maximum decoded picture buffer size, given in units of 8/3 macroblocks. See RFC 6184, page 46 for further details.

`max-fs`

If specified and supported by the software, `max-fs` specifies the maximum size of a single video frame, given as a number of macroblocks.

`max-mbps`

If specified and supported by the software, this value is an integer specifying the maximum rate at which macroblocks should be processed per second (in macroblocks per second).

`max-smbps`

If specified and supported by the software, this specifies an integer stating the maximum static macroblock processing rate in static macroblocks per second (using the hypothetical assumption that all macroblocks are static macroblocks).

##### Parameters with specific requirements

These parameters may or may not be required, but have some special requirement when used.

`packetization-mode`

All endpoints are required to support mode 1 (non-interleaved mode). Support for other packetization modes is optional, and the parameter itself is not required to be specified.

`sprop-parameter-sets`

Sequence and picture information for AVC can be sent either in-band or out-of-band. When AVC is used with WebRTC, this information *must* be signaled in-band; the `sprop-parameter-sets` parameter must therefore *not* be included in the SDP.

##### Parameters which must be specified

These parameters must be specified whenever using AVC in a WebRTC connection.

`profile-level-id`

All WebRTC implementations are *required* to specify and interpret this parameter in their SDP, identifying the sub-profile used by the codec. The specific value that is set is not defined; what matters is that the parameter be used at all. This is useful to note, since in RFC 6184 ("RTP Payload Format for H.264 Video"), `profile-level-id` is entirely optional.

#### Other requirements

For the purposes of supporting switching between portrait and landscape orientations, there are two methods that can be used. The first is the video orientation (CVO) header extension to the RTP protocol. However, if this isn't signaled as supported in the SDP, then it's encouraged that browsers support Display Orientation SEI messages, though not required.

Unless signaled otherwise, the pixel aspect ratio is 1:1, indicating that pixels are square.

#### Other notes

The payload format used for AVC in WebRTC is described in RFC 6184: RTP Payload Format for H.264 Video. AVC implementations for WebRTC are required to support the special "filler payload" and "full frame freeze" SEI messages; these are used to support switching among multiple input streams seamlessly.

Supported audio codecs
----------------------

The audio codecs which RFC 7874 mandates that all WebRTC-compatible browsers must support are shown in the table below.

Mandatory audio codecs| Codec name | Browser compatibility |
| --- | --- |
| Opus | Chrome, Edge, Firefox, Safari |
| G.711 PCM (A-law) | Chrome, Firefox, Safari |
| G.711 PCM (µ-law) | Chrome, Firefox, Safari |

See below for more details about any WebRTC-specific considerations that exist for each codec listed above.

It's useful to note that RFC 7874 defines more than a list of audio codecs that a WebRTC-compliant browser must support; it also provides recommendations and requirements for special audio features such as echo cancellation, noise reduction, and audio leveling.

**Note:** The list above indicates the minimum required set of codecs that all WebRTC-compatible endpoints are required to implement. A given browser may also support other codecs; however, cross-platform and cross-device compatibility may be at risk if you use other codecs without carefully ensuring that support exists in all browsers your users might choose.

In addition to the mandatory audio codecs, some browsers support additional codecs as well. Those are listed in the following table.

Other audio codecs| Codec name | Browser compatibility |
| --- | --- |
| G.722 | Chrome, Firefox, Safari |
| iLBC | Chrome, Safari |
| iSAC | Chrome, Safari |

**Internet Low Bitrate Codec** (**iLBC**) is an open-source narrow-band codec developed by Global IP Solutions and now Google, designed specifically for streaming voice audio. Google and some other browser developers have adopted it for WebRTC.

The **Internet Speech Audio Codec** (**iSAC**) is another codec developed by Global IP Solutions and now owned by Google, which has open-sourced it. It's used by Google Talk, QQ, and other instant messaging clients and is specifically designed for voice transmissions which are encapsulated within an RTP stream.

**Comfort noise** (**CN**) is a form of artificial background noise which is used to fill gaps in a transmission instead of using pure silence. This helps to avoid a jarring effect that can occur when voice activation and similar features cause a stream to stop sending data temporarily—a capability known as Discontinuous Transmission (DTX). In RFC 3389, a method for providing an appropriate filler to use during silences.

Comfort Noise is used with G.711, and may potentially be used with other codecs that don't have a built-in CN feature. Opus, for example, has its own CN capability; as such, using RFC 3389 CN with the Opus codec is not recommended.

An audio sender is never required to use discontinuous transmission or comfort noise.

### Opus

The Opus format, defined by RFC 6716 is the primary format for audio in WebRTC. The RTP payload format for Opus is found in RFC 7587. You can find more general information about Opus and its capabilities, and how other APIs can support Opus, in the corresponding section of our guide to audio codecs used on the web.

Both the speech and general audio modes should be supported. Opus's scalability and flexibility are useful when dealing with audio that may have varying degrees of complexity. Its support of in-band stereo signals allows support for stereo without complicating the demultiplexing process.

The entire range of bit rates supported by Opus (6 kbps to 510 kbps) is supported in WebRTC, with the bit rate allowed to be dynamically changed. Higher bit rates typically improve quality.

#### Bit rate recommendations

Given a 20 millisecond frame size, the following table shows the recommended bit rates for various forms of media.

| Media type | Recommended bit rate range |
| --- | --- |
| Narrow-band speech (NB) | 8 to 12 kbps |
| Wide-band speech (WB) | 16 to 20 kbps |
| Full-band speech (FB) | 28 to 40 kbps |
| Full-band monaural music (FB mono) | 48 to 64 kbps |
| Full-band stereo music (FB stereo) | 64 to 128 kbps |

The bit rate may be adjusted at any time. In order to avoid network congestion, the average audio bit rate should not exceed the available network bandwidth (minus any other known or anticipated added bandwidth requirements).

### G.711

G.711 defines the format for **Pulse Code Modulation** (**PCM**) audio as a series of 8-bit integer samples taken at a sample rate of 8,000 Hz, yielding a bit rate of 64 kbps. Both µ-law and A-law encodings are allowed.

G.711 is defined by the ITU and its payload format is defined in RFC 3551, section 4.5.14.

WebRTC requires that G.711 use 8-bit samples at the standard 64 kbps rate, even though G.711 supports some other variations. Neither G.711.0 (lossless compression), G.711.1 (wideband capability), nor any other extensions to the G.711 standard are mandated by WebRTC.

Due to its low sample rate and sample size, G.711 audio quality is generally considered poor by modern standards, even though it's roughly equivalent to what a landline telephone sounds like. It is generally used as a least common denominator to ensure that browsers can achieve an audio connection regardless of platforms and browsers, or as a fallback option in general.

Specifying and configuring codecs
---------------------------------

### Getting the supported codecs

Because a given browser and platform may have different availability among the potential codecs—and may have multiple profiles or levels supported for a given codec—the first step when configuring codecs for an `RTCPeerConnection` is to get the list of available codecs. To do this, you first have to establish a connection on which to get the list.

There are a couple of ways you can do this. The most efficient way is to use the static method `RTCRtpSender.getCapabilities()` (or the equivalent `RTCRtpReceiver.getCapabilities()` for a receiver), specifying the type of media as the input parameter. For example, to determine the supported codecs for video, you can do this:

```js
codecList = RTCRtpSender.getCapabilities("video").codecs;
```

 Now `codecList` is an array `codec` objects, each describing one codec configuration.
 Also present in the list will be entries for retransmission (RTX), redundant coding (RED), and forward error correction (FEC).

If the connection is in the process of starting up, you can use the `icegatheringstatechange` event to watch for the completion of ICE candidate gathering, then fetch the list.

```js
let codecList = null;

peerConnection.addEventListener("icegatheringstatechange", (event) => {
  if (peerConnection.iceGatheringState === "complete") {
    const senders = peerConnection.getSenders();

    senders.forEach((sender) => {
      if (sender.track.kind === "video") {
        codecList = sender.getParameters().codecs;
        return;
      }
    });
  }

  codecList = null;
});
```

The event handler for `icegatheringstatechange` is established; in it, we look to see if the ICE gathering state is `complete`, indicating that no further candidates will be collected. The method `RTCPeerConnection.getSenders()` is called to get a list of all the `RTCRtpSender` objects used by the connection.

 With that in hand, we walk through the list of senders, looking for the first one whose `MediaStreamTrack` indicates that it's `kind` is `video`, indicating that the track's data is video media.
 We then call that sender's `getParameters()` method and set `codecList` to the `codecs` property in the returned object, and then return to the caller.

If no video track is found, we set `codecList` to `null`.

On return, then, `codecList` is either `null` to indicate that no video tracks were found or it's an array of `RTCRtpCodecParameters` objects, each describing one permitted codec configuration. Of special importance in these objects: the `payloadType` property, which is a one-byte value which uniquely identifies the described configuration.

**Note:** The two methods for obtaining lists of codecs shown here use different output types in their codec lists. Be aware of this when using the results.

### Customizing the codec list

Once you have a list of the available codecs, you can alter it and then send the revised list to `RTCRtpTransceiver.setCodecPreferences()` to rearrange the codec list. This changes the order of preference of the codecs, letting you tell WebRTC to prefer a different codec over all others.

```js
function changeVideoCodec(mimeType) {
  const transceivers = peerConnection.getTransceivers();

  transceivers.forEach((transceiver) => {
    const kind = transceiver.sender.track.kind;
    let sendCodecs = RTCRtpSender.getCapabilities(kind).codecs;
    let recvCodecs = RTCRtpReceiver.getCapabilities(kind).codecs;

    if (kind === "video") {
      sendCodecs = preferCodec(mimeType);
      recvCodecs = preferCodec(mimeType);
      transceiver.setCodecPreferences([...sendCodecs, ...recvCodecs]);
    }
  });

  peerConnection.onnegotiationneeded();
}
```

In this sample, the function `changeVideoCodec()` takes as input the MIME type of the codec you wish to use. The code starts by getting a list of all of the `RTCPeerConnection`'s transceivers.

Then, for each transceiver, we get the kind of media represented by the transceiver from the `RTCRtpSender`'s track's `kind`. We also get the lists of all codecs supported by the browser for both sending and receiving video, using the `getCapabilities()` static method of both `RTCRtpSender` and `RTCRtpReceiver`.

If the media is video, we call a method called `preferCodec()` for both the sender's and receiver's codec lists; this method rearranges the codec list the way we want (see below).

Finally, we call the `RTCRtpTransceiver`'s `setCodecPreferences()` method to specify that the given send and receive codecs are allowed, in the newly rearranged order.

That's done for each transceiver on the `RTCPeerConnection`; once all of the transceivers have been updated, we call the `onnegotiationneeded` event handler, which will create a new offer, update the local description, send the offer along to the remote peer, and so on, thereby triggering the renegotiation of the connection.

The `preferCodec()` function called by the code above looks like this to move a specified codec to the top of the list (to be prioritized during negotiation):

```js
function preferCodec(codecs, mimeType) {
  let otherCodecs = [];
  let sortedCodecs = [];
  let count = codecs.length;

  codecs.forEach((codec) => {
    if (codec.mimeType === mimeType) {
      sortedCodecs.push(codec);
    } else {
      otherCodecs.push(codec);
    }
  });

  return sortedCodecs.concat(otherCodecs);
}
```

This code is just splitting the codec list into two arrays: one containing codecs whose MIME type matches the one specified by the `mimeType` parameter, and the other with all the other codecs. Once the list has been split up, they're concatenated back together with the entries matching the given `mimeType` first, followed by all of the other codecs. The rearranged list is then returned to the caller.

Default codecs
--------------

Unless otherwise specified, the default—or, more accurately, preferred—codecs requested by each browser's implementation of WebRTC are shown in the table below.

Preferred codecs for WebRTC in major web browsers|  | Audio | Video |
| --- | --- | --- |
| Chrome |  |  |
| Edge |  |  |
| Firefox |  | VP9 (Firefox 46 and later)VP8 |
| Opera |  |  |
| Safari |  |  |

Choosing the right codec
------------------------

Before choosing a codec that isn't one of the mandatory codecs (VP8 or AVC for video and Opus or PCM for audio), you should seriously consider the potential drawbacks: in particular, only these codecs can be generally assumed to be available on essentially all devices that support WebRTC.

If you choose to prefer a codec other than the mandatory ones, you should at least allow for fallback to one of the mandatory codecs if support is unavailable for the codec you prefer.

### Audio

In general, if it's available and the audio you wish to send has a sample rate greater than 8 kHz, you should strongly consider using Opus as your primary codec. For voice-only connections in a constrained environment, using G.711 at an 8 kHz sample rate can provide an acceptable experience for conversation, but typically you'll use G.711 as a fallback option, since there are other options which are more efficient and sound better, such as Opus in its narrowband mode.

### Video

There are a number of factors that come into play when deciding upon a video codec (or set of codecs) to support.

#### Licensing terms

Before choosing a video codec, make sure you're aware of any licensing requirements around the codec you select; you can find information about possible licensing concerns in our main guide to video codecs used on the web. Of the two mandatory codecs for video—VP8 and AVC/H.264—only VP8 is completely free of licensing requirements. If you select AVC, make sure you're; aware of any potential fees you may need to pay; that said, the patent holders have generally said that most typical website developers shouldn't need to worry about paying the license fees, which are typically focused more on the developers of the encoding and decoding software.

**Warning:** The information here does *not* constitute legal advice! Be sure to confirm your exposure to liability before making any final decisions where potential exists for licensing issues.

#### Power needs and battery life

Another factor to consider—especially on mobile platforms—is the impact a codec may have on battery life. If a codec is handled in hardware on a given platform, that codec is likely to allow for much better battery life and less heat production.

For example, Safari for iOS and iPadOS introduced WebRTC with AVC as the only supported video codec. AVC has the advantage, on iOS and iPadOS, of being able to be encoded and decoded in hardware. Safari 12.1 introduced support for VP8 within IRC, which improves interoperability, but at a cost—VP8 has no hardware support on iOS devices, so using it causes increased processor impact and reduced battery life.

#### Performance

Fortunately, VP8 and AVC perform similarly from an end-user perspective, and are equally adequate for use in videoconferencing and other WebRTC solutions. The final decision is yours. Whichever you choose, be sure to read the information provided in this article about any particular configuration issues you may need to contend with for that codec.

Keep in mind that choosing a codec that isn't on the list of mandatory codecs likely runs the risk of selecting a codec which isn't supported by a browser your users might prefer. See the article Handling media support issues in web content to learn more about how to offer support for your preferred codecs while still being able to fall back on browsers that don't implement that codec.

Security implications
---------------------

There are interesting potential security issues that come up while selecting and configuring codecs. WebRTC video is protected using Datagram Transport Layer Security (DTLS), but it is theoretically possible for a motivated party to infer the amount of change that's occurring from frame to frame when using variable bit rate (VBR) codecs, by monitoring the stream's bit rate and how it changes over time. This could potentially allow a bad actor to infer something about the content of the stream, given the ebb and flow of the bit rate.

For more about security considerations when using AVC in WebRTC, see RFC 6184, section 9: RTP Payload Format for H.264 Video: Security Considerations.

RTP payload format media types
------------------------------

It may be useful to refer to the IANA's list of RTP payload format media types; this is a complete list of the MIME media types defined for *potential* use in RTP streams, such as those used in WebRTC. Most of these are not used in WebRTC contexts, but the list may still be useful.

See also RFC 4855, which covers the registry of media types.

See also
--------

* WebRTC API
* Introduction to WebRTC protocols
* WebRTC connectivity
* Guide to video codecs used on the web
* Guide to audio codecs used on the web
* Digital video concepts
* Digital audio concepts


Codecs in common media types - Web media technologies

Codecs in common media types
============================

At a fundamental level, you can specify the type of a media file using a simple MIME type, such as `video/mp4` or `audio/mpeg`. However, many media types—especially those that support video tracks—can benefit from the ability to more precisely describe the format of the data within them. For instance, just describing a video in an MPEG-4 file with the MIME type `video/mp4` doesn't say anything about what format the actual media within takes.

For that reason, the `codecs` parameter can be added to the MIME type describing media content. With it, container-specific information can be provided. This information may include things like the profile of the video codec, the type used for the audio tracks, and so forth.

This guide briefly examines the syntax of the media type `codecs` parameter and how it's used with the MIME type string to provide details about the contents of audio or video media beyond indicating the container type.

Container format MIME types
---------------------------

The MIME type for a container format is expressed by stating the type of media (`audio`, `video`, etc.), then a slash character (`/`), then the format used to contain the media:

`audio/mpeg`

An audio file using the MPEG file type, such as an MP3.

`video/ogg`

A video file using the Ogg file type.

`video/mp4`

A video file using the MPEG-4 file type.

`video/quicktime`

A video file in Apple's QuickTime format. As noted elsewhere, this format was once commonly used on the web but no longer is, since it required a plugin to use.

However, each of these MIME types is vague. All of these file types support a variety of codecs, and those codecs may have any number of profiles, levels, and other configuration factors. For this reason, you may want to include the `codecs` parameter along with the media type.

Basic syntax
------------

You can add the `codecs` parameter to the media type. To do so, append a semicolon (`;`) followed by `codecs=` and then the string describing the format of the contents of the file. Some media types only let you specify the names of the codecs to use, while others allow you to specify various constraints on those codecs as well. You can specify multiple codecs by separating them with commas.

`audio/ogg; codecs=vorbis`

An Ogg file containing a Vorbis audio track.

`video/webm; codecs="vp8, vorbis"`

A WebM file containing VP8 video and/or Vorbis audio.

`video/mp4; codecs="avc1.4d002a"`

An MPEG-4 file containing AVC (H.264) video, Main Profile, Level 4.2.

As is the case with any MIME type parameter, `codecs` must be changed to `codecs*` (note the asterisk character, `*`) if any of the properties of the codec use special characters which must be percent-encoded per RFC 2231, section 4: MIME Parameter Value and Encoded Word Extensions. You can use the JavaScript `encodeURI()` function to encode the parameter list; similarly, you can use `decodeURI()` to decode a previously encoded parameter list.

**Note:** When the `codecs` parameter is used, the specified list of codecs must include every codec used for the contents of the file. The list may also contain codecs not present in the file.

Codec options by container
--------------------------

The containers below support extended codec options in their `codecs` parameters:

* 3GP
* AV1
* ISO BMFF
* MPEG-4
* QuickTime
* WebM

Several of the links above go to the same section; that's because those media types are all based on ISO Base Media File Format (ISO BMFF), so they share the same syntax.

### AV1

The syntax of the `codecs` parameter for AV1 is defined the AV1 Codec ISO Media File Format Binding specification, section 5: Codecs Parameter String.
```

av01.P.LLT.DD[.M.CCC.cp.tc.mc.F]
```

**Note:** Chromium-based browsers will accept any subset of the optional parameters (rather than all or none, as required by the specification).

This codec parameter string's components are described in more detail in the table below. Each component is a fixed number of characters long; if the value is less than that length, it must be padded with leading zeros.

AV1 codec parameter string components| Component | Details |
| --- | --- |
| `P` | The one-digit profile number:

AV1 profile numbers| Profile number | Description |
| --- | --- |
| 0 | "Main" profile; supports YUV 4:2:0 or monochrome bitstreams with bit depth of 8 or 10 bits per component. |
| 1 | "High" profile adds support for 4:4:4 chroma subsampling. |
| 2 | "Professional" profile adds support for 4:2:2 chroma subsampling and 12-bit per component color. |

 |
| `LL` |
 The two-digit level number, which is converted to the X.Y format level format, where `X = 2 + (LL >> 2)` and `Y = LL & 3`.
 See Appendix A, section 3 in the AV1 Specification for details.
  |
| `T` |
 The one-character tier indicator. For the Main tier (`seq_tier` equals 0), this character is the letter `M`.
 For the High tier (`seq_tier` is 1), this character is the letter `H`.
 The High tier is only available for level 4.0 and up.
  |
| `DD` | The two-digit component bit depth. This value must be one of 8, 10, or 12; which values are valid varies depending on the profile and other properties. |
| `M` |
 The one-digit monochrome flag; if this is 0, the video includes the U and V planes in addition to the Y plane.
 Otherwise, the video data is entirely in the Y plane and is therefore monochromatic.
 See [YUV](/en-US/docs/Web/Media/Formats/Video\_concepts#yuv) for details on how the YUV color system works.
 The default value is 0 (not monochrome).
  |
| `CCC` |
`CCC` indicates the chroma subsampling as three digits.
 The first digit is `subsampling_x`, the second is `subsampling_y`.
 If both of those are 1, the third is the value of `chroma_sample_position`; otherwise, the third digit is always 0.
 This, combined with the `M` component, can be used to construct the chroma subsampling format:

Determining the chroma subsampling format| subsampling\_x | subsampling\_y | Monochrome flag | Chroma subsampling format |
| --- | --- | --- | --- |
| 0 | 0 | 0 | YUV 4:4:4 |
| 1 | 0 | 0 | YUV 4:2:2 |
| 1 | 1 | 0 | YUV 4:2:0 |
| 1 | 1 | 1 | YUV 4:0:0 (Monochrome) |

The third digit in `CCC` indicates the chroma sample position, with a value of 0 indicating that the position is unknown and must be separately provided during decoding; a value of 1 indicating that the sample position is horizontally collocated with the (0, 0) luma sample; and a value of 2 indicating that the sample position is collocated with (0, 0) luma.
The default value is `110` (4:2:0 chroma subsampling). |
| `cp` |
 The two-digit `color_primaries` value indicates the color system used by the media.
 For example, BT.2020/BT.2100 color, as used for HDR video, is `09`.
 The information for this—and for each of the remaining components—is found in the Color config semantics section of the AV1 specification.
 The default value is `01` (ITU-R BT.709).
  |
| `tc` |
 The two-digit `transfer_characteristics` value. This value defines the function used to map the gamma (delightfully called the "opto-electrical transfer function" in technical parlance) from the source to the display.
 For example, 10-bit BT.2020 is `14`.
 The default value is `01` (ITU-R BT.709).
  |
| `mc` |
 The two-digit `matrix_coefficients` constant selects the matrix coefficients used to convert the red, blue, and green channels into luma and chroma signals.
 For example, the standard coefficients used for BT.709 are indicated using the value `01`.
 The default value is `01` (ITU-R BT.709).
  |
| `F` |
 A one-digit flag indicating whether the color should be allowed to use the full range of possible values (`1`), or should be constrained to those values considered legal for the specified color configuration (that is, the **studio swing representation**).
 The default is 0 (use the studio swing representation).
  |

All fields from `M` (monochrome flag) onward are optional; you may stop including fields at any point (but can't arbitrarily leave out fields). The default values are included in the table above. Some example AV1 codec strings:

`av01.2.15M.10.0.100.09.16.09.0`

AV1 Professional Profile, level 5.3, Main tier, 10 bits per color component, 4:2:2 chroma subsampling using ITU-R BT.2100 color primaries, transfer characteristics, and YCbCr color matrix. The studio swing representation is indicated.

`av01.0.15M.10`

AV1 Main Profile, level 5.3, Main tier, 10 bits per color component. The remaining properties are taken from the defaults: 4:2:0 chroma subsampling, BT.709 color primaries, transfer characteristics, and matrix coefficients. Studio swing representation.

### ISO Base Media File Format: MP4, QuickTime, and 3GP

All media types based upon the ISO Base Media File Format (ISO BMFF) share the same syntax for the `codecs` parameter. These media types include MPEG-4 (and, in fact, the QuickTime file format upon which MPEG-4 is based) as well as 3GP. Both video and audio tracks can be described using the `codecs` parameter with the following MIME types:

| MIME type | Description |
| --- | --- |
| `audio/3gpp` | 3GP audio (RFC 3839: MIME Type Registrations for 3rd generation Partnership Project (3GP) Multimedia files) |
| `video/3gpp` | 3GP video (RFC 3839: MIME Type Registrations for 3rd generation Partnership Project (3GP) Multimedia files) |
| `audio/3gp2` | 3GP2 audio (RFC 4393: MIME Type Registrations for 3GPP2 Multimedia files) |
| `video/3gp2` | 3GP2 video (RFC 4393: MIME Type Registrations for 3GPP2 Multimedia files) |
| `audio/mp4` | MP4 audio (RFC 4337: MIME Type Registration for MPEG-4) |
| `video/mp4` | MP4 video (RFC 4337: MIME Type Registration for MPEG-4) |
| `application/mp4` | Non-audiovisual media encapsulated in MPEG-4 |

Each codec described by the `codecs` parameter can be specified either as the container's name (`3gp`, `mp4`, `quicktime`, etc.) or as the container name plus additional parameters to specify the codec and its configuration. Each entry in the codec list may contain some number of components, separated by periods (`.`).

The syntax for the value of `codecs` varies by codec; however, it always starts with the codec's four-character identifier, a period separator (`.`), followed by the Object Type Indication (OTI) value for the specific data format. For most codecs, the OTI is a two-digit hexadecimal number; however, it's six hexadecimal digits for AVC (H.264).

Thus, the syntaxes for each of the supported codecs look like this:

`cccc[.pp]*` (Generic ISO BMFF)

Where `cccc` is the four-character ID for the codec and `pp` is where zero or more two-character encoded property values go.

`mp4a.oo[.A]` (MPEG-4 audio)

Where `oo` is the Object Type Indication value describing the contents of the media more precisely and `A` is the one-digit *audio* OTI. The possible values for the OTI can be found on the MP4 Registration Authority website's Object Types page. For example, Opus audio in an MP4 file is `mp4a.ad`. For further details, see MPEG-4 audio.

`mp4v.oo[.V]` (MPEG-4 video)

Here, `oo` is again the OTI describing the contents more precisely, while `V` is the one-digit *video* OTI.

`avc1[.PPCCLL]` (AVC video)

`PPCCLL` are six hexadecimal digits specifying the profile number (`PP`), constraint set flags (`CC`), and level (`LL`). See AVC profiles for the possible values of `PP`.

The constraint set flags byte is comprised of one-bit Boolean flags, with the most significant bit being referred to as flag 0 (or `constraint_set0_flag`, in some resources), and each successive bit being numbered one higher. Currently, only flags 0 through 2 are used; the other five bits *must* be zero. The meanings of the flags vary depending on the profile being used.

The level is a fixed-point number, so a value of `14` (decimal 20) means level 2.0 while a value of `3D` (decimal 61) means level 6.1. Generally speaking, the higher the level number, the more bandwidth the stream will use and the higher the maximum video dimensions are supported.

#### AVC profiles

The following are the AVC profiles and their profile numbers for use in the `codecs` parameter, as well as the value to specify for the constraints component, `CC`.

| Profile | Number (Hex) | Constraints byte |
| --- | --- | --- |
| **Constrained Baseline Profile (CBP)** CBP is primarily a solution for scenarios in which resources are constrained, or resource use needs to be controlled to minimize the odds of the media performing poorly. | `42` | `40` |
| **Baseline Profile (BP)** Similar to CBP but with more data loss protections and recovery capabilities. This is not as widely used as it was before CBP was introduced. All CBP streams are considered to also be BP streams. | `42` | `00` |
| **Extended Profile (XP)** Designed for streaming video over the network, with high compression capability and further improvements to data robustness and stream switching. | `58` | `00` |
| **Main Profile (MP)** The profile used for standard-definition digital television being broadcast in MPEG-4 format. *Not* used for high-definition television broadcasts. This profile's importance has faded since the introduction of the High Profile—which was added for HDTV use—in 2004. | `4D` | `00` |
| **High Profile (HiP)** Currently, HiP is the primary profile used for broadcast and disc-based HD video; it's used both for HD TV broadcasts and for Blu-Ray video. | `64` | `00` |
| **Progressive High Profile (PHiP)** Essentially High Profile without support for field coding. | `64` | `08` |
| **Constrained High Profile** PHiP, but without support for bi-predictive slices ("B-slices"). | `64` | `0C` |
| **High 10 Profile (Hi10P)** High Profile, but with support for up to 10 bits per color component. | `6E` | `00` |
| **High 4:2:2 Profile (Hi422P)** Expands upon Hi10P by adding support for 4:2:2 chroma subsampling along with up to10 bits per color component. | `7A` | `00` |
| **High 4:4:4 Predictive Profile (Hi444PP)** In addition to the capabilities included in Hi422P, Hi444PP adds support for 4:4:4 chroma subsampling (in which no color information is discarded). Also includes support for up to 14 bits per color sample and efficient lossless region coding. The option to encode each frame as three separate color planes (that is, each color's data is stored as if it were a single monochrome frame). | `F4` | `00` |
| **High 10 Intra Profile** High 10 constrained to all-intra-frame use. Primarily used for professional apps. | `6E` | `10` |
| **High 4:2:2 Intra Profile** The Hi422 Profile with all-intra-frame use. | `7A` | `10` |
| **High 4:4:4 Intra Profile** The High 4:4:4 Profile constrained to use only intra frames. | `F4` | `10` |
| **CAVLC 4:4:4 Intra Profile** The High 4:4:4 Profile constrained to all-intra use, and to using only CAVLC entropy coding. | `44` | `00` |
| **Scalable Baseline Profile** Intended for use with video conferencing as well as surveillance and mobile uses, the SVC Baseline Profile is based on AVC's Constrained Baseline profile. The base layer within the stream is provided at a high quality level, with some number of secondary substreams that offer alternative forms of the same video for use in various constrained environments. These may include any combination of reduced resolution, reduced frame rate, or increased compression levels. | `53` | `00` |
| **Scalable Constrained Baseline Profile** Primarily used for real-time communication applications. Not yet supported by WebRTC, but an extension to the WebRTC API to allow SVC is in development. | `53` | `04` |
| **Scalable High Profile** Meant mostly for use in broadcast and streaming applications. The base (or highest quality) layer must conform to the AVC High Profile. | `56` | `00` |
| **Scalable Constrained High Profile** A subset of the Scalable High Profile designed mainly for real-time communication. | `56` | `04` |
| **Scalable High Intra Profile** Primarily useful only for production applications, this profile supports only all-intra usage. | `56` | `20` |
| **Stereo High Profile** The Stereo High Profile provides stereoscopic video using two renderings of the scene (left eye and right eye). Otherwise, provides the same features as the High profile. | `80` | `00` |
| **Multiview High Profile** Supports two or more views using both temporal and MVC inter-view prediction. *Does not support* field pictures or macroblock-adaptive frame-field coding. | `76` | `00` |
| **Multiview Depth High Profile** Based on the High Profile, to which the main substream must adhere. The remaining substreams must match the Stereo High Profile. | `8A` | `00` |

#### MPEG-4 audio

When the value of an entry in the `codecs` list begins with `mp4a`, the syntax of the value should be:
```

mp4a.oo[.A]
```

Here, `oo` is the two-digit hexadecimal Object Type Indication which specifies the codec class being used for the media. The OTIs are assigned by the MP4 Registration Authority, which maintains a list of the possible OTI values. A special value is `40`; this indicates that the media is MPEG-4 audio (ISO/IEC 14496 Part 3). In order to be more specific still, a third component—the Audio Object Type—is added for OTI `40` to narrow the type down to a specific subtype of MPEG-4.

The Audio Object Type is specified as a one or two digit *decimal* value (unlike most other values in the `codecs` parameter, which use hexadecimal). For example, MPEG-4's AAC-LC has an audio object type number of `2`, so the full `codecs` value representing AAC-LC is `mp4a.40.2`.

Thus, ER AAC LC, whose Audio Object Type is 17, can be represented using the full `codecs` value `mp4a.40.17`. Single digit values can be given either as one digit (which is the best choice, since it will be the most broadly compatible) or with a leading zero padding it to two digits, such as `mp4a.40.02`.

**Note:** The specification originally mandated that the Audio Object Type number in the third component be only one decimal digit. However, amendments to the specification over time extended the range of these values well beyond one decimal digit, so now the third parameter may be either one or two digits. Padding values below 10 with a leading `0` is optional. Older implementations of MPEG-4 codecs may not support two-digit values, however, so using a single digit when possible will maximize compatibility.

The Audio Object Types are defined in ISO/IEC 14496-3 subpart 1, section 1.5.1. The table below provides a basic list of the Audio Object Types and in the case of the more common object types provides a list of the profiles supporting it, but you should refer to the specification for details if you need to know more about the inner workings of any given MPEG-4 audio type.

MPEG-4 audio object types| ID | Audio Object Type | Profile support |
| --- | --- | --- |
| `0` | NULL |  |
| `1` | AAC Main | Main |
| `2` | AAC LC (Low Complexity) | Main, Scalable, HQ, LD v2, AAC, HE-AAC, HE-AAC v2 |
| `3` | AAC SSR (Scalable Sampling Rate) | Main |
| `4` | AAC LTP (Long Term Prediction) | Main, Scalable, HQ |
| `5` | SBR (Spectral Band Replication) | HE-AAC, HE-AAC v2 |
| `6` | AAC Scalable | Main, Scalable, HQ |
| `7` | TwinVQ (Coding for ultra-low bit rates) | Main, Scalable |
| `8` | CELP (Code-Excited Linear Prediction) | Main, Scalable, Speech, HQ, LD |
| `9` | HVXC (Harmonic Vector Excitation Coding) | Main, Scalable, Speech, LD |
| `10` – `11` | *Reserved* |  |
| `12` | TTSI (Text to Speech Interface) | Main, Scalable, Speech, Synthetic, LD |
| `13` | Main Synthetic | Main, Synthetic |
| `14` | Wavetable Synthesis |  |
| `15` | General MIDI |  |
| `16` | Algorithmic Synthesis and Audio Effects |  |
| `17` | ER AAC LC (Error Resilient AAC Low-Complexity) | HQ, Mobile Internetworking |
| `18` | *Reserved* |  |
| `19` | ER AAC LTP (Error Resilient AAC Long Term Prediction) | HQ |
| `20` | ER AAC Scalable (Error Resilient AAC Scalable) | Mobile Internetworking |
| `21` | ER TwinVQ (Error Resilient TwinVQ) | Mobile Internetworking |
| `22` | ER BSAC (Error Resilient Bit-Sliced Arithmetic Coding) | Mobile Internetworking |
| `23` |
 ER AAC LD (Error Resilient AAC Low-Delay; used for two-way
 communication)
  | LD, Mobile Internetworking |
| `24` | ER CELP (Error Resilient Code-Excited Linear Prediction) | HQ, LD |
| `25` | ER HVXC (Error Resilient Harmonic Vector Excitation Coding) | LD |
| `26` | ER HILN (Error Resilient Harmonic and Individual Line plus Noise) |  |
| `27` | ER Parametric (Error Resilient Parametric) |  |
| `28` | SSC (Sinusoidal Coding) |  |
| `29` | PS (Parametric Stereo) | HE-AAC v2 |
| `30` | MPEG Surround |  |
| `31` | *Escape* |  |
| `32` | MPEG-1 Layer-1 |  |
| `33` | MPEG-1 Layer-2 (MP2) |  |
| `34` | MPEG-1 Layer-3 (MP3) |  |
| `35` | DST (Direct Stream Transfer) |  |
| `36` | ALS (Audio Lossless) |  |
| `37` | SLS (Scalable Lossless) |  |
| `38` | SLS Non-core (Scalable Lossless Non-core) |  |
| `39` | ER AAC ELD (Error Resilient AAC Enhanced Low Delay) |  |
| `40` | SMR Simple (Symbolic Music Representation Simple) |  |
| `41` | SMR Main (Symbolic Music Representation Main) |  |
| `42` | *Reserved* |  |
| `43` | SAOC (Spatial Audio Object Coding)

 Defined in
 ISO/IEC 14496-3:2009/Amd.2:2010(E).
  |  |
| `44` | LD MPEG Surround (Low Delay MPEG Surround)
Defined in ISO/IEC 14496-3:2009/Amd.2:2010(E). |  |
| `45` and up | *Reserved* |  |

### WebM

The basic form for a WebM `codecs` parameter is to list one or more of the four WebM codecs by name, separated by commas. The table below shows some examples:

| MIME type | Description |
| --- | --- |
| `video/webm;codecs="vp8"` | A WebM video with VP8 video in it; no audio is specified. |
| `video/webm;codecs="vp9"` | A WebM video with VP9 video in it. |
| `audio/webm;codecs="vorbis"` | Vorbis audio in a WebM container. |
| `audio/webm;codecs="opus"` | Opus audio in a WebM container. |
| `video/webm;codecs="vp8,vorbis"` | A WebM container with VP8 video and Vorbis audio. |
| `video/webm;codecs="vp9,opus"` | A WebM container with VP9 video and Opus audio. |

The strings `vp8.0` and `vp9.0` also work, but are not recommended.

#### ISO Base Media File Format syntax

As part of a move toward a standardized and powerful format for the `codecs` parameter, WebM is moving toward describing *video* content using a syntax based on that defined by the ISO Base Media File Format. This syntax is defined in VP Codec ISO Media File Format Binding, in the section Codecs Parameter String. The audio codec continues to be indicated as either `vorbis` or `opus`.

In this format, the `codecs` parameter's value begins with a four-character code identifying the codec being used in the container, which is then followed by a series of period (`.`) separated two-digit values.
```

cccc.PP.LL.DD
cccc.PP.LL.DD.CC.cp.tc.mc.FF
```

The first four components are required; everything from `CC` (chroma subsampling) onward is optional, but all or nothing. Each of these components is described in the following table. Following the table are some examples.

WebM codecs parameter components| Component | Details |
| --- | --- |
| `cccc` |
 A four-character code indicating which indicates which of the possible codecs is being described.
 Potential values are:

Four-character codes for WebM-supported codecs| Four-character code | Codec |
| --- | --- |
| `vp08` | VP8 |
| `vp09` | VP9 |
| `vp10` | VP10 |

 |
| `PP` | The two-digit profile number, padded with leading zeroes if necessary to be exactly two digits.

WebM profile numbers| Profile | Description |
| --- | --- |
| `00` |
 Only 4:2:0 (chroma subsampled both horizontally and vertically).
 Allows only 8 bits per color component.
  |
| `01` |
 All chroma subsampling formats are allowed.
 Allows only 8 bits per color component.
  |
| `02` |
 Only 4:2:0 (chroma subsampled both horizontally and vertically).
 Supports 8, 10, or 12 bits per color sample component.
  |
| `03` |
 All chroma subsampling formats are allowed.
 Supports 8, 10, or 12 bits per color sample component.
  |

 |
| `LL` |
 The two-digit level number.
 The level number is a fixed-point notation, where the first digit is the ones digit, and the second digit represents
 tenths.
 For example, level 3 is `30` and level 6.1 is `61`.
  |
| `DD` | The bit depth of the luma and color component values; permitted values are 8, 10, and 12. |
| `CC` |
 A two-digit value indicating which chroma subsampling format to use.
 The following table lists permitted values; see [Chroma subsampling](/en-US/docs/Web/Media/Formats/Video\_concepts#chroma\_subsampling) in our "Digital video concepts" guide for additional information about this topic and others.

WebM chroma subsampling identifiers| Value | Chroma subsampling format |
| --- | --- |
| `00` | 4:2:0 with the chroma samples sited interstitially between the pixels |
| `01` | 4:2:0 chroma subsampling with the samples collocated with luma (0, 0) |
| `02` | 4:2:2 chroma subsampling (4 out of each 4 horizontal pixels' luminance are used) |
| `03` | 4:4:4 chroma subsampling (every pixel's luminance and chrominance are both retained) |
| `04` | *Reserved* |

 |
| `cp` |
 A two-digit integer specifying which of the color primaries from Section 8.1 of the ISO/IEC 23001-8:2016 standard.
 This component, and every component after it, is optional.

The possible values of the color primaries component are:

ISO/IEC Color primary identifiers| Value | Details |
| --- | --- |
| `00` | *Reserved for future use by ITU or ISO/IEC* |
| `01` |
 BT.709, sRGB, sYCC. BT.709 is the standard for high definition (HD) television; sRGB is the most common color space used for computer displays.
 Broadcast BT.709 uses 8-bit color depth with the legal range being from 16 (black) to 235 (white).
  |
| `02` | Image characteristics are unknown, or are to be determined by the application |
| `03` | *Reserved for future use by ITU or ISO/IEC* |
| `04` | BT.470 System M, NTSC (standard definition television in the United States) |
| `05` | BT.470 System B, G; BT.601; BT.1358 625; BT.1700 625 PAL and 625 SECAM |
| `06` |
 BT.601 525; BT.1358 525 or 625; BT.1700 NTSC; SMPTE 170M.
 *Functionally identical to `7`.* |
| `70` | SMPTE 240M (historical).
 *Functionally identical to `6`.* |
| `08` | Generic film |
| `09` |
 BT.2020; BT.2100.
 Used for ultra-high definition (4K) High Dynamic Range (HDR) video, these have a very wide color gamut and support 10-bit and 12-bit color component depths.
  |
| `10` |
 SMPTE ST 428 (D-Cinema Distribution Master: Image characteristics).
 Defines the uncompressed image characteristics for DCDM.
  |
| `11` |
 SMPTE RP 431 (D-Cinema Quality: Reference projector and environment).
 Describes the reference projector and environment conditions that provide a consistent film presentation experience.
  |
| `12` |
 SMPTE EG 432 (Digital Source Processing: Color Processing for D-Cinema).
 Engineering guideline making color signal decoding recommendations for digital movies.
  |
| `13` – `21` | *Reserved for future use by ITU-T or ISO/IEC* |
| `22` | EBU Tech 3213-E |
| `23` – `255` | *Reserved for future use by ITU-T or ISO/IEC* |

 |
| `tc` |
 A two-digit integer indicating the
 `transferCharacteristics` for the video.
 This value is from Section 8.2 of ISO/IEC 23001-8:2016, and indicates the transfer characteristics to be used when adapting the decoded color to the render target.
  |
| `mc` |
 The two-digit value for the `matrixCoefficients` property.
 This value comes from the table in Section 8.3 of the ISO/IEC 23001-8:2016 specification.
 This value indicates which set of coefficients to use when mapping from the native red, blue, and green primaries to the luma and chroma signals.
 These coefficients are in turn used with the equations found in that same section.
  |
| `FF` |
 Indicates whether to restrict the black level and color range of each color component to the legal range.
 For 8-bit color samples, the legal range is 16-235.
 A value of `00` indicates that these limitations should be enforced, while a value of `01` allows the full range of possible values for each component, even if the resulting color is out of bounds for the color system.
  |

#### WebM media type examples

`video/webm;codecs="vp08.00.41.08,vorbis"`

VP8 video, profile 0 level 4.1, using 8-bit YUV with 4:2:0 chroma subsampling, using BT.709 color primaries, transfer function, and matrix coefficients, with the luminance and chroma values encoded within the legal ("studio") range. The audio is in Vorbis format.

`video/webm;codecs="vp09.02.10.10.01.09.16.09.01,opus"`

VP9 video, profile 2 level 1.0, with 10-bit YUV content using 4:2:0 chroma subsampling, BT.2020 primaries, ST 2084 EOTF (HDR SMPTE), BT.2020 non-constant luminance color matrix, and full-range chroma and luma encoding. The audio is in Opus format.

Using the codecs parameter
--------------------------

You can use the `codecs` parameter in a few situations. Firstly, you can use it with the `<source>` element when creating an `<audio>` or `<video>` element, in order to establish a group of options for the browser to choose from when selecting the format of the media to present to the user in the element.

You can also use the codecs parameter when specifying a MIME media type to the `MediaSource.isTypeSupported()` method; this method returns a Boolean which indicates whether or not the media is likely to work on the current device.

See also
--------

* Web media technologies
* The `<source>` element, child of the `<audio>` and `<video>` elements
* Guide to media types and formats on the web
* Guide to audio codecs used on the web
* Guide to video codecs used on the web
* Codecs used by WebRTC
* Getting the correct HTML codecs parameter for an AV1 video


Handling media support issues in web content - Web media technologies

Handling media support issues in web content
============================================

One of the realities of working with audio and video presentation and manipulation on the web is that there are a number of media formats available, of varying degrees of popularity and with a variety of capabilities. The availability of choices is good for the user, in that they can choose the format that suits their needs best. There is a drawback, however: because there are so many to choose from, with so many different kinds of licenses and design principles involved, each web browser developer is left to its own devices when deciding which media file types and codecs to support.

This places a small, but reasonably easily overcome, burden on the web developer: to properly handle the situation when the user's browser can't handle a particular type of media. This guide covers techniques you can use to develop web content that meets your media needs while providing the most broadly compatible experience possible. Topics we will examine fallbacks, baseline media formats, and error handling practices that will let your content work in as many situations as possible.

Using poster frames
-------------------

A **poster frame** is a still image that's representative of the content of a video. This may be the first frame of video; however, in many cases, the first frame is blank, or contains nothing but the logo of a business, or some other image that doesn't give the reader any context for the video's contents.

A good poster frame is one which is either a representative of the video's contents, or is an image which isn't even from the video itself but contains imagery and/or text which gives the reader a useful idea of the video's contents. In the case of an action movie, for example, the poster frame might be an iconic exemplar image from one of the film's best-known scenes.

A similar concept can be applied to still images; if an image you wish to present is very large and may take time to download (especially for slower devices or connections), you can offer a lower-resolution or alternate version that will be displayed until the full-quality version is available to be displayed.

We'll look at both of these scenarios and how to implement them.

### Poster frames for video

### Progressive images

Images—whether embedded using `<img>` or `<picture>`—don't support a concept similar to poster frames. However, there are ways to present an image in low quality while it's still loading. This requires creating your images using **progressive** formats, such as progressive JPEG or interlaced PNG.

Once your image has been converted to progressive form, you can use it as usual.

```html
<img
 src="/images/staff-photo-huge-progressive.jpg"
 alt="Staff Photo, taken in January of 1972" />
```

When using a progressive image, the data is stored in such a way that the browser is able to render a low-quality representation of the image as soon as possible, then update the image as it loads—or after it's finished loading—to present it in full quality.

**Note:** Progressive (or interlaced) images are inherently slightly larger than the non-progressive versions of the same images. Whether the interlacing will benefit your users is up to you to determine.

Specifying multiple sources
---------------------------

Checking compatibility in JavaScript
------------------------------------

`HTMLMediaElement.canPlayType` and `MediaSource.isTypeSupported()`…

Detecting playback errors
-------------------------

Adapting presentation with CSS
------------------------------

Memory management
-----------------