Audio and Video Delivery - Web media technologies

Audio and Video Delivery
========================

We can deliver audio and video on the web in a number of ways, ranging from 'static' media files to adaptive live streams. This article is intended as a starting point for exploring the various delivery mechanisms of web-based media and compatibility with popular browsers.

The Audio and Video Elements
----------------------------

Whether we are dealing with pre-recorded audio files or live streams, the mechanism for making them available through the browser's `<audio>` and `<video>` elements remains pretty much the same. Currently, to support all browsers we need to specify two formats, although with the adoption of MP3 and MP4 formats in Firefox and Opera, this is changing fast. You can find compatibility information in the Guide to media types and formats on the web.

To deliver video and audio, the general workflow is usually something like this:

1. Check what format the browser supports via feature detection (usually a choice of two, as stated above).
2. If the browser doesn't support playback of any of the provided formats natively, either present a still image or use a fallback technology to present the video.
3. Identify how you want to play/instantiate the media (e.g. a `<video>` element, or `document.createElement('video')` perhaps?)
4. Deliver the media file to the player.
### HTML Audio

```html
<audio controls preload="auto">
  <source src="audiofile.mp3" type="audio/mpeg" />

  <!-- fallback for browsers that don't support mp3 -->
  <source src="audiofile.ogg" type="audio/ogg" />

  <!-- fallback for browsers that don't support audio tag -->
  <a href="audiofile.mp3">download audio</a>
</audio>
```

The code above will create an audio player that attempts to preload as much audio as possible for smooth playback.

**Note:** The `preload` attribute may be ignored by some mobile browsers.

For further info see Cross Browser Audio Basics (HTML Audio In Detail)

### HTML Video

```html
<video
 controls
 width="640"
 height="480"
 poster="initialimage.png"
 autoplay
 muted>
  <source src="videofile.mp4" type="video/mp4" />

  <!-- fallback for browsers that don't support mp4 -->
  <source src="videofile.webm" type="video/webm" />

  <!-- specifying subtitle files -->
  <track src="subtitles\_en.vtt" kind="subtitles" srclang="en" label="English" />
  <track
 src="subtitles\_no.vtt"
 kind="subtitles"
 srclang="no"
 label="Norwegian" />

  <!-- fallback for browsers that don't support video tag -->
  <a href="videofile.mp4">download video</a>
</video>
```

The code above creates a video player of dimensions 640x480 pixels, displaying a poster image until the video is played. We instruct the video to autoplay but to be muted by default.

**Note:** The `autoplay` attribute may be ignored by some mobile browsers. Also, the autoplay feature can be controversial when misused. It's strongly recommended that you read the Autoplay guide for media and Web Audio APIs to learn how to use autoplay wisely.

For further info see <video> element and Creating a cross-browser video player.

### JavaScript Audio

```js
const myAudio = document.createElement("audio");

if (myAudio.canPlayType("audio/mpeg")) {
  myAudio.setAttribute("src", "audiofile.mp3");
} else if (myAudio.canPlayType("audio/ogg")) {
  myAudio.setAttribute("src", "audiofile.ogg");
}

myAudio.currentTime = 5;
myAudio.play();
```

We set the source of the audio depending on the type of audio file the browser supports, then set the play-head 5 seconds in and attempt to play it.

**Note:** Play will be ignored by most browsers unless issued by a user-initiated event.

It's also possible to feed an `<audio>` element a base64 encoded WAV file, allowing to generate audio on the fly:

```html
<audio id="player" src="data:audio/x-wav;base64,UklGRvC…"></audio>
```

Speak.js employs this technique.

### JavaScript Video

```js
const myVideo = document.createElement("video");

if (myVideo.canPlayType("video/mp4")) {
  myVideo.setAttribute("src", "videofile.mp4");
} else if (myVideo.canPlayType("video/webm")) {
  myVideo.setAttribute("src", "videofile.webm");
}

myVideo.width = 480;
myVideo.height = 320;
```

We set the source of the video depending on the type of video file the browser supports we then set the width and height of the video.

Web Audio API
-------------

In this example we retrieve an MP3 file using the `fetch()` API, load it into a source, and play it.

```js
let audioCtx;
let buffer;
let source;

async function loadAudio() {
  try {
    // Load an audio file
    const response = await fetch("viper.mp3");
    // Decode it
    buffer = await audioCtx.decodeAudioData(await response.arrayBuffer());
  } catch (err) {
    console.error(`Unable to fetch the audio file. Error: ${err.message}`);
  }
}

const play = document.getElementById("play");
play.addEventListener("click", async () => {
  if (!audioCtx) {
    audioCtx = new AudioContext();
    await loadAudio();
  }
  source = audioCtx.createBufferSource();
  source.buffer = buffer;
  source.connect(audioCtx.destination);
  source.start();
  play.disabled = true;
});
```

You can run the full example live, or view the source.

Find more about Web Audio API basics in Using the Web Audio API.

getUserMedia / Stream API
-------------------------

It's also possible to retrieve a live stream from a webcam and/or microphone using `getUserMedia` and the Stream API. This makes up part of a wider technology known as WebRTC (Web Real-Time Communications) and is compatible with the latest versions of Chrome, Firefox and Opera.

To grab the stream from your webcam, first set up a `<video>` element:

```html
<video id="webcam" width="480" height="360"></video>
```

Next, if supported connect the webcam source to the video element:

```js
if (navigator.mediaDevices) {
  navigator.mediaDevices
    .getUserMedia({ video: true, audio: false })
    .then(function onSuccess(stream) {
      const video = document.getElementById("webcam");
      video.autoplay = true;
      video.srcObject = stream;
    })
    .catch(function onError() {
      alert(
        "There has been a problem retrieving the streams - are you running on file:/// or did you disallow access?",
      );
    });
} else {
  alert("getUserMedia is not supported in this browser.");
}
```

To find out more, read our `MediaDevices.getUserMedia` page.

Mediastream Recording
---------------------

New standards are being rolled out to allow your browser to grab media from your mic or camera using `getUserMedia` and record it instantly using the new MediaStream Recording API. You take the stream you receive from `getUserMedia`, pass it to a `MediaRecorder` object, take the resulting output and feed it to your audio or video source\*.

The main mechanism is outlined below:

```js
navigator.mediaDevices
  .getUserMedia({ audio: true })
  .then(function onSuccess(stream) {
    const recorder = new MediaRecorder(stream);

    const data = [];
    recorder.ondataavailable = (e) => {
      data.push(e.data);
    };
    recorder.start();
    recorder.onerror = (e) => {
      throw e.error || new Error(e.name); // e.name is FF non-spec
    };
    recorder.onstop = (e) => {
      const audio = document.createElement("audio");
      audio.src = window.URL.createObjectURL(new Blob(data));
    };
    setTimeout(() => {
      rec.stop();
    }, 5000);
  })
  .catch(function onError(error) {
    console.log(error.message);
  });
```

See MediaStream Recording API for more details.

Media Source Extensions (MSE)
-----------------------------

Media Source Extensions is a W3C working draft that plans to extend `HTMLMediaElement` to allow JavaScript to generate media streams for playback. Allowing JavaScript to generate streams facilitates a variety of use cases like adaptive streaming and time shifting live streams.

### Encrypted Media Extensions (EME)

Encrypted Media Extensions is a W3C proposal to extend `HTMLMediaElement`, providing APIs to control playback of protected content.

The API supports use cases ranging from simple clear key decryption to high value video (given an appropriate user agent implementation). License/key exchange is controlled by the application, facilitating the development of robust playback applications supporting a range of content decryption and protection technologies.

One of the principal uses of EME is to allow browsers to implement DRM (Digital Rights Management), which helps to prevent web-based content (especially video) from being copied.

### Adaptive Streaming

New formats and protocols are being rolled out to facilitate adaptive streaming. Adaptive streaming media means that the bandwidth and typically quality of the stream can change in real-time in reaction to the user's available bandwidth. Adaptive streaming is often used in conjunction with live streaming where smooth delivery of audio or video is paramount.

The main formats used for adaptive streaming are HLS and MPEG-DASH. MSE has been designed with DASH in mind. MSE defines byte streams according to ISOBMFF and M2TS (both supported in DASH, the latter supported in HLS). Generally speaking, if you are interested in standards, are looking for flexibility, or wish to support most modern browsers, you are probably better off with DASH.

**Note:** Currently Safari does not support DASH although dash.js will work on newer versions of Safari scheduled for release with OSX Yosemite.

DASH also provides a number of profiles including simple onDemand profiles that require no preprocessing and splitting up of media files. There are also a number of cloud based services that will convert your media to both HLS and DASH.

For further information see Live streaming web audio and video.

Customizing Your Media Player
-----------------------------

You may decide that you want your audio or video player to have a consistent look across browsers, or just wish to tweak it to match your site. The general technique for achieving this is to omit the `controls` attribute so that the default browser controls are not displayed, create custom controls using HTML and CSS, then use JavaScript to link your controls to the audio/video API.

If you need something extra, it's possible to add features that are not currently present in default players, such as playback rate, quality stream switches or even audio spectrums. You can also choose how to make your player responsive — for example you might remove the progress bar under certain conditions.

You may detect click, touch and/or keyboard events to trigger actions such as play, pause and scrubbing. It's often important to remember keyboard controls for user convenience and accessibility.

A quick example — first set up your audio and custom controls in HTML:

```html
<audio
 id="my-audio"
 src="http://jPlayer.org/audio/mp3/Miaow-01-Tempered-song.mp3"></audio>
<button id="my-control">play</button>
```

add a bit of JavaScript to detect events to play and pause the audio:

```js
window.onload = () => {
  const myAudio = document.getElementById("my-audio");
  const myControl = document.getElementById("my-control");

  function switchState() {
    if (myAudio.paused) {
      myAudio.play();
      myControl.textContent = "pause";
    } else {
      myAudio.pause();
      myControl.textContent = "play";
    }
  }

  function checkKey(e) {
    if (e.keycode === 32) {
      // space bar
      switchState();
    }
  }

  myControl.addEventListener(
    "click",
    () => {
      switchState();
    },
    false,
  );

  window.addEventListener("keypress", checkKey, false);
};
```

You can try this example out here. For more information, see Creating your own custom audio player.

Other tips for audio/video
--------------------------

### Stopping the download of media

While stopping the playback of media is as easy as calling the element's `pause()` method, the browser keeps downloading the media until the media element is disposed of through garbage collection.

Here's a trick that stops the download at once:

```js
const mediaElement = document.querySelector("#myMediaElementID");
mediaElement.removeAttribute("src");
mediaElement.load();
```

By removing the media element's `src` attribute and invoking the load() method, you release the resources associated with the video, which stops the network download. You must call `load()` after removing the attribute, because just removing the `src` attribute does not invoke the load algorithm. If the `<video>` element also has `<source>` element descendants, those should also be removed before calling `load()`.

Note that just setting the `src` attribute to an empty string will actually cause the browser to treat it as though you're setting a video source to a relative path. This causes the browser to attempt another download to something that is unlikely to be a valid video.

### Seeking through media

Media elements provide support for moving the current playback position to specific points in the media's content. This is done by setting the value of the `currentTime` property on the element; see `HTMLMediaElement` for further details on the element's properties. Set the value to the time, in seconds, at which you want playback to continue.

You can use the element's `seekable` property to determine the ranges of the media that are currently available for seeking to. This returns a `TimeRanges` object listing the ranges of times that you can seek to.

```js
const mediaElement = document.querySelector("#mediaElementID");
mediaElement.seekable.start(0); // Returns the starting time (in seconds)
mediaElement.seekable.end(0); // Returns the ending time (in seconds)
mediaElement.currentTime = 122; // Seek to 122 seconds
mediaElement.played.end(0); // Returns the number of seconds the browser has played
```

### Specifying playback range

When specifying the URI of media for an `<audio>` or `<video>` element, you can optionally include additional information to specify the portion of the media to play. To do this, append a hash mark ("#") followed by the media fragment description.

A time range is specified using the syntax:
```

#t=[starttime][,endtime]
```

The time can be specified as a number of seconds (as a floating-point value) or as an hours/minutes/seconds time separated with colons (such as 2:05:01 for 2 hours, 5 minutes, and 1 second).

A few examples:

`http://example.com/video.ogv#t=10,20`

Specifies that the video should play the range 10 seconds through 20 seconds.

`http://example.com/video.ogv#t=,10.5`

Specifies that the video should play from the beginning through 10.5 seconds.

`http://example.com/video.ogv#t=,02:00:00`

Specifies that the video should play from the beginning through two hours.

`http://example.com/video.ogv#t=60`

Specifies that the video should start playing at 60 seconds and play through the end of the video.

Error handling
--------------

Errors get delivered to the child `<source>` elements corresponding to the sources resulting in the error.

This lets you detect which sources failed to load, which may be useful. Consider this HTML:

```html
<video>
<source id="mp4\_src"
 src="video.mp4"
 type='video/mp4; codecs="avc1.42E01E, mp4a.40.2"'>
</source>
<source id="3gp\_src"
 src="video.3gp"
 type='video/3gpp; codecs="mp4v.20.8, samr"'>
</source>
<source id="ogg\_src"
 src="video.ogv"
 type='video/ogv; codecs="theora, vorbis"'>
</source>
</video>
```

Since Firefox doesn't support MP4 and 3GP on some platforms due to their patent-encumbered nature, the `<source>` elements with the IDs "mp4\_src" and "3gp\_src" will receive `error` events before the Ogg resource is loaded. The sources are tried in the order in which they appear, and once one loads successfully, the remaining sources aren't tried at all.

### Checking whether the browser supports the supplied formats

Use the following verified sources within your audio and video elements to check support.

* Audio MP3 (`type="audio/mpeg"`): http://jPlayer.org/audio/mp3/Miaow-01-Tempered-song.mp3 (play the MP3 audio live.)
* Audio MP4 (`type="audio/mp4"`): http://jPlayer.org/audio/m4a/Miaow-01-Tempered-song.m4a (play the MP4 audio live.)
* Audio Ogg (`type="audio/ogg"`): http://jPlayer.org/audio/ogg/Miaow-01-Tempered-song.ogg (play the OGG audio live.)
* Video MP4 (`type="video/mp4"`): http://jPlayer.org/video/m4v/Big\_Buck\_Bunny\_Trailer.m4v (play the MP4 video live.)
* Video WebM (`type="video/webm"`): http://jPlayer.org/video/webm/Big\_Buck\_Bunny\_Trailer.webm (play the WebM video live.)
* Video Ogg (`type="video/ogg"`): http://jPlayer.org/video/ogv/Big\_Buck\_Bunny\_Trailer.ogv (play the OGG video live.)

If these don't play then the browser you are testing doesn't support the given format. Consider using a different format or using a fallback.

If these work but the files you are supplying don't, there are two possible issues:

#### 1. The media server is not delivering the correct mime types with the file

Although this is usually supported, you may need to add the following to your media server's `.htaccess` file.
```

# AddType TYPE/SUBTYPE EXTENSION

AddType audio/mpeg mp3
AddType audio/mp4 m4a
AddType audio/ogg ogg
AddType audio/ogg oga

AddType video/mp4 mp4
AddType video/mp4 m4v
AddType video/ogg ogv
AddType video/webm webm
AddType video/webm webmv
```

#### 2. Your files have been encoded incorrectly

Your files may have been encoded incorrectly — try encoding using one of the following tools, which are proven to be pretty reliable:

* Audacity — Free Audio Editor and Recorder
* Miro — Free, open-source music and video player
* Handbrake — Open Source Video Transcoder
* Firefogg — Video and Audio encoding for Firefox
* FFmpeg2 — Comprehensive command line encoder
* Libav — Comprehensive command line encoder
* Vid.ly — Video player, transcoding and delivery
* Internet Archive — Free transcoding and storage
### Detecting when no sources have loaded

To detect that all child `<source>` elements have failed to load, check the value of the media element's `networkState` attribute. If this is `HTMLMediaElement.NETWORK_NO_SOURCE`, you know that all the sources failed to load.

If at that point you add another source, by inserting a new `<source>` element as a child of the media element, Gecko attempts to load the specified resource.

### Showing fallback content when no source could be decoded

Another way to show the fallback content of a video, when none of the sources could be decoded in the current browser, is to add an error handler on the last source element. Then you can replace the video with its fallback content:

```html
<video controls>
  <source src="dynamicsearch.mp4" type="video/mp4"></source>
  <a href="dynamicsearch.mp4">
    <img src="dynamicsearch.jpg" alt="Dynamic app search in Firefox OS">
  </a>
  <p>Click image to play a video demo of dynamic app search</p>
</video>
```

```js
const v = document.querySelector("video");
const sources = v.querySelectorAll("source");
const lastsource = sources[sources.length - 1];
lastsource.addEventListener(
  "error",
  (ev) => {
    const d = document.createElement("div");
    d.innerHTML = v.innerHTML;
    v.parentNode.replaceChild(d, v);
  },
  false,
);
```

Audio/Video JavaScript Libraries
--------------------------------

A number of audio and video JavaScript libraries exist. The most popular libraries allow you to choose a consistent player design over all browsers and provide a fallback for browsers that don't support audio and video natively. Fallbacks have historically used now-obsolete plugins such as Adobe Flash or Microsoft Silverlight plugins to provide a media player in non-supporting browsers, although these are no longer supported on modern computers. Other functionality such as the `<track>` element for subtitles can also be provided through media libraries.

### Audio only

* SoundManager
* AmplitudeJS
* HowlerJS
### Video only

* flowplayer: Gratis with a flowplayer logo watermark. Open source (GPL licensed.)
* JWPlayer: Requires registration to download. Open Source Edition (Creative Commons License.)
* SublimeVideo: Requires registration. Form based set up with domain specific link to CDN hosted library.
* Video.js: Gratis and Open Source (Apache 2 Licensed.)
### Audio and Video

* jPlayer: Gratis and Open Source (MIT Licensed.)
* mediaelement.js: Gratis and Open Source (MIT Licensed.)
### Web Audio API

* AudioContext monkeypatch: A polyfill for older versions of the Web Audio API; Open Source (Apache 2 Licensed.)
Basic tutorials
---------------

Creating a cross-browser video player

A guide to creating a basic cross browser video player using the `<video>` element.

Video player styling basics

With the cross-browser video player put in place in the previous article, this article now looks at providing some basic, responsive styling for the player.

Cross-browser audio basics

This article provides a basic guide to creating an HTML audio player that works cross browser, with all the associated attributes, properties and events explained, and a quick guide to custom controls created using the Media API.

Media buffering, seeking, and time ranges

Sometimes it's useful to know how much `<audio>` or `<video>` has downloaded or is playable without delay — a good example of this is the buffered progress bar of an audio or video player. This article discusses how to build a buffer/seek bar using TimeRanges, and other features of the media API.

HTML playbackRate explained

The `playbackRate` property allows us to change the speed or rate at which a piece of web audio or video is playing. This article explains it in detail.

Using the Web Audio API

Explains the basics of using the Web Audio API to grab, manipulate and play back an audio source.

Streaming media tutorials
-------------------------

Live streaming web audio and video

Live streaming technology is often employed to relay live events such as sports, concerts and more generally TV and Radio programmes that are output live. Often shortened to just streaming, live streaming is the process of transmitting media 'live' to computers and devices. This is a fairly complex and nascent subject with a lot of variables, so in this article we'll introduce you to the subject and let you know how you can get started.

Setting up adaptive streaming media sources

Let's say you want to set up an adaptive streaming media source on a server, to be consumed inside an HTML media element. How would you do that? This article explains how, looking at two of the most common formats: MPEG-DASH and HLS (HTTP Live Streaming.)

DASH Adaptive Streaming for HTML 5 Video

Details how to set up adaptive streaming using DASH and WebM.

Advanced tutorials
------------------

* Adding captions and subtitles to HTML video
    + : This article explains how to add captions and subtitles to HTML `<video>`, using Web\_Video\_Text\_Tracks\_Format and the `<track>` element.
    + : A guide to writing cross browser Web Audio API code.
* Easy audio capture with the MediaRecorder API
    + : Explains the basics of using the MediaStream Recording API to directly record a media stream.

**Note:** Firefox OS versions 1.3 and above support the RTSP protocol for streaming video delivery. A fallback solution for older versions would be to use `<video>` along with a suitable format for Gecko (such as WebM) to serve fallback content. More information will be published on this in good time.

References
----------

* The video element
* HTMLVideoElement API
* MediaSource API
* Web Audio API
* MediaStream Recording API
* getUserMedia
* Event reference > Media


Using images in HTML - Web media technologies

Using images in HTML
====================

The HTML `<img>` element lets you embed images into an HTML document, while the `<picture>` element enables responsive images. In this guide you'll find links to resources that deal with adding images to websites.

References
----------

These articles cover some of the HTML elements and CSS properties that are used to control how images are displayed on the web.

### HTML

`<img>`

The **HTML `<img>` element** is used to embed an image in a webpage. It supports a wide range of attributes that control how the image behaves and allows you to add important information like `alt` text for people who don't see the image.

`<picture>`

The **HTML `<picture>` element** contains zero or more `<source>` elements and one `<img>` element which provides versions of an image for different display/device scenarios. The browser considers each child `<source>` element and chooses the best match among them.

### CSS

`object-fit`

The **`object-fit`** CSS property sets how the content of a replaced element, such as an `<img>` or `<video>`, should be resized to fit its container.

`object-position`

The **`object-position`** CSS property specifies the alignment of content in a selected replaced element within the element's box. Areas of the box which aren't covered by the replaced element's object will show the element's background.

`background-image`

The **`background-image`** CSS property sets one or more background images for an element.

Guides
------

These articles provide guidance on selecting and configuring image types.

Image file type and format guide

A guide to the various image file types commonly supported by web browsers including details about their individual use cases, capabilities, and compatibility factors. Additionally, this article provides guidance for selecting the best image file type for a given situation.

Including `width` and `height` attributes helps avoid jank

This changes how browsers load images so their aspect ratios are calculated by the browser early on and can later be used to reserve the display size of an image before it's loaded.

Other topics
------------

Related topics that may be of interest.

Learn HTML: Responsive Images

In this article, we'll learn about the concept of responsive images — images that work well on devices with widely differing screen sizes, resolutions, and other such features — and look at what tools HTML provides to help implement them.


DASH Adaptive Streaming for HTML 5 Video - Web media technologies

DASH Adaptive Streaming for HTML 5 Video
========================================

Dynamic Adaptive Streaming over HTTP (DASH) is an adaptive streaming protocol. This means that it allows for a video stream to switch between bit rates on the basis of network performance, in order to keep a video playing.

Browser Support
---------------

Firefox 21 includes an implementation of DASH for HTML WebM video which is turned off by default. It can be enabled via "about:config" and the "media.dash.enabled" preference.

Firefox 23 removed support for DASH for HTML WebM video. It will be replaced by an implementation of the Media Source Extensions API which will allow support for DASH via JavaScript libraries such as dash.js. See bug 778617 for details.

Using DASH - Server Side
------------------------

First you'll need to convert your WebM video to a DASH manifest with the accompanying video files in various bit rates. To start with you'll only need the ffpmeg program from ffmpeg.org, with libvpx and libvorbis support for WebM video and audio, at least version 2.5 (probably; this was tested with 3.2.5).

### 1. Use your existing WebM file to create one audio file and multiple video files

For example:

The file ***in.video*** can be any container with at least one audio and one video stream that can be decoded by ffmpeg,

Create the audio using:

```bash
ffmpeg -i in.video -vn -acodec libvorbis -ab 128k -dash 1 my_audio.webm
```

Create each video variant.

```bash
ffmpeg -i in.video -c:v libvpx-vp9 -keyint\_min 150 -g 150 -tile-columns 4 -frame-parallel 1 -f webm -dash 1 \
-an -vf scale=160:90 -b:v 250k -dash 1 video_160x90_250k.webm
```

```bash
ffmpeg -i in.video -c:v libvpx-vp9 -keyint\_min 150 -g 150 -tile-columns 4 -frame-parallel 1  -f webm -dash 1 \
-an -vf scale=320:180 -b:v 500k -dash 1 video_320x180_500k.webm
```

```bash
ffmpeg -i in.video -c:v libvpx-vp9 -keyint\_min 150 -g 150 -tile-columns 4 -frame-parallel 1  -f webm -dash 1 \
-an -vf scale=640:360 -b:v 750k -dash 1 video_640x360_750k.webm
```

```bash
ffmpeg -i in.video -c:v libvpx-vp9 -keyint\_min 150 -g 150 -tile-columns 4 -frame-parallel 1  -f webm -dash 1 \
-an -vf scale=640:360 -b:v 1000k -dash 1 video_640x360_1000k.webm
```

```bash
ffmpeg -i in.video -c:v libvpx-vp9 -keyint\_min 150 -g 150 -tile-columns 4 -frame-parallel 1  -f webm -dash 1 \
-an -vf scale=1280:720 -b:v 1500k -dash 1 video_1280x720_1500k.webm
```

Or do it in all in one command.

```bash
ffmpeg -i in.video -c:v libvpx-vp9 -keyint\_min 150 \
-g 150 -tile-columns 4 -frame-parallel 1 -f webm -dash 1 \
-an -vf scale=160:90 -b:v 250k -dash 1 video_160x90_250k.webm \
-an -vf scale=320:180 -b:v 500k -dash 1 video_320x180_500k.webm \
-an -vf scale=640:360 -b:v 750k -dash 1 video_640x360_750k.webm \
-an -vf scale=640:360 -b:v 1000k -dash 1 video_640x360_1000k.webm \
-an -vf scale=1280:720 -b:v 1500k -dash 1 video_1280x720_1500k.webm
```

### 2. Create the manifest file

```bash
ffmpeg \
  -f webm_dash_manifest -i video_160x90_250k.webm \
  -f webm_dash_manifest -i video_320x180_500k.webm \
  -f webm_dash_manifest -i video_640x360_750k.webm \
  -f webm_dash_manifest -i video_1280x720_1500k.webm \
  -f webm_dash_manifest -i my_audio.webm \
  -c copy \
  -map 0 -map 1 -map 2 -map 3 -map 4 \
  -f webm_dash_manifest \
  -adaptation\_sets "id=0,streams=0,1,2,3 id=1,streams=4" \
  my_video_manifest.mpd
```

The `-map` arguments correspond to the input files in the sequence they are given; you should have one for each file. The `-adaptation_sets` argument assigns them into adaptation sets; for example, this creates one set (0) that contains the streams 0, 1, 2 and 3 (the videos), and another set (1) that contains only stream 4, the audio stream.

Put the manifest and the associated video files on your web server or CDN. DASH works via HTTP, so as long as your HTTP server supports byte range requests, and it's set up to serve `.mpd` files with `mimetype="application/dash+xml"`, then you're all set.

Using DASH - Client Side
------------------------

You'll want to modify your web page to point to the DASH manifest first, instead of directly to a particular video file:

```html
<video>
  <source src="movie.mpd" />
  <source src="movie.webm" />
  Your browser does not support the video tag.
</video>
```

That's it! If DASH is supported by the browser, your video will now stream adaptively.

Links
-----

WebM DASH Specification at The WebM Project

DASH Industry Forum

WebM project description of how to create DASH files with FFMPEG


Autoplay guide for media and Web Audio APIs - Web media technologies

Autoplay guide for media and Web Audio APIs
===========================================

Automatically starting the playback of audio (or videos with audio tracks) immediately upon page load can be an unwelcome surprise to users. While autoplay of media serves a useful purpose, it should be used carefully and only when needed. In order to give users control over this, browsers often provide various forms of autoplay blocking. In this guide, we'll cover autoplay functionality in the various media and Web Audio APIs, including a brief overview of how to use autoplay and how to work with browsers to handle autoplay blocking gracefully.

Autoplay blocking is *not* applied to `<video>` elements when the source media does not have an audio track, or if the audio track is muted. Media with an active audio track are considered to be **audible**, and autoplay blocking applies to them. **Inaudible** media are not affected by autoplay blocking.

Autoplay and autoplay blocking
------------------------------

The term **autoplay** refers to any feature that causes media to begin to play without the user specifically requesting that playback begin. This includes both the use of HTML attributes to autoplay media as well as the use of JavaScript code to start playback outside the context of handling user input.

That means that both of the following are considered autoplay behavior, and are therefore subject to the browser's autoplay blocking policy:

```html
<audio src="/music.mp3" autoplay></audio>
```

and

```js
audioElement.play();
```

The following web features and APIs may be affected by autoplay blocking:

* The HTML `<audio>` and `<video>` elements
* The Web Audio API

From the user's perspective, a web page or app that spontaneously starts making noise without warning can be jarring, inconvenient, or off-putting. Because of that, browsers generally only allow autoplay to occur successfully under specific circumstances.

### Autoplay availability

As a general rule, you can assume that media will be allowed to autoplay only if *at least one* of the following is true:

* The audio is muted or its volume is set to 0
* The user has interacted with the site (by clicking, tapping, pressing keys, etc.)
* If the site has been allowlisted; this may happen either automatically if the browser determines that the user engages with media frequently, or manually through preferences or other user interface features
* If the autoplay Permissions Policy is used to grant autoplay support to an `<iframe>` and its document.

 Otherwise, the playback will likely be blocked.
 The exact situations that result in blocking, and the specifics of how sites become allowlisted, vary from browser to browser, but the above are good guidelines to go by.

For details, see the autoplay policies for Google Chrome and WebKit.

**Note:** Put another way, playback of any media that includes audio is generally blocked if the playback is programmatically initiated in a tab which has not yet had any user interaction. Browsers may additionally choose to block under other circumstances.

Autoplay of media elements
--------------------------

Now that we've covered what autoplay is and what can prevent autoplay from being allowed, we'll look at how your website or app can automatically play media upon page load, how to detect when autoplay fails to occur, and tips for coping when autoplay is denied by the browser.

### The autoplay attribute

 The simplest way to automatically play content is to add the `autoplay` attribute to your `<audio>` or `<video>` element, which sets the `autoplay` property on the element to `true`.
 When `autoplay` is `true`, the media will automatically begin to play as soon as possible after the following have occurred:

* The page is allowed to use autoplay functionality
* The element has been created during page load
* Enough media has been received to begin playback and continue to play through to the end of the media without interruption, assuming there are no dramatic changes in network performance or bandwidth.

#### Example: The autoplay attribute

An `<audio>` element using the `autoplay` attribute might look like this:

```html
<audio id="musicplayer" autoplay>
  <source src="/music/chapter1.mp3" />
</audio>
```

#### Example 2: Detecting whether autoplay is allowed

 If autoplay is important for your application, you may need to customize behavior based on whether or not autoplay is allowed, disallowed, or only supported for inaudible content.
 For example, if your application needs to autoplay a video and you know that the page only allows the autoplay of inaudible content, you can either mute it or supply a video with no audio track.
 Similarly, if you know that autoplay is not allowed at all, you might provide a default image for the video (using the `poster` attribute), or choose to defer loading the video until it is requested.

The `Navigator.getAutoplayPolicy()` method can be used to check the autoplay policy for a type of media feature (i.e. all media elements, or all audio contexts) in a document, or to check whether a specific media element or audio context can autoplay.

 The example below shows how you pass the `mediaelement` string to get the autoplay policy for all media elements in the document (pass `audiocontext` to get the policy for audio contexts).
 The code assumes `video` is an `HTMLVideoElement` media element using the `<video>` tag or `HTMLVideoElement`, and that it is configured to autoplay with audio by default.
 If autoplay is only allowed for inaudible content, we mute the audio; if autoplay is disallowed, we make sure that a placeholder image is displayed for the video.

```js
if (navigator.getAutoplayPolicy("mediaelement") === "allowed") {
  // The video element will autoplay with audio.
} else if (navigator.getAutoplayPolicy("mediaelement") === "allowed-muted") {
  // Mute audio on video
  video.muted = true;
} else if (navigator.getAutoplayPolicy("mediaelement") === "disallowed") {
  // Set a default placeholder image.
  video.poster = "http://example.com/poster\_image\_url";
}
```

 The code to test a specific element or audio context is the same, except that you pass in the element or context to test rather than the type string.
 Here we pass in the `video` object we want to test.

```js
if (navigator.getAutoplayPolicy(video) === "allowed") {
  // The video element will autoplay with audio.
} else if (navigator.getAutoplayPolicy(video) === "allowed-muted") {
  // Mute audio on video
  video.muted = true;
} else if (navigator.getAutoplayPolicy(video) === "disallowed") {
  // Set a default placeholder image.
  video.poster = "http://example.com/poster\_image\_url";
}
```

 The autoplay policy for a type may change due to user interaction with the site, page, or a particular element.
 Similarly, on some browsers the policy for a specific element might change even though the policy for the type has not (for example, on browsers where touching a particular element can allow just that element to autoplay).

As there is no way to be notified when the autoplay policy has changed (either for a type or element), generally we recommend that the policy is checked when the page is loaded, using the type.

#### Example 3: Detecting autoplay failure as a fallback

No specific event (or other notification) is triggered by autoplay success or failure, so browsers that do not support `Navigator.getAutoplayPolicy()` have no easy way to determine if autoplay is supported, or to react when it is triggered or not triggered.

 One approach is to listen for the first instance of the `play` event, which is fired on the media element when is resumed after being paused *and* when autoplay occurs.
 That means that the first time the `play` event is fired, you know your media is being started for the first time after the page is opened,

Consider this HTML for a media element:

```html
<video src="myvideo.mp4" id="video" autoplay></video>
```

Here we have a `<video>` element whose `autoplay` attribute is set and with a `play` event handler set up; the event is handled by a function called `handleFirstPlay()`, which receives as input the `play` event.

`handleFirstPlay()` looks like this:

```js
const video = document.getElementById("video");
video.addEventListener("play", handleFirstPlay, false);

let hasPlayed = false;
function handleFirstPlay(event) {
  if (!hasPlayed) {
    hasPlayed = true;

    // Remove listener so this only gets called once.
    const vid = event.target;
    vid.removeEventListener("play", handleFirstPlay);

    // Start whatever you need to do after first playback has started
  }
}
```

 After getting a reference to the video element from the `Event` object's `target`, we use it to remove the event listener.
 This will prevent any future `play` events from being delivered to the handler. That could happen if the video is paused and resumed by the user or automatically by the browser when the document is in a background tab.

At this point, your site or app can begin whatever it needs to do that relies upon the video having been started up.

### The play() method

The term "autoplay" also refers to scenarios in which a script tries to trigger the playback of media that includes audio, outside the context of handling a user input event. This is done by calling the media element's `play()` method.

**Note:** It is strongly recommended that you use the `autoplay` attribute whenever possible, because support for autoplay preferences are more widespread for the `autoplay` attribute than for other means of playing media automatically. It also lets the browser take responsibility for starting playback, letting it optimize the timing of that taking place.

#### Example: Playing video

This simple example plays the first `<video>` element found in the document. `play()` won't let the playback begin unless the document has permission to automatically play media.

```js
document.querySelector("video").play();
```

#### Example: Handling play() failures

It's much easier to detect a failure to autoplay media when you use the `play()` method to start it. `play()` returns a `Promise` which is resolved once the media successfully begins to play, and is rejected when playback fails to begin (such as if autoplay is denied). When autoplay fails, you likely will want to offer a way for the user to manually tell the browser to ask the user to grant permission to play media.

You might use code like this to accomplish the job:

```js
let startPlayPromise = videoElem.play();

if (startPlayPromise !== undefined) {
  startPlayPromise
    .then(() => {
      // Start whatever you need to do only after playback
      // has begun.
    })
    .catch((error) => {
      if (error.name === "NotAllowedError") {
        showPlayButton(videoElem);
      } else {
        // Handle a load or playback error
      }
    });
}
```

The first thing we do with the result of `play()` is make sure it's not `undefined`. We check for this because in earlier versions of the HTML specification, `play()` didn't return a value. Returning a promise to allow you to determine success or failure of the operation was added more recently. Checking for `undefined` prevents this code from failing with an error on older versions of web browsers.

If the promise returned by `play()` is resolved without error, the `then()` clause is run and can begin whatever needs to be done when autoplay has begun.

We then add a `catch()` handler to the promise. This looks at the error's `name` to see if it's `NotAllowedError`. This indicates that playback failed due to a permission issue, such as autoplay being denied. If that's the case, we should present a user interface to let the user manually start playback; that's handled here by a function `showPlayButton()`.

Any other errors are handled as appropriate.

If you want to start playing the video after the first interaction with the page, `setInterval()` might be used to achieve this:

```js
let playAttempt = setInterval(() => {
  videoElem
    .play()
    .then(() => {
      clearInterval(playAttempt);
    })
    .catch((error) => {
      console.log("Unable to play the video, User has not interacted yet.");
    });
}, 3000);
```

Autoplay using the Web Audio API
--------------------------------

In the Web Audio API, a website or app can start playing audio using the `start()` method on a source node linked to the `AudioContext`. Doing so outside the context of handling a user input event is subject to autoplay rules.

The autoplay Permissions Policy
-------------------------------

In addition to the browser-side management and control over autoplay functionality described above, a web server can also express its willingness to allow autoplay to function. The HTTP `Permissions-Policy` header's `autoplay` directive is used to control which domains, if any, can be used to autoplay media. By default, the `autoplay` Permissions Policy is set to `self`, indicating that autoplay is permitted as they're hosted on the same domain as the document.

You can also specify an empty allowlist (`()`) to disable autoplay entirely, `*` to allow autoplay from all domains, or one or more specific origins from which media can be automatically played. These origins are separated by space characters.

**Note:** The specified Permissions Policy applies to the document and every `<iframe>` nested within it, unless those frames include an `allow`, which sets a new Permissions Policy for that frame and all frames nested within it.

When using the `allow` attribute on an `<iframe>` to specify a Permissions Policy for that frame and its nested frames, you can also specify the value `'src'` to allow autoplay of media only from the same domain as that specified by the frame's `src` attribute.

### Example: Allowing autoplay only from the document's domain

To use the `Permissions-Policy` header to only allow media to autoplay from the document's origin:

```http
Permissions-Policy: autoplay=(self)
```

To do the same for an `<iframe>`:

```html
<iframe src="mediaplayer.html" allow="autoplay"> </iframe>
```

### Example: Allowing autoplay and fullscreen mode

Adding Fullscreen API permission to the previous example results in a `Permissions-Policy` header like the following if fullscreen access is allowed regardless of the domain; a domain restriction can be added as well as needed.

```http
Permissions-Policy: autoplay=(self), fullscreen=(self)
```

The same permissions, granted using the `<iframe>` element's `allow` property, look like this:

```html
<iframe src="mediaplayer.html" allow="autoplay; fullscreen"> </iframe>
```

### Example: Allowing autoplay from specific sources

The `Permissions-Policy` header to allow media to be played from both the document's (or `<iframe>`'s) own domain and `https://example.media` looks like this:

```http
Permissions-Policy: autoplay=(self "https://example.media")
```

An `<iframe>` can be written to specify that this autoplay policy should be applied to itself and any child frames would be written thusly:

```html
<iframe
 width="300"
 height="200"
 src="mediaplayer.html"
 allow="autoplay 'src' https://example.media">
</iframe>
```

### Example: Disabling autoplay

Setting the `autoplay` Permissions Policy to `()`/`none` disables autoplay entirely for the document or `<iframe>` and all nested frames. The HTTP header is:

```http
Permissions-Policy: autoplay=()
```

Using the `<iframe>`'s `allow` attribute:

```html
<iframe src="mediaplayer.html" allow="autoplay 'none'"> </iframe>
```

Best practices
--------------

Tips and recommended best practices to help you make the most of working with autoplay are offered here.

### Handling autoplay failure with media controls

A common use case for autoplay is to automatically begin to play a video clip that goes along with an article, an advertisement, or a preview of the page's main functionality. To autoplay videos like these, you have two options: don't have an audio track, or have an audio track but configure the `<video>` element to mute the audio by default, like this:

```html
<video
 src="/videos/awesomevid.webm"
 controls
 autoplay
 playsinline
 muted></video>
```

This video element is configured to include the user controls (typically play/pause, scrubbing through the video's timeline, volume control, and muting); also, since the `muted` attribute is included, and the `playsinline` attribute that is required for autoplay in Safari, the video will autoplay but with the audio muted. The user has the option, however, of re-enabling the audio by clicking on the unmute button in the controls.

Browser configuration options
-----------------------------

Browsers may have preferences that control the way autoplay works, or how autoplay blocking is handled. Here, any such preferences that may be of special significance or importance to you as a web developer are listed. These include any that may aid in testing or debugging as well as any that could be set in a way that you need to be prepared to handle.

### Firefox

`media.allowed-to-play.enabled`

A Boolean preference which specifies whether the `HTMLMediaElement.allowedToPlay` property is exposed to the web. This is currently `false` by default (except in nightly builds, where it's `true` by default). If this is `false`, the `allowedToPlay` property is missing from the `HTMLMediaElement` interface, and is thus not present on either `<audio>` or `<video>` elements.

`media.autoplay.allow-extension-background-pages`

This Boolean preference, if `true`, allows browser extensions' background scripts to autoplay audio media. Setting this value to `false` disables this capability. The default value is `true`.

`media.autoplay.allow-muted`

A Boolean preference which if `true` (the default) allows audio media which is currently muted to be automatically played. If this has been changed to `false`, media with an audio track will not be permitted to play even if muted.

`media.autoplay.block-webaudio`

 A Boolean preference that indicates whether to apply autoplay blocking to the Web Audio API.
 If `false`, web audio is always allowed to autoplay.
 If `true`, audio contexts are only able to play on pages once there has been Sticky activation.
 The default is set to `true`.

`media.autoplay.default`

An integer preference which specifies whether per-domain configuration for autoplay support by default is allowed (`0`), blocked (`1`), or prompt-on-use (`2`). The default value is `0`.

`media.autoplay.enabled.user-gestures-needed` (Nightly builds only)

A Boolean preference which controls whether detection of user gestures is allowed to override the setting of `media.autoplay.default`. If `media.autoplay.default` is *not* set to `0` (autoplay allowed by default), this preference being `true` allows autoplay of media with audio tracks anyway if the page has been activated by user gestures, and media that isn't audible is not restricted at all.

`media.block-autoplay-until-in-foreground`

A Boolean preference which indicates whether media playback is blocked when started on a background tab. The default value, `true`, means that even when otherwise available, autoplay won't take place until after a tab is brought to the foreground. This prevents the distracting situation in which a tab begins playing sound and the user can't find the tab among all their tabs and windows.

See also
--------

* Web media technologies
* Video and audio content (Learning guide)
* Using the Web Audio API
* Cross-browser audio basics


Using audio and video in HTML - Web media technologies

Using audio and video in HTML
=============================

The HTML `<audio>` and `<video>` elements let you embed audio and video content into a web page. More than that, their underlying interfaces are the key to manipulating and playing back media in general, even offscreen. In this guide, we'll review the media elements and how to use them in HTML content.

**Note:** This guide is a planned update to integrate content from various scattered places on MDN into one cohesive document or document set. The work is not yet scheduled but hopefully will come soon. For now, some of the key information you may be looking for can be found in our Learning Area article on the topic.

We don't have a particularly good guide to using these objects offscreen at this time, although Audio and video manipulation may be a good start.

See also
--------

* Web media technologies


Audio and video manipulation - Web media technologies

Audio and video manipulation
============================

The beauty of the web is that you can combine technologies to create new forms. Having native audio and video in the browser means we can use these data streams with technologies such as `<canvas>`, WebGL or Web Audio API to modify audio and video directly, for example adding reverb/compression effects to audio, or grayscale/sepia filters to video. This article provides a reference to explain what you need to do.

Video manipulation
------------------

The ability to read the pixel values from each frame of a video can be very useful.

### Video and canvas

The `<canvas>` element provides a surface for drawing graphics onto web pages; it is very powerful and can be coupled tightly with video.

The general technique is to:

1. Write a frame from the `<video>` element to the `<canvas>` element.
2. Read the data from the `<canvas>` element and manipulate it.
3. Write the manipulated data to your "display" `<canvas>` (which effectively can be the same element).
4. Pause and repeat.

For example, let's process a video to display it in greyscale. In this case, we'll show both the source video and the output greyscale frames. Ordinarily, if you were implementing a "play video in greyscale" feature, you'd probably add `display: none` to the style for the `<video>` element, to keep the source video from being drawn to the screen while showing only the canvas showing the altered frames.

#### HTML

We can set up our video player and `<canvas>` element like this:

```html
<video id="my-video" controls width="480" height="270" crossorigin="anonymous">
  <source
 src="https://jplayer.org/video/webm/Big\_Buck\_Bunny\_Trailer.webm"
 type="video/webm" />
  <source
 src="https://jplayer.org/video/m4v/Big\_Buck\_Bunny\_Trailer.m4v"
 type="video/mp4" />
</video>

<canvas id="my-canvas" width="480" height="270"></canvas>
```

#### JavaScript

This code handles altering the frames.

```js
const processor = {
  timerCallback() {
    if (this.video.paused || this.video.ended) {
      return;
    }
    this.computeFrame();
    setTimeout(() => {
      this.timerCallback();
    }, 16); // roughly 60 frames per second
  },

  doLoad() {
    this.video = document.getElementById("my-video");
    this.c1 = document.getElementById("my-canvas");
    this.ctx1 = this.c1.getContext("2d");

    this.video.addEventListener(
      "play",
      () => {
        this.width = this.video.width;
        this.height = this.video.height;
        this.timerCallback();
      },
      false,
    );
  },

  computeFrame() {
    this.ctx1.drawImage(this.video, 0, 0, this.width, this.height);
    const frame = this.ctx1.getImageData(0, 0, this.width, this.height);
    const l = frame.data.length / 4;

    for (let i = 0; i < l; i++) {
      const grey =
        (frame.data[i \* 4 + 0] +
          frame.data[i \* 4 + 1] +
          frame.data[i \* 4 + 2]) /
        3;

      frame.data[i \* 4 + 0] = grey;
      frame.data[i \* 4 + 1] = grey;
      frame.data[i \* 4 + 2] = grey;
    }
    this.ctx1.putImageData(frame, 0, 0);

    return;
  },
};
```

Once the page has loaded you can call

```js
processor.doLoad();
```

#### Result

This is a pretty simple example showing how to manipulate video frames using a canvas. For efficiency, you should consider using `requestAnimationFrame()` instead of `setTimeout()` when running on browsers that support it.

You can achieve the same result by applying the `grayscale()` CSS function to the source `<video>` element.

**Note:** Due to potential security issues if your video is on a different domain than your code, you'll need to enable CORS (Cross Origin Resource Sharing) on your video server.

### Video and WebGL

WebGL is a powerful API that uses canvas to draw hardware-accelerated 3D or 2D scenes. You can combine WebGL and the `<video>` element to create video textures, which means you can put video inside 3D scenes.

**Note:** You can find the source code of this demo on GitHub (see it live also).

### Playback rate

We can also adjust the rate that audio and video plays at using an attribute of the `<audio>` and `<video>` element called `playbackRate`. `playbackRate` is a number that represents a multiple to be applied to the rate of playback, for example 0.5 represents half speed while 2 represents double speed.

Note that the `playbackRate` property works with both `<audio>` and `<video>`, but in both cases, it changes the playback speed but *not* the pitch. To manipulate the audio's pitch you need to use the Web Audio API. See the `AudioBufferSourceNode.playbackRate` property.

#### HTML

```html
<video
 id="my-video"
 controls
 src="https://jplayer.org/video/m4v/Big\_Buck\_Bunny\_Trailer.m4v"></video>
```

#### JavaScript

```js
const myVideo = document.getElementById("my-video");
myVideo.playbackRate = 2;
```

#### Editable example
```

<video id="my-video" controls width="480" height="270">
  <source
 src="https://jplayer.org/video/webm/Big\_Buck\_Bunny\_Trailer.webm"
 type="video/webm" />
  <source
 src="https://jplayer.org/video/m4v/Big\_Buck\_Bunny\_Trailer.m4v"
 type="video/mp4" />
</video>
<div class="playable-buttons">
  <input id="edit" type="button" value="Edit" />
  <input id="reset" type="button" value="Reset" />
</div>
<textarea id="code" class="playable-code">
const myVideo = document.getElementById('my-video');
myVideo.playbackRate = 2;</textarea
>
```

```
const textarea = document.getElementById("code");
const reset = document.getElementById("reset");
const edit = document.getElementById("edit");
const code = textarea.value;

function setPlaybackRate() {
  eval(textarea.value);
}

reset.addEventListener("click", () => {
  textarea.value = code;
  setPlaybackRate();
});

edit.addEventListener("click", () => {
  textarea.focus();
});

textarea.addEventListener("input", setPlaybackRate);
window.addEventListener("load", setPlaybackRate);
```

**Note:** Try the playbackRate example live.

Audio manipulation
------------------

`playbackRate` aside, to manipulate audio you'll typically use the Web Audio API.

### Selecting an audio source

The Web Audio API can receive audio from a variety of sources, then process it and send it back out to an `AudioDestinationNode` representing the output device to which the sound is sent after processing.

| If the audio source is… | Use this Web Audio node type |
| --- | --- |
| An audio track from an HTML `<audio>` or `<video>` element | `MediaElementAudioSourceNode` |
| A plain raw audio data buffer in memory | `AudioBufferSourceNode` |
| An oscillator generating a sine wave or other computed waveform | `OscillatorNode` |
| An audio track from WebRTC (such as the microphone input you can get using `getUserMedia()`. | `MediaStreamAudioSourceNode` |

### Audio filters

The Web Audio API has a lot of different filter/effects that can be applied to audio using the `BiquadFilterNode`, for example.

#### HTML

```html
<video id="my-video" controls src="myvideo.mp4" type="video/mp4"></video>
```

#### JavaScript

```js
const context = new AudioContext();
const audioSource = context.createMediaElementSource(
  document.getElementById("my-video"),
);
const filter = context.createBiquadFilter();
audioSource.connect(filter);
filter.connect(context.destination);

// Configure filter
filter.type = "lowshelf";
filter.frequency.value = 1000;
filter.gain.value = 25;
```

#### Editable example
```

<video id="my-video" controls width="480" height="270" crossorigin="anonymous">
  <source
 src="https://jplayer.org/video/webm/Big\_Buck\_Bunny\_Trailer.webm"
 type="video/webm" />
  <source
 src="https://jplayer.org/video/m4v/Big\_Buck\_Bunny\_Trailer.m4v"
 type="video/mp4" />
</video>
<div class="playable-buttons">
  <input id="edit" type="button" value="Edit" />
  <input id="reset" type="button" value="Reset" />
</div>
<textarea id="code" class="playable-code">
  filter.type = "lowshelf";
  filter.frequency.value = 1000;
  filter.gain.value = 25;
</textarea>
```

```
const context = new AudioContext();
const audioSource = context.createMediaElementSource(
  document.getElementById("my-video"),
);
const filter = context.createBiquadFilter();
audioSource.connect(filter);
filter.connect(context.destination);

const textarea = document.getElementById("code");
const reset = document.getElementById("reset");
const edit = document.getElementById("edit");
const code = textarea.value;

function setFilter() {
  eval(textarea.value);
}

reset.addEventListener("click", () => {
  textarea.value = code;
  setFilter();
});

edit.addEventListener("click", () => {
  textarea.focus();
});

textarea.addEventListener("input", setFilter);
window.addEventListener("load", setFilter);
```

**Note:** unless you have CORS enabled, to avoid security issues your video should be on the same domain as your code.

#### Common audio filters

These are some common types of audio filter you can apply:

* Low Pass: Allows frequencies below the cutoff frequency to pass through and attenuates frequencies above the cutoff.
* High Pass: Allows frequencies above the cutoff frequency to pass through and attenuates frequencies below the cutoff.
* Band Pass: Allows a range of frequencies to pass through and attenuates the frequencies below and above this frequency range.
* Low Shelf: Allows all frequencies through, but adds a boost (or attenuation) to the lower frequencies.
* High Shelf: Allows all frequencies through, but adds a boost (or attenuation) to the higher frequencies.
* Peaking: Allows all frequencies through, but adds a boost (or attenuation) to a range of frequencies.
* Notch: Allows all frequencies through, except for a set of frequencies.
* All Pass: Allows all frequencies through, but changes the phase relationship between the various frequencies.

**Note:** See `BiquadFilterNode` for more information.

### Convolutions and impulses

It's also possible to apply impulse responses to audio using the `ConvolverNode`. An **impulse response** is the sound created after a brief impulse of sound (like a hand clap). An impulse response will signify the environment in which the impulse was created (for example, an echo created by clapping your hands in a tunnel).

#### Example

```js
const convolver = context.createConvolver();
convolver.buffer = this.impulseResponseBuffer;
// Connect the graph.
source.connect(convolver);
convolver.connect(context.destination);
```

See this Codepen for an applied (but very, very silly; like, little kids will giggle kind of silly) example.

### Spatial audio

We can also position audio using a **panner node**. A panner node—`PannerNode`—lets us define a source cone as well as positional and directional elements, all in 3D space as defined using 3D cartesian coordinates.

#### Example

```js
const panner = context.createPanner();
panner.coneOuterGain = 0.2;
panner.coneOuterAngle = 120;
panner.coneInnerAngle = 0;

panner.connect(context.destination);
source.connect(panner);
source.start(0);

// Position the listener at the origin.
context.listener.setPosition(0, 0, 0);
```

**Note:** You can find an example on our GitHub repository (see it live also).

### JavaScript codecs

It's also possible to manipulate audio at a low level using JavaScript. This can be useful should you want to create audio codecs.

Libraries currently exist for the following formats:

* AAC: AAC.js
* ALAC: alac.js
* FLAC: flac.js
* MP3: mp3.js
* Opus: Opus.js
* Vorbis: vorbis.js

**Note:** At Audiocogs, you can Try out a few demos; Audiocogs also provides a framework, Aurora.js, which is intended to help you author your own codecs in JavaScript.

Examples
--------

* Various Web Audio API (and other) examples
* THREE.js Video Cube example
* Convolution Effects in Real-Time
See also
--------

### Tutorials

* Manipulating Video Using Canvas
* HTML playbackRate explained
* Using the Web Audio API
* Web audio spatialization basics
* Using Video frames as a WebGL Texture (You can also the THREE.js WebGL library (and others) to achieve this effect)
* Animating Textures in WebGL
* Developing Game Audio with the Web Audio API (Room effects and filters) (2012)
### Reference

* The `<audio>` and `<video>` elements
* The `HTMLMediaElement` API
* The `<canvas>` element
* Web Audio API
* AudioContext
* More info on Spatial Audio
* Web media technologies


Media type and format guide: image, audio, and video content - Web media technologies

Media type and format guide: image, audio, and video content
============================================================

Since nearly its beginning, the web has included support for some form of visual media presentation. Originally, these capabilities were limited, and were expanded organically, with different browsers finding their own solutions to the problems around including still and video imagery on the web. The modern web has powerful features to support the presentation and manipulation of media, with several media-related APIs supporting various types of content. Generally, the media formats supported by a browser are entirely up to the browser's creators, which can complicate the work of a web developer.

This guide provides an overview of the media file types, codecs, and algorithms that may comprise media used on the web. It also provides browser support information for various combinations of these, and suggestions for prioritization of formats, as well as which formats excel at specific types of content.

References
----------

### Images

Image file type and format guide

A guide to the main image file types and content formats used on the Internet. This includes a high level overview of: browser support, benefits, and limitations, along with best-practice guidelines to help web designers choose the right image file format for specific types of content.

### Media file types and codecs

Media containers (file types)

A guide to the file types that contain media data. Some are audio-specific, while others may be used for either audio or combined audiovisual content such as movies. Includes overviews of each of the file types supported by the major web browsers, along with browser support information and supported features.

Web audio codec guide

A guide to the audio codecs allowed for by the common media containers, as well as by the major browsers. Includes benefits, limitations, key specifications and capabilities, and use cases. It also covers each browser's support for using the codec in given containers.

Web video codec guide

This article provides basic information about the video codecs supported by the major browsers, as well as some that are not commonly supported but that you might still run into. It also covers codec capabilities, benefits, limitations, and browser support levels and restrictions.

Codecs in common media types

When specifying the MIME type describing a media format, you can provide details using the `codecs` parameter as part of the type string. This guide describes the format and possible values of the `codecs` parameter for the common media types.

Codecs used by WebRTC

WebRTC doesn't use a container, but instead streams the encoded media itself from peer to peer using `MediaStreamTrack` objects to represent each audio or video track. This guide discusses the codecs commonly used with WebRTC.

Guides
------

### Concepts

Digital audio concepts

An introduction to how audio is converted into digital form and stored for use by computers. It explains basic concepts about how audio is sampled, as well as concepts such as sample rate, audio frames, and audio compression.

Digital video concepts

A guide to fundamental concepts involved with digital video as used on the web, including basics about color formats, chroma subsampling, how human perception influences video coding, and so forth.

### Tutorials and how-tos

Learning: Video and audio content

This tutorial introduces and details the use of media on the web.

Handling media support issues in web content

In this guide, we look at how to build web content that maximizes quality or performance while providing the broadest possible compatibility, by choosing media formats wisely, and offering fallbacks and alternate formats where it would be helpful.

Other topics
------------

Media Capabilities API

The Media Capabilities API lets you discover the encoding and decoding capabilities of the device your app or site is running on. This lets you make real-time decisions about what formats to use and when.


Guide to streaming audio and video - Web media technologies

Guide to streaming audio and video
==================================

In this guide, we'll examine the techniques used to stream audio and/or video media on the web, and how you can optimize your code, your media, your server, and the options you use while performing the streaming to bring out the best quality and performance possible.

Protocols
---------

In addition to the configuration of the server and the streaming code, there are sometimes special protocols which can be used to optimize performance.

### HTTPS Live Streaming

**HTTPS Live Streaming** (**HLS**) is a protocol developed by Apple and supported by Safari on all of their platforms. HLS may also be supported in other environments, although in some cases this support is conditional.

For example, because many websites' mobile-specific content assume that mobile browsers support HLS, Firefox for Android does as well, in order to avoid strange compatibility errors from occurring due to this assumption being incorrect. The desktop version of Firefox, however, does not support HLS.

HLS uses playlists to allow the user to not only select the media to stream, but also to choose among versions or forms of the same media. For example, HLS lets the server stream a video with multiple audio streams which the user can choose from, in order to hear their own language. Additionally, forms of the stream can be provided which are optimized for different network conditions. In this way, live streams can be made flexible and highly performant.

As of mid-2017, HLS has been standardized as RFC 8216.

See also
--------

* Web media technologies
* Guide to media types and formats on the web
* `<audio>` and `<video>`