index.bs

<pre class='metadata'>
Title: Picture-in-Picture
Shortname: picture-in-picture
Level: None
Status: ED
ED: https://w3c.github.io/picture-in-picture/
TR: https://www.w3.org/TR/picture-in-picture/
Favicon: https://raw.githubusercontent.com/google/material-design-icons/master/action/2x_web/ic_picture_in_picture_alt_black_48dp.png
Group: mediawg
Markup Shorthands: markdown yes
Repository: w3c/picture-in-picture
!Web Platform Tests: <a href="https://github.com/web-platform-tests/wpt/tree/master/permissions-policy">permissions-policy/</a><br/><a href="https://github.com/web-platform-tests/wpt/tree/master/picture-in-picture">picture-in-picture/</a>
Editor: François Beaufort, w3cid 81174, Google LLC https://www.google.com, fbeaufort@google.com
Former Editor: Mounir Lamouri, w3cid 45389, Google LLC https://www.google.com, mlamouri@google.com
Abstract: This specification provides APIs to allow websites to
Abstract: create a floating video window always on top of other windows so that
Abstract: users may continue consuming media while they interact with other
Abstract: content sites, or applications on their device.
</pre>

<pre class="anchors">
spec: HTML; urlPrefix: https://html.spec.whatwg.org/multipage/
    type: dfn
        urlPrefix: media.html
            text: media element event task source
    type: dfn
        urlPrefix: interaction.html
            text: system visibility state
spec: Fullscreen; urlPrefix: https://fullscreen.spec.whatwg.org
    type: dfn; text: fullscreen flag; url: #fullscreen-flag
    type: dfn; text: fullscreenElement; url: dom-document-fullscreenelement
    type: dfn; text: exit fullscreen; url: fully-exit-fullscreen
spec: Remote-Playback; urlPrefix: https://w3c.github.io/remote-playback/#dfn-
    type: dfn
        text: local playback device
        text: local playback state
</pre>

<pre class="link-defaults">
spec:dom; type:attribute; text:bubbles
spec:dom; type:dfn; for:NamedNodeMap; text:element
spec:dom; type:dfn; text:origin
spec:dom; type:interface; text:Document
spec:html; type:attribute; for:HTMLMediaElement; text:readyState
spec:html; type:dfn; for:/; text:browsing context
spec:infra; type:dfn; text:user agent
spec:infra; type:dfn; for:list; text:item
</pre>

# Introduction # {#intro}

<em>This section is non-normative.</em>

Many users want to continue consuming media while they interact with other
content, sites, or applications on their device. A common UI affordance for
this type of activity is Picture-in-Picture (PiP), where the video is contained
in a separate miniature window that is always on top of other windows. This
window stays visible even when the user agent is not visible.
Picture-in-Picture is a common platform-level feature among desktop and mobile
OSs.

This specification extends {{HTMLVideoElement}} allowing websites
to initiate and control this behavior by exposing the following sets of properties:

* Notify the website when it enters and leaves Picture-in-Picture mode.
* Allow the website to trigger Picture-in-Picture mode via a user gesture on a
    video element.
* Allow the website to know the size of the Picture-in-Picture window and notify
    the website when it changes.
* Allow the website to exit Picture-in-Picture mode.
* Allow the website to check if Picture-in-Picture mode can be triggered.


# Examples # {#examples}

## Add a custom Picture-in-Picture button ## {#example-add-custom-pip-button}

```html
<video id="video" src="https://example.com/file.mp4"></video>

<button id="togglePipButton"></button>

<script>
  const video = document.getElementById("video");
  const togglePipButton = document.getElementById("togglePipButton");

  // Hide button if Picture-in-Picture is not supported or disabled.
  togglePipButton.hidden =
    !document.pictureInPictureEnabled || video.disablePictureInPicture;

  togglePipButton.addEventListener("click", async () => {
    // If there is no element in Picture-in-Picture yet, let's request
    // Picture-in-Picture for the video, otherwise leave it.
    try {
      if (document.pictureInPictureElement) {
        await document.exitPictureInPicture();
      } else {
        await video.requestPictureInPicture();
      }
    } catch (err) {
      // Video failed to enter/leave Picture-in-Picture mode.
    }
  });
</script>
```

## Monitor video Picture-in-Picture changes ## {#example-monitor-video-pip-changes}

```html
<video id="video" src="https://example.com/file.mp4"></video>

<script>
  const video = document.getElementById("video");

  video.addEventListener("enterpictureinpicture", (event) => {
    // Video entered Picture-in-Picture mode.
    const pipWindow = event.pictureInPictureWindow;
    console.log(`Picture-in-Picture window width: ${pipWindow.width}`);
    console.log(`Picture-in-Picture window height: ${pipWindow.height}`);
  });

  video.addEventListener("leavepictureinpicture", () => {
    // Video left Picture-in-Picture mode.
  });
</script>
```

## Update video size based on Picture-in-Picture window size changes ## {#example-update-video-on-window-size-changes}

```html
<video id="video" src="https://example.com/file.mp4"></video>

<button id="pipButton"></button>

<script>
  const video = document.getElementById("video");
  const pipButton = document.getElementById("pipButton");

  pipButton.addEventListener("click", async () => {
    try {
      await video.requestPictureInPicture();
    } catch (error) {
      // Video failed to enter Picture-in-Picture mode.
    }
  });

  video.addEventListener("enterpictureinpicture", (event) => {
    // Video entered Picture-in-Picture mode.
    const pipWindow = event.pictureInPictureWindow;
    updateVideoSize(pipWindow.width, pipWindow.height);
    pipWindow.addEventListener("resize", onPipWindowResize);
  });

  video.addEventListener("leavepictureinpicture", (event) => {
    // Video left Picture-in-Picture mode.
    const pipWindow = event.pictureInPictureWindow;
    pipWindow.removeEventListener("resize", onPipWindowResize);
  });

  function onPipWindowResize(event) {
    // Picture-in-Picture window has been resized.
    const { width, height } = event.target;
    updateVideoSize(width, height);
  }

  function updateVideoSize(width, height) {
    // TODO: Update video size based on pip window width and height.
  }
</script>
```

# Concepts # {#concepts}

## Internal Slot Definitions ## {#defines}

A <a>user agent</a> has:

1. An <dfn>initiators of active Picture-in-Picture sessions</dfn>
    list of zero or more <a>origins</a>, which is initially empty.

Note: In case a <a>user agent</a> supports multiple Picture-in-Picture
  windows, the list allows duplicates.

An origin is said to have an active Picture-in-Picture session if any
of the origins in <a>initiators of active Picture-in-Picture sessions</a>
are <a>same origin-domain</a> with origin.

## Request Picture-in-Picture ## {#request-pip}

When the <dfn>request Picture-in-Picture algorithm</dfn> with |video| is invoked,
the user agent MUST run the following steps:

1. If <a>Picture-in-Picture support</a> is `false`, throw a
    {{NotSupportedError}} and abort these steps.
2. If the document is not <a>allowed to use</a> the <a>policy-controlled feature</a>
    named `"picture-in-picture"`, throw a {{SecurityError}} and abort these
    steps.
3. If |video|'s {{readyState}} attribute is {{HAVE_NOTHING}}, throw a
    {{InvalidStateError}} and abort these steps.
4. If |video| has no video track, throw a {{InvalidStateError}} and abort
    these steps.
5. If |video|'s {{HTMLVideoElement/disablePictureInPicture}} is true,
    the user agent MAY throw an {{InvalidStateError}} and abort these steps.
6. If {{pictureInPictureElement}} is `null` and the <a>relevant global object</a>
    of <a>this</a> does not have <a>transient activation</a>, throw a
    {{NotAllowedError}} and abort these steps.
7. If |video| is {{pictureInPictureElement}}, abort these steps.
8. Set {{pictureInPictureElement}} to |video|.
9. Let <dfn>Picture-in-Picture window</dfn> be a new instance of
    {{PictureInPictureWindow}} associated with {{pictureInPictureElement}}.
10. Append <a>relevant settings object</a>'s <a>origin</a> to
    <a>initiators of active Picture-in-Picture sessions</a>.
11. <a>Queue a task</a> to <a>fire an event</a> named
    {{enterpictureinpicture}} using {{PictureInPictureEvent}} at the
    |video| with its {{bubbles}} attribute initialized to `true` and its
    {{PictureInPictureEvent/pictureInPictureWindow}} attribute initialized to
    <a>Picture-in-Picture window</a>.
12. If {{pictureInPictureElement}} is <a>fullscreenElement</a>, it is
    RECOMMENDED to <a>exit fullscreen</a>.

It is RECOMMENDED that video frames are not rendered in the page and in the
Picture-in-Picture window at the same time but if they are, they MUST be kept
in sync.

When a video is played in Picture-in-Picture mode, the states SHOULD transition
as if it was played inline. That means that the events SHOULD fire at the same
time, calling methods SHOULD have the same behaviour, etc. However, the user
agent MAY transition out of Picture-in-Picture when the video element enters a
state that is considered not compatible with Picture-in-Picture.

Styles applied to |video| (such as opacity, visibility, transform, etc.) MUST
NOT apply in the Picture-in-Picture window. Its aspect ratio is based on the
video size.

It is also RECOMMENDED that the Picture-in-Picture window has a maximum and
minimum size. For example, it could be restricted to be between a quarter and
a half of one dimension of the screen.

## Exit Picture-in-Picture ## {#exit-pip}

When the <dfn>exit Picture-in-Picture algorithm</dfn> is invoked,
the user agent MUST run the following steps:

1. If {{pictureInPictureElement}} is `null`, throw a {{InvalidStateError}} and
    abort these steps.
2. Run the <a>close window algorithm</a> with the <a>Picture-in-Picture
    window</a> associated with {{pictureInPictureElement}}.
3. <a>Queue a task</a> to <a>fire an event</a> named
    {{leavepictureinpicture}} using {{PictureInPictureEvent}} at the
    |video| with its {{bubbles}} attribute initialized to `true` and its
    {{PictureInPictureEvent/pictureInPictureWindow}} attribute initialized to
    <a>Picture-in-Picture window</a> associated with {{pictureInPictureElement}}.
4. Unset {{pictureInPictureElement}}.
5. Remove one <a>item</a> matching <a>relevant settings object</a>'s <a>origin</a> from
    <a>initiators of active Picture-in-Picture sessions</a>.

It is NOT RECOMMENDED that the video playback state changes when the <a>exit
Picture-in-Picture algorithm</a> is invoked. The website SHOULD be in control
of the experience if it is website initiated. However, the user agent MAY expose
Picture-in-Picture window controls that change video playback state (e.g.,
pause).

As one of the <a>unloading document cleanup steps</a>, run the <a>exit
Picture-in-Picture algorithm</a>.

## Disable Picture-in-Picture ## {#disable-pip}

Some pages may want to disable Picture-in-Picture mode for a video element; for
example, they may want to prevent the user agent from suggesting a
Picture-in-Picture context menu in some cases.
To support these use cases, a new {{disablePictureInPicture}}
attribute is added to the list of content attributes for video elements.

The {{disablePictureInPicture}} IDL attribute MUST <a>reflect</a> the content
attribute of the same name.

If the {{disablePictureInPicture}} attribute is present on the video element,
the user agent MAY prevent the video element from playing in Picture-in-Picture
mode or present any UI to do so.

When the {{disablePictureInPicture}} attribute is added to a |video| element,
the user agent MAY run these steps:

1. Reject any pending promises returned by the {{requestPictureInPicture()}}
    method with {{InvalidStateError}}.
2. If |video| is {{pictureInPictureElement}}, run the <a>exit
    Picture-in-Picture algorithm</a>.

## Interaction with Fullscreen ## {#fullscreen}

It is RECOMMENDED to run the <a>exit Picture-in-Picture algorithm</a> when the
{{pictureInPictureElement}} <a>fullscreen flag</a> is set.

## Interaction with Remote Playback ## {#remote-playback}

The [[Remote-Playback]] specification defines a <a>local playback device</a>
and a <a>local playback state</a>. For the purpose of Picture-in-Picture, the
playback is local and regardless of whether it is played in page or in
Picture-in-Picture.

## Interaction with Media Session ## {#media-session}

The API will have to be used with the [[MediaSession]] API for customizing the
available controls on the Picture-in-Picture window.

## Interaction with Page Visibility ## {#page-visibility}

When {{pictureInPictureElement}} is set, the Picture-in-Picture window MUST
be visible, even when the <a>Document</a> is not in focus or hidden. The user
agent SHOULD provide a way for users to manually close the Picture-in-Picture
window.

The Picture-in-Picture window visibility MUST NOT be taken into account by the
user agent to determine if the <a>system visibility state</a> of a 
<a for="/">traversable navigable</a> has changed.

## One Picture-in-Picture window ## {#one-pip-window}

Operating systems with a Picture-in-Picture API usually restrict
Picture-in-Picture mode to only one window.  Whether only one window is allowed
in Picture-in-Picture mode will be left to the implementation and the platform.
However, because of the one Picture-in-Picture window limitation, the
specification assumes that a given {{Document}} can only have one
Picture-in-Picture window.

What happens when there is a Picture-in-Picture request while a window is
already in Picture-in-Picture will be left as an implementation detail: the
current Picture-in-Picture window could be closed, the Picture-in-Picture
request could be rejected or even two Picture-in-Picture windows could be
created. Regardless, the User Agent will have to fire the appropriate events
in order to notify the website of the Picture-in-Picture status changes.

# API # {#api}

## Extensions to <code>HTMLVideoElement</code> ## {#htmlvideoelement-extensions}

<pre class="idl">
partial interface HTMLVideoElement {
  [NewObject] Promise&lt;PictureInPictureWindow&gt; requestPictureInPicture();

  attribute EventHandler onenterpictureinpicture;
  attribute EventHandler onleavepictureinpicture;

  [CEReactions] attribute boolean disablePictureInPicture;
};
</pre>

The {{requestPictureInPicture()}} method, when invoked, MUST
return <a>a new promise</a> |promise| and run the following steps <a>in
parallel</a>:

1. Let |video| be the video element on which the method was invoked.
2. Run the <a>request Picture-in-Picture algorithm</a> with |video|.
3. If the previous step threw an exception, reject |promise| with that
    exception and abort these steps.
4. [=/Resolve=] |promise| with the <a>Picture-in-Picture window</a> associated with
    {{pictureInPictureElement}}.

## Extensions to <code>Document</code> ## {#document-extensions}

<pre class="idl">
partial interface Document {
  readonly attribute boolean pictureInPictureEnabled;

  [NewObject] Promise&lt;undefined&gt; exitPictureInPicture();
};
</pre>

The {{pictureInPictureEnabled}} attribute's getter must return `true` if
<a>Picture-in-Picture support</a> is `true` and <a>this</a> is
<a>allowed to use</a> the feature indicated by attribute name
`picture-in-picture`, and `false` otherwise.

<dfn>Picture-in-Picture support</dfn> is `false` if there's a user preference
that disables it or a platform limitation. It is `true` otherwise.

The {{exitPictureInPicture()}} method, when invoked, MUST
return <a>a new promise</a> |promise| and run the following steps <a>in
parallel</a>:

1. Run the <a>exit Picture-in-Picture algorithm</a>.
2. If the previous step threw an exception, reject |promise| with that
    exception and abort these steps.
3. [=/Resolve=] |promise|.

## Extension to <code>DocumentOrShadowRoot</code> ## {#documentorshadowroot-extension}

<pre class="idl">
partial interface mixin DocumentOrShadowRoot {
  readonly attribute Element? pictureInPictureElement;
};
</pre>

The {{pictureInPictureElement}} attribute's getter must run these steps:

1. If <a>this</a> is a <a for=/>shadow root</a> and its <a for=DocumentFragment>host</a>
    is not <a>connected</a>, return `null` and abort these steps.
2. Let |candidate| be the result of <a>retargeting</a> Picture-in-Picture
    element against <a>this</a>.
3. If |candidate| and <a>this</a> are in the same <a>tree</a>,
    return |candidate| and abort these steps.
4. Return `null`.

## Interface <code>PictureInPictureWindow</code> ## {#interface-picture-in-picture-window}

<pre class="idl">
[Exposed=Window]
interface PictureInPictureWindow : EventTarget {
  readonly attribute long width;
  readonly attribute long height;

  attribute EventHandler onresize;
};
</pre>

A {{PictureInPictureWindow}} instance represents a <a>Picture-in-Picture
window</a> associated with an {{HTMLVideoElement}}. When instantiated, an
instance of {{PictureInPictureWindow}} has its |state| set to `opened`.

When the <dfn>close window algorithm</dfn> with an instance of
{{PictureInPictureWindow}} is invoked, its |state| is set to `closed`.

The {{width}} attribute MUST return the width in <a lt=px value>CSS pixels</a> of the
<a>Picture-in-Picture window</a> associated with {{pictureInPictureElement}} if
the |state| is `opened`. Otherwise, it MUST return 0.

The {{height}} attribute MUST return the height in <a lt=px value>CSS pixels</a> of the
<a>Picture-in-Picture window</a> associated with {{pictureInPictureElement}} if
the |state| is `opened`. Otherwise, it MUST return 0.

When the size of the <a>Picture-in-Picture window</a> associated with
{{pictureInPictureElement}} changes, the user agent MUST <a>queue a task</a> to
<a>fire an event</a> named {{resize}} at {{pictureInPictureElement}}.

## Event types ## {#event-types}

<pre class="idl">
[Exposed=Window]
interface PictureInPictureEvent : Event {
    constructor(DOMString type, PictureInPictureEventInit eventInitDict);
    [SameObject] readonly attribute PictureInPictureWindow pictureInPictureWindow;
};

dictionary PictureInPictureEventInit : EventInit {
    required PictureInPictureWindow pictureInPictureWindow;
};
</pre>

: <dfn event for="HTMLVideoElement">`enterpictureinpicture`</dfn>
:: Fired on a {{HTMLVideoElement}} when it enters Picture-in-Picture.
: <dfn event for="HTMLVideoElement">`leavepictureinpicture`</dfn>
:: Fired on a {{HTMLVideoElement}} when it leaves Picture-in-Picture mode.
: <dfn event for="PictureInPictureWindow">`resize`</dfn>
:: Fired on a {{PictureInPictureWindow}} when it changes size.

## Task source ## {#task-source}

The <a>task source</a> for all the tasks queued in this specification is the
<a>media element event task source</a> of the video element in question.

## CSS pseudo-class ## {#css-pseudo-class}

The `:picture-in-picture` <a>pseudo-class</a> MUST match the Picture-in-Picture
element. It is different from the {{pictureInPictureElement}} as it does NOT
apply to the shadow host chain.

# Security considerations # {#security-considerations}

<em>This section is non-normative.</em>

To limit potential abuse through spoofing, the API applies only to
{{HTMLVideoElement}}. User interaction with the Picture-in-Picture window
is intentionally limited so that the only effect is on the Picture-in-Picture
window itself or the media being played.

## Secure Context ## {#secure-context}

The API is not limited to [[SECURE-CONTEXTS]] because it exposes a feature
to web applications that user agents usually offer natively on all media
regardless of the <a>browsing context</a>.

## Permissions Policy ## {#permissions-policy}

This specification defines a <a>policy-controlled feature</a> named
`"picture-in-picture"` that controls whether the <a>request Picture-in-Picture
algorithm</a> may return a {{SecurityError}} and whether
{{pictureInPictureEnabled}} is `true` or `false`.

The <a>default allowlist</a> for this feature is `*`.

# Acknowledgments # {#acknowledgments}

Thanks to Jennifer Apacible, Zouhir Chahoud, Marcos Cáceres, Philip Jägenstedt,
Jeremy Jones, Chris Needham, Jer Noble, Justin Uberti, Yoav Weiss, and Eckhart
Wörner for their contributions to this document.