Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

First proposal of dom layers #293

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions explainer.md
Original file line number Diff line number Diff line change
Expand Up @@ -304,3 +304,38 @@ const layer = xrMediaBinding.createQuadLayer(video, { layout: 'stereo-top-bottom
```

This will then cause only the top half of the video to show to the left eye and the bottom half of the video to show to the right eye. If more complex layouts are required than are described by the `XRLayerLayout` enum then the video must be manually rendered using an `XRWebGLBinding` layer instead.

## DOM layers
WebXR and WebXR Layers are very flexible because they give you complete control over every pixel.
However, this loses the feature set of HTML which makes it much harder to create 2D UI, provide interactivity and accessible content.

To give developers this ability back, this spec will introduce the notion of `DOM Layers`.
Much like Media and WebGL Layers, these layers are defined as a 2D plane that is composited in the scene by the system Compositor.
However, the content of this layer will be drawn by the browser itself as if it was another browser window. You pass a URL to the layer creation function and the layer will be drawn as if it was a popup window.

To mitigate security concerns the following limitations are applied:
- The URL has to be same origin as the your session
- The layer is not allowed to navigate to a different origin
- The layer is not allowed to create nested context (ie no \<iframe>)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure whether the security needs to be more restrictive than an iframe. One would like to be able to show external content, when it is allowed by the origin and external source, like in this mockup SpaceX control room
For same origin content I think one should be able to use the events in DOM to react in the 3d environment, like here Virtual shop

Because layer content is only accessible to the system compositor,
the author will not be able to read any pixel data from the layer
(just like with media layers).

Creating the layer will immediately start its browser session. The session won't be released until the layer's `destroy()` method is called.

Hit testing is done according to the `targetRayMode` of the current session. Layers that do not have a `blendTextureSourceAlpha` will block hit testing of DOM layers.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sessions don't have targetRayModes, inputs do.

Also, I think a bit more detail in needed about the behavior when a layer does have blendTextureSourceAlpha.


To create DOM layers, an XRDOMBinding must be created, similar to the XRWebGLBinding:
```js
const xrDOMBinding = new XRMDOMBinding(xrSession);
```

Use this factory object to create DOM Layers:

```js
const domlayer = xrDOMBinding.createCylinderLayer
("https://mysite.html/layercontent.html",
{ referenceSpace: xrReferenceSpace,
transform: new XRRigidTransform({z: -2})});
```
144 changes: 144 additions & 0 deletions webxrlayers-1.bs
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,8 @@ spec: html;
type: dfn; text: request the xr permission
spec:webidl;
type:dfn; text:new
spec:dom;
type:dfn; text:origin
</pre>

<pre class="anchors">
Expand Down Expand Up @@ -135,6 +137,8 @@ spec: ECMAScript; urlPrefix: https://tc39.github.io/ecma262/#
type: dfn; text: Realm; url: realm
spec: compositing-1; urlPrefix: https://www.w3.org/TR/compositing-1/
type: dfn; text: source-over; url: porterduffcompositingoperators_srcover
spec:fetch; urlPrefix: https://fetch.spec.whatwg.org/
type:dfn; text:cors-protocol; url: cors-protocol
</pre>


Expand Down Expand Up @@ -282,6 +286,13 @@ The "[=feature descriptor/layers=]" [=feature descriptor=] has a [=feature requi
NOTE: This means that executing the {{Permissions/request(permissionDesc)}} API with "[=feature descriptor/layers=]" will
not enable layers support for the current active session.

<div class="unstable">

If an application wants to create [[#domlayer|DOM layers]], the session MUST be requested with an appropriate [=feature descriptor=]. The string "<dfn for="feature descriptor">dom-layers</dfn>" is introduced
by this module as a new valid [=feature descriptor=] for the WebXR Layers feature.

</div>

Layer types {#xrlayertypes}
===========

Expand Down Expand Up @@ -388,6 +399,7 @@ or a {{WebGL2RenderingContext}} |context|, the user agent MUST run the following
</dl>
1. Set [=this=] {{XRCompositionLayer/opacity}} to <code>1.0</code>.


</div>

<div class="algorithm" data-algorithm="calling destroy on a layer">
Expand Down Expand Up @@ -1959,6 +1971,137 @@ When this method is invoked, the user agent MUST run the following steps:

ISSUE: define how the {{XREquirectLayer}}'s parameters affect the video display.

DOM layer creation {#domlayer}
==================

<div class="unstable">

Description {#xrdomlayerdescription}
-----------
DOM Layers allow the display of HTML content within an immersive session.
They are only enabled when the "[=feature descriptor/dom-layers=]" [=feature descriptor=] is granted to the current {{XRSession}}.

The [=origin=] of the URL MUST be the same as the [=origin=] of the URL that created the immersive session.
In addition, the `[=child-src=]` Content Security Policy with the document's [=origin=] MUST be applied to the loaded document.

NOTE: this means that the experience can only open pages from the same [=origin=] and these pages can't open iframes to other origins or navigate to other origins.

XRDOMLayerInit {#xrdomlayerinittype}
--------------
The {{XRDOMLayerInit}} dictionary represents a set of configurable values that describe how an {{XRCompositionLayer}} containing a web page
is initialized.

<pre class="idl">
dictionary XRDOMLayerInit {
required XRSpace space;
XRRigidTransform? transform;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is the DOM content sized? The width and height in the QuadLayer are in-world and it would be difficult to infer a required pixel scale from them. Similar things apply to the other layers.

};
</pre>

The <dfn dict-member for="XRDOMLayerInit">space</dfn> attribute defines the spatial relationship with the user’s physical environment.

XRDOMQuadLayerInit {#xrdomquadlayerinittype}
------------------
The {{XRDOMQuadLayerInit}} dictionary represents a set of configurable values that describe how an {{XRQuadLayer}} containing a web page
is initialized.

<pre class="idl">
dictionary XRDOMQuadLayerInit : XRDOMLayerInit {
float? width;
float? height;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are these nullable? My guess would be so that a default ratio can be determined from the window dimensions, in which case the windowPixel* attribs I mentioned above are all the more important.

At least a brief note describing the intent behind making these nullable would be appreciated, even if the exact logic will be fleshed out later.

};
</pre>

XRDOMCylinderLayerInit {#xrdomcylinderlayerinittype}
----------------------
The {{XRDOMCylinderLayerInit}} dictionary represents a set of configurable values that describe how an {{XRCylinderLayer}} containing a web page
is initialized.

<pre class="idl">
dictionary XRDOMCylinderLayerInit : XRDOMLayerInit {
float radius = 2.0;
float centralAngle = 0.78539;
float? aspectRatio;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto for mentioning why this is nullable.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

though I assume we'll get this when the algorithm is filled in.

};
</pre>

XRDOMQuadLayerResult {#xrdomquadlayerresulttype}
--------------------
The {{XRDOMQuadLayerResult}} class contains the result of calling {{XRDOMBinding/createQuadLayer()}}.

<pre class="idl">
[SecureContext, Exposed=Window]
interface XRDOMQuadLayerResult {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if I like this pattern, because it breaks from the other two layer sources. You can no longer use the result of the create*Layer call directly. Especially because it's feasible that some developers may pass the URL for the layer and then never interact with the layer's window contents again.

Could we instead add a getWindow() method to the XRDOMLayerBinding? Yes, it wouldn't apply to every layer, but we already have that situation with the WebGL layers and get[Sub]Image().

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, I'd rather do some getWindow() thing if it's really necessary

[SameObject] readonly attribute XRQuadLayer layer;
[SameObject] readonly attribute WindowProxy window;
};
</pre>

{{XRDOMQuadLayerResult/layer}} contains the layer that was created with the {{XRDOMQuadLayerInit}} parameters.

{{XRDOMQuadLayerResult/window}} contains the {{WindowProxy}} that is associated with the newly created browsing context in the layer.

XRDOMCylinderLayerResult {#xrdomcylinderlayerresulttype}
--------------------
The {{XRDOMCylinderLayerResult}} class contains the result of calling {{XRDOMBinding/createCylinderLayer()}}.

<pre class="idl">
[SecureContext, Exposed=Window]
interface XRDOMCylinderLayerResult {
[SameObject] readonly attribute XRCylinderLayer layer;
[SameObject] readonly attribute WindowProxy window;
};
</pre>

{{XRDOMCylinderLayerResult/layer}} contains the layer that was created with the {{XRDOMQuadLayerInit}} parameters.

{{XRDOMCylinderLayerResult/window}} contains the {{WindowProxy}} that is associated with the newly created browsing context in the layer.

XRDOMBinding {#xrdombindingtype}
------------
The {{XRDOMBinding}} object is used to create layers that display a web page.

<pre class="idl">
[Exposed=Window] interface XRDOMBinding {
constructor(XRSession session);

XRDOMQuadLayerResult createQuadLayer(DOMString url, optional XRDOMQuadLayerInit init = {});
XRDOMCylinderLayerResult createCylinderLayer(DOMString video, optional XRDOMCylinderLayerInit init = {});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
XRDOMCylinderLayerResult createCylinderLayer(DOMString video, optional XRDOMCylinderLayerInit init = {});
XRDOMCylinderLayerResult createCylinderLayer(DOMString url, optional XRDOMCylinderLayerInit init = {});

};
</pre>

ISSUE: the init dictionaries shouldn't be optional. This is bikeshed issue 1566.

<!--
Each {{XRDOMBinding}} has an associated <dfn dfn-for="XRDOMBinding">session</dfn>, which is the
{{XRSession}} it was created with.
-->

NOTE: It is possible to create more than one {{XRDOMBinding}}. The lifetime of a layer is not tied
to the lifetime of the {{XRDOMBinding}} that created it.

<div class="algorithm" data-algorithm="createDOMQuadLayerAlgo">
The <dfn method for="XRDOMBinding">createQuadLayer(DOMString |url|, XRQuadLayerInit |init|)</dfn> method creates a new {{XRDOMQuadLayerResult}} |result|.

When this method is invoked, the user agent MUST run the following steps:
1. |url|
1. |init|
1. |result|

</div>

<div class="algorithm" data-algorithm="createDOMCylinderLayerAlgo">
The <dfn method for="XRDOMBinding">createCylinderLayer(DOMString |url|, XRDOMCylinderLayerInit |init|)</dfn> method creates a new {{XRDOMCylinderLayerResult}} |result|.

When this method is invoked, the user agent MUST run the following steps:
1. |url|
1. |init|
1. |result|

</div>

</div>

Events {#events}
======

Expand Down Expand Up @@ -2089,6 +2232,7 @@ monoscopic devices.

XRView changes {#xrviewchanges}
--------------

Each [=view=] MUST define a <dfn>recommended WebGL color texture resolution</dfn> which represents a best estimate of the WebGL texture
resolution large enough to contain the view.

Expand Down