From 3f24fad9c0a6a3ed0d9aafd524a87771d2899434 Mon Sep 17 00:00:00 2001 From: Domenic Denicola Date: Mon, 3 Feb 2025 15:08:40 +0900 Subject: [PATCH] Make all prerenders go via prefetching first (#359) This allows deduplicating a good amount of the No-Vary-Search handling that both needed to do, and eliminates the entire "prerender records" concept. Fixes #320 by properly specifying Sec-Purpose for prerenders. --- prefetch.bs | 33 ++++++++------ prerendering.bs | 100 +++++++++++++++++-------------------------- speculation-rules.bs | 6 ++- 3 files changed, 64 insertions(+), 75 deletions(-) diff --git a/prefetch.bs b/prefetch.bs index b1cab4a..ba87382 100644 --- a/prefetch.bs +++ b/prefetch.bs @@ -182,6 +182,7 @@ A prefetch record is a [=struct=] with the following [=struct/ * label, a [=string=]
This is intended for use by a specification or [=implementation-defined=] feature to identify which prefetches it created. It might also associate other data with this struct.
+* prerendering traversable, a [=prerendering traversable=], "`to be created`", or null (null by default) * state, which is "`ongoing`" (the default), "`completed`", or "`canceled`"
"`canceled`" indicates that the prefetch was aborted by the author or user, or terminated by the user agent.
* fetch controller, a [=fetch controller=] (a new [=fetch controller=] by default) @@ -533,10 +534,11 @@ Modify the [=snapshot source snapshot params=] algorithm to set the return value 1. If |prefetchRecord| was given, then: 1. Let |purpose| be a [=structured header/List=] containing the [=structured header/Token=] `prefetch`. 1. If |prefetchRecord|'s [=prefetch record/anonymization policy=] [=prefetch IP anonymization policy/requires anonymity=] for |request|, then: - 1. Add a parameter whose key is "`anonymous-client-ip`" and whose value is true to the `prefetch` token in |purpose|. + 1. Add a parameter whose key is "`anonymous-client-ip`" and whose value is true to the `prefetch` token in |purpose|. 1. The user agent must use a [=connection=] which anonymizes the client IP address (e.g., using a proxy) when fetching |request|, or set |response| to a [=network error=] and [=iteration/break=].

At the moment, how IP anonymization is achieved is handwaved. This will probably be done in an [=implementation-defined=] manner using some kind of proxy or relay. Ideally this would be plumbed down to [=obtain a connection=], and possibly even the mechanism could be further standardized.

+ 1. If |prefetchRecord|'s [=prefetch record/prerendering traversable=] is not null, then add a parameter whose key is "`prerender`" and whose value is true to the `prefetch` token in |purpose|. 1. [=header list/Set a structured field value=] given (`` `Sec-Purpose` ``, |purpose|) in |request|'s [=request/header list=].
@@ -662,13 +664,13 @@ This section contains patches to [[NAVIGATION-TIMING]]. The list of sufficiently strict speculative navigation referrer policies is a list containing the following: "", "`strict-origin-when-cross-origin`", "`strict-origin`", "`same-origin`", "`no-referrer`".
- To prefetch given a {{Document}} document and a [=prefetch record=] |prefetchRecord|, perform the following steps. + To prefetch given a {{Document}} |document| and a [=prefetch record=] |prefetchRecord|, perform the following steps. 1. Let |sourceSnapshotParams| be the result of [=snapshotting source snapshot params=] given |document|. 1. Let |targetSnapshotParams| be the result of [=snapshotting target snapshot params=] given |document|'s [=node navigable=]. 1. Set |prefetchRecord|'s [=prefetch record/source partition key=] to the result of [=determining the network partition key=] given |document|'s [=relevant settings object=]. 1. [=Assert=]: |prefetchRecord|'s [=prefetch record/URL=]'s [=url/scheme=] is an [=HTTP(S) scheme=]. - 1. [=list/Append=] |prefetchRecord| to |document|'s [=Document/prefetch records=] + 1. [=list/Append=] |prefetchRecord| to |document|'s [=Document/prefetch records=]. 1. Set |prefetchRecord|'s [=prefetch record/start time=] to the [=current high resolution time=] for the [=relevant global object=] of |document|. 1. Set |prefetchRecord|'s [=prefetch record/sandboxing flag set=] to the result of [=determining the creation sandboxing flags=] for |document|'s [=Document/browsing context=] given |document|'s [=node navigable=]'s [=navigable/container=]. 1. Let |referrerPolicy| be |prefetchRecord|'s [=prefetch record/referrer policy=] if |prefetchRecord|'s [=prefetch record/referrer policy=] is not the empty string, and |document|'s [=Document/policy container=]'s [=policy container/referrer policy=] otherwise. @@ -768,19 +770,13 @@ The list of sufficiently strict speculative navigation referrer policies`` `Sec-Purpose` `` HTTP request header specifies that the request serves one or more purposes other than requesting the resource for immediate use by the user. -The header field is an [[RFC9651]] Structured Header whose value must be a [=structured header/List=]. Its ABNF is: +The header field is an [[RFC9651]] Structured Header whose value must be a [=structured header/List=]. -``` -Sec-Purpose = sf-list -``` +It may contain an [=structured header/Item=] member which is the [=structured header/Token=] `prefetch`. If so, this indicates the request's purpose is to download a resource it is anticipated will be fetched shortly. -It may contain an [=structured header/Item=] member which is the [=structured header/Token=] "`prefetch`". If so, this indicates the request's purpose is to download a resource it is anticipated will be fetched shortly. +The following parameters are defined for the `prefetch` token: -
TODO: Are there normative implications of this that should be specified here?
- -The following parameters are defined for the "`prefetch`" token: - -* A parameter whose key is "`anonymous-client-ip`". +* A parameter whose key is "`anonymous-client-ip`". If present with a value other than boolean false (`` `?0` `` in the field value), this parameter indicates that the prefetch request is being made using an anonymous client IP. Consequently, servers should not rely on it matching, or sharing a geographic location or network operator with, the client's IP address from which a non-prefetch request would have been made. @@ -792,6 +788,17 @@ The following parameters are defined for the "`prefetch`" token: This specification conforms to this advice; the [=prefetch=] algorithm does not emit non-boolean values.
+* A parameter whose key is "`prerender`". + + If present with a value other than boolean false (`` `?0` `` in the field value), this parameter indicates that the prefetch request is being made in anticipation of a prerender. + +
+ A future specification might define assign more specific meaning to non-boolean values. For now, they are treated the same as true. Implementations are advised not to emit such values. + + This specification conforms to this advice; the [=prefetch=] algorithm does not emit non-boolean values. +
+ +

Security considerations

See Security considerations (Speculation Rules). diff --git a/prerendering.bs b/prerendering.bs index 4cb525b..4e36b72 100644 --- a/prerendering.bs +++ b/prerendering.bs @@ -92,6 +92,8 @@ spec: nav-speculation; urlPrefix: prefetch.html text: supports prefetch; url: supports-prefetch text: list of sufficiently-strict speculative navigation referrer policies text: wait for a matching prefetch record; url: wait-for-a-matching-prefetch-record + for: prefetch record + text: prerendering traversable spec: RFC8941; urlPrefix: https://www.rfc-editor.org/rfc/rfc8941.html type: dfn text: structured header; url: #section-1 @@ -222,32 +224,6 @@ The prerendering getter steps are to return
-Every {{Document}} has prerender records, which is a [=list=] of [=prerender records=]. This is used to fulfill [=navigate|navigations=] to a given URL by instead [=prerendering traversable/activating=] the corresponding prerendering traversable. - -A prerender record is a [=struct=] with the following [=struct/items=]: -* starting URL, a [=URL=] -* No-Vary-Search hint, a [=URL search variance=] (the [=default URL search variance=] by default) -* start time, a {{DOMHighResTimeStamp}} (0.0 by default) -* prerendering traversable, a [=prerendering traversable=] - -
- A [=prerender record=] |prerenderRecord| matches a URL given a [=URL=] |url| if the following algorithm returns true: - 1. If |prerenderRecord|'s [=prerender record/starting URL=] is equal to |url|, return true. - 1. Let |searchVariance| be |prerenderRecord|'s [=prerender record/prerendering traversable=]'s [=prerendering traversable/prerender initial response search variance=]. - 1. If |searchVariance| is not null: - 1. If |prerenderRecord|'s [=prerender record/starting URL=] and |url| are [=equivalent modulo search variance=] given |searchVariance|, return true. - 1. Return false. -
- -
- A [=prerender record=] |prerenderRecord| is expected to match a URL given a [=URL=] |url| if the following algorithm returns true: - 1. If |prerenderRecord| [=prerender record/matches a URL=] given |url|, return true. - 1. If |prerenderRecord|'s [=prerender record/prerendering traversable=]'s [=prerendering traversable/prerender initial response search variance=] is null: - 1. Let |expectedSearchVariance| be |prerenderRecord|'s [=prerender record/No-Vary-Search hint=]. - 1. If |prerenderRecord|'s [=prerender record/starting URL=] and |url| are [=equivalent modulo search variance=] given |expectedSearchVariance|, return true. - 1. Return false. -
- Every {{Document}} has a post-prerendering activation steps list, which is a [=list=] where each [=list/item=] is a series of algorithm steps. For convenience, we define the post-prerendering activation steps list for any platform object |platformObject| as:
@@ -260,6 +236,8 @@ Every {{Document}} has a post-prerendering activation steps Every {{Document}} has an activation start time, which is initially a {{DOMHighResTimeStamp}} with a time value of zero. +

Prerender algorithms

+
[=User agents=] may choose to initiate prerendering without a referrer document, for example as a result of the address bar or other browser user interactions. @@ -271,13 +249,15 @@ Every {{Document}} has an activation start time, which 1. Set |prerenderingTraversable|'s [=navigable/loading mode=] to "`prerender`". - 1. [=Navigate=] |prerenderingTraversable| to |startingURL| using |prerenderingTraversable|'s [=navigable/active document=]. + 1. Let |prefetchRecord| be a new [=prefetch record=] whose [=prefetch record/URL=] is |startingURL|, [=prefetch record/anonymization policy=] is null, [=prefetch record/referrer policy=] is the empty string, [=prefetch record/No-Vary-Search hint=] is the [=default URL search variance=], [=prefetch record/label=] is "`browser UI`", and [=prefetch record/prerendering traversable=] is |prerenderingTraversable|. -

We treat this initial navigations as |prerenderingTraversable| navigating itself, which will ensure all relevant security checks pass. + 1. [=Prefetch=] given |prerenderingTraversable|'s [=navigable/active document=] and |prefetchRecord|. + + 1. [=Navigate=] |prerenderingTraversable| to |startingURL| using |prerenderingTraversable|'s [=navigable/active document=]. - 1. Let |record| be a [=prerender record=] with [=prerender record/starting URL=] |startingURL| and [=prerender record/prerendering traversable=] |prerenderingTraversable|. +

We treat this initial navigation as |prerenderingTraversable| navigating itself, which will ensure all relevant security checks pass. - 1. When the user indicates they wish to commit a navigation to an |activationURL| which is a [=URL=] such that |record| [=prerender record/matches a URL=] given |activationURL|: + 1. When the user indicates they wish to commit a navigation to an |activationURL| which is a [=URL=] such that |prefetchRecord| [=prefetch record/matches a URL=] given |activationURL|: 1. If the user indicates they wish to create a new user-visible [=top-level traversable=] while doing so (e.g., by pressing Shift+Enter after typing in the address bar): @@ -295,9 +275,9 @@ Every {{Document}} has an activation start time, which

- To start referrer-initiated prerendering given a [=URL=] |startingURL|, a {{Document}} |referrerDoc|, a [=referrer policy=] |referrerPolicy|, and a [=URL search variance=] |nvsHint|: + To start referrer-initiated prerendering given a {{Document}} |referrerDoc| and a [=prefetch record=] |prefetchRecord|: - 1. [=Assert=]: |startingURL|'s [=url/scheme=] is an [=HTTP(S) scheme=]. + 1. [=Assert=]: |prefetchRecord|'s [=prefetch record/URL=]'s [=url/scheme=] is an [=HTTP(S) scheme=]. 1. If |referrerDoc|'s [=node navigable=] is not a [=top-level traversable=], then return. @@ -307,26 +287,25 @@ Every {{Document}} has an activation start time, which

This avoids having to deal with any potential opener relationship between the prerendering traversable and |referrerDoc|'s [=Document/browsing context=]'s [=opener browsing context=]. - 1. If |referrerDoc|'s [=Document/origin=] is not [=/same site=] with |startingURL|'s [=url/origin=], then return. + 1. If |referrerDoc|'s [=Document/origin=] is not [=/same site=] with |prefetchRecord|'s [=prefetch record/URL=]'s [=url/origin=], then return.

Currently, cross-site prerendering is not specified or implemented, although we have various ideas about how it could work in this repository's explainers. - 1. [=list/For each=] |record| of |referrerDoc|'s [=Document/prerender records=]: - 1. If |record|'s [=prerender record/starting URL=] is |startingURL|, then return. + 1. [=Assert=]: |prefetchRecord|'s [=prefetch record/prerendering traversable=] is "`to be created`". 1. Let |prerenderingTraversable| be the result of [=creating a new top-level traversable=]. 1. Set |prerenderingTraversable|'s [=navigable/loading mode=] to "`prerender`". - 1. Let |prerenderRecord| be a [=prerender record=] with [=prerender record/starting URL=] |startingURL|, [=prerender record/No-Vary-Search hint=] |nvsHint|, [=prerender record/start time=] the [=current high resolution time=] for the [=relevant global object=] of |referrerDoc|, and [=prerender record/prerendering traversable=] |prerenderingTraversable|. + 1. Set |prefetchRecord|'s [=prefetch record/prerendering traversable=] to |prerenderingTraversable|. - 1. [=list/Append=] |prerenderRecord| to |referrerDoc|'s [=Document/prerender records=]. + 1. Set |prerenderingTraversable|'s [=prerendering traversable/remove from referrer=] to be an algorithm which sets |prefetchRecord|'s [=prefetch record/prerendering traversable=] to null. - 1. Set |prerenderingTraversable|'s [=prerendering traversable/remove from referrer=] to be an algorithm which [=list/removes=] |prerenderRecord| from |referrerDoc|'s [=Document/prerender records=]. +

As with all [=top-level traversables=], the [=prerendering traversable=] can be [=destroy a top-level traversable|destroyed=] for any reason, for example if it becomes unresponsive, performs a restricted operation, or if the user agent believes prerendering takes too many resources. In such cases, the [=prefetch record=] will remain, so it can be used to fulfill future navigations, even if the [=prerendering traversable=] has been destroyed. -

As with all [=top-level traversables=], the [=prerendering traversable=] can be [=destroy a top-level traversable|destroyed=] for any reason, for example if it becomes unresponsive, performs a restricted operation, or if the user agent believes prerendering takes too many resources. + 1. [=Prefetch=] given |referrerDoc| and |prefetchRecord|. - 1. [=Navigate=] |prerenderingTraversable| to |startingURL| using |referrerDoc|, with [=navigate/referrerPolicy=] set to |referrerPolicy|. + 1. [=Navigate=] |prerenderingTraversable| to |prefetchRecord|'s [=prefetch record/URL=] using |referrerDoc|, with [=navigate/referrerPolicy=] set to |prefetchRecord|'s [=prefetch record/referrer policy=].

@@ -445,23 +424,24 @@ Every {{Document}} has an activation start time, which
- To find a matching complete prerender record given a {{Document}} |predecessorDocument| and [=URL=] |url|: + To find a matching prerendered prefetch record given a {{Document}} |predecessorDocument| and [=URL=] |url|: 1. Let |recordToUse| be null. - 1. [=list/For each=] |record| of |predecessorDocument|'s [=Document/prerender records=]: - 1. If |record|'s [=prerender record/prerendering traversable=]'s [=navigable/active document=]'s [=Document/is initial about:blank=] is true, then [=iteration/continue=]. - 1. If |record|'s [=prerender record/starting URL=] is equal to |url|: + 1. [=list/For each=] |record| of |predecessorDocument|'s [=Document/prefetch records=]: + 1. If |record|'s [=prefetch record/prerendering traversable=] is not a [=prerendering traversable=], then [=iteration/continue=]. + 1. If |record|'s [=prefetch record/prerendering traversable=]'s [=navigable/active document=]'s [=Document/is initial about:blank=] is true, then [=iteration/continue=]. + 1. If |record|'s [=prefetch record/URL=] is equal to |url|: 1. Set |recordToUse| to |record|. 1. [=iteration/Break=]. - 1. If |recordToUse| is null and |record| [=prerender record/matches a URL=] given |url|: + 1. If |recordToUse| is null and |record| [=prefetch record/matches a URL=] given |url|: 1. Set |recordToUse| to |record|. 1. If |recordToUse| is not null: - 1. [=list/Remove=] |recordToUse| from |predecessorDocument|'s [=Document/prerender records=]. + 1. [=list/Remove=] |recordToUse| from |predecessorDocument|'s [=Document/prefetch records=]. 1. Return |recordToUse|.
- To wait for a matching prerendering record given a [=navigable=] |navigable|, a [=URL=] |url|, a string |cspNavigationType|, and a [=POST resource=], string, or null |documentResource|: + To wait for a matching prerendered prefetch record given a [=navigable=] |navigable|, a [=URL=] |url|, a string |cspNavigationType|, and a [=POST resource=], string, or null |documentResource|: 1. [=Assert=]: this is running [=in parallel=]. 1. If any of the following conditions hold: @@ -478,20 +458,18 @@ Every {{Document}} has an activation start time, which 1. Let |predecessorDocument| be |navigable|'s [=navigable/active document=]. 1. Let |cutoffTime| be null. 1. While true: - 1. Let |completeRecord| be the result of [=finding a matching complete prerender record=] given |predecessorDocument| and |url|. + 1. Let |completeRecord| be the result of [=finding a matching prerendered prefetch record=] given |predecessorDocument| and |url|. 1. If |completeRecord| is not null, return |completeRecord|. 1. Let |potentialRecords| be an empty [=list=]. - 1. [=list/For each=] |record| of |predecessorDocument|'s [=Document/prerender records=]: + 1. [=list/For each=] |record| of |predecessorDocument|'s [=Document/prefetch records=]: 1. If all of the following are true, then [=list/append=] |record| to |potentialRecords|: - * |record|'s [=prerender record/prerendering traversable=]'s [=navigable/active document=]'s [=Document/is initial about:blank=] is true. - * |record| [=prerender record/is expected to match a URL=] given |url|. - * |cutoffTime| is null or |record|'s [=prerender record/start time=] is less than |cutoffTime|. + * |record|'s [=prefetch record/prerendering traversable=] is a [=prerendering traversable=]; + * |record|'s [=prefetch record/prerendering traversable=]'s [=navigable/active document=]'s [=Document/is initial about:blank=] is true; + * |record| [=prefetch record/is expected to match a URL=] given |url|; and + * |cutoffTime| is null or |record|'s [=prefetch record/start time=] is less than |cutoffTime|. 1. If |potentialRecords| [=list/is empty=], return null. - 1. Wait until the [=navigable/ongoing navigation=] of the [=prerender record/prerendering traversable=] of any element of |predecessorDocument|'s [=Document/prerender records=] changes. - 1. If |cutoffTime| is null and any element of |potentialRecords| has a [=prerender record/prerendering traversable=] whose [=navigable/active document=]'s [=Document/is initial about:blank=] is false, set |cutoffTime| to the [=current high resolution time=] for the [=relevant global object=] of |predecessorDocument|. - -

See also: [=wait for a matching prefetch record=]. The logic for blocking on ongoing prerenders is similar to the prefetch case.

- + 1. Wait until the [=navigable/ongoing navigation=] of the [=prefetch record/prerendering traversable=] of any element of |predecessorDocument|'s [=Document/prefetch records=] changes. + 1. If |cutoffTime| is null and any element of |potentialRecords| has a [=prefetch record/prerendering traversable=] whose [=navigable/active document=]'s [=Document/is initial about:blank=] is false, set |cutoffTime| to the [=current high resolution time=] for the [=relevant global object=] of |predecessorDocument|.
Patch the [=navigate=] algorithm to allow the [=prerendering traversable/activate|activation=] of a [=prerendering traversable=] in place of a normal navigation as follows: @@ -499,13 +477,13 @@ Patch the [=navigate=] algorithm to allow the [=prerendering traversable/activat
In [=navigate=], insert the following steps as the first ones after we go [=in parallel=]: - 1. Let |matchingPrerenderRecord| be the result of [=waiting for a matching prerendering record=] given |navigable|, url, cspNavigationType, and documentResource. + 1. Let |record| be the result of [=waiting for a matching prerendered prefetch record=] given |navigable|, url, cspNavigationType, and documentResource. - 1. If |matchingPrerenderRecord| is not null, then: + 1. If |record| is not null, then: - 1. Let |matchingPrerenderedNavigable| be |matchingPrerenderRecord|'s [=prerender record/prerendering traversable=]. + 1. Let |matchingPrerenderedNavigable| be |record|'s [=prefetch record/prerendering traversable=]. - 1. Let |startingURL| be |matchingPrerenderRecord|'s [=prerender record/starting URL=]. + 1. Let |startingURL| be |record|'s [=prefetch record/URL=]. 1. [=prerendering traversable/Activate=] |matchingPrerenderedNavigable| in place of |navigable| given historyHandling, |startingURL|, url, and navigationId. diff --git a/speculation-rules.bs b/speculation-rules.bs index 9397562..06630e7 100644 --- a/speculation-rules.bs +++ b/speculation-rules.bs @@ -75,6 +75,7 @@ spec: nav-speculation; urlPrefix: prefetch.html text: No-Vary-Search hint; url: prefetch-record-no-vary-search-hint text: label; url: prefetch-record-label text: state; url: prefetch-record-state + text: prerendering traversable; url: prefetch-record-prerendering-traversable text: cancel and discard; url: prefetch-record-cancel-and-discard text: matches a URL; url: prefetch-record-matches-a-url text: prefetch IP anonymization policy; url: prefetch-ip-anonymization-policy @@ -604,7 +605,10 @@ A prerender candidate is a [=struct=] with the following [=struct/ite 1. Let |prefetchRecord| be a new [=prefetch record=] whose [=prefetch record/URL=] is |prefetchCandidate|'s [=prefetch candidate/URL=], [=prefetch record/anonymization policy=] is |prefetchCandidate|'s [=prefetch candidate/anonymization policy=], [=prefetch record/referrer policy=] is |prefetchCandidate|'s [=prefetch candidate/referrer policy=], [=prefetch record/No-Vary-Search hint=] is |prefetchCandidate|'s [=prefetch candidate/No-Vary-Search hint=], and [=prefetch record/label=] is "`speculation-rules`". 1. [=Prefetch=] given |document| and |prefetchRecord|. 1. [=list/For each=] |prerenderCandidate| of |prerenderCandidates|: - 1. The user agent may [=start referrer-initiated prerendering=] given |prerenderCandidate|'s [=prerender candidate/URL=], |document|, |prerenderCandidate|'s [=prerender candidate/referrer policy=], and |prerenderCandidate|'s [=prerender candidate/No-Vary-Search hint=]. + 1. The user agent may run the following steps: + 1. Let |prefetchRecord| be a new [=prefetch record=] whose [=prefetch record/URL=] is |prerenderCandidate|'s [=prerender candidate/URL=], [=prefetch record/anonymization policy=] is null, [=prefetch record/referrer policy=] is |prerenderCandidate|'s [=prerender candidate/referrer policy=], [=prefetch record/No-Vary-Search hint=] is |prerenderCandidate|'s [=prerender candidate/No-Vary-Search hint=], [=prefetch record/label=] is "`speculation-rules`", and [=prefetch record/prerendering traversable=] is "`to be created`". + + 1. The user agent may [=start referrer-initiated prerendering=] given |document| and |prefetchRecord|. The user agent can use |prerenderCandidate|'s [=prerender candidate/target navigable name hint=] as a hint to their implementation of the [=start referrer-initiated prerendering=] algorithm. This hint indicates that the web developer expects the eventual [=prerendering traversable/activate|activation=] of the created [=prerendering traversable=] to be in place of a particular predecessor traversable: the one that would be chosen by the invoking the [=rules for choosing a navigable=] given |prerenderCandidate|'s [=prerender candidate/target navigable name hint=] and |document|'s [=node navigable=].