Explicitly dedupe prefetches and prerenders

Closes #315. The rule is that the first candidate wins.

prefetch.bs

@@ -133,10 +133,16 @@ spec: no-vary-search; urlPrefix: https://httpwg.org/http-extensions/draft-ietf-h
     text: URL search variance; url: name-data-model
     text: obtain a URL search variance; url: name-obtain-a-url-search-varianc
     text: equivalent modulo search variance; url: name-comparing
+  type: http-header
+    text: No-Vary-Search; url: name-http-header-field-definitio
 spec: resource-timing; urlPrefix: https://w3c.github.io/resource-timing/
   type: dfn; for: PerformanceResourceTiming; text: delivery type; url: dfn-delivery-type
 </pre>
 
+<style>
+dfn var { font-style: italic; } /* override bad base.css */
+</style>
+
 <h2 id="concepts">Concepts</h2>
 
 In light of <a href="https://privacycg.github.io/storage-partitioning/">storage partitioning</a>, this specification defines prefetch for navigations which would occur within the same partition (for example, top-level navigations within the same site) and for navigations which would occur in a separate partition (for example, top-level navigations to a different site).
@@ -299,6 +305,27 @@ The user agent may [=prefetch record/cancel and discard=] records from the [=Doc
     <p class="note">The reasoning for setting the cutoff time *after* waiting for a prefetch record to finish is to allow for flexibility in selecting a prefetch to serve the navigation while still guaranteeing falling back to a non-prefetched navigation in the case of repeated prefetch failures. We allow blocking on prefetch attempts which started before we see an attempt fail, but we don't block on subsequent attempts. Notably, this approach: does not finalize the set of prefetches to block on at the start of the navigation; allows a prefetch which started and completed after the navigation started to serve the navigation; avoids the use of a fixed timeout, which would be arbitrary and detrimental to the use of prefetch with slower servers; and blocks on, at most, two nearly-consecutive prefetches before falling back to a conventional navigation.</p>
 </div>
 
+<div algorithm="has a matching prefetch record">
+  A {{Document}} |document| <dfn export>has a matching prefetch record</dfn> given a [=prefetch record=] |recordUnderConsideration| and an optional boolean <dfn for="has a matching prefetch record" export>|checkPrerender|</dfn> (default false) if the following algorithm returns true:
+
+  1. [=list/For each=] |record| of |document|'s [=Document/prefetch records=]:
+    1. If |record|'s [=prefetch record/state=] is "`canceled`", then [=iteration/continue=].
+
+    1. If |record|'s [=prefetch record/anonymization policy=] does not equal |recordUnderConsideration|'s [=prefetch record/anonymization policy=], then [=iteration/continue=].
+
+    1. If |checkPrerender| is true and |record|'s [=prefetch record/prerendering traversable=] is null, then [=iteration/continue=].
+
+    1. If |record| [=prefetch record/is expected to match a URL=] given |recordUnderConsideration|'s [=prefetch record/URL=], then return true.
+
+      <p class="note">This will take into account either |record|'s [=prefetch record/response=]'s actual [:No-Vary-Search:] header, if it is present, or its [=prefetch record/No-Vary-Search hint=], if the response has not yet been received.</p>
+
+    1. If |recordUnderConsideration| [=prefetch record/is expected to match a URL=] given |record|'s [=prefetch record/URL=], then return true.
+
+      <p class="note">This will only take into account |recordUnderConsideration|'s [=prefetch record/No-Vary-Search hint=], since this algorithm is never called when |recordUnderConsideration|'s [=prefetch record/response=] is popuulated.</p>
+
+  1. Return false.
+</div>
+
 <div algorithm>
     To <dfn export>create navigation params from a prefetch record</dfn> given a [=navigable=] |navigable|, a [=document state=] |documentState|, a [=navigation id=] |navigationId|, a {{NavigationTimingType}} |navTimingType|, a [=request=] |request|, a [=prefetch record=] |record|, a [=target snapshot params=] |targetSnapshotParams|, and a [=source snapshot params=] |sourceSnapshotParams|, perform the following steps.
 
@@ -666,6 +693,7 @@ The <dfn>list of sufficiently strict speculative navigation referrer policies</d
 <div algorithm>
     To <dfn export>prefetch</dfn> given a {{Document}} |document| and a [=prefetch record=] |prefetchRecord|, perform the following steps.
 
+    1. If |document| [=has a matching prefetch record=] given |prefetchRecord|, then return.
     1. Let |sourceSnapshotParams| be the result of [=snapshotting source snapshot params=] given |document|.
     1. Let |targetSnapshotParams| be the result of [=snapshotting target snapshot params=] given |document|'s [=node navigable=].
     1. Set |prefetchRecord|'s [=prefetch record/source partition key=] to the result of [=determining the network partition key=] given |document|'s [=relevant settings object=].

prerendering.bs

@@ -92,8 +92,11 @@ spec: nav-speculation; urlPrefix: prefetch.html
     text: supports prefetch; url: supports-prefetch
     text: list of sufficiently-strict speculative navigation referrer policies
     text: wait for a matching prefetch record; url: wait-for-a-matching-prefetch-record
+    text: has a matching prefetch record
     for: prefetch record
       text: prerendering traversable
+    for: has a matching prefetch record
+      text: checkPrerender
 spec: RFC8941; urlPrefix: https://www.rfc-editor.org/rfc/rfc8941.html
   type: dfn
     text: structured header; url: #section-1
@@ -291,6 +294,8 @@ Every {{Document}} has an <dfn for="Document">activation start time</dfn>, which
 
      <p class="note">Currently, cross-site prerendering is not specified or implemented, although we have various ideas about how it could work in this repository's explainers.
 
+  1. If |referrerDoc| [=has a matching prefetch record=] given |prefetchRecord| with <i>[=has a matching prefetch record/checkPrerender=]</i> set to true, then return.
+
   1. [=Assert=]: |prefetchRecord|'s [=prefetch record/prerendering traversable=] is "`to be created`".
 
   1. Let |prerenderingTraversable| be the result of [=creating a new top-level traversable=].

speculation-rules.bs

@@ -627,6 +627,28 @@ A <dfn>prerender candidate</dfn> is a [=struct=] with the following [=struct/ite
   <p class="note">A user agent's heuristics for enacting non-eager candidates could incorporate a [=prefetch candidate/No-Vary-Search hint=]. For example, a user hovering over a link whose URL and a candidate's [=prefetch candidate/URL=] are [=equivalent modulo search variance=] given the candidate's [=prefetch candidate/No-Vary-Search hint=] could indicate to the user agent that enacting it would be useful.</p>
 
   Notwithstanding the above, user agents should prioritize user preferences (express and implied, such as a low-data-usage mode) over eagerness expressed by the author.
+
+  <div class="example" id="example-multiple-matching-rules">
+    <p>It's possible for multiple speculation rules to generate "the same" speculative navigation request, represented in the above algorithm by multiple similar candidates. For example, consider
+
+    <pre highlight="json">
+    {
+      "prefetch": [
+        {
+          "urls": ["next.html"]
+        },
+        {
+          "urls": ["next.html"],
+          "referrer_policy": "no-referrer"
+        }
+      ]
+    }
+    </pre>
+
+    <p>Because [=prefetching=] or [=start referrer-initiated prerendering|prerendering=] is a no-op if there is already a matching request in progress, following the above algorithm will generally result in the first rule winning. So in our example, the request will be made with the default referrer policy, not <code>[="no-referrer"=]</code>.</p>
+
+    <p>Because of the "<span class="allow-2119">may</span>" step in the algorithm, user agents could technically instead choose to let the second rule win, by skipping the first candidate for [=implementation-defined=] reasons, but this is expected to be rare. Usually, a user agent would either skip none or all of the matching rules, at least within a given [=prefetch candidate/eagerness=] level.</p>
+  </div>
 </div>
 
 <p class="issue">