Allow opt-in for bundling data requests together from client-side

[shamsup]

Creating a discussion for a conversation from discord that was in the supporters channel

It would be awesome if there was a way to bundle multiple loader requests together during client-side navigation and for refetching after actions for environments like AWS lambda. The current model almost guarantees that one or more requests will have a cold start time because they are concurrent synchronous invocations with a regular API gateway integration. A single lambda instance can't handle concurrent requests, so firing off 4 fetches forces there to be 4 instances available.

We actually did it that way in the beginning but it makes cookie paths unpredictable, you can't cache a specific loader (there's a huge cartesian product of URLs if we sent every route to a single endpoint to get all the data) and we wouldn't be able to put each route on a different lambda (in the future)--which is actually more in line with a lambda architecture.
— @ryanflorence in a reply in discord, added here for context

Regarding Cookie Paths

With the way loader requests work currently, the requests are sent from the path the user is currently on, and that would not change based on bundling all simultaneous loader fetching into a single request.

Regarding Caching

Remix' current pattern is relatively good for caching individual loaders while adhering to the constraint of the cookie paths, although it does mean that the root loader data will already need to be cached once per route in the app in order to have a cache hit. For example, GET /users/1?_data=root and GET /users/1/settings?_data=root would already create a cache miss in most child routes for refreshing the root loader data.

To illustrate, if you are on a page, /users/1/settings, and you POST to your action, Remix will send off these requests individually:

GET /users/1/settings?_data=root
GET /users/1/settings?_data=routes/users
GET /users/1/settings?_data=routes/users/$id
GET /users/1/settings?_data=routes/users/$id/settings

In a typical client-side navigation scenario, the user might follow a pattern like / -> /users/1 -> /users/1/settings to get to this page initially. This would result in only the last request in the list possibly being a cache hit for the current user, and we'd probably want to ignore or prevent that anyway, since we're probably modifying that data in our action.

This multiple request pattern, while slightly better for CDN and browser caching, is incredibly bad for AWS Lambda-based environments, since an AWS Lambda instance can only handle 1 concurrent request at a time. This model guarantees that however many requests miss a caching layer will require a new Lambda instance. In the example, only the last request would possibly be a cache hit, so at least 3 available instances would be required to serve this single user action (1 for the action, which could be reused for 1 loader request, then 2 others for the other loader cache-misses). This leads to higher possibility of cold starts (higher latency), higher invocation count (higher cost), and less ability to optimize using caching patterns like singletons, in-memory LRU-caches, and per-request dataloader since the caches would not be shared across the instances that are most likely accessing related data.

I'm proposing that I should be able to opt into merging those 4 requests into a single request:

GET /users/1/settings?_data=routes/root&_data=routes/users&_data=routes/users/$id&_data=routes/users/$id/settings

While it is slightly less cache-friendly because individual loaders are not cached for this single path, this would be a huge cost and performance optimization for Remix on AWS Lambda, especially since remix-workloads (in my experience) are mostly idle waiting on network I/O rather than CPU-bound where handling multiple loaders would cause a bottleneck.

This pattern would still offer the ability to use other opt-in performance optimizations remix provides, such as shouldReload, and offers more flexibility for other optimizations.

Regarding Routes as Separate Lambda Functions

This would be an awesome opt-in for specific routes, like resource routes that use special libraries such as image optimization or generating PDFs. However, I don't see a huge benefit if the functions are created based on pathname outside of those uses. As illustrated in the example, the routes/users/$id/settings function would still need all the code and libraries required for any parent route loaders in order to adequately handle the other requests while adhering to the cookie path constraint. Removing special libraries from the general server bundle would be great for cold-starts and reducing bundle size, but I don't see the impact going much further than that. Additionally, spreading traffic across multiple different functions reduces the potential for reusing instances, although it would mean you can provision different concurrency for hot paths and less common paths. In my opinion, this implementation complexity (for the Remix team) would be a wash, since it would be easier just to create a separate image-resizing or PDF-generating function that I could call from a resource route.