WIP: File handler for juniper with rocket

[Asone]

Disclaimer : This issue and the associated branch is mostly a personal analytic note around file handling with Rust, Rocket & Juniper more than a formal implementation.

File upload and multipart/form-data specification

File upload on an HTTP requires an implementation of the RFC-7578 which describe multipart/form-data formatted HTTP requests.

GraphQL and file upload specification

Official GraphQL specification does not provide any specification around file upload.

When thinking about it it makes sense as GQL is made to provide a query language more than a protocol whereas file upload seems to be more tied to protocol specifications.

Therefore Juniper does not implement file upload handling and does not intend to do it as per the comment of this issue.

This does not mean there is no approach that could be built to handle file upload with GraphQL. Two main ways tends to be described :

Attach a REST HTTP endpoint

The first one is to create an HTTP REST endpoint that will handle file upload and retrieve an Id or data that will be later attached to a mutation GraphQL request in order to point out to the server which files should be manipulated.

It is an easy solution to implement and quite compatible with Rocket as per the library described in the File upload with rocket paragraph.

There is some trade-off however. As the file upload and the file processing gets splitted, if no mutation request the uploaded file will lie dead on the filesystem unless a mechanism gets implemented to regularly clean the temporary folder from dead files.

Use unofficial specification

Even if there is no official specification, many GraphQL clients and servers uses an unofficial specification to allow handling such feature.

Apollo, Altair and async-graphql seem to rely on this spec to handle uploading files to the server and process said files.

The main trade-off of this implementation is that as being non-standard it can not offer warranty that all gql clients will handle file upload the same way.

Also it bring some technical complexity on top of the original multipart/form-data implementation complexity due to the described structure as we'll see later in the technical solution analysis.

File upload with Rocket

Rocket does not currently provide an easily usable native implementation of multipart handling. This topic is subject to an issue in rocket repo.

The crate rocket-multipart-form-data exists trying to ease file upload handling.

What the above crate allows is to declare specific fields expected for a multipart/form-data request and parse the declared fields to fetch the provided data.

Any data attached to the form that won't have been declared won't be parsed and will be ignored.

Current choice of implementation

I decided to give a try into implementing the unofficial spec implementation.

A few reasons for that :

• Avoiding to have dead files on the filesystem or to have the necessity of building a clean-up job
• Build a file handler that corresponds to the initial main current PoC which was about LN over GQL more than REST itself.
• Implementation seems quite more challenging, so more fun to break my teeth onto such challenge 😄 .

Implementation complexities & Solution approach

Understanding how Juniper handles requests

To provide such implementation we must first understand how Juniper handles requests and the provided content-type.
Scrapping the source code of Juniper we the following code, we can find a FromData guard that pre-processes the request :

https://github.com/graphql-rust/juniper/blob/64fb83f5aa865962527cf4ff691ef277f7147b84/juniper_rocket/src/lib.rs#L345-L367

The first block of interest is the following one :

let is_json = match content_type {
    Some(("application", "json")) => true,
    Some(("application", "graphql")) => false,
    _ => return Box::pin(async move { Forward(data) }).await,
};

We must note how if content-type header is not application/json or application/graphql the request guard forwards to the next guard.

The second block of interest is this one, a few lines below :

Success(GraphQLRequest(if is_json {
      match serde_json::from_str(&body) {
          Ok(req) => req,
          Err(e) => return Failure((Status::BadRequest, format!("{}", e))),
      }
  } else {
      GraphQLBatchRequest::Single(http::GraphQLRequest::new(body, None, None))
  }))

When provided with a request, if the content-type is application/json, Juniper will parse the provided body and forward it to the GraphQLRequest. If the content-type is application/graphql the body will be provided as native request to the GraphQLRequest object. Any other declared content type will be forwarded to the next data guard, which includes our multipart/form-data request.

This is important for our implementation as it shows 2 things :

1. There is no direct way for the fromData to handle the multipart form data request, no matter the content of said request.
2. We will have to provide a GraphQLRequest instance with the part of our request that represents the graphQL query and make the other parts of the request ( i.e : the uploaded files ) accessible during query execution to allow us to process the files.

Unofficial specification data structure

To implement a data guard that will allow us to handle our request we need to look first at what will be the structure of our data sent through the multipart form.

According to the specification, when calling a multipart/form-data request we should be provided with a request that transcripted to json typed should look like this :

{
   "operations" : object,
   "map" : object,
   [key: string | number] : binary
}

• operations field is the field that should contain our graphQL query, providing the queries and/or mutations, the query variables and the operation name(s).
• map should be an object that describes the mapping to the other fields containing the files
• the part annotated as [key: string | number] should be the fields that contains the binary data of the files

Implementation process

We will split the implementation process :

1. Create a parser based on the rocket-multipart-form-data process that will allow us to provide the operations field to the GraphQLRequest object
2. Find a mechanism that will allow us to attach either the raw data or raw pointers to the temp file(s) so the files can be manipulated during the query execution
   3. Find a way to implement safety checks onto the files to avoid processing files that should be rejected automatically ( file size, file type, etc).

[Asone]

First step should be almost complete with this version of the handler : https://github.com/Asone/graphQLN/blob/5c9d43d79a20c95df912428bf3e65ca7c37d3db9/src/graphql/multipart/upload_request.rs

The next steps will probably need to have a good chunk of the code refactored and provided file is still quite dirty

[Asone]

Find a mechanism that will allow us to attach either the raw data or raw pointers to the temp file(s) so the files can be manipulated during the query execution

This version seems to work on my local environment and provide the desired feature.

[Asone]

An apparent functional handler has been pushed here :

https://github.com/Asone/graphQLN/tree/28-handle-file-upload-with-graphql/juniper_rocket_multipart_handler

An example on how to pass the files to the operations execution is provided here :

https://github.com/Asone/graphQLN/blob/01da87010275b6079adbdfc26423358369417beb/src/app.rs#L111-L120