editing for butterfly webserver

[thejohnhoffer]

Summary

These changes allow real-time collaborative editing through a websocket connection to the Butterfly server. Butterfly currently serves a static build of Neuroglancer (from this repository) to give Harvard researchers an interface to view and modify terabytes of scanned and/or labeled volumes.

Background

	Here is a 2D section from a labeled volume. Each label should represent a neuron, but often the labels need to be combined (merged) or divided (split) in order to truly mark a single complete brain cell.

Our goal is to perform two operations on a labeled volume:

1. Merge 2+ labels into one new label
2. Split One label into 2+ subregions

The butterfly server allows multiple concurrent users to send and receive merge and split information following by the Websocket API described here. The Neuroglancer edits here:

1. Define UI to merge any number of neighboring labels in 2D or 3D
2. Send merge and split requests to butterfly using the Websocket API
3. Apply all merge commands to visually join labels in all 2D or 3D views

The user interface for splitting labels is still under development.

Admittedly, the fork of Neuroglancer developed by the Seung Lab lets users merge and split labels. After reading the code and reaching out to them, we learned their branch relies on several remote-procedure calls specific to their custom graph server that requires a hierarchical merge tree representation of the labels. Such a representation was considered infeasible by our group.

So, this branch implements a simple websocket interface to manage edits stored in the butterfly webserver. While we could target a different open-source server such as the Boss, our butterfly server reads directly from several file formats used daily by members of our group. Most importantly, the butterfly server presented the quickest solution for writing the backend of the websocket interface.

Interface

The Websocket connection URL is constructed in third_party/dojo/websocket.js. An example URL of wss://dojo.rc.fas.harvard.edu/ws/JWR::2018_01_18::8x8y8z/id matches:

${wss}://${host}/ws/${key}/${channel}

Where:

• {wss} protocol determines whether (wss) or not (ws) to use SSL encryption
• {host} provides the hostname of the NDStore datasource
• {key} gives an arbitrary string identifying the resource location
• {channel} names a specific image source within a given resource

The data exchanged between server and client meets the following format:

{
  "error": "",
  "action": "save",
  "merge": [
     ["id","id","id"],
     ["id","id","id","id"],
     ....
  ],
  "split": [
    "id:sub:x0:x1:y0:y1:z0:z1",
    "id:sub:x0:x1:y0:y1:z0:z1",
     ....
  ]
}

To send a subset of available data, use "action": save. To send all data, use "action": restore. The value of "error": can be any arbitrary string. No changes to client or server state should occur for error values other than "".

The strings id,sub,x0,x1,y0,y1,z0, and z1 should all be standard base-10 string representations of integers.

The value of "merge": can be an empty array [], or an array of arrays of id values defining image labels to be joined into a single label. Example: [["1","2","3"],["9","100"]]

The value of "split": can be an empty array [], or an array of strings of id:sub:x0:x1:y0:y1:z0:z1 values defining image labels to be joined into a single label. A new label sub should be applied to all voxels that match 3 conditions:

1. Occur within an existing id label in the original data
2. Have all full-resolution coordinates above x0,y0,z0
3. Have all full-resolution coordinates below x1,y1,z1

Example: ["1:0:50:50:50:100:50:50","1:1:0:0:0:100:100:100"]

TODO

• Allow websocket editing for all supported API, not only NDStore
• Correctly implement UI for "split:" commands, not only "merge:" commands
• Develop an API for compressed binary data, not only a string representation