New trait allowing sensors to pass data references

[ddkohler]

With sensors that acquire large amounts of data (>~ MB), the TCP network transfer becomes a bottleneck on acquisition. Writing to file would be a much more performant method.

I want to formally discuss the prospect of a new trait (writes-measured or similar) that allows sensors to pass on references to data, rather than the data itself. The trait would have a new message (get_measured_reference or similar) to retrieve the reference.

Bluesky supports such references, so we should introduce a trait that works well in their system. From what I gather, a single file (let's say hdf5 format) is used for an acquisition, and each measurement Events point to a Datum document that specifies how to extract the specific measurement from the entire file. So a measured_filepath property, with a setter and getter, seem necessary for this trait. The trait's data reference will likely be some form of dictionary with information on how to lookup the data within the file.

For the purposes of streaming data (i.e. not doing an acquisition, but perhaps viewing data live for alignment), it would be desirable to have some option for a writing mode that buffers the last N acquisitions in a file so SMWR can be used to poll the latest element (assuming SWMR works well enough). The buffer is desirable to avoid using up disc space when not needed. Another (easier) option is to introduce a property to turn on/off writing.