-
Notifications
You must be signed in to change notification settings - Fork 3
Specification
It is assumed that motorized micromanipulators are connected to the PC. The code is primarily developed for the Luigs & Neumann SM-10 controller, which is connected through USB with serial communication protocol. A typical use case is two XYZ micromanipulators placed on an XY stage, with the Z axis of the microscope (focal plane) also motorized. In addition, a camera can be connected. Finally, one may also have a tactile screen.
An important point is that XYZ coordinates are relative to each device, and in fact to each device axis. For example, the X axis of micromanipulators is oblique.
The goal is to move the manipulators to precise positions relative to the stage or camera view, either:
- clicking on a screen position
- moving the manipulator to a predefined screen position (eg center or other)
In addition, the movements must be safe. A simple idea is: first move on the focal plane, then along the X-axis of the manipulator.
We want to:
- avoid collisions between pipettes, and with the objective
- avoid collisions of pipette on the coverslip
- alert when a given stage position is not reachable by a pipette
Device coordinates can be mapped to a common coordinate system, typically the stage and microscope system, or the camera system.
- Manual calibration. This procedure assumes that the MPs are mounted on the XY stage. The XY stage and microscope are moved to a reference position. Then the user brings the MP (with a pipette) in the center of the field and in focus. The center is indicated by a cross on the screen, or the user by simply use the camera software, and use a window corner as the point of reference. We record the XYZ coordinates of the MP. Then the user moves jointly the MP and stage to another point, so as put the pipette in the same visual position as before. With 4 points, we obtain enough information to calculate the transformation between coordinate systems.
To this efficiently and accurately, the best way is to use a 2-step method:
- A first calibration where the pipette stays within the field of view.
- A second calibration with large movements (eg 2 mm), where results of the first calibration is used to move automatically the pipette to the target position.
When the pipette is changed, and in case it has different length, recalibration can be done with just the first step.
- Automatic calibration. The same as manual calibration, but the MP is moved automatically with an image processing algorithm. The simplest option is to use template matching:
- First the user puts the pipette tip in the center and in focus.
- Photos are taken at different Z positions around this position.
- Template matching is used to locate the tip on the screen.
The same 2-step method as above can be used.
Note that here we are looking for a transformation from (XYZ manipulator, XY stage) -> (XY screen, focal Z).
We could use a gamepad to control the manipulators, for example the Xbox One controller. It features two joysticks plus a pad and several buttons.
Typically, one wants to move the MP in a focal plane, but the X axis of the MP is not aligned with the focal plane. Thus we want to allow movements along "virtual axes" that are within a focal plane. Or more simply, we want to constrain movements within a focal plane. This could be done with the pad of the game controller.
Image processing can be done with OpenCV. Kivy allows writing multitouch GUIs in Python.