-
Notifications
You must be signed in to change notification settings - Fork 3
Specification
It is assumed that motorized micromanipulators are connected to the PC. The code is primarily developed for the Luigs & Neumann SM-10 controller, which is connected through USB with serial communication protocol. A typical use case is two XYZ micromanipulators placed on an XY stage, with the Z axis of the microscope (focal plane) also motorized. In addition, a camera can be connected. Finally, one may also have a tactile screen.
An important point is that XYZ coordinates are relative to each device, and in fact to each device axis. For example, the X axis of micromanipulators is oblique.
The goal is to move the manipulators to precise positions relative to the stage or camera view, either:
- clicking on a screen position
- moving the manipulator to a predefined screen position (eg center or other)
In addition, the movements must be safe. A simple idea is: first move on the focal plane, then along the X-axis of the manipulator.
Device coordinates can be mapped to a common coordinate system, typically the stage and microscope system, or the camera system.
- Manual calibration. This procedure assumes that the MPs are mounted on the XY stage. The XY stage and microscope are moved to a reference position. Then the user brings the MP (with a pipette) in the center of the field and in focus. The center is indicated by a cross on the screen, or the user by simply use the camera software, and use a window corner as the point of reference. We record the XYZ coordinates of the MP. Then the user moves the MP to another point, and then the stage and focal plane are moved so as put the pipette in the same visual position as before. With 4 points, we obtain enough information to calculate the transformation between coordinate systems.
To this efficiently and accurately, the best way is to use a 2-step method:
- First
- Second
When the pipette is changed, and in case it has different length, recalibration can be done with just the first step.
- Automatic calibration. The same as manual calibration, but the MP is moved automatically with an image processing algorithm that locates the pipette tip and puts it in focus. This could use for example a corner detector and an autofocus algorithm. This is more difficult when the pipette is not in focus, but we could take an intermediate approach, such as doing the first placement manually, then letting the automatic procedure take place.
Typically, one wants to move the MP in a focal plane, but the X axis of the MP is not aligned with the focal plane. Thus we want to allow movements along "virtual axes" that are within a focal plane. Or more simply, we want to constrain movements within a focal plane. One way would be to have computer controls (keys) that puts the MPs in movement along particular directions. But in fact it is simpler to let the user use the physical interface (knobs) and correct the movements so as stay in a fixed focal plane.
These are movements that involved more than 1 MP. For example:
- Moving 1 or 2 MPs together with the microscope focus (Z).
- Moving the MPs (mounted on stage) in the opposite direction of stage movement, so as to keep them still relative to the microscope.
We could use a gamepad to control the manipulators, for example the Xbox One controller. It features two joysticks plus a pad and several buttons.
Image processing can be done with OpenCV. Kivy allows writing multitouch GUIs in Python.