README.md

Back to Projects List

Touch and pen interactions

Key Investigators

• Kyle Sunderland (Queen's University)
• Jarett Rushmore (Boston University)

Project Description

Objective

Implement and integrate robust interactions using both touchscreen gestures and peripherals such as the Microsoft Surface Pen, and Microsoft Surface Dial.

Approach and Plan

1. Demonstrate and receive feedback on the current state of gesture and peripheral interactions in 3D Slicer
2. Determine the requirements for the utilization of interactions during segmentation
3. Update interactions in VTK and Slicer based on feedback

Progress and Next Steps

• VTK pull request for fixing touchscreen gestures
  • Video of touch interactions in VTK
• Slicer Pull request for adding touchscreen gestures
• Added tablet mode to segment editor
  • Tablet mode rejects regular mouse events and instead only acknowledges tablet events within the segment editor
• Added interaction with button on surface pen to switch between the last active segment editor effect

Feedback

• Would be useful to have a button on the tablet that increments/deincrements through the slices

Issues

• For Microsoft Surface Pen, pen button events are sent as "Windows Key" + F18-F20, however Slicer is only currently detectinbg the "Windows Key" (Qt::Key_Meta)
• Improvements have been added to Qt, starting with the currently Slicer incompatible version of 5.12 (https://bugreports.qt.io/browse/QTBUG-53739)

Result

Segmentation using Surface Pen

Background and References

• WIP Slicer branch
• WIP VTK branch