Tested with python 3.10 on a Stretch RE3 with a tablet tool. Development Notice: The code in this repo is a work-in-progress. The code in this repo may be unstable, since we are actively conducting development. Since we have performed limited testing, you may encounter unexpected behaviors.
Stretch Show Tablet contains ROS2 packages for autonomously presenting a tablet to a human. These packages leverage the human pose estimation deep model from Stretch Deep Perception. The tablet location is specified relative to the human's pose estimate.
These packages are integrated into the Stretch Web Teleoperation repository, and can also be used as a standalone ROS2 action. See the Examples section below for usage examples.
This repository contains two ROS2 packages: stretch_show_tablet and stretch_show_tablet_interfaces. Inverse kinematics are handled using the Pinocchio IK solver from stretch_web_teleop_helpers in the Stretch Web Teleoperation repository.
This package contains the ROS Actions, planning framework, and human perception logic required to perceive humans and plan tablet showing locations.
plan_tablet_pose_service.pyaccepts aPlanTabletPoseservice request and returns the location of a tablet relative to a human for easy viewing.show_tablet_server.pyexposes a ROS Action of typeShowTablet. This action estimates a human's pose, uses the pose estimate to plan a tablet location for easy viewing, and then sends motion commands to thestretch_driverto move the robot's end effector to the target location.
This package contains the service and action message definitions required by the stretch_show_tablet action servers and services.
Clone this repository into your ROS2 workspace, such as:
cd ~/ament_ws/src
git clone https://github.com/hello-lamsey/stretch_show_tabletInstall ROS dependencies and python dependencies using the following commands:
On your robot:
cd ~/ament_ws
colcon build --packages-select stretch_show_tablet stretch_show_tablet_interfacesTODO: add rosdep install instructions for web teleop (IK solver)
On your robot:
cd ~/ament_ws/src/stretch_show_tablet
pip install requirements.txtThere are several ways to interact with the Stretch Show Tablet action server.
The best way to interact with the Stretch Show Tablet action server is through the Stretch Web Teleop application.
TODO: Add details for how to enable tablet showing in web teleop!
In two terminals, run the following commands:
Terminal one: ros2 launch stretch_show_tablet demo_show_tablet.launch.py
Terminal two: ros2 run stretch_show_tablet demo_show_tablet
The demo_show_tablet contains a command line menu system for testing the tablet showing routine. Enter commands to enable the pose estimator and jog the robot's camera to point at a human. You can then send a ShowTablet action request to the show_tablet_server action server. The action server will estimate the human's pose and plan a tablet location for easy viewing.
It is also possible to directly interact with the show_tablet_server action server using the ros2 action send_goal command. For example, to send a ShowTablet action request to the show_tablet_server action server, run the following commands one at a time:
ros2 launch stretch_show_tablet demo_show_tablet.launch.py
ros2 service call /toggle_body_pose_estimator std_srvs/srv/SetBool "{data: True}"
ros2 action send_goal /show_tablet stretch_show_tablet_interfaces/action/ShowTablet number_of_pose_estimates:\ 10The Stretch Show Tablet action uses the human pose estimator from Stretch Deep Perception inside of Stretch ROS2.
The location of the tablet is specified relative to the human's head. The orientation of the tablet is computed using the location of the human's shoulders - in most cases, the tablet's surface is parallel to the plane containing the person's shoulders and the gravity vector.