Gazebo_IC382_Simulation

This platform provides a simulation environment for Hong Kong Polytechnic University Industrial Centre Course (IC382) Rescue Robot Simulation . The simulation tasks are mainly focus on computer vision and deep learning. The robot should have the ability to control itself in order to reach the goal meanwhile, it should be able to classify the turning directions based on the traffic light pattern.

1. Demo and Layout

1.1 Simulation Field

1.2 Target Performance

2. Virtual Vision Integration

2.1 Basic virtual camera

2.2 Lane Detection based on Hough Transform

3. Deep Learning (object detection) Integration

3.1 Vision based control

3.2 Tensorboard Training Evaluation

The network trained based on ssdmobilienet_v1 with coco dataset.

4. Graph and Visualization

4.1 Tracked robot URDF

4.2 ROS graph

**5. Help **

5.1 Usage of scripts

1. camera_checker.sh: it is a script for you to check whether the python can receive gazebo virutal camera image or not. It will display the raw image from ros topic

2. cv_lane_detector.sh: it is a script to perform lane detection using the virutal camera as a source but it is not compatible for tensorflow (py3)

3. dataset_prepare.sh: it is a script that keep on taking saving pictures in order to help you prepare object detection dataset.

4. fake_velocity.sh: it is a script to publish twist message to control the robot directly.

5. run_gazebo.sh: it is a script to start gazebo and load the world

6. simple_controller.sh: it is a script to allow manual control of the robot.

7. visualize_nodes.sh: it is a script to generate ros graph

8. camera_publish.sh: it is a script to publish the image as a jpg from python2 opencv

9. camera_subscriber.sh: it is a script to show published image and perform lane detection (testing)

10. TensorRT-ROS-Bridge.sh: it is a script which does not belong to ROS. It loads published image and perform AI object detection and other image processsing.

11. vision_control.sh: it is a script to perform PID control based on vision and deep learning.

5.2 Startup Procedures

1. ./run_gazebo.sh (load the world) [ROS]
2. ./camera_publish.sh (publish the image) [ROS]
3. ./TensorRT-ROS-Bridge.sh (enable AI and vision processing module) [NOT ROS]
4. ./vision_control.sh [ROS]

**Remeber to put correct models to ~/.gazebo/models, otherwise the world cannnot be loaded. Also, put budiling_editor_models under ~/

**6. Reference **

TensorRT Object Detection Program Framework -> https://github.com/jkjung-avt/tf_trt_models