-
Notifications
You must be signed in to change notification settings - Fork 0
Home
This Wiki outlines the process of collecting and processing data for YOLO training. This can be done from any host machine, but the tutorial assumes MacOS.
- Clone the TAUV-Tools GitHub repository. Run
$ git clone https://github.com/Tartan-AUV/TAUV-Tools.gitin terminal. - Clone the VideoLabeling GitHub repository. Run
https://github.com/Tartan-AUV/VideoLabeling.gitin terminal. - Make sure all necessary libraries and packages are installed on the host machine. Lists of these can be found in the Prerequisites section of the VideoLabeling repository README.md and the Prerequisites section of the video_extraction directory README.md in TAUV-Tools.
- Download VLC or some other .mp4 viewer.
This assumes that video data has been collected and saved to the Jetson as rosbag data.
- Turn on the Jetson. Plug in a battery to the submarine powerboard or connect it to the e-bench power supply (set to 6 amps and 15 volts, connecting black before red). Insert the key, and wait to see that the Jetson lights turns on.
- Turn on the router. Connect the power cable (plugs in to the left side of the box), switch on the router (switch on left side of the box next to the power plug), and see that the router lights turn on.
- Connect to the network. Join the
submariners-5gnetwork from your host machine (password istartanauv). Connect the Jetson to the router via ethernet (pink cable directly from the router to either the Jetson or front OAK-D camera). A light on the router numbered 1 - 4 (corresponding to the ethernet port the Jetson is attached to) and the green light by the ethernet port on the Jetson should each be lit if connected correctly. - ssh to the Jetson. Use command
$ ssh tartanauv@tartanauv(password istauv). In the case that this raises the following error (or similar), use command$ ssh [email protected]. If you have any other trouble with this step, ensure that the the Jetson is connected via ethernet correctly as described in step 3.
ssh: Could not resolve hostname tartanauv: nodename nor servname provided, or not known
- Plug SanDisk ssd into Jetson.
- Mount the file system as follows:
- Run
lsblkto list all connected drives, and find the one that follows/dev/sdx1.- This will most likely be
/dev/sda1but can be/dev/sdb1,/dev/sdbc1, or so on.
- This will most likely be
- Run
sudo mount -o rw,uid=1000,gid=1000,user,exec,umask=003 /dev/sda1 /home/tartanauv/datato mount the ssd to the Jetson file system at location/home/tartanauv/data.- Replace
/dev/sda1with whatever name you found on the first command.
- Replace
- Run
- Find the path to rosbag data. This is likely
/home/tartanauv/shared/bags/testbut may be different depending on the specified location at the time of rosbag collection. Ensure all bags are labeled with date of test asYYYY-MM-DD-xx-xx-xx_xx.bagby runningls <path_to_bags>. - Run
sudo rsync -P -r <path_to_bags> /home/tartanauv/data, replacing<path_to_bags>as found in step 3. This copies the folder containing rosbag data to the mounted ssd. There will be a progress bar for each individual bag.
- Plug SanDisk ssh into host machine.
- Run
diskutil listto list all partitions of the host disk. Find the/dev/diskXlisted as (external, physical) that matches description of the ssd. An example using an ssd named "ExtremeSSD" with identifierdisk4s1follows.
% diskutil list
dev/disk0 (internal):
...
/dev/disk4 (external, physical):
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *2.0 TB disk4
1: Microsoft Basic Data ExtremeSSD 2.0 TB disk4s1
- Run
diskutil mount /dev/diskXs1. This will mount the ssd to the host MacBook file system, located at/Volumes/<ssd_name>. A confirmation like as follows will be printed.
Volume ExtremeSSD on /dev/disk4s1 mounted
/Volumes/<ssd_name>/<bags_folder_name> will be the input directory for video_extraction.
- Navigate to
~/TAUV-Tools/video_extractionin your host machine terminal. - Run
python3 export_images.py <input_dir> --output_dir <output_dir>. This command using the running example directory names is as follows, with an output folder namedproclocated on the ssd.
python3 export_images.py /Volumes/ExtremeSSD/test --output_dir /Volumes/ExtremeSSD/proc
Running this script creates a directory for each of the collected rosbags containing a subdirectory of .png images and an .mp4 video for each of the collected rostopics (usually darknet detections, depth, and color from front and bottom cameras). More details on specific prerequisites can be found in the README.md of the video_extraction directory in TAUV-Tools. Note that the output directory of video_extraction will serve as the input directory for get_yaml_folder in the next step.
- Navigate to
~/VideoLabelingin your host machine terminal. - Ensure that the
data_classes.txtanddefault.yamlare each up to date with all necessary classes for training (and that they match). If another data class name must be added, simply add another line with the name to the end ofdata_classes.txtand add the following to the end ofdefault.yaml.
<data_class_name>
- [-1, -1]
- Run
python3 get_yaml_folder.py <input_dir> <output_dir_mp4> <output_dir_yaml>. This script takes the output ofvideo_extractionand fills two directories: (1) a folder containing each of the .mp4 files located throughout the input directory (at the location specified by<output_dir_mp4>) and (2) a folder containing blank yaml files corresponding to each of these videos (at the location specified by<output_dir_yaml>). - Load the folder of of the videos in an .mp4 viewer (VLC works nicely here), and for each video do the following.
- Open the video's corresponding .yaml file in a text editor. This file will have the same name as the video, just with the .yaml extension.
- Step through the video. For each interval of time that any object of a data class appears in the frame, make an annotation beneath that data class name on the .yaml file. The default .yaml file has the
[-1, -1]written beneath each data class name, which indicates that it does not appear at all throughout the video. Overwrite this line for any object that does appear in the video, and leave it for any object that does not. The following is an example .yaml file for a video labeling cats and dogs in which a cat appears twice (from second 1 to second 4 and again from second 6 to second 10) and no dog appears.
cat
- [1, 4]
- [6, 10]
dog
- [-1, -1]
At this point, you should have edited each .yaml file according to its corresponding video, and you're ready to trim the videos such that each interval is accounted for. Note that the <output_dir_mp4> directory containing .mp4 videos and the <output_dir_yaml> directory containing the updated .yaml files will be used as input to trim_all in the next step.
- Navigate to
~/VideoLabelingin your host machine terminal. - Run
python3 trim_all.py <yaml_dir> <mp4_dir> <output_dir>.<mp4_dir>should be the folder of all videos output byget_yaml_folder, and<yaml_dir>should be the path to the folder containing the .yaml files you've updated in the last step.
This script will take each of the intervals indicated in each .yaml file and create trimmed videos in which a set of objects is present the entire time. Note that the output directory of this step will be the input to find_all_bb in the next.
- Navigate to
~/VideoLabelingin your host machine terminal. - Open
find_bb.pyin a text editor. Ensure that, from line 160 onward, each of the data classes accounted for indata_classes.txthas a corresponding key assigned for its identification. If you must add a key to another data class, avoid the keysnandq, as these have different key bindings. Add the following to the end of the file and before theelse:statement.
elif chr(key) == <unused_char>
current_class = <data_class_name>
- Run
python3 find_bb.py -hto see the directions for labeling the first set of bounding boxes for each trimmed video. Familiarize yourself with the commands. You can practice placing bounding boxes on a singular video by runningpython3 find_bb.py <path_to_video>. - Run
python3 find_all_bb.py <trimmed_videos_dir>. The path<trimmed_videos_dir>should be the output directory oftrim_allfrom the prior step that contains each of the trimmed videos. This script will callfind_bbfor each of the videos in the input directory, automatically loading each frame that you must manually annotate.
This script will create a _rects.txt file that corresponds to the bounding boxes of each trimmed video, adding it to <trimmed_videos_dir>. These are necessary for input to track_all in the next step.
When running find_all_bb, once you hit q to save a specific set of boxes, you can't adjust them. If you notice you've made a mistake, take note of the name of the incorrectly labeled video. You can go back and overwrite the incorrect _rects.txt file by running python3 find_bb.py <path_to_video>.
- Navigate to
~/VideoLabelingin your host machine terminal. - Run
python3 tracking.py -hto see ...
...
...
...
...
...
...
...
...
...
...
...
...
...