Detecting Obstacles Part 1: Finding image features

To detect obstacles we use something called OpenCV. OpenCV is an image processing program that can be used for real time imaging and static image analysis. What this basically means is that it provides an easy interface to perform things like edge detection in an image, or motion tracking etc. This is important for our cameras which we use on our robot, we use these primarily to detect lines.

In fact the application of OpenCV for the most part is pretty easy to describe. We need to have a way to see white lines, thus we use OpenCV to perform line detection. Unfortunately this alone is not all that simple.

Lets consider our environment and the tools we have at our disposal. First, ignoring cones, and other obstacles, we find that we are on a roughly flat plane (this will be important later). The plane is made up of grass and lines. If this was it, actually detecting the lines would be trivial, we would just look at what is white/what ever color the lines are, and we simply avoid those lines. But lets think about the real world for a second

In the real world we have the sun, which can obscure lines that we can clearly see (but a robot may not) due to reflection off of due on the grass, or other natural phenomena. Additionally though we can safely assume that our terrain is roughly flat, grass is not, and may induce a lot of noise even with out extra environmental factors. Thus we can't simply extract the white from an image with even close to 100% accuracy in most common conditions.

Take a look at these examples:

What a wimpy white line.

and look at this one, even for a human it gets hard to tell what color the line is at the extremes of perspective.

This phenomena in Image Processing a lot of the time is referred to as noise. The most common way to get rid of noise in an image? Smooth the image out (smooth out the "bumps" in the image). To this end we apply something you all know called a "blur", any one whose ever used any paint program more advanced than Microsoft's probably knows what this is. In most cases Guassian Blur will do fine, however OpenCV offers a variety of other blurs that may work better under certain noise condition. Additionally lets say you are looking for colors of a certain saturation (for example the lines may be yellow in our circumstance) we could transform the image color space from RGB (red green blue) to HSV (Hue Saturation Value) or to YCbCr(way more complicated, used in lossy image compression). If we were to use HSV we would be able to change the color range we are looking for to be in terms of saturation and value, basically the amount of color in an image and the amount of "lightness" in an image. This way we ignore the dark green grass and only look at light line colors.

What that looks like in pseudo code is as follows.

image get_light_colors(image source)
{	
	image source_hsv = convert_to_hsv(source)
	guassian_blur(source_hsv)
	image dest_hsv(source_hsv.dimensions)
	scalar min_bound(0, 0, 250)
	scalar max_bound(0, 20, 255)
	InRangeS(souce_hsv, min_bound, max_bound, dest_hsv)
	return dest_hsv
}

This will return an image which will have everything that was light colored in white. This is great and all, but just because we know it is as light as our line should be doesn't mean we actually know what we found was a line, and while multiple Gaussian blurs can get rid of the one off bright spots, eventually you'll muddy the image enough you won't be able to figure out what the heck it is (and neither will the robot). What we need is a way to detect if the color we are looking at is actually the type of object we want to avoid, IE a white line. Here is where a concept called Hough Line Transform. I'll let the link explain most of it, but basically for each point x,y, check all possible lines that could go through that point given the polar coordinate equivalent. The more lines that intersect the more likely that those groups of xy points lay on the same line

This technique only works on binary images, luckily our dest_hsv image returns a binary image. Before we run that however, it might be best to run an edge detector. Explaining the specifics of how this edge detector is out of the scope of this section, however computerphile made a great video explaining the technique. All you need to know is that this will detect edges in the image, and any weird one off points will be ignored if they aren't edges. Again this returns a binary image. What we might do now is run a canny edge detector on our binary image, get a binary image back out and then run our line finding algorithm to find lines. What will probably work better in practice is simply running the edge detector, and not using the line finder. The detector will be able to follow curved lines better and if enough care has been done to remove junk artifacts (light reflecting off of grass etc) we shouldn't have a problem getting a binary representation of the white lines in the image.

We can't just run an edge detector on a binary image however, otherwise it will just return the edges we'll either just have to use the line finder or use the edge detector on the original image (but grey-scaled) and & it with the color filter.

Pseudo code is as follows

image get_white_lines(image source)
{
	colors = get_light_colors(source)
	image source_edge = grayscale(source)
	guassian_blur(source_edge)
	image dest_edge(source_edge.dimensions)
	canny(source_edge, dest_edge)
	image final = colors & dest_edge
	return final
}		

With this we will hopefully have the edge or line segments that correspond to the white lines on the ground, but what happens when you have multiple sensing devices and multiple cameras? Our project has just that problem. The solution to having multi domain sensors it to perform projections, which is explained in part 2