Advanced Lane Finding

In this project, your goal is to write a software pipeline to identify the lane boundaries in a video from a front-facing camera on a car. The camera calibration images, test road images, and project videos are available in the project repository.

Camera Calibration

Briefly state how you computed the camera matrix and distortion coefficients. Provide an example of a distortion corrected calibration image.

In the camera_calibration.py module, I wrote pipeline that loads the images from the camera_cal directory and applies cv2.findChessboardCorners, cv2.drawChessboardCorners to find the points required by cv2.calibrateCamera which in its turn calculates the camera matrix and distortion coefficients. These values are stored at data/camera_calibration.p, to avoid running calibration for the next steps.

The results can be seem bellow:

Pipeline (test images)

Provide an example of a distortion-corrected image.

In the undistort_image.py module, the calibration data is loaded and the undistort function is exposed. Here is the result of applying to the test_images:

Describe how (and identify where in your code) you used color transforms, gradients or other methods to create a thresholded binary image. Provide an example of a binary image result.

In the binary_threshold.py module, I reused and refactored the code from course material and exposed the following functions:

• color_threshold: combines the thresholds for the HLS channels (25° < H < 93° - emphasis on yellow, L > 88% and S > 88%).

• absolute_threshold: combines the cv2.Sobel thresholds for X (from 21% to 50%) and Y (from 25% to 50%) axis.

• gradient_threshold: combines the cv2.Sobel thresholds for magnitude (from 32% to 57%) and directional (from 0° to 6°) gradients.

• combined_threshold: combines the color, absolute and magnitude gradient thresholds (directional gradient threshold was excluded because I failed to find a range that would boost signal/noise).

Describe how (and identify where in your code) you performed a perspective transform and provide an example of a transformed image.

In the perspective_transform.py module, I exposed the function warp which transforms the input image according to coordinates extracted from the first test image. It employs cv2.getPerspectiveTransform and cv2.warpPerspective.

Describe how (and identify where in your code) you identified lane-line pixels and fit their positions with a polynomial?

In the lane_finder.py module, I expose the function find_lane takes a binary warped image and execute the following steps:

1. If there were previous lines found, applies a mask that will clear any pixel that is not with the tolerance from the previous lines trend;
2. Iterate through windows from the botton to the top of the image:
   1. Calculates the histogram of full pixels for in vertical axis for the current window, using numpy.sum;
   2. Finds the histogram peaks using the indexes function from PeakUtils library;
   3. Sorts the peaks for left and right sides of the image;
   4. Maps the peaks indexes into the image's coordinate system.
3. Creates instances of the Left and Right classes with the found coordinates;
   • The Line base class exposes the get_fit method that should return a second degree polynomial fit;
4. Creates and returns an instance of the Lane class.

Describe how (and identify where in your code) you calculated the radius of curvature of the lane and the position of the vehicle with respect to center.

• The radius of curvature is calculated according the formula provided in project instructions by the method calculate_curvature from the Line class;
• The lane position is calculated by the method get_position from the Lane class;
• To calculate those values in meters, I had to estimate how to scale the image coordinates:
  • In the horizontal axis, I found the separation between lanes (3.67 meters according this document) measured about 820 pixels;
  • In the vertical axis, I found the lane markings length (3.67 meters according this document) measured about 120 pixels;
  • These scaling parameters where defined as constants X_METERS_PER_PIXEL and Y_METERS_PER_PIXEL.

Provide an example image of your result plotted back down onto the road such that the lane area is identified clearly.

Pipeline (video)

Provide a link to your final video output. Your pipeline should perform reasonably well on the entire project video (wobbly lines are ok but no catastrophic failures that would cause the car to drive off the road!)

Video output: output.mp4

Discussion

Briefly discuss any problems / issues you faced in your implementation of this project. Where will your pipeline likely fail? What could you do to make it more robust?

• I spend a long time yak-shaving, since I wanted to make sure to understand and familiarize myself with all the components I leveraged in this pipeline:
  • Python;
  • Numpy;
  • OpenCV;
  • PeakUtils.
• I didn't read and leveraged any particular Python styling guidelines, but I refactored it a few times to improve readability and flexibility;
• A good chunk of the time was also spent in finding and fine tuning the parameters in the binary_threshold.py module;
• I also tried different algorithms for the sliding windows, including the histogram and convolution methods explained in course material, but at the end I got similar, if not better, results with a simple to read approach using the PeakUtils library;
• I am not happy with the curvature values, I would expect very high numbers (close to infinite) for straights, but the position values seem accurate enough in the centimeters order of magnitude;
• We can see in the output video that algorithm doesn't perform well in a few situations:
  • When there are surface transitions (from asphalt to concrete and vice-versa) and when there are shadows in the road:
    • In this situation, there is noise introduced by the vertical axis from the absolute threshold and the magnitude gradient threshold;
  • When there are bumps in the road:
    • I suspect this confuses the smoothing algorithm;
    • Image stabilization could prevent this issue;
• The pipeline also would have trouble if there where visible objects in the road or close to the lines;
• In general, I believe that a more advanced signal processing algorithm could make the pipeline more robust.