lane detection, lane centering

Introduction

Traffic accidents have become one of the most serious problems in today’s world. Increase in the number of vehicles, human errors towards traffic rules and the difficulty to oversee situational dangers by drivers are contributing to the majority of accidents on the road. Lane detection is an essential component for autonomous vehiclesLane detection is an important component of advanced driver assistance systems (ADAS) and autonomous vehicles, as it provides information about the road layout and the position of the vehicle within the lane, which is crucial for navigation and safety. Lane detection system is an advanced driver assistance system (ADAS) technology that uses cameras or sensors to identify and track the lane markings on the road. Its primary purpose is to assist drivers in staying within their designated lanes and avoiding unintentional lane departures. The system analyzes the captured images or sensor data in real-time and provides feedback to the driver through visual, auditory, or haptic alerts.

Types of Lane marking

Types of lane boundaries: (a) Dashed. (b) Dashed-solid. (c) Soliddashed. (d) Single solid. (e) Double solid.

A road is any area the public is reasonably allowed to drive on including streets, highways, riverbeds, beaches, wharves and car parks.But in terms of what we conventionally consider to be a road, there are different types:

Single-lane roads

Unmarked: These tend to be called lanes, alleys, backroads and drives. They consist of a narrow road, often barely wide enough for two vehicles to pass, with no or few markings. They might be sealed, but they could be gravel or dirt.

Marked: These tend to be called streets, roads, routes, single carriageways, highways, or boulevards. They consist of one lane in either direction separated by a centre line

Multi-lane roads

Three lanes: a road with a marked overtaking lane with a priority in one direction has a passing lane. The opposite direction can use it if the way is clear and there's no restriction such as bollards, fences or a solid yellow line. Some normal streets or roads in urban areas have two lanes in one direction and one in the other.

Four lanes: These tend to be either dual carriageways (usually roads in urban areas) or expressways (high-speed roads that don't qualify for motorway status). Some 'arterial roads' are dual carriageways, but they could be single carriageways in places. Sometimes the opposing lanes are separated with a median barrier or median strip.

Four or more lanes: street or road (when used in a 50kph urban setting), or motorway when it is high-speed with grade-separated access (i.e. on-ramps and off-ramps)

How it works

Sensor/Camera Acquisition: The system uses one or more cameras or sensors to capture the view of the road ahead. These sensors are typically mounted on the vehicle's front windshield or in other strategic positions.

Image Processing: The captured images or sensor data are processed using computer vision algorithms. These algorithms analyze the pixels or sensor readings to identify lane markings, such as solid lines, dashed lines, or other road boundaries.

Lane Marking Extraction: The system extracts relevant lane markings from the processed data. It distinguishes between different types of lines, such as lane dividers, centerlines, or edge lines, to accurately determine the vehicle's position within the lanes.

Lane Tracking and Positioning: Once the lane markings are identified, the system tracks their position relative to the vehicle's current position. It calculates the lateral deviation of the vehicle from the center of the detected lane and provides feedback to the driver accordingly.

Lane Departure Warning: If the system detects that the vehicle is drifting out of its lane without the driver signaling an intention to change lanes, it issues a warning to alert the driver. The warning can be in the form of visual alerts on the dashboard, audible alarms, or vibrations in the steering wheel or seat.

Lane detection systems can enhance road safety by preventing accidents caused by unintended lane departures, drowsy driving, or distraction. They are often integrated with other ADAS features, such as adaptive cruise control, collision warning, or automatic emergency braking, to provide a comprehensive safety package for vehicles.

Types of Lane marking

Single-lane roads

Marked: These tend to be called streets, roads, routes, single carriageways, highways, or boulevards. They consist of one lane in either direction separated by a centre line

Multi-lane roads

Four or more lanes: street or road (when used in a 50kph urban setting), or motorway when it is high-speed with grade-separated access (i.e. on-ramps and off-ramps)

How it works

Algorithm of Lane Detection

Lane detection in Advanced Driver Assistance Systems (ADAS) involves various algorithms to identify and track lanes on the road. While there isn't a single standard algorithm, the process typically involves several steps:

Image/Video Preprocessing:
- Input frames from cameras mounted on the vehicle undergo preprocessing.
- Common techniques include grayscale conversion, color space transformation, noise reduction, and contrast enhancement to improve lane visibility.

Edge Detection:
- Techniques like Canny edge detection may be used to highlight significant edges in the image that could potentially represent lane markings.

Region of Interest (ROI) Selection:
- Define the area within the image where the lanes are expected to be present.
- Usually, this is the lower part of the image, as lanes are primarily located in that area.

Lane Line Detection:
- Hough Transform: Transform the edge-detected image to a Hough space to identify lines. Lines representing the lanes in the image are then extracted.
- Alternatively, machine learning-based approaches (e.g., deep learning, specifically convolutional neural networks) are increasingly being used to directly predict lane markings.

Lane Fitting and Tracking:
- Once the lane lines are detected, a curve or line fitting algorithm (like polynomial fitting) is applied to determine the mathematical representation of the lanes.
- Kalman filters or other tracking algorithms might be employed to predict and track lane positions across consecutive frames, allowing for a more stable estimation of the lane lines.

Lane Classification and Validation:
- Validate detected lane lines to ensure they are plausible and correspond to actual lanes.
- Classification methods may differentiate between different types of lane markings (solid, dashed, double lines) or identify erroneous detections.
Post-processing and Visualization:
- After detecting and validating lanes, the results are often post-processed to smooth out lane estimations or fill gaps in the detections.
- Finally, the detected lanes are overlaid or annotated on the original image or video to provide a visual representation for the driver or for further processing by other ADAS systems.

Each step can involve variations in algorithms and methodologies, and the effectiveness of lane detection systems often depends on the combination of these techniques and how they are fine-tuned for specific road conditions, lighting, and environmental factors.

Tutorial

# importing libraries
import cv2
import numpy as np
def nothing(x):
pass
cv2.namedWindow("Trackbar")
cv2.createTrackbar("Threshold1", "Trackbar", 0, 255, nothing)
cv2.createTrackbar("Threshold2", "Trackbar", 0, 255, nothing)
# Create a VideoCapture object and read from input file
cap = cv2.VideoCapture ( '4K Seattle Streets - Car Driving Relax Video - Washington State, USA.mp4' )

# Check if camera opened successfully
if (cap.isOpened () == False):
print ( "Error opening video file" )

# Read until video is completed
while (cap.isOpened ()):

# Capture frame-by-frame
ret , frame = cap.read ()
if ret == True:
# Display the resulting frame
cv2.imshow ( 'Frame' , frame )
thresh1 = cv2.getTrackbarPos ( "Threshold1" , "Trackbar" )
thresh2 = cv2.getTrackbarPos ( "Threshold2" , "Trackbar" )

height = frame.shape[0]
width = frame.shape[1]
# region_of_interest_vertices = [(0, height), (width/2, height/1.37), (width-0, height)]
#region_of_interest_vertices = [(0, height), (width/2, height/2), (width-0, height)]
region_of_interest_vertices = [(0 ,height), (width/2, height/1.5), (width - 0, height)]
# Region of interest
def region_of_interest(frame , vertices):
mask = np.zeros_like ( frame )
#channel_count = frame.shape[2]
match_mask_color = 255 # (255,) * channel_count
cv2.fillPoly ( mask , vertices , match_mask_color )
masked_image = cv2.bitwise_and ( frame , mask )
return masked_image

gray = cv2.cvtColor ( frame , cv2.COLOR_BGR2GRAY ) # converting to gray-scale
cv2.imshow ( "gray" , gray ) # displaying the video
blurred = cv2.GaussianBlur ( gray , (15 , 15) , 0 ) # converting to Blur
edged = cv2.Canny ( blurred , thresh1 , thresh2 , 3 ) # Find the edges on Blur as per value in thres1 and2

cropped = region_of_interest ( edged ,np.array ( [region_of_interest_vertices] , np.int32 ) )
cv2.imshow ( "cropped" , cropped )
#cv2.resizeWindow("cropped",500,200)

dilated = cv2.dilate ( cropped , (1 , 1) , iterations = 2 )
cv2.imshow ( "dilated" , dilated )
contours , hierarchy = cv2.findContours ( dilated.copy () , cv2.RETR_EXTERNAL , cv2.CHAIN_APPROX_NONE )

cv2.drawContours ( frame , contours , -1 , (0 , 255 , 0) , 2 )
cv2.imshow ( "countours" ,frame )
#cv2.resizeWindow("countours" ,500, 200)

# Press Q on keyboard to exit
if cv2.waitKey ( 25 ) & 0xFF == ord ( 'q' ):
break

# Break the loop
else:
break

# When everything done, release
# the video capture object
cap.release ()

# Closes all the frames
cv2.destroyAllWindows ()

Output