作者:Dolly_Vaishnav
编译:东岸因为@一点人工一点智能
自动驾驶汽车是人工智能领域最具颠覆性的创新之一。它们借助深度学习算法不断推动社会发展,并在移动领域创造新的机遇。自动驾驶汽车可以去任何传统汽车可以去的地方,也能像经验丰富的人类驾驶员一样完成各种操作。但是,正确的训练是非常重要的。在自动驾驶汽车的训练过程中,车道检测是其中的一个重要步骤,也是最初要完成的步骤。今天,我们将学习如何使用视频进行车道检测。
01 车道检测步骤简要
车道检测需要检测自动驾驶车辆的行驶路径,并避免进入其他车道的风险。车道识别算法通过分析视觉输入可以识别车道的位置和边界。先进的驾驶辅助系统(ADAS)和自动驾驶车辆系统都对它们有很大依赖。今天我们将讨论其中一种车道检测算法。涉及的步骤如下:
· 捕获和解码视频文件:我们将使用VideoFileClip对象捕获视频,并在捕获初始化后对每个视频帧进行解码(即转换为一系列图像)。
· 图像的灰度转换:视频帧采用RGB格式,将RGB转换为灰度是因为处理单通道图像比处理三通道彩色图像更快。
· 降噪:噪声可能会产生虚假的边缘,因此在进一步处理之前,必须进行图像平滑处理。高斯模糊(Gaussian blur)用于执行此过程。高斯模糊是一种典型的图像滤波技术,用于降低噪声并增强图像特征。权重是使用高斯分布选择的,并且每个像素经过加权平均处理,考虑到周围的像素。通过减少高频元素并提高整体图像质量,这种模糊技术可以创建更柔和、更能满足视觉舒适要求的图像。
· Canny边缘检测器:它在模糊图像的所有方向上计算梯度,并追踪强度变化较大的边缘。
· 兴趣区域:此步骤是仅考虑道路车道覆盖的区域。在这里创建了一个掩码,其尺寸与我们的道路图像相同。此外,对我们的Canny图像的每个像素与该掩码执行按位AND操作。最终掩盖了Canny图像,并显示了由掩码的多边形轮廓追踪的兴趣区域。
· 霍夫线变换:在图像处理中,霍夫变换是一种用于找到基本几何对象(如线条和圆)的特征提取方法。通过将图像空间转换为参数空间,它可以通过累积投票点(voting points)来识别形状。我们将在算法中使用概率霍夫线变换。为了在保持形状检测准确性的同时加快处理速度,霍夫变换已经通过概率霍夫变换进行了扩展,它随机选择一部分图像点,并仅对这些点应用霍夫变换。
· 在图像或视频上绘制线条:在使用霍夫线变换识别我们感兴趣区域的车道线之后,我们将它们叠加在我们的视觉输入(视频流/图像)上。
数据集:为了演示该算法的工作原理,我们将使用一段道路的视频文件。您可以从以下 GitHub 链接下载数据集 - 数据集(https://github.com/rslim087a/road-video)。
注意:此代码在Google Colab中实现。如果您在其他编辑器上工作,您可能需要对代码进行一些修改,因为Colab与OpenCV存在一些依赖性问题。
02 实施道路车道检测的步骤
步骤1:在Python中安装OpenCV库。
!pip install -q opencv-python
步骤2:导入必要的库。
# Libraries for working with image processing
import numpy as np
import pandas as pd
import cv2
from google.colab.patches import cv2_imshow
# Libraries needed to edit/save/watch video clips
from moviepy import editor
import moviepy
步骤3:为算法定义驱动函数。
def process_video(test_video, output_video): """ Read input video stream and produce a video file with detected lane lines. Parameters: test_video: location of input video file output_video: location where output video file is to be saved """ # read the video file using VideoFileClip without audio input_video = editor.VideoFileClip(test_video, audio=False) # apply the function "frame_processor" to each frame of the video # will give more detail about "frame_processor" in further steps # "processed" stores the output video processed = input_video.fl_image(frame_processor) # save the output video stream to an mp4 file processed.write_videofile(output_video, audio=False)
步骤4:定义“frame_processor”函数,在帧上进行所有的处理以检测车道线。
def frame_processor(image):
"""
Process the input frame to detect lane lines.
Parameters:
image: image of a road where one wants to detect lane lines
(we will be passing frames of video to this function)
"""
# convert the RGB image to Gray scale
grayscale = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# applying gaussian Blur which removes noise from the image
# and focuses on our region of interest
# size of gaussian kernel
kernel_size = 5
# Applying gaussian blur to remove noise from the frames
blur = cv2.GaussianBlur(grayscale, (kernel_size, kernel_size), 0)
# first threshold for the hysteresis procedure
low_t = 50
# second threshold for the hysteresis procedure
high_t = 150
# applying canny edge detection and save edges in a variable
edges = cv2.Canny(blur, low_t, high_t)
# since we are getting too many edges from our image, we apply
# a mask polygon to only focus on the road
# Will explain Region selection in detail in further steps
region = region_selection(edges)
# Applying hough transform to get straight lines from our image
# and find the lane lines
# Will explain Hough Transform in detail in further steps
hough = hough_transform(region)
#lastly we draw the lines on our resulting frame and return it as output
result = draw_lane_lines(image, lane_lines(image, hough))
return result
Output:
步骤5:区域选择
到目前为止,我们已经将帧从RGB转换为灰度图像,应用了高斯模糊以减少噪声,并使用了Canny边缘检测。接下来,我们将选择我们想要检测道路车道的区域。
def region_selection(image):
"""
Determine and cut the region of interest in the input image.
Parameters:
image: we pass here the output from canny where we have
identified edges in the frame
"""
# create an array of the same size as of the input image
mask = np.zeros_like(image)
# if you pass an image with more then one channel
if len(image.shape) > 2:
channel_count = image.shape[2]
ignore_mask_color = (255,) * channel_count
# our image only has one channel so it will go under "else"
else:
# color of the mask polygon (white)
ignore_mask_color = 255
# creating a polygon to focus only on the road in the picture
# we have created this polygon in accordance to how the camera was placed
rows, cols = image.shape[:2]
bottom_left = [cols * 0.1, rows * 0.95]
top_left = [cols * 0.4, rows * 0.6]
bottom_right = [cols * 0.9, rows * 0.95]
top_right = [cols * 0.6, rows * 0.6]
vertices = np.array([[bottom_left, top_left, top_right, bottom_right]], dtype=np.int32)
# filling the polygon with white color and generating the final mask
cv2.fillPoly(mask, vertices, ignore_mask_color)
# performing Bitwise AND on the input image and mask to get only the edges on the road
masked_image = cv2.bitwise_and(image, mask)
return masked_image
Output:
步骤6:现在我们将使用概率霍夫变换来识别上述函数输出图像中的直线。
def hough_transform(image):
"""
Determine and cut the region of interest in the input image.
Parameter:
image: grayscale image which should be an output from the edge detector
"""
# Distance resolution of the accumulator in pixels.
rho = 1
# Angle resolution of the accumulator in radians.
theta = np.pi/180
# Only lines that are greater than threshold will be returned.
threshold = 20
# Line segments shorter than that are rejected.
minLineLength = 20
# Maximum allowed gap between points on the same line to link them
maxLineGap = 500
# function returns an array containing dimensions of straight lines
# appearing in the input image
return cv2.HoughLinesP(image, rho = rho, theta = theta, threshold = threshold,
minLineLength = minLineLength, maxLineGap = maxLineGap)
Output:
[[[284 180 382 278]]
[[281 180 379 285]]
[[137 274 183 192]]
[[140 285 189 188]]
[[313 210 388 285]]
[[139 285 188 188]]
[[132 282 181 194]]
[[146 285 191 196]]
[[286 187 379 284]]]
第7步:在视频帧上绘制线条
现在我们已经使用霍夫变换获取了坐标,我们将在原始图像(帧)上绘制它们,但是由于我们得到的坐标超过了2条线,所以我们将首先找到左侧和右侧车道的斜率,然后将它们叠加在原始图像上。
我们在这里定义了4个函数来帮助在输入帧上绘制左侧和右侧车道:
· 平均斜率截距:该函数接收霍夫变换线条并计算它们的斜率和截距。如果一条线的斜率为负,则它属于左车道,否则属于右车道。然后我们计算左车道和右车道的加权平均斜率和截距。
· 像素点:通过使用线的斜率、截距和y值,我们找到线的x值,并返回车道的x和y坐标作为整数。
· 车道线:调用平均斜率截距和像素点的函数,计算出右车道和左车道的坐标。
· 绘制车道线:该函数在输入帧上绘制道路的左车道和右车道。返回输出帧,然后存储在我们的驱动函数“process_video”的变量“processed”中。
def average_slope_intercept(lines):
"""
Find the slope and intercept of the left and right lanes of each image.
Parameters:
lines: output from Hough Transform
"""
left_lines = [] #(slope, intercept)
left_weights = [] #(length,)
right_lines = [] #(slope, intercept)
right_weights = [] #(length,)
for line in lines:
for x1, y1, x2, y2 in line:
if x1 == x2:
continue
# calculating slope of a line
slope = (y2 - y1) / (x2 - x1)
# calculating intercept of a line
intercept = y1 - (slope * x1)
# calculating length of a line
length = np.sqrt(((y2 - y1) ** 2) + ((x2 - x1) ** 2))
# slope of left lane is negative and for right lane slope is positive
if slope < 0:
left_lines.append((slope, intercept))
left_weights.append((length))
else:
right_lines.append((slope, intercept))
right_weights.append((length))
#
left_lane = np.dot(left_weights, left_lines) / np.sum(left_weights) if len(left_weights) > 0 else None
right_lane = np.dot(right_weights, right_lines) / np.sum(right_weights) if len(right_weights) > 0 else None
return left_lane, right_lane
def pixel_points(y1, y2, line):
"""
Converts the slope and intercept of each line into pixel points.
Parameters:
y1: y-value of the line's starting point.
y2: y-value of the line's end point.
line: The slope and intercept of the line.
"""
if line is None:
return None
slope, intercept = line
x1 = int((y1 - intercept)/slope)
x2 = int((y2 - intercept)/slope)
y1 = int(y1)
y2 = int(y2)
return ((x1, y1), (x2, y2))
def lane_lines(image, lines):
"""
Create full lenght lines from pixel points.
Parameters:
image: The input test image.
lines: The output lines from Hough Transform.
"""
left_lane, right_lane = average_slope_intercept(lines)
y1 = image.shape[0]
y2 = y1 * 0.6
left_line = pixel_points(y1, y2, left_lane)
right_line = pixel_points(y1, y2, right_lane)
return left_line, right_line
def draw_lane_lines(image, lines, color=[255, 0, 0], thickness=12):
"""
Draw lines onto the input image.
Parameters:
image: The input test image (video frame in our case).
lines: The output lines from Hough Transform.
color (Default = red): Line color.
thickness (Default = 12): Line thickness.
"""
line_image = np.zeros_like(image)
for line in lines:
if line is not None:
cv2.line(line_image, *line, color, thickness)
return cv2.addWeighted(image, 1.0, line_image, 1.0, 0.0)
Output:
03 实时道路车道检测的完整代码
import numpy as np
import pandas as pd
import cv2
from google.colab.patches import cv2_imshow
#Import everything needed to edit/save/watch video clips
from moviepy import editor
import moviepy
def region_selection(image):
"""
Determine and cut the region of interest in the input image.
Parameters:
image: we pass here the output from canny where we have
identified edges in the frame
"""
# create an array of the same size as of the input image
mask = np.zeros_like(image)
# if you pass an image with more then one channel
if len(image.shape) > 2:
channel_count = image.shape[2]
ignore_mask_color = (255,) * channel_count
# our image only has one channel so it will go under "else"
else:
# color of the mask polygon (white)
ignore_mask_color = 255
# creating a polygon to focus only on the road in the picture
# we have created this polygon in accordance to how the camera was placed
rows, cols = image.shape[:2]
bottom_left = [cols * 0.1, rows * 0.95]
top_left = [cols * 0.4, rows * 0.6]
bottom_right = [cols * 0.9, rows * 0.95]
top_right = [cols * 0.6, rows * 0.6]
vertices = np.array([[bottom_left, top_left, top_right, bottom_right]], dtype=np.int32)
# filling the polygon with white color and generating the final mask
cv2.fillPoly(mask, vertices, ignore_mask_color)
# performing Bitwise AND on the input image and mask to get only the edges on the road
masked_image = cv2.bitwise_and(image, mask)
return masked_image
def hough_transform(image):
"""
Determine and cut the region of interest in the input image.
Parameter:
image: grayscale image which should be an output from the edge detector
"""
# Distance resolution of the accumulator in pixels.
rho = 1
# Angle resolution of the accumulator in radians.
theta = np.pi/180
# Only lines that are greater than threshold will be returned.
threshold = 20
# Line segments shorter than that are rejected.
minLineLength = 20
# Maximum allowed gap between points on the same line to link them
maxLineGap = 500
# function returns an array containing dimensions of straight lines
# appearing in the input image
return cv2.HoughLinesP(image, rho = rho, theta = theta, threshold = threshold,
minLineLength = minLineLength, maxLineGap = maxLineGap)
def average_slope_intercept(lines):
"""
Find the slope and intercept of the left and right lanes of each image.
Parameters:
lines: output from Hough Transform
"""
left_lines = [] #(slope, intercept)
left_weights = [] #(length,)
right_lines = [] #(slope, intercept)
right_weights = [] #(length,)
for line in lines:
for x1, y1, x2, y2 in line:
if x1 == x2:
continue
# calculating slope of a line
slope = (y2 - y1) / (x2 - x1)
# calculating intercept of a line
intercept = y1 - (slope * x1)
# calculating length of a line
length = np.sqrt(((y2 - y1) ** 2) + ((x2 - x1) ** 2))
# slope of left lane is negative and for right lane slope is positive
if slope < 0:
left_lines.append((slope, intercept))
left_weights.append((length))
else:
right_lines.append((slope, intercept))
right_weights.append((length))
#
left_lane = np.dot(left_weights, left_lines) / np.sum(left_weights) if len(left_weights) > 0 else None
right_lane = np.dot(right_weights, right_lines) / np.sum(right_weights) if len(right_weights) > 0 else None
return left_lane, right_lane
def pixel_points(y1, y2, line):
"""
Converts the slope and intercept of each line into pixel points.
Parameters:
y1: y-value of the line's starting point.
y2: y-value of the line's end point.
line: The slope and intercept of the line.
"""
if line is None:
return None
slope, intercept = line
x1 = int((y1 - intercept)/slope)
x2 = int((y2 - intercept)/slope)
y1 = int(y1)
y2 = int(y2)
return ((x1, y1), (x2, y2))
def lane_lines(image, lines):
"""
Create full lenght lines from pixel points.
Parameters:
image: The input test image.
lines: The output lines from Hough Transform.
"""
left_lane, right_lane = average_slope_intercept(lines)
y1 = image.shape[0]
y2 = y1 * 0.6
left_line = pixel_points(y1, y2, left_lane)
right_line = pixel_points(y1, y2, right_lane)
return left_line, right_line
def draw_lane_lines(image, lines, color=[255, 0, 0], thickness=12):
"""
Draw lines onto the input image.
Parameters:
image: The input test image (video frame in our case).
lines: The output lines from Hough Transform.
color (Default = red): Line color.
thickness (Default = 12): Line thickness.
"""
line_image = np.zeros_like(image)
for line in lines:
if line is not None:
cv2.line(line_image, *line, color, thickness)
return cv2.addWeighted(image, 1.0, line_image, 1.0, 0.0)
def frame_processor(image):
"""
Process the input frame to detect lane lines.
Parameters:
image: image of a road where one wants to detect lane lines
(we will be passing frames of video to this function)
"""
# convert the RGB image to Gray scale
grayscale = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# applying gaussian Blur which removes noise from the image
# and focuses on our region of interest
# size of gaussian kernel
kernel_size = 5
# Applying gaussian blur to remove noise from the frames
blur = cv2.GaussianBlur(grayscale, (kernel_size, kernel_size), 0)
# first threshold for the hysteresis procedure
low_t = 50
# second threshold for the hysteresis procedure
high_t = 150
# applying canny edge detection and save edges in a variable
edges = cv2.Canny(blur, low_t, high_t)
# since we are getting too many edges from our image, we apply
# a mask polygon to only focus on the road
# Will explain Region selection in detail in further steps
region = region_selection(edges)
# Applying hough transform to get straight lines from our image
# and find the lane lines
# Will explain Hough Transform in detail in further steps
hough = hough_transform(region)
#lastly we draw the lines on our resulting frame and return it as output
result = draw_lane_lines(image, lane_lines(image, hough))
return result
# driver function
def process_video(test_video, output_video):
"""
Read input video stream and produce a video file with detected lane lines.
Parameters:
test_video: location of input video file
output_video: location where output video file is to be saved
"""
# read the video file using VideoFileClip without audio
input_video = editor.VideoFileClip(test_video, audio=False)
# apply the function "frame_processor" to each frame of the video
# will give more detail about "frame_processor" in further steps
# "processed" stores the output video
processed = input_video.fl_image(frame_processor)
# save the output video stream to an mp4 file
processed.write_videofile(output_video, audio=False)
# calling driver function
process_video('input.mp4','output.mp4')
Final Output:
04 结论
我们介绍了使用Canny边缘检测器和Hough变换来检测道路车道的其中一种方法。其他一些道路车道检测的方法使用了复杂的神经网络和传感器数据。
—— 精彩推荐 ——
3. 自动驾驶视觉感知算法