SNE-RoadSeg: Incorporating Surface Normal Information into Semantic
Segmentation for Accurate Freespace Detection
Abstract
Freespace detection is an essential component of visual perception for
self-driving cars. The recent efforts made in data-fusion convolutional
neural networks (CNNs) have significantly improved semantic driving
scene segmentation. Freespace can be hypothesized as a ground plane, on
which the points have similar surface normals. Hence, in this paper, we
first introduce a novel module, named surface normal estimator (SNE),
which can infer surface normal information from dense depth/disparity
images with high accuracy and efficiency. Furthermore, we propose a
data-fusion CNN architecture, referred to as RoadSeg, which can extract
and fuse features from both RGB images and the inferred surface normal
information for accurate freespace detection. For research purposes, we
publish a large-scale synthetic freespace detection dataset, named
Ready-to-Drive (R2D) road dataset, collected under different
illumination and weather conditions. The experimental results
demonstrate that our proposed SNE module can benefit all the
state-of-the-art CNNs for freespace detection, and our SNE-RoadSeg
achieves the best overall performance among different datasets.