Advanced Driver Assistance System

Published Date: 02 Nov 2017

Introduction

In 21st century, car is essential part of our life and is ready to change from luxury to convenience. Intelligent Driver Support Systems will help vehicle drivers to react to changing road conditions which can potentially improve safety. Road signs are an important road asset that is used by drivers to drive under safety regulations to avoid accidents and keep the order on the road tracks. Road and traffic signs must be properly installed in the necessary locations and an inventory of them is ideally needed to help ensure adequate updating and maintenance. With the expansion in road network, motorization and urbanization in the country, the number of road accidents have surged. Road traffic injuries (RTIs) and fatalities have emerged as a major public health concern. RTIs has become one of the leading causes of deaths, disabilities and hospitalizations which impose severe socio-economic costs across the world [1].

Traffic sign detection work was initiated in Japan in 1984. The main objective was to try and use various computer vision algorithms and methods for the detection of the objects in outdoor scenes. Thereafter researches formed various research groups to deal with problem. During the calendar year 2010, there were close to 5 lakh road accidents in India, which resulted in more than 1.3 lakh deaths and inflicted injured on 5.2 lakh persons. These numbers translate into one road accident every minute and one road accident death every 4 minute. More than half of these victims are in active age group of 25-65 years. Following graph in figure 1 gives the total number of accidents took place and people killed from 2001 to 2010 (provisional) [1].

G:\1. Segmentation\Workshops and Conferances\2013\ACIIDS 2013\LNCS-Office2007\fig1bmp.bmp

Fig. . Number of accidents and persons involved

The design and functioning of the road transport system is intended to ensure socio-economically efficient and sustainable transportation. A road traffic sign detection and recognition system could be developed as part of an Intelligent Transport Systems (ITS) that continuously monitors the driver, the vehicle, and the road to inform the driver in just-in-time about upcoming decisions regarding navigation and potentially risky traffic situations.

Advanced driver assistance system

Designing smarter and intelligent vehicles, aiming to minimize the number of accidents due to drivers negligence or wrong-decision, which can be faced during the drive, is one of burning topics of today’s automotive technology. These systems enhances the safety by informing driver of speed limits, road regulations and immediate danger like blind turn, school ahead, railway crossing, road work progress etc. Many systems aiming in enhancing drivers comfort and safety are in computer vision research. Lane departing warning system, pedestrian detection, and traffic sign recognition are few of them.

These types of systems are out of reach of common people, since they are used in few car manufactures like Volvo, BMW and Opel. Mostly such systems require sensors, radars, lidar or transmitters. Development of such systems comes with lots of problems. This includes i) poor visibility because of bad weather, illuminations change and resolution ii) occlusion of road signs and rotation iii) real time execution and speed of performance iv) variation of traffic signs in different countries. Our computer vision approach is based on the camera installed in small and family cars. We intend to give cost effective solution to the problem of road sign detection.

Need of standardization

The rules on the road are both the traffic laws and the informal rules that may be developed over time to facilitate the orderly and timely flow of traffic. Rules on the road are the basic practices and procedures that road users follow; they manage interactions with other vehicles and pedestrians. In 1968 the Europe countries signed an international treaty, called the Vienna convention on road traffic, for the basic traffic rules. The aim of standardizing traffic regulations in participating countries in order to facilitate international road traffic and to increase road safety. A part of this treaty defined the traffic signs and signals. As a result, in Europe the traffic signs are well standardized, although not all countries are participants of these rules and local variations in practice may be found. In spite of appearances of traffic signs being strictly pre-scribed by the Vienna convention, there still exist variations between countries which have signed the treaty. The variations are seemingly irrelevant for a human, but might pose significant challenges for a computer vision algorithm [3]. Figure 2 shows the variations of traffic sign across five different Asian countries related to school, kindergarten and nursery ahead.

G:\1. Segmentation\Workshops and Conferances\2013\ACIIDS 2013\LNCS-Office2007\fig2bmp.bmp

Fig. . "School Ahead" signs in different countries

Similarly figure 3 gives the different road signs which are used to indicate tunnels in various countries.

G:\1. Segmentation\Workshops and Conferances\2013\ACIIDS 2013\LNCS-Office2007\Fig3.bmp

Fig. . Different signs for indicating tunnels in different countries

There is subtle difference between these icons which may not be noticed by human eyes for recognition but may influence the performance of detection algorithm. There is a need to standardize these icons which are used for various informative signs. Use of HCI concepts and human factors in designing these icons could be incorporated.

Literature review

In most of the past traffic sign recognition techniques [4-7], the first work is to detect the location of each traffic sign in an image. There are three major approaches to detecting traffic signs: detection using colour information, detection using shape information, and detection using both colour and shape information. There is a wide range of color based detection techniques used by many researchers. In [8] the technique is based on calculating the distance in RGB space between the pixel colour and a reference colour. They used thresholding approach to segment pixels in a digital image into object pixels and background pixels. [9] Suggested six modules in there system: colour segmentation, edge localisation, RGB differencing, edge detection, histogram extraction, and classification. There algorithm is capable of recognizing the Stop, Yield, and Do-Not-Enter traffic warning signs. Shape is an important attribute of road signs. Therefore, this attribute can be used for sign detection. Using shape features does not require color information and can be achieved in gray scale images. [10] discussed in there paper, an adaptive driver support system named as Driver Advocate , merging various AI techniques, in particular, agents, ontology, production systems and machine learning technologies.

Framework for road sign detection

An intelligent agent is entity acting in a particular environment, whose actions are based on the perceptions received from the environment, in such a way its behavior is considered intelligent by expert. There are several factors that drove the decision to choose the agent architecture. First, the DA system must be robust to unanticipated input and to dynamic reconfiguration [10]. The first step in the detection phase as specified in figure 4 is pre-processing, which may include several operations. This operation corrects an image which is influenced by noise, motion blur, out-of-focus blur, distortion caused by low resolution. Secondly, feature images are extracted from the original image. These feature images contains relevant information of the original image, but in a reduced representation. Thereafter, the traffic signal has to be separated from the background. This can be done with simple segmentation techniques. After the segmentation phase follows feature color and shape feature extraction part. In the last part of the detection phase the potential traffic signs are detected from the segmented images, by using the extracted features of the previous phase.

G:\1. Segmentation\Workshops and Conferances\2013\ACIIDS 2013\LNCS-Office2007\fig44.bmp

Fig. . General framework for road sign detection process

The efficiency and speed of the detection phase are important factors in the whole process, because it reduces the search space and indicates only potential regions. After detection we can further analyze the image with several operations and modify it or extract further necessary information from it. Thereafter, in the recognition phase, the detected traffic signs can be classified into the necessary categories as shown in the figure 5.

G:\1. Segmentation\Workshops and Conferances\2013\ACIIDS 2013\LNCS-Office2007\FIG7.bmp

Fig. . Classification of the detected sign

Proposed system

An Agent is a self-contained software element responsible for performing part of a programmatic process. Therefore such agent contains some level of intelligence and performs their task independently and may cooperate with other agents. A system to detect and recognize road and traffic signs should be able to work in real time and in all circumstances. This part is designed by using Microsoft visual C++ 6.0 with OpenCV image development package. Some heuristics are used to identify the shapes. Proposed method work on similar framework, where circular and triangular signs are detected using color thresholding. This is a non-linear operation that converts a gray-scale image into a binary image where the two levels are assigned to pixels that are below or above the specified threshold value.

Video captured from camera is queried for each frame to continuously obtain any traffic sign on the road. Color space is changed from RGB to HSV before image filtering is performed. We constantly search for red color in the left-half region of video frame, since road signs are located at the left side of the road in India. Figure 6 gives two snapshot of the triangular road sign detected. Detected signs are cropped and will be displayed on TFT monitor for driver assistance. Here we have shown the cropped road sign in the separate window on the right side to signify detected sign.

G:\1. Segmentation\Workshops and Conferances\2013\ACIIDS 2013\LNCS-Office2007\fig5.bmp

Fig. . Triangular road sign detection

Similarly Hough transform is used to detect circles in a grayscale image using Intel’s OpenCV libraries. The process of extracting circular signs in given below.

1. Load an image

2. Convert it to grayscale

3. Apply a Gaussian blur to reduce noise and avoid false circle detection

4. Apply Hough Circle Transform

5. Draw the detected circular road sign

6. Display the detected circular red traffic sign

Few snapshots of circular signs taken at different location are shown in figure 7.

G:\1. Segmentation\Workshops and Conferances\2013\ACIIDS 2013\LNCS-Office2007\fig6.bmp

Fig. . Circular road sign detection

System developed gives results for all types of circular and triangular signs. Signs which are placed on the roads and are not traffic signs are also falsely detected by this method. User interface agent will help driver to get speech output of the road sign detected by the system.

Conclusions

The traffic sign recognition is a very helpful driver assistance technique for increasing traffic and driver safety. The future Intelligent Vehicles would take some decisions about their speed, trajectory, etc. depending on the signs detected. In this paper, we have presented agent based traffic sign detection method which is often based on the general framework discussed. Currently the video was captured using a digital camera and the result was shown by supplying video stream to the system. Proposed method gives good results for triangular and circular road signs detection.