IoT programming to Develop Self Driving Robotics Car using OpenCV and Other Emerging Technologies

—The advancement of Artificial Intelligence & Deep Learning has catalyzed the field of technology. The progression in these fields is exponentially increasing, and the discoveries which were once just an imagination are now changed into reality. The evolution of cars each year has made a lot of difference in people travelling from one place to another. One such reform involving Artificial Intelligence & Deep Learning is the birth of a self-driving car. The future is here where one can reach their destination hassle-free safely without the fear of accidents. This paper introduces a practical model of the self-driving robotics car, which can travel from one position to another on different types of tracks. A Pi-camera module is attached with the help of Raspberry Pi, which sends series of image frames to the Convolutional neural network, which then foretells the car to move in a specific direction, i.e. right, left, forward and reverse direction. The outcome is the robotics car, which travels in the desired direction without any individual effort.


I. INTRODUCTION
An autonomous vehicle, a self-driving car, is a connected automobile that relies on hardware, software and machine learning to travel in various conditions considering all the obstacles and road conditions utilizing real-time sensory data [1].Today, Tesla, Google, Uber, and General Motors are all trying to create their self-driving cars that can run on real-world roads.Many analysts predict that within the next five years, we will start to have fully autonomous vehicles running in our cities.Within two to three decades, nearly all cars will be fully independent.There are various levels of different automation, according to SAE International [2].Each level depends upon the human involvement associated with handling the automobile, which is as follows No Automation (level 0), Driver Assist (level 1), Partial Automation (level 2), Conditional Automation (level 3), High Automation (level 4), Full Automation (level 5).A lot of failures do exist in different models, but each day due to more enhanced techniques and algorithms, prospects to create high chances of yielding better results are more.
The technology inside self-driving cars considers many important factors like who monitors the road?Steering, acceleration, declaration, monitoring surroundings, fallback for self-driving failures, automation taking full control.Motor vehicle accidents are the primary cause of death of teenagers in the United States-with as many as 8,000 deaths per year have been caused by drivers aged between 16-20.Statistics in the US show the largest percentage of vehicle fatalities are by teens, and according to study, American teen drivers cost the Americans over $34 billion annually.Many metropolitan cities, larger states, and federal agencies are actively encouraging autonomous vehicle deployment, but this is not universal; this is because the problem arises on trusting on an autonomous vehicle.A lot of challenges are faced in the self-driving industry each day, and many new issues arise after applying regressive testing methodologies, here are a few challenges faced namely, i.
It is challenging to handle real-time obstacles for self-driving cars. ii.
Driving requires many complex social interactions.
iii.Bad weather makes everything trickier.i. Level 0 -A car with absolutely no automation is termed as level 0. ii.
Level 1 -Drivers Assistance, the features here include rear camera, lane discipline assistance which tells driver to follow certain traffic laws and adds a few safety features.iii.
Level 2-The automated system takes control on driving aspects such as steering, acceleration and deceleration when approached a pedestrian or an obstacle.The driver however can manage everything.
Level 3 -Uber's self-driving car is an example of Level 3 because while the carrier controls most of the navigation, but still the driver is required.v.
Level 4 -Level 4 is defined as high automation.
Google's Waymo the autonomous vehicle company comes under Level 4 autonomy.vi.
Level 5 is termed as full automation; the self-driving car has complete control over all systems.
For partial and high-level automation i.e.Level 4 and Level 5 self-driving autonomous vehicles, the AI system will have full control of all the systems.In theory, the AI system will not get drunk and will not get exhausted while driving a car and is considered as justification for pursuing actual self-driving cars.
No more intoxicated drivers because of sophisticated AI-driven self-driving cars.No more human faulty chauffeurs that snap and opt to drive into a roadside fence.Human decisions and individual exemptions will go out the window, and driverless cars will not get angry.
This paper introduces partially level 4 and level 5 based automation, as it has features of both high and full automation.The self-driving car system consists of three subsystems: input system (camera, ultrasonic sensor), processing unit (computer) and RC car control unit.

A. Input Unit
A Raspberry Pi board (model B+), connected with a Pi camera module and an HC-SR04 ultrasonic sensor is used to obtain input data.Two python programs run on Raspberry Pi for streaming colour video and ultrasonic sensors to send data to the computer via a network, which is a local Wi-Fi connection, connected to the same network.In order to produce low latency video streaming, video resolution is reduced or scaled down to QVGA (320×240) resolution.

B. Processing Unit
The processing unit (computer) manages various tasks: receiving data from Raspberry Pi, neural network training and prediction(steering), object detection (stop sign), distance measurement (using the ultrasonic sensor), and sending directions to Arduino through the USB connection.

TCP Server
A TCP server program runs on the computer to receive the video, which converts the video into streamed image frames and also collects ultrasonic sensor data from the Raspberry Pi.
Image frames are transformed into grayscale and are decoded into numpy arrays, which are then fed into the neural network.

Neural Network
One benefit of using a neural network is that once that it is trained, and it only needs to load trained parameters later.Once trained, prediction can be very fast, and only one part of the image frame is used, which is the lower half of the input image frame for training and then predicts direction to turn.
There are approximately 38,000 nodes in the input layer, and then only 32 nodes are used in the hidden layer, and these are selected relatively arbitrary.The output layer constitutes four layers where every node assigns steering control instructions: left, right, forward and reverse sequentially.

Object Detection
This paper adopted the shape-based approach and uses Haar feature-based cascade classifier algorithm for object detection.
Considering every object needs its own classifier and understands the same process in training and detection, this paper is only centered on stop sign detection.OpenCV offers a trainer as well as a detector.Positive samples (containing target object) were acquired using Pi camera, and the captured video is converted to image frames and cropped to get the desired object which are visible.Negative samples (without target object), are obtained without using obstacles.Using the cv2 package, which supports OpenCV in python, the frames are used to apply different image processing techniques to get the desired object.These are fed into the network for training and get desired results.In this paper, the execution is distributed into different steps: the pi camera module, which is connected to Raspberry Pi captures the video, this captured video is converted into image frames which are fed into the convolutional neural network.

III. TECHNOLOGY -SELF DRIVING CAR
Various algorithms are applied into the network for several epochs.The algorithm predicts the direction and conveys the control to Arduino UNO.Finally, the signal received triggers the controller to move the car in specified direction.The HC-SR04 ultrasonic sensor is connected to Raspberry Pi using a small piece of breadboard and some jumper cables.
The sensor and Pi are mounted on the car, the ultrasonic module is used to constantly measure distances.The Trigger accepts a 3.3V output from the Pi but the Echo pin must be converted to 3.3V from 5V before it reaches the GPIO input.The Pi Camera is also connected to Pi, it is used to capture the video and convert the video into image frames and pass in onto CNN.

B. Regression Algorithm
This algorithm is used for object localization, predicting movement and also object detection.The working of regression algorithm compares variables to predict an objects position and distance.There are different kinds of regression algorithm used in self-driving cars like decision forest regression, neural network regression and Bayesian regression.A combination of decision forest and Bayesian is best to use as it is good is predicting events and provides fast online detection.

C. Decision Algorithm
The decision matrix algorithm analyzes a set of data and its corresponding values and then identifies and evaluates the performance of the associated data.As the name suggests, they are used for decision making.Whether a vehicle demands to brake or initiates to move in a particular direction based on the level of confidence, these algorithms recognize and predict the next movement of objects.The decision matrix algorithms have several decision models trained independently, and these models predict to make an overall prediction while minimizing the chances of errors in decision making.

D. Pattern Recognition Algorithm
The images captured through a sensored camera in selfdriving autonomous, which includes an Advanced Driver Assistance Systems (ADAS) consists of all sets of environmental data and refining of the image frames, discovers the instances of an object category and rules out irrelevant data in the image frame.Before analyzing the objects, the identification of patterns is an essential step in a dataset.These kinds of algorithms are termed as data reduction algorithms.
In this paper all types of algorithms were used one at a time on the self-driving autonomous car to check how it recognizes the image frames and train the model.Haar's cascade classifier was easy to implement on the cost-effective hardware, it also yielded better result with ease of execution.Other 3 techniques took lof of time and process to run on the board had look of issues due to multiple reasons like development of the machine learning was difficult on the hardware.All these algorithms can be applied but would rather require additional hardware development kits like Google coral, intended to bolster the development of machine learning models which are the edge.The accuracy obtained using Haar's cascade classifier was close to 60% on training samples and 80+ % on testing samples.In fig 6, the accuracy on training samples can be seen.

RESULTS
In this paper, after training the model several times with 32 videos of different tracks with many failed attempts, the model was successfully able to run the car on the track as desired in forward, right, left directions.Stopping the vehicle if we find an obstacle or seeing a stop sign wasn't that accurate compared to moving the car is a specified direction, but if tried out using different algorithms in the future, it would yield better results.

CONCLUSION
In this paper, the overall design of the autonomous car model presented was cost-effective and was done using the OpenCV package in python.In the field of an autonomous self-driving car, the proposed work yields better hassle-free transport system.The real time capabilities can be extended to real world so that the travel is safe and less drunk drivers and lesser accidents, also the concern on auto insurance based on driver and demerit system would are lesser risk in insurance practice.Vulnerable testing of autonomous car is currently better, but machines cannot be trusted with 100% automation when human life is vital here, so universally it is still considered a low based industry, whereas in the future it may play a very important role in transportation.

LIMITATION IN THE CURRENT SYSTEM
Hardware and Software learn and react from different situations by machine learning.Few situations are like unpredictable agent behaviour, complexity in perception selfdriving security, visibility occlusion from weather, optical illusions like a mirror reflection.Most of the real-world scenarios aren't considered, so extending the model by trying out different sensors and actuators would enhance the system and would also increase the accuracy and safety features of the autonomous car, which is very vital in the self-driving automotive industry.

CONTRIBUTION AND FUTURE WORK
In advancement to the current model, the system can be optimized by using more data and train model for a longer number of epochs so that it yields better results in terms of self-driving.The car moves slightly out of the track, as lane discipline is an important rule in traffic laws.If we current this in the future, it would make system more efficient in terms of traffic laws, and it will also be added as a safety feature.In the future, more enhanced algorithms and sensors can be applied to yield better results.

Fig. 3 .
Fig. 3. Concept of execution of autonomous car block diagram

Fig. 5 .
Fig. 5. Sample images of the track on which the car is trained.The car was trained on different combinations of tracks like straight, curved.Training the vehicle on the straight track was much more comfortable as the image frames were much easier to obtain and train the model, a total of 15-20 videos were recorded to obtain better results, the images were re-sized to 320*240 on which the network was trained.The Convolutional Neural Network had 128 input nodes, two hidden layers of 32 nodes each and finally, the output layer consisting of 4 nodes for each of the four output.IV.AUTONOMOUS CAR MODEL DESIGN

Fig. 6 .
Fig. 6.Accuracy of the trained autonomous car model using Haar's Cascade Classifier.