Moving Vehicles Detection and Tracking on Highways and Transportation System for Smart Cities

Introduction

Now a day’s the transportation system has become an integral part of day to day human’s lives. Around 40 percent of the world's population spends at least an hour on the roads/streets per day (Aastho, 2010). Not only this National Highways plays a major role in connecting cities and towns, hence they are extensively used for transportation systems as well as for travelling between cities and towns. Nowadays, in parallel with the increasing population, the increase in the number of vehicles increases the time people spend in traffic. In order to solve the problems caused by these, increases (Yadav, D. K., 2019, Yadav, D. K., & Singh, K. 2019). Travel and Transportation issues become a difficult task when the system and the behavior of user’s are very difficult to frame and to predict travel patterns/behaviors. Therefore, Machine Learning and Deep Learning algorithms may be helpful to overcome the challenges of simultaneously increasing travel demand, increasing congestion, road safety, traffic prediction, etc. These challenges emerge due to continuous growth of rural and urban vehicles. Due to exponential growth in population and mainly in developing countries like India etc. Phillips, D. J., et al., 2019, Wu, D.,  et al., 2017, Pan, G., et al., 2017).

Artificial Intelligence and Machine Learning (AI & ML) is a wide area of computer science and engineering field that makes machines work like the human being. It is utilized to search for issues that are hard to clarify utilizing conventional experiential techniques (Song, H., et al., 2019). The AI based techniques can be implemented as Knowledge-Based Systems (KBS) and as an Artificial Neural Network (ANN) (Guerrero-Ibáñez, et al., 2018, Kukkala, V. K., et al., 2018, Khan, T., et al., 2019).The KBS systems are those where AI works, based on the predetermined rules defined in the algorithm by humans. The artificial neural network, on the other hand, are systems of neurons connected and designed onto various layers, their working is very similar to the human brain, they take some input list and based on the input list, the ANNs produce required outputs (Chen, L., et al., 2018, Yazdi, M., & Bouwmans, T., 2018, Microsoft Asia News Center, 2019, Machin, M., et al ,2018, Özdağ, M. E., & Atasoy, N. A., 2019). The Deep learning is based on ANN but involves a greater number of hidden neurons and hidden layers than traditional ANNs. Deep learning has proven to be a huge success in the aspects of natural language processing (NLP), speech recognition, computer vision and item recommendation. Various problematic datasets are publicly available and some of them are used in this paper too (Goyette, N., et al 2012, Microsoft, 2020 ). They also achieve state-of-art efficiency in multiple classification and prediction task in transport scenarios (Polson, N. G., & Sokolov, V. O., 2017, Saluja, N., 2019, Inkoom, S., et al., 2019). As long as enough training data and GPU resources are available, it is possible that conventional machine learning methods can be overridden by deep learning models (Abduljabbar, R., 2019).

According to the WHO Global Road Safety Report 2018, there were over 1.5 lakhs of casualties in India due to road accidents alone (NCRB 2019). With 1.94% of the overall traffic, national highways accounted for 30.2% of road injuries and 35.7% of fatalities in 2018. For 2.97 percent of the length of the routes, state highways account for 25.2 percent and 26.8 percent of injuries and fatalities respectively. 5.8% of deaths attributable to collisions are due to driving on the wrong side of the road. Cell phone use accounts for 2.4% of deaths, and another 2.8% of casualties were due to drunken driving. The machine learning and artificial intelligent based applications are used in transportation for intelligent buses, connected busses, smart roads and computer vision enabled vehicles (Huang, T., Wang, S., & Sharma, A. (2020), Yuan, T., et al 2019, Lanner, 2019). Apart from these IoT-enabled devices to communicate in wireless network along with messaging system, alert, voice-based system is inbuilt in vehicles now a days to upgrade the transportation system (Zeng, Q., et al, 2020, Xu, X., et al, 2019). Another application which is highly recommended in smart vehicles to check ECG, EEG, EDA etc for driver or other passengers. Such ITS based applications monitor driver’s (i) health and feelings monitoring, (ii) sleepy cautioning (iii) alert control.

Literature Survey

The Scientists and Researchers have done their research on the problems on highways and transportation systems, based on the data that is being collected from different sources (NCRB, 2019, Huang, T., Wang, S., & Sharma, A., 2020, Yuan, T., 2019, Lanner, 2019). They have developed numerous D/L and M/L models for accident prediction, highway safety, designing and controlling transport network structures, intelligent transport systems, traffic flow prediction, travel demand prediction, automated driving (self-driving cars), traffic signal control, crack condition of the roads on highways, etc (Yadav, D. K. 2019, Microsoft Asia News Center, 2019, Zeng, Q., et al., 2020, Xu, X., et al 2019).

For real time assessment of highway traffic monitoring, a system has been proposed. In this, the onboard vehicle equipment and the roadside units (RSUs) work together to assess the contingency of an occurrence under the artificial intelligence criterion. They specifically focus on two paradigms of AI, i.e. (1) vector support machines (SVMs) and (2) Artificial Neural Network (ANNs) (Wu, D., 2018, Song, H., et al., 2019, Guerrero-Ibáñez, J., et al 2019).

The data on the number of vehicles and vehicle categories play a crucial role in the management of highways. Because of the several types of vehicles like Bus, truck, Bike, tractor etc. types of vehicles, their identification remains challenge that directly affects the precision of the vehicle count. A vehicle recognition system based on computer vision has been proposed to address this issue. YOLOv3 network is used to detect the type of vehicle and their trajectories are found with the help of ORB algorithm (Song, H., et al., 2019).

The number of vehicles and the form of vehicles play an important role in the management of highways. This issue must be resolved using some ML and DL models. A vehicle identification system based on computer vision has been proposed to address this issue. YOLOv3 network is used to detect the type of vehicle and their trajectories are found with the help of ORB algorithm (Song, H., et al 2019).

Fig. 1. Moving Vehicle Counting (Song, H., et al 2019)

Traffic flow prediction is a very crucial step in designing a successful intelligent transport system. The deep learning models have been able to predict the traffic density with the help of big data on the highways. “The techniques used are Long Short Time Memory (LSTM), Recurrent Neural Network (RNN), Stacked Long Short Term Memory (S-LSTM), Gated Recurrent Unit (GRU) and Bidirectional Long Short Term Memory (B-LSTM) neural networks” (Özdağ, M. E., & Atasoy, N. A.2019). A deep learning model is developed that mix up the linear model computed using L1 regularization and layer based sequences. Prediction of the traffic flow is a challenge because of the irregularities occurs due to congestion, breakdown and transition between free flows. They demonstrate that deep learning structures can capture these non-linear spatio-temporal effects. It introduces the idea of using ML to construct a global highway safety (SFP) feature that can be used to predict the predicted frequency of crashes for various routes from various regions. As an alternative to regression models used for crash modeling, a common DL model known as the Deep Belief Network (DBN) has been implemented (Song, H., et al 2019).

The conditions of the pavements is regularly tested by the transport department through visual analysis, image recognition and other learning algorithms. These techniques are effective, but mistakes, ambiguity and overfitting are highly probable. A research has been conducted to predict pavement crack ratings using recursive partitioning and artificial neural networks (ANNs and deep learning frameworks) and has proven to be a good technique for detecting cracks (Chen, L., 2018, Yazdi, M., & Bouwmans, T. 2018, Özdağ, M. E., & Atasoy, N. A.2019, Polson, N. G., & Sokolov, V. O. 2017).

(Phillip et. al.2017) explored a real-time prediction based method that automatically analyzed the collision risk from the monocular video data to automate the transportation system. (Kukkala et. al. 2018) proposed a method for evaluating the path for vehicles running on the highways in transportation system. It also assist the drivers through the driver-assistance systems along with autonomous system. (Song et al. 2019) has focused on the literatures and examined possible work for moving motor vehicle identification and incorporate system on highways/roads scene using the DL approach. Such work enhances the real-time based techniques for intelligent transportation system. (Peng et  al. 2019)  has investigated a method for haze removal in the colored images. This method uses airlight white correction along with the local light filtering technique. (Ma et. al., 2009) has focused on real time highway traffic condition and then assess the system for Vehicle Infrastructure Integration (VII) using AI for developing an Intelligent Transportation System.

Usage and Role of AI and ML techniques in Highways

This section describes the different ways of applying Machine Learning, Deep Learning and Artificial Neural Network models in highway management and transportation systems and how these models are making lives easier and efficient.

1. Highway Crash Detection and Crash prediction

To reduce the adverse effect of crashes on highways, it is becoming very essential for the traffic management centers to timely get the information about the crash, preferably before a crash. To avoid unusual highway traffic and secondary crashes it is very essential to get the crash prediction on time (Aastho 2010, Saluja, N., 2019, NCRB, 2019, Lanner 2019). The researchers investigated many systems for timely and reliable identification of crashes to assist in the management of road accidents. Nowadays we have different sources of real time data available and therefore we can use them and build very precise models. (Huang et. al. 2020) was conducted a study on using DL models to detect collision risks and collision detection. For crash detection, convolutional neural networks (CNN) were used and found to be performing better than the regular models, when provided with stable training data without overfitting. Data from different time slots were checked for prediction to further explore the model prediction power model. The result of their analysis shows that better collision detection and very close collision prediction results are available in the deep learning model.

1. Crash hotspot identification on Highways

The Hotspots are the roadway sites or places where the frequency of occurring of a crash is high. Expected Equivalent Property Damage only (EEPDO) has been used for identification of crash hotspot. There are various techniques for crash detection and crash prediction but it is very difficult to correctly predict for a crash to happen (Aastho 2010, Pan, G., Fu, L., & Thakali, L. 2017, Saluja, N., 2019, NCRB, 2019). The crashes are random and rare, i.e. fluctuating with time and space. (Wu et. al., 2017) compared the performance of machine learning algorithms, KNN algorithm and Negative Binomial (NB) to find an estimate of EEPDO. Negative Binomial assumes that the primary data follows a certain Gamma distribution that is not commonly retained for crash data. The result of their experiment showed that the K-Nearest Neighbor (KNN) algorithm outperformed the Negative Binomial in finding accurate value for EEPDO that helps in identifying crash hotspots.

1. Monitoring Driver Behavior

On highways the drivers drove their vehicle at very high speed which might lead to dangerous accidents, these accidents can cause property damage, life damage, and other damages. These accidents largely depend on the driver’s behavior. Mr. Dhammasaroj, Vice President of General Administration of PTT Global Chemical Public Company Limited (GC), explored how to reduce the risks of road/highway travel. This system's mainly focused to detect if the driver driving the vehicle is feeling distracted or sleepy. The working of this application can be outlined as: the company's vehicles are equipped with driver-focused cameras and a GPS (Global Positioning System) for detecting speed. The information on facial recognition is gathered and moved to the cloud where machine learning is used to analyze it. If the driver showed the sign of risks then they will intimate by an immediate alarm, and then a new driver may be sent off by the fleet manager, if necessary. Machine learning from the data collected can continuously enhance the identification of sleepiness as the device is used more and may also be able to identify specific behavioral signals. Over a time, the approach gets intelligent and helps to predict and avoid accidents even more accurately (Yazdi, M., & Bouwmans, T., 2018).

1. Intelligent Transport System (ITS)

Intelligent transport systems (ITS) are more likely in the future to be a major component of smart cities. It is today’s demand. ITS makes use of the information technology and sensing technology to improve the transportation and transit systems. Few of the applications and services of the ITS systems are in public transit system management, traffic management, self-driving vehicles and traveler information systems, etc. Nowadays we have abundant data available collected from various sources like in vehicle sensors, cameras, etc. These collected data can be very useful in making Machine Learning and Artificial Intelligence models for ITS (Machin, M.,, et al, 2018). (Machin, M.,, et al, 2018) discussed about the use of various AI techniques that can improve the ITS systems. They conducted their study into three main areas of ITS i.e., (1) Vehicle Control, (2) Traffic forecast and Control (iii) Road Safety and Accident prognostic. The selected AI techniques that were used are: (1) ANNs, (2) GAs, (3) FLs, and (4) ESs. The result of their study can be summarized as: For vehicle control systems the most widely used AI technique was Genetic Algorithm and also GAs are suitable for multi-objective improvements. Artificial Neural Networks were used for traffic prediction and traffic control services. For road safety and accident prediction it was found that for estimating the accident frequency FL seems suitable and for injury severity in traffic accident ANNs was performing well.

1. Crack Condition of Highway Pavements

The pavements are the surfacing of a road that helps in absorbing or transmitting the load to the sub-base and underlying soil. Heat and cold weather causes the pavement to expand and contract that eventually causes cracks in pavement. So it has become a necessity to make sure that the pavements are in correct shape. Because it is very expensive to make a new pavement rather than to maintain it (Microsoft Asia News Center, 2019). (Inkooma et. al. 2020) investigated the use of ML algorithms to predict the crack rating of pavements. ANNs and recursive partitioning were the algorithms used. They found in their results that both the ANNs & recursive partitioning can be used to predict crack conditions. Based on their quality of-fit statistics, mean absolute deviation (MAD< 0.4) and root mean square errors, crack ratings were observed ( RMSE between 0.30 and 0.65 ).

1. Traffic Surveillance and Traffic Flow

Traffic surveillance means monitoring the traffic flow on highways and roads. Traffic surveillance can help in monitoring of the roads for accidents, closures and also in highway management not only this, it is useful in making decisions regarding future road development and constructions (Pan, G., Fu, L., & Thakali, L. 2017, Özdağ, M. E., & Atasoy, N. A.2019, Polson, N. G., & Sokolov, V. O. (2017). Now a days, researches are trying to focus on IoT and sensor enabled vehicles which uses computer vision technology and work in cloud environment through wireless network. For example, a sensing technology-based application combined with ICT to improve the intelligent transport system, such as when a vehicle is involved in a road accident due to a sudden pothole opening and the vehicle is stuck inside. (as seen in Fig. 2). So, such applications are very helpful for drivers to get out of such kind of sudden danger zones. Various technological and problematic aspects haves been depicted in (Goyette, N., 2012, Microsoft, 2020).

Fig. 2.Segmented Results

For example, an application based on sensing technology that can be combined with information investigated a vehicle recognition system based on computer vision and an intensive counting system based on DL models. In the proposed traffic sensing the main highway and further categorized into the remote and the proximal region in the proposed traffic sensing and counting method by using the newly proposed image segmentation system. An important way to improve vehicle detection results. Then, to decide the type and position of the vehicle, the above two positions are put on the YOLOv3 network. Finally, the vehicle's trajectories are obtained by the ORB algorithm, which can be used to measure the vehicle's driving direction and to estimate the number of unique vehicle trajectories. Then, to decide the type and position of the vehicle, the above two positions are put on the YOLOv3 network. To authenticate and confirm the proposed method of segmentation, highway surveillance image sequences focused on different scenes are used. The test results confirm that high detection accuracy can be given by using the proposed segmentation process, especially for smaller vehicles (Song, H. et al,2019).

Results

The experimental results along with the analysis reveals that simulation work has been performed over highway frame sequences provided by changedetection dataset (Goyette, N, 2012). This frame sequence is freely and publicly available for research work and each frame is having size of 320X240 in gray scale. The implementation work is performed on Matlab2014b platform on the Windows 8.1. The achieved outcomes are categorized in two phases which are qualitative analysis (visual results: Table-1) and quantitative analysis (quantitative results: Table-2, Table-3)

Qualitative Results

This section focused on the qualitative results and depicted a comparative analysis of the various state-of-the-art methods over publicly available highway frame sequence. The qualitative (visual) results are available in the Table 1. As per visually detected results, it seems our proposed results are better than other considered peer methods.

Table 1. Qualitative Analysis

Quantitative Results and their Analysis

In this section, the proposed work explored the quantitative analysis along with considered state-of-the-art methods that can be visualize through Table 2 and Table 3. The error is computed through the given metrics (Microsoft, 2020, Polson, N. G., et al 2017, Yuan).

FP_Error = FP *100 / total_no_of_rows * total_no_of_columns;                                                (1)             

FN_Error = FN *100 / total_no_of_rows * total_no_of_columns;                                          (2)

Total_Error = FP_Error + FP_Error;                                                                                        (3)

Here, the value of F₁-score is simply computed through the harmonic mean of precision (P_av) and average recall (R_av) values. The value of precision, recall and F₁-score are computed though the following equations. 

P_av = Σ_nTP / Σ_n (TP+FP)                                                                                                                  (4)

R_av = Σ_nTP / Σ_n (TP+FP)                                                                                                                  (5)

F-measure_av = 2 * (P_av* R_av) / (P_av+ R_av) (6)

Fig. 3. Error analysis

The F-Measure or F₁-score is computed for analysis of the performance of proposed method with the peer methods. Maximum value of F₁-score indicated outcome. So, according to analyzing the performance using F₁-score, the performance of the proposed work have better results as per in comparison with (state-of-the-art) methods.

Table 2. Quantitative Analysis: Precision, Recall and F₁-score

Similarly, this work also evaluated the value of true positive rate (TPR), false positive rate (FPR), percentage of bad classification (PBC) and accuracy. These values are shown in Table-3. The proposed work has depicted better TPR and FPR as Grimson and Haque shows more value of TPR but are having several misclassified pixels that maximizes the false positive rate. Here, the proposed method has less misclassification rate i.e. FPR. In case of percentage of bad classification the proposed work depicted minimum value of bad classification, so it classified maximum pixels correctly that indicates less error. After post-processing the performance of the proposed results demonstrates better accuracy value. So, the overall performance through achieved results depicted that proposed method is better against the considered peer methods.

Table 3.Quantitative Analysis: TPR, FPR, PBC and Accuracy Measure

Conclusion

Now a days, in real-time environment various problems has been resolved through artificial intelligent techniques to develop automatic evaluation based system. In this paper, the proposed work discusses various machine learning and artificial intelligence based state-of-the-art models that solves problems related to detection, tracking, counting of moving vehicles especially for transportation systems and highways. This paper work has depicted the real-time based methodologies to automate the transportation system. Such work is helpful to develop detection and tracking based pre-crash alert system, driver assistance system etc. the proposed background subtraction based method has been experimented over high dataset that is freely and publicly available along with ground truth. The experimental results and analysis demonstrates the better performance of the proposed work against the available peer methods. Now a days, such autonomous techniques are helpful for smart cities through intelligent transportation systems.

Acknowledgement

The authors are very thankful for changedetection.net (Goyette, N., et al 2012) for providing problematic video sequence with ground truth for academic and research purpose.

Conflict of interest

The authors declare no potential conflict of interest regarding the publication of this work. In addition, the ethical issues including plagiarism, informed consent, misconduct, data fabrication and, or falsification, double publication and, or submission, and redundancy have been completely witnessed by the authors.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article