An official website of the United States government
Recent technological developments have attracted the use of machine learning technologies and sensors in various pavement maintenance and rehabilitation studies. To avoid excessive road damages, which cause high road maintenance costs, reduced mobility, vehicle damages, and safety concerns, the periodic maintenance of roads is necessary. As part of maintenance works, road pavement conditions should be monitored continuously. This monitoring is possible using modern distress detection methods that are simple to use, comparatively cheap, less labor-intensive, faster, safer, and able to provide data on a real-time basis. This paper proposed and developed two models: computer vision and sensor-based. The computer vision model was developed using the You Only Look Once (YOLOv5) algorithm for detecting and classifying pavement distresses into nine classes. The sensor-based model combined eight Controller Area Network (CAN) bus sensors available in most new vehicles to predict pavement distress. This research employed an extreme gradient boosting model (XGBoost) to train the sensor-based model. The results showed that the model achieved 98.42% and 97.99% area under the curve (AUC) metrics for training and validation datasets, respectively. The computer vision model attained an accuracy of 81.28% and an F1-score of 76.40%, which agree with past studies. The results indicated that both computer vision and sensor-based models proved highly efficient in predicting pavement distress and can be used to complement each other. Overall, computer vision and sensor-based tools provide cheap and practical road condition monitoring compared to traditional manual instruments.
more »
« less
This content will become publicly available on February 1, 2026
Detection of Flexible Pavement Surface Cracks in Coastal Regions Using Deep Learning and 2D/3D Images
Pavement surface distresses are analyzed by transportation agencies to determine section performance across their pavement networks. To efficiently collect and evaluate thousands of lane-miles, automated processes utilizing image-capturing techniques and detection algorithms are applied to perform these tasks. However, the precision of this novel technology often leads to inaccuracies that must be verified by pavement engineers. Developments in artificial intelligence and machine learning (AI/ML) can aid in the progress of more robust and precise detection algorithms. Deep learning models are efficient for visual distress identification of pavement. With the use of 2D/3D pavement images, surface distress analysis can help train models to efficiently detect and classify surface distresses that may be caused by traffic loading, weather, aging, and other environmental factors. The formation of these distresses is developing at a higher rate in coastal regions, where extreme weather phenomena are more frequent and intensive. This study aims to develop a YOLOv5 model with 2D/3D images collected in the states of Louisiana, Mississippi, and Texas in the U.S. to establish a library of data on pavement sections near the Gulf of Mexico. Images with a resolution of 4096 × 2048 are annotated by utilizing bounding boxes based on a class list of nine distress and non-distress objects. Along with emphasis on efforts to detect cracks in the presence of background noise on asphalt pavements, six scenarios for augmentation were made to evaluate the model’s performance based on flip probability in the horizontal and vertical directions. The YOLOv5 models are able to detect defined distresses consistently, with the highest mAP50 scores ranging from 0.437 to 0.462 throughout the training scenarios.
more »
« less
- Award ID(s):
- 2213694
- PAR ID:
- 10612885
- Publisher / Repository:
- MDPI
- Date Published:
- Journal Name:
- Sensors
- Volume:
- 25
- Issue:
- 4
- ISSN:
- 1424-8220
- Page Range / eLocation ID:
- 1145
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Pavement surveying and distress mapping is completed by roadway authorities to quantify the topical and structural damage levels for strategic preventative or rehabilitative action. The failure to time the preventative or rehabilitative action and control distress propagation can lead to severe structural and financial loss of the asset requiring complete reconstruction. Continuous and computer-aided surveying measures not only can eliminate human error when analyzing, identifying, defining, and mapping pavement surface distresses, but also can provide a database of road damage patterns and their locations. The database can be used for timely road repairs to gain the maximum durability of the asphalt and the minimum cost of maintenance. This paper introduces an autonomous surveying scheme to collect, analyze, and map the image-based distress data in real time. A descriptive approach is considered for identifying cracks from collected images using a convolutional neural network (CNN) that classifies several types of cracks. Typically, CNN-based schemes require a relatively large processing power to detect desired objects in images in real time. However, the portability objective of this work requires to utilize low-weight processing units. To that end, the CNN training was optimized by the Bayesian optimization algorithm (BOA) to achieve the maximum accuracy and minimum processing time with minimum neural network layers. First, a database consisting of a diverse population of crack distress types such as longitudinal, transverse, and alligator cracks, photographed at multiple angles, was prepared. Then, the database was used to train a CNN whose hyperparameters were optimized using BOA. Finally, a heuristic algorithm is introduced to process the CNN’s output and produce the crack map. The performance of the classifier and mapping algorithm is examined against still images and videos captured by a drone from cracked pavement. In both instances, the proposed CNN was able to classify the cracks with 97% accuracy. The mapping algorithm is able to map a diverse population of surface cracks patterns in real time at the speed of 11.1 km per hour.more » « less
-
Road authorities worldwide can leverage the advances in vehicle technology by continuously monitoring their roads’ conditions to minimize road maintenance costs. The existing methods for carrying out road condition surveys involve manual observations using standard survey forms, performed by qualified personnel. These methods are expensive, time-consuming, infrequent, and can hardly provide real-time information. Some automated approaches also exist but are very expensive since they require special vehicles equipped with computing devices and sensors for data collection and processing. This research aims to leverage the advances in vehicle technology in providing a cheap and real-time approach to carry out road condition monitoring (RCM). This study developed a deep learning model using the You Only Look Once, Version 5 (YOLOv5) algorithm that was trained to capture and categorize flexible pavement distresses (FPD) and reached 95% precision, 93.4% recall, and 97.2% mean Average Precision. Using vehicle built-in cameras and GPS sensors, these distresses were detected, images were captured, and locations were recorded. This was validated on campus roads and parking lots using a car featured with a built-in camera and GPS. The vehicles’ built-in technologies provided a more cost-effective and efficient road condition monitoring approach that could also provide real-time road conditions.more » « less
-
Abstract Maintaining airport runways is crucial for safety and efficiency, yet traditional monitoring relies on manual inspections, prone to time consumption and inaccuracy. This study pioneers the utilization of low‐cost dashcam imagery for the detection and geolocation of airport runway pavement distresses, employing novel deep‐learning frameworks. A significant contribution of our work is the creation of the first public dataset specifically designed for this purpose, addressing a critical gap in the field. This dataset, enriched with diverse distress types under various environmental conditions, enables the development of an automated, cost‐effective method that substantially enhances airport maintenance operations. Leveraging low‐cost dashcam technology in this unique scenario, our approach demonstrates remarkable potential in improving the efficiency and safety of airport runway inspections, offering a scalable solution for infrastructure management. Our findings underscore the benefits of integrating advanced imaging and artificial intelligence technologies, paving the way for advancements in airport maintenance practices.more » « less
-
Mobile headsets should be capable of understanding 3D physical environments to offer a truly immersive experience for augmented/mixed reality (AR/MR). However, their small form-factor and limited computation resources make it extremely challenging to execute in real-time 3D vision algorithms, which are known to be more compute-intensive than their 2D counterparts. In this paper, we propose DeepMix, a mobility-aware, lightweight, and hybrid 3D object detection framework for improving the user experience of AR/MR on mobile headsets. Motivated by our analysis and evaluation of state-of-the-art 3D object detection models, DeepMix intelligently combines edge-assisted 2D object detection and novel, on-device 3D bounding box estimations that leverage depth data captured by headsets. This leads to low end-to-end latency and significantly boosts detection accuracy in mobile scenarios. A unique feature of DeepMix is that it fully exploits the mobility of headsets to fine-tune detection results and boost detection accuracy. To the best of our knowledge, DeepMix is the first 3D object detection that achieves 30 FPS (i.e., an end-to-end latency much lower than the 100 ms stringent requirement of interactive AR/MR). We implement a prototype of DeepMix on Microsoft HoloLens and evaluate its performance via both extensive controlled experiments and a user study with 30+ participants. DeepMix not only improves detection accuracy by 9.1--37.3% but also reduces end-to-end latency by 2.68--9.15×, compared to the baseline that uses existing 3D object detection models.more » « less
