skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Real-Time Barge Detection Using Traffic Cameras and Deep Learning on Inland Waterways
Inland waterways are critical for freight movement, but limited means exist for monitoring their performance and usage by freight-carrying vessels (e.g., barges). Although methods to track vessels (e.g., tug and tow boats) are publicly available through Automatic Identification System (AIS), ways to track freight tonnages and commodity flows carried on barges along these critical marine highways are nonexistent, especially in real-time settings. This study developed a method to detect barge traffic on inland waterways using existing traffic cameras with opportune viewing angles. Deep learning models You Only Look Once (YOLO), Single Shot MultiBox Detector (SSD), and EfficientDet were employed to detect the presence of vessels/barges from video and classify them (no vessel or barge, vessel without barge, vessel with barge, barge). A dataset of 331 annotated images was collected from five existing traffic cameras along the Mississippi and Ohio Rivers for model development. YOLOv8 achieved an F1-score of 96%, outperforming YOLOv5, SSD, and EfficientDet at 86%, 79%, and 77%, respectively. Sensitivity analysis was carried out for weather conditions (rain, fog) and location (Mississippi and Ohio River). A background subtraction technique normalized the video images across the various locations for the location sensitivity analysis. This model could be used to detect the presence of barges along river segments, which could be used for anonymous bulk commodity tracking and monitoring. Such data are valuable for long-range transportation planning efforts carried out by public transportation agencies, and for operational and maintenance planning conducted by federal agencies such as the U.S. Army Corps of Engineers.  more » « less
Award ID(s):
2042870
PAR ID:
10584404
Author(s) / Creator(s):
; ; ; ;
Publisher / Repository:
Transportation Research Record
Date Published:
Journal Name:
Transportation Research Record: Journal of the Transportation Research Board
ISSN:
0361-1981
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Effective transportation performance measurement (TPM) benefits from ubiquitous transportation system monitoring both spatially and temporally. In the context of freight-oriented TPM, traditional devices such as inductive loops, cameras, manual counts, and so forth, may fail to provide comprehensive and high-resolution coverage, providing, for example, only volume counts for a small subset of links across a large network with no indication of trip linkages. New sources of big data from mobile sensors including on-board global positioning system (GPS) devices allow more comprehensive network coverage and insights into trip chaining behaviors. However, to gain actionable insights into system performance from large and noisy streams of mobile sensor data, it is necessary to mine it for relevant operational characteristics of the trucks it represents. Such characteristics include stop locations, stop duration, stop time of day, trip length, and trip duration. To address this methodological need, this paper presents three heuristic algorithms: “stop identification,”“path identification,” and “trip identification.” To address the issue of determining relevant operational characteristics, a multinomial logit (MNL) model approach is applied to determine the commodity carried based on the outputs of the heuristic algorithms. The MNL model is novel in that it relates operational characteristics to commodity carried thus filling a critical data gap that currently limits the development of advanced freight forecasting models. The set of models developed in this paper allow large-scale GPS data to be used for freight planning while maintaining levels of data anonymity that allow such data to be shared with public agencies. 
    more » « less
  2. Deep learning algorithms are exceptionally valuable tools for collecting and analyzing the catastrophic readiness and countless actionable flood data. Convolutional neural networks (CNNs) are one form of deep learning algorithms widely used in computer vision which can be used to study flood images and assign learnable weights to various objects in the image. Here, we leveraged and discussed how connected vision systems can be used to embed cameras, image processing, CNNs, and data connectivity capabilities for flood label detection. We built a training database service of >9000 images (image annotation service) including the image geolocation information by streaming relevant images from social media platforms, Department of Transportation (DOT) 511 traffic cameras, the US Geological Survey (USGS) live river cameras, and images downloaded from search engines. We then developed a new python package called “FloodImageClassifier” to classify and detect objects within the collected flood images. “FloodImageClassifier” includes various CNNs architectures such as YOLOv3 (You look only once version 3), Fast R–CNN (Region-based CNN), Mask R–CNN, SSD MobileNet (Single Shot MultiBox Detector MobileNet), and EfficientDet (Efficient Object Detection) to perform both object detection and segmentation simultaneously. Canny Edge Detection and aspect ratio concepts are also included in the package for flood water level estimation and classification. The pipeline is smartly designed to train a large number of images and calculate flood water levels and inundation areas which can be used to identify flood depth, severity, and risk. “FloodImageClassifier” can be embedded with the USGS live river cameras and 511 traffic cameras to monitor river and road flooding conditions and provide early intelligence to emergency response authorities in real-time. 
    more » « less
  3. Real-time object detection is essential for AI-based intelligent traffic management. However, growing complexities of deep learning models for object detection cause increased latency and resource requirements. To tackle the challenge, we introduce a new approach, named AROD (Adaptive Real-Time Object Detection), that infers the pixel motion speed in continuous traffic video frames and skips redundant frames when the pixel velocity is low. Thereby, AROD aims to significantly enhance the efficiency and scalability, sustaining the accuracy of object detection. Our evaluation using real-world traffic videos reveals that our method for pixel velocity inference via lightweight deep learning reduces the RMSE (Root Mean Square Error) by up to two orders of magnitude compared to state-of-the-art approaches. AROD improves the frame processing rate of YOLOv5, SSD, and EfficientDet by approximately 32-61\%, 110-174\%, and 120-213\%, respectively. AROD considerably enhances scalability by supporting real-time object detection for up to three concurrent traffic video streams on a commodity machine. Moreover, AROD demonstrates its generalizability by supporting competitive accuracy in object detection for a separate traffic video that was fully hidden during training. 
    more » « less
  4. Real-time object detection is essential for AI-based intelligent traffic management. However, growing complexities of deep learning models for object detection cause increased latency and resource requirements. To tackle the challenge, we introduce a new approach, named AROD (Adaptive Real-Time Object Detection), that infers the pixel motion speed in continuous traffic video frames and skips redundant frames when the pixel velocity is low. Thereby, AROD aims to significantly enhance the efficiency and scalability, sustaining the accuracy of object detection. Our evaluation using real-world traffic videos reveals that our method for pixel velocity inference via lightweight deep learning reduces the RMSE (Root Mean Square Error) by up to two orders of magnitude compared to state-of-the-art approaches. AROD improves the frame processing rate of YOLOv5, SSD, and EfficientDet by approximately 32-61%, 110-174%, and 120-213%, respectively. AROD considerably enhances scalability by supporting real-time object detection for up to three concurrent traffic video streams on a commodity machine. Moreover, AROD demonstrates its generalizability by supporting competitive accuracy in object detection for a separate traffic video that was fully hidden during training. 
    more » « less
  5. Abstract The Mississippi River represents a major commercial waterway, and periods of anomalously low river levels disrupt riverine transport. These low-flow events occur periodically, with a recent event in the fall of 2022 slowing barge traffic and generating sharp increases in riverine transportation costs. Here we combine instrumental river gage observations from the lower Mississippi River with output from the Community Earth System Model v2 Large Ensemble (LENS2) to evaluate historical trends and future projections of Mississippi River low streamflow extremes, place the 2022 low-flow event in a broader temporal context, and assess the hydroclimatic mechanisms that mediate the occurrence of low-flows. We show that the severity and duration of low-flow events gradually decreased between 1950 and 1980 coincident with the establishment of artificial reservoirs. In the context of the last ∼70 years, the 2022 low-flow event was less severe in terms of stage or discharge minima than other low-flow events of the mid- and late-20th century. Model simulations from the LENS2 dataset show that, under a moderate-high emissions scenario (SSP3-7.0), the severity and duration of low-flow events is projected to decrease through to the end of the 21st century. Finally, we use the large sample size afforded by the LENS2 dataset to show that low-flow events on the Mississippi River are associated with cold tropical Pacific forcing (i.e. La Niña conditions), providing support for the hypothesis that the El Niño-Southern Oscillation plays a critical role in mediating Mississippi River discharge extremes. We anticipate that our findings describing the trends in and hydroclimatic mechanisms of Mississippi River low-flow occurrence will aid water resource managers to reduce the negative impacts of low water levels on riverine transport. 
    more » « less