skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Low-cost camera and 2-D LIDAR fusion for target vehicle corner detection and tracking: Applications to micromobility devices
This paper develops a cost-effective vehicle detection and tracking system based on fusion of a 2-D LIDAR and a monocular camera to protect electric micromobility devices, especially e-scooters, by predicting the real- time danger of a car- scooter collision. The cost and size disadvantages of 3-D LIDAR sensors make them an unsuitable choice for micromobility devices. Therefore, a 2-D RPLIDAR Mapper sensor is used. Although low-cost, this sensor comes with major shortcomings such as the narrow vertical field of view and its low density of data points. Due to these factors, the sensor does not have a robust output in outdoor applications, and the measurements keep jumping and sliding on the vehicle surface. To improve the performance of the LIDAR, a single monocular camera is fused with the LIDAR data not only to detect vehicles, but also to separately detect the front and side of a target vehicle and to find its corner. It is shown that this corner detection method is more accurate than strategies that are only based on the LIDAR data. The corner measurements are used in a high-gain observer to estimate the location, velocity, and orientation of the target vehicle. The developed system is implemented on a Ninebot e-scooter platform, and multiple experiments are performed to evaluate the performance of the algorithm.  more » « less
Award ID(s):
2038403
PAR ID:
10482124
Author(s) / Creator(s):
;
Publisher / Repository:
Elsevier
Date Published:
Journal Name:
Mechanical Systems and Signal Processing
Volume:
206
Issue:
C
ISSN:
0888-3270
Page Range / eLocation ID:
110891
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    High-resolution vehicle trajectory data can be used to generate a wide range of performance measures and facilitate many smart mobility applications for traffic operations and management. In this paper, a Longitudinal Scanline LiDAR-Camera model is explored for trajectory extraction at urban arterial intersections. The proposed model can efficiently detect vehicle trajectories under the complex, noisy conditions (e.g., hanging cables, lane markings, crossing traffic) typical of an arterial intersection environment. Traces within video footage are then converted into trajectories in world coordinates by matching a video image with a 3D LiDAR (Light Detection and Ranging) model through key infrastructure points. Using 3D LiDAR data will significantly improve the camera calibration process for real-world trajectory extraction. The pan-tilt-zoom effects of the traffic camera can be handled automatically by a proposed motion estimation algorithm. The results demonstrate the potential of integrating longitudinal-scanline-based vehicle trajectory detection and the 3D LiDAR point cloud to provide lane-by-lane high-resolution trajectory data. The resulting system has the potential to become a low-cost but reliable measure for future smart mobility systems. 
    more » « less
  2. Micromobility usage has increased significantly in the last several years as exemplified by shared escooters and privately owned bicycles. In this study, we use traffic camera footage to observe the behavior of over 700 shared e-scooters and privately owned bicycles in Asbury Park, New Jersey. We address the following questions: (1) What are the behavioral differences between bicycle and e-scooter usage in terms of helmet use, bike lane / sidewalk use, gender split, group riding, and by time of day? (2) Are more protective conditions associated with helmet use and bike lane / sidewalk use? And (3) what is the gender split between e-scooter users and cyclists? We find notable differences in safety precautions: around one third of cyclists but no shared e-scooter users were observed wearing a helmet. Among cyclists, helmet use was more prominent among men than women. However, men were more likely to ride on the road than women. We also found that the gender split was narrower among e-scooter users, with a nearly even gender split – as opposed to cyclists, where only 21% of cyclists were observed to be women. Our findings suggest that e-scooter users take fewer safety precautions, in that they are less likely to use a bike lane and to wear a helmet. We conclude with policy implications with regards to safety and gender differences between these two modes. 
    more » « less
  3. Multi-sensor fusion has been widely used by autonomous vehicles (AVs) to integrate the perception results from different sensing modalities including LiDAR, camera and radar. Despite the rapid development of multi-sensor fusion systems in autonomous driving, their vulnerability to malicious attacks have not been well studied. Although some prior works have studied the attacks against the perception systems of AVs, they only consider a single sensing modality or a camera-LiDAR fusion system, which can not attack the sensor fusion system based on LiDAR, camera, and radar. To fill this research gap, in this paper, we present the first study on the vulnerability of multi-sensor fusion systems that employ LiDAR, camera, and radar. Specifically, we propose a novel attack method that can simultaneously attack all three types of sensing modalities using a single type of adversarial object. The adversarial object can be easily fabricated at low cost, and the proposed attack can be easily performed with high stealthiness and flexibility in practice. Extensive experiments based on a real-world AV testbed show that the proposed attack can continuously hide a target vehicle from the perception system of a victim AV using only two small adversarial objects. 
    more » « less
  4. Electric(e)-scooters have emerged as a popular, ubiquitous, and first/last-mile micromobility transportation option within and across many cities worldwide. With the increasing situation-awareness and on-board computational capability, such intelligent micromobility has become a critical means of understanding the rider's interactions with other traffic constituents (called Rider-to-X Interactions, RXIs), such as pedestrians, cars, and other micromobility vehicles, as well as road environments, including curbs, road infrastructures, and traffic signs. How to interpret these complex, dynamic, and context-dependent RXIs, particularly for the rider-centric understandings across different data modalities --- such as visual, behavioral, and textual data --- is essential for enabling safer and more comfortable micromobility riding experience and the greater good of urban transportation networks. Under a naturalistic riding setting (i.e., without any unnatural constraint on rider's decision-making and maneuvering), we have designed, implemented, and evaluated a pilot Cross-modality E-scooter Naturalistic Riding Understanding System, namely CENRUS, from a human-centered AI perspective. We have conducted an extensive study with CENRUS in sensing, analyzing, and understanding the behavioral, visual, and textual annotation data of RXIs during naturalistic riding. We have also designed a novel, efficient, and usable disentanglement mechanism to conceptualize and understand the e-scooter naturalistic riding processes, and conducted extensive human-centered AI model studies. We have performed multiple downstream tasks enabled by the core model within CENRUS to derive the human-centered AI understandings and insights of complex RXIs, showcasing such downstream tasks as efficient information retrieval and scene understanding. CENRUS can serve as a foundational system for safe and easy-to-use micromobility rider assistance as well as accountable use of micromobility vehicles. 
    more » « less
  5. This paper presents a cost-effective, non-intrusive, and easy-to-deploy traffic count data collection method using two-dimensional light-detection and ranging (LiDAR) technology. The proposed method integrates a LiDAR sensor, continuous wavelet transform (CWT), and support vector machine (SVM) into a single framework for traffic count. LiDAR is adopted since the technology is economical and easily accessible. Moreover, its 360° visibility and accurate distance information make it more reliable compared with radar, which uses electromagnetic waves instead of light rays. The obtained distance data are converted into the signals. CWT is employed to detect any deviation in distance profile, because of its efficiency in detecting modest changes over a period of time. SVM is one of the supervised machine learning tools for data classification and regression. In the methodology, the SVM is applied to classify the distance data points obtained from the sensor into detection and non-detection cases, which are highly complex. Proof-of-concept (POC) test is conducted in three different places in Newark, New Jersey, to examine the performance of the proposed method. The POC test results demonstrate that the proposed method achieves acceptable performances in vehicle count collection, resulting in 83–94% accuracy. It is discovered that the accuracy of the proposed method is affected by the color of the exterior surface of a vehicle. 
    more » « less