More Like this

1. Robust multimodal vehicle detection in foggy weather using complementary lidar and radar signals

   Qian, Kun; Zhu, Shilin; Zhang, Xinyu; Li, Erran Li (June 2021, IEEE Conference on Computer Vision and Pattern Recognition)

   null (Ed.)

   Vehicle detection with visual sensors like lidar and camera is one of the critical functions enabling autonomous driving. While they generate fine-grained point clouds or high-resolution images with rich information in good weather conditions, they fail in adverse weather (e.g., fog) where opaque particles distort lights and significantly reduce visibility. Thus, existing methods relying on lidar or camera experience significant performance degradation in rare but critical adverse weather conditions. To remedy this, we resort to exploiting complementary radar, which is less impacted by adverse weather and becomes prevalent on vehicles. In this paper, we present Multimodal Vehicle Detection Network (MVDNet), a two-stage deep fusion detector, which first generates proposals from two sensors and then fuses region-wise features between multimodal sensor streams to improve final detection results. To evaluate MVDNet, we create a procedurally generated training dataset based on the collected raw lidar and radar signals from the open-source Oxford Radar Robotcar. We show that the proposed MVDNet surpasses other state-of-the-art methods, notably in terms of Average Precision (AP), especially in adverse weather conditions. The code and data are available at https://github.com/qiank10/MVDNet. 

   more » « less
2. MoPac+: Multimodal Modeling of the Pavement cracks using hard-example mining and context-aware feature aggregation

   https://doi.org/10.1016/j.measurement.2026.120896  

   Biswas, Debojyoti; Scouten, Andrew; Gong, Haitao; Wang, Feng; Tešić, Jelena (May 2026, Measurement)

   Automated pavement distress (crack) detection through image analysis remains critical for pavement condition evaluation, maintenance planning, project selection, and asset management. While deep learning has significantly advanced pavement distress detection, current datasets and methodologies predominantly focus on common crack types (e.g., longitudinal/transverse), leaving rare distress patterns and fine-grained categorizations inadequately addressed. Existing approaches further demonstrate limitations in handling irregular crack morphologies and datascarce scenarios. This paper introduces MoPac+, a novel anchor-free cascade framework integrating multi-stage crack modeling with adaptive hard-example mining and loss reweighting, specifically designed for robust pavement deterioration localization and classification in aerial imagery. Our methodology addresses two key challenges: (1) uncontrolled crack geometry with significant intra-class size variation, addressed through anchor-free detection modules, and (2) grayscale image limitations mitigated by pre-processing techniques that enhance distressed regions through strategic dilation and class-balancing operations. The proposed system particularly improves the detection efficiency of rare classes by implementing gradient-harmonized loss allocation and hierarchical feature fusion. Comprehensive experiments demonstrate substantial performance gains, achieving a 6.9% relative improvement in mean average precision (mAP) over baselineYOLOv8 models while maintaining computational efficiency. These advancements establish new state-of-the-art benchmarks for imbalanced pavement distress recognition in resource-constrained environments. 

   more » « less
3. On Model Calibration for Long-Tailed Object Detection and Instance Segmentation

   Pan, Tai-Yu; Zhang, Cheng; Li, Yandong; Hu, Hexiang; Xuan, Dong; Changpinyo, Soravit; Gong, Boqing; Chao, Wei-Lun (January 2021, Advances in neural information processing systems)

   Vanilla models for object detection and instance segmentation suffer from the heavy bias toward detecting frequent objects in the long-tailed setting. Existing methods address this issue mostly during training, e.g., by re-sampling or re- weighting. In this paper, we investigate a largely overlooked approach — post- processing calibration of confidence scores. We propose NORCAL, Normalized Calibration for long-tailed object detection and instance segmentation, a simple and straightforward recipe that reweighs the predicted scores of each class by its training sample size. We show that separately handling the background class and normalizing the scores over classes for each proposal are keys to achieving superior performance. On the LVIS dataset, NORCAL can effectively improve nearly all the baseline models not only on rare classes but also on common and frequent classes. Finally, we conduct extensive analysis and ablation studies to offer insights into various modeling choices and mechanisms of our approach. Our code is publicly available at https://github.com/tydpan/NorCal. 

   more » « less
4. Long-tail Detection with Effective Class-Margins

   Cho, Jang Hyun; Krähenbühl, Philipp (October 2022, European Conference on Computer Vision)

   Large-scale object detection and instance segmentation face a severe data imbalance. The finer-grained object classes become, the less frequent they appear in our datasets. However, at test-time, we expect a detector that performs well for all classes and not just the most frequent ones. In this paper, we provide a theoretical understanding of the long-trail detection problem. We show how the commonly used mean average precision evaluation metric on an unknown test set is bound by a margin-based binary classification error on a long-tailed object detection training set. We optimize margin-based binary classification error with a novel surrogate objective called \textbf{Effective Class-Margin Loss} (ECM). The ECM loss is simple, theoretically well-motivated, and outperforms other heuristic counterparts on LVIS v1 benchmark over a wide range of architecture and detectors. 

   more » « less
5. 3D-FAD: Informativeness-guided Multimodal 3D Object Detection for Federated Autonomous Driving

   https://doi.org/10.1145/3820499  

   Gan, Maolin; Liu, Li; Alam, Samiul; Liu, Jiancheng; Zhang, Mi; Zeng, Huacheng; Cao, Zhichao (July 2026, ACM Transactions on Sensor Networks)

   Autonomous driving has been adopted in various applications, including taxis, logistics, and agriculture, to address labor shortages and enhance safety. A critical component is 3D object detection, relying on deep neural networks (DNNs) that integrate LiDAR and camera data for robust road planning. Federated Autonomous Driving (FAD) utilizes Federated Learning (FL) to train these DNNs, addressing the demand for diverse training datasets while preserving client privacy. However, extending the existing FAD framework to 3D object detection remains challenging due to insufficient consideration of modality heterogeneity and informativeness. The article introduces3D-FAD, a pioneering informativeness-guided multimodal FAD framework.3D-FADis the first to explicitly integrate modality informativeness into multimodal 3D object detection for FAD by a convolution-based encoder. It utilizes this informativeness for efficient client selection and aggregation at the server. Furthermore,3D-FADintroduces Masked Modality AutoEncoders to impute deficient LiDAR features for clients with limited LiDAR FOV and low-cost cameras in real-world autonomous driving. Our experiments on real-world autonomous driving datasets nuScene and Waymo show that3D-FADoutperforms baselines by up to 34.1% and 39.2% in key metrics, along with better performance across different object classes and environments. 

   more » « less