Search for: All records

Despite significant strides in achieving vehicle autonomy, robust perception under low-light conditions still remains a persistent challenge. In this study, we investigate the potential of multispectral imaging, thereby leveraging deep learning models to enhance object detection performance in the context of nighttime driving. Features encoded from the red, green, and blue (RGB) visual spectrum and thermal infrared images are combined to implement a multispectral object detection model. This has proven to be more effective compared to using visual channels only, as thermal images provide complementary information when discriminating objects in low-illumination conditions. Additionally, there is a lack of studies on effectively fusing these two modalities for optimal object detection performance. In this work, we present a framework based on the Faster R-CNN architecture with a feature pyramid network. Moreover, we design various fusion approaches using concatenation and addition operators at varying stages of the network to analyze their impact on object detection performance. Our experimental results on the KAIST and FLIR datasets show that our framework outperforms the baseline experiments of the unimodal input source and the existing multispectral object detectors 

Humanoid robots have had significant research interest in the past two decades. Their classification as mobile manipulators allows them to work in unstructured environments creating new possibilities for human-robot interaction. Object grasping and manipulation are essential and enabling capabilities for mobile humanoid robots that require reliable perception. This paper presents a perception approach using depth images from an RGB-D camera to estimate the work plane and estimate object positions relative to the robot. Results from experiments with a set of object shapes and scenarios are presented.