More Like this

1. A Deep Learning Approach for Lane Detection

   https://doi.org/10.1109/ITSC48978.2021.9564965  

   Getahun, Tesfamchael; Karimoddini, Ali; Mudalige, Priyantha (September 2021, 2021 IEEE International Intelligent Transportation Systems Conference (ITSC))

   State-of-the-art lane detection methods use a variety of deep learning techniques for lane feature extraction and prediction, demonstrating better performance than conventional lane detectors. However, deep learning approaches are computationally demanding and often fail to meet real-time requirements of autonomous vehicles. This paper proposes a lane detection method using a light-weight convolutional neural network model as a feature extractor exploiting the potential of deep learning while meeting real-time needs. The developed model is trained with a dataset containing small image patches of dimension 16 × 64 pixels and a non-overlapping sliding window approach is employed to achieve fast inference. Then, the predictions are clustered and fitted with a polynomial to model the lane boundaries. The proposed method was tested on the KITTI and Caltech datasets and demonstrated an acceptable performance. We also integrated the detector into the localization and planning system of our autonomous vehicle and runs at 28 fps in a CPU on image resolution of 768 × 1024 meeting real-time requirements needed for self-driving cars. 

   more » « less
2. Fine-Grained Textual Knowledge Transfer to Improve RNN Transducers for Speech Recognition and Understanding

   https://doi.org/10.1109/ICASSP49357.2023.10094997  

   Sunder, Vishal; Thomas, Samuel; Kuo, Hong-Kwang J.; Kingsbury, Brian; Fosler-Lussier, Eric (June 2023, International Conference on Acoustics, Speech and Signal Processing)

   RNN Tranducer (RNN-T) technology is very popular for building deployable models for end-to-end (E2E) automatic speech recognition (ASR) and spoken language understanding (SLU). Since these are E2E models operating on speech directly, there remains a potential to improve their performance using purely text based models like BERT, which have strong language understanding capabilities. In this paper, we propose a new training criteria for RNN-T based E2E ASR and SLU to transfer BERT’s knowledge into these systems. In the first stage of our proposed mechanism, we improve ASR performance by using a fine-grained, tokenwise knowledge transfer from BERT. In the second stage, we fine-tune the ASR model for SLU such that the above knowledge is explicitly utilized by the RNN-T model for improved performance. Our techniques improve ASR performance on the Switchboard and CallHome test sets of the NIST Hub5 2000 evaluation and on the recently released SLURP dataset on which we achieve a new state-of-the-art performance. For SLU, we show significant improvements on the SLURP slot filling task, outperforming HuBERT-base and reaching a performance close to HuBERTlarge. Compared to large transformer based speech models like HuBERT, our model is significantly more compact and uses only 300 hours of speech pretraining data. 

   more » « less
3. Evaluation Metrics for Object Detection for Autonomous Systems

   Badithela Apurva; Wongpiromsarn, Tichakorn; Murray, Richard M. (October 2023, 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS))

   This paper studies the evaluation of learning-based object detection models in conjunction with model-checking of formal specifications defined on an abstract model of an autonomous system and its environment. In particular, we define two metrics – proposition-labeled and class-labeled confusion matrices – for evaluating object detection, and we incorporate these metrics to compute the satisfaction probability of system-level safety requirements. While confusion matrices have been effective for comparative evaluation of classification and object detection models, our framework fills two key gaps. First, we relate the performance of object detection to formal requirements defined over downstream high-level planning tasks. In particular, we provide empirical results that show that the choice of a good object detection algorithm, with respect to formal requirements on the overall system, significantly depends on the downstream planning and control design. Secondly, unlike the traditional confusion matrix, our metrics account for variations in performance with respect to the distance between the ego and the object being detected. We demonstrate this framework on a car-pedestrian example by computing the satisfaction probabilities for safety requirements formalized in Linear Temporal Logic (LTL). 

   more » « less
4. RAF-RCNN: Adaptive Feature Transfer from Clear to Rainy Conditions for Improved Object Detection

   https://doi.org/10.1109/ITSC58415.2024.10920096  

   Wei, Chuheng; Wu, Guoyuan; Barth, Matthew J (September 2024, IEEE)

   : In the challenging realm of object detection under rainy conditions, visual distortions significantly hinder accuracy. This paper introduces Rain-Adapt Faster RCNN (RAF-RCNN), an innovative end-to-end approach that merges advanced deraining techniques with robust object detection. Our method integrates rain removal and object detection into a single process, using a novel feature transfer learning approach for enhanced robustness. By employing the Extended Area Structural Discrepancy Loss (EASDL), RAF-RCNN enhances feature map evaluation, leading to significant performance improvements. In quantitative testing of the Rainy KITTI dataset, RAF-RCNN achieves a mean Average Precision (mAP) of 51.4% at IOU [0.5, 0.95], exceeding previous methods by at least 5.5%. These results demonstrate RAF-RCNN's potential to significantly enhance perception systems in intelligent transportation, promising substantial improvements in reliability and safety for autonomous vehicles operating in varied weather conditions. 

   more » « less
5. A Novel Application of Levenshtein Distance for Assessment of High-Level Motor Planning Underlying Performance During Learning of Complex Motor Sequences

   https://doi.org/10.1123/jmld.2018-0060  

   Hauge, Theresa C.; Katz, Garrett E.; Davis, Gregory P.; Jaquess, Kyle J.; Reinhard, Matthew J.; Costanzo, Michelle E.; Reggia, James A.; Gentili, Rodolphe J. (April 2020, Journal of Motor Learning and Development)

   null (Ed.)

   Few studies have examined high-level motor plans underlying cognitive-motor performance during practice of complex action sequences. These investigations have assessed performance through fairly simple metrics without examining how practice affects the structures of action sequences. By adapting the Levenshtein distance (LD) method to the motor domain, we propose a computational approach to accurately capture performance dynamics during practice of action sequences. Practice performance dynamics were assessed by computing the LD based on the number of insertions, deletions, and substitutions of actions needed to transform any sequence into a reference sequence (having a minimal number of actions to complete the task). Also, combining LD-based performance with mental workload metrics allowed assessment of cognitive-motor efficiency dynamics. This approach was tested on the Tower of Hanoi task. The findings revealed that throughout practice this method could capture: i) action sequence performance improvements as indexed by a reduced LD (decrease of insertions and substitutions), ii) structural modifications of the high-level plans, iii) an attenuation of mental workload, and iv) enhanced cognitive-motor efficiency. This effort complements prior work examining the practice of complex action sequences in healthy adults and has potential for probing cognitive-motor impairment in clinical populations as well as the development/assessment of cognitive robotic controllers. 

   more » « less