More Like this

1. Multimodal Routing: Improving Local and Global Interpretability of Multimodal Language Analysis.

   Tsai, Y.-H. ; Ma, M. ; Yang, M. ; Salakhutdinov, R. ; Morency, L.-P. ( January 2020 , Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020)

   null (Ed.)
2. Ensemble Sequence Level Training for Multimodal MT: OSU-Baidu WMT18 Multimodal Machine Translation System Report

   Zheng, Renjie ; Yang, Yilin ; Ma, Mingbo ; Huang, Liang ( November 2018 , In Proceedings of WMT 2018)
3. Multimodal Routing: Improving Local and Global Interpretability of Multimodal Language Analysis

   Tsai, Y.-H. ; Ma, M. ; Yang, M. ; Salakhutdinov, R. ; Morency, L.-P. ( January 2020 , Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP))
4. Multimodal approaches to functional connectivity in autism spectrum disorders: An integrative perspective: Multimodal Connectivity in ASDs

   https://doi.org/10.1002/dneu.22570  

   Mash, Lisa E. ; Reiter, Maya A. ; Linke, Annika C. ; Townsend, Jeanne ; Müller, Ralph-Axel ( May 2018 , Developmental Neurobiology)
5. Multimodal Context-Based Continuous Authentication

   https://doi.org/10.1109/IJCB57857.2023.10448626  

   Aathreya, Saandeep ; Chaudhary, Meghna ; Neal, Tempestt ; Canavan, Shaun ( September 2023 , IEEE)

   We present a new multimodal, context-based dataset for continuous authentication. The dataset contains 27 subjects, with an age range of [8, 72], where data has been collected across multiple sessions while the subjects are watching videos meant to elicit an emotional response. Collected data includes accelerometer data, heart rate, electrodermal activity, skin temperature, and face videos. We also propose a baseline approach for fair comparisons when using the proposed dataset. The approach uses a combination of a pretrained backbone network with supervised contrastive loss for face. Time-series features are also extracted, from the physiological signals, which are used for classification. This approach, on the proposed dataset, results in an average accuracy, precision, and recall of 76.59%, 88.90, and 53.25, respectively, on electrical signals, and 90.39%, 98.77, and 75.71, respectively on face videos. 

   more » « less