More Like this

1. Multimodal Transformer for Unaligned Multimodal Language Sequences

   https://doi.org/10.18653/v1/P19-1656  

   Tsai, Yao-Hung Hubert ; Bai, Shaojie ; Liang, Paul Pu ; Kolter, J. Zico ; Morency, Louis-Philippe ; Salakhutdinov, Ruslan ( January 2019 , Proceedings of the Annual Meeting of the Association for Computational Linguistics)
2. Multimodal Routing: Improving Local and Global Interpretability of Multimodal Language Analysis.

   Tsai, Y.-H. ; Ma, M. ; Yang, M. ; Salakhutdinov, R. ; Morency, L.-P. ( January 2020 , Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020)

   null (Ed.)
3. Multimodal Routing: Improving Local and Global Interpretability of Multimodal Language Analysis

   Tsai, Y.-H. ; Ma, M. ; Yang, M. ; Salakhutdinov, R. ; Morency, L.-P. ( January 2020 , Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP))
4. Multimodal approaches to functional connectivity in autism spectrum disorders: An integrative perspective: Multimodal Connectivity in ASDs

   https://doi.org/10.1002/dneu.22570  

   Mash, Lisa E. ; Reiter, Maya A. ; Linke, Annika C. ; Townsend, Jeanne ; Müller, Ralph-Axel ( May 2018 , Developmental Neurobiology)
5. Multimodal Context-Based Continuous Authentication

   https://doi.org/10.1109/IJCB57857.2023.10448626  

   Aathreya, Saandeep ; Chaudhary, Meghna ; Neal, Tempestt ; Canavan, Shaun ( September 2023 , IEEE)

   We present a new multimodal, context-based dataset for continuous authentication. The dataset contains 27 subjects, with an age range of [8, 72], where data has been collected across multiple sessions while the subjects are watching videos meant to elicit an emotional response. Collected data includes accelerometer data, heart rate, electrodermal activity, skin temperature, and face videos. We also propose a baseline approach for fair comparisons when using the proposed dataset. The approach uses a combination of a pretrained backbone network with supervised contrastive loss for face. Time-series features are also extracted, from the physiological signals, which are used for classification. This approach, on the proposed dataset, results in an average accuracy, precision, and recall of 76.59%, 88.90, and 53.25, respectively, on electrical signals, and 90.39%, 98.77, and 75.71, respectively on face videos. 

   more » « less