More Like this

1. Multimodal Transformer for Unaligned Multimodal Language Sequences

   https://doi.org/10.18653/v1/P19-1656  

   Tsai, Yao-Hung Hubert; Bai, Shaojie; Liang, Paul Pu; Kolter, J. Zico; Morency, Louis-Philippe; Salakhutdinov, Ruslan (January 2019, Proceedings of the Annual Meeting of the Association for Computational Linguistics)
2. Multimodal Routing: Improving Local and Global Interpretability of Multimodal Language Analysis

   Tsai, Y.-H.; Ma, M.; Yang, M.; Salakhutdinov, R.; Morency, L.-P. (January 2020, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP))
3. Multimodal Routing: Improving Local and Global Interpretability of Multimodal Language Analysis.

   Tsai, Y.-H.; Ma, M.; Yang, M.; Salakhutdinov, R.; Morency, L.-P. (January 2020, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020)

   null (Ed.)
4. Temporal Multimodal Multivariate Learning

   https://doi.org/10.1145/3534678.3539159  

   Park, Hyoshin; Darko, Justice; Deshpande, Niharika; Pandey, Venktesh; Su, Hui; Ono, Masahiro; Barkley, Dedrick; Folsom, Larkin; Posselt, Derek; Chien, Steve (August 2022, ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD2022))

   We introduce temporal multimodal multivariate learning, a new family of decision making models that can indirectly learn and transfer online information from simultaneous observations of a probability distribution with more than one peak or more than one outcome variable from one time stage to another. We approximate the posterior by sequentially removing additional uncertainties across different variables and time, based on data-physics driven correlation, to address a broader class of challenging time-dependent decision-making problems under uncertainty. Extensive experiments on real-world datasets ( i.e., urban traffic data and hurricane ensemble forecasting data) demonstrate the superior performance of the proposed targeted decision-making over the state-of-the-art baseline prediction methods across various settings. 

   more » « less
5. Deep Multimodal Complementarity Learning

   https://doi.org/10.1109/tnnls.2022.3165180  

   Wang, Daheng; Zhao, Tong; Yu, Wenhao; Chawla, Nitesh V.; Jiang, Meng (April 2022, IEEE Transactions on Neural Networks and Learning Systems)