skip to main content


Title: Robust Multi-view Representation: A Unified Perspective from Multi-view Learning to Domain Adaption

Multi-view data are extensively accessible nowadays thanks to various types of features, different view-points and sensors which tend to facilitate better representation in many key applications. This survey covers the topic of robust multi-view data representation, centered around several major visual applications. First of all, we formulate a unified learning framework which is able to model most existing multi-view learning and domain adaptation in this line. Following this, we conduct a comprehensive discussion across these two problems by reviewing the algorithms along these two topics, including multi-view clustering, multi-view classification, zero-shot learning, and domain adaption. We further present more practical challenges in multi-view data analysis. Finally, we discuss future research including incomplete, unbalance, large-scale multi-view learning. This would benefit AI community from literature review to future direction.

 
more » « less
Award ID(s):
1651902
NSF-PAR ID:
10065420
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
IJCAI
Page Range / eLocation ID:
5434 to 5440
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Data from many real-world applications can be naturally represented by multi-view networks where the different views encode different types of relationships (e.g., friendship, shared interests in music, etc.) between real-world individuals or entities. There is an urgent need for methods to obtain low-dimensional, information preserving and typically nonlinear embeddings of such multi-view networks. However, most of the work on multi-view learning focuses on data that lack a network structure, and most of the work on network embeddings has focused primarily on single-view networks. Against this background, we consider the multi-view network representation learning problem, i.e., the problem of constructing low-dimensional information preserving embeddings of multi-view networks. Specifically, we investigate a novel Generative Adversarial Network (GAN) framework for Multi-View Network Embedding, namely MEGAN, aimed at preserving the information from the individual network views, while accounting for connectivity across (and hence complementarity of and correlations between) different views. The results of our experiments on two real-world multi-view data sets show that the embeddings obtained using MEGAN outperform the state-of-the-art methods on node classification, link prediction and visualization tasks.

     
    more » « less
  2. null (Ed.)
    Clustering is a machine learning paradigm of dividing sample subjects into a number of groups such that subjects in the same groups are more similar to those in other groups. With advances in information acquisition technologies, samples can frequently be viewed from different angles or in different modalities, generating multi-view data. Multi-view clustering, that clusters subjects into subgroups using multi-view data, has attracted more and more attentions. Although MVC methods have been developed rapidly, there has not been enough survey to summarize and analyze the current progress. Therefore, we propose a novel taxonomy of the MVC approaches. Similar with machine learning methods, we categorize them into generative and discriminative classes. In discriminative class, based on the way to integrate multiple views, we split it further into five groups: Common Eigenvector Matrix, Common Coefficient Matrix, Common Indicator Matrix, Direct Combination and Combination After Projection. Furthermore, we discuss the relationships between MVC and some related topics: multi-view representation, ensemble clustering, multi-task clustering, multi-view supervised and semi-supervised learning. Several representative real-world applications are elaborated for practitioners. Some commonly used multi-view datasets are introduced and several representative MVC algorithms from each group are run to conduct the comparison to analyze how and why they perform on those datasets. To promote future development of MVC approaches, we point out several open problems that may require further investigation and thorough examination. 
    more » « less
  3. Data from many real-world applications can be nat- urally represented by multi-view networks where the different views encode different types of rela- tionships (e.g., friendship, shared interests in mu- sic, etc.) between real-world individuals or enti- ties. There is an urgent need for methods to ob- tain low-dimensional, information preserving and typically nonlinear embeddings of such multi-view networks. However, most of the work on multi- view learning focuses on data that lack a net- work structure, and most of the work on net- work embeddings has focused primarily on single- view networks. Against this background, we con- sider the multi-view network representation learn- ing problem, i.e., the problem of constructing low- dimensional information preserving embeddings of multi-view networks. Specifically, we investigate a novel Generative Adversarial Network (GAN) framework for Multi-View Network Embedding, namely MEGAN, aimed at preserving the informa- tion from the individual network views, while ac- counting for connectivity across (and hence com- plementarity of and correlations between) differ- ent views. The results of our experiments on two real-world multi-view data sets show that the em- beddings obtained using MEGAN outperform the state-of-the-art methods on node classification, link prediction and visualization tasks. 
    more » « less
  4. Collaborative object localization aims to collaboratively estimate locations of objects observed from multiple views or perspectives, which is a critical ability for multi-agent systems such as connected vehicles. To enable collaborative localization, several model-based state estimation and learning-based localization methods have been developed. Given their encouraging performance, model-based state estimation often lacks the ability to model the complex relationships among multiple objects, while learning-based methods are typically not able to fuse the observations from an arbitrary number of views and cannot well model uncertainty. In this paper, we introduce a novel spatiotemporal graph filter approach that integrates graph learning and model-based estimation to perform multi-view sensor fusion for collaborative object localization. Our approach models complex object relationships using a new spatiotemporal graph representation and fuses multi-view observations in a Bayesian fashion to improve location estimation under uncertainty. We evaluate our approach in the applications of connected autonomous driving and multiple pedestrian localization. Experimental results show that our approach outperforms previous techniques and achieves the state-of-the-art performance on collaborative localization. 
    more » « less
  5. Abstract

    Current radio frequency (RF) classification techniques assume only one target in the field of view. Multi‐target recognition is challenging because conventional radar signal processing results in the superposition of target micro‐Doppler signatures, making it difficult to recognise multi‐target activity. This study proposes an angular subspace projection technique that generates multiple radar data cubes (RDC) conditioned on angle (RDC‐ω). This approach enables signal separation in the raw RDC, making possible the utilisation of deep neural networks taking the raw RF data as input or any other data representation in multi‐target scenarios. When targets are in closer proximity and cannot be separated by classical techniques, the proposed approach boosts the relative signal‐to‐noise ratio between targets, resulting in multi‐view spectrograms that boosts the classification accuracy when input to the proposed multi‐view DNN. Our results qualitatively and quantitatively characterise the similarity of multi‐view signatures to those acquired in a single‐target configuration. For a nine‐class activity recognition problem, 97.8% accuracy in a 3‐person scenario is achieved, while utilising DNN trained on single‐target data. We also present the results for two cases of close proximity (sign language recognition and side‐by‐side activities), where the proposed approach has boosted the performance.

     
    more » « less