In recent years, face recognition systems have achieved
exceptional success due to promising advances in deep
learning architectures. However, they still fail to achieve the expected accuracy when matching profile images against a
gallery of frontal images. Current approaches either perform
pose normalization (i.e., frontalization) or disentangle
pose information for face recognition. We instead propose a
new approach to utilize pose as auxiliary information via
an attention mechanism. In this paper, we hypothesize that
pose-attended information using an attention mechanism
can guide contextual and distinctive feature extraction from
profile faces, which further benefits better representation
learning in an embedded domain. To achieve this, first, we
design a unified coupled profile-to-frontal face recognition
network. It learns the mapping from faces to a compact embedding
subspace via a class-specific contrastive loss. Second,
we develop a novel pose attention block (PAB) to specially
guide the pose-agnostic feature extraction from profile
faces. To be more specific, PAB is designed to explicitly
help the network to focus on important features along
both “channel” and “spatial” dimensions while learning
discriminative yet pose-invariant features in an embedding
subspace. To validate the effectiveness of our proposed
method, we conduct experiments on both controlled and in the-
wild benchmarks including Multi-PIE, CFP, and IJB-C, and
show superiority over the state-of-the-art.
more »
« less
Face Sketch Matching via Coupled Deep Transform Learning
Face sketch to digital image matching is an important
challenge of face recognition that involves matching across
different domains. Current research efforts have primarily
focused on extracting domain invariant representations or
learning a mapping from one domain to the other. In this
research, we propose a novel transform learning based approach
termed as DeepTransformer, which learns a transformation
and mapping function between the features of
two domains. The proposed formulation is independent of
the input information and can be applied with any existing
learned or hand-crafted feature. Since the mapping function
is directional in nature, we propose two variants of
DeepTransformer: (i) semi-coupled and (ii) symmetricallycoupled
deep transform learning. This research also uses
a novel IIIT-D Composite Sketch with Age (CSA) variations
database which contains sketch images of 150 subjects
along with age-separated digital photos. The performance
of the proposed models is evaluated on a novel application
of sketch-to-sketch matching, along with sketch-to-digital
photo matching. Experimental results demonstrate the robustness
of the proposed models in comparison to existing
state-of-the-art sketch matching algorithms and a commercial
face recognition system.
more »
« less
- NSF-PAR ID:
- 10053777
- Date Published:
- Journal Name:
- International Joint Conference on Biometrics (IJCB)
- Page Range / eLocation ID:
- 5429 to 5438
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Recent years have witnessed a growing body of research on autonomous activity recognition models for use in deployment of mobile systems in new settings such as when a wearable system is adopted by a new user. Current research, however, lacks comprehensive frameworks for transfer learning. Specifically, it lacks the ability to deal with partially available data in new settings. To address these limitations, we propose {\it OptiMapper}, a novel uninformed cross-subject transfer learning framework for activity recognition. OptiMapper is a combinatorial optimization framework that extracts abstract knowledge across subjects and utilizes this knowledge for developing a personalized and accurate activity recognition model in new subjects. To this end, a novel community-detection-based clustering of unlabeled data is proposed that uses the target user data to construct a network of unannotated sensor observations. The clusters of these target observations are then mapped onto the source clusters using a complete bipartite graph model. In the next step, the mapped labels are conditionally fused with the prediction of a base learner to create a personalized and labeled training dataset for the target user. We present two instantiations of OptiMapper. The first instantiation, which is applicable for transfer learning across domains with identical activity labels, performs a one-to-one bipartite mapping between clusters of the source and target users. The second instantiation performs optimal many-to-one mapping between the source clusters and those of the target. The many-to-one mapping allows us to find an optimal mapping even when the target dataset does not contain sufficient instances of all activity classes. We show that this type of cross-domain mapping can be formulated as a transportation problem and solved optimally. We evaluate our transfer learning techniques on several activity recognition datasets. Our results show that the proposed community detection approach can achieve, on average, 69%$ utilization of the datasets for clustering with an overall clustering accuracy of 87.5%. Our results also suggest that the proposed transfer learning algorithms can achieve up to 22.5% improvement in the activity recognition accuracy, compared to the state-of-the-art techniques. The experimental results also demonstrate high and sustained performance even in presence of partial data.more » « less
-
The continuing growth of scientific publications has posed a double-challenge to researchers, to not only grasp the overall research trends in a scientific domain, but also get down to research details embedded in a collection of core papers. Existing work on science mapping provides multiple tools to visualize research trends in domain on macro-level, and work from the digital humanities have proposed text visualization of documents, topics, sentences, and words on micro-level. However, existing micro-level text visualizations are not tailored for scientific paper corpus, and cannot support meso-level scientific reading, which aligns a set of core papers based on their research progress, before drilling down to individual papers. To bridge this gap, the present paper proposes LitStoryTeller+, an interactive system under a unified framework that can support both meso-level and micro-level scientific paper visual storytelling. More specifically, we use entities (concepts and terminologies) as basic visual elements, and visualize entity storylines across papers and within a paper borrowing metaphors from screen play. To identify entities and entity communities, named entity recognition and community detection are performed. We also employ a variety of text mining methods such as extractive text summarization and comparative sentence classification to provide rich textual information supplementary to our visualizations. We also propose a top-down story-reading strategy that best takes advantage of our system. Two comprehensive hypothetical walkthroughs to explore documents from the computer science domain and history domain with our system demonstrate the effectiveness of our story-reading strategy and the usefulness of LitStoryTeller+.more » « less
-
Forensic application of automatically matching skull with face images is an important research area linking biometrics with practical applications in forensics. It is an opportunity for biometrics and face recognition researchers to help the law enforcement and forensic experts in giving an identity to unidentified human skulls. It is an extremely challenging problem which is further exacerbated due to lack of any publicly available database related to this problem. This is the first research in this direction with a twofold contribution: (i) introducing the first of its kind skullface image pair database, IdentifyMe, and (ii) presenting a preliminary approach using the proposed semi-supervised formulation of transform learning. The experimental results and comparison with existing algorithms showcase the challenging nature of the problem. We assert that the availability of the database will inspire researchers to build sophisticated skull-to-face matching algorithms.more » « less
-
The success and impact of activity recognition algorithms largely depends on the availability of the labeled training samples and adaptability of activity recognition models across various domains. In a new environment, the pre-trained activity recognition models face challenges in presence of sensing bias- ness, device heterogeneities, and inherent variabilities in human behaviors and activities. Activity Recognition (AR) system built in one environment does not scale well in another environment, if it has to learn new activities and the annotated activity samples are scarce. Indeed building a new activity recognition model and training the model with large annotated samples often help overcome this challenging problem. However, collecting annotated samples is cost-sensitive and learning activity model at wild is computationally expensive. In this work, we propose an activity recognition framework, UnTran that utilizes source domains' pre-trained autoencoder enabled activity model that transfers two layers of this network to generate a common feature space for both source and target domain activities. We postulate a hybrid AR framework that helps fuse the decisions from a trained model in source domain and two activity models (raw and deep-feature based activity model) in target domain reducing the demand of annotated activity samples to help recognize unseen activities. We evaluated our framework with three real-world data traces consisting of 41 users and 26 activities in total. Our proposed UnTran AR framework achieves ≈ 75% F1 score in recognizing unseen new activities using only 10% labeled activity data in the target domain. UnTran attains ≈ 98% F1 score while recognizing seen activities in presence of only 2-3% of labeled activity samples.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/ICCV.2017.579
