In computer vision, tracking humans across camera views
remains challenging, especially for complex scenarios with frequent
occlusions, significant lighting changes and other difficulties. Under
such conditions, most existing appearance and geometric cues are
not reliable enough to distinguish humans across camera views. To
address these challenges, this paper presents a stochastic attribute
grammar model for leveraging complementary and discriminative human
attributes for enhancing cross-view tracking. The key idea of our
method is to introduce a hierarchical representation, parse graph, to
describe a subject and its movement trajectory in both space and time
domains. This results in a hierarchical compositional representation,
comprising trajectory entities of varying level, including human boxes,
3D human boxes, tracklets and trajectories. We use a set of grammar
rules to decompose a graph node (e.g. tracklet) into a set of children
nodes (e.g. 3D human boxes), and augment each node with a set
of attributes, including geometry (e.g., moving speed, direction), accessories
(e.g., bags), and/or activities (e.g., walking, running). These
attributes serve as valuable cues, in addition to appearance features
(e.g., colors), in determining the associations of human detection boxes
across cameras. In particular, the attributes of a parent node are inherited
by its children nodes, resulting in consistency constraints over
the feasible parse graph. Thus, we cast cross-view human tracking as
finding the most discriminative parse graph for each subject in videos.
We develop a learning method to train this attribute grammar model from
weakly supervised training data. To infer the optimal parse graph and its
attributes, we develop an alternative parsing method that employs both
top-down and bottom-up computations to search the optimal solution.
We also explicitly reason the occlusion status of each entity in order
to deal with significant changes of camera viewpoints. We evaluate the
proposed method over public video benchmarks and demonstrate with
extensive experiments that our method clearly outperforms state-of-theart
tracking methods.
more »
« less
Recognizing Part Attributes With Insufficient Data
Recognizing the attributes of objects and their parts is central to many computer vision applications. Although great progress has been made to apply object-level recognition, recognizing the attributes of parts remains less applicable since the training data for part attributes recognition is usually scarce especially for internet-scale applications. Furthermore, most existing part attribute recognition methods rely on the part annotations which are more expensive to obtain. In order to solve the data insufficiency problem and get rid of dependence on the part annotation, we introduce a novel Concept Sharing Network (CSN) for part attribute recognition. A great advantage of CSN is its capability of recognizing the part attribute (a combination of part location and appearance pattern) that has insufficient or zero training data, by learning the part location and appearance pattern respectively from the training data that usually mix them in a single label. Extensive experiments on CUB, Celeb A, and a newly proposed human attribute dataset demonstrate the effectiveness of CSN and its advantages over other methods, especially for the attributes with few training samples. Further experiments show that CSN can also perform zero-shot part attribute recognition.
more »
« less
- Award ID(s):
- 1815561
- NSF-PAR ID:
- 10161300
- Date Published:
- Journal Name:
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Page Range / eLocation ID:
- 350 to 360
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
We develop a rigorous mathematical analysis of zero-shot learning with attributes. In this setting, the goal is to label novel classes with no training data, only detectors for attributes and a description of how those attributes are correlated with the target classes, called the class-attribute matrix. We develop the first non-trivial lower bound on the worst-case error of the best map from attributes to classes for this setting, even with perfect attribute detectors. The lower bound characterizes the theoretical intrinsic difficulty of the zero-shot problem based on the available information---the class-attribute matrix---and the bound is practically computable from it. Our lower bound is tight, as we show that we can always find a randomized map from attributes to classes whose expected error is upper bounded by the value of the lower bound. We show that our analysis can be predictive of how standard zero-shot methods behave in practice, including which classes will likely be confused with others.more » « less
-
Facial attribute recognition is conventionally computed from a single image. In practice, each subject may have multiple face images. Taking the eye size as an example, it should not change, but it may have different estimation in multiple images, which would make a negative impact on face recognition. Thus, how to compute these attributes corresponding to each subject rather than each single image is a profound work. To address this question, we deploy deep training for facial attributes prediction, and we explore the inconsistency issue among the attributes computed from each single image. Then, we develop two approaches to address the inconsistency issue. Experimental results show that the proposed methods can handle facial attribute estimation on either multiple still images or video frames, and can correct the incorrectly annotated labels. The experiments are conducted on two large public databases with annotations of facial attributes.more » « less
-
Modern recognition systems require large amounts of supervision to achieve accuracy. Adapting to new domains requires significant data from experts, which is onerous and can become too expensive. Zero-shot learning requires an annotated set of attributes for a novel category. Annotating the full set of attributes for a novel category proves to be a tedious and expensive task in deployment. This is especially the case when the recognition domain is an expert domain. We introduce a new field-guide-inspired approach to zero-shot annotation where the learner model interactively asks for the most useful attributes that define a class. We evaluate our method on classification benchmarks with attribute annotations like CUB, SUN, and AWA2 and show that our model achieves the performance of a model with full annotations at the cost of significantly fewer number of annotations. Since the time of experts is precious, decreasing annotation cost can be very valuable for real-world deployment.more » « less
-
Recent studies have revealed that sensitive and private attributes could be decoded from sEMG signals, which incurs significant privacy threats to the users of sEMG applications. Most researches so far focus on improving the accuracy and reliability of sEMG models, but much less attention has been paid to their privacy. To fill this gap, this paper implemented and evaluated a framework to optimize the sEMG-based data-sharing mechanism. Our primary goal is to remove sensitive attributes in the sEMG features before sharing them with primary tasks while maintaining the data utility. We disentangled the identity-invariant task-relevant representations from original sEMG features. We shared it with the downstream pattern recognition tasks to reduce the chance of sensitive attributes being inferred by potential attackers. The proposed method was evaluated on data from twenty subjects, with training and testing data acquired 3-25 days apart. Experimental results show that the disentangled representations significantly lower the success rate of identity inference attacks compared to the original feature and its sparse representations generated by the state-of-the-art feature projection methods. Furthermore, the utility of the disentangled representation is also evaluated in hand gesture recognition tasks, showing superior performance over other methods. This work shows that disentangled representations of sEMG signals are a promising solution for privacy-reserving applications.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/ICCV.2019.00044
