What We Could Learn About Holistic Face Processing Only From Nonface Objects

Cronbach (1957)

advocated integrating an experimental approach, in which we manipulate variables, and a correlational approach, in which we study individual differences. Recent advances to understand holistic processing have required the strengths of both approaches, but also (perhaps surprisingly) the use of non-face objects to study a hallmark of face processing. This review is in two acts. In Act I, I outline why an individual differences approach is necessary to clarify what we measure when we study holistic processing. This work has primarily used faces, with non-face objects used in control conditions. In Act II, I argue that to study the causes of holistic effects we need experimental manipulations of the type and amount of experience, which requires non-face objects. I review work that integrates experimental manipulations of experience with the study of resulting individual differences in holistic effects, heeding Cronbach's advice.

ACT I -The importance of measurement.

Composites of parts from different faces have been used for more than 30 years to measure holistic processing. Young et al. (1987) found that people are slower to name the top half of a famous face when aligned with the bottom half of another famous person, relative to when the parts are misaligned. Hole (1994) reported a similar effect in a matching paradigm with unfamiliar faces and Farah et al. (1998) developed a selective-attention paradigm in which a tobe-ignored face part is either congruent or incongruent with a same/different decision about a cued part (see Figure 1). In this task, holistic processing (I remain intentionally agnostic to the underlying mechanism and simply define it as a hypothetical latent construct) is inferred from a congruency effect (CE) that reveals the influence of the to-be-ignored part. A meta-analysis of 48 studies (Richler et al., 2014) found the CE to be large, with no evidence of publication bias (when effects are inflated because studies with non-significant results are not published).

Holistic processing originally drew interest from psychologists because of its importance for face perception. For example, aside from special cases with shapes designed to elicit particularly strong perceptual grouping (Zhao et al., 2016), or when observers have expertise with a category (a topic I return to later), CEs in the composite task are small or non-existent for non-face objects (Richler et al, 2014;Meinhardt et al., 2014). Almost every aspect of the task has been manipulated to study its underlying mechanisms. Examples of such manipulations, with reference to numbers in Figure 1, include the following. The parts of the study (1) or test image (2) in a composite task trial can be shown either aligned or misaligned. The timing of the study (3) or of the test image (4) can be manipulated independently, each with or without a mask. When the taskrelevant part is not the same throughout the experiment or is not blocked, the cue (5) indicating the to-be-attended part can appear early (before or during the study image) or later, during the mask or test image. Feedback (6) may or may not be provided after the subject makes a same or different response.

Setting aside the fact that research with this paradigm has been used to advocate for a representational (Farah et al., 1998), perceptual (Meinhardt et al., 2014) decisional (Richler et al., 2008) or attentional (Fitousi, 2016) locus of holistic processing, two points are critical for the interpretation of CEs. First, in normal observers the CE for faces is robust and larger than that for objects, across essentially any combination of these manipulations. Second, the average CE varies in magnitude as a function of many of these manipulations. For instance, larger CEs are obtained in children and older adults when the cue appears on the test image rather than at study (Meinhardt et al., 2017), and in young adults, the CEs get smaller with feedback especially with long presentation times (Meinhardt et al., 2011). Thus, manipulations that are not critical to obtaining a CE influence its magnitude, a reminder that no measure is a pure reflection of a single construct (the unobservable cause of specific effects). The composite task is often inferred to tap in a putative process (holistic processing) that is more important for faces than non-face objects, but it also produces measurements that can reflect other influences such as motivation, cognitive control and attention.

Given these contaminating influences on performance, an individual differences approach can help assess how various tasks measure holistic processing per se. To establish the validity of holistic processing beyond specific effects, we can assess convergence using very different methods. In fact, with different measures of holistic processing, we could extract a latent variable that better reflects the underlying construct, remote from the idiosyncrasies of each task. But so far, the field has accumulated failures to demonstrate convergence between different measures generally considered relevant for holistic processing. For instance, Rezlescu et al. (2017) found little to no correlation between the face inversion effect (a large cost in performance for upsidedown faces), the part-whole effect (poorer recognition of parts in isolation than in whole faces) and the alignment effect in the composite task (see also DeGutis, 2013). Klargaard et al. (2018) found no relation between the face inversion effects on perceptual and memory tasks. Another problem is that modest correlations can reflect a confound, when both tasks use a small number of repeating faces, because some people learn more from repetition than others (Richler et al., 2015).

Measures targeting holistic processing may fail to correlate because their reliability is too poor (i.e., a measure that does not correlate with itself cannot correlate with other measures). This is a common problem for tasks designed to capture group effects, even if they are very sensitive for that purpose (Hedge et al., 2018). This challenge is beginning to be addressed with efforts to create measures with better psychometric properties. Holistic effects in some tasks, such as the part-whole paradigm, have been found to have too poor reliability to be valid (Sunday et al., 2017). Work based on the composite task has been more successful. Psychophysical methods have been used for reliable measurement of holistic effects in face composites, resulting in correlations across composite effects for gender and for age judgments (Gray et al., 2019). Another test, the Vanderbilt Holistic Processing Test for faces (VHPT-F, Richler et al., 2014b, Figure 2a), measure CEs with higher reliability than standard composite paradigms, with other advantages. It uses 3 alternatives on each trial to reduce guessing and problematic response biases (Richler & Gauthier, 2014b) and it does not repeat faces over trials to avoid contamination from learning effects (Richler et al., 2015). The CEs measured in the VHPT-F are larger than in the Figure 1. An example of an incongruent trial in a composite task, in which the cued part is the top, the response is "different" but the to-be-ignored bottom halves are the same. Numbers indicate various parts of the design that have been manipulated across various studies with influence on the magnitude of the congruency effect. Citations for example of studies that manipulated these factors are given in the text. matching version of the task and are independent from individual differences in cognitive control (Gauthier et al., 2018). The VHPT-F measures a stable trait based on 6 months test-retest (Richler et al., 2014b).

Unfortunately, better reliability does not guarantee higher correlations with other variables. Many authors assume that faces are processed efficiently because of holistic processing and that the best face recognizers should show more holistic processing. Early work on this topic suffered from the use of measurements that were insufficiently reliable. Even in large samples, work with the VHPT-F (or similar measures achieving acceptable reliability) revealed no correlation between CEs and face recognition ability (Richler et al., 2014b;Verhallen et al., 2017). A different approach found that better face recognizers use less, and relatively local, information relative to poorer face recognizers (Royer et al., 2018). The surprising finding that holistic processing of faces, even measured reliably, does not relate to face recognition ability motivates the work discussed in Act II.

ACT II -The problem with experience.

That face recognition ability does not predict holistic face processing is surprising, given evidence that experts (but not novices) show evidence of holistic effects with non-face objects (such as cars, fingerprints, chess pieces arrangements or novel objects, Richler et al., 2014). However, most of us have considerable experience with faces, and even the most limited amounts of experience with faces could be more than sufficient for high levels of holistic processing. For instance, studies find comparable CEs for faces of other ethnicities compared to one's own (e.g., Horry et al., 2015;Harrison et al., 2014). One study even found holistic effects for an artificial race of faces without any prior experience, suggesting that anything that looks like a face benefits from our vast experience with them (Chua et al., 2014).

A similar point was made by Oruc et al. (2018), on the basis that variability in experience only produce correlations between face identity and face expression recognition abilities at low-level of experience (such as in people with Autism). Similarly, the role of experience in holistic processing cannot be adequately tested with faces, because face recognition performance eventually becomes less sensitive to differences in experience -this is well illustrated by the developmental curve on a face recognition task, which increases sharply between age 10 and 20, but stays relatively stable for the next several decades (Germine et al., 2011).

In contrast, artificial objects are uniquely suited to investigate the effect of experience and the causes of holistic effects. Experience can change the perception of novel objects surprisingly rapidly. Diamond & Carey (1986) pioneered the idea that faces became special through experience. But with world dog experts with over 30 years of experience, it seemed like a great deal of experience was required for face-like processing. My dissertation work training experts with novel objects called Greebles demonstrated that 7 hours of learning could result in behavioral and neural hallmarks of face processing, including CEs in the composite task (Gauthier & Tarr, 2002). More recently, we trained 246 people for only 90 min per category, for four categories of novel objects, and measured CEs for these categories as well as a fifth category they had no experience with (Richler et al., 2019). With such a large sample, this short training was sufficient to produce an interaction between training and congruency effects in a composite task, in both accuracy and response times. Not only was the training short, it did not require learning any names -subjects only played a Space Invaders game requiring attention to the shape of individual objects.

To better measure individual differences in training effects on holistic effects, we recently designed a version of the VHPT for three categories of novel objects (VHPT-NO, Figure 2b). With Bayesian evidence, we first ascertained that we could not find any CE in novices on the test (Chua & Gauthier, in press). We then integrated experimental and correlational approaches in a study in which we trained 50 subjects each for a total of 10 hours, parametrically manipulating the level of experience with different categories (Chua and Gauthier, 2019). Importantly, we matched the level of experience for two of the categories, so if they received a little less than one hour (or up to 5 hours) of training for Greebles, the same was true for the other category (called Ziggerins). This resulted in a significant linear increase in the CE as a function of experience. The correlational analyses afforded by the VHPT-NOs revealed that holistic effects for Greebles and Ziggerins was correlated, as a result of correlated experience. Perhaps most interestingly, CEs in the VHPT-NO for the category that received the most training (about 8 hours) were comparable in magnitude to those obtained for faces in the VHPT-F: the effect size (mean CE divided by its standard deviation) was numerically larger for the trained objects than for faces. This confirms why it is difficult to observe effects of experience in holistic processing with faces. Adults have considerably more experience with faces, even those of less familiar groups (as in other-race effect studies), than they would ever receive in our studies.

Because we can manipulate the kind of experience people have with novel objects, they allow us to study the mechanisms that drive holistic effects. One of the first accounts of holistic effects postulates representations in which parts are not differentiated (Farah et al., 1998). Using training with novel objects, we challenged this theory by testing whether holistic effects can arise from experience with only parts (Chua, Richler & Gauthier, 2015). We trained 80 subjects to identify objects from two categories ("Ploks" and "Glips"): Ploks varied 10 times more in their top than bottom halves, and the reverse was true for Glips. By the end of a 3-sessions training, we measured CEs with objects made of top and bottom parts that were diagnostic during training, or objects made of previously non-diagnostic parts. We found face-like CEs only for objects made of diagnostic parts, even though such objects had never been seen during training. This led to the learned attention account of holistic effects, according to CEs stem from a history of learning to attend to the parts of objects. With faces, we cannot dissociate learning that many parts of a face contain diagnostic information from learning that these parts appear together.

Recent work using modeling of information processing architecture likewise found no support for various versions of integration of face parts into representations, and also suggested that learned attention may better explain performance in composite tasks (Cheng et al., 2018).

Theories in many areas of psychology propose that experience affects performance through changes in attentional mechanisms, although experience does not always increase failures of selective attention as it does with CEs. In the categorization literature, stimuli are categorized as varying along dimensions that are separable (like color and shape) or integral (like value and chroma of a color patch), with integrality indicating that attention cannot be applied selectively to one dimension. Color experts have been found to have better selective attention to dimensions of color (Burns & Shepp, 1988) and some research suggest that learning a category can increase perceptual discrimination (Goldstone, 1994) as well as perceptual separability (Soto & Ashby, 2015). Theories of perceptual learning also reveal that spatial attention and attentional suppression can be influenced by practice (e.g., Ahissar & Hochstein, 2000;Vidnyanszky & Sohn, 2005). Work on cognitive control shows that learned attentional settings can be applied to trained categories of objects, transferring to novel members of those categories (Bugg et al., 2011). While it is not yet clear which of these accounts best applies to CEs with faces and objects of expertise, training studies on holistic processing suggest that an attentional locus is likely (see also Fitousi, 2016;Cheng et al., 2018).

Conclusion

Many areas of psychology rely on the concept of holistic processing to study the developments of such effects, understand deficits, or social influences on perception. But any use of tasks meant to measure holistic processing requires caution. While more reliable measures are being created, measurements of holistic processing remain grounded in specific operational definitions still in search of a clear construct. CEs in the composite paradigm have been useful in both experimental and correlational studies. Training work with non-face novel objects reveals the conditions under which holistic effects arise, because experience, both its amount and type, can be experimentally manipulated. This research supports a learned attention model of holistic effects, whereby experience individuating objects results in difficulty ignoring specific parts that have proved diagnostic in the past. This line of research suggests that while holistic processing is not facespecific, most holistic processing may be domain-specific, because it arises through experience with the diagnostic parts of objects from a specific category. While the CE for faces can now be measured reliably, it has been more difficult to determine, using faces, if this variability comes from experience or from other influences. Up until now at least, it appears that we have learned more about the causes of holistic face processing using novel objects than using faces. powerful means of evaluating the construct validity of operational definitions. By seeking operational definitions that meet the standards of both approaches, we can ensure our experimental studies can test hypotheses that truly address the underlying causes of stable dispositions like holistic processing.