More Like this

1. Holistic face recognition is an emergent phenomenon of spatial processing in face-selective regions

   https://doi.org/10.1038/s41467-021-24806-1  

   Poltoratski, Sonia ; Kay, Kendrick ; Finzi, Dawn ; Grill-Spector, Kalanit ( August 2021 , Nature Communications)

   Spatial processing by receptive fields is a core property of the visual system. However, it is unknown how spatial processing in high-level regions contributes to recognition behavior. As face inversion is thought to disrupt typical holistic processing of information in faces, we mapped population receptive fields (pRFs) with upright and inverted faces in the human visual system. Here we show that in face-selective regions, but not primary visual cortex, pRFs and overall visual field coverage are smaller and shifted downward in response to face inversion. From these measurements, we successfully predict the relative behavioral detriment of face inversion at different positions in the visual field. This correspondence between neural measurements and behavior demonstrates how spatial processing in face-selective regions may enable holistic perception. These results not only show that spatial processing in high-level visual regions is dynamically used towards recognition, but also suggest a powerful approach for bridging neural computations by receptive fields to behavior.

    

   more » « less
2. Using population receptive field models to elucidate spatial integration in high-level visual cortex

   https://doi.org/https://doi.org/10.32470/CCN.2019.1178-0  

   Poltoratski, Sonia Kay ( September 2019 , 2019 Conference on Cognitive Computational Neuroscience, 13-16 September 2019, Berlin, Germany)

   While spatial information and biases have been consistently reported in high-level face regions, the functional contribution of this information toward face recognition behavior is unclear. Here, we propose that spatial integration of information plays a critical role in a hallmark phenomenon of face perception: holistic processing, or the tendency to process all features of a face concurrently rather than independently. We sought to gain insight into the neural basis of face recognition behavior by using a voxelwise encoding model of spatial selectivity to characterize the human face network using both typical face stimuli, and stimuli thought to disrupt normal face perception. We mapped population receptive fields (pRFs) using 3T fMRI in 6 participants using upright as well as inverted faces, which are thought to disrupt holistic processing. Compared to upright faces, inverted faces yielded substantial differences in measured pRF size, position, and amplitude. Further, these differences increased in magnitude along the face network hierarchy, from IOG- to pFus- and mFus-faces. These data suggest that pRFs in high-level regions reflect complex stimulus- dependent neural computations that underlie variations in recognition performance. 

   more » « less
3. Visual Psychophysics for Making Face Recognition Algorithms More Explainable

   https://doi.org/10.1007/978-3-030-01267-0  

   Webster, Brandon R ; Kwon, So Yon ; Clarizio, Christopher ; Anthony, Samuel E ; Scheirer, Walter J ( July 2018 , European Conference on Computer Vision 2018)

   Abstract. Scientific fields that are interested in faces have developed their own sets of concepts and procedures for understanding how a tar- get model system (be it a person or algorithm) perceives a face under varying conditions. In computer vision, this has largely been in the form of dataset evaluation for recognition tasks where summary statistics are used to measure progress. While aggregate performance has continued to improve, understanding individual causes of failure has been difficult, as it is not always clear why a particular face fails to be recognized, or why an impostor is recognized by an algorithm. Importantly, other fields studying vision have addressed this via the use of visual psychophysics: the controlled manipulation of stimuli and careful study of the responses they evoke in a model system. In this paper, we suggest that visual psychophysics is a viable methodology for making face recognition algo- rithms more explainable. A comprehensive set of procedures is developed for assessing face recognition algorithm behavior, which is then deployed over state-of-the-art convolutional neural networks and more basic, yet still widely used, shallow and handcrafted feature-based approaches. 

   more » « less
4. The development of race effects in face processing from childhood through adulthood: Neural and behavioral evidence

   https://doi.org/10.1111/desc.13058  

   Golarai, Golijeh ; Ghahremani, Dara G. ; Greenwood, Anders C. ; Gabrieli, John D. E. ; Eberhardt, Jennifer L. ( December 2020 , Developmental Science)

   Most adults are better at recognizing recently encountered faces of their own race, relative to faces of other races. In adults, this race effect in face recognition is associated with differential neural representations of own‐ and other‐race faces in the fusiform face area (FFA), a high‐level visual region involved in face recognition. Previous research has linked these differential face representations in adults to viewers’ implicit racial associations. However, despite the fact that the FFA undergoes a gradual development which continues well into adulthood, little is known about the developmental time‐course of the race effect in FFA responses. Also unclear is how this race effect might relate to the development of face recognition or implicit associations with own‐ or other‐races during childhood and adolescence. To examine the developmental trajectory of these race effects, in a cross‐sectional study of European American (EA) children (ages 7–11), adolescents (ages 12–16) and adults (ages 18–35), we evaluated responses to adult African American (AA) and EA face stimuli, using functional magnetic resonance imaging and separate behavioral measures outside the scanner. We found that FFA responses to AA and EA faces differentiated during development from childhood into adulthood; meanwhile, the magnitudes of race effects increased in behavioral measures of face‐recognition and implicit racial associations. These three race effects were positively correlated, even after controlling for age. These findings suggest that social and perceptual experiences shape a protracted development of the race effect in face processing that continues well into adulthood.

    

   more » « less
5. Intracranial Electroencephalography and Deep Neural Networks Reveal Shared Substrates for Representations of Face Identity and Expressions

   https://doi.org/10.1523/JNEUROSCI.1277-22.2023  

   Schwartz, Emily ; Alreja, Arish ; Richardson, R. Mark ; Ghuman, Avniel ; Anzellotti, Stefano ( May 2023 , The Journal of Neuroscience)

   According to a classical view of face perception (Bruce and Young, 1986; Haxby et al., 2000), face identity and facial expression recognition are performed by separate neural substrates (ventral and lateral temporal face-selective regions, respectively). However, recent studies challenge this view, showing that expression valence can also be decoded from ventral regions (Skerry and Saxe, 2014; Li et al., 2019), and identity from lateral regions (Anzellotti and Caramazza, 2017). These findings could be reconciled with the classical view if regions specialized for one task (either identity or expression) contain a small amount of information for the other task (that enables above-chance decoding). In this case, we would expect representations in lateral regions to be more similar to representations in deep convolutional neural networks (DCNNs) trained to recognize facial expression than to representations in DCNNs trained to recognize face identity (the converse should hold for ventral regions). We tested this hypothesis by analyzing neural responses to faces varying in identity and expression. Representational dissimilarity matrices (RDMs) computed from human intracranial recordings (n= 11 adults; 7 females) were compared with RDMs from DCNNs trained to label either identity or expression. We found that RDMs from DCNNs trained to recognize identity correlated with intracranial recordings more strongly in all regions tested—even in regions classically hypothesized to be specialized for expression. These results deviate from the classical view, suggesting that face-selective ventral and lateral regions contribute to the representation of both identity and expression.

   SIGNIFICANCE STATEMENTPrevious work proposed that separate brain regions are specialized for the recognition of face identity and facial expression. However, identity and expression recognition mechanisms might share common brain regions instead. We tested these alternatives using deep neural networks and intracranial recordings from face-selective brain regions. Deep neural networks trained to recognize identity and networks trained to recognize expression learned representations that correlate with neural recordings. Identity-trained representations correlated with intracranial recordings more strongly in all regions tested, including regions hypothesized to be expression specialized in the classical hypothesis. These findings support the view that identity and expression recognition rely on common brain regions. This discovery may require reevaluation of the roles that the ventral and lateral neural pathways play in processing socially relevant stimuli.

    

   more » « less