More Like this

1. Verbal labels promote representational alignment even in the absence of communication

   Suffill, E. (January 2020, Proceedings for the 42nd Annual Meeting of the Cognitive Science Society)

   Denison, S.; Mack, M.; Xu, Y.; Armstrong, B.C. (Ed.)

   What affects whether one person represents an item in a similar way to another person? We examined the role of verbal labels in promoting representational alignment. Three groups of participants sorted novel shapes on perceived similarity. Prior to sorting, participants in two of the groups were pre-exposed to the shapes using a simple visual matching task and in one of these groups, shapes were accompanied by one of two novel category labels. Exposure with labels led people to represent the shapes in a more categorical way and to increased alignment between sorters, despite the two categories being visually distinct and participants in both pre-exposure conditions receiving identical visual experience of the shapes. Results hint that labels play a role in aligning people's mental representations, even in the absence of communication 

   more » « less
2. Mind Melds: Verbal Labels Induce Greater Representational Alignment

   https://doi.org/10.1162/opmi_a_00153  

   Suffill, Ellise; van_Paridon, Jeroen; Lupyan, Gary (January 2024, Open Mind)

   What determines whether two people represent something in a similar way? We examined the role of verbal labels in promoting representational alignment. Across two experiments, three groups of participants sorted novel shapes from two visually dissimilar categories. Prior to sorting, participants in two of the groups were pre-exposed to the shapes using a simple visual matching task designed to reinforce the visual category structure. In one of these groups, participants additionally heard one of two nonsense category labels accompanying the shapes. Exposure to these redundant labels led people to represent the shapes in a more categorical way, which led to greater alignment between sorters. We found this effect of label-induced alignment despite the two categories being highly visually distinct and despite participants in both pre-exposure conditions receiving identical visual experience with the shapes. Experiment 2 replicated this basic result using more even more stringent testing conditions. The results hint at the possibly extensive role that labels may play in aligning people’s mental representations. 

   more » « less
3. Shared spatiotemporal category representations in biological and artificial deep neural networks

   https://doi.org/https://doi.org/10.1371/journal.pcbi.1006327  

   Greene, Michelle R.; Hansen, Bruce C. (October 2018, PLOS computational biology)

   Visual scene category representations emerge very rapidly, yet the computational transformations that enable such invariant categorizations remain elusive. Deep convolutional neural networks (CNNs) perform visual categorization at near human-level accuracy using a feedforward architecture, providing neuroscientists with the opportunity to assess one successful series of representational transformations that enable categorization in silico. The goal of the current study is to assess the extent to which sequential scene category representations built by a CNN map onto those built in the human brain as assessed by high-density, time-resolved event-related potentials (ERPs). We found correspondence both over time and across the scalp: earlier (0–200 ms) ERP activity was best explained by early CNN layers at all electrodes. Although later activity at most electrode sites corresponded to earlier CNN layers, activity in right occipito-temporal electrodes was best explained by the later, fully-connected layers of the CNN around 225 ms post-stimulus, along with similar patterns in frontal electrodes. Taken together, these results suggest that the emergence of scene category representations develop through a dynamic interplay between early activity over occipital electrodes as well as later activity over temporal and frontal electrodes. 

   more » « less
4. Relating Representational Geometry to Cortical Geometry in the Visual Cortex

   Acosta, Francisco; Conwell, Colin; Sanborn, Sophia; Klindt, David; Miolane, Nina (December 2023, NeurIPS Workshop on Unifying Representations)

   A fundamental principle of neural representation is to minimize wiring length by spatially organizing neurons according to the frequency of their communication [Sterling and Laughlin, 2015]. A consequence is that nearby regions of the brain tend to represent similar content. This has been explored in the context of the visual cortex in recent works [Doshi and Konkle, 2023, Tong et al., 2023]. Here, we use the notion of cortical distance as a baseline to ground, evaluate, and interpret measures of representational distance. We compare several popular methods—both second-order methods (Representational Similarity Analysis, Centered Kernel Alignment) and first-order methods (Shape Metrics)—and calculate how well the representational distance reflects 2D anatomical distance along the visual cortex (the anatomical stress score). We evaluate these metrics on a large-scale fMRI dataset of human ventral visual cortex [Allen et al., 2022b], and observe that the 3 types of Shape Metrics produce representational-anatomical stress scores with the smallest variance across subjects, (Z score = -1.5), which suggests that first-order representational scores quantify the relationship between representational and cortical geometry in a way that is more invariant across different subjects. Our work establishes a criterion with which to compare methods for quantifying representational similarity with implications for studying the anatomical organization of high-level ventral visual cortex. 

   more » « less
5. Measuring individual differences in visual and verbal thinking styles

   Warford, Noel; Kunda, Maithilee (January 2018, 40th Annual Meeting of the Cognitive Science Society)

   Do people have dispositions towards visual or verbal thinking styles, i.e., a tendency towards one default representational modality versus the other? The problem in trying to answer this question is that visual/verbal thinking styles are challenging to measure. Subjective, introspective measures are the most common but often show poor reliability and validity; neuroimaging studies can provide objective evidence but are intrusive and resource-intensive. In previous work, we observed that in order for a purely behavioral testing method to be able to objectively evaluate a person’s visual/verbal thinking style, 1) the task must be solvable equally well using either visual or verbal mental representations, and 2) it must offer a secondary behavioral marker, in addition to primary performance measures, that indicates which modality is being used. We collected four such tasks from the psychology literature and conducted a small pilot study with adult participants to see the extent to which visual/verbal thinking styles can be differentiated using an individual’s results on these tasks. 

   more » « less