skip to main content


Title: Top-Down Attention Guidance Shapes Action Encoding in the pSTS
Abstract

The posterior superior temporal sulcus (pSTS) is a brain region characterized by perceptual representations of human body actions that promote the understanding of observed behavior. Increasingly, action observation is recognized as being strongly shaped by the expectations of the observer (Kilner 2011; Koster-Hale and Saxe 2013; Patel et al. 2019). Therefore, to characterize top-down influences on action observation, we evaluated the statistical structure of multivariate activation patterns from the action observation network (AON) while observers attended to the different dimensions of action vignettes (the action kinematics, goal, or identity of avatars jumping or crouching). Decoding accuracy varied as a function of attention instruction in the right pSTS and left inferior frontal cortex (IFC), with the right pSTS classifying actions most accurately when observers attended to the action kinematics and the left IFC classifying most accurately when observed attended to the actor’s goal. Functional connectivity also increased between the right pSTS and right IFC when observers attended to the actions portrayed in the vignettes. Our findings are evidence that the attentive state of the viewer modulates sensory representations in the pSTS, consistent with proposals that the pSTS occupies an interstitial zone mediating top-down context and bottom-up perceptual cues during action observation.

 
more » « less
Award ID(s):
1658278 1658560
NSF-PAR ID:
10488796
Author(s) / Creator(s):
; ; ; ; ;
Publisher / Repository:
Oxford University Press
Date Published:
Journal Name:
Cerebral Cortex
Volume:
31
Issue:
7
ISSN:
1047-3211
Page Range / eLocation ID:
3522 to 3535
Subject(s) / Keyword(s):
action observation action observation network attention multivariate pattern analysis posterior superior temporal sulcus
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Observing the actions of others engages a core action observation network (AON) that includes the bilateral inferior frontal cortex (IFC), posterior superior temporal sulcus (pSTS) and inferior parietal lobule (IPL) (Caspers et al., 2010). Each region in the AON has functional properties that are heterogeneous and include representing the perceptual properties of action, predicting action outcomes and making inferences as to the goals of the actor. Critically, recent evidence shows that neural representations within the pSTS are sharpened when attending to the kinematics of the actor, such that the top-down guided attention reshapes underlying neural representations. In this study we evaluate how attention alters network connectivity within the AON as a system. Cues directed participant's attention to the goal, kinematics, or identity depicted in short action animations while brain responses were measured by fMRI. We identified those parcels within the AON with functional connectivity modulated by task. Results show that connectivity between the right pSTS and right IFC, and bilateral extended STS (STS+) were modulated during action observation such that connections were strengthened when the participant was attending to the action than goal. This finding is contrasted by the univariate results, which no univariate modulations in these brain regions except for right IFC. Using the functional networks defined by Yeo et al. (2011), we identified the parcels that are modulated by the attention to consist mainly of the fronto-parietal control network and default mode networks. These results are consistent with models of top-down feedback from executive system in the IFC to pSTS and implicates a right lateralized dual pathway model for action observation when focused on whole-body kinematics. 
    more » « less
  2. Olanoff, D. ; Johnson, K. ; Spitzer, S. (Ed.)
    A key aspect of professional noticing includes attending to students’ mathematics (Jacobs et al., 2010). Initially, preservice teachers (PSTs) may attend to non-mathematics specific aspects of a classroom before attending to children’s procedures and then, eventually their conceptual reasoning (Barnhart & van Es, 2015). Use of 360 videos has been observed to increase the likelihood that PSTs will attend to more mathematics-specific student actions. This is due to an increased perceptual capacity, or the capacity of a representation to convey what is perceivable in a scenario (Kosko et al., in press). A 360 camera records a classroom omnidirectionally, allowing PSTs viewing the video to look in any direction. Moreover, several 360 cameras can be used in a single room to allow the viewer to move from one point in the recorded classroom to another; defined by Zolfaghari et al., 2020 as multi-perspective 360 video. Although multiperspective 360 has tremendous potential for immersion and presence (Gandolfi et al., 2021), we have not located empirical research clarifying whether or how this may affect PSTs’ professional noticing. Rather, most published research focuses on the use of a single camera. Given the dearth of research, we explored PSTs’ viewing of and teacher noticing related to a six-camera multiperspective 360 video. We examined 22 early childhood PSTs’ viewing of a 4th grade class using pattern blocks to find an equivalent fraction to 3/4. Towards the end of the video, one student suggested 8/12 as an equivalent fraction, but a peer claimed it was 9/12. The teacher prompts the peer to “prove it” and a brief discussion ensues before the video ends. After viewing the video, PSTs’ written noticings were solicited and coded. In our initial analysis, we examined whether PSTs attended to students’ fraction reasoning. Although many PSTs attended to whether 8/12 or 9/12 was the correct answer, only 7 of 22 attended to students’ part-whole reasoning of the fractions. Next, we examined the variance in how frequently PSTs switched their camera perspective using the unalikeability statistic. Unalikeability (U2) is a nonparametric measure of variance, ranging from 0 to 1, for nominal variables (Kader & Perry, 2007). Participants scores ranged from 0 to 0.80 (Median=0.47). We then compared participants’ U2 statistics for whether they attended (or not) to students mathematical reasoning in their written noticing. Findings revealed no statistically significant difference (U=38.5, p=0.316). On average, PSTs used 2-3 camera perspectives, and there was no observable benefit to using a higher number of cameras. These findings suggest that multiple perspectives may be useful for some, but not all PSTs’. 
    more » « less
  3. Abstract

    The grouping of sensory stimuli into categories is fundamental to cognition. Previous research in the visual and auditory systems supports a two‐stage processing hierarchy that underlies perceptual categorization: (a) a “bottom‐up” perceptual stage in sensory cortices where neurons show selectivity for stimulus features and (b) a “top‐down” second stage in higher level cortical areas that categorizes the stimulus‐selective input from the first stage. In order to test the hypothesis that the two‐stage model applies to the somatosensory system, 14 human participants were trained to categorize vibrotactile stimuli presented to their right forearm. Then, during an fMRI scan, participants actively categorized the stimuli. Representational similarity analysis revealed stimulus selectivity in areas including the left precentral and postcentral gyri, the supramarginal gyrus, and the posterior middle temporal gyrus. Crucially, we identified a single category‐selective region in the left ventral precentral gyrus. Furthermore, an estimation of directed functional connectivity delivered evidence for robust top‐down connectivity from the second to first stage. These results support the validity of the two‐stage model of perceptual categorization for the somatosensory system, suggesting common computational principles and a unified theory of perceptual categorization across the visual, auditory, and somatosensory systems.

     
    more » « less
  4. For students to advance beyond arithmetic, they must learn how to attend to the structure of math notation. This process can be challenging due to students' left-to-right computing tendencies. Brackets are used in mathematics to indicate precedence but can also be used as superfluous cues and perceptual grouping mechanisms in instructional materials to direct students’ attention and facilitate accurate and efficient problem solving. This online study examines the impact of operator position and superfluous brackets on students’ performance solving arithmetic problems. A total of 528 students completed a baseline assessment of math knowledge, then were randomly assigned to one of six conditions that varied in the placement of higher-order operator and the presence or absence of superfluous brackets: [a] brackets-left (e.g., (5 * 4) + 2 + 3), [b] no brackets-left (e.g., 5 * 4 + 2 + 3), [c] brackets-center (e.g., 2 + (5 * 4) + 3), [d] no brackets-center (e.g., 2 + 5 * 4 + 3), [e] brackets-right (e.g., 2 + 3 + (5 * 4)), and [f] no brackets-right (e.g., 2 + 3 + 5 * 4). Participants simplified expressions in an online learning platform with the goal to “master” the content by answering three questions correctly in a row. Results showed that, on average, students were more accurate in problem solving when the higher-order operator was on the left side and less accurate when it was on the right compared to in the center. There was also a main effect of the presence of brackets on mastery speed. However, interaction effects showed that these main effects were driven by the center position: superfluous brackets only improved accuracy when students solved expressions with brackets with the operator in the center. This study advances research on perceptual learning in math by revealing how operator position and presence of superfluous brackets impact students’ performance. Additionally, this research provides implications for instructors who can use perceptual cues to support students during problem solving. 
    more » « less
  5. For students to advance beyond arithmetic, they must learn how to attend to the structure of math notation. This process can be challenging due to students' left-to-right computing tendencies. Brackets are used in mathematics to indicate precedence but can also be used as superfluous cues and perceptual grouping mechanisms in instructional materials to direct students’ attention and facilitate accurate and efficient problem solving. This online study examines the impact of operator position and superfluous brackets on students’ performance solving arithmetic problems. A total of 528 students completed a baseline assessment of math knowledge, then were randomly assigned to one of six conditions that varied in the placement of higher-order operator and the presence or absence of superfluous brackets: 1) brackets-left (e.g., (5 * 4) + 2 + 3), 2) no brackets-left (e.g., 5 * 4 + 2 + 3), 3) brackets-center (e.g., 2 + (5 * 4) + 3), 4) no brackets-center (e.g., 2 + 5 * 4 + 3), 5) brackets-right (e.g., 2 + 3 + (5 * 4)), and 6) no brackets-right (e.g., 2 + 3 + 5 * 4). Participants simplified expressions in an online learning platform with the goal to “master” the content by answering three questions correctly in a row. Results showed that, on average, students were more accurate in problem solving when the higher-order operator was on the left side and less accurate when it was on the right compared to the center. There was also a main effect of the presence of brackets on mastery speed. However, interaction effects showed that these main effects were driven by the center position: superfluous brackets only improved accuracy when students solved expressions with brackets with the operator in the center. This study advances research on perceptual learning in math by revealing how operator position and presence of superfluous brackets impact students’ performance. Additionally, this research provides implications for instructors who can use perceptual cues to support students during problem solving. 
    more » « less