skip to main content


Title: Improving analytical reasoning and argument understanding: a quasi-experimental field study of argument visualization
Abstract

The ability to analyze arguments is critical for higher-level reasoning, yet previous research suggests that standard university education provides only modest improvements in students’ analytical-reasoning abilities. What pedagogical approaches are most effective for cultivating these skills? We investigated the effectiveness of a 12-week undergraduate seminar in which students practiced a software-based technique for visualizing the logical structures implicit in argumentative texts. Seminar students met weekly to analyze excerpts from contemporary analytic philosophy papers, completed argument visualization problem sets, and received individualized feedback on a weekly basis. We found that seminar students improved substantially more on LSAT Logical Reasoning test forms than did control students (d = 0.71, 95% CI: [0.37, 1.04],p < 0.001), suggesting that learning how to visualize arguments in the seminar led to large generalized improvements in students’ analytical-reasoning skills. Moreover, blind scoring of final essays from seminar students and control students, drawn from a parallel lecture course, revealed large differences in favor of seminar students (d = 0.87, 95% CI: [0.26, 1.48],p = 0.005). Seminar students understood the arguments better, and their essays were more accurate and effectively structured. Taken together, these findings deepen our understanding of how visualizations support logical reasoning and provide a model for improving analytical-reasoning pedagogy.

 
more » « less
NSF-PAR ID:
10154115
Author(s) / Creator(s):
; ; ;
Publisher / Repository:
Nature Publishing Group
Date Published:
Journal Name:
npj Science of Learning
Volume:
3
Issue:
1
ISSN:
2056-7936
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Background

    This paper contributes to current discussions about supporting prospective teachers (PSTs) in developing skills of noticing students’ mathematical thinking. We draw attention to PSTs’ initial noticing skills (prior to instruction focused on supporting noticing) as PSTs engage in analyzing written artifacts of student work and video-records. We examined and compared PSTs’ noticing skills as they analyzed how students reason about, generalize, and justify generalizations of figural patterns given student written work and video records. We identified aspects of student thinking about generalizations and justifications, which PSTs addressed and interpreted. We also examined how PSTs respond to students as they analyze student thinking given written artifacts of student work or video-records of small group discussions, and we identified the foci of PSTs’ responding practice.

    Results

    Our data revealed that PSTs’ initial noticing skills of student generalizations and justifications differed while accounting for ways in which student thinking was externalized (written work or video-records). PSTs’ attending-and-interpreting and their responding practices were focused on mathematically significant aspects of student thinking to a greater extent in the context of analyzing written artifacts compared to video records. While analyzing students’ written work, PSTs demonstrated greater attention to ways in which students analyzed patterns, students’ generalization strategies, and justifications linked to an understanding of the pattern structure, compared to analyzing student thinking captured via videos.

    Conclusion

    Our results document that without providing any intentional support for PSTs’ noticing skills, PSTs are more deliberate to focus on mathematically significant aspects of student thinking while analyzing written artifacts of student work compared to video-records. We believe that the analysis of student written work might demand from PSTs to be more analytical. While examining written representations, PSTs have to reconstruct students’ reasoning. Unlike the videos where the students tell or use gestures to express their thinking, written work provides fewer clues about student thinking. Thus, written work demands a deeper level of engagement from PSTs as they strive to understand student reasoning. Our study extends research on PSTs’ noticing skills by documenting differences in PSTs’ noticing in relation to the nature of artifacts of student work that PSTs analyze. Our work also adds to prior research on PSTs’ noticing by characterizing specific aspects of students’ thinking about pattern generalizations and justifications that PSTs address as they analyze student thinking and respond to students.

     
    more » « less
  2. Engineering education traditionally focuses on technical content and problem-solving, leaving little room in the curriculum to examine broader environmental and sociotechnical impacts of engineering work. However, if engineers wish to have intentional, positive influences on these broader impacts, skills for reflective thinking and ethical decision-making are essential. The arts and humanities can provide important and often neglected perspectives for engineers in developing such skills. In a recent seminar course for civil/environmental engineers, we explored ways of developing these skills through activities including Visual Thinking Strategies (VTS), in-class readings & discussions, essay writing, and portfolio assignments. In this paper, we present selected findings from this experimental course. While the class was small, comprised of a dozen graduate students, results were encouraging. For example, findings from qualitative thematic analysis of pre- and post-course essays showed an increase in recognition of the importance of breadth of knowledge and/or perspective. Similarly, pre-post Likert-type survey results showed a statistically significant increase (p<0.005, d=1, n=10) in Contextual Competence, a self-reported measure of ability to anticipate and understand the impacts and constraints of broader contexts on engineering solutions. These findings are preliminary but suggest the course helped students develop capacity for reflection through arts- and humanities-based activities. 
    more » « less
  3. Understanding an online argumentative discussion is essential for understanding users' opinions on a topic and their underlying reasoning. A key challenge in determining completeness and persuasiveness of argumentative discussions is to assess how arguments under a topic are connected in a logical and coherent manner. Online argumentative discussions, in contrast to essays or face-to-face communication, challenge techniques for judging argument relevance because online discussions involve multiple participants and often exhibit incoherence in reasoning and inconsistencies in writing style. We define relevance as the logical and topical connections between small texts representing argument fragments in online discussions. We provide a corpus comprising pairs of sentences, labeled with argumentative relevance between the sentences in each pair. We propose a computational approach relying on content reduction and a Siamese neural network architecture for modeling argumentative connections and determining argumentative relevance between texts. Experimental results indicate that our approach is effective in measuring relevance between arguments, and outperforms strong and well-adopted baselines.Further analysis demonstrates the benefit of using our argumentative relevance encoding on a downstream task, predicting how impactful an online comment is to certain topic, comparing to encoding that does not consider logical connection. 
    more » « less
  4. Background Online physician reviews are an important source of information for prospective patients. In addition, they represent an untapped resource for studying the effects of gender on the doctor-patient relationship. Understanding gender differences in online reviews is important because it may impact the value of those reviews to patients. Documenting gender differences in patient experience may also help to improve the doctor-patient relationship. This is the first large-scale study of physician reviews to extensively investigate gender bias in online reviews or offer recommendations for improvements to online review systems to correct for gender bias and aid patients in selecting a physician. Objective This study examines 154,305 reviews from across the United States for all medical specialties. Our analysis includes a qualitative and quantitative examination of review content and physician rating with regard to doctor and reviewer gender. Methods A total of 154,305 reviews were sampled from Google Place reviews. Reviewer and doctor gender were inferred from names. Reviews were coded for overall patient experience (negative or positive) by collapsing a 5-star scale and coded for general categories (process, positive/negative soft skills), which were further subdivided into themes. Computational text processing methods were employed to apply this codebook to the entire data set, rendering it tractable to quantitative methods. Specifically, we estimated binary regression models to examine relationships between physician rating, patient experience themes, physician gender, and reviewer gender). Results Female reviewers wrote 60% more reviews than men. Male reviewers were more likely to give negative reviews (odds ratio [OR] 1.15, 95% CI 1.10-1.19; P<.001). Reviews of female physicians were considerably more negative than those of male physicians (OR 1.99, 95% CI 1.94-2.14; P<.001). Soft skills were more likely to be mentioned in the reviews written by female reviewers and about female physicians. Negative reviews of female doctors were more likely to mention candor (OR 1.61, 95% CI 1.42-1.82; P<.001) and amicability (OR 1.63, 95% CI 1.47-1.90; P<.001). Disrespect was associated with both female physicians (OR 1.42, 95% CI 1.35-1.51; P<.001) and female reviewers (OR 1.27, 95% CI 1.19-1.35; P<.001). Female patients were less likely to report disrespect from female doctors than expected from the base ORs (OR 1.19, 95% CI 1.04-1.32; P=.008), but this effect overrode only the effect for female reviewers. Conclusions This work reinforces findings in the extensive literature on gender differences and gender bias in patient-physician interaction. Its novel contribution lies in highlighting gender differences in online reviews. These reviews inform patients’ choice of doctor and thus affect both patients and physicians. The evidence of gender bias documented here suggests review sites may be improved by providing information about gender differences, controlling for gender when presenting composite ratings for physicians, and helping users write less biased reviews. 
    more » « less
  5. Background

    A lack of in utero imaging data hampers our understanding of the connections in the human fetal brain. Generalizing observations from postmortem subjects and premature newborns is inaccurate due to technical and biological differences.

    Purpose

    To evaluate changes in fetal brain structural connectivity between 23 and 35 weeks postconceptional age using a spatiotemporal atlas of diffusion tensor imaging (DTI).

    Study Type

    Retrospective.

    Population

    Publicly available diffusion atlases, based on 60 healthy women (age 18–45 years) with normal prenatal care, from 23 and 35 weeks of gestation.

    Field Strength/Sequence

    3.0 Tesla/DTI acquired with diffusion‐weighted echo planar imaging (EPI).

    Assessment

    We performed whole‐brain fiber tractography from DTI images. The cortical plate of each diffusion atlas was segmented and parcellated into 78 regions derived from the Edinburgh Neonatal Atlas (ENA33). Connectivity matrices were computed, representing normalized fiber connections between nodes. We examined the relationship between global efficiency (GE), local efficiency (LE), small‐worldness (SW), nodal efficiency (NE), and betweenness centrality (BC) with gestational age (GA) and with laterality.

    Statistical Tests

    Linear regression was used to analyze changes in GE, LE, NE, and BC throughout gestation, and to assess changes in laterality. Thet‐tests were used to assess SW.P‐values were corrected using Holm‐Bonferroni method. A correctedP‐value <0.05 was considered statistically significant.

    Results

    Network analysis revealed a significant weekly increase in GE (5.83%/week, 95% CI 4.32–7.37), LE (5.43%/week, 95% CI 3.63–7.25), and presence of SW across GA. No significant hemisphere differences were found in GE (P = 0.971) or LE (P = 0.458). Increasing GA was significantly associated with increasing NE in 41 nodes, increasing BC in 3 nodes, and decreasing BC in 2 nodes.

    Data Conclusion

    Extensive network development and refinement occur in the second and third trimesters, marked by a rapid increase in global integration and local segregation.

    Level of Evidence

    3

    Technical Efficacy

    Stage 2

     
    more » « less