skip to main content


Title: Examining the arguments surrounding the argument-based approach to validation: A systematic review of validation methodology
As early as Descartes (1637/1970), logic and reason have been positioned as tools for individuals to advance their own understanding. By contrast, argumentation is an interactive, social exercise used for persuasion, collective cognition, and to advance shared knowledge (Mercier & Sperber, 2011, 2017). When one advances an argument, subjects it to the tests and challenges of others, and responds to questions and counterarguments, one’s thinking improves (Mercier & Sperber, 2017). Through argumentation, groups produce correct solutions more often than individuals (Moshman & Geil, 1998) and individual accuracy improves as well (Castelain, Girotto, Jamet, & Mercier, 2016). Since it was formally introduced by Kane (1990, 1992), the argument-based approach to validation has been promoted in the field of educational and psychological measurement as the preferred method for validating interpretations and uses of test scores (AERA, APA, & NCME, 2014; Kane, 2013; Schilling & Hill, 2007). Scholars continue to debate the best approaches for developing and supporting validity arguments, however (for examples, see Brennan, 2013; Kane, 2007).  more » « less
Award ID(s):
1644314
NSF-PAR ID:
10064697
Author(s) / Creator(s):
; ; ; ; ; ;
Date Published:
Journal Name:
American Educational Research Association
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    Since it was formalized by Kane, the argument‐based approach to validation has been promoted as the preferred method for validating interpretations and uses of test scores. Because validation is discussed in terms of arguments, and arguments are both interactive and social, the present review systematically examines the scholarly arguments which appear in 83 papers on argument‐based validation methods published in peer‐reviewed journals. Findings suggest that scholars generally agree on the nature and importance of argument‐based validation but disagree on whether validation should be structured or unstructured, formal or informal. Implications are discussed, including promotion of theStandards for Educational and Psychological Testing(AERA, APA, and NCME) as a foundation for consensus in the field.

     
    more » « less
  2. Hidden curriculum (HC) consist of the particular assumptions that are held by individuals about schooling that are manifested in practice (Smith, 2014). These assumptions can be recognized through socio cultural interactions, experiences with their physical surroundings, or exposure to virtual environments (The Glossary of Education Reform, 2017; Killick , 2016; Margolis, 2001; Smith, 2014). HC has been explored widely in fields such as education, psychology, business, and medicine (Baird, Bracken, & Grierson, 2016; Borges, Ferreira, Borges de Oliveria , Macini , Caldana , 2017; Cotton, Winter, & Bailey, 2013; Joughin , 2010; Margolis, 2001; Rabah , 2012; Smith, 2014) but is relatively unaddressed in engineering (Erickson, 2007; Villanueva et al., 2018) and more specifically neither the positive or negative implications of HC in engineering have been explored. This study sought to use a mixed method approach to understand the mechanisms behind HC recognition (via emotions and self efficacy) for engineering students and faculty nationwide. 
    more » « less
  3. The social/technical dualism in the engineering curriculum leaves students ill-prepared to tackle real-world technical problems in their social, economic, and political contexts (Cech, 2013; Faulkner, 2007; Trevelan, 2010, 2014). Increasingly, students have expressed the desire for their technical courses to show the interplay between social and technical considerations (Leydens & Lucena, 2017), but they have few opportunities to develop these sociotechnical ways of thinking (i.e., values, attitudes, and skills that integrate the social and technical). Instead, students are left to infer engineering as technically neutral through the instructional decisions that make up an engineering curriculum (Cech, 2013; Trevelan, 2014). In this study, we focus on how students understand the role of sociotechnical thinking in engineering. Particularly, this study centers seven minoritized students in an introductory engineering computation class who are pursuing an engineering degree. The study takes place at a medium private university in New England. These seven students are from a group of roughly seventy students split between two of the five sections for the course. These two sections were recently revised to include more sociotechnical readings, discussions, and homework facilitated with learning assistants. We are interested in understanding the self-described sense of belonging that these students feel as they relate it to learning about engineering as a sociotechnical field. While the dualism between engineering's technical and social dimensions has been studied in ASEE LEES papers, articles in Engineering Studies, broader engineering education research, and Science, Technology, and Science publications (e.g., Cech, 2013; Faulkner, 2007; Leydens & Lucena, 2017; Riley, 2017; Wisnioski, 2012), there is a need to connect this vast literature with the similarly extensive research on students' sense of belonging and engineering identity development, specifically for those students who have historically been excluded from engineering. Specifically, we draw on W.E.B. DuBois's notion of a 'double consciousness' from the Souls of Black Folks (1903) as a lens through which to understand how these seven students take on the political, economic, and social dimensions presented to them through a first-year engineering curricular redesign around engineering as sociotechnical. We note the small-n design of this study (Slaton & Pawley, 2018). The seven interviewed students are gender and racial minorities in engineering. However, we note that they do not represent all minoritized students in engineering, and to respect and elevate their experiences, we take a narrative approach. This study is intended to center the perspectives and experiences of these seven students as they navigate an engineering learning environment. We do not intend for the findings to be generalizable or exhaustive but informative as we think about scaling up the sociotechnical curricular redesign in engineering at this university and more broadly. 
    more » « less
  4. null (Ed.)
    Entrepreneurial education has been rapidly expanding within universities over the past 15 years with colleges of engineering being amongst the most active participants in embedding entrepreneurship into curricular and cocurricular activities (Pittaway & Cope, 2007). Well-developed and theoretically grounded educational interventions have been shown to increase entrepreneurial skills and perception among students. (Pittaway & Cope, 2007; Matlay & Caray, 2007; Duval-Couetil & Wheadon, 2013; Duval-Couetil & Rheed-Roads, 2012). Organizations including the National Science Foundation through the Lean Launch Curriculum and I-Corps program, VentureWell through curriculum development grants and their E-Team program, and the Kern Family Foundation through the Kern Entrepreneurial Education Network (KEEN) have provided significant funding to embed and transform entrepreneurial teaching and practice into colleges of engineering (Matthew et al., 2017; Pistrui, Blessing & Mekemson, 2008; Smith et al. 2017). This activity combines with an added emphasis among engineering programs to develop an entrepreneurial mindset among their engineering students with the belief that this will lead to them being more productive and innovative whether their career path leads them into established industry (becoming “intrapreneurs”) or later as entrepreneurs. While this trend toward developing more entrepreneurially minded engineering students is supported by global economic trends and a rapidly changing work environment, one factor has been largely overlooked in this process. Statistically, most entrepreneurial ventures fail, with disproportionately large value being created from a minority of entrepreneurial endeavors (Coats, 2019). Given this fact, until we find ways to drastically increase the success rate of entrepreneurial ventures, as we increase engineering students’ exposure to entrepreneurship, we are also increasing their exposure to failure very early in their careers. With this exposure, it is unknown whether sufficient preparation and education around project/venture failure is occurring to properly equip entrepreneurially minded engineering students to learn and grow from entrepreneurial failure. In this work in progress study, current and former engineering students who formed entrepreneurial ventures and experienced either failure of the venture or significant failure during the venture are interviewed to better understand the influences that led to both adaptive and maladaptive responses to these failures. Participants have been selected from those that have received funding through the national VentureWell E-Team program. This program awards three levels of funding and provides mentorship, training, and networking for the teams. The study uses the framework developed by Henry, Shorter, Charkoudian, Heemstra, and Corwin (2018) in which they associate pre-failure dispositions related to fixed and growth mindset (Dweck, 2000, 2006) and mastery vs. performance disposition (Pintrich, 2000 a, b). Our work will utilize this framework to guide the research, but more importantly will provide a unique context for analysis, specifically within engineering entrepreneurship, which will add to the body of work and expand the understanding of this pre-failure/post-failure disposition framework. Initial interview data and analysis will be presented in the context of this framework with preliminary insights to be shared with those in the field. 
    more » « less
  5. Abstract

    Tracing the development of Lehman Brothers from their roots as a dry goods store in Montgomery, Alabama, into a Fortune 500 global financial services company that collapsed in the 2007 financial crisis reveals the contours of US‐style racial capitalism. We highlight how race and capital are geographically rooted in the United States. At the heart of our argument is an understanding of the geographic imperative of capital which creates and exploits differences to wrench capital’s profits from the blood, sweat, and toil of racialised bodies. Perhaps most geographically significant, we advance a second, interrelated argument, the geographic case for reparations. By arguing the system itself is entirely and wholly wrapped in race in ways that extend and advance practices of exploitation, we understand reparations that do not fundamentally undermine, transform, destabilise or smash the system as doomed to be short‐term solutions to long‐term structural problems.

     
    more » « less