An official website of the United States government
ObjectivesIn calls for excellent and equitable Computer Science (CS) education, the wordrigoroften appears, but it often goes undefined. The goal of this work is to understand how CS teachers, instructors, and students conceive of rigor. Research Questions:1) What do CS instructors think rigor is? and 2) What do students think rigor is? Methods:Using the principles of phenomenological research, we conducted a semi-structured interview study with 10 post-secondary CS students, 10 secondary CS teachers, and 9 post-secondary CS instructors, to understand their conceptions of rigor. Results:Analysis showed that no participants had the same understanding of rigor. We found that participants had abstractPrinciples of Rigorwhich included: Precision, Systematic Thought Process, Depth of Understanding, and Challenge. They also had concreteObservations of Rigorthat included Time and Effort, Intrinsic Drive, Productive Failure, Struggle, Outcomes, and Gatekeeping. Participants also sharedConditions for Rigorwhich included Expectations, Standards, Community Support, and Resources. Implications:Our data supports prior work that educators are using different definitions of rigor. This implies that each educator holds different expectations for students, without necessarily communicating these expectations to their students. In the best case, this might confuse students; in the worst case, it reinforces hegemonic norms which can lead to gatekeeping which prevents students from fully participating in the CS field. Based on these insights, we argue that to commit to the idea of quality CS learning, the community must discard the use of this concept of rigor to justify student learning and re-imagine alternate benchmarks.
more »
« less
This content will become publicly available on December 2, 2026
Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigor
In AI research and practice, rigor remains largely understood in terms of methodological rigor — such as whether mathematical, statistical, or computational methods are correctly applied. We argue that this narrow conception of rigor has contributed to the concerns raised by the responsible AI community, including overblown claims about AI capabilities. Our position is that a broader conception of what rigorous AI research and practice should entail is needed. We believe such a conception — in addition to a more expansive understanding of (1) methodological rigor — should include aspects related to (2) what background knowledge informs what to work on (epistemic rigor); (3) how disciplinary, community, or personal norms, standards, or beliefs influence the work (normative rigor); (4) how clearly articulated the theoretical constructs under use are (conceptual rigor); (5) what is reported and how (reporting rigor); and (6) how well-supported the inferences from existing evidence are (interpretative rigor). In doing so, we also aim to provide useful language and a framework for much-needed dialogue about the AI community’s work by researchers, policymakers, journalists, and other stakeholders.
more »
« less
- PAR ID:
- 10654842
- Publisher / Repository:
- NeurIPS
- Date Published:
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The ubiquity of AI in society means the time is ripe to consider what educated 21st century digital citizens should know about this subject. In May 2018, the Association for the Advancement of Artificial Intelligence (AAAI) and the Computer Science Teachers Association (CSTA) formed a joint working group to develop national guidelines for teaching AI to K-12 students. Inspired by CSTA's national standards for K-12 computing education, the AI for K-12 guidelines will define what students in each grade band should know about artificial intelligence, machine learning, and robotics. The AI for K-12 working group is also creating an online resource directory where teachers can find AI- related videos, demos, software, and activity descriptions they can incorporate into their lesson plans. This blue sky talk invites the AI research community to reflect on the big ideas in AI that every K-12 student should know, and how we should communicate with the public about advances in AI and their future impact on society. It is a call to action for more AI researchers to become AI educators, creating resources that help teachers and students understand our work.more » « less
-
What should we do with emotion AI? Should we regulate, ban, promote, or re-imagine it? Emotion AI, a class of affective computing technologies used in personal and social computing, comprises emergent and controversial techniques aiming to classify human emotion and other affective phenomena. Industry, policy, and scientific actors debate potential benefits and harms, arguing for polarized futures ranging from panoptic expansion to complete bans. Emotion AI is proposed, deployed, and sometimes withdrawn in collaborative contexts such as education, hiring, healthcare, and service work. Proponents expound these technologies’ benefits for well-being and security, while critics decry privacy harms, civil liberties risks, bias, and shaky scientific foundations, and gaps between technologies’ capabilities and how they are marketed and legitimized. This panel brings diverse disciplinary perspectives into discussion about the history of emotions—as an example of ’intimate’ data—in computing, how emotion AI is legitimized, people’s experiences with and perceptions of emotion AI in social and collaborative settings, emotion AI’s development practices, and using design research to re-imagine emotion AI. These issues are relevant to the CSCW community in designing, evaluating, and regulating algorithmic sensing technologies including and beyond emotion-sensing.more » « less
-
The future of AI will be determined in part by how its developers are educated. Thus, how computer science (CS) education incorporates instruction in various aspects of AI will have a substantial impact on AI's evolution. Understanding how and what CS educators think about AI education is, therefore, an important piece of the landscape in anticipating -- and shaping -- the future of AI. However, little is known about how educators perceive the role of AI education in CS education, and there is no consensus yet regarding what AI topics should be taught to all students. This paper helps to fill that gap by presenting a qualitative analysis of data collected from high school CS instructors, higher education CS faculty, and those working in the tech industry as they reflected on their priorities for high school CS instruction and on anticipated changes in high school, college, and workplace CS. We conclude with recommendations for the CS education research community around AI in K-12, particularly at the high school level.more » « less
-
null (Ed.)Current attempts at methodological reform in sciences come in response to an overall lack of rigor in methodological and scientific practices in experimental sciences. However, most methodological reform attempts suffer from similar mistakes and over-generalizations to the ones they aim to address. We argue that this can be attributed in part to lack of formalism and first principles. Considering the costs of allowing false claims to become canonized, we argue for formal statistical rigor and scientific nuance in methodological reform. To attain this rigor and nuance, we propose a five-step formal approach for solving methodological problems. To illustrate the use and benefits of such formalism, we present a formal statistical analysis of three popular claims in the metascientific literature: (i) that reproducibility is the cornerstone of science; (ii) that data must not be used twice in any analysis; and (iii) that exploratory projects imply poor statistical practice. We show how our formal approach can inform and shape debates about such methodological claims.more » « less
