Title: Interpretation and Use Statements for Instruments in Mathematics Education
This Research Commentary addresses the need for an instrument abstract—termed an Interpretation and Use Statement (IUS)—to be included when mathematics educators present instruments for use by others in journal articles and other communication venues (e.g., websites and administration manuals). We begin with presenting the need for IUSs, including the importance of a focus on interpretation and use. We then propose a set of elements—identified by a group of mathematics education researchers, instrument developers, and psychometricians—to be included in the IUS. We describe the development process, the recommended elements for inclusion, and two example IUSs. Last, we present why IUSs have the potential to benefit end users and the field of mathematics education. more »« less
Bostic, J.; Krupa, E.; Folger, T.; Bentley, B.; Stokes, D.
(, Psychology of Mathematics Education North America)
A. Lischka, E. Dyer
(Ed.)
Validity and validation is central to conducting high quality quantitative mathematics education scholarship. This presentation aims to support scholars engaged in quantitative research by providing information about the degrees to which validity evidence related to their instrument use or interpretation, were found in mathematics education scholarship. Findings have potential to steer future quantitatively focused scholarship and support equity aims.
Carney, M. B.; Champion, J.; Crawford, A.
(, The 43rd Meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education, Virtual/Philadelphia, USA.)
Olanoff, D.; Johnson, K.; Spitzer, S. M.
(Ed.)
Mathematics education needs measures that can be used to research and/or evaluate the impact of professional development for constructs that are broadly relevant to the field. To address this need we developed the Priorities for Mathematics Instruction (PMI) survey consisting of two scales focused on the constructs of Explicit Attention to Concepts (EAC) and Student Opportunities to Struggle (SOS) – which have been linked to increased student understanding and achievement. We identified the most critical assumptions that underlie the proposed interpretation and use of the scale scores and then examined the related validity evidence. We found the evidence for each assumption supports the proposed interpretation and use of the scale scores.
Integrated approaches to teaching science, technology, engineering, and mathematics (commonly referred to as STEM education) in K-12 classrooms have resulted in a growing number of teachers incorporating engineering in their science classrooms. Such changes are a result of shifts in science standards to include engineering as evidenced by the Next Generation Science Standards. To date, 20 states and the District of Columbia have adopted the NGSS and another 24 have adopted standards based on the Framework for K-12 Science Education. Despite the increased presence of engineering and integrated STEM education in K-12 education, there are several concerns to consider. One concern is the limited availability of observation instruments appropriate for instruction where multiple STEM disciplines are present and integrated with one another. Addressing this concern requires the development of a new observation instrument, designed with integrated STEM instruction in mind. An instrument such as this has implications for both research and practice. For example, research using this instrument could help educators compare integrated STEM instruction across grade bands. Additionally, this tool could be useful in the preparation of pre-service teachers and professional development of in-service teachers new to integrated STEM education and formative learning through professional learning communities or classroom coaching. The work presented here describes in detail the development of an integrated STEM observation instrument that can be used for both research and practice. Over a period of approximately 18-months, a team of STEM educators and educational researchers developed a 10-item integrated STEM observation instrument for use in K-12 science and engineering classrooms. The process of developing the instrument began with establishing a conceptual framework, drawing on the integrated STEM research literature, national standards documents, and frameworks for both K-12 engineering education and integrated STEM education. As part of the instrument development process, the project team had access to over 2000 classroom videos where integrated STEM education took place. Initial analysis of a selection of these videos helped the project team write a preliminary draft instrument consisting of 52 items. Through several rounds of revisions, including the construction of detailed scoring levels of the items and collapsing of items that significantly overlapped, and piloting of the instrument for usability, items were added, edited, and/or removed for various reasons. These reasons included issues concerning the intricacy of the observed phenomenon or the item not being specific to integrated STEM education (e.g., questioning). In its final form, the instrument consists of 10 items, each comprising four descriptive levels. Each item is also accompanied by a set of user guidelines, which have been refined by the project team as a result of piloting the instrument and reviewed by external experts in the field. The instrument has shown to be reliable with the project team and further validation is underway. This instrument will be of use to a wide variety of educators and educational researchers looking to understand the implementation of integrated STEM education in K-12 science and engineering classrooms.
Dare, E. A.; Hiwatig, B.; Keratithamkul, K.; Ellis, J. A.; Roehrig, G. H.; Ring-Whalen, E. A.; Rouleau, M. D.; Faruqi, F.; Rice, C.; Titu, P.; et al
(, ASEE Annual Conference proceedings)
null
(Ed.)
Integrated approaches to teaching science, technology, engineering, and mathematics (commonly referred to as STEM education) in K-12 classrooms have resulted in a growing number of teachers incorporating engineering in their science classrooms. Such changes are a result of shifts in science standards to include engineering as evidenced by the Next Generation Science Standards. To date, 20 states and the District of Columbia have adopted the NGSS and another 24 have adopted standards based on the Framework for K-12 Science Education. Despite the increased presence of engineering and integrated STEM education in K-12 education, there are several concerns to consider. One concern is the limited availability of observation instruments appropriate for instruction where multiple STEM disciplines are present and integrated with one another. Addressing this concern requires the development of a new observation instrument, designed with integrated STEM instruction in mind. An instrument such as this has implications for both research and practice. For example, research using this instrument could help educators compare integrated STEM instruction across grade bands. Additionally, this tool could be useful in the preparation of pre-service teachers and professional development of in-service teachers new to integrated STEM education and formative learning through professional learning communities or classroom coaching. The work presented here describes in detail the development of an integrated STEM observation instrument - the STEM Observation Protocol (STEM-OP) - that can be used for both research and practice. Over a period of approximately 18-months, a team of STEM educators and educational researchers developed a 10-item integrated STEM observation instrument for use in K-12 science and engineering classrooms. The process of developing the STEM-OP began with establishing a conceptual framework, drawing on the integrated STEM research literature, national standards documents, and frameworks for both K-12 engineering education and integrated STEM education. As part of the instrument development process, the project team had access to over 2000 classroom videos where integrated STEM education took place. Initial analysis of a selection of these videos helped the project team write a preliminary draft instrument consisting of 79 items. Through several rounds of revisions, including the construction of detailed scoring levels of the items and collapsing of items that significantly overlapped, and piloting of the instrument for usability, items were added, edited, and/or removed for various reasons. These reasons included issues concerning the intricacy of the observed phenomenon or the item not being specific to integrated STEM education (e.g., questioning). In its final form, the STEM-OP consists of 10 items, each comprising four descriptive levels. Each item is also accompanied by a set of user guidelines, which have been refined by the project team as a result of piloting the instrument and reviewed by external experts in the field. The instrument has shown to be reliable with the project team and further validation is underway. The STEM-OP will be of use to a wide variety of educators and educational researchers looking to understand the implementation of integrated STEM education in K-12 science and engineering classrooms.
Forde, Elizabeth N.; Robinson, Latanya; Ellis, Joshua A.; Dare, Emily A.
(, Disciplinary and Interdisciplinary Science Education Research)
Effective K-12 integrated STEM education should reflect an intentional effort to adequately represent and facilitate each of its component disciplines in a meaningful way. However, most research in this space has been conducted within the context of science classrooms, ignoring mathematics. Also missing from the literature is research that examines the level of cognitive demand required from mathematical tasks present within integrated STEM lessons. In order to seek insight pertaining to this gap in the literature, we sought to better understand how science teachers use mathematics within K-12 integrated STEM instruction. We used an explanatory sequential mixed methods research design to explore the enactment of mathematics in integrated STEM lessons that focus on physical, earth, and life science content. We first examined 2030 sets of video-recorded classroom observation scores generated from the 10-item STEM Observation Protocol (STEM-OP) designed for observing integrated STEM education in K-12 classrooms. We compared the STEM-OP scores of classroom observations that included mathematics with those that did not. This quantitative analysis was followed by a closer, more in-depth qualitative examination of how mathematics was employed, focusing on the degree of cognitive demand. To do this, we coded and analyzed transcripts from video-recorded classroom observations in which mathematical content was present. Our study yielded two main findings about mathematics in integrated STEM lessons: (1) the presence of mathematical content resulted in higher STEM-OP scores on nearly all items, and (2) mathematical tasks within these lessons were categorized as requiring mainly low levels of cognitive demand from students. This study highlights the need for the increased inclusion of mathematical tasks in integrated STEM teaching. Implications for including higher-order mathematical thinking within integrated STEM teaching are discussed.
Carney, Michele B., Bostic, Jonathan, Krupa, Erin, and Shih, Jeff. Interpretation and Use Statements for Instruments in Mathematics Education. Retrieved from https://par.nsf.gov/biblio/10420404. Journal for Research in Mathematics Education 53.4 Web. doi:10.5951/jresematheduc-2020-0087.
Carney, Michele B., Bostic, Jonathan, Krupa, Erin, & Shih, Jeff. Interpretation and Use Statements for Instruments in Mathematics Education. Journal for Research in Mathematics Education, 53 (4). Retrieved from https://par.nsf.gov/biblio/10420404. https://doi.org/10.5951/jresematheduc-2020-0087
@article{osti_10420404,
place = {Country unknown/Code not available},
title = {Interpretation and Use Statements for Instruments in Mathematics Education},
url = {https://par.nsf.gov/biblio/10420404},
DOI = {10.5951/jresematheduc-2020-0087},
abstractNote = {This Research Commentary addresses the need for an instrument abstract—termed an Interpretation and Use Statement (IUS)—to be included when mathematics educators present instruments for use by others in journal articles and other communication venues (e.g., websites and administration manuals). We begin with presenting the need for IUSs, including the importance of a focus on interpretation and use. We then propose a set of elements—identified by a group of mathematics education researchers, instrument developers, and psychometricians—to be included in the IUS. We describe the development process, the recommended elements for inclusion, and two example IUSs. Last, we present why IUSs have the potential to benefit end users and the field of mathematics education.},
journal = {Journal for Research in Mathematics Education},
volume = {53},
number = {4},
author = {Carney, Michele B. and Bostic, Jonathan and Krupa, Erin and Shih, Jeff},
}
Warning: Leaving National Science Foundation Website
You are now leaving the National Science Foundation website to go to a non-government website.
Website:
NSF takes no responsibility for and exercises no control over the views expressed or the accuracy of
the information contained on this site. Also be aware that NSF's privacy policy does not apply to this site.
An official website of the United States government
