skip to main content

Attention:

The NSF Public Access Repository (PAR) system and access will be unavailable from 8:00 PM ET on Friday, March 21 until 8:00 AM ET on Saturday, March 22 due to maintenance. We apologize for the inconvenience.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


This content will become publicly available on December 5, 2025

Title: Validation of an Instrument to Measure Self-Efficacy in Information Security
Problem. Extant measures of students’ cybersecurity self-efficacy lack sufficient evidence of validity based on internal structure. Such evidence of validity is needed to enhance confidence in conclusions drawn from use of self-efficacy measures in the cybersecurity domain. Research Question. To address this identified problem, we sought to answer our research question: What is the underlying factor structure of a new self-efficacy for Information Security measure? Method. We leveraged exploratory factor analysis (EFA) to deter- mine the number of factors underlying a new measure of student self-efficacy to conduct information security. This measure was created to align with the five elements of the information security section of the K-12 Cybersecurity Education framework. Participants were 190 undergraduate students recruited from computer science courses across the U.S. Findings. Results from the EFA indicated that a four-factor solution best fit the data while maximizing interpretability of the factors. The internal reliability of the measure was quite strong (𝛼 = .99). Implications. The psychometric quality of this measure was demonstrated, and thus evidence of validity based on internal structure has been established. Future work will conduct a confirmatory factor analysis (CFA) and assess measurement invariance across sub- groups of interest (e.g., over- vs. under-represented race/ethnicity groups, gender).  more » « less
Award ID(s):
2028426
PAR ID:
10560734
Author(s) / Creator(s):
;
Publisher / Repository:
ACM
Date Published:
ISBN:
9798400705984
Page Range / eLocation ID:
214 to 220
Format(s):
Medium: X
Location:
Virtual Event NC USA
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ( , 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference)
    null (Ed.)
    The purpose of this study is to re-examine the validity evidence of the engineering design self-efficacy (EDSE) scale scores by Carberry et al. (2010) within the context of secondary education. Self-efficacy refers to individuals’ belief in their capabilities to perform a domain-specific task. In engineering education, significant efforts have been made to understand the role of self-efficacy for students considering its positive impact on student outcomes such as performance and persistence. These studies have investigated and developed measures for different domains of engineering self-efficacy (e.g., general academic, domain-general, and task-specific self-efficacy). The EDSE scale is a frequently cited measure that examines task-specific self-efficacy within the domain of engineering design. The original scale contains nine items that are intended to represent the engineering design process. Initial score validity evidence was collected using a sample consisting of 202 respondents with varying degrees of engineering experience including undergraduate/graduate students and faculty members. This scale has been primarily used by researchers and practitioners with engineering undergraduate students to assess changes in their engineering design self-efficacy as a result of active learning interventions, such as project-based learning. Our work has begun to experiment using the scale in a secondary education context in conjunction with an increased introduction to engineering in K-12 education. Yet, there still is a need to examine score validity and reliability of this scale in non-undergraduate populations such as secondary school student populations. This study fills this important gap by testing construct validity of the original nine items of the EDSE scale, supporting proper use of the scale for researchers and practitioners. This study was conducted as part of a larger, e4usa project investigating the development and implementation of a yearlong project-based engineering design course for secondary school students. Evidence of construct validity and reliability was collected using a multi-step process. First, a survey that includes the EDSE scale was administered to the project participating students at nine associated secondary schools across the US at the beginning of Spring 2020. Analysis of collected data is in progress and includes Exploratory Factor Analysis (EFA) on the 137 responses. The evidence of score reliability will be obtained by computing the internal consistency of each resulting factor. The resulting factor structure and items will be analyzed by comparing it with the original EDSE scale. The full paper will provide details about the psychometric evaluation of the EDSE scale. The findings from this paper will provide insights on the future usage of the EDSE scale in the context of secondary engineering education. 
    more » « less
  2. ; ( , Journal of Psychoeducational Assessment)

    Since its development in 2006, the Longitudinal Assessment of Engineering Self-Efficacy (LAESE) V3.0 instrument with six constructs indicated by 31 items has been a popular tool used in engineering education research in the United States. However, there has been lack of validity and reliability evidence in the literature beyond its initial development, with an indication of multicollinearity between its two engineering self-efficacy constructs. This study aimed to rescale the LAESE V3.0 through factor analyses after a modification of items, providing construct validity evidence for the revised instrument. With data from 997 engineering students at three institutions, exploratory and confirmatory factor analyses resulted in the Revised LAESE V3.0, consisting of 16 items loading across four factors in a good model fit range: Engineering Self-Efficacy, Engineering Career Expectations, Sense of Belonging, and Coping Self-Efficacy. The nonlinear SEM (structural equation modeling) reliability coefficients for individual factors ranged from .76 to .84, with the overall Omega for the ordinal data of .92, demonstrating acceptable internal consistency reliability.

     
    more » « less
  3. ( , ASEE Annual Conference & Exposition)
    Measures of subject-related role identities in physics and math have been developed from research on the underlying constructs of identity in science education. The items for these measures capture three constructs of identity: students’ interest in the subject, students’ feeling of recognition by others, and students’ beliefs about their performance/competence in the subject area. In prior studies with late secondary and early post-secondary students, participants did not distinguish between performance beliefs (e.g., believing that they can do well in a particular subject) and competence beliefs (e.g., believing that they can understand a particular subject); therefore, performance/competence beliefs are measured as a single construct. These validated measures have been successful in predicting STEM career choices including physics, math, and engineering. Based on these measures of identity, literature on engineering identity, and my prior work on understanding engineering choice and belongingness through students’ science and math identities at the transition from high school to college, I developed a set of new engineering identity measures that capture and overall identification as an engineer, future engineering career identification, and students’ engineering-related interest, recognition, and performance/competence beliefs. I conducted a pilot survey of 371 first-year engineering students at three institutions within the U.S. during the spring semester of 2015. An exploratory factor analysis (EFA) was performed to examine the underlying structure of the piloted questions about students’ engineering identity. The measures loaded on three separate constructs that were consistent with the hypothesized constructs of interest, performance/competence and recognition. The developed items were used in a subsequent study deployed in the fall semester of 2015 that measured more than 2500 first-year engineering students’ attitudes and beliefs at four institutions within the U.S. The data on engineering identity measures from this second survey were analyzed using confirmatory factor analysis (CFA). The results indicated that the developed measures do extract a significant portion of the average variance in the latent constructs and the internal consistency of the measures (Cronbach’s α) falls within the acceptable and better range. The development of these items provides ways for engineering education researchers to more deeply explore the underlying self-beliefs in students’ engineering identity formation through quantitative measures with strong evidence for validity. 
    more » « less
  4. ; ; ; ; ( , Proceedings for the American Society for Engineering Education Annual Conference, June 25-28. Columbus, OH.)
    When survey time is limited, education researchers may be faced with the choice of using an extremely brief measure of innovativeness or using no measure at all. To meet the need for a very brief measure, a 5-item innovation self-efficacy (ISE.5) scale was developed using the 19- item Dyer et al. Innovative Behavior Scale (IBS) as a starting point, adapted for undergraduate engineering students, and then condensed using confirmatory factor analysis. The ISE.5 measures innovation self-efficacy as a unitary construct drawn from Dyer et al.’s five innovative behavior components (Questioning, Observing, Experimenting, Networking Ideas and Associational Thinking) and has good internal and external validity as well as good test-retest reliability. The ISE.5 (as a measure of innovation self-efficacy) is shown to be an important mediator between innovation interests and a desire to pursue innovative work as a career postgraduation. This mediator relationship is consistent among important sub-populations of engineering students such as females, underrepresented minorities and first generation college students. While not a substitute for a full multi-factor innovation assessment tool, the ISE.5 can serve as an important indicator of innovation self-efficacy among an undergraduate engineering student population. 
    more » « less
  5. ( , Proceedings of the 43rd Meeting of the International Group for the Psychology of Mathematics Education)
    Problem solving is central to mathematics learning (NCTM, 2014). Assessments are needed that appropriately measure students’ problem-solving performance. More importantly, assessments must be grounded in robust validity evidence that justifies their interpretations and outcomes (AERA et al., 2014). Thus, measures that are grounded in validity evidence are warranted for use by practitioners and scholars. The purpose of this presentation is to convey validity evidence for a new measure titled Problem-Solving Measure for grade four (PSM4). The research question is: What validity evidence supports PSM4 administration? The PSM4 is one assessment within the previously published PSM series designed for elementary and middle grades students. Problems are grounded in Schoenfeld’s (2011) framework and rely upon Verschaffel et al. (1999) perspective that word problems be open, complex, and realistic. The mathematics in the problems is tied to USA grade-level content and practice standards (CCSSI, 2010). 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email