skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


  • NSF Public Access
  • Search Results
  • Persuasion with Precision: Using Natural Language Processing to Improve Instrument Fidelity for Risk Communication Experimental Treatments
Title: Persuasion with Precision: Using Natural Language Processing to Improve Instrument Fidelity for Risk Communication Experimental Treatments
Instrument fidelity in message testing research hinges upon how precisely messages operationalize treatment conditions. However, numerous message testing studies have unmitigated threats to validity and reliability because no established procedures exist to guide construction of message treatments. Their construction typically occurs in a black box, resulting in suspect inferential conclusions about treatment effects. Because a mixed methods approach is needed to enhance instrument fidelity in message testing research, this article contributes to the field of mixed methods research by presenting an integrated multistage procedure for constructing precise message treatments using an exploratory sequential mixed methods design. This work harnesses the power of integration through crossover analysis to improve instrument fidelity in message testing research through the use of natural language processing (NLP).  more » « less
Award ID(s):
1757351
PAR ID:
10413749
Author(s) / Creator(s):
; ; ; ; ; ; ; ;
Date Published:
Journal Name:
Journal of Mixed Methods Research
ISSN:
1558-6898
Page Range / eLocation ID:
155868982210969
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; (, Journal of the Royal Statistical Society Series B: Statistical Methodology)
    Abstract The ability to predict individualized treatment effects (ITEs) based on a given patient's profile is essential for personalized medicine. We propose a hypothesis testing approach to choosing between two potential treatments for a given individual in the framework of high-dimensional linear models. The methodological novelty lies in the construction of a debiased estimator of the ITE and establishment of its asymptotic normality uniformly for an arbitrary future high-dimensional observation, while the existing methods can only handle certain specific forms of observations. We introduce a testing procedure with the type I error controlled and establish its asymptotic power. The proposed method can be extended to making inference for general linear contrasts, including both the average treatment effect and outcome prediction. We introduce the optimality framework for hypothesis testing from both the minimaxity and adaptivity perspectives and establish the optimality of the proposed procedure. An extension to high-dimensional approximate linear models is also considered. The finite sample performance of the procedure is demonstrated in simulation studies and further illustrated through an analysis of electronic health records data from patients with rheumatoid arthritis. 
    more » « less
  2. ; ; ; ; (, PLoS ONE)
    Sun, Daner (Ed.)
    The use of online instruction for undergraduate STEM courses is growing rapidly. While researchers and practitioners have access to validated instruments for studying the practice of teaching in face-to-face classrooms, analogous tools do not yet exist for online instruction. These tools are needed for quality design and control purposes. To meet this need, this project developed an observational protocol that can be used to collect non-evaluative data for the description, study, and improvement of online, undergraduate STEM courses. The development of this instrument used a sequential exploratory mixed methods approach to the research, design, pilot-testing, refinement and implementation of the protocol. Pairs of researchers tested the final version of this instrument, observing completed online undergraduate STEM courses. Across 2,394 pairs of observations, the observers recorded the same indication (yes or no to the presence of some course element) 1,853 times for an agreement rate of 77.4%, falling above the 75% threshold for an acceptable level of agreement. There was a wide range in the inter-rater reliability rates among items and further revisions were made to the instrument. This foundational work-in-progress instrument should be further developed and used by practitioners who are interested in learning about and reflecting on their online teaching practice. 
    more » « less
  3. ; ; (, Stat)
    Individuals may respond to treatments with significant heterogeneity. To optimize the treatment effect, it is necessary to recommend treatments based on individual characteristics. Existing methods in the literature for learning individualized treatment regimes are usually designed for randomized studies with binary treatments. In this study, we propose an algorithm to extend random forest of interaction trees (Su et al., 2009) to accommodate multiple treatments. By integrating the generalized propensity score into the interaction tree growing process, the proposed method can handle both randomized and observational study data with multiple treatments. The performance of the proposed method, relative to existing approaches in the literature, is evaluated through simulation studies. The proposed method is applied to an assessment of multiple voluntary educational programmes at a large public university. 
    more » « less
  4. ; ; ; ; ; (, Journal of Technology and Science Education)
    An authentic, interdisciplinary, research and problem-based integrated science, technology, engineering, and mathematics (STEM) project may be ideal for encouraging scientific inquiry and developing teamwork among undergraduate students, but it also presents challenges. The authors describe how two interdisciplinary teams (n=6) of undergraduate college students built integrated STEM projects in a research based internship setting, and then collaboratively brought the project to fruition to include designing lessons and activities shared with K-12 students in a classroom setting. Each three person undergraduate team consisted of two STEM majors and one Education major. The Education majors are a special focus for this study. Interviews, field observations, and lesson plan artifacts collected from the undergraduate college students were analyzed according to authenticity factors, the authentic scientific inquiry instrument, and an integrated STEM instrument. The authors highlight areas of strength and weakness for both teams and explore how preservice teachers contributed to integrated STEM products and lessons. Teacher educators might apply recommendations for teacher preparation and professional development when facilitating authentic scientific inquiry and integrated STEM topics with both STEM and non-STEM educators. Undergraduate college students were challenged to fully integrate the STEM disciplines, transitions between them, and the spaces between them where multiple disciplines existed. By describing the challenges of integrating the spaces between STEM, the authors offer a description of the undergraduate college students’ experiences in an effort to expand the common message beyond a flat approach of try this activity because it works, to a more robust message of try this type of engagement and purposefully organize for maximum results. 
    more » « less
  5. ; ; ; ; ; ; (, ASME 2020 Dynamic Systems and Control Conference)
    null (Ed.)
    This paper presents a systematic design of high-fidelity tele-operated scaled vehicles to be used as a research and development platform for intelligent transportation technologies. Compared to computer simulation and full-scale physical tests, the use of high-fidelity scaled setups provides advantages on testing time and financial effectiveness. The physical design of the vehicles features a 1:14 scale with realistic appearance licensed by car manufacturers. Customized steering system and propulsion control provide high-fidelity maneuver characteristics. Remote control is deployed using a target-host structure over WiFi and can provide seamless switching between human driving and autonomous/assisted driving on the host side. Several possible solutions for real-time panoramic vision feedback are explored, with a tri-camera design based on parallel acquisition interfaces adopted. An adaptive color compression technique is developed to shorten the video streaming latency. A customized miniature LIDAR system is introduced to provide an ultra-small package for on-board installation. As a solution balancing between test fidelity and costs, the proposed scaled vehicles are especially suitable for validation tests during the early stage of research and development. With a long-term goal of developing a multi-vehicle traffic network test platform, ongoing and future work on the construction of scaled buildings and road systems is also discussed. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email