An official website of the United States government
There is widespread agreement about the need to assess the success of programs training scientists to communicate more effectively with non-professional audiences. However, there is little agreement about how that should be done. What do we mean when we talk about “effective communication”? What should we measure? How should we measure it? Evaluation of communication training programs often incorporates the views of students or trainers themselves, although this is widely understood to bias the assessment. We recently completed a 3-year experiment to use audiences of non-scientists to evaluate the effect of training on STEM (Science, Technology, Engineering and Math) graduate students’ communication ability. Overall, audiences rated STEM grad students’ communication performance no better after training than before, as we reported in Rubega et al. 2018. However, audience ratings do not reveal whether training changed specific trainee communication behaviors (e.g., jargon use, narrative techniques) even if too little to affect trainees’ overall success. Here we measure trainee communication behavior directly, using multiple textual analysis tools and analysis of trainees’ body language during videotaped talks. We found that student use of jargon declined after training but that use of narrative techniques did not increase. Flesch Reading Ease and Flesch-Kincaid Grade Level scores, used as indicators of complexity of sentences and word choice, were no different after instruction. Trainees’ movement of hands and hesitancy during talks was correlated negatively with audience ratings of credibility and clarity; smiling, on the other hand, was correlated with improvement in credibility, clarity and engagement scores given by audience members. We show that objective tools can be used to measure the success of communication training programs, that non-verbal cues are associated with audience judgments, and that an intensive communication course does change some, if not all, communication behaviors.
more »
« less
Assessment by Audiences Shows Little Effect of Science Communication Training
As the science community has recognized the vital role of communicating to the public, science communication training has proliferated. The development of rigorous, comparable approaches to assessment of training has not kept pace. We conducted a fully controlled experiment using a semester-long science communication course, and audience assessment of communicator performance. Evaluators scored the communication competence of trainees and their matched, untrained controls, before and after training. Bayesian analysis of the data showed very small gains in communication skills of trainees, and no difference from untrained controls. High variance in scores suggests little agreement on what constitutes “good” communication.
more »
« less
- Award ID(s):
- 2022036
- PAR ID:
- 10286231
- Date Published:
- Journal Name:
- Science Communication
- Volume:
- 43
- Issue:
- 2
- ISSN:
- 1075-5470
- Page Range / eLocation ID:
- 139 to 169
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
While over one-third of the U.S. economy and much of our national security infrastructure directly depends on precision timing, there has been to date no educational workforce development program in the US dedicated to training young talent in the timekeeping technologies that underpin our society. The Alabama Collaborative for Contemporary Education in Precision Timing (ACCEPT) Program is a new, 5-year National Research Traineeship program funded by the National Science Foundation, designed to train the next generation of graduate (MS and PhD) degree holders in a field of critical important to our nation. ACCEPT will provide a comprehensive training and educational opportunity for trainees from physics, mathematics, and engineering. Trainees will combine coursework across these three departments with professional development in critical areas identified by precision timing experts (teamwork, leadership, ethics, communication), and put their training into practice via research experiences with ACCEPT partners, student-led initiatives, and networking at conferences and workshops. In this paper, we present the current objectives, vision, and methodology of our new program, initial steps toward building a comprehensive training facility, and initial research and demonstration projects.more » « less
-
Despite being disproportionately impacted by health disparities, Black, Hispanic, Indigenous, and other underrepresented populations account for a significant minority of graduates in biomedical data science-related disciplines. Given their commitment to educating underrepresented students and trainees, minority serving institutions (MSIs) can play a significant role in enhancing diversity in the biomedical data science workforce. Little has been published about the reach, curricular breadth, and best practices for delivering these data science training programs. The purpose of this paper is to summarize six Research Centers in Minority Institutions (RCMIs) awarded funding from the National Institute of Minority Health Disparities (NIMHD) to develop new data science training programs. A cross-sectional survey was conducted to better understand the demographics of learners served, curricular topics covered, methods of instruction and assessment, challenges, and recommendations by program directors. Programs demonstrated overall success in reach and curricular diversity, serving a broad range of students and faculty, while also covering a broad range of topics. The main challenges highlighted were a lack of resources and infrastructure and teaching learners with varying levels of experience and knowledge. Further investments in MSIs are needed to sustain training efforts and develop pathways for diversifying the biomedical data science workforce.more » « less
-
To enable the sustainable use of their ocean resources, capacity for ocean science and observations is important for every coastal nation. In many developing areas of the world, capability for ocean science and observations is not yet adequate to meet management needs. International organizations have employed a variety of capacity development approaches to assist developing countries in building self-sustaining ocean science and observational communities. This article describes the lessons learned from visiting scientist programs conducted for more than a decade by the Partnership for Observation of the Global Ocean (POGO) and the Scientific Committee on Oceanic Research (SCOR) that dispatched ocean scientists to developing countries to train hundreds of individuals in a variety of ocean science and observation topics and techniques. From these programs, SCOR and POGO have learned that training in-country has multiple benefits to trainees, host institutions, and trainers, benefits that are not achievable when students leave their countries. These benefits include more cost-effective training on issues relevant to the host institutions using locally available technology, as well as the ability to reach a large number of trainees. Lessons learned from the POGO and SCOR programs can be used to inform the future capacity-development activities of POGO and SCOR, as well as other organizations, to improve, enhance, and expand the use of in-country training and mentoring. Such approaches could contribute to the capacity development efforts of the UN Decade of Ocean Science for Sustainable Development.more » « less
-
Wolfson, Adele (Ed.)Diversity-focused committees continue to play essential roles in the efforts of professional scientific societies to foster inclusion and facilitate the professional development of underrepresented minority (URM) young scientists in their respective scientific disciplines. Until recently, the efforts of these committees have remained independent and disconnected from one another. Funding from the National Science Foundation has allowed several of these committees to come together and form the Alliance to Catalyze Change for Equity in STEM Success, herein referred to as ACCESS. The overall goal of this meta-organization is to create a community in which diversity-focused committees can interact, synergize, share their collective experiences, and have a unified voice on behalf of URM trainees in science, technology, engineering, and mathematics disciplines. In this Essay, we compare and contrast the broad approaches that scientific societies in ACCESS use to implement and assess their travel award programs for URM trainees. We also report a set of recommendations, including both short- and long-term outcomes assessment in populations of interest and specialized programmatic activities coupled to travel award programs.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1177/1075547020971639
