More Like this

1. Reproducibility in Scientific Computing

   https://doi.org/10.1145/3186266  

   Ivie, Peter; Thain, Douglas (July 2018, ACM Computing Surveys)
2. Containers for computational reproducibility

   https://doi.org/10.1038/s43586-023-00236-9  

   Moreau, David; Wiebels, Kristina; Boettiger, Carl (December 2023, Nature Reviews Methods Primers)
3. Reproducibility in the Classroom

   https://doi.org/10.1146/annurev-statistics-112723-034436  

   Dogucu, Mine (March 2025, Annual Review of Statistics and Its Application)

   Difficulties in reproducing results from scientific studies have lately been referred to as a reproducibility crisis. Scientific practice depends heavily on scientific training. What gets taught in the classroom is often practiced in labs, fieldwork, and data analysis. The importance of reproducibility in the classroom has gained momentum in statistics education in recent years. In this article, we review the existing literature on reproducibility education. We delve into the relationship between computing tools and reproducibility through visiting historical developments in this area. We share examples for teaching reproducibility and reproducible teaching while discussing the pedagogical opportunities created by these examples as well as challenges that the instructors should be aware of. We detail the use of teaching reproducibility and reproducible teaching practices in an introductory data science course. Lastly, we provide recommendations on reproducibility education for instructors, administrators, and other members of the scientific community. 

   more » « less
4. Summer of Reproducibility: Building Global Capacity for Practical Reproducibility through Hands-On Mentorship

   Lieggi, Stephanie; Fund, Fraida; Keahey, Kate; Richardson, Marc (July 2025, ACM)

   Thanks to increasing awareness of the importance of reproducibility in computer science research, initiatives such as artifact review and badging have been introduced to encourage reproducible research in this field. However, making "practical reproducibility" truly widespread requires more than just incentives. It demands an increase in capacity for reproducible research among computer scientists - more tools, workflows, and exemplar artifacts, and more human resources trained in best practices for reproducibility. In this paper, we describe our experiences in the first two years of the Summer of Reproducibility (SoR), a mentoring program that seeks to build global capacity by enabling students around the world to work with expert mentors while producing reproducibility artifacts, tools, and education materials. We give an overview of the program, report preliminary outcomes, and discuss plans to evolve this program. 

   more » « less
5. Reproducibility metrics for CRISPR screens

   https://doi.org/10.1101/2022.02.19.480892  

   Billmann, Maximilian; Ward, Henry N.; Aregger, Michael; Costanzo, Michael; Andrews, Brenda J.; Boone, C; Moffat, J; Myers, Chad L. (February 2022, bioRxiv)

   CRISPR screens are used extensively to systematically interrogate the phenotype-to-genotype problem. In contrast to early CRISPR screens, which defined core cell fitness genes, most current efforts now aim to identify context-specific phenotypes that differentiate a cell line, genetic background or condition of interest, such as a drug treatment. While CRISPR-related technologies have shown great promise and a fast pace of innovation, a better understanding of standards and methods for quality assessment of CRISPR screen results is crucial to guide technology development and application. Specifically, many commonly used metrics for quantifying screen quality do not accurately measure the reproducibility of context-specific hits. We highlight the importance of reporting reproducibility statistics that directly relate to the purpose of the screen and suggest the use of metrics that are sensitive to context-specific signal. 

   more » « less