More Like this

1. Toward Scientific Evidence Standards in Empirical Computer Science

   Timothy Kluthe, Brett A. (March 2023, Dagstuhl reports)

   Many scientific fields of study use formally established evidence standards during the peer review and evaluation process, such as Consolidated Standards of Reporting Trials (CONSORT) in medical research, the What Works Clearinghouse (WWC) used in education in the United States, or the APA Journal Article Reporting Standards (JARS) in psychology. The basis for these standards is community agreement on what to report in empirical studies. Such standards achieve two key goals. First, they make it easier to compare studies, facilitating replications, through transparent reporting and sharing of data, which can provide confidence that multiple research teams can obtain the same results. Second, they establish community agreement on how to report on and evaluate studies using different methodologies. The discipline of computer science does not have formalized evidence standards, even for major conferences or journals. This Dagstuhl Seminar has three primary objectives: 1. To establish a process for creating or adopting an existing evidence standard for empirical research in computer science. 2. To build a community of scholars that can discuss what a general standard should include. 3. To kickstart the discussion with scholars from software engineering, human-computer interac- tion, and computer science education. In order to better discuss and understand the implications of such standards across several empirical subfields of computer science and to facilitate adoption, we brought together participants from a range of backgrounds; including academia and industry, software engineering, computer- human interaction and computer science education, as well as representatives from several prominent journals. 

   more » « less
2. Standardizing data reporting in the research community to enhance the utility of open data for SARS-CoV-2 wastewater surveillance

   https://doi.org/10.1039/D1EW00235J  

   McClary-Gutierrez, Jill S.; Aanderud, Zachary T.; Al-faliti, Mitham; Duvallet, Claire; Gonzalez, Raul; Guzman, Joe; Holm, Rochelle H.; Jahne, Michael A.; Kantor, Rose S.; Katsivelis, Panagis; et al (July 2021, Environmental Science: Water Research & Technology)

   null (Ed.)

   SARS-CoV-2 RNA detection in wastewater is being rapidly developed and adopted as a public health monitoring tool worldwide. With wastewater surveillance programs being implemented across many different scales and by many different stakeholders, it is critical that data collected and shared are accompanied by an appropriate minimal amount of meta-information to enable meaningful interpretation and use of this new information source and intercomparison across datasets. While some databases are being developed for specific surveillance programs locally, regionally, nationally, and internationally, common globally-adopted data standards have not yet been established within the research community. Establishing such standards will require national and international consensus on what meta-information should accompany SARS-CoV-2 wastewater measurements. To establish a recommendation on minimum information to accompany reporting of SARS-CoV-2 occurrence in wastewater for the research community, the United States National Science Foundation (NSF) Research Coordination Network on Wastewater Surveillance for SARS-CoV-2 hosted a workshop in February 2021 with participants from academia, government agencies, private companies, wastewater utilities, public health laboratories, and research institutes. This report presents the primary two outcomes of the workshop: (i) a recommendation on the set of minimum meta-information that is needed to confidently interpret wastewater SARS-CoV-2 data, and (ii) insights from workshop discussions on how to improve standardization of data reporting. 

   more » « less
3. Foundations of plasma standards

   https://doi.org/10.1088/1361-6595/acb810  

   Alves, Luís L.; Becker, Markus M.; van Dijk, Jan; Gans, Timo; Go, David B.; Stapelmann, Katharina; Tennyson, Jonathan; Turner, Miles M.; Kushner, Mark J. (February 2023, Plasma Sources Science and Technology)

   Abstract The field of low-temperature plasmas (LTPs) excels by virtue of its broad intellectual diversity, interdisciplinarity and range of applications. This great diversity also challenges researchers in communicating the outcomes of their investigations, as common practices and expectations for reporting vary widely in the many disciplines that either fall under the LTP umbrella or interact closely with LTP topics. These challenges encompass comparing measurements made in different laboratories, exchanging and sharing computer models, enabling reproducibility in experiments and computations using traceable and transparent methods and data, establishing metrics for reliability, and in translating fundamental findings to practice. In this paper, we address these challenges from the perspective of LTP standards for measurements, diagnostics, computations, reporting and plasma sources. This discussion on standards, or recommended best practices, and in some cases suggestions for standards or best practices, has the goal of improving communication, reproducibility and transparency within the LTP field and fields allied with LTPs. This discussion also acknowledges that standards and best practices, either recommended or at some point enforced, are ultimately a matter of judgment. These standards and recommended practices should not limit innovation nor prevent research breakthroughs from having real-time impact. Ultimately, the goal of our research community is to advance the entire LTP field and the many applications it touches through a shared set of expectations. 

   more » « less
4. Verifiable Obstacle Detection

   https://doi.org/10.1109/ISSRE55969.2022.00017  

   Bansal, Ayoosh; Kim, Hunmin; Yu, Simon; Li, Bo; Hovakimyan, Naira; Caccamo, Marco; Sha, Lui (October 2022, Verifiable Obstacle Detection)

   Perception of obstacles remains a critical safety concern for autonomous vehicles. Real-world collisions have shown that the autonomy faults leading to fatal collisions originate from obstacle existence detection. Open source autonomous driving implementations show a perception pipeline with complex interdependent Deep Neural Networks. These networks are not fully verifiable, making them unsuitable for safety-critical tasks. In this work, we present a safety verification of an existing LiDAR based classical obstacle detection algorithm. We establish strict bounds on the capabilities of this obstacle detection algorithm. Given safety standards, such bounds allow for determining LiDAR sensor properties that would reliably satisfy the standards. Such analysis has as yet been unattainable for neural network based perception systems. We provide a rigorous analysis of the obstacle detection s 

   more » « less
5. A community-developed extension to Darwin Core for reporting the chronometric age of specimens

   https://doi.org/10.1371/journal.pone.0261044  

   Brenskelle, Laura; Wieczorek, John; Davis, Edward; Wallis, Neill J.; Emery, Kitty; LeFebvre, Michelle J.; Guralnick, Rob (September 2022, PLOS ONE)

   Haldorai, Anandakumar (Ed.)

   Darwin Core, the data standard used for sharing modern biodiversity and paleodiversity occurrence records, has previously lacked proper mechanisms for reporting what is known about the estimated age range of specimens from deep time. This has led to data providers putting these data in fields where they cannot easily be found by users, which impedes the reuse and improvement of these data by other researchers. Here we describe the development of the Chronometric Age Extension to Darwin Core, a ratified, community-developed extension that enables the reporting of ages of specimens from deeper time and the evidence supporting these estimates. The extension standardizes reporting about the methods or assays used to determine an age and other critical information like uncertainty. It gives data providers flexibility about the level of detail reported, focusing on the minimum information needed for reuse while still allowing for significant detail if providers have it. Providing a standardized format for reporting these data will make them easier to find and search and enable researchers to pinpoint specimens of interest for data improvement or accumulate more data for broad temporal studies. The Chronometric Age Extension was also the first community-managed vocabulary to undergo the new Biodiversity Informatics Standards (TDWG) review and ratification process, thus providing a blueprint for future Darwin Core extension development. 

   more » « less