skip to main content


Title: A Roadmap to Robust Science for High-throughput Applications: The Developers’ Perspective
Scientists using the high-throughput computing (HTC) paradigm for scientific discovery rely on complex software systems and heterogeneous architectures that must deliver robust science (i.e., ensuring performance scalability in space and time; trust in technology, people, and infrastructures; and reproducible or confirmable research). Developers must overcome a variety of obstacles to pursue workflow interoperability, identify tools and libraries for robust science, port codes across different architectures, and establish trust in non-deterministic results. This poster presents recommendations to build a roadmap to overcome these challenges and enable robust science for HTC applications and workflows. The findings were collected from an international community of software developers during a Virtual World Cafe in May 2021.  more » « less
Award ID(s):
2028923 2028930
NSF-PAR ID:
10392351
Author(s) / Creator(s):
; ; ; ; ;
Editor(s):
IEEE Computer Society
Date Published:
Journal Name:
IEEE International Conference on Cluster Computing, CLUSTER 2021
Page Range / eLocation ID:
807 to 808
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. IEEE Computer Science (Ed.)
    This poster presents our first steps to define a roadmap to robust science for high-throughput applications used in scientific discovery. These applications combine multiple components into increasingly complex multi-modal workflows that are often executed in concert on heterogeneous systems. The increasing complexity hinders the ability of scientists to generate robust science (i.e., ensuring performance scalability in space and time; trust in technology, people, and infrastructures; and reproducible or confirmable research). Scientists must withstand and overcome adverse conditions such as heterogeneous and unreliable architectures at all scales (including extreme scale), rigorous testing under uncertainties, unexplainable algorithms in machine learning, and black-box methods. This poster presents findings and recommendations to build a roadmap to overcome these challenges and enable robust science. The data was collected from an international community of scientists during a virtual world cafe in February 2021 
    more » « less
  2. This report is intended to provide value to scientists, engineers, software developers, designers, analysts, regulators, students, and other stakeholders associated with (or intending to work with) computational models related to the mechanics of materials and structures (MOMS). This includes both modelers and experimentalists within the materials science and engineering, mechanical engineering, solid mechanics, structural dynamics, and related communities, spanning academic, industrial, and government affiliation sectors. This report was written with two types of people in mind: novices who have little or no prior experience in robust verification and validation (V&V) and associated/inseparable uncertainty quantification (UQ) practices, and those who have some V&V/UQ experience, but want to establish more rigorous practices. More specifically, researchers, developers, and students associated with materials (both structural and soft materials) and solid mechanics modeling, who utilize advanced computation, materials data, and/or experimental validation tools, should find the information in this report especially useful. It is critical that the community widely adopts robust V&V/UQ practices in order to improve trust, reduce risk, and improve the reliability of MOMS computational models. Beyond practitioners in this field, other stakeholders who can influence the future of advanced computational modeling associated with MOMS should find this report useful, as well. This includes individuals who support financial and/ or time investments in science and technologies surrounding computational modeling, such as funding officers and other decision-makers at federal agencies, and leaders/managers in industry. Educators teaching undergraduate and graduate courses related to MOMS, as well as department heads and/or deans within the relevant disciplines, also could use the information in this report to advance associated curricula and enhance research products. 
    more » « less
  3. Topology optimization is a systematic method of generating designs to meet specific engineering requirements. It is exploited today in several industries including aircraft, automobile, and machinery, and it strongly complements the emerging field of additive manufacturing. Yet, the wide-spread use of topology optimization has been deterred due to high computational cost and significant software/hardware investment. In this paper, we propose a cloud based topology optimization (CTO) framework to overcome these challenges, thereby promoting the wider use of topology optimization. CTO requires a confluence of several methods and technologies, each of which is discussed in this paper. First and foremost, CTO requires a fast 3D topology optimization method that can respond rapidly to multiple clients. Here, PareTO, a topological sensitivity based method is used as the backbone of the framework. PareTO relies on limited-memory finite element analysis with a deflated linear solver that is designed to exploit multi-core and many-core architectures. At the client-end, the framework relies on JavaScript based WebGL and ThreeJS technologies to display 3D geometry and formulate structural problems within a browser. Finally, Ajax, php and HTML5 technologies are exploited to achieve asynchronous and robust user experience. An implementation of this framework is available at www.cloudtopopt.com; to use this free service, JavaScript must be enabled within the browser. 
    more » « less
  4. CitSci.org is a global citizen science software platform and support organization housed at Colorado State University. The mission of CitSci is to help people do high quality citizen science by amplifying impacts and outcomes. This platform hosts over one thousand projects and a diverse volunteer base that has amassed over one million observations of the natural world, focused on biodiversity and ecosystem sustainability. It is a custom platform built using open source components including: PostgreSQL, Symfony, Vue.js, with React Native for the mobile apps. CitSci sets itself apart from other Citizen Science platforms through the flexibility in the types of projects it supports rather than having a singular focus. This flexibility allows projects to define their own datasheets and methodologies. The diversity of programs we host motivated us to take a founding role in the design of the PPSR Core, a set of global, transdisciplinary data and metadata standards for use in Public Participation in Scientific Research (Citizen Science) projects. Through an international partnership between the Citizen Science Association, European Citizen Science Association, and Australian Citizen Science Association, the PPSR team and associated standards enable interoperability of citizen science projects, datasets, and observations. Here we share our experience over the past 10+ years of supporting biodiversity research both as developers of the CitSci.org platform and as stewards of, and contributors to, the PPSR Core standard. Specifically, we share details about: the origin, development, and informatics infrastructure for CitSci our support for biodiversity projects such as population and community surveys our experiences in platform interoperability through PPSR Core working with the Zooniverse, SciStarter, and CyberTracker data quality data sharing goals and use cases. the origin, development, and informatics infrastructure for CitSci our support for biodiversity projects such as population and community surveys our experiences in platform interoperability through PPSR Core working with the Zooniverse, SciStarter, and CyberTracker data quality data sharing goals and use cases. We conclude by sharing overall successes, limitations, and recommendations as they pertain to trust and rigor in citizen science data sharing and interoperability. As the scientific community moves forward, we show that Citizen Science is a key tool to enabling a systems-based approach to ecosystem problems. 
    more » « less
  5. Gibbons, Jeremy (Ed.)
    CONTEXT The success of QuickCheck has led to the development of property-based testing (PBT) libraries for many languages and the process is getting increasing attention. However, unlike regular testing, PBT is not widespread in collegiate curricula. Furthermore, the value of PBT is not limited to software testing. The growing use of formal methods in, and the growth of software synthesis, all create demand for techniques to train students and developers in the art of specification writing. We posit that PBT forms a strong bridge between testing and the act of specification: it’s a form of testing where the tester is actually writing abstract specifications. INQUIRY Even well-informed technologists mention the difficulty of finding good motivating examples for its use. We take steps to fill this lacuna. APPROACH & KNOWLEDGE We find that the use of “relational” problems—those for which an input may admit multiple valid outputs—easily motivates the use of PBT. We also notice that such problems are readily available in the computer science pantheon of problems (e.g., many graph and sorting algorithms). We have been using these for some years now to teach PBT in collegiate courses. GROUNDING In this paper, we describe the problems we use and report on students’ completion of them. We believe the problems overcome some of the motivation issues described above. We also show that students can do quite well at PBT for these problems, suggesting that the topic is well within their reach. In the process, we introduce a simple method to evaluate the accuracy of their specifications, and use it to characterize their common mistakes. IMPORTANCE Based on our findings, we believe that relational problems are an underutilized motivating example for PBT. We hope this paper initiates a catalog of such problems for educators (and developers) to use, and also provides a concrete (though by no means exclusive) method to analyze the quality of PBT. 
    more » « less