Search for: All records

The Allocations Service for the Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS) program is charged with accepting, reviewing, and processing researchers’ requests to use resources that are integrated into the ACCESS ecosystem. We present as a case study the metrics framework used to evaluate the Allocations Service project, a metrics framework that aligns with the project’s goals and identifies key performance indicators (KPIs). Several of our top-level KPIs reflect complex concepts and are composite measures built from suites of metrics compiled from two primary sources: a well-instrumented allocations and accounting system and an annual survey of the ACCESS researcher community. This approach allows us to describe and measure complex concepts such as “democratization” and “ecosystem access time” in a quantitative manner and to target improvements to project activities. The metrics framework is augmented by metrics to measure the performance of the project team, to describe general ecosystem and allocations activity, and to capture publications from the researcher community. We used this framework to gather and present data as part of the ACCESS Allocations Service first annual NSF panel review. The metrics were largely successful at communicating our progress, but we also encountered a few unexpected technical issues with the data and calculations themselves, which we are continuing to refine. Presented here as a case study, this approach to a metrics framework for the Allocations Service has proved valuable in complementing more subjective descriptions of the project, its accomplishments, and progress toward our goals.

 

This paper uses accounting concepts—particularly the concept of Return on Investment (ROI)—to reveal the quantitative value of scientific research pertaining to a major US cyberinfrastructure project (XSEDE—the eXtreme Science and Engineering Discovery Environment). XSEDE provides operational and support services for advanced information technology systems, cloud systems, and supercomputers supporting non-classified US research, with an average budget for XSEDE of US$20M+ per year over the period studied (2014–2021). To assess the financial effectiveness of these services, we calculated a proxy for ROI, and converted quantitative measures of XSEDE service delivery into financial values using costs for service from the US marketplace. We calculated two estimates of ROI: a Conservative Estimate, functioning as a lower bound and using publicly available data for a lower valuation of XSEDE services; and a Best Available Estimate, functioning as a more accurate estimate, but using some unpublished valuation data. Using the largest dataset assembled for analysis of ROI for a cyberinfrastructure project, we found a Conservative Estimate of ROI of 1.87, and a Best Available Estimate of ROI of 3.24. Through accounting methods, we show that XSEDE services offer excellent value to the US government, that the services offered uniquely by XSEDE (that is, not otherwise available for purchase) were the most valuable to the facilitation of US research activities, and that accounting-based concepts hold great value for understanding the mechanisms of scientific research generally.

 

The need to train sustainability scientists and engineers to address the complex problems of our world has never been more apparent. We organized an interdisciplinary team of instructors from universities in the states of Maine, New Hampshire, and Rhode Island who designed, taught, and assessed a multi-university course to develop the core competencies necessary for advancing sustainability solutions. Lessons from the course translate across sustainability contexts, but our specific focus was on the issues and trade-offs associated with dams. Dams provide numerous water, energy, and cultural services to society while exacting an ecological toll that disrupts the flow of water, fish, and sediment in rivers. Like many natural resource management challenges, effective dam decisions require collaboration among diverse stakeholders and disciplines. We linked key sustainability principles and practices related to interdisciplinarity, stakeholder engagement, and problem-solving to student learning outcomes that are generalizable beyond our dam-specific context. Students and instructors co-created class activities to build capacity for interdisciplinary collaboration and encourage student leadership and creativity. Assessment results show that students responded positively to activities related to stakeholder engagement and interdisciplinary collaboration, particularly when practicing nested discussion and intrapersonal reflection. These activities helped broaden students’ perspectives on sustainability problems and built greater capacity for constructive communication and student leadership. 

Aging infrastructure and growing interests in river restoration have led to a substantial rise in dam removals in the United States. However, the decision to remove a dam involves many complex trade-offs. The benefits of dam removal for hazard reduction and ecological restoration are potentially offset by the loss of hydroelectricity production, water supply, and other important services. We use a multiobjective approach to examine a wide array of trade-offs and synergies involved with strategic dam removal at three spatial scales in New England. We find that increasing the scale of decision-making improves the efficiency of trade-offs among ecosystem services, river safety, and economic costs resulting from dam removal, but this may lead to heterogeneous and less equitable local-scale outcomes. Our model may help facilitate multilateral funding, policy, and stakeholder agreements by analyzing the trade-offs of coordinated dam decisions, including net benefit alternatives to dam removal, at scales that satisfy these agreements.