More Like this

1. Taming Performance Variability

   Maricq, A; Duplyakin, D; Jimenez, I; Maltzahn, C; Stutsman, R; Ricci, R (October 2018, Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI ’18))

   The performance of compute hardware varies: software run repeatedly on the same server (or a different server with supposedly identical parts) can produce performance results that differ with each execution. This variation has important effects on the reproducibility of systems research and ability to quantitatively compare the performance of different systems. It also has implications for commercial computing, where agreements are often made conditioned on meeting specific performance targets. Over a period of 10 months, we conducted a large-scale study capturing nearly 900,000 data points from 835 servers. We examine this data from two perspectives: that of a service provider wishing to offer a consistent environment, and that of a systems researcher who must understand how variability impacts experimental results. From this examination, we draw a number of lessons about the types and magnitudes of performance variability and the effects on confidence in experiment results. We also create a statistical model that can be used to understand how representative an individual server is of the general population. The full dataset and our analysis tools are publicly available, and we have built a system to interactively explore the data and make recommendations for experiment parameters based on statistical analysis of historical data. 

   more » « less
2. Variability-Inducing Requirements for Programs: Increasing Solution Variability for Similarity Checking

   https://doi.org/10.1145/3587102.3588855  

   Pang, Ashley; Vahid, Frank (June 2023, ITiCSE 2023: Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education)
3. Alluvial River Width Variability

   https://doi.org/10.4211/hs.56e0ac25249d4451845a843eb3154954  

   Phillips, Colin (November 2024, HydroShare)

   Data compilations of bankfull downstream hydraulic geometry for alluvial rivers, lidar derived high-resolution spatial series of bankfull width for 67 sites, and hydrograph metrics for sites with USGS hydrographs. This compilation is composed of three datasets: (1) a compilation of alluvial river geometry at bankfull for a variety of hydraulic attributes; (2) x, y, and z coordinates of channel bank lines for a selection of sites and their associated river widths derived from high-resolution lidar topography; and (3) statistics describing the hydrographs for a subset of the larger compilation. These data are divided into two primary sets, the larger compilation and a smaller set of sites where the bankfull width was derived from lidar topography. For the high-resolution dataset, the data are available as the coordinates of the bank lines, spatial series of distance downstream and bankfull width, and the spatial series filtered for quality. 

   more » « less
4. An Empirical Study of Real-World Variability Bugs Detected by Variability-Oblivious Tools

   https://doi.org/10.1145/3338906.3338967  

   Mordahl, Austin; Oh, Jeho; Koc, Ugur; Wei, Shiyi; Gazzillo, Paul (August 2019, ESEC/FSE 2019: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering)

   Many critical software systems developed in C utilize compile-time configurability. The many possible configurations of this software make bug detection through static analysis difficult. While variability-aware static analyses have been developed, there remains a gap between those and state-of-the-art static bug detection tools. In order to collect data on how such tools may perform and to develop real-world benchmarks, we present a way to leverage configuration sampling, off-the-shelf “variability-oblivious” bug detectors, and automatic feature identification techniques to simulate a variability-aware analysis. We instantiate our approach using four popular static analysis tools on three highly configurable, real-world C projects, obtaining 36,061 warnings, 80% of which are variability warnings. We analyze the warnings we collect from these experiments, finding that most results are variability warnings of a variety of kinds such as NULL dereference. We then manually investigate these warnings to produce a benchmark of 77 confirmed true bugs (52 of which are variability bugs) useful for future development of variability-aware analyses. 

   more » « less
5. Hypolimnetic Hypoxia Increases the Biomass Variability and Compositional Variability of Crustacean Zooplankton Communities

   https://doi.org/10.3390/w11102179  

   Doubek, Jonathan P.; Campbell, Kylie L.; Lofton, Mary E.; McClure, Ryan P.; Carey, Cayelan C. (October 2019, Water)

   In freshwater lakes and reservoirs, climate change and eutrophication are increasing the occurrence of low-dissolved oxygen concentrations (hypoxia), which has the potential to alter the variability of zooplankton seasonal dynamics. We sampled zooplankton and physical, chemical and biological variables (e.g., temperature, dissolved oxygen, and chlorophyll a) in four reservoirs during the summer stratified period for three consecutive years. The hypolimnion (bottom waters) of two reservoirs remained oxic throughout the entire stratified period, whereas the hypolimnion of the other two reservoirs became hypoxic during the stratified period. Biomass variability (measured as the coefficient of the variation of zooplankton biomass) and compositional variability (measured as the community composition of zooplankton) of crustacean zooplankton communities were similar throughout the summer in the oxic reservoirs; however, biomass variability and compositional variability significantly increased after the onset of hypoxia in the two seasonally-hypoxic reservoirs. The increase in biomass variability in the seasonally-hypoxic reservoirs was driven largely by an increase in the variability of copepod biomass, while the increase in compositional variability was driven by increased variability in the dominance (proportion of total crustacean zooplankton biomass) of copepod taxa. Our results suggest that hypoxia may increase the seasonal variability of crustacean zooplankton communities. 

   more » « less