More Like this

1. Protecting Cyber-Physical System Testbeds from Red-Teaming/Blue-Teaming Experiments Gone Awry

   https://doi.org/10.1007/978-3-031-21280-2_8  

   Talukder, M.R.H.; Amin, M.A.; Ray, I (November 2022, Information Security Practice and Experience. ISPEC 2022. Lecture Notes in Computer Science)

   Su, C.; Gritzalis, D.; Piuri, V. (Ed.)

   Many cyber-physical systems (CPS) are critical infrastructure. Security attacks on these critical systems can have catastrophic consequences, putting human lives at risk. Consequently, it is very important to pace CPS systems to red-teaming/blue teaming exercises to understand vulnerabilities and the progression/impact of cyber attacks on them. Since it is not always prudent to conduct such security exercises on live CPS, researchers use CPS testbeds to conduct security-related experiments. Often, such testbeds are very expensive. Since attack scripts used in red-teaming/blue-teaming exercises are, in the strictest sense of the term, malicious in nature, there is a need to protect the testbed itself from these attack experiments that have the potential to go awry. Moreover, when multiple experiments are conducted on the same testbed, there is a need to maintain isolation among these experiments so that no experiment can accidentally or maliciously affect/compromise others. In this work, we describe a novel security architecture and framework to ensure protection of security-related experiments on a CPS testbed and at the same time support secure communication services among simultaneously running experiments based on well-formulated access control policies. 

   more » « less
2. Trailblazing Roblox Virtual Synthetic Testbed Development for Human-Robot Teaming Studies

   https://doi.org/10.1177/1071181322661470  

   Raimondo, Felix R.; Wolff, Alexandra T.; Hehr, Alexander J.; Peel, Matthew A.; Wong, Margaret E.; Chiou, Erin K.; Demir, Mustafa; Cookea, Nancy J. (September 2022, Proceedings of the Human Factors and Ergonomics Society Annual Meeting)

   Virtual testbeds are fundamental to the success of research on cognitive work in safety-critical domains. A testbed that can meet researchers' objectives and create a sense of reality for participants positively impacts the research process; they have the potential to allow researchers to address questions not achievable in physical environments. This paper discusses the development of a synthetic task environment (STE) for Urban Search and Rescue (USAR) to advance the boundaries of Human-Robot Teams (HRTs) using Roblox. Virtual testbeds can simulate USAR task environments and HRT interactions. After assessing alternative STE platforms, we discovered Roblox not only met our research capabilities but also would prove invaluable for research teams without substantial coding experience. This paper outlines the design process of creating an STE to meet our research team's objectives. 

   more » « less
3. Testbeds and Evaluation Frameworks for Anomaly Detection within Built Environments: A Systematic Review

   https://doi.org/10.1145/3722213  

   Alosaimi, Mohammed; Rana, Omer; Perera, Charith (September 2025, ACM Computing Surveys)

   The Internet of Things (IoT) has revolutionized built environments by enabling seamless data exchange among devices such as sensors, actuators, and computers. However, IoT devices often lack robust security mechanisms, making them vulnerable to cyberattacks, privacy breaches, and operational anomalies caused by environmental factors or device faults. While anomaly detection techniques are critical for securing IoT systems, the role of testbeds in evaluating these techniques has been largely overlooked. This systematic review addresses this gap by treating testbeds as first-class entities essential for the standardized evaluation and validation of anomaly detection methods in built environments. We analyze testbed characteristics, including infrastructure configurations, device selection, user-interaction models, and methods for anomaly generation. We also examine evaluation frameworks, highlighting key metrics and integrating emerging technologies such as edge computing and 5G networks into testbed design. By providing a structured and comprehensive approach to testbed development and evaluation, this paper offers valuable guidance to researchers and practitioners in enhancing the reliability and effectiveness of anomaly detection systems. Our findings contribute to the development of more secure, adaptable, and scalable IoT systems, ultimately improving the security, resilience, and efficiency of built environments. 

   more » « less
4. A Computationally Efficient, High-Fidelity Testbed for Building Climate Control

   https://doi.org/10.1115/1.4048895  

   Kircher, Kevin J.; Schaefer, Walter; Max Zhang, K. (February 2021, ASME Journal of Engineering for Sustainable Buildings and Cities)

   Abstract Advanced building climate control systems have the potential to significantly reduce greenhouse gas emissions and energy costs, but more research is needed to bring these systems to market. A key component of building control research is testing algorithms through simulation. Many high-fidelity simulation testbeds exist, but they tend to be complex and opaque to users. Simpler, more transparent testbeds also exist, but they tend to neglect important nonlinearities and disturbances encountered in practice. In this paper, we develop a simulation testbed that is computationally efficient, transparent and high fidelity. We validate the testbed empirically, then demonstrate its use through the examples of system identification, online state and parameter estimation, and model predictive control (MPC). The testbed is intended to enable rapid, reliable analysis of building control algorithms, thereby accelerating progress toward reducing greenhouse gas emissions at scale. We call the resulting testbed and supporting functions the bldg toolbox, which is free, open source, and available online. 

   more » « less
5. Lessons Learned from the Chameleon Testbed

   Keahey, K.; Anderson, J.; Zhen, Z; Riteau, P.; Ruth, P.; Stanzione, D.; Cevik, M.; Colleran, J.; Gunawi, H.S.; Hammock, C.; et al (July 2020, Proceedings of the 2020 USENIX Annual Technical Conference (USENIX ATC '20))

   The Chameleon testbed is a case study in adapting the cloud paradigm for computer science research. In this paper, we explain how this adaptation was achieved, evaluate it from the perspective of supporting the most experiments for the most users, and make a case that utilizing mainstream technology in research testbeds can increase efficiency without compro- mising on functionality. We also highlight the opportunity inherent in the shared digital artifacts generated by testbeds and give an overview of the efforts we’ve made to develop it to foster reproducibility. 

   more » « less