More Like this

1. Exploitation of network-segregated CPU resources in CMS

   https://doi.org/10.1051/epjconf/202125102020  

   Acosta-Silva, C.; Delgado Peris, A.; Flix, J.; Frey, J.; Hernández, J.M.; Yzquierdo, A. Pérez-Calero; Tannenbaum, T. (January 2021, EPJ Web of Conferences)

   Biscarat, C.; Campana, S.; Hegner, B.; Roiser, S.; Rovelli, C.I.; Stewart, G.A. (Ed.)

   CMS is tackling the exploitation of CPU resources at HPC centers where compute nodes do not have network connectivity to the Internet. Pilot agents and payload jobs need to interact with external services from the compute nodes: access to the application software (CernVM-FS) and conditions data (Frontier), management of input and output data files (data management services), and job management (HTCondor). Finding an alternative route to these services is challenging. Seamless integration in the CMS production system without causing any operational overhead is a key goal. The case of the Barcelona Supercomputing Center (BSC), in Spain, is particularly challenging, due to its especially restrictive network setup. We describe in this paper the solutions developed within CMS to overcome these restrictions, and integrate this resource in production. Singularity containers with application software releases are built and pre-placed in the HPC facility shared file system, together with conditions data files. HTCondor has been extended to relay communications between running pilot jobs and HTCondor daemons through the HPC shared file system. This operation mode also allows piping input and output data files through the HPC file system. Results, issues encountered during the integration process, and remaining concerns are discussed. 

   more » « less
2. Distributed VR: An Analysis of Inter-Server Traffic Through a LAN

   https://doi.org/10.1109/LANMAN61958.2024.10621900  

   Ali, Mahad; Yuksel, Murat (July 2024, IEEE)

   Immersive Virtual Reality (VR) applications demand low network latency, large bandwidth, and substantial computational resources. Despite significant progress in addressing these challenges, creating Distributed VR environments remains complex. Existing VR deployments are predominantly centralized. Extending VR to a distributed setup requires solving scalability challenges of the network support needed for VR servers distributed across a network. In particular, the scale of traffic between distributed VR servers and the interaction of this VR traffic's size with various features of the VR applications are unexplored. In this study, we present and evaluate a distributed multi-server VR environment based on Mozilla's popular open-source platform, Hubs, on a local area network (LAN). By conducting traffic measurements, we evaluate how the network traffic volume to support such distributed VR setups may evolve. Our work assesses the feasibility of creating such distributed VR environments. We find that the inter-server traffic exhibits logarithmic increase with respect to the client count when the clients make human-like movements, pointing to the scalability potential of Distributed VR environments. Additionally, the study lays the foundation for future optimizations, aiming to enhance the distributed VR experience for users. 

   more » « less
3. A Theory of Auto-Scaling for Resource Reservation in Cloud Services

   https://doi.org/10.1287/stsy.2021.0091  

   Psychas, Konstantinos; Ghaderi, Javad (February 2022, Stochastic Systems)

   We consider a distributed server system consisting of a large number of servers, each with limited capacity on multiple resources (CPU, memory, etc.). Jobs with different rewards arrive over time and require certain amounts of resources for the duration of their service. When a job arrives, the system must decide whether to admit it or reject it, and if admitted, in which server to schedule it. The objective is to maximize the expected total reward received by the system. This problem is motivated by control of cloud computing clusters, in which jobs are requests for virtual machines (VMs) or containers that reserve resources for various services, and rewards represent service priority of requests or price paid per time unit of service. We study this problem in an asymptotic regime where the number of servers and jobs’ arrival rates scale by a factor L, as L becomes large. We propose a resource reservation policy that asymptotically achieves at least 1/2, and under certain monotone property on jobs’ rewards and resources, at least [Formula: see text] of the optimal expected reward. The policy automatically scales the number of VM slots for each job type as the demand changes and decides in which servers the slots should be created in advance, without the knowledge of traffic rates. 

   more » « less
4. Simple Policies for Multiresource Job Scheduling

   Chen, Zhongrui; Grosof, Isaac; Berg, Benjamin (September 2024, Association for Computing Machinery)

   Data center workloads are composed of multiresource jobs requiring a variety of computational resources including CPU cores, memory, disk space, and hardware accelerators. Mod- ern servers can run multiple jobs in parallel, but a set of jobs can only run in parallel if the server has sufficient resources to satisfy the demands of each job. It is generally hard to find sets of jobs that perfectly utilize all server resources, and choosing the wrong set of jobs can lead to low resource uti- lization. This raises the question of how to allocate resources across a stream of arriving multiresource jobs to minimize the mean response time across jobs — the mean time from when a job arrives to the system until it is complete. Current policies for scheduling multiresource jobs are com- plex to analyze and hard to implement. We propose a class of simple policies, called Markovian Service Rate (MSR) policies. We show that the class of MSR policies is throughput- optimal, in that if a policy exists that can stabilize the sys- tem, then an MSR policy exists that stabilizes the system. We derive bounds on the mean response time under an MSR policy, and show how our bounds can be used to choose an MSR policy that minimizes mean response time. 

   more » « less
5. Accelerating Machine Learning Inference with GPUs in ProtoDUNE Data Processing

   https://doi.org/10.1007/s41781-023-00101-0  

   Cai, Tejin; Herner, Kenneth; Yang, Tingjun; Wang, Michael; Acosta Flechas, Maria; Harris, Philip; Holzman, Burt; Pedro, Kevin; Tran, Nhan (October 2023, Computing and Software for Big Science)

   Abstract We study the performance of a cloud-based GPU-accelerated inference server to speed up event reconstruction in neutrino data batch jobs. Using detector data from the ProtoDUNE experiment and employing the standard DUNE grid job submission tools, we attempt to reprocess the data by running several thousand concurrent grid jobs, a rate we expect to be typical of current and future neutrino physics experiments. We process most of the dataset with the GPU version of our processing algorithm and the remainder with the CPU version for timing comparisons. We find that a 100-GPU cloud-based server is able to easily meet the processing demand, and that using the GPU version of the event processing algorithm is two times faster than processing these data with the CPU version when comparing to the newest CPUs in our sample. The amount of data transferred to the inference server during the GPU runs can overwhelm even the highest-bandwidth network switches, however, unless care is taken to observe network facility limits or otherwise distribute the jobs to multiple sites. We discuss the lessons learned from this processing campaign and several avenues for future improvements. 

   more » « less