More Like this

1. Network Capabilities for the HL-LHC Era

   https://doi.org/10.1051/epjconf/202024507051  

   Babik, Marian; McKee, Shawn (January 2020, EPJ Web of Conferences)

   Doglioni, C.; Kim, D.; Stewart, G.A.; Silvestris, L.; Jackson, P.; Kamleh, W. (Ed.)

   High Energy Physics (HEP) experiments rely on the networks as one of the critical parts of their infrastructure both within the participating laboratories and sites as well as globally to interconnect the sites, data centres and experiments instrumentation. Network virtualisation and programmable networks are two key enablers that facilitate agile, fast and more economical network infrastructures as well as service development, deployment and provisioning. Adoption of these technologies by HEP sites and experiments will allow them to design more scalable and robust networks while decreasing the overall cost and improving the effectiveness of the resource utilization. The primary challenge we currently face is ensuring that WLCG and its constituent collaborations will have the networking capabilities required to most effectively exploit LHC data for the lifetime of the LHC. In this paper we provide a high level summary of the HEPiX NFV Working Group report that explored some of the novel network capabilities that could potentially be deployment in time for HL-LHC. 

   more » « less
2. TRACER (TRACe route ExploRer): A tool to explore OSG/WLCG network route topologies

   https://doi.org/10.1142/S0217751X21300052  

   Tretyakov, Evgeniy; Artamonov, Alexey; Grigorieva, Maria; Klimentov, Alexei; McKee, Shawn; Vukotic, Ilija (February 2021, International Journal of Modern Physics A)

   null (Ed.)

   The experiments at the Large Hadron Collider (LHC) rely upon a complex distributed computing infrastructure (WLCG) consisting of hundreds of individual sites worldwide at universities and national laboratories, providing about half a billion computing job slots and an exabyte of storage interconnected through high speed networks. Wide Area Networking (WAN) is one of the three pillars (together with computational resources and storage) of LHC computing. More than 5 PB/day are transferred between WLCG sites. Monitoring is one of the crucial components of WAN and experiments operations. In the past years all experiments have invested significant effort to improve monitoring and integrate networking information with data management and workload management systems. All WLCG sites are equipped with perfSONAR servers to collect a wide range of network metrics. We will present the latest development to provide the 3D force directed graph visualization for data collected by perfSONAR. The visualization package allows site admins, network engineers, scientists and network researchers to better understand the topology of our Research and Education networks and it provides the ability to identify nonreliable or/and nonoptimal network paths, such as those with routing loops or rapidly changing routes. 

   more » « less
3. Modeling Distributed Computing Infrastructures for HEP Applications

   https://doi.org/10.1051/epjconf/202429504032  

   Horzela, Maximilian; Casanova, Henri; Giffels, Manuel; Gottmann, Artur; Hofsaess, Robin; Quast, Günter; Rossi_Tisbeni, Simone; Streit, Achim; Suter, Frédéric (January 2024, EPJ Web of Conferences)

   De_Vita, R; Espinal, X; Laycock, P; Shadura, O (Ed.)

   Predicting the performance of various infrastructure design options in complex federated infrastructures with computing sites distributed over a wide area network that support a plethora of users and workflows, such as the Worldwide LHC Computing Grid (WLCG), is not trivial. Due to the complexity and size of these infrastructures, it is not feasible to deploy experimental test-beds at large scales merely for the purpose of comparing and evaluating alternate designs. An alternative is to study the behaviours of these systems using simulation. This approach has been used successfully in the past to identify efficient and practical infrastructure designs for High Energy Physics (HEP). A prominent example is the Monarc simulation framework, which was used to study the initial structure of the WLCG. New simulation capabilities are needed to simulate large-scale heterogeneous computing systems with complex networks, data access and caching patterns. A modern tool to simulate HEP workloads that execute on distributed computing infrastructures based on the SimGrid and WRENCH simulation frameworks is outlined. Studies of its accuracy and scalability are presented using HEP as a case-study. Hypothetical adjustments to prevailing computing architectures in HEP are studied providing insights into the dynamics of a part of the WLCG and candidates for improvements. 

   more » « less
4. Understanding Container Network Interface Plugins: Design Considerations and Performance

   https://doi.org/10.1109/LANMAN49260.2020.9153266  

   Qi, Shixiong; Kulkarni, Sameer G; Ramakrishnan, K. K. (July 2020, 26th {IEEE} International Symposium on Local and Metropolitan Area Networks, {LANMAN} 2020, Orlando, FL, USA, July 13-15, 2020)

   Kubernetes, an open-source container orchestration platform, has been widely adopted by cloud service providers (CSPs) for its advantages in simplifying container deployment, scalability and scheduling. Networking is one of the central components of Kubernetes, providing connectivity between different pods (group of containers) both within the same host and across hosts. To bootstrap Kubernetes networking, the Container Network Interface (CNI) provides a unified interface for the interaction between container runtimes. There are several CNI implementations, available as open-source ‘CNI plugins’. While they differ in functionality and performance, it is a challenge for a cloud provider to differentiate and choose the appropriate plugin for their environment. In this paper, we compare the various open source CNI plugins available from the community, qualitatively and through detailed quantitative measurements. With our experimental evaluation, we analyze the overheads and bottlenecks for each CNI plugin, as a result of the network model it implements, interaction with the host network protocol stack and the network policies implemented in iptables rules. The choice of the CNI plugin may also be based on whether intra-host or inter-host communication dominates. 

   more » « less
5. Packet-Level Analytics in Software without Compromises

   Michel, O; Sonchack, J; Keller, E; Smith, J M (July 2018, USENIX Workshop on Hot Topics in Cloud Computing (HotCloud))

   Traditionally, network monitoring and analytics systems rely on aggregation (e.g., flow records) or sampling to cope with high packet rates. This has the downside that, in doing so, we lose data granularity and accu- racy, and, in general, limit the possible network analytics we can perform. Recent proposals leveraging software- defined networking or programmable hardware provide more fine-grained, per-packet monitoring but are still based on the fundamental principle of data reduction in the network, before analytics. In this paper, we pro- vide a first step towards a cloud-scale, packet-level mon- itoring and analytics system based on stream processing entirely in software. Software provides virtually unlim- ited programmability and makes modern ( e.g.,machine-learning) network analytics applications possible. We identify unique features of network analytics applica- tions which enable the specialization of stream process- ing systems. As a result, an evaluation with our pre- liminary implementation shows that we can scale up to several million packets per second per core and together with load balancing and further optimizations, the vision of cloud-scale per-packet network analytics is possible. 

   more » « less