Search for: All records

The efficiency of high energy physics workflows relies on the ability to rapidly transfer data among the sites where the data is processed and analyzed. The best data transfer tools should provide a simple and reliable solution for local, regional, national and in some cases intercontinental data transfers. This work outlines the results of data transfer tool tests using internal and external (simulated latency and packet loss) in 100 Gbps testbeds and compares the results among the existing solutions, while also treating the issue of tuning parameters and methods to help optimize the rates of transfers. Many tools have been developed to facilitate data transfers over wide area networks. However, few studies have shown the tools’ requirements, use cases, and reliability through comparative measurements. Here, we were evaluating a variety of high-performance data transfer tools used today in the LHC and other scientific communities, such as FDT, WDT, and NDN in different environments. Furthermore, this test was made to reproduce real-world data transfer examples to analyse each tool’s strengths and weaknesses, including the fault tolerance of the tools when we have packet loss. By comparing the tools in a controlled environment, we can shed light on the tool’s relative reliability and usability for academia and industry. Also, this work highlights the best tuning parameters for WAN and LAN transfers for maximum performance, in several cases. 

The Large Hadron Collider (LHC) experiments distribute data by leveraging a diverse array of National Research and Education Networks (NRENs), where experiment data management systems treat networks as a “blackbox” resource. After the High Luminosity upgrade, the Compact Muon Solenoid (CMS) experiment alone will produce roughly 0.5 exabytes of data per year. NREN Networks are a critical part of the success of CMS and other LHC experiments. However, during data movement, NRENs are unaware of data priorities, importance, or need for quality of service, and this poses a challenge for operators to coordinate the movement of data and have predictable data flows across multi-domain networks. The overarching goal of SENSE (The Software-defined network for End-to-end Networked Science at Exascale) is to enable National Labs and universities to request and provision end-to-end intelligent network services for their application workflows leveraging SDN (Software-Defined Networking) capabilities. This work aims to allow LHC Experiments and Rucio, the data management software used by CMS Experiment, to allocate and prioritize certain data transfers over the wide area network. In this paper, we will present the current progress of the integration of SENSE, Multi-domain end-to-end SDN Orchestration with QoS (Quality of Service) capabilities, with Rucio, the data management software used by CMS Experiment. 

This work presents the design and implementation of an Open Storage System plugin for XRootD, utilizing Named Data Networking (NDN). This represents a significant step in integrating NDN, a prominent future Internet architecture, with the established data management systems within CMS. We show that this integration enables XRootD to access data in a location transparent manner, reducing the complexity of data management and retrieval. Our approach includes the creation of the NDNc software library, which bridges the existing NDN C++ library with the high-performance NDN-DPDK data-forwarding system. This paper outlines the design of the plugin and preliminary results of data transfer tests using both internal and external 100 Gbps testbed. 

This paper presents the rapid progress, vision and outlook across multiple state of the art development lines within the Global Network Advancement Group (GNA-G) and its Data Intensive Sciences and SENSE/AutoGOLE working groups, which are designed to meet the present and future needs and address the challenges of the Large Hadron Collider and other science programs with global reach. Since it was founded in the Fall of 2019 and the working groups were formed in 2020, in partnership with ESnet, Internet2, CENIC, GEANT, ANA, RNP, StarLight, NRP, N-DISE, AmLight, and many other leading research and education networks and network R&D projects, as well as Caltech, UCSD/SDSC, Fermilab, CERN, LBL, and many other leading universities and laboratories, the GNA-G working groups have deployed two virtual circuit and programmable testbeds spanning six continents which supports continuous developments aimed at the next generation of programmable networks interworking with the science programs’ computing and data management systems. The talk covers examples of recent progress in developing and deploying new methods and approaches in multidomain virtual circuits, flow steering, path selection, load balancing and congestion avoidance, segment routing and machine learning based traffic prediction and optimization. 

As in-vehicle communication becomes more complex, the automotive community is exploring various architectural options such as centralized and zonal architectures for their numerous benefits. Common characteristics of these architectures include the need for high-bandwidth communication and security, which have been elusive with standard automotive architectures. Further, as automotive communication technologies evolve, it is also likely that multiple link-layer technologies such as CAN and Automotive Ethernet will co-exist. These alternative architectures promise to integrate these diverse sets of technologies. However, architectures that allow such co-existence have not been adequately explored. In this work we explore a new network architecture called Named Data Networking (NDN) to achieve multiple goals: provide a foundational security infrastructure and bridge different link layer protocols such as CAN, LIN, and automotive Ethernet into a unified communication system. We have created a proof-of-concept bench-top testbed using CAN HATS and Raspberry PIs that replay real traffic over CAN and Ethernet to demonstrate how NDN can provide a secure, high-speed bridge between different automotive link layers. We also show how NDN can support communication between centralized or zonal high-power compute components. Security is achieved through digitally signing all Data packets between these components, preventing unauthorized ECUs from injecting arbitrary data into the network. We also demonstrate NDN's ability to prevent DoS and replay attacks between different network segments connected through NDN.