A continuing trend in many scientific disciplines is the growth in the volume of data collected by scientific instruments and the desire to rapidly and efficiently distribute this data to the scientific community. As both the data volume and number of subscribers grows, a reliable network multicast is a promising approach to alleviate the demand for the bandwidth needed to support efficient data distribution to multiple, geographically-distributed, research communities. In prior work, we identified the need for a reliable network multicast: scientists engaged in atmospheric research subscribing to meteorological file-streams. An application called Local Data Manager (LDM) is used to disseminate meteorological data to hundreds of subscribers. This paper presents a high-performance, reliable network multicast solution, Dynamic Reliable File-Stream Multicast Service (DRFSM), and describes a trial deployment comprising eight university campuses connected via Research-and-Education Networks (RENs) and Internet2 and a DRFSM-enabled LDM (LDM7). Using this deployment, we evaluated the DRFSM architecture, which uses network multicast with a reliable transport protocol, and leverages Layer-2 (L2) multipoint Virtual LAN (VLAN/MPLS). A performance monitoring system was developed to collect the realtime performance of LDM7. The measurements showed that our proof-of-concept prototype worked significantly better than the current production LDM (LDM6) in two ways. First, LDM7 distributes data faster than LDM6. With six subscribers and a 100 Mbps bandwidth limit setting, an almost 22-fold improvement in delivery time was observed with LDM7. Second, LDM7 significantly reduces the bandwidth requirement needed to deliver data to subscribers. LDM7 needed 90% less bandwidth than LDM6 to achieve a 20 Mbps average throughput across four subscribers.
more »
« less
A Trial Deployment of a Reliable Network-Multicast Application across Internet2
A continuing trend in many scientific disciplines is the growth in the volume of data collected by scientific instruments and the desire to rapidly and efficiently distribute this data to the scientific community. Transferring these large data sets to a geographically distributed research community consumes significant network bandwidth. As both the data volume and number of subscribers grows, reliable network multicast is a promising approach to reduce the rate of growth of the bandwidth needed to support efficient data distribution. In prior work, we identified a need for reliable network multicast: scientists engaged in atmospheric research subscribing to meteorological file-streams. Specifically, the University Cooperation Atmospheric Research (UCAR) uses the Local Data Manager (LDM) to disseminate data. This work describes a trial deployment of a multicast-enabled LDM, in which eight university campuses are connected via corresponding regional Research-and-Education Networks (RENs) and Internet2. Using this deployment, we evaluated the new version of LDM, LDM7, which uses network multicast with a reliable transport protocol, and leverages Layer-2 (L2) multipoint Virtual LAN (VLANIMPLS). A performance monitoring system was deployed to collect real-time performance of LDM7, which showed that our proof-of-concept prototype worked significantly better than the current production LDM, LDM6, in two ways: (i) LDM7 can distribute file streams faster than LDM6. With six subscribers, an almost 22-fold improvement was observed with LDM7 at 100 Mbps. And (ii) to achieve a similar performance, LDM7 significantly reduces the need for bandwidth, which reduced the bandwidth requirement by about 90% over LDM6 to achieve 20 Mbps average throughput across four subscribers.
more »
« less
- Award ID(s):
- 1659174
- PAR ID:
- 10226043
- Date Published:
- Journal Name:
- 2020 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS)
- Page Range / eLocation ID:
- 22 to 32
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
In prior work, we proposed a cross-layer architecture called Multicast-Push Unicast-Pull (MPUP) for Software Defined Networks (SDN) to support a reliable file-stream multicast application. In this work, we improved the algorithms used to set parameters: transport-layer sender retransmission timer, VLAN rate (which is also the sending rate) and sender-buffer size. Experimental evaluation using feeds with metadata collected from real meteorology file streams was conducted. A significant finding is that the throughput achieved is smaller than the VLAN/sending rate even though file blocks are multicast continuously in UDP datagrams. Sender-buffer waiting times and propagation delays are the main reasons for the degraded throughput. For example, increasing the VLAN rate from 20 Mbps to 500 Mbps, reduced the degradation from 90% to 45%. However, the degradation increased from 45% to 58% when the VLAN rate was increased from 500 Mbps to 1 Gbps. We found an increase in the number of block retransmissions at the higher rates, which explains this increased degradation. Increasing RTT from 0.1 ms to 100 ms caused throughput to drop from 274.8 Mbps to 27.6 Mbps on a 500 Mbps VLAN. If transmission delay was a significant component in total latency, then throughput degradation relative to VLAN rate would be small; however, the meteorology file-streams used in our study have small-sized data products. Due to bandwidth borrowing between VLAN and IP-routed services, VLAN utilization is not important, and hence we recommend using the smallest rate at which sender-buffer waiting times are insignificant.more » « less
-
The networking industry is offering new services leveraging recent technological advances in connectivity, storage, and computing such as mobile communications and edge computing. In this regard, extended reality, a term encompassing virtual reality, augmented reality, and mixed reality, can provide unprecedented user experience and pioneering service opportunities such as: live concerts, sports, and other events; interactive gaming and entertainment; immersive education, training, and demos. These services require high-bandwidth, low-latency, and reliable connections, and are supported by next-generation ultra-reliable and low-latency communications in the vision of 6G mobile communication systems. In this work, we devise a novel scheme, called backup from different data centers with multicast and adaptive bandwidth provisioning, to admit reliable, low-latency, and high-bandwidth extended reality live streams in next-generation networks. We consider network services where contents are non-cacheable and investigate how backup services can be offered by different data centers with multicast and adaptive bandwidth provisioning. Our proposed service-provisioning scheme provides protection not only against link failures in the physical network but also against computing and storage failures in data centers. We develop scalable algorithms for the service-provisioning scheme and evaluate their performance on various complex network instances in a dynamic environment. Numerical results show that, compared to conventional service-provisioning schemes such as those seeking backup services from the same data center, our proposed service-provisioning scheme efficiently utilizes network resources, ensures higher reliability, and guarantees low latency; hence, it is highly suitable for extended reality live streams.more » « less
-
Efficiently transferring data over long-distance, high-speed networks requires optimal utilization of available network bandwidth. One effective method to achieve this is through the use of parallel TCP streams. This approach allows applications to leverage network parallelism, thereby enhancing transfer throughput. However, determining the ideal number of parallel TCP streams can be challenging due to non-deterministic background traffic sharing the network, as well as non-stationary and partially observable network signals. We present a novel learning-based approach that utilizes deep reinforcement learning (DRL) to determine the optimal number of parallel TCP streams. Our DRL-based algorithm is designed to intelligently utilize available network bandwidth while adapting to different network conditions. Unlike rule-based heuristics, which lack generalization in unknown network scenarios, our DRL-based solution can dynamically adjust the parallel TCP stream numbers to optimize network bandwidth utilization without causing network congestion and ensuring fairness among competing transfers. We conducted extensive experiments to evaluate our DRL-based algorithm’s performance and compared it with several state-of-the-art online optimization algorithms. The results demonstrate that our algorithm can identify nearly optimal solutions 40% faster while achieving up to 15% higher throughput. Furthermore, we show that our solution can prevent network congestion and distribute the available network resources fairly among competing transfers, unlike a discriminatory algorithm.more » « less
-
Sub-terahertz (THz) wireless communication links require low-SWaP (size, weight, and power) software defined radio (SDR) modems to achieve efficient and reliable data transmission. This research presents the design, development, and experimentation of an SDR system operating in the 135-150 GHz frequency range, utilizing simple I/Q modulation techniques such as differential binary phase shift keying (DBPSK). The system integrates advanced components, including Virginia Diodes (VDI) 110-170 GHz compact upconverter (CCU) and compact downconverter (CCD), high-gain lens horn antennas from Anteral (40 dBi), and the Xilinx RF-SoC ZCU-111 for real time DSP. A 500 MHz IF is implemented on RF-SoC with baseband bandwidth 64 MHz and data rate 64 Mbps via DBPSK modulation. For 20 dBm transmit power at 147 GHz, the nearfield SNR was measured to be 55 dB at 1m lens-to-lens separation for a baseband of 64 MHz. Simulation models of propagation predict 64 Mbps is possibly viable at up to 2 km in a point-to-point connection for a BER of < 10−3. The SDR was realized on the Xilinx PYNQ platform, offering a user-friendly interface while being adaptable to high data rate applications. This digital design is particularly suited for deployment in scenarios such as vehicle-to-vehicle communication, backhaul networks, and data center level interconnects. The research explored challenges related to synchronization, signal integrity, and environmental sensitivity, which are critical for maintaining reliable communication in a 147 GHz channel. A real-time text messaging application demonstrated correct operation of the PYNQ modem.more » « less
An official website of the United States government

