An official website of the United States government
In prior work, we proposed a cross-layer architecture called Multicast-Push Unicast-Pull (MPUP) for Software Defined Networks (SDN) to support a reliable file-stream multicast application. In this work, we improved the algorithms used to set parameters: transport-layer sender retransmission timer, VLAN rate (which is also the sending rate) and sender-buffer size. Experimental evaluation using feeds with metadata collected from real meteorology file streams was conducted. A significant finding is that the throughput achieved is smaller than the VLAN/sending rate even though file blocks are multicast continuously in UDP datagrams. Sender-buffer waiting times and propagation delays are the main reasons for the degraded throughput. For example, increasing the VLAN rate from 20 Mbps to 500 Mbps, reduced the degradation from 90% to 45%. However, the degradation increased from 45% to 58% when the VLAN rate was increased from 500 Mbps to 1 Gbps. We found an increase in the number of block retransmissions at the higher rates, which explains this increased degradation. Increasing RTT from 0.1 ms to 100 ms caused throughput to drop from 274.8 Mbps to 27.6 Mbps on a 500 Mbps VLAN. If transmission delay was a significant component in total latency, then throughput degradation relative to VLAN rate would be small; however, the meteorology file-streams used in our study have small-sized data products. Due to bandwidth borrowing between VLAN and IP-routed services, VLAN utilization is not important, and hence we recommend using the smallest rate at which sender-buffer waiting times are insignificant.
more »
« less
A Trial Deployment of a Reliable Network-Multicast Application across Internet2
A continuing trend in many scientific disciplines is the growth in the volume of data collected by scientific instruments and the desire to rapidly and efficiently distribute this data to the scientific community. Transferring these large data sets to a geographically distributed research community consumes significant network bandwidth. As both the data volume and number of subscribers grows, reliable network multicast is a promising approach to reduce the rate of growth of the bandwidth needed to support efficient data distribution. In prior work, we identified a need for reliable network multicast: scientists engaged in atmospheric research subscribing to meteorological file-streams. Specifically, the University Cooperation Atmospheric Research (UCAR) uses the Local Data Manager (LDM) to disseminate data. This work describes a trial deployment of a multicast-enabled LDM, in which eight university campuses are connected via corresponding regional Research-and-Education Networks (RENs) and Internet2. Using this deployment, we evaluated the new version of LDM, LDM7, which uses network multicast with a reliable transport protocol, and leverages Layer-2 (L2) multipoint Virtual LAN (VLANIMPLS). A performance monitoring system was deployed to collect real-time performance of LDM7, which showed that our proof-of-concept prototype worked significantly better than the current production LDM, LDM6, in two ways: (i) LDM7 can distribute file streams faster than LDM6. With six subscribers, an almost 22-fold improvement was observed with LDM7 at 100 Mbps. And (ii) to achieve a similar performance, LDM7 significantly reduces the need for bandwidth, which reduced the bandwidth requirement by about 90% over LDM6 to achieve 20 Mbps average throughput across four subscribers.
more »
« less
- Award ID(s):
- 1659174
- PAR ID:
- 10226043
- Date Published:
- Journal Name:
- 2020 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS)
- Page Range / eLocation ID:
- 22 to 32
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Efficiently transferring data over long-distance, high-speed networks requires optimal utilization of available network bandwidth. One effective method to achieve this is through the use of parallel TCP streams. This approach allows applications to leverage network parallelism, thereby enhancing transfer throughput. However, determining the ideal number of parallel TCP streams can be challenging due to non-deterministic background traffic sharing the network, as well as non-stationary and partially observable network signals. We present a novel learning-based approach that utilizes deep reinforcement learning (DRL) to determine the optimal number of parallel TCP streams. Our DRL-based algorithm is designed to intelligently utilize available network bandwidth while adapting to different network conditions. Unlike rule-based heuristics, which lack generalization in unknown network scenarios, our DRL-based solution can dynamically adjust the parallel TCP stream numbers to optimize network bandwidth utilization without causing network congestion and ensuring fairness among competing transfers. We conducted extensive experiments to evaluate our DRL-based algorithm’s performance and compared it with several state-of-the-art online optimization algorithms. The results demonstrate that our algorithm can identify nearly optimal solutions 40% faster while achieving up to 15% higher throughput. Furthermore, we show that our solution can prevent network congestion and distribute the available network resources fairly among competing transfers, unlike a discriminatory algorithm.more » « less
-
Sub-terahertz (THz) wireless communication links require low-SWaP (size, weight, and power) software defined radio (SDR) modems to achieve efficient and reliable data transmission. This research presents the design, development, and experimentation of an SDR system operating in the 135-150 GHz frequency range, utilizing simple I/Q modulation techniques such as differential binary phase shift keying (DBPSK). The system integrates advanced components, including Virginia Diodes (VDI) 110-170 GHz compact upconverter (CCU) and compact downconverter (CCD), high-gain lens horn antennas from Anteral (40 dBi), and the Xilinx RF-SoC ZCU-111 for real time DSP. A 500 MHz IF is implemented on RF-SoC with baseband bandwidth 64 MHz and data rate 64 Mbps via DBPSK modulation. For 20 dBm transmit power at 147 GHz, the nearfield SNR was measured to be 55 dB at 1m lens-to-lens separation for a baseband of 64 MHz. Simulation models of propagation predict 64 Mbps is possibly viable at up to 2 km in a point-to-point connection for a BER of < 10−3. The SDR was realized on the Xilinx PYNQ platform, offering a user-friendly interface while being adaptable to high data rate applications. This digital design is particularly suited for deployment in scenarios such as vehicle-to-vehicle communication, backhaul networks, and data center level interconnects. The research explored challenges related to synchronization, signal integrity, and environmental sensitivity, which are critical for maintaining reliable communication in a 147 GHz channel. A real-time text messaging application demonstrated correct operation of the PYNQ modem.more » « less
-
Traditionally, quantitative lidar techniques like differential absorption lidar (DIAL) and high-spectral-resolution lidar (HSRL) utilize high-power-aperture product designs, which partially compensates for the need to take discrete deriva-tives of noisy data in post-processing (for number density for DIAL and extinction for HSRL) and provides for high-performance measurements, i.e., higher resolu-tion, accuracy, or precision. Conversely, low-power-aperture product lidar designs are easier to make eye-safe, reliable, and cost-effective, which are important attributes for network development and field deployment. The atmospheric science community has expressed the need for high-quality, quantitative, robust, network deployable, and cost-effective sensors for a variety of applications such as improved numerical weather forecasting – in essence requiring the best of both worlds without the accompanying drawbacks. In response to this need, the National Center for Atmospheric Research and Montana State University have been developing the MicroPulse DIAL (MPD) architecture for thermodynamic profiling in the lower atmosphere. The MPD architecture takes advantage of the benefits of low-power, low-cost laser diodes, and fiber optics to achieve quantitative profiling leverag-ing narrowband filtering and efficient elastic scattering. A field-deployable MPD instrument capable of humidity, quantitative aerosol, and temperature profiling has recently been developed. This presentation will describe the current status of this thermodynamic profiler and the initial results from a recent field deployment. Emphasis will be given to the analysis of the temperature data including compar-isons to co-located radiosondes to describe current performance.more » « less
-
null (Ed.)The volume of data moving through a network increases with new scientific experiments and simulations. Network bandwidth requirements also increase proportionally to deliver data within a certain time frame. We observe that a significant portion of the popular dataset is transferred multiple times to different users as well as to the same user for various reasons. In-network data caching for the shared data has shown to reduce the redundant data transfers and consequently save network traffic volume. In addition, overall application performance is expected to improve with in-network caching because access to the locally cached data results in lower latency. This paper shows how much data was shared over the study period, how much network traffic volume was consequently saved, and how much the temporary in-network caching increased the scientific application performance. It also analyzes data access patterns in applications and the impacts of caching nodes on the regional data repository. From the results, we observed that the network bandwidth demand was reduced by nearly a factor of 3 over the study period.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/INDIS51933.2020.00008
