Search for: All records

A continuing trend in many scientific disciplines is the growth in the volume of data collected by scientific instruments and the desire to rapidly and efficiently distribute this data to the scientific community. Transferring these large data sets to a geographically distributed research community consumes significant network bandwidth. As both the data volume and number of subscribers grows, reliable network multicast is a promising approach to reduce the rate of growth of the bandwidth needed to support efficient data distribution. In prior work, we identified a need for reliable network multicast: scientists engaged in atmospheric research subscribing to meteorological file-streams. Specifically, the University Cooperation Atmospheric Research (UCAR) uses the Local Data Manager (LDM) to disseminate data. This work describes a trial deployment of a multicast-enabled LDM, in which eight university campuses are connected via corresponding regional Research-and-Education Networks (RENs) and Internet2. Using this deployment, we evaluated the new version of LDM, LDM7, which uses network multicast with a reliable transport protocol, and leverages Layer-2 (L2) multipoint Virtual LAN (VLANIMPLS). A performance monitoring system was deployed to collect real-time performance of LDM7, which showed that our proof-of-concept prototype worked significantly better than the current production LDM, LDM6, in two ways: (i) LDM7 can distribute file streams faster than LDM6. With six subscribers, an almost 22-fold improvement was observed with LDM7 at 100 Mbps. And (ii) to achieve a similar performance, LDM7 significantly reduces the need for bandwidth, which reduced the bandwidth requirement by about 90% over LDM6 to achieve 20 Mbps average throughput across four subscribers. 

In prior work, we proposed a cross-layer architecture called Multicast-Push Unicast-Pull (MPUP) for Software Defined Networks (SDN) to support a reliable file-stream multicast application. In this work, we improved the algorithms used to set parameters: transport-layer sender retransmission timer, VLAN rate (which is also the sending rate) and sender-buffer size. Experimental evaluation using feeds with metadata collected from real meteorology file streams was conducted. A significant finding is that the throughput achieved is smaller than the VLAN/sending rate even though file blocks are multicast continuously in UDP datagrams. Sender-buffer waiting times and propagation delays are the main reasons for the degraded throughput. For example, increasing the VLAN rate from 20 Mbps to 500 Mbps, reduced the degradation from 90% to 45%. However, the degradation increased from 45% to 58% when the VLAN rate was increased from 500 Mbps to 1 Gbps. We found an increase in the number of block retransmissions at the higher rates, which explains this increased degradation. Increasing RTT from 0.1 ms to 100 ms caused throughput to drop from 274.8 Mbps to 27.6 Mbps on a 500 Mbps VLAN. If transmission delay was a significant component in total latency, then throughput degradation relative to VLAN rate would be small; however, the meteorology file-streams used in our study have small-sized data products. Due to bandwidth borrowing between VLAN and IP-routed services, VLAN utilization is not important, and hence we recommend using the smallest rate at which sender-buffer waiting times are insignificant.