skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Enabling TCP Pacing using Programmable Data Plane Switches
Previous studies have observed that TCP pacing evenly spacing out packets-minimizes traffic burstiness, reduces packet losses, and increases throughput. However, the main drawback of pacing is that the number of flows and the bottleneck link capacity must be known in advance. With this information, pacing is achieved by manually tuning sender nodes to send at rates that aggregate to the bottleneck capacity. This paper proposes a scheme based on programmable switches by which rates are dynamically adjusted. These switches store the network's state in the data plane and notify sender nodes to update their pacing rates when the network's state changes, e.g., a new flow joins or leaves the network. The scheme uses a custom protocol that is encapsulated inside the IP Options header field and thus is compatible with legacy switches (i.e., the scheme does not require all switches to be programmable). Furthermore, the processing overhead at programmable switches is minimal, as custom packets are only generated when a flow joins or leaves the network. Simulation results conducted in Mininet demonstrate that the proposed scheme is capable of dynamically notifying hosts to adapt the pacing rate with a minimum delay, increasing throughput, mitigating the TCP sawtooth behavior, and achieving better fairness among concurrent flows. The proposed scheme and preliminary results are particularly attractive to applications such as Science DMZ, where typically a small number of large flows must share the bandwidth capacity.  more » « less
Award ID(s):
1829698
PAR ID:
10119182
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
2019 42nd International Conference on Telecommunications and Signal Processing (TSP)
Page Range / eLocation ID:
273 to 277
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract—Recent work has demonstrated how programmable switches can effectively detect attack traffic, such as denial- of-service attacks in the midst of high-volume network traffic. However, these techniques primarily rely on sampling- or sketch- based data structures that can only be used to approximate the characteristics of dominant flows in the network. As a result, such techniques are unable to effectively detect slow attacks such as SYN port scans, SSH brute forcing, or HTTP connection exploits, which do so by stealthily adding only a few packets to the network. In this work we explore how the combination of programmable switches, Smart network interface cards (sNICs), and hosts can enable fine-grained analysis of every flow in a cloud network, even those with only a small number of packets. We focus on analyzing packets at the start of each flow, as those packets often can help indicate whether a flow is benign or suspicious, e.g., by detecting an attack which fails to complete the TCP handshake in order to waste server connection resources. Our approach leverages the high-speed processing of a programmable switch while overcoming its primary limitation – very limited memory capacity – by judiciously sending some state for processing to the sNIC or the host which typically has more memory, but lower bandwidth. Achieving this requires careful design of data structures on the switch, such as a bloom filter and flow logs, and communication protocols between the switch, sNIC, and host, to coordinate state. 
    more » « less
  2. Recent work has demonstrated how programmable switches can effectively detect attack traffic, such as denial-of- service attacks in the midst of high-volume network traffic. However, these techniques primarily rely on sampling- or sketch- based data structures that can only be used to approximate the characteristics of dominant flows in the network. As a result, such techniques are unable to effectively detect slow attacks such as SYN port scans, SSH brute forcing, or HTTP connection exploits, which do so by stealthily adding only a few packets to the network. In this work we explore how the combination of programmable switches, Smart network interface cards (sNICs), and hosts can enable fine-grained analysis of every flow in a cloud network, even those with only a small number of packets. We focus on analyzing packets at the start of each flow, as those packets often can help indicate whether a flow is benign or suspicious, e.g., by detecting an attack which fails to complete the TCP handshake in order to waste server connection resources. Our approach leverages the high-speed processing of a programmable switch while overcoming its primary limitation - very limited memory capacity - by judiciously sending some state for processing to the sNIC or the host which typically has more memory, but lower bandwidth. Achieving this requires careful design of data structures on the switch, such as a bloom filter and flow logs, and communication protocols between the switch, sNIC , and host, to coordinate state. 
    more » « less
  3. Recent work shows that programmable switches can effectively detect attack traffic, such as denial-of-service attacks in the midst of high-volume network traffic. However, these techniques primarily rely on sampling or sketch-based data structures, which can only be used to approximate the characteristics of dominant flows in the network. As a result, such techniques are unable to effectively detect low-volume attacks that stealthily add only a few packets to the network. Our work explores how the combination of programmable switches, Smart network interface cards, and hosts can enable fine-grained analysis of every flow in a network, even those with only a small number of packets. We focus on analyzing packets at the start of each flow, as those packets often can help indicate whether a flow is benign or suspicious. We propose a unified architecture that spans the full programmable dataplane to take advantage of the strengths of each type of device. We are developing new filter data structures to efficiently track flows on the switch, dataplane-based communication protocols to quickly coordinate between devices, and caching approaches on the SmartNIC that help minimize the traffic load reaching the host. Our preliminary prototype can handle the full pipe bandwidth of 1.4 Tbps of traffic entering the Tofino switch, forward only 20 Gbps to the SmartNIC, and minimize the traffic load to 5 Gbps reaching the host due to our efficient flow filter, packet batching, and SmartNIC-based cache. 
    more » « less
  4. Overlay networks serve as the de facto network virtualization technique for providing connectivity among distributed containers. Despite the flexibility in building customized private container networks, overlay networks incur significant performance loss compared to physical networks (i.e., the native). The culprit lies in the inclusion of multiple network processing stages in overlay networks, which prolongs the network processing path and overloads CPU cores. In this paper, we propose mFlow, a novel packet steering approach to parallelize the in-kernel data path of network flows. mFlow exploits packet-level parallelism in the kernel network stack by splitting the packets of the same flow into multiple micro-flows, which can be processed in parallel on multiple cores. mFlow devises new, generic mechanisms for flow splitting while preserving in-order packet delivery with little overhead. Our evaluation with both micro-benchmarks and real-world applications demonstrates the effectiveness of mFlow, with significantly improved performance – e.g., by 81% in TCP throughput and 139% in UDP compared to vanilla overlay networks. mFlow even achieved higher TCP throughput than the native (e.g., 29.8 vs. 26.6 Gbps). 
    more » « less
  5. HPC networks and campus networks are beginning to leverage various levels of network programmability ranging from programmable network configuration (e.g., NETCONF/YANG, SNMP, OF-CONFIG) to software-based controllers (e.g., OpenFlow Controllers) to dynamic function placement via network function virtualization (NFV). While programmable networks offer new capabilities, they also make the network more difficult to debug. When applications experience unexpected network behavior, there is no established method to investigate the cause in a programmable network and many of the conventional troubleshooting debugging tools (e.g., ping and traceroute) can turn out to be completely useless. This absence of troubleshooting tools that support programmability is a serious challenge for researchers trying to understand the root cause of their networking problems. This paper explores the challenges of debugging an all-campus science DMZ network that leverages SDN-based network paths for high-performance flows. We propose Flow Tracer, a light-weight, data-plane-based debugging tool for SDN-enabled networks that allows end users to dynamically discover how the network is handling their packets. In particular, we focus on solving the problem of identifying an SDN path by using actual packets from the flow being analyzed as opposed to existing expensive approaches where either probe packets are injected into the network or actual packets are duplicated for tracing purposes. Our simulation experiments show that Flow Tracer has negligible impact on the performance of monitored flows. Moreover, our tool can be extended to obtain further information about the actual switch behavior, topology, and other flow information without privileged access to the SDN control plane. 
    more » « less