skip to main content


Title: Flow-Level Loss Detection with Δ-Sketches
Packet drops caused by congestion are a fundamental problem in network operation. Yet, it is difficult to detect where drops are happening, let alone which flows are most affected. Detecting the small-timescale drops caused by short bursts of traffic is even more challenging, and traditional monitoring techniques can easily miss them. To uncover packet drops as they occur inside a switch, the analysis must be real-time, fine-grained, and efficient. However, modern switches have distributed packet-processing pipelines that see either the arriving or departing traffic, but not the packet drops. Plus, they do not have enough memory to store per-flow state. Our MIDST system addresses these challenges through a distributed compact data structure with lightweight coordination between ingress and egress pipelines. MIDST identifies the flows experiencing loss, as well as the bursty flows responsible, across different burst durations. Our evaluation with real-world traces and TCP connections shows that MIDST uses little memory (e.g., 320KB) while providing high accuracy (95% to 98%) under varying loss rates and burst durations. We evaluate a low-rate DDoS attack and demonstrate the potential use of our measurement results for attack detection and mitigation.  more » « less
Award ID(s):
2106946 2333887
PAR ID:
10356047
Author(s) / Creator(s):
; ; ; ; ;
Date Published:
Journal Name:
Proceedings of ACM SIGCOMM Symposium on SDN Research (SOSR '22)
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Programming Protocol-independent Packet Processors (P4) is an open-source domain-specific language to aid the data plane devices in programming packet forwarding. It has a variety of constructs optimized for this purpose. With P4, one can program ASICs, PISA chips, FPGAs, and many network devices since the language constructs allow true independence in some aspects that OpenFlow could not support. However, there are some challenges facing this technology. The first challenge is that P4 does not account for malicious traffic detection in the data plane pipeline. 2. The controllers have no secure medium of attack signature exchange. This ongoing work presents a multichain solution for detecting malicious traffic and exchanging attack signatures among controllers. This architecture uses an Artificial Immune System (AIS) based Intrusion Detection System (IDS), which runs on a distributed blockchain network, to introspect the P4 data plane to analyze and detect anomaly traffic flows. This IDS resides on the SideChain smart contracts and constantly monitors the traffic flow at the data planes based on introspection. Once malicious traffic is detected on any SideChain, the signatures are extracted and passed through the signature forwarding node to the MainChain for real-time storage. The malicious signatures are sent to all controllers via the mainchain network. We minimize the congestion the solution can cause to the P4 network by utilizing a load balancer to serve the SideChain. To evaluate the performance, we evaluate the False Positive Rate (FPR), Detection Rate (DR), and Accuracy (ACC) of the IDS. We also compute the execution time, performance overhead, and scalability of the proposed solution. 
    more » « less
  2. A Distributed Denial of Service (DDoS) attack is an attempt to make an online service, a network, or even an entire organization, unavailable by saturating it with traffic from multiple sources. DDoS attacks are among the most common and most devastating threats that network defenders have to watch out for. DDoS attacks are becoming bigger, more frequent, and more sophisticated. Volumetric attacks are the most common types of DDoS attacks. A DDoS attack is considered volumetric, or high-rate, when within a short period of time it generates a large amount of packets or a high volume of traffic. High-rate attacks are well-known and have received much attention in the past decade; however, despite several detection and mitigation strategies have been designed and implemented, high-rate attacks are still halting the normal operation of information technology infrastructures across the Internet when the protection mechanisms are not able to cope with the aggregated capacity that the perpetrators have put together. With this in mind, the present paper aims to propose and test a distributed and collaborative architecture for online high-rate DDoS attack detection and mitigation based on an in-memory distributed graph data structure and unsupervised machine learning algorithms that leverage real-time streaming data and analytics. We have successfully tested our proposed mechanism using a real-world DDoS attack dataset at its original rate in pursuance of reproducing the conditions of an actual large scale attack. 
    more » « less
  3. Despite advances in network security, attacks targeting mission critical systems and applications remain a significant problem for network and datacenter providers. Existing telemetry platforms detect volumetric attacks at terabit scales using approximation techniques and coarse grain analysis. However, the prevalence of low and slow attacks that require very little bandwidth, makes flow-state tracking critical to overall attack mitigation. Traffic queries deployed on network switches are often limited by hardware constraints, preventing them from carrying out flow tracking features required to detect stealthy attacks. Such attacks can go undetected in the midst of high traffic volumes. We design SmartWatch, a novel flow state tracking and flow logging system at line rate, using SmartNICs to optimize performance and simultaneously detect a number of stealthy attacks. SmartWatch leverages advances in switch based network telemetry platforms to process the bulk of the traffic and only forward suspicious traffic subsets to the SmartNIC. The programmable network switches perform coarse-grained traffic analysis while the SmartNIC conducts the finer-grained analysis which involves additional processing of the packet as a 'bump-in-the-wire'. A control loop between the SmartNIC and programmable switch tunes the queries performed in the switch to direct the most appropriate traffic subset to the SmartNIC. SmartWatch's cooperative monitoring approach yields 2.39 times better detection rate compared to existing platforms deployed on programmable switches. SmartWatch can detect covert timing channels and perform website fingerprinting more efficiently compared to standalone programmable switch solutions, relieving switch memory and control-plane processor resources. Compared to host-based approaches, SmartWatch can reduce the packet processing latency by 72.32%. 
    more » « less
  4. Modern end-host network stacks have to handle traffic from tens of thousands of flows and hundreds of virtual machines per single host, to keep up with the scale of modern clouds. This can cause congestion for traffic egressing from the end host. The effects of this congestion have received little attention. Currently, an overflowing queue, like a kernel queuing discipline, will drop incoming packets. Packet drops lead to worse network and CPU performance by inflating the time to transmit the packet as well as spending extra effort on retransmissions. In this paper, we show that current end-host mechanisms can lead to high CPU utilization, high tail latency, and low throughput in cases of congestion of egress traffic within the end host. We present zD, a framework for applying backpressure from a congested queue to traffic sources at end hosts that can scale to thousands of flows. We implement zD to apply backpressure in two settings: i) between TCP sources and kernel queuing discipline, and ii) between VMs as traffic sources and kernel queuing discipline in the hypervisor. zD improves throughput by up to 60%, and improves tail RTT by at least 10x at high loads, compared to standard kernel implementation. 
    more » « less
  5. Recent work has demonstrated how programmable switches can effectively detect attack traffic, such as denial-of- service attacks in the midst of high-volume network traffic. However, these techniques primarily rely on sampling- or sketch- based data structures that can only be used to approximate the characteristics of dominant flows in the network. As a result, such techniques are unable to effectively detect slow attacks such as SYN port scans, SSH brute forcing, or HTTP connection exploits, which do so by stealthily adding only a few packets to the network. In this work we explore how the combination of programmable switches, Smart network interface cards (sNICs), and hosts can enable fine-grained analysis of every flow in a cloud network, even those with only a small number of packets. We focus on analyzing packets at the start of each flow, as those packets often can help indicate whether a flow is benign or suspicious, e.g., by detecting an attack which fails to complete the TCP handshake in order to waste server connection resources. Our approach leverages the high-speed processing of a programmable switch while overcoming its primary limitation - very limited memory capacity - by judiciously sending some state for processing to the sNIC or the host which typically has more memory, but lower bandwidth. Achieving this requires careful design of data structures on the switch, such as a bloom filter and flow logs, and communication protocols between the switch, sNIC , and host, to coordinate state. 
    more » « less