Software-based Network functions (NFs) have evolved significantly in recent years and have become an integral part of service provider, enterprise, and data center networks. These NFs are typically high speed packet processing engines functioning as a bump-in-the-wire on the data path and may need to process several million packets per second, as link rates scale up from 10G, to 40G, to 100Gbps. They support a variety of in-network services such as network address translation (NAT), firewalls (FW), intrusion detection and prevention (IDS/IPS), etc. Network flows typically pass through more than one NF, being processed in a specific order referred to as a Service Function Chain (SFC). Figure 1 shows a simple chain consisting of NAT, FW, IDS and rate limiter NFs.
NFs operate inline with the network forwarding datapath, and as such, NF failure or underlying hardware failures (server node, link) can significantly disrupt network operations. Hence providing NF failure resiliency is critical. Further, we observe that NFs differ in their computational complexity and can drastically vary in their packet processing rates [1]. In order to meet varying traffic demands and to meet chain-wide performance goals, the NFs in a chain may need to be elastically scaledi.e., networks have to dynamically adapt the number of NF instances and balance the load across them. Works such as Pico replication [2], FTMB [3] and Reinforce [4] 1 make use of stateful NFs that maintain their state locally within the NF instance and share the state across multiple NF instances through message passing or other traditional operating system shared memory constructs. However, support for shared state in these works is limited to the instances within a single server node. In contrast, Stateless NFs are a new architectural approach for managing state in network function virtualization (NFV) platforms. Stateless NFs decouple the existing design of NFs into a stateless processing component and a data store layer. They break the tight coupling of state with the processing components, thus seeking to enable a more elastic and resilient network function infrastructure [5]. Works such as CHC [6] have followed such an approach and extended the state management aspects to facilitate very low cost (perpacket processing latency) chain-wide failure resiliency.
In this work, we compare stateful and stateless NF management and deployment paradigms. We implement and deploy stateful and stateless NFs and NF chains on OpenNetVM [7], a DPDK based high performance NFV platform. We provide both qualitative and quantitative analysis and results on the impact of incorporating stateful or stateless NFs on our NFV platform. We specifically target the impact on NF performance (failure-free operation) and their ability to support features such as elastic scaling and fault-tolerance while addressing non-determinism and having chain-wide consistent operations.
Some NFs may be inherently stateless i.e., they do not maintain any state associated with packet processing e.g., stateless firewalls utilize static pre-configured access control rules to block certain packets. However, a large number of NFs are stateful and maintain flow/packet specific state information
Per-flow state, etc.
Shared counters, etc.
Per-flow, Shared counters, etc.
Per-flow, Shared counters, etc.
Per-flow, Shared counters, etc.
Local State (Cached) e.g., IDS, load balancer, etc. [3]. The state maintained by the NFs may correspond to i) per flow statusi.e., state for each new flow e.g., application delivery controllers and stateful firewalls; ii) per packet status -state associated with the processing of each individual packet by the NFs e.g., IDS.
In this work, we focus only on the stateful NFs.
In [8], they categorize the state maintained by the NFs into i) Internal -ephemeral state: it is of no consequence outside that NF instance's execution, e.g., application logic, resource mappings (CPU core, configuration files), etc. ii) External state -Partitioned and coherent state: NF state that is required for, and impacts, the packet processing. It includes Partitioned statee.g., per flow state, that is often specific to an NF instance and differs across different replicas, and coherent statee.g., shared global counters and state associated with a group of flows, which can be updated by different NF instances and need to be kept consistent across NF replicas. In addition, changes made to the packet due to the NF processing also correspond to the state e.g., NAT, load balancer NFs modify the TCP/IP headers; Firewalls, IDS/IPS may modify the packet routing state across the NF chains. Figure 2 (a) shows the NF state partitioning and packet processing with the Stateful NFs.
Non-determinism (ND) is pervasive in NFs [3], [4]. Output from two identical NFs, even when provided with identical inputs, can differ due to non-determinism exhibited by the NFs in processing the packets. ND can be due to a) the local behavior of each NF i.e., hardware dependence whose outcome cannot be predicted, such as hardware clocks, random number generators, etc., or race conditions in accessing shared variables among different NF threads; b) the behavior of the network connecting them, as well, e.g., the order of packet arrival and subsequent processing either due to random packets being lost, dropped or marked for ECN [9].
For example, rate limiter NFs that restrict the maximum number of connections for different clients may end up rejecting or terminating different connections either due to a race condition among NF threads accessing and updating the shared connection counter variable or due to ordering of packet processing within the NF itself. Similarly, a load balancer NF that assigns each TCP connection to one server from a pool of backend servers may end up choosing different backend servers for the same flow across different NF instances when the backend server selection logic is based on system specific calls like random(). This also impacts the state at the external clients and may potentially disrupt network services as the clients may end up losing the connection/session state maintained at their respective ends. Thus, ND further complicates the state management of NFs when providing resiliency.
Recent work, such as StatelessNF [5] and CHC [6] have proposed an alternative approach to manage NF state by decoupling the NF state from the NF processing instance and externalizing that state to an in-memory database like Redis [10], RAMCloud [11] etc. as shown in Fig. 2 (b). Note: The Externalized data store is decoupled from the NF instance and can be implemented and run as a single/cluster of processes or containers on the same or different compute nodes. Any state access operations from the NF instance to the externalized state (e.g., add/read/write/modify/delete) require inter-node or interprocess communication e.g., remote procedure call, HTTP/TCP socket etc., based on the location of the externalized state. To avoid the communication overheads, most of the research works [5], [6] consider an in-memory data store that allows partitioning and storing of the database on the DRAM of the local node (e.g., Redis client process). Despite, it should be noted that with user space NFs, the access to state through the database APIs typically incurs the overhead of an additional context switch to kernel space. Further, just as with stateful NFs, the externalized state can also be cached locally within the NF instance that can exploit the local cache and alleviate communication overheads by allowing the processing of a large batch of packets.
A key advantage with externalizing state is that when any NF instance fails, the state in the externalized data store is not impacted, and is still available for the replica NF to seamlessly failover, which only requires the flows to be redirected to a replica NF. e.g., When the rate limiter NF fails, the flows from the IDS-1 and IDS-2 can be redirected to the new instance of the rate-limiter as shown in Fig. 1 without the need to re-synchronize and update the state at the replica instance. The approaches using stateless NFs with externalized state are better suited for a Microservices architecture [12]. Stateless NFs allow seamless scaling of NF instances and failover to different NF instances, without having the state replicated to a distinct replica that is selected a priori. In addition, since the state is decoupled from the NF instance, asymmetric routing of packets from different flows which may be common in multipath routing [13] can be supported as welle.g., in Fig. 1, packets of flows processed at NF instances (FW-1, IDS-1) in the service chain may instead be processed by NF instances FW-2 and IDS-2 without any loss of information 2 .
Architecturally having externalized state for NFs might seem a step in the right direction, but the challenges in terms of performance, addressing non-determinism and complexity of operation need to be carefully analyzed including the overhead of employing them for high speed packet processing NFs has to be empirically determined (as we do in this paper).
Elastic scaling is the ability to adapt NF instances to changing application traffic patterns with automatic scale-out (add) or scale-in (remove) of the NFs. With traditional stateful NFs, when the NFs are elastically scaled, it is essential to first update the associated state for the portion of the traffic that is distributed across these new NF instances before updating the route for the flows. Works such as [14], [8] ensure the corresponding NF state is replicated before migrating flows to the new instance. However, with stateless NFs (with no caching) e.g., as in [5], flows can be instantly redistributed without the need to worry about any state update. But, cache-based stateless NFs [6] require the cached state to be flushed out and synchronized to an externalized data store before routing the traffic to the new NF instances. We analyze the penalty for such state migration for the stateful and stateless (with cache) approaches. Further, with NF chains, it is necessary to ensure the flow's affinity for an NF instance (upstream NFs in the chain) is maintained when any NF in the chain is elastically scaled. e.g., in Fig. 1, when the FW and IDS NFs are scaled to a new FW-2 and IDS-2 instances, FW-2 needs to ensure that flows that have their state at IDS-1 are routed towards IDS-1 and only the flows that have their state migrated to new IDS instance are routed and served at IDS-2.
Unlike elastic scaling, providing resiliency for an NF or NF chain failure requires more careful consideration of NF state management. Accordingly, earlier works have distinguished two approaches with 'Active:Standby' mode of operation viz. i) checkpoint only [8], [2] and ii) replay based approaches [3], [6], [4]. In either case, it is necessary to setup a standby replica NF and NF chain instances a priori, and perform periodic state updates (synchronization) on the standby instances. Although most works [2], [5], [3] address fault tolerance, only [6], [4] specifically address chain-wide failure resiliency.
Therefore, addressing elastic scaling and fault tolerance for NFs is a major challenge -the solution needs to ensure: i) consistent state updates across NF instances, since any loss of state can not only degrade performance, but can also disrupt correct operation of the network service; ii) overcome nondeterminism to ensure state consistency; and iii) have lowoverhead on normal operation to ensure high packet processing rate and low packet-processing latency. [3], [4] consider network functions to be stateful and correspondingly provide support to migrate state across different NF instances to facilitate elastic scaling and failure resiliency, while the works [5], [6] consider the NFs to be stateless having externalized the state to a datastore.
Elastic Scaling Split/Merge [8] defines state access APIs to read and update the internal state of virtualized NFs being moved across hosts. It relies on the ability to identify perflow state to provide consistent migration. FlexNFV [15] is a DPDK based framework that periodically monitors NF load on a service chain, and performs timely scaling of NFs to evenly distribute the load among available instances. In work [16], authors propose a proactive approach to scale and provision NF instances ahead of time based on the estimated flow rates using an efficient online learning method. Fault tolerance and high availability: Pico Replication [2] relies on flow-group based NF state transfers i.e., application level NF state check-pointing to address high availability for the stateful NFs. During the check-pointing, to ensure correctness, it pauses the packet processing of the flows and buffers all the input and output packets which results in significant throughput and latency overhead during the failure-free operation. On the other hand, FTMB [3] relies on packet replay and periodic (coarse-grain) check-pointing of the NF state. It logs all the input packets and the per packet access log for the shared variables in the NF that account for non-determinism, which are necessary to replay and restore the state correctly on the replica NF. However, both do not address fault tolerance for NF chains and do not provide any NF chain-wide consistent recovery. REINFORCE [4] fills this gap with an efficient chain level replication scheme which does not excessively impact the normal operation as well does not place any restrictions on replay mode to ensure correctness of the replica state.
We leverage our previous work on REIN-FORCE [4] to support stateful NFs. REINFORCE is built on OpenNetVM [7] -a DPDK based high performance NFV platform. Each NF maintains a 64KB local memory block to maintain NF state e.g., per flow state information. In addition 64MB of a shared memory block is provided to maintain the global shared state across multiple instances of the same kind of network function e.g., global counters.
Stateless NFs: We implement stateless NFs with the assistance of Redis [10] as a backend data-store. We used Redis version 2.8.4. and the latest version of Hiredis -a minimalist 'C' client library to integrate with our NFs. Since we built REINFORCE on OpenNetVM, we leverage the same base platform to build the stateless NFs as in CHC [6].
We customized and built the existing NFs (Basic monitor (BM), Vlan Tag (QoS), Load balancer (LB) and Deep Packet Inspection (DPI)) to read and export the state variables to Redis, so that core processing logic of the NFs is unchanged, and only the relevant state access operations are modified. Note, we do not change any of the dynamic memory and in-packet processing functionality. Only the static counter variables (Svs) are exported to Redis.
We tested for varying state update patterns to compare the impact of using synchronous (sync) and asynchronous (async) state update operations. Further, we enabled a local cache of variables for each of the NF state variables for both the synchronous (sync+c) and asynchronous (async+c) state updates. These state variables are updated on the backend database only after performing a batch of packet processing operations. In addition, we did not wait for acknowledgements in the case of asynchronous operations (for both async, async+c cases) so that packet processing is not stalled and can be performed concurrently with state update operations.
We experimented with several in-memory databases and narrowed our implementation to leverage Redis as the preferred datastore due to performance, tuning support for different configuration parameters, stability and 'C' plugin availability that enables easy integration with our platform. Further, in order to improve performance, we tuned the Redis configuration parameters as follows:
1. We enabled TCP keep-alives so that once the connection is setup by an NF, it is reused for the entire session, without requiring the TCP connection to be setup for every request. 2. We disabled the transparent huge pages and RDB persistence options to avoid the overhead of disk operations. 3. Also, to avoid excessive logging overheads, we set the loglevel to 'warnings-only' mode.
We use an experimental testbed consisting of five Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz servers, each with 157GB RAM, (two sockets with 28 cores each), running Ubuntu SMP Linux kernel 3.19.0-39-lowlatency. Additionally, we have a source and sink node at either end. We deploy the Redis datastore on both predecessor and primary nodes. For these experiments, nodes were connected back-to-back with dual-port 10Gbps DPDK compatible NICs to avoid any switch-induced overheads. We keep a dedicated 10Gbps link for Redis traffic, while the NFs communicate over a separate 10Gbps DPDK port. We use the DPDK-based high speed traffic generator, Moongen [17] to generate line rate traffic (14.88Mpps). We vary the traffic rate as needed for each of the different experiments. For the NF chain scenario, we deploy the entire NF chain on single (primary) node.
We compare REINFORCE with CHC [6]. We implement a simplified version of CHC, where the NF state is externalized to a Redis datastore [10]. The NFs cache the state locally and perform asynchronous state update operations after processing a batch of 256 packets. All our experiments with REINFORCE use a small batch size of 32 packets. However for CHC, we set batch size to 256 packets, as the smaller batch (32) limited CHC's throughput to less than 3Mpps 3 .
We experimented with standard benchmark tools available with the Redis [10] Aerospike [18] in-memory cluster databases that can be used to externalize NF state. With Redis, we observed that we could achieve a maximum of 1.65 million read and 1.32 million writes transactions per second (tps) respectively on a single node, for transferring 8 bytes of data each time. Note: For highest performance, we tuned the 'parallel connections' and 'pipeline (in-flight requests)' parameters, and set the parallel connections to 200 and pipeline (in-flight requests) to 256. In fact, the default Redis parameters (parallel connections = 50, pipeline = 1) result in less than 100K read/write operations per second. With Aerospike, we observed 350K read and 370K write tps respectively. We chose Redis because it was easy to integrate using the Hiredis 'C' plugin with our NFV platform.
First, we profile the impact on throughput for using the external state store in both synchronous and asynchronous updates. We use a simple forwarder NF with a packet counter variable exported to the externalized state store as a toy NF, to demonstrate the impact of synchronous and asynchronous modes of state update, with local caches, for different batch sizes. The state update to the data store is carried out after processing a batch of packets.
From Figure 3a, we observe that with no caching, where the state variable is updated to the datastore after processing each and every single packet (i.e., with batch size=1, update operations are effectively operating with no local cache), both synchronous and asynchronous modes results in very low throughput (0.01 and 0.5Mpps respectively). Although, this mode ensures strict correctness, it incurs a very high penalty on throughput and latency. We gradually increase the batch size for updating the datastore. Here, variables are cached locally and are updated to the datastore only after processing a fixed batch of packets. We observe that even with a very large batch size of 2048, synchronous state updates can at-best achieve a performance of 6Mpps, while the asynchronous state updates (without waiting for acknowledgements) can achieve line rate throughput (14.88Mpps) for batch sizes above 1024. However, asynchronous state updates require the platform to take additional measures to ensure correct state recovery in the event of failures. This includes, at the very least during normal operation, tracking of failed updates (based on the acknowledgements), and ensure correct versioning of the states to be maintained both at the datastore and the local cache of the NF. Further, as suggested by CHC [6], to achieve correct recovery in the event of failures, it is necessary to keep track of the previous version of the state variables in the datastore along with additional metadata (last NF, and the packet responsible for state update) that can help suppress any duplicate updates. Figure 3b shows the latency profile for the baseline, REIN-FORCE and CHC with synchronous and asynchronous modes of operation with the local cache, and a state update batch size of 256. The round-trip latency for CHC with synchronous mode incurs high penalty, while the asynchronous mode is only marginally better than REINFORCE.
With stateful NFs, elastic scaling requires to synchronize/transfer state to the newly instantiated NFs. Hence, we profile and update 8 bytes of data achieves a maximum of 1.65 million read and 1.32 million write transactions per second (Tps) on a single node. The overhead for using in-memory databases comes primarily from the socket I/O (read/write system calls), that involves user-space to kernel-space context switching, and thus reduced performance for DPDK-based user-space NFs.
Further, a major challenge arises in ensuring the correctness and consistency of the externalized state with respect to failed NFs that might have partially processed the packets and updated the state locally or synced only a portion of the state updates to the database, before crashing. In such scenarios, both the NFs and the database need to maintain additional version control for each state update, so that state updates can be validated before being committed. This would further reduce the NF processing capacity [6]. Moreover, with the Stateless NF approach there is a need to instrument and refactor the NF code to externalize the NF state. It requires all the internal NF state entities to be expressed in a welldefined key-value store mode. While this may be easily dealt with for per-flow state, it can be difficult to express shared persession state as well as internal state variables in this manner. Also, typically NFs allocate and release memory dynamically (via alloc and free callback functions as in nDPI). Although ephemeral, these states may also need to be externalized to ensure operational correctness, which can result in significant state update overhead. On the other hand, the complexity of supporting non-determinism and chain-wide correctness in both the stateful and stateless NFs is non-trivial. To ensure correctness, stateless NFs would require additional support from the externalized databases to provide version control and roll back of the committed state.
Stateless NFs with externalized state is promising, decoupling state and processing for NFV. However, the performance challenges with externalized data stores suggest that they need to be adopted with care for high speed packet processing NFs.
VII. CONCLUSION In this paper, we analyzed two NF state management (i.e., traditional in-memory stateful NFs and stateless NFs) approaches that have been proposed for addressing elastic scaling and fault tolerance. Ensuring correctness and consistent state update and recovery for NF chains face similar challenges (addressing non-determinism and chain-wide consistency) for both state management approaches. Stateless NFs, although promising, fall short of achieving line-rate packet processing capabilities and stateful NFs offer much higher performance and correctness under non-deterministic packet processing. Advancements in userspace in-memory databases and persistent storage can continue to help externalizing state for specialized applications contexts, especially when NF processing is entirely deterministic.
This is based on the assumption that all the state is externalized, and no state is cached locally within the NFs
Authorized licensed use limited to: The George Washington University. Downloaded on August 31,2020 at 19:37:18 UTC from IEEE Xplore. Restrictions apply.
Note: Our results (throughput) for CHC are better than those presented in the CHC paper[6], and the results may depend on the actual CHC implementation and its optimized datastore.