An official website of the United States government
Tiered distributed computing systems, where components run in Internet-of-things devices, in edge computers, and in the cloud, introduce unique difficulties in maintaining consistency of shared data while ensuring availability. A major source of difficulty is the highly variable network latencies that applications must deal with. It is well known in distributed computing that when network latencies rise sufficiently, one or both of consistency and availability must be sacrificed. This paper quantifies consistency and availability and gives an algebraic relationship between these quantities and network latencies. The algebraic relation is linear in a max-plus algebra and supports heterogeneous networks, where the communication latency between 2 components may differ from the latency between another 2 components. We show how to make use of this algebraic relation to guide design, enabling software designers to specify consistency and availability requirements, and to derive from those the requirements on network latencies. We show how to design systems to fail in predictable ways when the network latency requirements are violated, by choosing to sacrifice either consistency or availability.
more »
« less
Mapping Study on Constraint Consistency Checking in Distributed Enterprise Systems
Constraint consistency errors in distributed systems can lead to fatal consequences when left unobserved and undetected. The primary goal of quality engineers should be to avoid system inconsistencies in general. However, it is typically a much more straight forward process in monolith-like systems with one codebase than in distributed solutions where heterogeneity occurs across modules. In this paper, we raise the research question of what is the existing state-of-the-art and research literature practice when it comes to consistency checking in distributed systems. We conducted a systematic search for existing work and assess the evidence to categorize the approaches and to identify used techniques. Identified works offer interesting directions and achievements. Often the works share tool prototypes and instruments to build on the top of when performing further research in this direction and we share them in this paper. Finally, we discuss open challenges and gaps in this field to promote the interest of the research audience.
more »
« less
- Award ID(s):
- 1854049
- PAR ID:
- 10203753
- Date Published:
- Journal Name:
- RACS '20: Proceedings of the International Conference on Research in Adaptive and Convergent Systems
- Page Range / eLocation ID:
- 167 to 174
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)This paper shows how to use bounded-time recovery (BTR) to defend distributed systems against non-crash faults and attacks. Unlike many existing fault-tolerance techniques, BTR does not attempt to completely mask all symptoms of a fault; instead, it ensures that the system returns to the correct behavior within a bounded amount of time. This weaker guarantee is sufficient, e.g., for many cyber-physical systems, where physical properties - such as inertia and thermal capacity - prevent quick state changes and thus limit the damage that can result from a brief period of undefined behavior. We present an algorithm called REBOUND that can provide BTR for the Byzantine fault model. REBOUND works by detecting faults and then reconfiguring the system to exclude the faulty nodes. This supports very fine-grained responses to faults: for instance, the system can move or replace existing tasks, or drop less critical tasks entirely to conserve resources. REBOUND can take useful actions even when a majority of the nodes is compromised, and it requires less redundancy than full fault-tolerance.more » « less
-
Despite the increased accuracy of intrusion detection systems (IDS) in identifying cyberattacks in computer networks and devices connected to the internet, distributed or coordinated attacks can still go undetected or not detected on time. The single vantage point limits the ability of these IDSs to detect such attacks. Due to this reason, there is a need for attack characteristics’ exchange among different IDS nodes. Researchers proposed a cooperative intrusion detection system to share these attack characteristics effectively. This approach was useful; however, the security of the shared data cannot be guaranteed. More specifically, maintaining the integrity and consistency of shared data becomes a significant concern. In this paper, we propose a blockchain-based solution that ensures the integrity and consistency of attack characteristics shared in a cooperative intrusion detection system. The proposed architecture achieves this by detecting and preventing fake features injection and compromised IDS nodes. It also facilitates scalable attack features exchange among IDS nodes, ensures heterogeneous IDS nodes participation, and it is robust to public IDS nodes joining and leaving the network. We evaluate the security analysis and latency. The result shows that the proposed approach detects and prevents compromised IDS nodes, malicious features injection, manipulation, or deletion, and it is also scalable with low latency.more » « less
-
null (Ed.)Under the linear regression framework, we study the variable selection problem when the underlying model is assumed to have a small number of nonzero coefficients. Non-convex penalties in speci c forms are well-studied in the literature for sparse estimation. A recent work, Ahn, Pang, and Xin (2017), has pointed out that nearly all existing non-convex penalties can be represented as difference-of-convex (DC) functions, which are the difference of two convex functions, while itself may not be convex. There is a large existing literature on optimization problems when their objectives and/or constraints involve DC functions. Efficient numerical solutions have been proposed. Under the DC framework, directional-stationary (d-stationary) solutions are considered, and they are usually not unique. In this paper, we show that under some mild conditions, a certain subset of d-stationary solutions in an optimization problem (with a DC objective) has some ideal statistical properties: namely, asymptotic estimation consistency, asymptotic model selection consistency, asymptotic efficiency. Our assumptions are either weaker than or comparable with those conditions that have been adopted in other existing works. This work shows that DC is a nice framework to offer a uni ed approach to these existing works where non-convex penalties are involved. Our work bridges the communities of optimization and statistics.more » « less
-
Data centers are increasingly equipped with RDMAs. These network interfaces mark the advent of a new distributed system model where a node can directly access the remote memory of another. They have enabled microsecond-scale replicated services. The underlying replication protocols of these systems execute all operations under strong consistency. However, strong consistency can hinder response time and availability, and recent replication models have turned to a hybrid of strong and relaxed consistency. This paper presents RDMA replicated data types, the first hybrid replicated data types for the RDMA network model. It presents a novel operational semantics for these types that considers three distinct categories of methods and captures their re- quired coordination, and formally proves that they preserve convergence and integrity. It implements these semantics in a system called Hamband that leverages direct remote accesses to efficiently implement the required coordination protocols. The empirical evaluation shows that Hamband outperforms the throughput of existing message-based and SMR-based implementations by more than 4x.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3400286.3418257
