skip to main content


Title: Database and Distributed Computing Foundations of Blockchains
The uprise of Bitcoin and other peer-to-peer cryptocurrencies has opened many interesting and challenging problems in cryptography, distributed systems, and databases. The main underlying data structure is blockchain, a scalable fully replicated structure that is shared among all participants and guarantees a consistent view of all user transactions by all participants in the system. In this tutorial, we discuss the basic protocols used in blockchain, and elaborate on its main advantages and limitations. To overcome these limitations, we provide the necessary distributed systems background in managing large scale fully replicated ledgers, using Byzantine Agreement protocols to solve the consensus problem. Finally, we expound on some of the most recent proposals to design scalable and efficient blockchains in both permissionless and permissioned settings. The focus of the tutorial is on the distributed systems and database aspects of the recent innovations in blockchains  more » « less
Award ID(s):
1815733 1703560
NSF-PAR ID:
10113700
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
ACM SIGMOD International Conference on Management of Data
Page Range / eLocation ID:
2036 to 2041
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Darmont, J ; Novikov, B. ; Wrembel, R. (Ed.)
    Bitcoin [12] is a successful and interesting example of a global scale peer-to-peer cryptocurrency that integrates many techniques and protocols from cryptography, distributed systems, and databases. The main underlying data structure is blockchain, a scalable fully replicated structure that is shared among all participants and guarantees a consistent view of all user transactions by all participants in the system. In a blockchain, nodes agree on their shared states across a large network of untrusted participants. Although originally devised for cryptocurrencies, recent systems exploit its many unique features such as transparency, provenance, fault tolerance, and authenticity to support a wide range of distributed applications. Bitcoin and other cryptocurrencies use permissionless blockchains. In a permissionless blockchain, the network is public, and anyone can participate without a specific identity. Many other distributed applications, such as supply chain management and healthcare, are deployed on permissioned blockchains consisting of a set of known, identified nodes that still might not fully trust each other. This paper illustrates some of the main challenges and opportunities from a database perspective in the many novel and interesting application domains of blockchains. These opportunities are illustrated using various examples from recent research in both permissionless and permissioned blockchains. Two main themes unite the various examples: (1) the important role of distribution and consensus in managing large scale systems and (2) the need to tolerate malicious failures. The advent of cloud computing and large data centers shifted large scale data management infrastructures from centralized databases to distributed systems. One of the main challenges in designing distributed systems is the need for fault-tolerance. Cloud-based systems typically assume trusted infrastructures, since data centers are owned by the enterprises managing the data, and hence the design typically only assumes and tolerates crash failures. The advent of blockchain and the underlying premise that copies of the blockchain are distributed among untrusted entities has shifted the focus of fault-tolerance from tolerating crash failures to tolerating malicious failures. These interesting and challenging settings pose great opportunities for database researchers. 
    more » « less
  2. The unique features of blockchains such as immutability, transparency, provenance, and authenticity have been used by many large-scale data management systems to deploy a wide range of distributed applications including supply chain management, healthcare, and crowdworking in permissioned settings. Unlike permissionless settings, e.g., Bitcoin, where the network is public, and anyone can participate without a specific identity, a permissioned blockchain system consists of a set of known, identified nodes that might not fully trust each other. While the characteristics of permissioned blockchains are appealing to a wide range of largescale data management systems, these systems, have to satisfy four main requirements: confidentiality, verifiability, performance, and scalability. Various approaches have been developed in industry and academia to satisfy these requirements with varying assumptions and costs. The focus of this tutorial is on presenting many of these techniques while highlighting the trade-offs among them. We demonstrate the practicality of such techniques in real-life by presenting three different applications, i.e., supply chain management, large-scale databases, and multi-platform crowdworking environments, and show how those techniques can be utilized to meet the requirements of such applications 
    more » « less
  3. Public blockchains have spurred the growing popularity of decentralized transactions and smart contracts, especially on the financial market. However, public blockchains exhibit their limitations on the transaction throughput, storage availability, and compute capacity. To avoid transaction gridlock, public blockchains impose large fees and per-block resource limits, making it difficult to accommodate the ever-growing high transaction demand. Previous research endeavors to improve the scalability and performance of blockchain through various technologies, such as side-chaining, sharding, secured off-chain computation, communication network optimizations, and efficient consensus protocols. However, these approaches have not attained a widespread adoption due to their inability in delivering a cloud-like performance, in terms of the scalability in transaction throughput, storage, and compute capacity. In this work, we determine that the major obstacle to public blockchain scalability is their underlying unstructured P2P networks. We further show that a centralized network can support the deployment of decentralized smart contracts. We propose a novel approach for achieving scalable decentralization: instead of trying to make blockchain scalable, we deliver decentralization to already scalable cloud by using an Ethereum smart contract. We introduce Blockumulus, a framework that can deploy decentralized cloud smart contract environments using a novel technique called overlay consensus. Through experiments, we demonstrate that Blockumulus is scalable in all three dimensions: computation, data storage, and transaction throughput. Besides eliminating the current code execution and storage restrictions, Blockumulus delivers a transaction latency between 2 and 5 seconds under normal load. Moreover, the stress test of our prototype reveals the ability to execute 20,000 simultaneous transactions under 26 seconds, which is on par with the average throughput of worldwide credit card transactions. 
    more » « less
  4. null (Ed.)
    The current centralized model of the electricity market is not efficient in performing distributed energy transactions required for the transactive smart grid. One of the prominent solutions to this issue is to integrate blockchain technologies, which promise transparent, tamper-proof, and secure transaction systems specifically suitable for the decentralized and distributed energy markets. Blockchain has already been shown to successfully operate in a microgrid peer-to-peer (P2P) energy market. The prime determinant of different blockchain implementations is the consensus algorithm they use to reach consensus on which blocks/transactions to accept as valid in a distributed environment. Although different blockchain implementations have been proposed independently for P2P energy market in the microgrid, quantitative experimental analyses and comparison of the consensus algorithms that the different blockchains may use for energy markets, has not been studied. Identifying the right consensus algorithm to use is essential for scalability and operation of the energy market. To this end, we evaluate three popular consensus algorithms: (i) proof of work (PoW), (ii) proof of authority (PoA), and (iii) Istanbul Byzantine fault tolerance (IBFT), running them on a network of nodes set up using a network of docker nodes to form a microgrid energy market. Using a series of double auctions, we assess each algorithm's viability using different metrics, such as time to reach consensus and scalability. The results indicate that PoA is the most efficient and scalable consensus algorithm to hold double auctions in the smart grid. We also identified the minimum hardware specification necessary for devices such as smart meters, which may run these consensus algorithms 
    more » « less
  5. The current centralized model of the electricity market is not efficient in performing distributed energy transactions required for the transactive smart grid. One of the prominent solutions to this issue is to integrate blockchain technologies, which promise transparent, tamper-proof, and secure transaction systems specifically suitable for the decentralized and distributed energy markets. Blockchain has already been shown to successfully operate in a microgrid peer-to-peer (P2P) energy market. The prime determinant of different blockchain implementations is the consensus algorithm they use to reach consensus on which blocks/transactions to accept as valid in a distributed environment. Although different blockchain implementations have been proposed independently for P2P energy market in the microgrid, quantitative experimental analyses and comparison of the consensus algorithms that the different blockchains may use for energy markets, has not been studied. Identifying the right consensus algorithm to use is essential for scalability and operation of the energy market. To this end, we evaluate three popular consensus algorithms: (i) proof of work (PoW), (ii) proof of authority (PoA), and (iii) Istanbul Byzantine fault tolerance (IBFT), running them on a network of nodes set up using a network of docker nodes to form a microgrid energy market. Using a series of double auctions, we assess each algorithm’s viability using different metrics, such as time to reach consensus and scalability. The results indicate that PoA is the most efficient and scalable consensus algorithm to hold double auctions in the smart grid. We also identified the minimum hardware specification necessary for devices such as smart meters, which may run these consensus algorithms. 
    more » « less