skip to main content

Explore Research Products in the NSF-PAR

This content will become publicly available on May 1, 2024

Title: P2P Energy Trading through Prospect Theory, Differential Evolution, and Reinforcement Learning
Peer-to-peer (P2P) energy trading is a decentralized energy market where local energy prosumers act as peers, trading energy among each other. Existing works in this area largely overlook the importance of user behavioral modeling, assume users’ sustained active participation, and full compliance in the decision-making process. To overcome these unrealistic assumptions, and their deleterious consequences, in this paper we propose an automated P2P energy trading framework that specifically considers the users’ perception by exploiting prospect theory . We formalize an optimization problem that maximizes the buyers’ perceived utility while matching energy production and demand. We prove that the problem is NP-hard and we propose a Differential Evolution-based Algorithm for Trading Energy ( DEbATE ) heuristic. Additionally, we propose two automated pricing solutions to improve the sellers’ profit based on reinforcement learning. The first solution, named Pricing mechanism with Q-learning and Risk-sensitivity ( PQR ), is based on Q-learning. Additionally, the given scalability issues of PQR , we propose a Deep Q-Network-based algorithm called ProDQN that exploits deep learning and a novel loss function rooted in prospect theory. Results based on real traces of energy consumption and production, as well as realistic prospect theory functions, show that our approaches achieve \(26\% \) higher perceived value for buyers and generate \(7\% \) more reward for sellers, compared to recent state-of-the-art approaches.  more » « less
Award ID(s):
1936131 1943035
NSF-PAR ID:
10463458
Author(s) / Creator(s):
;
Date Published:
Journal Name:
ACM Transactions on Evolutionary Learning and Optimization
ISSN:
2688-299X
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ( , Energies)
    null (Ed.)
    Peer-to-peer energy trading within microgrid (MG) communities emerges as a key enabler of the future transactive distribution system and the transactive electricity market. Energy trading within MGs refers to the idea that the surplus energy of one MG can be used to satisfy the demand of another MG or a group of MGs that form an MG community. These communities can be dynamically established through time, based on the variations of demand and supply of the interconnected MGs. In many modern MGs, Electric Vehicles (EVs) have been considered as a viable storage option due to their ease of use (plug-and-play) and their growing adoption rates by drivers. On the other hand, the dynamic nature of EVs escalates the uncertainty in the transactive distribution system. In this paper, we study the problem of energy trading among MGs and EVs with the aim of power loss minimization where there is uncertainty. We propose a novel Bayesian Coalition Game (BCG) based algorithm, which allows the MGs and EVs to reduce the overall power loss by allowing them to form coalitions intelligently. The proposed scheme is compared with a conventional coalitional game theory-based approach and a Q-learning based approach. Our results show significant improvement over other compared techniques. 
    more » « less
  2. ; ; ( , 2019 IEEE Global Communications Conference (GLOBECOM))
    In this paper, a flexible resource sharing paradigm is introduced, to enable the allocation of users’ computing tasks in a social cloud computing system offering both Virtual Machines (VMs) and Serverless Computing (SC) functions. VMs are treated as a safe computing resource, while SC due to the uncertainty introduced by its shared nature, is treated as a common pool resource, being susceptible to potential over-exploitation. These computing options are differentiated based on the potential satisfaction perceived by the user, as well as their corresponding pricing, while taking into account the social interactions among the users. Considering the inherent uncertainty of the considered computing environment, Prospect Theory and the theory of the Tragedy of the Commons are adopted to properly reflect the users’ behavioral characteristics, i.e., gain-seeking or loss-averse behavior, as well as to formulate appropriate prospect-theoretic utility functions, embodying the social-aware and risk-aware user’s perceived satisfaction. A distributed maximization problem of each user’s expected prospect-theoretic utility is formulated as a non-cooperative game among the users and the corresponding Pure Nash Equilibrium (PNE), i.e., optimal computing jobs offloading to the VMs and the SC, is determined, while a distributed low-complexity algorithm that converges to the PNE is introduced. The performance and key principles of the proposed framework are demonstrated through modeling and simulation. 
    more » « less
  3. ; ; ( , Sensors)
    null (Ed.)
    Unmanned Aerial Vehicle (UAV)-assisted Multi-access Edge Computing (MEC) systems have emerged recently as a flexible and dynamic computing environment, providing task offloading service to the users. In order for such a paradigm to be viable, the operator of a UAV-mounted MEC server should enjoy some form of profit by offering its computing capabilities to the end users. To deal with this issue in this paper, we apply a usage-based pricing policy for allowing the exploitation of the servers’ computing resources. The proposed pricing mechanism implicitly introduces a more social behavior to the users with respect to competing for the UAV-mounted MEC servers’ computation resources. In order to properly model the users’ risk-aware behavior within the overall data offloading decision-making process the principles of Prospect Theory are adopted, while the exploitation of the available computation resources is considered based on the theory of the Tragedy of the Commons. Initially, the user’s prospect-theoretic utility function is formulated by quantifying the user’s risk seeking and loss aversion behavior, while taking into account the pricing mechanism. Accordingly, the users’ pricing and risk-aware data offloading problem is formulated as a distributed maximization problem of each user’s expected prospect-theoretic utility function and addressed as a non-cooperative game among the users. The existence of a Pure Nash Equilibrium (PNE) for the formulated non-cooperative game is shown based on the theory of submodular games. An iterative and distributed algorithm is introduced which converges to the PNE, following the learning rule of the best response dynamics. The performance evaluation of the proposed approach is achieved via modeling and simulation, and detailed numerical results are presented highlighting its key operation features and benefits. 
    more » « less
  4. ( , Proceedings ACMSIGMOD International Conference on Management of Data)
    The growing adoption of data analytics platforms and machine learning-based solutions for decision-makers creates a significant demand for datasets, which explains the appearance of data markets. In a well-functioning data market, sellers share data in exchange for money, and buyers pay for datasets that help them solve problems. The market raises sufficient money to compensate sellers and incentivize them to keep sharing datasets. This low-friction matching of sellers and buyers distributes the value of data among participants. But designing online data markets is challenging because they must account for the strategic behavior of participants. In this paper, we introduce techniques to protect data markets from strategic participants, even when the asset traded is data. We combine those techniques into a pricing algorithm specifically designed to trade data. The evaluation includes a user study and extensive simulations. Together, the evaluation demonstrates how participants strategize and the effectiveness of our techniques. 
    more » « less
  5. ; ; ; ; ; ( , IEEE Journal on Selected Areas in Communications)
    As distributed energy resources (DERs) are widely deployed, DC packetized power microgrids have been considered as a promising solution to incorporate DERs effectively and steadily. In this paper, we consider a DC packetized power microgrid, where the energy is dispatched in the form of power packets with the assist of a power router. However, the benefits of the microgrid can only be realized when energy subscribers (ESs) equipped with DERs actively participate in the energy market. Therefore, peer-to-peer (P2P) energy trading is necessary in the DC packetized power microgrid to encourage the usage of DERs. Different from P2P energy trading in AC microgrids, the dispatching capability of the router needs to be considered in DC microgrids, which will complicate the trading problem. To tackle this challenge, we formulate the P2P trading problem as an auction game, in which the demander ESs submit bids to compete for power packets, and a controller decides the energy allocation and power packet scheduling. Analysis of the proposed scheme is provided, and its effectiveness is validated through simulation. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email