skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: P2P Energy Trading through Prospect Theory, Differential Evolution, and Reinforcement Learning
Peer-to-peer (P2P) energy trading is a decentralized energy market where local energy prosumers act as peers, trading energy among each other. Existing works in this area largely overlook the importance of user behavioral modeling, assume users’ sustained active participation, and full compliance in the decision-making process. To overcome these unrealistic assumptions, and their deleterious consequences, in this paper we propose an automated P2P energy trading framework that specifically considers the users’ perception by exploiting prospect theory . We formalize an optimization problem that maximizes the buyers’ perceived utility while matching energy production and demand. We prove that the problem is NP-hard and we propose a Differential Evolution-based Algorithm for Trading Energy ( DEbATE ) heuristic. Additionally, we propose two automated pricing solutions to improve the sellers’ profit based on reinforcement learning. The first solution, named Pricing mechanism with Q-learning and Risk-sensitivity ( PQR ), is based on Q-learning. Additionally, the given scalability issues of PQR , we propose a Deep Q-Network-based algorithm called ProDQN that exploits deep learning and a novel loss function rooted in prospect theory. Results based on real traces of energy consumption and production, as well as realistic prospect theory functions, show that our approaches achieve \(26\% \) higher perceived value for buyers and generate \(7\% \) more reward for sellers, compared to recent state-of-the-art approaches.  more » « less
Award ID(s):
1936131 1943035
PAR ID:
10463458
Author(s) / Creator(s):
;
Date Published:
Journal Name:
ACM Transactions on Evolutionary Learning and Optimization
ISSN:
2688-299X
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; (, Proceedings of the 56th Hawaii International Conference on System Sciences)
    Utilizing distributed renewable and energy storage resources via peer-to-peer (P2P) energy trading has long been touted as a solution to improve energy system’s resilience and sustainability. Consumers and prosumers (those who have energy generation resources), however, do not have expertise to engage in repeated P2P trading, and the zero-marginal costs of renewables present challenges in determining fair market prices. To address these issues, we propose a multi-agent reinforcement learning (MARL) framework to help automate consumers’ bidding and management of their solar PV and energy storage resources, under a specific P2P clearing mechanism that utilizes the so-called supply-demand ratio. In addition, we show how the MARL framework can integrate physical network constraints to realize decentralized voltage control, hence ensuring physical feasibility of the P2P energy trading and paving ways for real-world implementations. 
    more » « less
  2. ; ; ; ; ; (, IEEE Journal on Selected Areas in Communications)
    As distributed energy resources (DERs) are widely deployed, DC packetized power microgrids have been considered as a promising solution to incorporate DERs effectively and steadily. In this paper, we consider a DC packetized power microgrid, where the energy is dispatched in the form of power packets with the assist of a power router. However, the benefits of the microgrid can only be realized when energy subscribers (ESs) equipped with DERs actively participate in the energy market. Therefore, peer-to-peer (P2P) energy trading is necessary in the DC packetized power microgrid to encourage the usage of DERs. Different from P2P energy trading in AC microgrids, the dispatching capability of the router needs to be considered in DC microgrids, which will complicate the trading problem. To tackle this challenge, we formulate the P2P trading problem as an auction game, in which the demander ESs submit bids to compete for power packets, and a controller decides the energy allocation and power packet scheduling. Analysis of the proposed scheme is provided, and its effectiveness is validated through simulation. 
    more » « less
  3. ; ; (, Energies)
    null (Ed.)
    Peer-to-peer energy trading within microgrid (MG) communities emerges as a key enabler of the future transactive distribution system and the transactive electricity market. Energy trading within MGs refers to the idea that the surplus energy of one MG can be used to satisfy the demand of another MG or a group of MGs that form an MG community. These communities can be dynamically established through time, based on the variations of demand and supply of the interconnected MGs. In many modern MGs, Electric Vehicles (EVs) have been considered as a viable storage option due to their ease of use (plug-and-play) and their growing adoption rates by drivers. On the other hand, the dynamic nature of EVs escalates the uncertainty in the transactive distribution system. In this paper, we study the problem of energy trading among MGs and EVs with the aim of power loss minimization where there is uncertainty. We propose a novel Bayesian Coalition Game (BCG) based algorithm, which allows the MGs and EVs to reduce the overall power loss by allowing them to form coalitions intelligently. The proposed scheme is compared with a conventional coalitional game theory-based approach and a Q-learning based approach. Our results show significant improvement over other compared techniques. 
    more » « less
  4. ; ; (, Sensors)
    null (Ed.)
    Unmanned Aerial Vehicle (UAV)-assisted Multi-access Edge Computing (MEC) systems have emerged recently as a flexible and dynamic computing environment, providing task offloading service to the users. In order for such a paradigm to be viable, the operator of a UAV-mounted MEC server should enjoy some form of profit by offering its computing capabilities to the end users. To deal with this issue in this paper, we apply a usage-based pricing policy for allowing the exploitation of the servers’ computing resources. The proposed pricing mechanism implicitly introduces a more social behavior to the users with respect to competing for the UAV-mounted MEC servers’ computation resources. In order to properly model the users’ risk-aware behavior within the overall data offloading decision-making process the principles of Prospect Theory are adopted, while the exploitation of the available computation resources is considered based on the theory of the Tragedy of the Commons. Initially, the user’s prospect-theoretic utility function is formulated by quantifying the user’s risk seeking and loss aversion behavior, while taking into account the pricing mechanism. Accordingly, the users’ pricing and risk-aware data offloading problem is formulated as a distributed maximization problem of each user’s expected prospect-theoretic utility function and addressed as a non-cooperative game among the users. The existence of a Pure Nash Equilibrium (PNE) for the formulated non-cooperative game is shown based on the theory of submodular games. An iterative and distributed algorithm is introduced which converges to the PNE, following the learning rule of the best response dynamics. The performance evaluation of the proposed approach is achieved via modeling and simulation, and detailed numerical results are presented highlighting its key operation features and benefits. 
    more » « less
  5. (, Proceedings ACMSIGMOD International Conference on Management of Data)
    The growing adoption of data analytics platforms and machine learning-based solutions for decision-makers creates a significant demand for datasets, which explains the appearance of data markets. In a well-functioning data market, sellers share data in exchange for money, and buyers pay for datasets that help them solve problems. The market raises sufficient money to compensate sellers and incentivize them to keep sharing datasets. This low-friction matching of sellers and buyers distributes the value of data among participants. But designing online data markets is challenging because they must account for the strategic behavior of participants. In this paper, we introduce techniques to protect data markets from strategic participants, even when the asset traded is data. We combine those techniques into a pricing algorithm specifically designed to trade data. The evaluation includes a user study and extensive simulations. Together, the evaluation demonstrates how participants strategize and the effectiveness of our techniques. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email