The fast development of electric vehicles (EV) and EV chargers introduces many factors that affect the grid. EV charging and charge scheduling also bring challenges to EV drivers and grid operators. In this work, we propose a human-centric, data-driven, city-scale, multivariate optimization approach for the EV-interfaced grid. This approach takes into account user historical driving and charging habits, user preferences, EV characteristics, city-scale mobility, EV charger availability and price, and grid capacity. The user preferences include the trade-off between cost and time to charge, as well as incentives to participate in different energy-saving programs. We leverage deep reinforcement learning (DRL) to make recommendations to EV drivers and optimize their welfare while enhancing grid performance.
more »
« less
This content will become publicly available on May 1, 2025
Reconfigurable-Intelligent-Surface-Enhanced Dynamic Resource Allocation for the Social Internet of Electric Vehicle Charging Networks with Causal-Structure-Based Reinforcement Learning
Charging stations and electric vehicle (EV) charging networks signify a significant advancement in technology as a frontier application of the Social Internet of Things (SIoT), presenting both challenges and opportunities for current 6G wireless networks. One primary challenge in this integration is limited wireless network resources, particularly when serving a large number of users within distributed EV charging networks in the SIoT. Factors such as congestion during EV travel, varying EV user preferences, and uncertainties in decision-making regarding charging station resources significantly impact system operation and network resource allocation. To address these challenges, this paper develops a novel framework harnessing the potential of emerging technologies, specifically reconfigurable intelligent surfaces (RISs) and causal-structure-enhanced asynchronous advantage actor–critic (A3C) reinforcement learning techniques. This framework aims to optimize resource allocation, thereby enhancing communication support within EV charging networks. Through the integration of RIS technology, which enables control over electromagnetic waves, and the application of causal reinforcement learning algorithms, the framework dynamically adjusts resource allocation strategies to accommodate evolving conditions in EV charging networks. An essential aspect of this framework is its ability to simultaneously meet real-world social requirements, such as ensuring efficient utilization of network resources. Numerical simulation results validate the effectiveness and adaptability of this approach in improving wireless network efficiency and enhancing user experience within the SIoT context. Through these simulations, it becomes evident that the developed framework offers promising solutions to the challenges posed by integrating the SIoT with EV charging networks.
more » « less- Award ID(s):
- 2128656
- PAR ID:
- 10541308
- Publisher / Repository:
- MDPI
- Date Published:
- Journal Name:
- Future Internet
- Volume:
- 16
- Issue:
- 5
- ISSN:
- 1999-5903
- Page Range / eLocation ID:
- 165
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
There is a growing interest in applying computational tools to the automatic discovery of social and economic behavior. For example, with decisions involving resource allocation related to public infrastructure, the ability to predict failures can allow for more efficient policy responses. In this paper, we use social data from a popular electric vehicle (EV) driver app to characterize the emerging EV charging station infrastructure. We introduce a typology of EV charging experiences collected from user reviews and deploy text classification algorithms, including convolutional neural networks (CNN), to automatically learn about potential failures. We use machine learning techniques as a pre-processing tool for econometric analyses on the quality of service delivery. After classifying the reviews into 9 main user topics and 34 subtopics, we find that the dominant issues in EV charging relate to station functionality and availability, which drive negative consumer experience. Contrary to the public discourse about EVs, range anxiety was not of large concern to existing EV drivers. Based on our findings, we move towards automated identification of failures in public charging infrastructure that can significantly reduce research evaluation costs through relatively simple computational solutions.more » « less
-
more » « less
This study investigates the problem of decentralized dynamic resource allocation optimization for ad-hoc network communication with the support of reconfigurable intelligent surfaces (RIS), leveraging a reinforcement learning framework. In the present context of cellular networks, device-to-device (D2D) communication stands out as a promising technique to enhance the spectrum efficiency. Simultaneously, RIS have gained considerable attention due to their ability to enhance the quality of dynamic wireless networks by maximizing the spectrum efficiency without increasing the power consumption. However, prevalent centralized D2D transmission schemes require global information, leading to a significant signaling overhead. Conversely, existing distributed schemes, while avoiding the need for global information, often demand frequent information exchange among D2D users, falling short of achieving global optimization. This paper introduces a framework comprising an outer loop and inner loop. In the outer loop, decentralized dynamic resource allocation optimization has been developed for self-organizing network communication aided by RIS. This is accomplished through the application of a multi-player multi-armed bandit approach, completing strategies for RIS and resource block selection. Notably, these strategies operate without requiring signal interaction during execution. Meanwhile, in the inner loop, the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm has been adopted for cooperative learning with neural networks (NNs) to obtain optimal transmit power control and RIS phase shift control for multiple users, with a specified RIS and resource block selection policy from the outer loop. Through the utilization of optimization theory, distributed optimal resource allocation can be attained as the outer and inner reinforcement learning algorithms converge over time. Finally, a series of numerical simulations are presented to validate and illustrate the effectiveness of the proposed scheme.
-
There is an increase in usage of smaller cells or femtocells to improve performance and coverage of next-generation heterogeneous wireless networks (HetNets). However, the interference caused by femtocells to neighboring cells is a limiting performance factor in dense HetNets. This interference is being managed via distributed resource allocation methods. However, as the density of the network increases so does the complexity of such resource allocation methods. Yet, unplanned deployment of femtocells requires an adaptable and self-organizing algorithm to make HetNets viable. As such, we propose to use a machine learning approach based on Q-learning to solve the resource allocation problem in such complex networks. By defining each base station as an agent, a cellular network is modeled as a multi-agent network. Subsequently, cooperative Q-learning can be applied as an efficient approach to manage the resources of a multi-agent network. Furthermore, the proposed approach considers the quality of service (QoS) for each user and fairness in the network. In comparison with prior work, the proposed approach can bring more than a four-fold increase in the number of supported femtocells while using cooperative Q-learning to reduce resource allocation overhead.more » « less
-
null (Ed.)Wireless charging coupled with computation offloading in edge networks offers a promising solution for realizing power-hungry and computation intensive applications on user devices. We consider a mutil-access edge computing (MEC) system with collocated MEC servers and base-stations/access points (BS/AP) supporting multiple users requesting data computation and wireless charging. We propose an integrated solution for wireless charging with computation offloading to satisfy the largest feasible proportion of requested wireless charging while keeping the total energy consumption at the minimum, subject to the MEC-AP transmit power and latency constraints. We propose a novel nested algorithm to jointly perform data partitioning, time allocation, transmit power control and design the optimal energy beamforming for wireless charging. Our resource allocation scheme offers a minimal energy consumption solution compared to other schemes while also delivering a higher amount of wirelessly transferred charge to the users. Even with data offloading, our proposed solution shows significant charging performance, comparable to the case of charging alone, hence showing the effectiveness of performing partial offloading jointly with wireless charging.more » « less
