More Like this

1. Subscription Models for Differential Access to Real-time Information

   Neupane, A.; Pandey, V.; Park, H. (January 2023, INFORMS Transportation and Logistics Society Second Triennial Conference)

   Traffic systems exhibit supply-side uncertainty which is alleviated through real-time information. This article explores subscription models for a private agency sharing data at a fixed rate. A multiclass strategy-based equilibrium model is developed for two classes of subscribed and unsubscribed travelers, whose optimal strategy given the link-state costs is modeled as a Markov decision process (MDP) and a partially-observable MDP, respectively. A utility-based subscription choice model is formulated to study the impacts of subscription rates on the percentage of travelers choosing to subscribe. Solutions to the fixed-point formulation are determined using iterative algorithms. The proposed subscription model can be used for designing optimal subscription rates in various settings where real-time information can be a valuable routing tool such as express lanes, parking systems, roadside delivery, and routing of vulnerable road users. 

   more » « less
2. A reinforcement learning approach for user-optimal parking searching strategy on a network exploiting network topology

   Xiao, J; Lou, Y (January 2019, 2019 Transportation Annual Conference)

   This paper investigates the idea of introducing learning algorithms into parking guidance and information systems that employ a central server, in order to provide estimated optimal parking searching strategies to travelers. The parking searching process on a network with uncertain parking availability can naturally be modeled as a Markov Decision Process (MDP). Such an MDP with full information can easily be solved by dynamic programming approaches. However, the probabilities of finding parking are difficult to define and calculate, even with accurate occupancy data. Learning algorithms are suitable for addressing this issue. The central server collects data from numerous travelers’ parking search experiences in the same area within a time window, computes approximated optimal parking searching strategy using a learning algorithm, and distributes the strategy to travelers. We propose an algorithm based on Q-learning, where the topology of the underlying transportation network is incorporated. This modification allows us to reduce the size of the problem dramatically, and thus the amount of data required to learn the optimal strategy. Numerical experiments conducted on a toy network show that the proposed learning algorithm outperforms the nearest-node greedy search strategy and the original Q-learning algorithm. Sensitivity analysis regarding the desired amount of training data is also performed. 

   more » « less
3. A traveler-centric mobility game: Efficiency and stability under rationality and prospect theory

   https://doi.org/10.1371/journal.pone.0285322  

   Chremos, I.V.; Malikopoulos, A.A. (May 2023, PloS one)

   In this paper, we study a routing and travel-mode choice problem for mobility systems with a multimodal transportation network as a “mobility game” with coupled action sets. We formulate an atomic routing game to focus on the travelers’ preferences and study the impact on the efficiency of the travelers’ behavioral decision-making under rationality and prospect theory. To control the innate inefficiencies, we introduce a mobility “pricing mechanism,” in which we model traffic congestion using linear cost functions while also considering the waiting times at different transport hubs. We show that the travelers’ selfish actions lead to a pure-strategy Nash equilibrium. We then perform a Price of Anarchy and Price of Stability analysis to establish that the mobility system’s inefficiencies remain relatively low and the social welfare at a NE remains close to the social optimum as the number of travelers increases. We deviate from the standard game-theoretic analysis of decision-making by extending our mobility game to capture the subjective behavior of travelers using prospect theory. Finally, we provide a detailed discussion of implementing our proposed mobility game. 

   more » « less
4. AoI-optimal Scheduling for Arbitrary K-channel Update-Through-Queue Systems

   Lee, Won_Jun; Wang, Chih-Chun (July 2024, IEEE)

   This work generalizes the Age-of-Information (AoI) minimization problem of update-through-queue systems such that in addition to deciding the waiting time, the sender also chooses over which “channel” each update packet will be served. Different channels have different costs, delays, and quality characteristics that reflect the scheduler’s selections of routing, communications, and update modes. Instead of considering only two channels with restricted parameters as in the existing works, this work studies the general K-channel problem with arbitrary parameters. The results show that both the optimal waiting time and the optimal channel-selection policies admit an elegant water-filling structure, and can be efficiently computed by the proposed low-complexity fixed-point-based numerical method. 

   more » « less
5. A Deep Reinforcement Learning Based Approach for Home Energy Management System

   https://doi.org/10.1109/ISGT45199.2020.9087647  

   Li, Hepeng; Wan, Zhiqiang; He, Haibo (February 2020, IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT 2020))

   Home energy management system (HEMS) enables residents to actively participate in demand response (DR) programs. It can autonomously optimize the electricity usage of home appliances to reduce the electricity cost based on time-varying electricity prices. However, due to the existence of randomness in the pricing process of the utility and resident's activities, developing an efficient HEMS is challenging. To address this issue, we propose a novel home energy management method for optimal scheduling of different kinds of home appliances based on deep reinforcement learning (DRL). Specifically, we formulate the home energy management problem as an MDP considering the randomness of real-time electricity prices and resident's activities. A DRL approach based on proximal policy optimization (PPO) is developed to determine the optimal DR scheduling strategy. The proposed approach does not need any information on the appliances' models and distribution knowledge of the randomness. Simulation results verify the effectiveness of our proposed approach. 

   more » « less