skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


  • NSF Public Access
  • Search Results
  • SMART-eFlo: An Integrated SUMO-Gym Framework for Multi-Agent Reinforcement Learning in Electric Fleet Management Problem
Title: SMART-eFlo: An Integrated SUMO-Gym Framework for Multi-Agent Reinforcement Learning in Electric Fleet Management Problem
Electric vehicles (EVs) have been used in the ride-hailing system in recent years, which brings the electric fleet management problem (EFMP) critical. This paper aims to leverage multi-agent reinforcement learning (MARL) in EFMP. In particular, we focus on how EVs learn to manage battery charging, pick up and drop off passengers. We propose an integrated SUMO-Gym framework based on the SUMO simulator to capture EVs’ asynchronous decisionmaking regarding charging and ride-hailing in complex traffic environments. We adopt a hierarchical reinforcement learning (HRL) scheme, where each EV decides to get charged or pick up a passenger on the upper level and chooses a charging station or passenger on the lower level. We develop a learning algorithm for the HRL scheme to solve EFMP and present numerical results about the efficiency of our algorithm and policies EVs have learned in EFMP. Our codes are available at https://github.com/LovelyBuggies/SUMO-Gym, which provides an open-source environment for researchers to design traffic scenarios and test RL algorithms for EFMP.  more » « less
Award ID(s):
2038984
PAR ID:
10447569
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC)
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; (, IEEE)
    Electrifying the ride-hailing services has the potential to significantly reduce greenhouse gas emissions in the shared mobility sector. However, these emission reduction benefits depend on the utilization of EVs to serve trip requests, especially during the fleet electrification process. In this paper, we evaluated the performance and emission impacts of ride-hailing service with three dispatching policies and various EV penetration levels in the ride-hailing fleet. A large-scale simulation platform was developed for the city of San Francisco in SUMO to enable the application of ride-hailing, electric vehicle charging, and idle vehicle repositioning. Simulation results indicate that with a 60% EVs in the simulated fleet, the off-peak EV priority policy and off-peak EV only policy can reduce CO2 emissions by 32% - 40% while preserving the mobility performance in terms of deadheading, total travel distance, and average rider pick-up time. It is important for ride-hailing platforms to increase the zero-emission rides and encourage ride pooling to comply with California’s Clean Miles Standard. 
    more » « less
  2. ; ; ; ; ; ; ; ; ; ; et al (, IEEE)
    Electric Vehicle (EV) charging has been a significant barrier to the widespread use of EVs. Traditional EV charging methods depend on cables, and there are concerns about safety, accessibility, convenience, and weather. A recent development, dynamic (or in-motion) wireless charging, enables EVs to charge wirelessly by incorporating charging infrastructure into roadways, allowing EVs to charge while moving. However, the energy transferred relies heavily on vehicle speed and time spent in the charging lane. This paper proposes an innovative solution that combines dynamic wire-less charging with Variable Speed Limit (VSL) control. This dynamic traffic control strategy adjusts speed limits based on real-time traffic, weather, and incidents. This integration of dynamic wireless charging and VSL has two potential benefits. First, it can motivate driver compliance with VSL through the incentive of charging. Second, it can promote smoother traffic flow and improve traffic safety by implementing lower speed limits at certain times. To verify these benefits, microscopic traffic simulations in SUMO were conducted under different EV penetration rates and VSL compliance rates. Simulation results reveal that the proposed approach can enhance dynamic wireless charging system performance while improving traffic flow and safety 
    more » « less
  3. ; ; (, International Regional Science Review)
    Advances in information technologies and vehicle automation have birthed new transportation services, including shared autonomous vehicles (SAVs). Shared autonomous vehicles are on-demand self-driving taxis, with flexible routes and schedules, able to replace personal vehicles for many trips in the near future. The siting and density of pick-up and drop-off (PUDO) points for SAVs, much like bus stops, can be key in planning SAV fleet operations, since PUDOs impact SAV demand, route choices, passenger wait times, and network congestion. Unlike traditional human-driven taxis and ride-hailing vehicles like Lyft and Uber, SAVs are unlikely to engage in quasi-legal procedures, like double parking or fire hydrant pick-ups. In congested settings, like central business districts (CBD) or airport curbs, SAVs and others will not be allowed to pick up and drop off passengers wherever they like. This paper uses an agent-based simulation to model the impact of different PUDO locations and densities in the Austin, Texas CBD, where land values are highest and curb spaces are coveted. In this paper 18 scenarios were tested, varying PUDO density, fleet size and fare price. The results show that for a given fare price and fleet size, PUDO spacing (e.g., one block vs. three blocks) has significant impact on ridership, vehicle-miles travelled, vehicle occupancy, and revenue. A good fleet size to serve the region’s 80 core square miles is 4000 SAVs, charging a $1 fare per mile of travel distance, and with PUDOs spaced three blocks of distance apart from each other in the CBD. 
    more » « less
  4. ; (, Sustainable Cities and Society)
    The transition to Electric Vehicles (EVs) for reducing urban greenhouse gas emissions is hindered by the lack of public charging infrastructure, particularly fast-charging stations. Given that electric vehicle fast charging stations (EVFCS) can burden the electricity grid, it is crucial for EVFCS to adopt sustainable energy supply methods while accommodating the growing demands of EVs. Despite recent research efforts to optimize the placement of renewable-powered EV charging stations, current planning methods face challenges when applied to a complex city scale and integrating with renewable energy resources. This study thus introduces a robust decision-making model for optimal EVFCS placement planning integrated with solar power supply in a large and complex urban environment (e.g., Chicago), utilizing an advantage actor-critic (A2C) deep reinforcement learning (DRL) approach. The model balances traffic demand with energy supply, strategically placing charging stations in areas with high traffic density and solar potential. As a result, the model is used to optimally place 1,000 charging stations with a random starting search approach, achieving total reward values of 74.30 %, and estimated the capacities of potential EVFCS. This study can inform the identification of suitable locations to advance the microgrid-based charging infrastructure systems in large urban environments. 
    more » « less
  5. ; ; ; (, Proceedings of the 27th ACM International Conference on Information and Knowledge Management)
    The knowledge of all occupied and unoccupied trips made by selfemployed drivers are essential for optimized vehicle dispatch by ride-hailing services (e.g., Didi Dache, Uber, Lyft, Grab, etc.). However, vehicles’ occupancy status is not always known to service operators due to adoption of multiple ride-hailing apps. In this paper, we propose a novel framework, Learning to INfer Trips (LINT), to infer occupancy of car trips by exploring characteristics of observed occupied trips. Two main research steps, stop point classification and structural segmentation, are included in LINT. In the first step, we represent a vehicle trajectory as a sequence of stop points, and assign stop points with pick-up, drop-off, and intermediate labels thus producing a stop point label sequence. In the second step, for structural segmentation, we further propose several segmentation algorithms, including greedy segmentation (GS), efficient greedy segmentation (EGS), and dynamic programming-based segmentation (DP) to infer occupied trip from stop point label sequences. Our comprehensive experiments on real vehicle trajectories from self-employed drivers show that (1) the proposed stop point classifier predicts stop point labels with high accuracy, and (2) the proposed segmentation algorithm GS delivers the best accuracy performance with efficient running time. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email