An official website of the United States government
Learning the human--mobility interaction (HMI) on interactive scenes (e.g., how a vehicle turns at an intersection in response to traffic lights and other oncoming vehicles) can enhance the safety, efficiency, and resilience of smart mobility systems (e.g., autonomous vehicles) and many other ubiquitous computing applications. Towards the ubiquitous and understandable HMI learning, this paper considers both spoken language (e.g., human textual annotations) and unspoken language (e.g., visual and sensor-based behavioral mobility information related to the HMI scenes) in terms of information modalities from the real-world HMI scenarios. We aim to extract the important but possibly implicit HMI concepts (as the named entities) from the textual annotations (provided by human annotators) through a novel human language and sensor data co-learning design. To this end, we propose CG-HMI, a novel Cross-modality Graph fusion approach for extracting important Human-Mobility Interaction concepts from co-learning of textual annotations as well as the visual and behavioral sensor data. In order to fuse both unspoken and spoken languages, we have designed a unified representation called the human--mobility interaction graph (HMIG) for each modality related to the HMI scenes, i.e., textual annotations, visual video frames, and behavioral sensor time-series (e.g., from the on-board or smartphone inertial measurement units). The nodes of the HMIG in these modalities correspond to the textual words (tokenized for ease of processing) related to HMI concepts, the detected traffic participant/environment categories, and the vehicle maneuver behavior types determined from the behavioral sensor time-series. To extract the inter- and intra-modality semantic correspondences and interactions in the HMIG, we have designed a novel graph interaction fusion approach with differentiable pooling-based graph attention. The resulting graph embeddings are then processed to identify and retrieve the HMI concepts within the annotations, which can benefit the downstream human-computer interaction and ubiquitous computing applications. We have developed and implemented CG-HMI into a system prototype, and performed extensive studies upon three real-world HMI datasets (two on car driving and the third one on e-scooter riding). We have corroborated the excellent performance (on average 13.11% higher accuracy than the other baselines in terms of precision, recall, and F1 measure) and effectiveness of CG-HMI in recognizing and extracting the important HMI concepts through cross-modality learning. Our CG-HMI studies also provide real-world implications (e.g., road safety and driving behaviors) about the interactions between the drivers and other traffic participants.
more »
« less
This content will become publicly available on August 22, 2025
Beyond "Taming Electric Scooters": Disentangling Understandings of Micromobility Naturalistic Riding
Electric(e)-scooters have emerged as a popular, ubiquitous, and first/last-mile micromobility transportation option within and across many cities worldwide. With the increasing situation-awareness and on-board computational capability, such intelligent micromobility has become a critical means of understanding the rider's interactions with other traffic constituents (called Rider-to-X Interactions, RXIs), such as pedestrians, cars, and other micromobility vehicles, as well as road environments, including curbs, road infrastructures, and traffic signs. How to interpret these complex, dynamic, and context-dependent RXIs, particularly for the rider-centric understandings across different data modalities --- such as visual, behavioral, and textual data --- is essential for enabling safer and more comfortable micromobility riding experience and the greater good of urban transportation networks. Under a naturalistic riding setting (i.e., without any unnatural constraint on rider's decision-making and maneuvering), we have designed, implemented, and evaluated a pilot Cross-modality E-scooter Naturalistic Riding Understanding System, namely CENRUS, from a human-centered AI perspective. We have conducted an extensive study with CENRUS in sensing, analyzing, and understanding the behavioral, visual, and textual annotation data of RXIs during naturalistic riding. We have also designed a novel, efficient, and usable disentanglement mechanism to conceptualize and understand the e-scooter naturalistic riding processes, and conducted extensive human-centered AI model studies. We have performed multiple downstream tasks enabled by the core model within CENRUS to derive the human-centered AI understandings and insights of complex RXIs, showcasing such downstream tasks as efficient information retrieval and scene understanding. CENRUS can serve as a foundational system for safe and easy-to-use micromobility rider assistance as well as accountable use of micromobility vehicles.
more »
« less
- Award ID(s):
- 2239897
- PAR ID:
- 10540815
- Publisher / Repository:
- ACM
- Date Published:
- Journal Name:
- Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Volume:
- 8
- Issue:
- 3
- ISSN:
- 2474-9567
- Page Range / eLocation ID:
- 1 to 24
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Electric scooters (or e-scooters) are among the most popular micromobility options that have experienced an enormous expansion in urban transportation systems across the world in recent years. Along with the increased usage of e-scooters, the increasing number of e-scooter-related injuries has also become an emerging global public health concern. However, little is known regarding the risk factors for e-scooter-related crashes and injury crashes. This study consisted of a two-phase survey questionnaire administered to a cohort of e-scooter riders (n = 210), which obtained exposure information on riders’ demographics, riding behaviors (including infrastructure selection), helmet use, and other crash-related factors. The risk ratios of riders’ self-reported involvement in an e-scooter-related crash (i.e., any crash versus no crash) and injury crash (i.e., injury crash versus non-injury crash) were estimated across exposure subcategories using the Negative Binomial regression approach. Males and frequent users of e-scooters were associated with an increased risk of e-scooter-related crashes of any type. For the e-scooter-related injury crashes, more frequently riding on bike lanes (i.e., greater than 25% of the time), either protected or unprotected, was identified as a protective factor. E-scooter-related injury crashes were more likely to occur among females, who reported riding on sidewalks and non-paved surfaces more frequently. The study may help inform public policy regarding e-scooter legislation and prioritize efforts to establish suitable road infrastructure for improved e-scooter riding safety.more » « less
-
This study investigated electric-scooter (e-scooter) rider behaviors and preferences to inform ways to increase safety for e-scooter riders. Data was collected from 329 e-scooter riders via two online and one in-person survey. Survey questions considered rider roadway infrastructure preferences, safety perceptions, and helmet-wearing behavior. Protected bike lanes were more commonly indicated as the safest infrastructure (62.4%) but were less likely to be the most preferred infrastructure (49.7%). Sidewalks were better matched between riders, indicating them as their preferred riding infrastructure (22.7%) and the perceived safest riding infrastructure (24.5%). Riders had low feelings of safety and preference for riding on major/neighborhood streets or on unprotected bike lanes. Riders reported significant concern about being hit by a moving vehicle, running into a pothole/rough roadway, and running into a moving vehicle. In line with the Theory of Planned Behavior, a significant relationship was found between the frequency of riding and helmet-wearing behavior, with more frequent riders being more likely to wear helmets. Findings suggest that existing roadway infrastructure may pose safety challenges and encourage rider-selected workarounds. Public policy may consider emphasizing protected bicycle lane development, rather than helmet mandates, to support e-scooter riding safety for all vulnerable road users.more » « less
-
Micromobility usage has increased significantly in the last several years as exemplified by shared escooters and privately owned bicycles. In this study, we use traffic camera footage to observe the behavior of over 700 shared e-scooters and privately owned bicycles in Asbury Park, New Jersey. We address the following questions: (1) What are the behavioral differences between bicycle and e-scooter usage in terms of helmet use, bike lane / sidewalk use, gender split, group riding, and by time of day? (2) Are more protective conditions associated with helmet use and bike lane / sidewalk use? And (3) what is the gender split between e-scooter users and cyclists? We find notable differences in safety precautions: around one third of cyclists but no shared e-scooter users were observed wearing a helmet. Among cyclists, helmet use was more prominent among men than women. However, men were more likely to ride on the road than women. We also found that the gender split was narrower among e-scooter users, with a nearly even gender split – as opposed to cyclists, where only 21% of cyclists were observed to be women. Our findings suggest that e-scooter users take fewer safety precautions, in that they are less likely to use a bike lane and to wear a helmet. We conclude with policy implications with regards to safety and gender differences between these two modes.more » « less
-
Self-driving vehicles are the latest innovation in improving personal mobility and road safety by removing arguably error-prone humans from driving-related tasks. Such advances can prove especially beneficial for people who are blind or have low vision who cannot legally operate conventional motor vehicles. Missing from the related literature, we argue, are studies that describe strategies for vehicle design for these persons. We present a case study of the participatory design of a prototype for a self-driving vehicle human-machine interface (HMI) for a graduate-level course on inclusive design and accessible technology. We reflect on the process of working alongside a co-designer, a person with a visual disability, to identify user needs, define design ideas, and produce a low-fidelity prototype for the HMI. This paper may benefit researchers interested in using a similar approach for designing accessible autonomous vehicle technology. INTRODUCTION The rise of autonomous vehicles (AVs) may prove to be one of the most significant innovations in personal mobility of the past century. Advances in automated vehicle technology and advanced driver assistance systems (ADAS) specifically, may have a significant impact on road safety and a reduction in vehicle accidents (Brinkley et al., 2017; Dearen, 2018). According to the Department of Transportation (DoT), automated vehicles could help reduce road accidents caused by human error by as much as 94% (SAE International, n.d.). In addition to reducing traffic accidents and saving lives and property, autonomous vehicles may also prove to be of significant value to persons who cannot otherwise operate conventional motor vehicles. AVs may provide the necessary mobility, for instance, to help create new employment opportunities for nearly 40 million Americans with disabilities (Claypool et al., 2017; Guiding Eyes for the Blind, 2019), for instance. Advocates for the visually impaired specifically have expressed how “transformative” this technology can be for those who are blind or have significant low vision (Winter, 2015); persons who cannot otherwise legally operate a motor vehicle. While autonomous vehicles have the potential to break down transportationmore » « less
