An official website of the United States government
This paper proposes an AR-based real-time mobile system for assistive indoor navigation with target segmentation (ARMSAINTS) for both sighted and blind or low-vision (BLV) users to safely explore and navigate in an indoor environment. The solution comprises four major components: graph construction, hybrid modeling, real-time navigation and target segmentation. The system utilizes an automatic graph construction method to generate a graph from a 2D floorplan and the Delaunay triangulation-based localization method to provide precise localization with negligible error. The 3D obstacle detection method integrates the existing capability of AR with a 2D object detector and a semantic target segmentation model to detect and track 3D bounding boxes of obstacles and people to increase BLV safety and understanding when traveling in the indoor environment. The entire system does not require the installation and maintenance of expensive infrastructure, run in real-time on a smartphone, and can easily adapt to environmental changes.
more »
« less
On avoiding Moving Objects for indoor autonomous quadrotors
A mini quadrotor can be used in many applications, such as indoor airborne surveillance, payload delivery, and warehouse monitoring. In these applications, vision-based autonomous navigation is one of the most interesting research topics because precise navigation can be implemented based on vision analysis. However, pixel-based vision analysis approaches require a high-powered computer, which is inappropriate to be attached to a small indoor quadrotor. This paper proposes a method called the Motion-vector-based Moving Objects Detection. This method detects and avoids obstacles using stereo motion vectors instead of individual pixels, thereby substantially reducing the data processing requirement. Although this method can also be used in the avoidance of stationary obstacles by taking into account the ego-motion of the quadrotor, this paper primarily focuses on providing our empirical verification on the real-time avoidance of moving objects.
more »
« less
- Award ID(s):
- 1521153
- PAR ID:
- 10021678
- Date Published:
- Journal Name:
- 2016 IEEE International Conference on Automation Science and Engineering (CASE),
- Page Range / eLocation ID:
- 503 to 508
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Real-time detection of 3D obstacles and recognition of humans and other objects is essential for blind or low- vision people to travel not only safely and independently but also confidently and interactively, especially in a cluttered indoor environment. Most existing 3D obstacle detection techniques that are widely applied in robotic applications and outdoor environments often require high-end devices to ensure real-time performance. There is a strong need to develop a low-cost and highly efficient technique for 3D obstacle detection and object recognition in indoor environments. This paper proposes an integrated 3D obstacle detection system implemented on a smartphone, by utilizing deep-learning-based pre-trained 2D object detectors and ARKit- based point cloud data acquisition to predict and track the 3D positions of multiple objects (obstacles, humans, and other objects), and then provide alerts to users in real time. The system consists of four modules: 3D obstacle detection, 3D object tracking, 3D object matching, and information filtering. Preliminary tests in a small house setting indicated that this application could reliably detect large obstacles and their 3D positions and sizes in the real world and small obstacles’ positions, without any expensive devices besides an iPhone.more » « less
-
Trajectory optimization o↵ers mature tools for motion planning in high-dimensional spaces under dynamic constraints. However, when facing complex configuration spaces, cluttered with obstacles, roboticists typically fall back to sampling-based planners that struggle in very high dimensions and with continuous di↵erential constraints. Indeed, obstacles are the source of many textbook examples of problematic nonconvexities in the trajectory-optimization prob- lem. Here we show that convex optimization can, in fact, be used to reliably plan trajectories around obstacles. Specifically, we consider planning problems with collision-avoidance constraints, as well as cost penalties and hard constraints on the shape, the duration, and the velocity of the trajectory. Combining the properties of B ́ezier curves with a recently-proposed framework for finding shortest paths in Graphs of Convex Sets (GCS), we formulate the planning problem as a compact mixed-integer optimization. In stark contrast with existing mixed-integer planners, the convex relaxation of our programs is very tight, and a cheap round- ing of its solution is typically sufficient to design globally-optimal trajectories. This reduces the mixed-integer program back to a simple convex optimization, and automatically provides optimality bounds for the planned trajectories. We name the proposed planner GCS, after its underlying optimization framework. We demonstrate GCS in simulation on a variety of robotic platforms, including a quadrotor flying through buildings and a dual-arm manipulator (with fourteen degrees of freedom) moving in a confined space. Using numerical experiments on a seven-degree-of-freedom manipulator, we show that GCS can outperform widely-used sampling-based planners by finding higher-quality trajectories in less time.more » « less
-
Autonomous navigation in construction environments is particularly challenging due to dynamic obstacles and uncertain surroundings. While recent advances in Building Information Modeling (BIM)-based planning have leveraged spatial and semantic information to improve navigation, most prior work assumes precise localization of the BIM model to enable global path planning. In contrast, this paper introduces an online replanning framework that registers obstacles on discovery within BIM and replans according to the updated semantic map. Our method integrates object-aware path planning by utilizing large language models (LLMs) to extract semantic danger sentiments from BIM-annotated objects and their spatial information about the mission environment. Additionally, we demonstrate practical feasibility by integrating a path tracking control, ensuring generated paths are not only safer but also realistically executable by mobile robots. Experimental results demonstrate an improved obstacle avoidance by 2.8× compared to traditional A* algorithms in dynamically updated environments.more » « less
-
Navigation and obstacle avoidance in aquatic en-vironments for autonomous surface vehicles (ASVs) in high-traffic maritime scenarios is still an open challenge, as the Convention on the International Regulations for Preventing Collisions at Sea (COLREGs) is not defined for multi-encounter situations. Current state-of-the-art methods resolve single-to-single encounters with sequential actions and assume that other obstacles follow COLREGs. Our work proposes a novel real-time non-myopic obstacle avoidance method, allowing an ASV that has only partial knowledge of the surroundings within the sensor radius to navigate in high-traffic maritime scenarios. Specifically, we achieve a holistic view of the feasible ASV action space able to avoid deadlock scenarios, by proposing (1) a clustering method based on motion attributes of other obstacles, (2) a geometric framework for identifying the feasible action space, and (3) a multi-objective optimization to determine the best action. Theoretical analysis and extensive realistic experiments in simulation considering real-world traffic scenarios demonstrate that our proposed real-time obstacle avoidance method is able to achieve safer trajectories than other state-of-the-art methods and that is robust to uncertainty present in the current information available to the ASV.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/COASE.2016.7743447
