Search for: All records

Efficient multi-robot task allocation (MRTA) is fundamental to various time-sensitive applications such as disaster response, warehouse operations, and construction. This paper tackles a particular class of these problems that we call MRTA-collective transport or MRTA-CT – here tasks present varying workloads and deadlines, and robots are subject to flight range, communication range, and payload constraints. For large instances of these problems involving 100s-1000’s of tasks and 10s-100s of robots, traditional non-learning solvers are often time-inefficient, and emerging learning-based policies do not scale well to larger-sized problems without costly retraining. To address this gap, we use a recently proposed encoder-decoder graph neural network involving Capsule networks and multi-head attention mechanism, and innovatively add topological descriptors (TD) as new features to improve transferability to unseen problems of similar and larger size. Persistent homology is used to derive the TD, and proximal policy optimization is used to train our TD-augmented graph neural network. The resulting policy model compares favorably to state-of-the-art non-learning baselines while being much faster. The benefit of using TD is readily evident when scaling to test problems of size larger than those used in training. 

The earth’s orbit is becoming increasingly crowded with debris that poses significant safety risks to the operation of existing and new spacecraft and satellites. The active tether-net system, which consists of a flexible net with maneuverable corner nodes, launched from a small autonomous spacecraft, is a promising solution to capturing and disposing of such space debris. The requirement of autonomous operation and the need to generalize over debris scenarios in terms of different rotational rates makes the capture process significantly challenging. The space debris could rotate about multiple axes, which along with sensing/estimation and actuation uncertainties, call for a robust, generalizable approach to guiding the net launch and flight – one that can guarantee robust capture. This paper proposes a decentralized actuation system combined with reinforcement learning based on prior work in designing and controlling this tether-net system. In this new system, four microsatellites with thrusters act as the corner nodes of the net, and can thus help control the flight of the net after launch. The microsatellites pull the net to complete the task of approaching and capturing the space debris. The proposed method uses a reinforcement learning framework that integrates a proximal policy optimization to find the optimal solution based on the dynamics simulation of the net and the MUs in Vortex Studio. The reinforcement learning framework finds the optimal trajectory that is both energy-efficient and ensures a desired level of capture quality 

Opportunistic Physics-mining Transfer Mapping Architecture (OPTMA) is a hybrid architecture that combines fast simplified physics models with neural networks in order to provide significantly improved generalizability and explainability compared to pure data-driven machine learning (ML) models. However, training OPTMA remains computationally inefficient due to its dependence on gradient-free solvers or back-propagation with supervised learning over expensively pre-generated labels. This paper presents two extensions of OPTMA that are not only more efficient to train through standard back-propagation but are readily deployable through the state-of-the-art library, PyTorch. The first extension, OPTMA-Net, presents novel manual reprogramming of the simplified physics model, expressing it in Torch tensor compatible form, thus naturally enabling PyTorch's in-built Auto-Differentiation to be used for training. Since manual reprogramming can be tedious for some physics models, a second extension called OPTMA-Dual is presented, where a highly accurate internal neural net is trained apriori on the fast simplified physics model (which can be generously sampled), and integrated with the transfer model. Both new architectures are tested on analytical test problems and the problem of predicting the acoustic field of an unmanned aerial vehicle. The interference of the acoustic pressure waves produced by multiple monopoles form the basis of the simplified physics for this problem statement. An indoor noise monitoring setup in motion capture environment provided the ground truth for target data. Compared to sequential hybrid and pure ML models, OPTMA-Net/Dual demonstrate several fold improvement in performing extrapolation, while providing orders of magnitude faster training times compared to the original OPTMA. 

As city populations continue to rise, urban air mobility (UAM) seeks to provide much needed relief from traffic congestion. UAM is enforced by electrical vertical takeoff and landing (eVTOL) vehicles, which operate out of a vertiport, akin to the relationship between planes and airports. The vertiport has an air traffic controller (ATC) tasked with managing each eVTOL, ensuring they reach their destinations on time and safely. This task allocation problem can be difficult due to inadvertent issues such as mechanical failure, inclement weather, collisions, among other uncertainties that may arise. This paper provides a novel solution to this Urban Air Mobility - Vertiport Schedule Management (UAM-VSM) problem through the utilization of graph convolutional networks (GCNs). GCNs allow us to add abstractions of the vertiport space and eVTOL space as graphs, and aggregate information for a centralized ATC agent to help generalize the environment. We use Unreal Engine combined with Airsim for high fidelity simulation. The proposed GRL agent will be trained in an environment without extra uncertainties and then tested with and without those uncertainties. The performance will be examined side by side with a random and first come first serve (FCFS) baseline. 

The collective operation of robots, such as unmanned aerial vehicles (UAVs) operating as a team or swarm, is affected by their individual capabilities, which in turn is dependent on their physical design, aka morphology. However, with the exception of a few (albeit ad hoc) evolutionary robotics methods, there has been very little work on understanding the interplay of morphology and collective behavior. There is especially a lack of computational frameworks to concurrently search for the robot morphology and the hyper-parameters of their behavior model that jointly optimize the collective (team) performance. To address this gap, this paper proposes a new co-design framework. Here the exploding computational cost of an otherwise nested morphology/behavior co-design is effectively alleviated through the novel concept of “talent” metrics; while also allowing significantly better solutions compared to the typically sub-optimal sequential morphology → behavior design approach. This framework comprises four major steps: talent metrics selection, talent Pareto exploration (a multi-objective morphology optimization process), behavior optimization, and morphology finalization. This co-design concept is demonstrated by applying it to design UAVs that operate as a team to localize signal sources, e.g., in victim search and hazard localization. Here, the collective behavior is driven by a recently reported batch Bayesian search algorithm called Bayes-Swarm. Our case studies show that the outcome of co-design provides significantly higher success rates in signal source localization compared to a baseline design, across a variety of signal environments and teams with 6 to 15 UAVs. Moreover, this co-design process provides two orders of magnitude reduction in computing time compared to a projected nested design approach. 

Topology and weight evolving artificial neural network (TWEANN) algorithms optimize the structure and weights of artificial neural networks (ANNs) simultaneously. The resulting networks are typically used as policy models for solving control and reinforcement learning (RL) type problems. This paper presents a neuroevolution algorithm that aims to address the typical stagnation and sluggish convergence issues present in other neuroevolution algorithms. These issues are often caused by inadequacies in population diversity preservation, exploration/exploitation balance, and search flexibility. This new algorithm, called the Adaptive Genomic Evolution of Neural-Network Topologies (AGENT), builds on the neuroevolution of augmenting topologies (NEAT) concept. Novel mechanisms for adapting the selection and mutation operations are proposed to favorably control population diversity and exploration/exploitation balance. The former is founded on a fundamentally new way of quantifying diversity by taking a graph-theoretic perspective of the population of genomes and inter-genomic differences. Further advancements to the NEAT paradigm occur through the incorporation of variable neuronal properties and new mutation operations that uniquely allow both the growth and pruning of ANN topologies during evolution. Numerical experiments with benchmark control problems adopted from the OpenAI Gym illustrate the competitive performance of AGENT against standard RL methods and adaptive HyperNEAT, and superiority over the original NEAT algorithm. Further parametric analysis provides key insights into the impact of the new features in AGENT. This is followed by evaluation on an unmanned aerial vehicle collision avoidance problem where maneuver planning models are learnt by AGENT with 33% reward improvement over 15 generations. 

This paper introduces a new graph neural network architecture for learning solutions of Capacitated Vehicle Routing Problems (CVRP) as policies over graphs. CVRP serves as an important benchmark for a wide range of combinatorial planning problems, which can be adapted to manufacturing, robotics and fleet planning applications. Here, the specific aim is to demonstrate the significant real-time executability and (beyond training) scalability advantages of the new graph learning approach over existing solution methods. While partly drawing motivation from recent graph learning methods that learn to solve CO problems such as multi-Traveling Salesman Problem (mTSP) and VRP, the proposed neural architecture presents a novel encoder-decoder architecture. Here the encoder is based on Capsule networks, which enables better representation of local and global information with permutation invariant node embeddings; and the decoder is based on the Multi-head attention (MHA) mechanism allowing sequential decisions. This architecture is trained using a policy gradient Reinforcement Learning process. The performance of our approach is favorably compared with state-of-the-art learning and non-learning methods for a benchmark suite of Capacitated-VRP (CVRP) problems. A further study on the CVRP with demand uncertainties is conducted to explore how this Capsule-Attention Mechanism architecture can be extended to handle real-world uncertainties by embedding them through the encoder.

 

The maintenance of wind farms is one of the major factors affecting their profitability. During preventive maintenance, the shutdown of wind turbines causes downtime energy losses. The selection of when and which turbines to maintain can significantly impact the overall downtime energy loss. This paper leverages a wind farm power generation model to calculate downtime energy losses during preventive maintenance for an offshore wind farm. Wake effects are considered to accurately evaluate power output under specific wind conditions. In addition to wind speed and direction, the influence of wake effects is an important factor in selecting time windows for maintenance. To minimize the overall downtime energy loss of an offshore wind farm caused by preventive maintenance, a mixed‐integer nonlinear optimization problem is formulated and solved by the genetic algorithm, which can select the optimal maintenance time windows of each turbine. Weather conditions are imposed as constraints to ensure the safety of maintenance personnel and transportation. Using the climatic data of Cape Cod, Massachusetts, the schedule of preventive maintenance is optimized for a simulated utility‐scale offshore wind farm. The optimized schedule not only reduces the annual downtime energy loss by selecting the maintenance dates when wind speed is low but also decreases the overall influence of wake effects within the farm. The portion of downtime energy loss reduced due to consideration of wake effects each year is up to approximately 0.2% of the annual wind farm energy generation across the case studies—with other stated opportunities for further profitability improvements.