skip to main content


Title: Learning efficient navigation in vortical flow fields
Abstract

Efficient point-to-point navigation in the presence of a background flow field is important for robotic applications such as ocean surveying. In such applications, robots may only have knowledge of their immediate surroundings or be faced with time-varying currents, which limits the use of optimal control techniques. Here, we apply a recently introduced Reinforcement Learning algorithm to discover time-efficient navigation policies to steer a fixed-speed swimmer through unsteady two-dimensional flow fields. The algorithm entails inputting environmental cues into a deep neural network that determines the swimmer’s actions, and deploying Remember and Forget Experience Replay. We find that the resulting swimmers successfully exploit the background flow to reach the target, but that this success depends on the sensed environmental cue. Surprisingly, a velocity sensing approach significantly outperformed a bio-mimetic vorticity sensing approach, and achieved a near 100% success rate in reaching the target locations while approaching the time-efficiency of optimal navigation trajectories.

 
more » « less
NSF-PAR ID:
10383657
Author(s) / Creator(s):
; ; ; ;
Publisher / Repository:
Nature Publishing Group
Date Published:
Journal Name:
Nature Communications
Volume:
12
Issue:
1
ISSN:
2041-1723
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. SUMMARY

    In previous publications, we presented a general framework, which we called ‘box tomography’, that allows the coupling of any two different numerical seismic wave propagation solvers, respectively outside and inside a target region, or ‘box’. The goal of such hybrid wavefield computations is to reduce the cost of computations in the context of full-waveform inversion for structure within the target region, when sources and/or receivers are located at large distances from the box. Previously, we had demonstrated this approach with sources and receivers outside the target region in a 2-D acoustic spherical earth model, and demonstrated and applied this methodology in the 3-D spherical elastic Earth in a continental scale inversion in which all stations were inside the target region. Here we extend the implementation of the approach to the case of a 3-D global elastic earth model in the case where both sources and stations are outside the box. We couple a global 3-D solver, SPECFEM3D_GLOBE, for the computation of the wavefield and Green’s functions in a reference 3-D model, with a regional 3-D solver, RegSEM, for the computation of the wavefield within the box, by means of time-reversal mirrors. We briefly review key theoretical aspects, showing in particular how only the displacement is needed to be stored at the boundary of the box. We provide details of the practical implementation, including the geometrical design of the mirrors, how we deal with different sizes of meshes in the two solvers, and how we address memory-saving through the use of B-spline compression of the recorded wavefield on the mirror. The proposed approach is numerically efficient but also versatile, since adapting it to other solvers is straightforward and does not require any changes in the solver codes themselves, as long as the displacement can be recovered at any point in time and space. We present benchmarks of the hybrid computations against direct computations of the wavefield between a source and an array of stations in a realistic geometry centred in the Yellowstone region, with and without a hypothetical plume within the ‘box’, and with a 1-D or a 3-D background model, down to a period of 20 s. The ultimate goal of this development is for applications in the context of imaging of remote target regions in the deep mantle, such as, for example, Ultra Low Velocity Zones.

     
    more » « less
  2. Intelligent mobile sensors, such as uninhabited aerial or underwater vehicles, are becoming prevalent in environmental sensing and monitoring applications. These active sensing platforms operate in unsteady fluid flows, including windy urban environments, hurricanes and ocean currents. Often constrained in their actuation capabilities, the dynamics of these mobile sensors depend strongly on the background flow, making their deployment and control particularly challenging. Therefore, efficient trajectory planning with partial knowledge about the background flow is essential for teams of mobile sensors to adaptively sense and monitor their environments. In this work, we investigate the use of finite-horizon model predictive control (MPC) for the energy-efficient trajectory planning of an active mobile sensor in an unsteady fluid flow field. We uncover connections between trajectories optimized over a finite-time horizon and finite-time Lyapunov exponents of the background flow, confirming that energy-efficient trajectories exploit invariant coherent structures in the flow. We demonstrate our findings on the unsteady double gyre vector field, which is a canonical model for chaotic mixing in the ocean. We present an exhaustive search through critical MPC parameters including the prediction horizon, maximum sensor actuation, and relative penalty on the accumulated state error and actuation effort. We find that even relatively short prediction horizons can often yield energy-efficient trajectories. We also explore these connections on a three-dimensional flow and ocean flow data from the Gulf of Mexico. These results are promising for the adaptive planning of energy-efficient trajectories for swarms of mobile sensors in distributed sensing and monitoring. 
    more » « less
  3.  
    more » « less
  4. Abstract

    ROV operations are mainly performed via a traditional control kiosk and limited data feedback methods, such as the use of joysticks and camera view displays equipped on a surface vessel. This traditional setup requires significant personnel on board (POB) time and imposes high requirements for personnel training. This paper proposes a virtual reality (VR) based haptic-visual ROV teleoperation system that can substantially simplify ROV teleoperation and enhance the remote operator's situational awareness.

    This study leverages the recent development in Mixed Reality (MR) technologies, sensory augmentation, sensing technologies, and closed-loop control, to visualize and render complex underwater environmental data in an intuitive and immersive way. The raw sensor data will be processed with physics engine systems and rendered as a high-fidelity digital twin model in game engines. Certain features will be visualized and displayed via the VR headset, whereas others will be manifested as haptic and tactile cues via our haptic feedback systems. We applied a simulation approach to test the developed system.

    With our developed system, a high-fidelity subsea environment is reconstructed based on the sensor data collected from an ROV including the bathymetric, hydrodynamic, visual, and vehicle navigational measurements. Specifically, the vehicle is equipped with a navigation sensor system for real-time state estimation, an acoustic Doppler current profiler for far-field flow measurement, and a bio-inspired artificial literal-line hydrodynamic sensor system for near-field small-scale hydrodynamics. Optimized game engine rendering algorithms then visualize key environmental features as augmented user interface elements in a VR headset, such as color-coded vectors, to indicate the environmental impact on the performance and function of the ROV. In addition, augmenting environmental feedback such as hydrodynamic forces are translated into patterned haptic stimuli via a haptic suit for indicating drift-inducing flows in the near field. A pilot case study was performed to verify the feasibility and effectiveness of the system design in a series of simulated ROV operation tasks.

    ROVs are widely used in subsea exploration and intervention tasks, playing a critical role in offshore inspection, installation, and maintenance activities. The innovative ROV teleoperation feedback and control system will lower the barrier for ROV pilot jobs.

     
    more » « less
  5. Abstract

    Multimarginal Optimal Transport (MOT) has attracted significant interest due to applications in machine learning, statistics, and the sciences. However, in most applications, the success of MOT is severely limited by a lack of efficient algorithms. Indeed, MOT in general requires exponential time in the number of marginalskand their support sizesn. This paper develops a general theory about what “structure” makes MOT solvable in$$\mathrm {poly}(n,k)$$poly(n,k)time. We develop a unified algorithmic framework for solving MOT in$$\mathrm {poly}(n,k)$$poly(n,k)time by characterizing the structure that different algorithms require in terms of simple variants of the dual feasibility oracle. This framework has several benefits. First, it enables us to show that the Sinkhorn algorithm, which is currently the most popular MOT algorithm, requires strictly more structure than other algorithms do to solve MOT in$$\mathrm {poly}(n,k)$$poly(n,k)time. Second, our framework makes it much simpler to develop$$\mathrm {poly}(n,k)$$poly(n,k)time algorithms for a given MOT problem. In particular, it is necessary and sufficient to (approximately) solve the dual feasibility oracle—which is much more amenable to standard algorithmic techniques. We illustrate this ease-of-use by developing$$\mathrm {poly}(n,k)$$poly(n,k)-time algorithms for three general classes of MOT cost structures: (1) graphical structure; (2) set-optimization structure; and (3) low-rank plus sparse structure. For structure (1), we recover the known result that Sinkhorn has$$\mathrm {poly}(n,k)$$poly(n,k)runtime; moreover, we provide the first$$\mathrm {poly}(n,k)$$poly(n,k)time algorithms for computing solutions that are exact and sparse. For structures (2)-(3), we give the first$$\mathrm {poly}(n,k)$$poly(n,k)time algorithms, even for approximate computation. Together, these three structures encompass many—if not most—current applications of MOT.

     
    more » « less