skip to main content


Title: Reinforcement learning of a multi-link swimmer at low Reynolds numbers
The use of machine learning techniques in the development of microscopic swimmers has drawn considerable attention in recent years. In particular, reinforcement learning has been shown useful in enabling swimmers to learn effective propulsion strategies through its interactions with the surroundings. In this work, we apply a reinforcement learning approach to identify swimming gaits of a multi-link model swimmer. The swimmer consists of multiple rigid links connected serially with hinges, which can rotate freely to change the relative angles between neighboring links. Purcell [“Life at low Reynolds number,” Am. J. Phys. 45, 3 (1977)] demonstrated how the particular case of a three-link swimmer (now known as Purcell's swimmer) can perform a prescribed sequence of hinge rotation to generate self-propulsion in the absence of inertia. Here, without relying on any prior knowledge of low-Reynolds-number locomotion, we first demonstrate the use of reinforcement learning in identifying the classical swimming gaits of Purcell's swimmer for case of three links. We next examine the new swimming gaits acquired by the learning process as the number of links increases. We also consider the scenarios when only a single hinge is allowed to rotate at a time and when simultaneous rotation of multiple hinges is allowed. We contrast the difference in the locomotory gaits learned by the swimmers in these scenarios and discuss their propulsion performance. Taken together, our results demonstrate how a simple reinforcement learning technique can be applied to identify both classical and new swimming gaits at low Reynolds numbers.  more » « less
Award ID(s):
1931292 1830958
PAR ID:
10440239
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
Physics of Fluids
Volume:
35
Issue:
3
ISSN:
1070-6631
Page Range / eLocation ID:
032003
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Biological microswimmers can coordinate their motions to exploit their fluid environment—and each other—to achieve global advantages in their locomotory performance. These cooperative locomotion require delicate adjustments of both individual swimming gaits and spatial arrangements of the swimmers. Here we probe the emergence of such cooperative behaviors among artificial microswimmers endowed with artificial intelligence. We present the first use of a deep reinforcement learning approach to empower the cooperative locomotion of a pair of reconfigurable microswimmers. The AI-advised cooperative policy comprises two stages: an approach stage where the swimmers get in close proximity to fully exploit hydrodynamic interactions, followed a synchronization stage where the swimmers synchronize their locomotory gaits to maximize their overall net propulsion. The synchronized motions allow the swimmer pair to move together coherently with an enhanced locomotion performance unattainable by a single swimmer alone. Our work constitutes a first step toward uncovering intriguing cooperative behaviors of smart artificial microswimmers, demonstrating the vast potential of reinforcement learning towards intelligent autonomous manipulations of multiple microswimmers for their future biomedical and environmental applications. 
    more » « less
  2. Abstract Swimming microorganisms switch between locomotory gaits to enable complex navigation strategies such as run-and-tumble to explore their environments and search for specific targets. This ability of targeted navigation via adaptive gait-switching is particularly desirable for the development of smart artificial microswimmers that can perform complex biomedical tasks such as targeted drug delivery and microsurgery in an autonomous manner. Here we use a deep reinforcement learning approach to enable a model microswimmer to self-learn effective locomotory gaits for translation, rotation and combined motions. The Artificial Intelligence (AI) powered swimmer can switch between various locomotory gaits adaptively to navigate towards target locations. The multimodal navigation strategy is reminiscent of gait-switching behaviors adopted by swimming microorganisms. We show that the strategy advised by AI is robust to flow perturbations and versatile in enabling the swimmer to perform complex tasks such as path tracing without being explicitly programmed. Taken together, our results demonstrate the vast potential of these AI-powered swimmers for applications in unpredictable, complex fluid environments. 
    more » « less
  3. Swimming at low Reynolds number is typically dominated by a large viscous drag, therefore microscale swimmers require non-reciprocal body deformation to generate locomotion. Purcell described a simple mechanical swimmer at the microscale consisting of three rigid components connected together with two hinges. Here we present a simple microswimmer consisting of two rigid paramagnetic particles with different sizes. When placed in an eccentric magnetic field, this simple microswimmer exhibits non-reciprocal body motion and its swimming locomotion can be directed in a controllable manner. Additional components can be added to create a multibody microswimmer, whereby the particles act cooperatively and translate in a given direction. For some multibody swimmers, the stochastic thermal forces fragment the arm, which therefore modifies the swimming strokes and changes the locomotive speed. This work offers insight into directing the motion of active systems with novel time-varying magnetic fields. It also reveals that Brownian motion not only affects the locomotion of reciprocal swimmers that are subject to the Scallop theorem, but also affects that of non-reciprocal swimmers. 
    more » « less
  4. We conduct experiments with flexible swimmers to address the impact of fluid viscoelasticity on their locomotion. The swimmers are composed of a magnetic head actuated in rotation by a frequency-controlled magnetic field and a flexible tail whose deformation leads to forward propulsion. We consider both viscous Newtonian and glucose-based Boger fluids with similar viscosities. We find that the elasticity of the fluid systematically enhances the locomotion speed of the swimmer and that this enhancement increases with Deborah number. Using particle image velocimetry to visualize the flow field, we find a significant difference in the amount of shear between the rear and leading parts of the swimmer head. We conjecture that viscoelastic normal stresses lead to a net elastic forces in the swimming direction and thus a faster swimming speed.

     
    more » « less
  5. Swimming at the microscale has recently garnered substantial attention due to the fundamental biological significance of swimming microorganisms and the wide range of biomedical applications for artificial microswimmers. These microswimmers invariably find themselves surrounded by different confining boundaries, which can impact their locomotion in significant and diverse ways. In this work, we employ a widely used three-sphere swimmer model to investigate the effect of confinement on swimming at low Reynolds numbers. We conduct theoretical analysis via the point-particle approximation and numerical simulations based on the finite element method to examine the motion of the swimmer along the centerline in a capillary tube. The axisymmetric configuration reduces the motion to one-dimensional movement, which allows us to quantify how the degree of confinement affects the propulsion speed in a simple manner. Our results show that the confinement does not significantly affect the propulsion speed until the ratio of the radius of the tube to the radius of the sphere is in the range of O(1)−O(10), where the swimmer undergoes substantial reduction in its propulsion speed as the radius of the tube decreases. We provide some physical insights into how reduced hydrodynamic interactions between moving spheres under confinement may hinder the propulsion of the three-sphere swimmer. We also remark that the reduced propulsion performance stands in stark contrast to the enhanced helical propulsion observed in a capillary tube, highlighting how the manifestation of confinement effects can vary qualitatively depending on the propulsion mechanisms employed by the swimmers.

     
    more » « less