The implementation of two‐photon polymerization (TPP) in the microrobotics community has permitted the fabrication of complex 3D structures at the microscale, creating novel platforms with potential biomedical applications for minimizing procedure invasiveness and diagnosis accuracy. Although advanced functionalities for manipulation and drug delivery tasks have been explored, one remaining challenge is achieving improved visualization, identification, and accurate closed‐loop control of microscale robots. To enable this, distinguishable identifying and trackable features must be included on the microrobot. Toward this end, the construction of micro‐ and nanoscale patterns using TPP is demonstrated for the first time on microrobot surfaces with the intent of mimicking color‐expressing nanostructures present on beetles or butterflies. The patterns provide identification and tracking targets due to their vivid color expression under visible light. Helical and rectangular microrobots are designed with the topical patterns and further functionalized with magnetic materials to be externally actuated by magnetic fields. Vision‐based tracking of a 20 μm × 30 μm colored feature on a 100 μm‐long helical microrobot using a fixed angular position light source during microrobotic motion is shown. This versatile structural color patterning approach shows great potential for the visual differentiation of various microrobots and tracking for improved closed‐loop control.
Swimming microrobots are increasingly developed with complex materials and dynamic shapes and are expected to operate in complex environments in which the system dynamics are difficult to model and positional control of the microrobot is not straightforward to achieve. Deep reinforcement learning is a promising method of autonomously developing robust controllers for creating smart microrobots, which can adapt their behavior to operate in uncharacterized environments without the need to model the system dynamics. This article reports the development of a smart helical magnetic hydrogel microrobot that uses the soft actor critic reinforcement learning algorithm to autonomously derive a control policy which allows the microrobot to swim through an uncharacterized biomimetic fluidic environment under control of a time‐varying magnetic field generated from a three‐axis array of electromagnets. The reinforcement learning agent learns successful control policies from both state vector input and raw images, and the control policies learned by the agent recapitulate the behavior of rationally designed controllers based on physical models of helical swimming microrobots. Deep reinforcement learning applied to microrobot control is likely to significantly expand the capabilities of the next generation of microrobots.
- PAR ID:
- 10368592
- Publisher / Repository:
- Wiley Blackwell (John Wiley & Sons)
- Date Published:
- Journal Name:
- Advanced Intelligent Systems
- Volume:
- 4
- Issue:
- 10
- ISSN:
- 2640-4567
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
-
Abstract A magnetic object subject to an external rotating magnetic field would be rotated due to the alignment tendency between its internal magnetization and the field. Based on this principle, 12 shapes of swimming microrobots around 1 mm long were designed and 3D-printed using biodegradable materials Poly (ethylene glycol) diacrylate (PEDGA). Their surface was decorated with superparamagnetic iron oxide nanoparticles to provide magnetic responsivity. An array of 12 permanent magnets generated a rotating uniform magnetic field (∼100 mT) to impose magnetic torque, which induces a tumbling motion in the microrobot. We developed a dynamic model that captured the behavior of swimming microrobots of different shapes and showed good agreement with experimental results. Among these 12 shapes, we found that microrobots with equal length, width, and depth performed better. The observed translational speed of the hollow cube microrobot can exceed 17.84 mm s −1 (17.84 body lengths/s) under a rotating magnetic field of 5.26 Hz. These microrobots could swim to the targeted sites in a simplified vessel branch. And a finite element model was created to simulate the motion of the swimming microrobot under a flow rate of 0.062 m s −1 .more » « less
-
more » « less
This study investigates the motion characteristics of soft alginate microrobots in complex fluidic environments utilizing wireless magnetic fields for actuation. The aim is to explore the diverse motion modes that arise due to shear forces in viscoelastic fluids by employing snowman-shaped microrobots. Polyacrylamide (PAA), a water-soluble polymer, is used to create a dynamic environment with non-Newtonian fluid properties. Microrobots are fabricated via an extrusion-based microcentrifugal droplet method, successfully demonstrating the feasibility of both wiggling and tumbling motions. Specifically, the wiggling motion primarily results from the interplay between the viscoelastic fluid environment and the microrobots’ non-uniform magnetization. Furthermore, it is discovered that the viscoelasticity properties of the fluid influence the motion behavior of the microrobots, leading to non-uniform behavior in complex environments for microrobot swarms. Through velocity analysis, valuable insights into the relationship between applied magnetic fields and motion characteristics are obtained, facilitating a more realistic understanding of surface locomotion for targeted drug delivery purposes while accounting for swarm dynamics and non-uniform behavior.
-
This paper presents our work over the last decade in developing functional microrobotic systems, which include wireless actuation of microrobots to traverse complex surfaces, addition of sensing capabilities, and independent actuation of swarms of microrobots. We will discuss our work on the design, fabrication, and testing of a number of different mobile microrobots that are able to achieve these goals. These microrobots include the microscale magnetorestrictive asymmetric bimorph microrobot ( μ MAB), our first attempt at magnetic actuation in the microscale; the microscale tumbling microrobot ( μ TUM), our microrobot capable of traversing complex surfaces in both wet and dry conditions; and the micro-force sensing magnetic microrobot ( μ FSMM), which is capable of real-time micro-force sensing feedback to the user as well as intuitive wireless actuation. Additionally, we will present our latest results on using local magnetic field actuation for independent control of multiple microrobots in the same workspace for microassembly tasks.more » « less
-
more » « less
Smooth camber morphing aircraft offer increased control authority and improved aerodynamic efficiency. Smart material actuators have become a popular driving force for shape changes, capable of adhering to weight and size constraints and allowing for simplicity in mechanical design. As a step towards creating uncrewed aerial vehicles (UAVs) capable of autonomously responding to flow conditions, this work examines a multifunctional morphing airfoil’s ability to follow commands in various flows. We integrated an airfoil with a morphing trailing edge consisting of an antagonistic pair of macro fiber composites (MFCs), serving as both skin and actuator, and internal piezoelectric flex sensors to form a closed loop composite system. Closed loop feedback control is necessary to accurately follow deflection commands due to the hysteretic behavior of MFCs. Here we used a deep reinforcement learning algorithm, Proximal Policy Optimization, to control the morphing airfoil. Two neural controllers were trained in a simulation developed through time series modeling on long short-term memory recurrent neural networks. The learned controllers were then tested on the composite wing using two state inference methods in still air and in a wind tunnel at various flow speeds. We compared the performance of our neural controllers to one using traditional position-derivative feedback control methods. Our experimental results validate that the autonomous neural controllers were faster and more accurate than traditional methods. This research shows that deep learning methods can overcome common obstacles for achieving sufficient modeling and control when implementing smart composite actuators in an autonomous aerospace environment.
