The implementation of two‐photon polymerization (TPP) in the microrobotics community has permitted the fabrication of complex 3D structures at the microscale, creating novel platforms with potential biomedical applications for minimizing procedure invasiveness and diagnosis accuracy. Although advanced functionalities for manipulation and drug delivery tasks have been explored, one remaining challenge is achieving improved visualization, identification, and accurate closed‐loop control of microscale robots. To enable this, distinguishable identifying and trackable features must be included on the microrobot. Toward this end, the construction of micro‐ and nanoscale patterns using TPP is demonstrated for the first time on microrobot surfaces with the intent of mimicking color‐expressing nanostructures present on beetles or butterflies. The patterns provide identification and tracking targets due to their vivid color expression under visible light. Helical and rectangular microrobots are designed with the topical patterns and further functionalized with magnetic materials to be externally actuated by magnetic fields. Vision‐based tracking of a 20 μm × 30 μm colored feature on a 100 μm‐long helical microrobot using a fixed angular position light source during microrobotic motion is shown. This versatile structural color patterning approach shows great potential for the visual differentiation of various microrobots and tracking for improved closed‐loop control.
MACHINE VISION TRACKING AND AUTOMATION OF A MICROROBOT (SAFAM)
In this paper, we propose a method for tracking a microrobot’s three-dimensional position using microscope machine vision. The microrobot, theSolid Articulated Four Axis Microrobot (sAFAM), is being developed to enable the assembly and manipulation of micro and nanoscale objects. In the future, arrays of sAFAMS working together can be integrated into a wafer-scale nanofactory, Prior to use, microrobots in this microfactory need calibration, which can be achieved using the proposed measurement technique. Our approach enables faster and more accurate mapping of microrobot translations and rotations, and orders of magnitude larger datasets can be created by automation. Cameras feeds on a custom microscopy system is fed into a data processing pipeline that enables tracking of the microrobot in real-time. This particular machine vision method was implemented with a help of OpenCV and Python and can be used to track the movement of other micrometer-sized features. Additionally, a script was created to enable automated repeatability tests for each of the six trajectories traversable by the robot. A more precise microrobot workable area was also determined thanks to the significantly larger datasets enabled by the combined automation and machine vision approaches.
Keywords: Micro robotics, machine vision, nano microscale manufacturing.
more »
« less
- Award ID(s):
- 1950137
- NSF-PAR ID:
- 10322712
- Date Published:
- Journal Name:
- Manufacturing science and engineering
- ISSN:
- 1096-6668
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
-
Tracking microrobots is challenging, considering their minute size and high speed. As the field progresses towards developing microrobots for biomedical applications and conducting mechanistic studies in physiologically relevant media (e.g., collagen), this challenge is exacerbated by the dense surrounding environments with feature size and shape comparable to microrobots. Herein, we report Motion Enhanced Multi-level Tracker (MEMTrack), a robust pipeline for detecting and tracking microrobots using synthetic motion features, deep learning-based object detection, and a modified Simple Online and Real-time Tracking (SORT) algorithm with interpolation for tracking. Our object detection approach combines different models based on the object's motion pattern. We trained and validated our model using bacterial micro-motors in collagen (tissue phantom) and tested it in collagen and aqueous media. We demonstrate that MEMTrack accurately tracks even the most challenging bacteria missed by skilled human annotators, achieving precision and recall of 77% and 48% in collagen and 94% and 35% in liquid media, respectively. Moreover, we show that MEMTrack can quantify average bacteria speed with no statistically significant difference from the laboriously-produced manual tracking data. MEMTrack represents a significant contribution to microrobot localization and tracking, and opens the potential for vision-based deep learning approaches to microrobot control in dense and low-contrast settings. All source code for training and testing MEMTrack and reproducing the results of the paper have been made publicly available this https URL.more » « less
-
null (Ed.)Abstract Microassembly systems utilizing precision robotics have long been used for realizing three-dimensional microstructures such as microsystems and microrobots. Prior to assembly, microscale components are fabricated using micro-electromechanical-system (MEMS) technology. The microassembly system then directs a microgripper through a series of automated or human-controlled pick-and-place operations. In this paper, we describe a novel custom microassembly system, named NEXUS, that can be used to prototype MEMS microrobots. The NEXUS integrates multi-degrees-of-freedom (DOF) precision positioners, microscope computer vision, and microscale process tools such as a microgripper and vacuum tip. A semi-autonomous human–machine interface (HMI) was programmed to allow the operator to interact with the microassembly system. The NEXUS human–machine interface includes multiple functions, such as positioning, target detection, visual servoing, and inspection. The microassembly system's HMI was used by operators to assemble various three-dimensional microrobots such as the Solarpede, a novel light-powered stick-and-slip mobile microcrawler. Experimental results are reported in this paper to evaluate the system's semi-autonomous capabilities in terms of assembly rate and yield and compare them to purely teleoperated assembly performance. Results show that the semi-automated capabilities of the microassembly system's HMI offer a more consistent assembly rate of microrobot components and are less reliant on the operator's experience and skill.more » « less
-
Microassembly systems utilizing precision robotics have long been used for realizing 3-dimensional microstructures such as microrobots. Prior to assembly, such components are fabricated using Micro-Electro-Mechanical-System (MEMS) technology. The microassembly system then directs a microgripper through automated or human-controlled pick-and-place operations. In this paper, we describe a novel custom microassembly system, named NEXUS. The NEXUS integrates multi-degree of freedom (DOF) precision positioners, microscope computer vision, and micro-scale process tools such as a microgripper and vacuum tip. A semi-autonomous human-machine interface (HMI) is programmed by NI LabVIEW® to allow the operator to interact with the microassembly system. The NEXUS human-machine interface includes multiple functions, such as positioning, target detection, visual servoing, and inspection. The microassembly system’s HMI was used by operators to assemble various 3-dimensional microrobots such as the Solarpede, a novel light-powered stick-and-slip mobile microcrawler. Experimental results are reported in this paper that evaluate the system’s semi-autonomous capabilities in terms of assembly rate and yield and compare them to purely teleoperated assembly performance. Results show that the semi-automated capabilities of the microassembly system’s HMI offer a more consistent assembly rate of microrobot components.more » « less
-
This paper presents the use of a micro-force sensing mobile microrobot (μFSMM) for in vitro biomedical applications. The μFSMM utilizes a vision-based force sensor end-effector, which computes the force based on the deflection of a compliant structure with a known stiffness using a computer vision tracking algorithm. The μFSMM is used to characterize the stiffness of several different alginate and hyaluronic acid hydrogel spheroid samples, which are typically used in 3D tissue engineered constructs for studying cellular behavior. Additionally, μFSMM is used to perform safe micromanipulation tasks with these spheroids. These experimental results showcase some of the applications of this unique microrobot design in the fields of mechanobiology, theranostics, and force-guided micromanipulation.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
