In this modern world, with the increase of complexity of many technologies, especially in the micro and nanoscale, the field of robotic manipulation has tremendously grown. Microrobots and other complex microscale systems are often to laborious to fabricate using standard microfabrication techniques, therefore there is a trend towards fabricating them in parts then assembling them together, mainly using micromanipulation tools. Here, a comprehensive and robust micromanipulation platform is presented, in which four micromanipulators can be used simultaneously to perform complex tasks, providing the user with an intuitive environment. The system utilizes a vision-based force sensor to aid with manipulation tasks and it provides a safe environment for biomanipulation. Lastly, virtual reality (VR) was incorporated into the system, allowing the user to control the probes from a more intuitive standpoint and providing an immersive platform for the future of micromanipulation.
more »
« less
Stiffness Characterization and Micromanipulation for Biomedical Applications using the Vision-based Force-Sensing Magnetic Mobile Microrobot
This paper presents the use of a micro-force sensing mobile microrobot (μFSMM) for in vitro biomedical applications. The μFSMM utilizes a vision-based force sensor end-effector, which computes the force based on the deflection of a compliant structure with a known stiffness using a computer vision tracking algorithm. The μFSMM is used to characterize the stiffness of several different alginate and hyaluronic acid hydrogel spheroid samples, which are typically used in 3D tissue engineered constructs for studying cellular behavior. Additionally, μFSMM is used to perform safe micromanipulation tasks with these spheroids. These experimental results showcase some of the applications of this unique microrobot design in the fields of mechanobiology, theranostics, and force-guided micromanipulation.
more »
« less
- Award ID(s):
- 1637961
- NSF-PAR ID:
- 10309216
- Date Published:
- Journal Name:
- International Conference on Manipulation, Automation and Robotics at Small Scales
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
more » « less
Stiffness and forces are two fundamental quantities essential to living cells and tissues. However, it has been a challenge to quantify both 3D traction forces and stiffness (or modulus) using the same probe in vivo. Here, we describe an approach that overcomes this challenge by creating a magnetic microrobot probe with controllable functionality. Biocompatible ferromagnetic cobalt-platinum microcrosses were fabricated, and each microcross (about 30 micrometers) was trapped inside an arginine–glycine–aspartic acid–conjugated stiff poly(ethylene glycol) (PEG) round microgel (about 50 micrometers) using a microfluidic device. The stiff magnetic microrobot was seeded inside a cell colony and acted as a stiffness probe by rigidly rotating in response to an oscillatory magnetic field. Then, brief episodes of ultraviolet light exposure were applied to dynamically photodegrade and soften the fluorescent nanoparticle–embedded PEG microgel, whose deformation and 3D traction forces were quantified. Using the microrobot probe, we show that malignant tumor–repopulating cell colonies altered their modulus but not traction forces in response to different 3D substrate elasticities. Stiffness and 3D traction forces were measured, and both normal and shear traction force oscillations were observed in zebrafish embryos from blastula to gastrula. Mouse embryos generated larger tensile and compressive traction force oscillations than shear traction force oscillations during blastocyst. The microrobot probe with controllable functionality via magnetic fields could potentially be useful for studying the mechanoregulation of cells, tissues, and embryos.
-
In this paper, we propose a method for tracking a microrobot’s three-dimensional position using microscope machine vision. The microrobot, theSolid Articulated Four Axis Microrobot (sAFAM), is being developed to enable the assembly and manipulation of micro and nanoscale objects. In the future, arrays of sAFAMS working together can be integrated into a wafer-scale nanofactory, Prior to use, microrobots in this microfactory need calibration, which can be achieved using the proposed measurement technique. Our approach enables faster and more accurate mapping of microrobot translations and rotations, and orders of magnitude larger datasets can be created by automation. Cameras feeds on a custom microscopy system is fed into a data processing pipeline that enables tracking of the microrobot in real-time. This particular machine vision method was implemented with a help of OpenCV and Python and can be used to track the movement of other micrometer-sized features. Additionally, a script was created to enable automated repeatability tests for each of the six trajectories traversable by the robot. A more precise microrobot workable area was also determined thanks to the significantly larger datasets enabled by the combined automation and machine vision approaches. Keywords: Micro robotics, machine vision, nano microscale manufacturing.more » « less
-
more » « less
The implementation of two‐photon polymerization (TPP) in the microrobotics community has permitted the fabrication of complex 3D structures at the microscale, creating novel platforms with potential biomedical applications for minimizing procedure invasiveness and diagnosis accuracy. Although advanced functionalities for manipulation and drug delivery tasks have been explored, one remaining challenge is achieving improved visualization, identification, and accurate closed‐loop control of microscale robots. To enable this, distinguishable identifying and trackable features must be included on the microrobot. Toward this end, the construction of micro‐ and nanoscale patterns using TPP is demonstrated for the first time on microrobot surfaces with the intent of mimicking color‐expressing nanostructures present on beetles or butterflies. The patterns provide identification and tracking targets due to their vivid color expression under visible light. Helical and rectangular microrobots are designed with the topical patterns and further functionalized with magnetic materials to be externally actuated by magnetic fields. Vision‐based tracking of a 20 μm × 30 μm colored feature on a 100 μm‐long helical microrobot using a fixed angular position light source during microrobotic motion is shown. This versatile structural color patterning approach shows great potential for the visual differentiation of various microrobots and tracking for improved closed‐loop control.
-
Tracking microrobots is challenging, considering their minute size and high speed. As the field progresses towards developing microrobots for biomedical applications and conducting mechanistic studies in physiologically relevant media (e.g., collagen), this challenge is exacerbated by the dense surrounding environments with feature size and shape comparable to microrobots. Herein, we report Motion Enhanced Multi-level Tracker (MEMTrack), a robust pipeline for detecting and tracking microrobots using synthetic motion features, deep learning-based object detection, and a modified Simple Online and Real-time Tracking (SORT) algorithm with interpolation for tracking. Our object detection approach combines different models based on the object's motion pattern. We trained and validated our model using bacterial micro-motors in collagen (tissue phantom) and tested it in collagen and aqueous media. We demonstrate that MEMTrack accurately tracks even the most challenging bacteria missed by skilled human annotators, achieving precision and recall of 77% and 48% in collagen and 94% and 35% in liquid media, respectively. Moreover, we show that MEMTrack can quantify average bacteria speed with no statistically significant difference from the laboriously-produced manual tracking data. MEMTrack represents a significant contribution to microrobot localization and tracking, and opens the potential for vision-based deep learning approaches to microrobot control in dense and low-contrast settings. All source code for training and testing MEMTrack and reproducing the results of the paper have been made publicly available this https URL.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/MARSS49294.2020.9307874
