skip to main content

This content will become publicly available on June 17, 2023

Title: GradTac: Spatio-Temporal Gradient Based Tactile Sensing
Tactile sensing for robotics is achieved through a variety of mechanisms, including magnetic, optical-tactile, and conductive fluid. Currently, the fluid-based sensors have struck the right balance of anthropomorphic sizes and shapes and accuracy of tactile response measurement. However, this design is plagued by a low Signal to Noise Ratio (SNR) due to the fluid based sensing mechanism “damping” the measurement values that are hard to model. To this end, we present a spatio-temporal gradient representation on the data obtained from fluid-based tactile sensors, which is inspired from neuromorphic principles of event based sensing. We present a novel algorithm (GradTac) that converts discrete data points from spatial tactile sensors into spatio-temporal surfaces and tracks tactile contours across these surfaces. Processing the tactile data using the proposed spatio-temporal domain is robust, makes it less susceptible to the inherent noise from the fluid based sensors, and allows accurate tracking of regions of touch as compared to using the raw data. We successfully evaluate and demonstrate the efficacy of GradTac on many real-world experiments performed using the Shadow Dexterous Hand, equipped with the BioTac SP sensors. Specifically, we use it for tracking tactile input across the sensor’s surface, measuring relative forces, detecting linear and more » rotational slip, and for edge tracking. We also release an accompanying task-agnostic dataset for the BioTac SP, which we hope will provide a resource to compare and quantify various novel approaches, and motivate further research. « less
; ; ; ; ;
Award ID(s):
1824198 2020624
Publication Date:
Journal Name:
Frontiers in Robotics and AI
Sponsoring Org:
National Science Foundation
More Like this
  1. Evolution has honed predatory skills in the natural world where localizing and intercepting fast-moving prey is required. The current generation of robotic systems mimics these biological systems using deep learning. High-speed processing of the camera frames using convolutional neural networks (CNN) (frame pipeline) on such constrained aerial edge-robots gets resource-limited. Adding more compute resources also eventually limits the throughput at the frame rate of the camera as frame-only traditional systems fail to capture the detailed temporal dynamics of the environment. Bio-inspired event cameras and spiking neural networks (SNN) provide an asynchronous sensor-processor pair (event pipeline) capturing the continuous temporal details of the scene for high-speed but lag in terms of accuracy. In this work, we propose a target localization system combining event-camera and SNN-based high-speed target estimation and frame-based camera and CNN-driven reliable object detection by fusing complementary spatio-temporal prowess of event and frame pipelines. One of our main contributions involves the design of an SNN filter that borrows from the neural mechanism for ego-motion cancelation in houseflies. It fuses the vestibular sensors with the vision to cancel the activity corresponding to the predator's self-motion. We also integrate the neuro-inspired multi-pipeline processing with task-optimized multi-neuronal pathway structure in primates andmore »insects. The system is validated to outperform CNN-only processing using prey-predator drone simulations in realistic 3D virtual environments. The system is then demonstrated in a real-world multi-drone set-up with emulated event data. Subsequently, we use recorded actual sensory data from multi-camera and inertial measurement unit (IMU) assembly to show desired working while tolerating the realistic noise in vision and IMU sensors. We analyze the design space to identify optimal parameters for spiking neurons, CNN models, and for checking their effect on the performance metrics of the fused system. Finally, we map the throughput controlling SNN and fusion network on edge-compatible Zynq-7000 FPGA to show a potential 264 outputs per second even at constrained resource availability. This work may open new research directions by coupling multiple sensing and processing modalities inspired by discoveries in neuroscience to break fundamental trade-offs in frame-based computer vision 1 .« less
  2. The accurate measurement of wall zeta potentials and solute–surface interaction length scales for electrolyte and non-electrolyte solutes, respectively, is critical to the design of many biomedical and microfluidic applications. We present a novel microfluidic approach using diffusioosmosis for measuring either the zeta potentials or the characteristic interaction length scales for surfaces exposed to, respectively, electrolyte or non-electrolyte solutes. When flows containing different solute concentrations merge in a junction, local solute concentration gradients can drive diffusioosmotic flow due to electrokinetic, steric, and other interactions between the solute molecules and solid surfaces. We demonstrate a microfluidic system consisting of a long, narrow pore connecting two large side channels in which solute concentration gradients drive diffusioosmosis within the pore, resulting in predictable fluid velocity/pressure and solute profiles. Furthermore, we present analytical results and a methodology to determine the zeta potential or interaction length scale for the pore surfaces based on the solute concentrations in the main side channels, the flow rate in the pore, and the pressure drop across the pore. We apply this method to the experimental data of Lee et al. to predict the zeta potentials of their system, and we use 3D numerical simulations to validate the theory and showmore »that end effects caused by the junctions are negligible for a wide range of parameters. Because the dynamics in the proposed system are driven by diffusioosmosis, this technique does not suffer from certain disadvantages associated with the use of sensitive electronics in traditional zeta potential measurement approaches such as streaming potential, streaming current, or electroosmosis. To the best of our knowledge this is the first flow-based approach to characterize surface/solute interactions with non-electrolyte solutes.« less
  3. We present a method for classifying the quality of near-contact grasps using spatial metrics that are recoverable from sensor data. Current methods often rely on calculating precise contact points, which are difficult to calculate in real life, or on tactile sensors or image data, which may be unavailable for some applications. Our method, in contrast, uses a mix of spatial metrics that do not depend on the fingers being in contact with the object, such as the object's approximate size and location. The grasp quality can be calculated {\em before} the fingers actually contact the object, enabling near-grasp quality prediction. Using a random forest classifier, the resulting system is able to predict grasp quality with 96\% accuracy using spatial metrics based on the locations of the robot palm, fingers and object. Furthermore, it can maintain an accuracy of 90\% when exposed to 10\% noise across all its inputs.
  4. The surface urban heat island (SUHI), which represents the difference of land surface temperature (LST) in urban relativity to neighboring non-urban surfaces, is usually measured using satellite LST data. Over the last few decades, advancements of remote sensing along with spatial science have considerably increased the number and quality of SUHI studies that form the major body of the urban heat island (UHI) literature. This paper provides a systematic review of satellite-based SUHI studies, from their origin in 1972 to the present. We find an exponentially increasing trend of SUHI research since 2005, with clear preferences for geographic areas, time of day, seasons, research foci, and platforms/sensors. The most frequently studied region and time period of research are China and summer daytime, respectively. Nearly two-thirds of the studies focus on the SUHI/LST variability at a local scale. The Landsat Thematic Mapper (TM)/Enhanced Thematic Mapper (ETM+)/Thermal Infrared Sensor (TIRS) and Terra/Aqua Moderate Resolution Imaging Spectroradiometer (MODIS) are the two most commonly-used satellite sensors and account for about 78% of the total publications. We systematically reviewed the main satellite/sensors, methods, key findings, and challenges of the SUHI research. Previous studies confirm that the large spatial (local to global scales) and temporal (diurnal,more »seasonal, and inter-annual) variations of SUHI are contributed by a variety of factors such as impervious surface area, vegetation cover, landscape structure, albedo, and climate. However, applications of SUHI research are largely impeded by a series of data and methodological limitations. Lastly, we propose key potential directions and opportunities for future efforts. Besides improving the quality and quantity of LST data, more attention should be focused on understudied regions/cities, methods to examine SUHI intensity, inter-annual variability and long-term trends of SUHI, scaling issues of SUHI, the relationship between surface and subsurface UHIs, and the integration of remote sensing with field observations and numeric modeling.« less
  5. Madden, John D. ; Anderson, Iain A. ; Shea, Herbert R. (Ed.)
    Ras Labs makes Synthetic Muscle™, which is a class of electroactive polymer (EAP) based materials and actuators that sense pressure (gentle touch to high impact), controllably contract and expand at low voltage (1.5 V to 50 V, including use of batteries), and attenuate force. We are in the robotics era, but robots do have their challenges. Currently, robotic sensing is mainly visual, which is useful up until the point of contact. To understand how an object is being gripped, tactile feedback is needed. For handling fragile objects, if the grip is too tight, breakage occurs, and if the grip is too loose, the object will slip out of the grasp, also leading to breakage. Rigid robotic grippers using a visual feedback loop can struggle to determine the exact point and quality of contact. Robotic grippers can also get a stuttering effect in the visual feedback loop. By using soft Synthetic Muscle™ based EAP pads as the sensors, immediate feedback was generated at the first point of contact. Because these pads provided a soft, compliant interface, the first point of contact did not apply excessive force, allowing the force applied to the object to be controlled. The EAP sensor could alsomore »detect a change in pressure location on its surface, making it possible to detect and prevent slippage by then adjusting the grip strength. In other words, directional glide provided feedback for the presence of possible slippage to then be able to control a slightly tighter grip, without stutter, due to both the feedback and the soft gentleness of the fingertip-like EAP pads themselves. The soft nature of the EAP fingertip pad also naturally held the gripped object, improving the gripping quality over rigid grippers without an increase in applied force. Analogous to finger-like tactile touch, the EAPs with appropriate coatings and electronics were positioned as pressure sensors in the fingertip or end effector regions of robotic grippers. This development of using Synthetic Muscle™ based EAPs as soft sensors provided for sensors that feel like the pads of human fingertips. Basic pressure position and magnitude tests have been successful, with pressure sensitivity down to 0.05 N. Most automation and robots are very strong, very fast, and usually need to be partitioned away from humans for safety reasons. For many repetitive tasks that humans do with delicate or fragile objects, it would be beneficial to use robotics; whether it is for agriculture, medical surgery, therapeutic or personal care, or in extreme environments where humans cannot enter, including with contagions that have no cure. Synthetic Muscle™ was also retrofitted as actuator systems into off-the-shelf robotic grippers and is being considered in novel biomimetic gripper designs, operating at low voltages (less than 50 V). This offers biomimetic movement by contracting like human muscles, but also exceeds natural biological capabilities by expanding under reversed electric polarity. Human grasp is gentle yet firm, with tactile touch feedback. In conjunction with shape-morphing abilities, these EAPs also are being explored to intrinsically sense pressure due to the correlation between mechanical force applied to the EAP and its electronic signature. The robotic field is experiencing phenomenal growth in this fourth phase of the industrial revolution, the robotics era. The combination of Ras Labs’ EAP shape-morphing and sensing features promises the potential for robotic grippers with human hand-like control and tactile sensing. This work is expected to advance both robotics and prosthetics, particularly for collaborative robotics to allow humans and robots to intuitively work safely and effectively together.« less