An official website of the United States government
During in-hand manipulation, robots must be able to continuously estimate the pose of the object in order to generate appropriate control actions. The performance of algorithms for pose estimation hinges on the robot's sensors being able to detect discriminative geometric object features, but previous sensing modalities are unable to make such measurements robustly. The robot's fingers can occlude the view of environment- or robot-mounted image sensors, and tactile sensors can only measure at the local areas of contact. Motivated by fingertip-embedded proximity sensors' robustness to occlusion and ability to measure beyond the local areas of contact, we present the first evaluation of proximity sensor based pose estimation for in-hand manipulation. We develop a novel two-fingered hand with fingertip-embedded optical time-of-flight proximity sensors as a testbed for pose estimation during planar in-hand manipulation. Here, the in-hand manipulation task consists of the robot moving a cylindrical object from one end of its workspace to the other. We demonstrate, with statistical significance, that proximity-sensor based pose estimation via particle filtering during in-hand manipulation: a) exhibits 50% lower average pose error than a tactile-sensor based baseline; b) empowers a model predictive controller to achieve 30% lower final positioning error compared to when using tactile-sensor based pose estimates.
more »
« less
This content will become publicly available on December 15, 2025
A Machine Learning Approach to Contact Localization in Variable Density Three-Dimensional Tactile Artificial Skin
Estimating the location of contact is a primary function of artificial tactile sensing apparatuses that perceive the environment through touch. Existing contact localization methods use flat geometry and uniform sensor distributions as a simplifying assumption, limiting their ability to be used on 3D surfaces with variable density sensing arrays. This paper studies contact localization on an artificial skin embedded with mutual capacitance tactile sensors, arranged non-uniformly in an unknown distribution along a semi-conical 3D geometry. A fully connected neural network is trained to localize the touching points on the embedded tactile sensors. The studied online model achieves a localization error of 5.7 ± 3.0 mm. This research contributes a versatile tool and robust solution for contact localization that is ambiguous in shape and internal sensor distribution.
more »
« less
- Award ID(s):
- 2222952
- PAR ID:
- 10615181
- Publisher / Repository:
- 2nd NeurIPS Workshop on Touch Processing: From Data to Knowledge
- Date Published:
- Format(s):
- Medium: X
- Location:
- Vancouver, Canada
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Tactile sensing is essential for robots to perceive and react to the environment. However, it remains a challenge to make large-scale and flexible tactile skins on robots. Industrial machine knitting provides solutions to manufacture customiz-able fabrics. Along with functional yarns, it can produce highly customizable circuits that can be made into tactile skins for robots. In this work, we present RobotSweater, a machine-knitted pressure-sensitive tactile skin that can be easily applied on robots. We design and fabricate a parameterized multi-layer tactile skin using off-the-shelf yarns, and characterize our sensor on both a flat testbed and a curved surface to show its robust contact detection, multi-contact localization, and pressure sensing capabilities. The sensor is fabricated using a well-established textile manufacturing process with a programmable industrial knitting machine, which makes it highly customizable and low-cost. The textile nature of the sensor also makes it easily fit curved surfaces of different robots and have a friendly appearance. Using our tactile skins, we conduct closed-loop control with tactile feedback for two applications: (1) human lead-through control of a robot arm, and (2) human-robot interaction with a mobile robot.more » « less
-
Sensitive and flexible pressure sensors have invoked considerable interest for a broad range of applications in tactile sensing, physiological sensing, and flexible electronics. The barrier between high sensitivity and low fabrication cost needs to be addressed to commercialize such flexible pressure sensors. A low-cost sacrificial template-assisted method for the capacitive sensor has been reported herein, utilizing a porous Polydimethylsiloxane (PDMS) polymer and a multiwalled carbon nanotube (MWCNT) composite-based dielectric layer. The sensor shows high sensitivity of 2.42 kPa−1 along with a low limit of detection of 1.46 Pa. The high sensitivity originates from adding MWCNT to PDMS, increasing the composite polymer’s dielectric constant. Besides this, the pressure sensor shows excellent stability at a cyclic loading of 9000 cycles, proving its reliability for long-lasting application in tactile and physiological sensing. The high sensitivity of the sensor is suitable for the detection of small deformations such as pulse waveforms as well as tactile pressure sensing. In addition, the paper demonstrates a simultaneous contact and non-contact sensing capability suitable for dual sensing (pressure and proximity) with a single data readout system. The dual-mode sensing capability may open opportunities for realizing compact systems in robotics, gesture control, contactless applications, and many more. The practicality of the sensor was shown in applications such as tactile sensing, Morse code generator, proximity sensing, and pulse wave sensing.more » « less
-
Tactile sensing is pivotal for enhancing robot manipulation abilities by providing crucial feedback for localized information. However, existing sensors often lack the necessary resolution and bandwidth required for intricate tasks. To address this gap, we introduce VibTac, a novel multi-modal tactile sensing finger designed to offer high-resolution and high-bandwidth tactile sensing simultaneously. VibTac seamlessly integrates vision-based and vibration-based tactile sensing modes to achieve high-resolution and high-bandwidth tactile sensing respectively, leveraging a streamlined human-inspired design for versatility in tasks. This paper outlines the key design elements of VibTac and its fabrication methods, highlighting the significance of the Elastomer Gel Pad (EGP) in its sensing mechanism. The sensor’s multi-modal performance is validated through 3D reconstruction and spectral analysis to discern tactile stimuli effectively. In experimental trials, VibTac demonstrates its efficacy by achieving over 90% accuracy in insertion tasks involving objects emitting distinct sounds, such as ethernet connectors. Leveraging vision-based tactile sensing for object localization and employing a deep learning model for “click” sound classification, VibTac showcases its robustness in real-world scenarios. Video of the sensor working can be accessed at https://youtu.be/kmKIUlXGroo.more » « less
-
In the future, human-robot interaction will include collaboration in close-quarters where the environment geometry is partially unknown. As a means for enabling such interaction, this paper presents a multi-modal sensor array capable of contact detection and localization, force sensing, proximity sensing, and mapping. The sensor array integrates Hall effect and time-of-flight (ToF) sensors in an I 2 C communication network. The design, fabrication, and characterization of the sensor array for a future in-situ collaborative continuum robot are presented. Possible perception benefits of the sensor array are demonstrated for accidental contact detection, mapping of the environment, selection of admissible zones for bracing, and constrained motion control of the end effector while maintaining a bracing constraint with an admissible rolling motion.more » « less
