An official website of the United States government
Vision-based methods are commonly used in robotic arm activity recognition. These approaches typically rely on line-of-sight (LoS) and raise privacy concerns, particularly in smart home applications. Passive Wi-Fi sensing represents a new paradigm for recognizing human and robotic arm activi- ties, utilizing channel state information (CSI) measurements to identify activities in indoor environments. In this paper, a novel machine learning approach based on discrete wavelet transform and vision transformers for robotic arm activity recognition from CSI measurements in indoor settings is proposed. This method outperforms convolutional neural network (CNN) and long short- term memory (LSTM) models in robotic arm activity recognition, particularly when LoS is obstructed by barriers, without relying on external or internal sensors or visual aids. Experiments are conducted using four different data collection scenarios and four different robotic arm activities. Performance results demonstrate that wavelet transform can significantly enhance the accuracy of visual transformer networks in robotic arms activity recognition.
more »
« less
RoboFiSense: Attention-Based Robotic Arm Activity Recognition With WiFi Sensing
Despite the current surge of interest in autonomous robotic systems, robot activity recognition within restricted in- door environments remains a formidable challenge. Conventional methods for detecting and recognizing robotic arms’ activities often rely on vision-based or light detection and ranging (LiDAR) sensors, which require line-of-sight (LoS) access and may raise privacy concerns, for example, in nursing facilities. This research pioneers an innovative approach harnessing channel state in- formation (CSI) measured from WiFi signals, subtly influenced by the activity of robotic arms. We developed an attention- based network to classify eight distinct activities performed by a Franka Emika robotic arm in different situations. Our proposed bidirectional vision transformer-concatenated (BiVTC) methodology aspires to predict robotic arm activities accurately, even when trained on activities with different velocities, all without dependency on external or internal sensors or visual aids. Considering the high dependency of CSI data on the environment motivated us study the problem of sniffer location selection, by systematically changing the sniffer’s location and collecting different sets of data. Finally, this paper also marks the first publication of the CSI data of eight distinct robotic arm activities, collectively referred to as RoboFiSense. This initiative aims to provide a benchmark dataset and baselines to the research community, fostering advancements in the field of robotics sensing.
more »
« less
- Award ID(s):
- 2121121
- PAR ID:
- 10535980
- Publisher / Repository:
- IEEE
- Date Published:
- Journal Name:
- IEEE Journal of Selected Topics in Signal Processing
- ISSN:
- 1932-4553
- Page Range / eLocation ID:
- 1 to 12
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Current models of bending in soft arms are formulated in terms of experimentally determined, arm-specific parameters, which cannot evaluate fundamental differences in soft robot arm design. Existing models are successful at improving control of individual arms but do not give insight into how the structure of the arm affects the arm’s capabilities. For example, omnidirectional soft robot arms most frequently have three parallel actuators, but may have four or more, while common biological arms, including octopuses, have tens of distinct longitudinal muscle bundles. This article presents a quasi-static analytical model of soft arms bent with longitudinal actuators, based on equilibrium principles and assuming an unknown neutral axis location. The model is presented as a generalizable framework and specifically implemented for an arm with [Formula: see text] fluid-driven actuators, a subset of which are pressurized to induce a bend with a certain curvature and direction. The presented implementation is validated experimentally using planar (2D) and spatial (3D) bends. The planar model is used to initially estimate pressure for a closed-loop curvature control system and to bound the accessible configurations for a rapidly-exploring random trees (RRT) motion planner. A three-segment planar arm is demonstrated to navigate along a planned trajectory through a gap in a wall. Finally, the model is used to explore how the arm morphology affects maximum curvature and directional resolution. This research analytically connects soft arm structure and actuator behavior to unloaded arm performance, and the results may be used to methodically design soft robot arms.more » « less
-
Persons with disabilities often rely on caregivers or family members to assist in their daily living activities. Robotic assistants can provide an alternative solution if intuitive user interfaces are designed for simple operations. Current humanrobot interfaces are still far from being able to operate in an intuitive way when used for complex activities of daily living (ADL). In this era of smartphones that are packed with sensors, such as accelerometers, gyroscopes and a precise touch screen, robot controls can be interfaced with smartphones to capture the user’s intended operation of the robot assistant. In this paper, we review the current popular human-robot interfaces, and we present three novel human-robot smartphone-based interfaces to operate a robotic arm for assisting persons with disabilities in their ADL tasks. Useful smartphone data, including 3 dimensional orientation and 2 dimensional touchscreen positions, are used as control variables to the robot motion in Cartesian teleoperation. In this paper, we present the three control interfaces, their implementation on a smartphone to control a robotic arm, and a comparison between the results on using the three interfaces for three different ADL tasks. The developed interfaces provide intuitiveness, low cost, and environmental adaptability.more » « less
-
Inspired by the embodied intelligence observed in octopus arms, we introduce magnetically controlled origami robotic arms based on Kresling patterns for multimodal deformations, including stretching, folding, omnidirectional bending, and twisting. The highly integrated motion of the robotic arms is attributed to inherent features of the reconfigurable Kresling unit, whose controllable bistable deploying/folding and omnidirectional bending are achieved through precise magnetic actuation. We investigate single- and multiple-unit robotic systems, the latter exhibiting higher biomimetic resemblance to octopus’ arms. We start from the single Kresling unit to delineate the working mechanism of the magnetic actuation for deploying/folding and bending. The two-unit Kresling assembly demonstrates the basic integrated motion that combines omnidirectional bending with deploying. The four-unit Kresling assembly constitutes a robotic arm with a larger omnidirectional bending angle and stretchability. With the foundation of the basic integrated motion, scalability of Kresling assemblies is demonstrated through distributed magnetic actuation of double-digit number of units, which enables robotic arms with sophisticated motions, such as continuous stretching and contracting, reconfigurable bending, and multiaxis twisting. Such complex motions allow for functions mimicking octopus arms that grasp and manipulate objects. The Kresling robotic arm with noncontact actuation provides a distinctive mechanism for applications that require synergistic robotic motions for navigation, sensing, and interaction with objects in environments with limited or constrained access. Based on small-scale Kresling robotic arms, miniaturized medical devices, such as tubes and catheters, can be developed in conjunction with endoscopy, intubation, and catheterization procedures using functionalities of object manipulation and motion under remote control.more » « less
-
Construction tasks involve various activities composed of one or more body motions. It is essential to understand the dynamically changing behavior and state of construction workers to manage construction workers effectively with regards to their safety and productivity. While several research efforts have shown promising results in activity recognition, further research is still necessary to identify the best locations of motion sensors on a worker’s body by analyzing the recognition results for improving the performance and reducing the implementation cost. This study proposes a simulation-based evaluation of multiple motion sensors attached to workers performing typical construction tasks. A set of 17 inertial measurement unit (IMU) sensors is utilized to collect motion sensor data from an entire body. Multiple machine learning algorithms are utilized to classify the motions of the workers by simulating several scenarios with different combinations and features of the sensors. Through the simulations, each IMU sensor placed in different locations of a body is tested to evaluate its recognition accuracy toward the worker’s different activity types. Then, the effectiveness of sensor locations is measured regarding activity recognition performance to determine relative advantage of each location. Based on the results, the required number of sensors can be reduced maintaining the recognition performance. The findings of this study can contribute to the practical implementation of activity recognition using simple motion sensors to enhance the safety and productivity of individual workers.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1109/JSTSP.2024.3416851
