Search for: All records

Microrobots powered by an external magnetic field could be used for sophisticated medical applications such as cell treatment, micromanipulation, and noninvasive surgery inside the body. Untethered microrobot applications can benefit from haptic technology and telecommunication, enabling telemedical micro-manipulation. Users can manipulate the microrobots with haptic feedback by interacting with the robot operating system remotely in such applications. Artificially created haptic forces based on wirelessly transmitted data and model-based guidance can aid human operators with haptic sensations while manipulating microrobots. The system presented here includes a haptic device and a magnetic tweezer system linked together using a network-based teleoperation method with motion models in fluids. The magnetic microrobots can be controlled remotely, and the haptic interactions with the remote environment can be felt in real time. A time-domain passivity controller is applied to overcome network delay and ensure stability of communication. This study develops and tests a motion model for microrobots and evaluates two image-based 3D tracking algorithms to improve tracking accuracy in various Newtonian fluids. Additionally, it demonstrates that microrobots can group together to transport multiple larger objects, move through microfluidic channels for detailed tasks, and use a novel method for disassembly, greatly expanding their range of use in microscale operations. Remote medical treatment in multiple locations, remote delivery of medication without the need for physical penetration of the skin, and remotely controlled cell manipulations are some of the possible uses of the proposed technology. 

Human emotions are expressed through multiple modalities, including verbal and non-verbal information. Moreover, the affective states of human users can be the indicator for the level of engagement and successful interaction, suitable for the robot to use as a rewarding factor to optimize robotic behaviors through interaction. This study demonstrates a multimodal human-robot interaction (HRI) framework with reinforcement learning to enhance the robotic interaction policy and personalize emotional interaction for a human user. The goal is to apply this framework in social scenarios that can let the robots generate a more natural and engaging HRI framework. 

Natural Human-Robot interaction (HRI) attracts considerable interest in letting robots understand the users’ emotional state. This paper demonstrates a method to introduce the affection model to the robotic system’s conversational agent to provide natural and empathetic HRI. We use a large-scale pre-trained language model and fine-tune it on a dialogue dataset with empathetic characteristics. Based on existing studies’ progress, we extend the current method and enable the agent to perform advanced sentiment analysis using the affection model. This dialogue agent will allow the robot to provide natural response along with emotion classification and the estimations of arousal and valence level. We evaluate our model using different metrics, comparing it with the recent studies and showing its emotion detection capacity. 

Decades of scientific research have been conducted on developing and evaluating methods for automated emotion recognition. With exponentially growing technology, there is a wide range of emerging applications that require emotional state recognition of the user. This paper investigates a robust approach for multimodal emotion recognition during a conversation. Three separate models for audio, video and text modalities are structured and fine-tuned on the MELD. In this paper, a transformer-based crossmodality fusion with the EmbraceNet architecture is employed to estimate the emotion. The proposed multimodal network architecture can achieve up to 65% accuracy, which significantly surpasses any of the unimodal models. We provide multiple evaluation techniques applied to our work to show that our model is robust and can even outperform the state-of-the-art models on the MELD. 

This paper presents a method for extracting novel spectral features based on a sinusoidal model. The method is focused on characterizing the spectral shapes of audio signals using spectra peaks in frequency sub-bands. The extracted features are evaluated for predicting the levels of emotional dimensions, namely arousal and valence. Principal component regression, partial least squares regression, and deep convolutional neural network (CNN) models are used as prediction models for the levels of the emotional dimensions. The experimental results indicate that the proposed features include additional spectral information that common baseline features may not include. Since the quality of audio signals, especially timbre, plays a major role in affecting the perception of emotional valence in music, the inclusion of the presented features will contribute to decreasing the prediction error rate.