More Like this

1. Exploring Multidimensional Measurements for Pain Evaluation using Facial Action Units

   https://doi.org/10.1109/FG47880.2020.00087  

   Xu, Xiaojing; de Sa, Virginia R. (January 2020, 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020))

   Although pain is widely recognized to be a multidimensional experience, it is typically measured by unidimensional patient self-reported visual analog scale (VAS). However, self-reported pain is subjective, difficult to interpret and sometimes impossible to obtain. Machine learning models have been developed to automatically recognize pain at both the frame level and sequence (or video) level. Many methods use or learn facial action units (AUs) defined by the Facial Action Coding System (FACS) for describing facial expressions with muscle movement. In this paper, we analyze the relationship between sequence-level multidimensional pain measurements and frame-level AUs and an AU derived pain-related measure, the Prkachin and Solomon Pain Intensity (PSPI). We study methods that learn sequence-level metrics from frame-level metrics. Specifically, we explore an extended multitask learning model to predict VAS from human-labeled AUs with the help of other sequence-level pain measurements during training. This model consists of two parts: a multitask learning neural network model to predict multidimensional pain scores, and an ensemble learning model to linearly combine the multidimensional pain scores to best approximate VAS. Starting from human-labeled AUs, the model achieves a mean absolute error (MAE) on VAS of 1.73. It outperforms provided human sequence-level estimates which have an MAE of 1.76. Combining our machine learning model with the human estimates gives the best performance of MAE on VAS of 1.48. 

   more » « less
2. Personalized Pain Detection in Facial Video with Uncertainty Estimation

   Xu, Xiaojing; de Sa, V.R. (January 2021, 43rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC))

   null (Ed.)

   Pain is a personal, subjective experience, and the current gold standard to evaluate pain is the Visual Analog Scale (VAS), which is self-reported at the video level. One problem with the current automated pain detection systems is that the learned model doesn’t generalize well to unseen subjects. In this work, we propose to improve pain detection in facial videos using individual models and uncertainty estimation. For a new test video, we jointly consider which individual models generalize well generally, and which individual models are more similar/accurate to this test video, in order to choose the optimal combination of individual models and get the best performance on new test videos. We show on the UNBCMcMaster Shoulder Pain Dataset that our method significantly improves the previous state-of-the-art performance. 

   more » « less
3. Experimental Exploration of Objective Human Pain Assessment Using Multimodal Sensing Signals

   https://doi.org/10.3389/fnins.2022.831627  

   Lin, Yingzi; Xiao, Yan; Wang, Li; Guo, Yikang; Zhu, Wenchao; Dalip, Biren; Kamarthi, Sagar; Schreiber, Kristin L.; Edwards, Robert R.; Urman, Richard D. (February 2022, Frontiers in Neuroscience)

   Optimization of pain assessment and treatment is an active area of research in healthcare. The purpose of this research is to create an objective pain intensity estimation system based on multimodal sensing signals through experimental studies. Twenty eight healthy subjects were recruited at Northeastern University. Nine physiological modalities were utilized in this research, namely facial expressions (FE), electroencephalography (EEG), eye movement (EM), skin conductance (SC), and blood volume pulse (BVP), electromyography (EMG), respiration rate (RR), skin temperature (ST), blood pressure (BP). Statistical analysis and machine learning algorithms were deployed to analyze the physiological data. FE, EEG, SC, BVP, and BP proved to be able to detect different pain states from healthy subjects. Multi-modalities proved to be promising in detecting different levels of painful states. A decision-level multi-modal fusion also proved to be efficient and accurate in classifying painful states. 

   more » « less
4. Investigating the Neurophysiological Effect of Thermal Environment on Individuals' Performance Using Electroencephalogram

   Wang, Xi; Li, Da; Menassa, Carol. C; and Kamat, Vineet R. (January 2019, International Conference on. Computing in Civil Engineering (i3CE): Smart Cities, Sustainability, and Resilience)

   The thermal environment has a great influence on individuals’ performance; however, factors such as one’s motivation to perform well under experimental conditions cause difficulties in assessing how room temperature affect subjects’ performance. One approach to overcome this problem is to understand the changes in individuals’ neurophysiological conditions. This paper reports on the results of an experiment where electroencephalogram (EEG) data were collected from 5 subjects while they performed four computerized cognitive tasks. Power spectral density of EEG signals in three different thermal environments, slightly cool, neutral, and slightly warm, was compared within subjects. In most cases, significant differences in PSD of the frontal theta (4–8 Hz) activity are observed, indicating individuals’ mental effort varies with room temperature. In the long run, the increased mental workload will reduce individuals’ performance and be detrimental to their productivity. The study indicates that the proposed method could be implemented on a larger scale for further studies. 

   more » « less
5. Too hot, too fast! Using the thermal grill illusion to explore dynamic thermal perception

   https://doi.org/10.1109/HAPTICS.2018.8357167  

   Patwardhan, Shriniwas; Kawazoe, Anzu; Kerr, David; Nakatani, Masashi; Visell, Yon (March 2018, 2018 IEEE Haptics Symposium (HAPTICS))

   Thermal perception is important in the experience of touching real objects, and thermal display devices are of growing interest for applications in virtual reality, medicine, and wearable technologies. In this paper, we designed a new thermal display, and investigated the perception of spatially varying thermal stimuli, including the thermal grill illusion. The latter is a perceptual effect in which a burning sensation is elicited in response to touching a surface composed of spatially juxtaposed warm and cool areas. Using a computer controlled thermal display, we present experiments in which we measured temporal correlates of the perception of spatially inhomogeneous stimuli, or thermal grills. We assessed the intensity of responses elicited by thermal grill stimuli with different temperature settings, and measured the response time until the onset of burning sensations. We found that thermal grills elicited highly stereotyped responses. The experimental results also indicated that as the temperature difference increases, the intensity increases monotonically, while the response time decreases monotonically. Consequently, perceived intensity was inversely correlated with response time. Under current physiological explanations, responses to thermal stimuli depend on tissue heating, neural processing, and the spatial distribution (or juxtaposition) of surface temperatures. The results of this study could help to inform models accounting for these factors, enabling new applications of the thermal grill illusion. 

   more » « less