Chronic pain is a prevalent problem that can cause continuous medical expenditures and reduce the quality of life. According to the 2019 National Health Interview Survey, 20.4% of adults have chronic pain, and 7.4% of adults have high-impact chronic pain that can frequently hinder their daily activities (Zelaya, Dahlhamer, Lucas, & Connor, 2020).
Assessing chronic pain is critical to developing effective management solutions, but traditional assessments rely on physician-patient communication or self-reporting, which may be unreliable (Lin et al., 2018). Recently, sensor-fusion pain assessment methods that use physiological signals have emerged, aiming to reduce inconsistencies. Machine learning has accelerated the development of these methods (Guo, Wang, Xiao, & Lin, 2021;Wang et al., 2022). Mobile health (m-health) solutions also have become popular due to their mobility and scalability (Sun et al., 2021). However, current m-health solutions for pain management either rely solely on self-report by patients or are limited to lab environments through pain stimulus tests, slowing down scaling (Pfeifer et al., 2020). This highlights the need for more accessible and objective pain assessment and management solutions that can be delivered remotely.
In this paper, an app that can have both clinical utility and in-home pain management functions was proposed. The app name is abbreviated from an NSF-funded project entitled Novel Computational Methods for Continuous Objective Multimodal Pain Assessment Sensing System (COMPASS) (Lin et al., 2022). It predicted objective pain levels using physiological signals and tracked daily activities affecting chronic pain. A usability test was designed to improve this app. The results showed the validity of using physiological signals to predict internalizing pain levels. The architecture and test results of the COMPASS app were introduced.
M-health apps, particularly e-diaries, have been shown to be valuable in managing chronic pain (Charoenpol, Tontisirin, Leerapan, Seangrung, & Finlayson, 2019). The dialogue-style action recording process triggers reflection, encouraging users to adopt new behaviors (Kocielnik, Xiao, Avrahami, & Hsieh, 2018), which is similar to behavioral intervention therapy. Mobile apps make this therapy more accessible and affordable, leading to profound clinical importance (Simon et al., 2021). However, the adoption of currently available mobile pain management applications is hindered due to their limitations. Reviews have found that most of those relying solely on self-reporting (Pfeifer et al., 2020;Zhao, Yoo, Lancey, & Varghese, 2019), leading to inconsistency in reported pain levels. To address this problem, physiological signals have been used and proved to be reliable (Alotaiby, Alshebeili, Aljafar, & Alsabhan, 2019). They utilized a common spatial pattern algorithm to extract features from ECG signals and then Support Vector Machine (SVM) was adopted to achieve an identification rate of 95.15%. It can be used in the classification between pain and no-pain status, achieving 86% of accuracy in five classes of pain (Modares-Haghighi, Boostani, Nami, & Sanei, 2021). At-home cold water pain tests with pre-prepared directions have also been shown to produce lab-level results, paving the way for the combination of the sensor-fusion pain assessment method with mobile apps (McIntyre et al., 2020).
To obtain a more specific estimation of pain levels, pain intensity started to be treated as a continuous variable so machine learning methods can be used (Lin et al., 2022). Some researchers developed and evaluated an automatic and adaptable pain assessment algorithm based on ECG features using real pain data on postoperative patients (Naeini et al., 2021). Thirtytwo features from both the time domain and frequency domain were extracted, and five classifiers were applied to perform five pain-level tasks with the highest accuracy of 84.79%. Respiration rate is also investigated in the pain assessment research. Rui Cao et al. (Cao, Aqajari, Naeini, & Rahmani, 2021) proposed an objective pain assessment method using respiration rates from real post-operative patients. A filter-based feature selection method was used to identify the top most significant features and 8 features were finally selected. Five classifiers were applied to perform the binary classification tasks, and an accuracy of 81.41% was achieved.
One core idea of pain management application is to trigger behavioral change. One of the most famous behavior change models is the Theory of Planned Behavior (Ajzen, 1991), which is widely applied to research fields. In 2019, TPB was refined by incorporating the goal system theory, resulting in the Theory of Reasoned Goal Pursuit (Ajzen & Kruglanski, 2019). Compared with the original Theory of Planned Behavior, the Theory of Reasoned Goal Pursuit is more suitable to be the base framework of the COMPASS app since it emphasizes the significance of the goal, which is considered the starting point of behavior change.
An AI-assisted Goal Pursuit framework in Fig. 1 has been developed based on the Theory of Reasoned Goal Pursuit model (Ajzen & Kruglanski, 2019) to guide the design of the COMPASS app. The AI pain assessment system is at the core of the COMPASS app. As depicted in the dashed border of Fig. 1, the AI pain assessment system predicts pain levels based on physiological data collected by sensors. Feedback from the AI pain assessment system helps users establish self-report standards, improving the accuracy and consistency of selfreported pain levels. Users can also provide feedback on the AI pain predictions, which aids in customizing the pain assessment algorithm for individual users.
The AI pain predictions can also subtly influence the users' attitudes towards behaviors. Feedback from the AI pain assessment provides an objective view of changes in users' pain sensitivity and pain levels. With objective data as a reference, users are more likely to believe that certain behaviors will benefit their overall pain management process.
In addition to the core AI pain assessment system, the COMPASS app also considers the role of Behavioral Control plays in behavior change (Ajzen & Kruglanski, 2019). In the framework, the COMPASS system affects "Pain-related Behavioral Control" by incorporating a daily check-in section. Users can customize their daily check-ins by selecting the behaviors they wish to monitor, such as sitting or diet shown in Fig. 2. By achieving small goals each day, users can feel more in control of their behaviors and experience a sense of accomplishment.
The COMPASS app is composed of four main sections, with the Home page serving as the gateway to each of these sections. Additionally, the Home page acts as a data dashboard (refer to the leftmost screen in Fig. 2), enabling users to view pain level trends and track their behavioral objectives quickly. In the Profile section, users can personalize their basic information and adjust interaction settings to enhance the app's suitability.
Bi-weekly cold water pain tests are scheduled in the COMPASS app. With the assistance of the instructions provided, users can perform the sensor connection, baseline recording, pain report training, and the final cold water pain report section (two Cold Water Pain Test screens in Fig. 2) independently from the comfort of their homes. A bi-weekly schedule, as opposed to weekly or monthly, is deemed to provide a good temporal spread, enabling to capture the variability of pain level and user comfort and safety.
The daily pain log check-in is the most frequently used section, which contains a pain level log and a key behavior log. For the pain level log, users connect the sensor to obtain a pain level prediction from the algorithm, followed by a self-report of their pain perception as feedback to the algorithm. Regarding the key behavior log, users record their sleep, sitting, emotions, and dietary data (two Daily Check-in screens in Fig. 2).
Based on the AI-assisted Goal Pursuit Model, the COMPASS app prototype is developed. An experiment is designed to explore the following research questions: RQ1: How does the physiological signal help assess users' pain level and how's the assessment quality?
RQ2: How easy is it for users who suffer from pain to navigate this app and perform cold water pain tests with it?
Participants. The majority of participants in this study were students enrolled at Northeastern University in Boston, Massachusetts. Individuals who were not affiliated with the university were also included. All participants were required to be adults and considered healthy. A total of 33 participants completed the questionnaires, consisting of 16 male and 17 female subjects. The racial and ethnic diversity of the participants consisted of White (81.8%), Latinx (15.2%), Asian (9.1%), and preferred not to respond (3%). 29 participants were pursuing bachelor's degrees, while the other four were pursuing master's degrees. Additionally, most participants used iOS smartphones (94.0%). All tests were conducted under the oversight and approval of Northeastern University's Institutional Review Board (IRB #21-11-18).
Experiment Process. Participants were firstly asked to finish a short questionnaire about their past m-health and painful experiences. They were then asked to perform six tasks listed in Table 1 on the app prototype. The tasks were designed to guide participants in exploring the app.
One key task was to perform a cold water pain test following the instructions provided on the app prototype. For each subject, 120 seconds of baseline data were first collected. Then, the subject placed their hand in a bucket of ice water. Every 20 seconds, the subject was prompted to choose a pain rating on a scale of 0-10 by pressing the buttons that appeared on the app screen (See the middle screen in Fig. 2.) The test concluded after 10 pain ratings were recorded (200 seconds). If the pain became too extreme at any time, the subject was able to terminate the test. Aidmed (Czekaj et al., 2020), a portable medical device, was attached to the subject's chest to collect the physiological signals, including electrocardiogram (ECG), respiration, and temperature data.
After completing all six tasks, participants were required to answer a short questionnaire regarding their experience and attitudes toward the app.
The initial section of the survey explored participants' history of painful experiences. Among the 33 subjects, 22 had never used any healthcare apps, while the remaining 11 reported previous usage. When faced with pain, in contrast to the 19 subjects who chose not to, only 4 subjects pursued further medical assistance from specialists. Six subjects reported having been hospitalized for chronic pain, and 8 subjects expressed difficulty in assessing the reasons for their discomfort accurately.
Interestingly, recurring pain was not a common experience among our subjects; only four reported pain recurrences in particular areas, while the majority experienced sporadic pain. However, this sporadic pain did not significantly impact their daily functioning. The rest four who did experience chronic pain sought medical care for persistent discomfort and tended to endure their pain until it subsided.
Subsequently, participants were asked to execute six tasks using the app prototype. Participants were allowed to ask for clarifications during the task, with the time still running. This decision might have influenced their success rates. As a result, we decided to adopt a time-based efficiency metric, that combines the success rate and the time spent on tasks. By measuring the time they took to complete (or abandon) each task, the time-based efficiency (Mifsud, n.d.) of the task can be calculated based on the equation below:
physical comfort of the wearable devices involved. The same number affirmed the general usefulness of the app. While user intention to integrate the app into their routines varied, thirteen participants were willing to make it a regular part of their day and eight participants indicated they would likely use the app periodically. As for the pain management question, 17 out of the 33 participants stated they would resort to the app when experiencing pain, while 6 subjects expressed a tentative willingness to do the same.
Where:
Time Based Efficiency = j=1 i=1 ti j NR N=number of tasks (here is 1) R=number of users (here is 33) ni j=The result of task i by user j; if user completed the task, ni j=1, otherwise ni j=0 ti j=The time user j spent on task i
Table 1 presents the time-based efficiency of the six tasks performed by the participants. As all participants were firsttime users, it is not surprising that they were more efficient in completing common digital app tasks, such as filling out the pre-questionnaire than in completing COMPASS app-specific tasks. However, the efficiency data indicates that the participants found the bi-weekly cold water pain test was easier to complete than the pain log daily check-in. This suggests that the information architecture and user flow of the check-in process need further improvement. Since the pain log check-in is a daily activity, confusion or difficulty in completing it may lead to reduced user engagement over time.
A post-task survey was conducted to gather the participants' feedback on their experiences interacting with the COMPASS app experience (Fig. 3). Almost 60% of the participants found the app straightforward and not overly complicated. In addition, about 80% felt confident using the app without requiring technical assistance. They also provided positive feedback on the data recording process, indicating it was a smooth experience. Sixteen people commented on the
As for the completion of the cold water pain test, four subjects terminated in the middle of their experiments. All data including physiological signals and pain level recordings from all subjects were recorded and compiled to be analyzed. The first measure that the team focused on was the change in pain level over time. As time continues, pain levels will rise as seen by the visible trend, mean and range in Fig. 4.
Physiological signals have been established as reliable indicators of an individual's varying states (Zhu, Kucyi, Kramer, & Lin, 2022). Key signals, including ECG, respiration, and temperature, were first preprocessed using min-max standardization (Van Gent, Farah, Nes, & van Arem, 2018). Subsequently, 20-second data segments were divided into five 4-second samples, each labeled with the pain level reported at the end of the 20-second period. The 0-10 pain level scale was categorized as low pain (0-5) and high pain (6-10) to balance two classes, which represented approximately 60.66% of the low pain class and 39.34% of the high pain class. Statistical features of each signal such as mean, standard deviation, max, min, and range, and physiological features were extracted from signals, including beats per minute, interbeat interval, standard deviation of RR intervals, standard deviation of successive differences, root mean square of successive differences, low frequency between 0.05-0.15Hz, high frequency between 0.15-0.5Hz, the ration of high frequency/low frequency, heart rate variability, and respiration rate. The collected data was split into 75% training and 25% testing sets. Features were selected by setting the 0.8 variance threshold on the training dataset. And Synthetic Minority Oversampling Technique technique (Fernández, Garcia, Herrera, & Chawla, 2018) was also applied to the training dataset to solve the imbalance problem. Decision tree and knearest neighbor machine learning models were applied to classify the pain classes, with 5-fold cross-validation. Precision, recall, and f-1 scores were selected as model evaluation metrics. Results demonstrated that the KNN model outperformed the DT model in predicting pain classes, achieving 70.3% F-1 score.
In conclusion, this paper has showcased the potential of mobile health applications in aiding individuals with pain tracking and management. By utilizing a cold water pain test, we've been able to gauge a subject's physiological sensitivity to pain. The application of machine learning algorithms to this physiological data has exhibited promising performance, with an F-1 score of 70.3% for the k-Nearest Neighbors model. Our usability testing showed that the majority of participants found the application user-friendly and identified areas for improvement. Moving forward, we will continue refining our app and center on making it more accessible, personalized, and user-friendly for those living with pain.