skip to main content

Title: Evaluating the Use of Online Self-Report Questionnaires as Clinically Valid Mental Health Monitoring Tools in the Clinical Whitespace

Although digital health solutions are increasingly popular in clinical psychiatry, one application that has not been fully explored is the utilization of survey technology to monitor patients outside of the clinic. Supplementing routine care with digital information collected in the “clinical whitespace” between visits could improve care for patients with severe mental illness. This study evaluated the feasibility and validity of using online self-report questionnaires to supplement in-person clinical evaluations in persons with and without psychiatric diagnoses. We performed a rigorous in-person clinical diagnostic and assessment battery in 54 participants with schizophrenia (N = 23), depressive disorder (N = 14), and healthy controls (N = 17) using standard assessments for depressive and psychotic symptomatology. Participants were then asked to complete brief online assessments of depressive (Quick Inventory of Depressive Symptomatology) and psychotic (Community Assessment of Psychic Experiences) symptoms outside of the clinic for comparison with the ground-truth in-person assessments. We found that online self-report ratings of severity were significantly correlated with the clinical assessments for depression (two assessments used: R = 0.63, p < 0.001; R = 0.73, p < 0.001) and psychosis (R = 0.62, p < 0.001). Our results demonstrate the feasibility and validity of collecting psychiatric symptom ratings through online surveys. Surveillance of this kind may be especially useful in detecting acute mental health crises between patient visits and can generally contribute to more comprehensive psychiatric treatment.

more » « less
Award ID(s):
Author(s) / Creator(s):
; ; ; ; ; ; ; ; ;
Publisher / Repository:
Springer Science + Business Media
Date Published:
Journal Name:
Psychiatric Quarterly
Medium: X Size: p. 221-231
["p. 221-231"]
Sponsoring Org:
National Science Foundation
More Like this
  1. Background: Deformational plagiocephaly and brachycephaly (DPB) is manifested in ~20% of newborns in the US. DPB can be effectively corrected by repositioning and/or physical therapy if detected and monitored before 4 months of age. The cranial index (CI) and cranial vault asymmetry index (CVAI) are used for DPB diagnosis and monitoring. As there is no current tool available for pediatricians or parents to quantitatively measure these indices at the point-of-care, we developed a smartphone app, called SoftSpot, that measures CI and CVAI from photographs of a child’s head to increase the chances of early detection and treatment. Objective: To prospectively evaluate the accuracy of the smartphone measurements of CI and CVAI in a clinical setting. Methods: Bird’s eye-view head photos of 117 infants aged 2-11 months (42 female, 75 male) were captured at a large multidisciplinary craniofacial center with the SoftSpot app (PediaMetrix Inc. Rockville, MD) using an iPhone X (Apple Inc., Cupertino, CA). The study was IRB approved and parent consent was obtained. Measurements included width, length, and diagonals of the patients’ head were obtained by a single CRNP and were used to calculate CI and CVAI as the ground truth. At least five images for each patient were chosen by an analyst, CI and CVAI were automatically measured by the proprietary algorithms of the app, and results were averaged for each patient. Automated and ground truth CI and CVAI measurements were compared using the Bland-Altman method and Spearman Correlation Coefficient after excluding outliers with mean absolute error (MAE) greater than two standard deviations. Results: MAE was 2.47 ± 1.68 for CI, 1.55 ± 1.03 for CVAI. Spearman correlation coefficients were 0.93 and 0.91 (p-values < 0.001) for CI and CVAI, respectively (see Fig. 2). Bland-Altman analysis (see Fig. 2 resulted in limits of agreement of [-4.41, 6.53] for CI and [-3.64, 3.68] for CVAI, with respective biases of 1.06 and 0.02. Conclusion: Our app measures CI and CVAI from head 2D photos with very high correlation to caliper-based measurements obtained in the craniofacial clinic. This prospective study demonstrates the clinical feasibility of using a smartphone app for cranial measurements at the point-of-care with the potential to early detect and monitor DPB. The app can potentially be used in telemedicine encounters when in-person visits are difficult due to circumstances like COVID-19 or for remote and underserved areas. 
    more » « less
  2. Abstract Introduction

    Utilization of telemedicine for health care delivery increased rapidly during the coronavirus disease 2019 (COVID‐19) pandemic. However, physical examination during telehealth visits remains limited. A novel telerehabilitation system—The Augmented Reality‐based Telerehabilitation System with Haptics (ARTESH)—shows promise for performing synchronous, remote musculoskeletal examination.


    To assess the potential of ARTESH in remotely examining upper extremity passive range of motion (PROM) and maximum isometric strength (MIS).


    In this cross‐sectional pilot study, we compared the in‐person (reference standard) and remote evaluations (ARTESH) of participants' upper extremity PROM and MIS in 10 shoulder and arm movements. The evaluators were blinded to each other's results.


    Participants underwent in‐person evaluations at a Veterans Affairs hospital's outpatient Physical Medicine and Rehabilitation (PM&R) clinic, and underwent remote examination using ARTESH with the evaluator located at a research lab 30 miles away, connected via a high‐speed network.


    Fifteen participants with upper extremity pain and/or weakness.


    Not applicable.

    Main Outcome Measures

    Inter‐rater agreement between in‐person and remote evaluations on 10 PROM and MIS movements and presence/absence of pain with movement was calculated.


    The highest inter‐rater agreements were noted in shoulder abduction and protraction PROM (kappa (κ) = 0.44, confidence interval (CI): −0.1 to 1.0), and in elbow flexion, shoulder abduction, and shoulder protraction MIS (κ = 0.63, CI: 0 to 1.0).


    This pilot study suggests that synchronous tele‐physical examination using the ARTESH system with augmented reality and haptics has the potential to provide enhanced value to existing telemedicine platforms. With the additional technological and procedural improvements and with an adequately powered study, the accuracy of ARTESH‐enabled remote tele‐physical examinations can be better evaluated.

    more » « less
  3. Abstract Aim

    The aim of this study is to develop a Smarthealth system of monitoring, modelling, and interactive recommendation solutions (for caregivers) for in‐home dementia patient care that focuses on caregiver–patient relationships.


    This descriptive study employs a single‐group, non‐randomized trial to examine functionality, effectiveness, feasibility, and acceptability of the novel Smarthealth system.


    Thirty persons with Alzheimer's Disease or related dementia and their family caregivers (N = 30 dyads) will receive and install Smarthealth technology in their home. There will be a 1‐month observation phase for collecting baseline mood states and a 2‐month implementation phase when caregivers will receive stress management techniques for each detected, negative mood state. Caregivers will report technique implementation and usefulness, sent via Ecological Momentary Assessment system to the study‐provided smartphone. Caregivers will provide daily, self‐reported mood and health ratings. Instruments measuring caregiver assessment of disruptive behaviours and their effect on caregivers; caregiver depressive symptoms, anxiety and stress; caregiver strain; and family functioning will be completed at baseline and 3 months. The study received funding in 2018 and ethics board approval in 2019.


    This study will develop and test novel in‐home technology to improve family caregiving relationships. Results from this study will help develop and improve the Smarthealth recommendation system and determine its usefulness, feasibility, and acceptability for persons with dementia and their family caregiver.


    The Smarthealth technology discussed will provide in‐home stress reduction resources at a time when older adults may be experiencing increasingly high rates of isolation and anxiety and caregiver dyads may be experiencing high levels of relationship strain.

    Trial Registration

    This study was registered with Clinical (Identifier NCT04536701).

    more » « less
  4. Background

    In recent years, epidemiological and clinical studies have revealed that depressive disorders can present in early childhood. To clarify the validity and prognostic significance of early childhood‐onset depression, we investigated diagnostic and functional outcomes in later childhood and adolescence.


    A community sample (N = 516) was assessed for psychopathology at ages 3 and 6 using the Preschool Age Psychiatric Assessment. When participants were 9, 12, and 15 years old, children and parents completed the Kiddie Schedule for Affective Disorders and Schizophrenia and measures of symptoms and functioning.


    In models adjusting for covariates, depressed 3/6‐year‐old children were more likely to experience subsequent episodes of depressive disorders and exhibited significantly higher rates of later anxiety disorder, attention deficit hyperactivity disorder, and suicidality compared to children without depressive disorders at age 3/6. Early childhood depression was also associated with higher levels of mother, but not child, reported depressive symptoms at age 15 compared to children without depressive disorders at age 3/6. Finally, depression at age 3/6 predicted lower levels of global and interpersonal functioning and higher rates of treatment at age 15 compared to children without depressive disorders at age 3/6.


    Results support the clinical significance of depression in 3/6‐year‐old children, although further studies with larger samples are needed.

    more » « less
  5. Abstract

    The COVID-19 pandemic has boosted digital health utilization, raising concerns about increased physicians’ after-hours clinical work (work-outside-work”). The surge in patients’ digital messages and additional time spent on work-outside-work by telemedicine providers underscores the need to evaluate the connection between digital health utilization and physicians’ after-hours commitments. We examined the impact on physicians’ workload from two types of digital demands - patients’ messages requesting medical advice (PMARs) sent to physicians’ inbox (inbasket), and telemedicine. Our study included 1716 ambulatory-care physicians in New York City regularly practicing between November 2022 and March 2023. Regression analyses assessed primary and interaction effects of (PMARs) and telemedicine on work-outside-work. The study revealed a significant effect ofPMARs on physicians’ work-outside-work and that this relationship is moderated by physicians’ specialties. Non-primary care physicians or specialists experienced a more pronounced effect than their primary care peers. Analysis of their telemedicine load revealed that primary care physicians received fewerPMARs and spent less time in work-outside-work with more telemedicine. Specialists faced increasedPMARs and did more work-outside-work as telemedicine visits increased which could be due to the difference in patient panels. ReducingPMARvolumes and efficient inbasket management strategies needed to reduce physicians’ work-outside-work. Policymakers need to be cognizant of potential disruptions in physicians carefully balanced workload caused by the digital health services.

    more » « less