skip to main content


Title: Comparison of In-Situ and Retrospective Self-Reports on Assessing Hearing Aid Outcomes
Abstract Background Ecological momentary assessment (EMA) is a methodology involving repeated surveys to collect in-situ self-reports that describe respondents' current or recent experiences. Audiology literature comparing in-situ and retrospective self-reports is scarce. Purpose To compare the sensitivity of in-situ and retrospective self-reports in detecting the outcome difference between hearing aid technologies, and to determine the association between in-situ and retrospective self-reports. Research Design An observational study. Study Sample Thirty-nine older adults with hearing loss. Data Collection and Analysis The study was part of a larger clinical trial that compared the outcomes of a prototype hearing aid (denoted as HA1) and a commercially available device (HA2). In each trial condition, participants wore hearing aids for 4 weeks. Outcomes were measured using EMA and retrospective questionnaires. To ensure that the outcome data could be directly compared, the Glasgow Hearing Aid Benefit Profile was administered as an in-situ self-report (denoted as EMA-GHABP) and as a retrospective questionnaire (retro-GHABP). Linear mixed models were used to determine if the EMA- and retro-GHABP could detect the outcome difference between HA1 and HA2. Correlation analyses were used to examine the association between EMA- and retro-GHABP. Results For the EMA-GHABP, HA2 had significantly higher (better) scores than HA1 in the GHABP subscales of benefit, residual disability, and satisfaction (p = 0.029–0.0015). In contrast, the difference in the retro-GHABP score between HA1 and HA2 was significant only in the satisfaction subscale (p = 0.0004). The correlations between the EMA- and retro-GHABP were significant in all subscales (p = 0.0004 to <0.0001). The strength of the association ranged from weak to moderate (r = 0.28–0.58). Finally, the exit interview indicated that 29 participants (74.4%) preferred HA2 over HA1. Conclusion The study suggests that in-situ self-reports collected using EMA could have a higher sensitivity than retrospective questionnaires. Therefore, EMA is worth considering in clinical trials that aim to compare the outcomes of different hearing aid technologies. The weak to moderate association between in-situ and retrospective self-reports suggests that these two types of measures assess different aspects of hearing aid outcomes.  more » « less
Award ID(s):
1838830
NSF-PAR ID:
10309713
Author(s) / Creator(s):
; ; ; ; ;
Date Published:
Journal Name:
Journal of the American Academy of Audiology
Volume:
31
Issue:
10
ISSN:
1050-0545
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    Abstract Background Ecological momentary assessment (EMA) is a methodology involving repeated surveys to collect in situ data that describe respondents' current or recent experiences and related contexts in their natural environments. Audiology literature investigating the test-retest reliability of EMA is scarce. Purpose This article examines the test-retest reliability of EMA in measuring the characteristics of listening contexts and listening experiences. Research Design An observational study. Study Sample Fifty-one older adults with hearing loss. Data Collection and Analysis The study was part of a larger study that examined the effect of hearing aid technologies. The larger study had four trial conditions and outcome was measured using a smartphone-based EMA system. After completing the four trial conditions, participants repeated one of the conditions to examine the EMA test-retest reliability. The EMA surveys contained questions that assessed listening context characteristics including talker familiarity, talker location, and noise location, as well as listening experiences including speech understanding, listening effort, loudness satisfaction, and hearing aid satisfaction. The data from multiple EMA surveys collected by each participant were aggregated in each of the test and retest conditions. Test-retest correlation on the aggregated data was then calculated for each EMA survey question to determine the reliability of EMA. Results At the group level, listening context characteristics and listening experience did not change between the test and retest conditions. The test-retest correlation varied across the EMA questions, with the highest being the questions that assessed talker location (median r = 1.0), reverberation (r = 0.89), and speech understanding (r = 0.85), and the lowest being the items that quantified noise location (median r = 0.63), talker familiarity (r = 0.46), listening effort (r = 0.61), loudness satisfaction (r = 0.60), and hearing aid satisfaction (r = 0.61). Conclusion Several EMA questions yielded appropriate test-retest reliability results. The lower test-retest correlations for some EMA survey questions were likely due to fewer surveys completed by participants and poorly designed questions. Therefore, the present study stresses the importance of using validated questions in EMA. With sufficient numbers of surveys completed by respondents and with appropriately designed survey questions, EMA could have reasonable test-retest reliability in audiology research. 
    more » « less
  2. Introduction: Back pain is one of the most common causes of pain in the United States. Spinal cord stimulation (SCS) is an intervention for patients with chronic back pain (CBP). However, SCS decreases pain in only 58% of patients and relies on self-reported pain scores as outcome measures. An SCS trial is temporarily implanted for seven days and helps to determine if a permanent SCS is needed. Patients that have a >50% reduction in pain from the trial stimulator makes them eligible for permanent implantation. However, self-reported measures reveal little on how mechanisms in the brain are altered. Other measurements of pain intensity, onset, medication, disabilities, depression, and anxiety have been used with machine learning to predict outcomes with accuracies <70%. We aim to predict long-term SCS responders at 6-months using baseline resting EEG and machine learning. Materials and Methods: We obtained 10-minutes of resting electroencephalography (EEG) and pain questionnaires from nine participants with CBP at two time points: 1) pre-trial baseline. 2) Six months after SCS permanent implant surgery. Subjects were designated as high or moderate responders based on the amount of pain relief provided by the long-term (post six months) SCS, and pain scored on a scale of 0-10 with 0 being no pain and 10 intolerable. We used the resting EEG from baseline to predict long-term treatment outcome. Resting EEG data was fed through a pipeline for classification and to map dipole sources. EEG signals were preprocessed using the EEGLAB toolbox. Independent component analysis and dipole fitting were used to linearly unmix the signal and to map dipole sources from the brain. Spectral analysis was performed to obtain the frequency distribution of the signal. Each power band, delta (1-4 Hz), theta (4-8 Hz), alpha (8-13 Hz), beta (13-30 Hz), and gamma (30-100 Hz), as well as the entire spectrum (1-100 Hz), were used for classification. Furthermore, dipole sources were ranked based on classification feature weights to determine the significance of specific regions in the brain. We used support vector machines to predict pain outcomes. Results and Discussion: We found higher frequency powerbands provide overall classification performance of 88.89%. Differences in power are seen between moderate and high responders in both the frontal and parietal regions for theta, alpha, beta, and the entire spectrum (Fig.1). This can potentially be used to predict patient response to SCS. Conclusions: We found evidence of decreased power in theta, alpha, beta, and entire spectrum in the anterior regions of the parietal cortex and posterior regions of the frontal cortex between moderate and high responders, which can be used for predicting treatment outcomes in long-term pain relief from SCS. Long-term treatment outcome prediction using baseline EEG data has the potential to contribute to decision making in terms of permanent surgery, forgo trial periods, and improve clinical efficiency by beginning to understand the mechanism of action of SCS in the human brain. 
    more » « less
  3. Abstract Background Ecological momentary assessment (EMA) often requires respondents to complete surveys in the moment to report real-time experiences. Because EMA may seem disruptive or intrusive, respondents may not complete surveys as directed in certain circumstances. Purpose This article aims to determine the effect of environmental characteristics on the likelihood of instances where respondents do not complete EMA surveys (referred to as survey incompletion), and to estimate the impact of survey incompletion on EMA self-report data. Research Design An observational study. Study Sample Ten adults hearing aid (HA) users. Data Collection and Analysis Experienced, bilateral HA users were recruited and fit with study HAs. The study HAs were equipped with real-time data loggers, an algorithm that logged the data generated by HAs (e.g., overall sound level, environment classification, and feature status including microphone mode and amount of gain reduction). The study HAs were also connected via Bluetooth to a smartphone app, which collected the real-time data logging data as well as presented the participants with EMA surveys about their listening environments and experiences. The participants were sent out to wear the HAs and complete surveys for 1 week. Real-time data logging was triggered when participants completed surveys and when participants ignored or snoozed surveys. Data logging data were used to estimate the effect of environmental characteristics on the likelihood of survey incompletion, and to predict participants' responses to survey questions in the instances of survey incompletion. Results Across the 10 participants, 715 surveys were completed and survey incompletion occurred 228 times. Mixed effects logistic regression models indicated that survey incompletion was more likely to happen in the environments that were less quiet and contained more speech, noise, and machine sounds, and in the environments wherein directional microphones and noise reduction algorithms were enabled. The results of survey response prediction further indicated that the participants could have reported more challenging environments and more listening difficulty in the instances of survey incompletion. However, the difference in the distribution of survey responses between the observed responses and the combined observed and predicted responses was small. Conclusion The present study indicates that EMA survey incompletion occurs systematically. Although survey incompletion could bias EMA self-report data, the impact is likely to be small. 
    more » « less
  4. Abstract STUDY QUESTION To what extent is exposure to cellular telephones associated with male fertility? SUMMARY ANSWER Overall, we found little association between carrying a cell phone in the front pants pocket and male fertility, although among leaner men (BMI <25 kg/m2), carrying a cell phone in the front pants pocket was associated with lower fecundability. WHAT IS KNOWN ALREADY Some studies have indicated that cell phone use is associated with poor semen quality, but the results are conflicting. STUDY DESIGN, SIZE, DURATION Two prospective preconception cohort studies were conducted with men in Denmark (n = 751) and in North America (n = 2349), enrolled and followed via the internet from 2012 to 2020. PARTICIPANTS/MATERIALS, SETTING, METHODS On the baseline questionnaire, males reported their hours/day of carrying a cell phone in different body locations. We ascertained time to pregnancy via bi-monthly follow-up questionnaires completed by the female partner for up to 12 months or until reported conception. We used proportional probabilities regression models to estimate fecundability ratios (FRs) and 95% confidence intervals (CIs) for the association between male cell phone habits and fecundability, focusing on front pants pocket exposure, within each cohort separately and pooling across the cohorts using a fixed-effect meta-analysis. In a subset of participants, we examined selected semen parameters (semen volume, sperm concentration and sperm motility) using a home-based semen testing kit. MAIN RESULTS AND THE ROLE OF CHANCE There was little overall association between carrying a cell phone in a front pants pocket and fecundability: the FR for any front pants pocket exposure versus none was 0.94 (95% CI: 0.0.83–1.05). We observed an inverse association between any front pants pocket exposure and fecundability among men whose BMI was <25 kg/m2 (FR = 0.72, 95% CI: 0.59–0.88) but little association among men whose BMI was ≥25 kg/m2 (FR = 1.05, 95% CI: 0.90–1.22). There were few consistent associations between cell phone exposure and semen volume, sperm concentration, or sperm motility. LIMITATIONS, REASONS FOR CAUTION Exposure to radiofrequency radiation from cell phones is subject to considerable non-differential misclassification, which would tend to attenuate the estimates for dichotomous comparisons and extreme exposure categories (e.g. exposure 8 vs. 0 h/day). Residual confounding by occupation or other unknown or poorly measured factors may also have affected the results. WIDER IMPLICATIONS OF THE FINDINGS Overall, there was little association between carrying one’s phone in the front pants pocket and fecundability. There was a moderate inverse association between front pants pocket cell phone exposure and fecundability among men with BMI <25 kg/m2, but not among men with BMI ≥25 kg/m2. Although several previous studies have indicated associations between cell phone exposure and lower sperm motility, we found few consistent associations with any semen quality parameters. STUDY FUNDING/COMPETING INTEREST(S) The study was funded by the National Institutes of Health, grant number R03HD090315. In the last 3 years, PRESTO has received in-kind donations from Sandstone Diagnostics (for semen kits), Swiss Precision Diagnostics (home pregnancy tests), Kindara.com (fertility app), and FertilityFriend.com (fertility app). Dr. L.A.W. is a fibroid consultant for AbbVie, Inc. Dr. H.T.S. reports that the Department of Clinical Epidemiology is involved in studies with funding from various companies as research grants to and administered by Aarhus University. None of these studies are related to the current study. Dr. M.L.E. is an advisor to Sandstone Diagnostics, Ro, Dadi, Hannah, and Underdog. Dr. G.J.S. holds ownership in Sandstone Diagnostics Inc., developers of the Trak Male Fertility Testing System. In addition, Dr. G.J.S. has a patent pending related to Trak Male Fertility Testing System issued. TRIAL REGISTRATION NUMBER N/A 
    more » « less
  5. Abstract: 100 words Jurors are increasingly exposed to scientific information in the courtroom. To determine whether providing jurors with gist information would assist in their ability to make well-informed decisions, the present experiment utilized a Fuzzy Trace Theory-inspired intervention and tested it against traditional legal safeguards (i.e., judge instructions) by varying the scientific quality of the evidence. The results indicate that jurors who viewed high quality evidence rated the scientific evidence significantly higher than those who viewed low quality evidence, but were unable to moderate the credibility of the expert witness and apply damages appropriately resulting in poor calibration. Summary: <1000 words Jurors and juries are increasingly exposed to scientific information in the courtroom and it remains unclear when they will base their decisions on a reasonable understanding of the relevant scientific information. Without such knowledge, the ability of jurors and juries to make well-informed decisions may be at risk, increasing chances of unjust outcomes (e.g., false convictions in criminal cases). Therefore, there is a critical need to understand conditions that affect jurors’ and juries’ sensitivity to the qualities of scientific information and to identify safeguards that can assist with scientific calibration in the courtroom. The current project addresses these issues with an ecologically valid experimental paradigm, making it possible to assess causal effects of evidence quality and safeguards as well as the role of a host of individual difference variables that may affect perceptions of testimony by scientific experts as well as liability in a civil case. Our main goal was to develop a simple, theoretically grounded tool to enable triers of fact (individual jurors) with a range of scientific reasoning abilities to appropriately weigh scientific evidence in court. We did so by testing a Fuzzy Trace Theory-inspired intervention in court, and testing it against traditional legal safeguards. Appropriate use of scientific evidence reflects good calibration – which we define as being influenced more by strong scientific information than by weak scientific information. Inappropriate use reflects poor calibration – defined as relative insensitivity to the strength of scientific information. Fuzzy Trace Theory (Reyna & Brainerd, 1995) predicts that techniques for improving calibration can come from presentation of easy-to-interpret, bottom-line “gist” of the information. Our central hypothesis was that laypeople’s appropriate use of scientific information would be moderated both by external situational conditions (e.g., quality of the scientific information itself, a decision aid designed to convey clearly the “gist” of the information) and individual differences among people (e.g., scientific reasoning skills, cognitive reflection tendencies, numeracy, need for cognition, attitudes toward and trust in science). Identifying factors that promote jurors’ appropriate understanding of and reliance on scientific information will contribute to general theories of reasoning based on scientific evidence, while also providing an evidence-based framework for improving the courts’ use of scientific information. All hypotheses were preregistered on the Open Science Framework. Method Participants completed six questionnaires (counterbalanced): Need for Cognition Scale (NCS; 18 items), Cognitive Reflection Test (CRT; 7 items), Abbreviated Numeracy Scale (ABS; 6 items), Scientific Reasoning Scale (SRS; 11 items), Trust in Science (TIS; 29 items), and Attitudes towards Science (ATS; 7 items). Participants then viewed a video depicting a civil trial in which the defendant sought damages from the plaintiff for injuries caused by a fall. The defendant (bar patron) alleged that the plaintiff (bartender) pushed him, causing him to fall and hit his head on the hard floor. Participants were informed at the outset that the defendant was liable; therefore, their task was to determine if the plaintiff should be compensated. Participants were randomly assigned to 1 of 6 experimental conditions: 2 (quality of scientific evidence: high vs. low) x 3 (safeguard to improve calibration: gist information, no-gist information [control], jury instructions). An expert witness (neuroscientist) hired by the court testified regarding the scientific strength of fMRI data (high [90 to 10 signal-to-noise ratio] vs. low [50 to 50 signal-to-noise ratio]) and gist or no-gist information both verbally (i.e., fairly high/about average) and visually (i.e., a graph). After viewing the video, participants were asked if they would like to award damages. If they indicated yes, they were asked to enter a dollar amount. Participants then completed the Positive and Negative Affect Schedule-Modified Short Form (PANAS-MSF; 16 items), expert Witness Credibility Scale (WCS; 20 items), Witness Credibility and Influence on damages for each witness, manipulation check questions, Understanding Scientific Testimony (UST; 10 items), and 3 additional measures were collected, but are beyond the scope of the current investigation. Finally, participants completed demographic questions, including questions about their scientific background and experience. The study was completed via Qualtrics, with participation from students (online vs. in-lab), MTurkers, and non-student community members. After removing those who failed attention check questions, 469 participants remained (243 men, 224 women, 2 did not specify gender) from a variety of racial and ethnic backgrounds (70.2% White, non-Hispanic). Results and Discussion There were three primary outcomes: quality of the scientific evidence, expert credibility (WCS), and damages. During initial analyses, each dependent variable was submitted to a separate 3 Gist Safeguard (safeguard, no safeguard, judge instructions) x 2 Scientific Quality (high, low) Analysis of Variance (ANOVA). Consistent with hypotheses, there was a significant main effect of scientific quality on strength of evidence, F(1, 463)=5.099, p=.024; participants who viewed the high quality evidence rated the scientific evidence significantly higher (M= 7.44) than those who viewed the low quality evidence (M=7.06). There were no significant main effects or interactions for witness credibility, indicating that the expert that provided scientific testimony was seen as equally credible regardless of scientific quality or gist safeguard. Finally, for damages, consistent with hypotheses, there was a marginally significant interaction between Gist Safeguard and Scientific Quality, F(2, 273)=2.916, p=.056. However, post hoc t-tests revealed significantly higher damages were awarded for low (M=11.50) versus high (M=10.51) scientific quality evidence F(1, 273)=3.955, p=.048 in the no gist with judge instructions safeguard condition, which was contrary to hypotheses. The data suggest that the judge instructions alone are reversing the pattern, though nonsignificant, those who received the no gist without judge instructions safeguard awarded higher damages in the high (M=11.34) versus low (M=10.84) scientific quality evidence conditions F(1, 273)=1.059, p=.30. Together, these provide promising initial results indicating that participants were able to effectively differentiate between high and low scientific quality of evidence, though inappropriately utilized the scientific evidence through their inability to discern expert credibility and apply damages, resulting in poor calibration. These results will provide the basis for more sophisticated analyses including higher order interactions with individual differences (e.g., need for cognition) as well as tests of mediation using path analyses. [References omitted but available by request] Learning Objective: Participants will be able to determine whether providing jurors with gist information would assist in their ability to award damages in a civil trial. 
    more » « less