skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


  • NSF Public Access
  • Search Results
  • Online-computer-mediated interviews and observations: Overcoming challenges and establishing best practices in a human-AI teaming context
Title: Online-computer-mediated interviews and observations: Overcoming challenges and establishing best practices in a human-AI teaming context
Prior research has established the feasibility of conducting online interviews and observations, yet there is limited guidance in how to interact with participants when conducting fully mediated research with screen-sharing and video. This study, conducted during early phases of COVID-19, included 15 volunteer tweet-annotators working with an emergency response organization. This method contribution uses cues-related and surveillance theories to reveal challenges and best practices when asking research participants to share their screen, be on video, and participate in a multiple-interview study. The findings suggest that researchers conducting online-mediated research should be prepared to provide technical support for the devices and interfaces participants use during the study, find ways to “see” beyond what is on the mediated screen, and consider ethical issues not often discussed. In addition to these findings, an output of this research is two brief training videos useful for other researchers embarking on conducting fully mediated research.  more » « less
Award ID(s):
2029692
PAR ID:
10230410
Author(s) / Creator(s):
; ; ; ; ; ; ;
Editor(s):
Bui, T.
Date Published:
Journal Name:
Proceedings of the Annual Hawaii International Conference on System Sciences
ISSN:
1060-3425
Page Range / eLocation ID:
2896-2905
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; (, 19th Web for All Conference (W4A’22))
    Researchers have adopted remote methods, such as online surveys and video conferencing, to overcome challenges in conducting in-person usability testing, such as participation, user representation, and safety. However, remote user evaluation on hardware testbeds is limited, especially for blind participants, as such methods restrict access to observations of user interactions. We employ smart glasses in usability testing with blind people and share our lessons from a case study conducted in blind participants’ homes (N=12), where the experimenter can access participants’ activities via dual video conferencing: a third-person view via a laptop camera and a first-person view via smart glasses worn by the participant. We show that smart glasses hold potential for observing participants’ interactions with smartphone testbeds remotely; on average 58.7% of the interactions were fully captured via the first-person view compared to 3.7% via the third-person. However, this gain is not uniform across participants as it is susceptible to head movements orienting the ear towards a sound source, which highlights the need for a more inclusive camera form factor. We also share our lessons learned when it comes to dealing with lack of screen readers, a rapidly draining battery, and Internet connectivity in remote studies with blind participants. 
    more » « less
  2. ; ; ; ; (, Proceedings of the ACM on Human-Computer Interaction)
    null (Ed.)
    The methods in which we study the online experiences of adolescents should be evidence-based and informed by youth. This is especially true when studying sensitive topics, such as the online risk behaviors of minors. We directly engaged 20 adolescents (ages 12-18) in the co-design of two different research methodologies (i.e., diary studies and analyzing social media trace data) for conducting adolescent online safety research. We also interviewed 13 of their parents to understand their perspectives. Overall, teens wanted to share their personal experiences and benefit society, while parents wanted researchers to tackle a topic that they felt was a prevalent problem for teens. Yet, they both had significant concerns regarding data privacy of the sensitive disclosures made by teens during such studies. Teens' feared getting in trouble. Participants emphasized the importance of developing a trusting relationship with the researcher to overcome these concerns. Participants also saw the potential for using the research study as a tool for risk-reporting and mitigation, where researchers could act as liaisons between the teens and other parties (e.g., counselors, law enforcement, parents) to share pertinent risk details and facilitate resources or even help teens directly by giving them strategies for mitigating online risks they encountered during the study. Our research delves into important ethical considerations for conducting risk-focused research with adolescents and uncovers the critical need for designing risk-based research for youth protection. We provide researchers with heuristic guidelines for conducting ethical research with vulnerable populations (i.e., adolescents) and keeping participants safe while doing so. 
    more » « less
  3. ; ; (, The Qualitative Report)
    null (Ed.)
    In this paper, we share our approach and the process for qualitative analysis of online video data recorded during an after-school robotics program that emphasized computational thinking (CT). Online research strategies may be necessary for various reasons such as when working with a geographically distributed research team, when conducting research with students in an online program, or when resources are inaccessible due to campus closures like those experienced during the COVID-19 pandemic. We followed a three-stage process during qualitative analysis of the videos that included planning and setup, online analysis of videos, and structural coding of memos to explore patterns across the data. Analysis was conducted with a combination of technologies including Google Drive for collaborative coding online and NVivo to collate and summarize findings. The methods and process we describe are readily applicable to other research studies that include video as part of the data set. 
    more » « less
  4. ; ; ; ; (, Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI '22))
    Deaf and Hard-of-Hearing (DHH) users face accessibility challenges during in-person and remote meetings. While emerging use of applications incorporating automatic speech recognition (ASR) is promising, more user-interface and user-experience research is needed. While co-design methods could elucidate designs for such applications, COVID-19 has interrupted in-person research. This study describes a novel methodology for conducting online co-design workshops with 18 DHH and hearing participant pairs to investigate ASR-supported mobile and videoconferencing technologies along two design dimensions: Correcting errors in ASR output and implementing notification systems for influencing speaker behaviors. Our methodological findings include an analysis of communication modalities and strategies participants used, use of an online collaborative whiteboarding tool, and how participants reconciled differences in ideas. Finally, we present guidelines for researchers interested in online DHH co-design methodologies, enabling greater geographically diversity among study participants even beyond the current pandemic. 
    more » « less
  5. ; ; (, PLOS ONE)
    Triberti, Stefano (Ed.)
    Communication of science through online media has become a primary means of disseminating and connecting science with a public audience. However, online media can come in many forms and stories of scientific discovery can be told by many individuals. We tested whether the relationship of a spokesperson to the science story being told (i.e., the narrative perspective) influences how people react and respond to online science media. We created five video stimuli that fell into three treatments: a scientist presenting their own research (male or female), a third-party summarizing research (male or female), and an infographic-like video with no on-screen presenter. Each of these videos presented the same fabricated science story about the discovery of a new ant species (Formicidae). We used Qualtrics to administer and obtain survey responses from 515 participants (~100 per video). Participants were randomly assigned to one of the videos and after viewing the stimulus answered questions assessing their perceptions of the video (trustworthiness and enjoyment), the spokesperson (trustworthiness and competence), scientists in general (competence and warmth), and attitudes towards the research topic and funding. Participants were also asked to recall what they had seen and heard. We determined that when participants watched a video in which a scientist presented their own research, participants perceived the spokesperson as having more expertise than a third-party presenter, and as more trustworthy and having more expertise than the no-spokesperson stimuli. Viewing a scientist presenting their own work also humanized the research, with participants more often including a person in their answer to the recall question. Overall, manipulating the narrative perspective of the source of a single online video communication effort is effective at impacting immediate objective outcomes related to spokesperson perceptions, but whether those objectives can positively influence long-term goals requires more investigation. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email