More Like this

1. Behavioral Intervention for Adults With Autism on Distribution of Attention in Triadic Conversations: A/B-Tested Pre-Post Study

   https://doi.org/10.2196/55339  

   Tepencelik, Onur Necip; Wei, Wenchuan; Luo, Mirabel; Cosman, Pamela; Dey, Sujit (January 2024, JMIR Formative Research)

   BackgroundCross-neurotype differences in social communication patterns contribute to high unemployment rates among adults with autism. Adults with autism can be unsuccessful in job searches or terminated from employment due to mismatches between their social attention behaviors and society’s expectations on workplace communication. ObjectiveWe propose a behavioral intervention concerning distribution of attention in triadic (three-way) conversations. Specifically, the objective is to determine whether providing personalized feedback to each individual with autism based on an analysis of their attention distribution behavior during an initial conversation session would cause them to modify their orientation behavior in a subsequent conversation session. MethodsOur system uses an unobtrusive head orientation estimation model to track the focus of attention of each individual. Head orientation sequences from a conversation session are analyzed based on five statistical domains (eg, maximum exclusion duration and average contact duration) representing different types of attention distribution behavior. An intervention is provided to a participant if they exceeded the nonautistic average for that behavior by at least 2 SDs. The intervention uses data analysis and video modeling along with a constructive discussion about the targeted behaviors. Twenty-four individuals with autism with no intellectual disabilities participated in the study. The participants were divided into test and control groups of 12 participants each. ResultsBased on their attention distribution behavior in the initial conversation session, 11 of the 12 participants in the test group received an intervention in at least one domain. Of the 11 participants who received the intervention, 10 showed improvement in at least one domain on which they received feedback. Independent t tests for larger test groups (df>15) confirmed that the group improvements are statistically significant compared with the corresponding controls (P<.05). Crawford-Howell t tests confirmed that 78% of the interventions resulted in significant improvements when compared individually against corresponding controls (P<.05). Additional t tests comparing the first conversation sessions of the test and control groups and comparing the first and second conversation sessions of the control group resulted in nonsignificant differences, pointing to the intervention being the main effect behind the behavioral changes displayed by the test group, as opposed to confounding effects or group differences. ConclusionsOur proposed behavioral intervention offers a useful framework for practicing social attention behavior in multiparty conversations that are common in social and professional settings. 

   more » « less
2. Real-time automated billing for tobacco treatment: developing and validating a scalable machine learning approach

   https://doi.org/10.1093/jamiaopen/ooaf039  

   Baughman, Derek J; Qassem, Layth; Sulieman, Lina; Matheny, Michael E; Fabbri, Daniel; Tindle, Hilary A; Goodman, Aubrey Cole; Nelson, Scott D; Wright, Adam (May 2025, JAMIA Open)

   Abstract ObjectivesTo develop CigStopper, a real-time, automated medical billing prototype designed to identify eligible tobacco cessation care codes, thereby reducing administrative workload while improving billing accuracy. Materials and MethodsChatGPT prompt engineering generated a synthetic corpus of physician-style clinical notes categorized for CPT codes 99406/99407. Practicing clinicians annotated the dataset to train multiple machine learning (ML) models focused on accurately predicting billing code eligibility. ResultsDecision tree and random forest models performed best. Mean performance across all models: PRC AUC = 0.857, F1 score = 0.835. Generalizability testing on deidentified notes confirmed that tree-based models performed best. DiscussionCigStopper shows promise for streamlining manual billing inefficiencies that hinder tobacco cessation care. ML methods lay the groundwork for clinical implementation based on good performance using synthetic data. Automating high-volume, low-value tasks simplify complexities in a multi-payer system and promote financial sustainability for healthcare practices. ConclusionCigStopper validates foundational methods for automating the discernment of appropriate billing codes for eligible smoking cessation counseling care. 

   more » « less
3. Large Language Model–Based Responses to Patients’ In-Basket Messages

   https://doi.org/10.1001/jamanetworkopen.2024.22399  

   Small, William R; Wiesenfeld, Batia; Brandfield-Harvey, Beatrix; Jonassen, Zoe; Mandal, Soumik; Stevens, Elizabeth R; Major, Vincent J; Lostraglio, Erin; Szerencsy, Adam; Jones, Simon; et al (July 2024, JAMA Network Open)

   ImportanceVirtual patient-physician communications have increased since 2020 and negatively impacted primary care physician (PCP) well-being. Generative artificial intelligence (GenAI) drafts of patient messages could potentially reduce health care professional (HCP) workload and improve communication quality, but only if the drafts are considered useful. ObjectivesTo assess PCPs’ perceptions of GenAI drafts and to examine linguistic characteristics associated with equity and perceived empathy. Design, Setting, and ParticipantsThis cross-sectional quality improvement study tested the hypothesis that PCPs’ ratings of GenAI drafts (created using the electronic health record [EHR] standard prompts) would be equivalent to HCP-generated responses on 3 dimensions. The study was conducted at NYU Langone Health using private patient-HCP communications at 3 internal medicine practices piloting GenAI. ExposuresRandomly assigned patient messages coupled with either an HCP message or the draft GenAI response. Main Outcomes and MeasuresPCPs rated responses’ information content quality (eg, relevance), using a Likert scale, communication quality (eg, verbosity), using a Likert scale, and whether they would use the draft or start anew (usable vs unusable). Branching logic further probed for empathy, personalization, and professionalism of responses. Computational linguistics methods assessed content differences in HCP vs GenAI responses, focusing on equity and empathy. ResultsA total of 16 PCPs (8 [50.0%] female) reviewed 344 messages (175 GenAI drafted; 169 HCP drafted). Both GenAI and HCP responses were rated favorably. GenAI responses were rated higher for communication style than HCP responses (mean [SD], 3.70 [1.15] vs 3.38 [1.20];P = .01,U = 12 568.5) but were similar to HCPs on information content (mean [SD], 3.53 [1.26] vs 3.41 [1.27];P = .37;U = 13 981.0) and usable draft proportion (mean [SD], 0.69 [0.48] vs 0.65 [0.47],P = .49,t = −0.6842). Usable GenAI responses were considered more empathetic than usable HCP responses (32 of 86 [37.2%] vs 13 of 79 [16.5%]; difference, 125.5%), possibly attributable to more subjective (mean [SD], 0.54 [0.16] vs 0.31 [0.23];P &lt; .001; difference, 74.2%) and positive (mean [SD] polarity, 0.21 [0.14] vs 0.13 [0.25];P = .02; difference, 61.5%) language; they were also numerically longer (mean [SD] word count, 90.5 [32.0] vs 65.4 [62.6]; difference, 38.4%), but the difference was not statistically significant (P = .07) and more linguistically complex (mean [SD] score, 125.2 [47.8] vs 95.4 [58.8];P = .002; difference, 31.2%). ConclusionsIn this cross-sectional study of PCP perceptions of an EHR-integrated GenAI chatbot, GenAI was found to communicate information better and with more empathy than HCPs, highlighting its potential to enhance patient-HCP communication. However, GenAI drafts were less readable than HCPs’, a significant concern for patients with low health or English literacy. 

   more » « less
4. Communicating about chemicals in cigarette smoke: impact on knowledge and misunderstanding

   https://doi.org/10.1136/tobaccocontrol-2018-054863  

   Lazard, Allison J; Byron, M Justin; Peters, Ellen; Brewer, Noel T (January 2019, Tobacco Control)

   null (Ed.)

   Background The USA must publicly share information about harmful and potentially harmful constituents (chemicals) in tobacco products. We sought to understand whether webpages with chemical information are “understandable and not misleading to a lay person.” Methods Participants were a national probability sample of US adults and adolescents ( n =1441, 18% smokers). In an online experiment, we randomly assigned participants to view one of the developed webpages (chemical names only, names with quantity ranges, names with visual risk indicators) or no webpage in phase one (between subjects). Participants completed a survey assessing knowledge, misunderstanding, perceived likelihood, perceived severity of health effects from smoking and quit intentions (smokers only). In phase two (within subjects), participants viewed all three webpage formats and reported webpage perceptions (clarity, usability, usefulness) and perceived impact (affect, elaboration, perceived effectiveness). Results In phase one, viewing any webpage led to more knowledge of chemicals (48%–54% vs 28% no webpage, p s<0.001) and health harms (77% vs 67% no webpage, p s<0.001). When exposed to any webpage, 5%–23% endorsed misunderstandings that some cigarettes are safer than others. Webpage format did not affect knowledge or reduce misunderstandings. Viewing any webpage led to higher perceived likelihood of experiencing health effects from smoking ( p < 0.001) and, among smokers, greater intentions to quit smoking ( p =0.04). In phase two, where participants viewed all formats, a visual risk indicator led to the highest perceived impact. Conclusions Knowledge of chemicals and health effects can increase after viewing a website. Yet, websites may not correct the misunderstanding that some cigarettes are safer. 

   more » « less
5. The First Generative AI Prompt-A-Thon in Healthcare: A Novel Approach to Workforce Engagement with a Private Instance of ChatGPT

   https://doi.org/10.1371/journal.pdig.0000394  

   Small, William R; Malhotra, Kiran; Major, Vincent J; Wiesenfeld, Batia; Lewis, Marisa; Grover, Himanshu; Tang, Huming; Banerjee, Arnab; Jabbour, Michael J; Aphinyanaphongs, Yindalon; et al (July 2024, PLOS Digital Health)

   Hastings, Janna (Ed.)

   BackgroundHealthcare crowdsourcing events (e.g. hackathons) facilitate interdisciplinary collaboration and encourage innovation. Peer-reviewed research has not yet considered a healthcare crowdsourcing event focusing on generative artificial intelligence (GenAI), which generates text in response to detailed prompts and has vast potential for improving the efficiency of healthcare organizations. Our event, the New York University Langone Health (NYULH) Prompt-a-thon, primarily sought to inspire and build AI fluency within our diverse NYULH community, and foster collaboration and innovation. Secondarily, we sought to analyze how participants’ experience was influenced by their prior GenAI exposure and whether they received sample prompts during the workshop. MethodsExecuting the event required the assembly of an expert planning committee, who recruited diverse participants, anticipated technological challenges, and prepared the event. The event was composed of didactics and workshop sessions, which educated and allowed participants to experiment with using GenAI on real healthcare data. Participants were given novel “project cards” associated with each dataset that illuminated the tasks GenAI could perform and, for a random set of teams, sample prompts to help them achieve each task (the public repository of project cards can be found athttps://github.com/smallw03/NYULH-Generative-AI-Prompt-a-thon-Project-Cards). Afterwards, participants were asked to fill out a survey with 7-point Likert-style questions. ResultsOur event was successful in educating and inspiring hundreds of enthusiastic in-person and virtual participants across our organization on the responsible use of GenAI in a low-cost and technologically feasible manner. All participants responded positively, on average, to each of the survey questions (e.g., confidence in their ability to use and trust GenAI). Critically, participants reported a self-perceived increase in their likelihood of using and promoting colleagues’ use of GenAI for their daily work. No significant differences were seen in the surveys of those who received sample prompts with their project task descriptions ConclusionThe first healthcare Prompt-a-thon was an overwhelming success, with minimal technological failures, positive responses from diverse participants and staff, and evidence of post-event engagement. These findings will be integral to planning future events at our institution, and to others looking to engage their workforce in utilizing GenAI. 

   more » « less