An official website of the United States government
ABSTRACT Early identification and intervention often leads to improved life outcomes for individuals with Autism Spectrum Disorder (ASD). However, traditional diagnostic methods are time-consuming, frequently delaying treatment. This study examines the application of machine learning (ML) techniques to 10-question Quantitative Checklist for Autism in Toddlers (QCHAT-10) datasets, aiming to evaluate the predictive value of questionnaire features and overall accuracy metrics across different cultures. We trained models using three distinct datasets from three different countries: Poland, New Zealand, and Saudi Arabia. The New Zealand and Saudi Arabian-trained models were both tested on the Polish dataset, which consisted of diagnostic class labels derived from clinical diagnostic processes. The Decision Tree, Random Forest, and XGBoost models were evaluated, with XGBoost consistently performing best. Feature importance rankings revealed little consistency across models; however, Recursive Feature Elimination (RFE) to select the models with the four most predictive features retained three common features. Both models performed similarly on the Polish test dataset with clinical diagnostic labels, with the New Zealand models with all 13 features achieving an AUROC of 0.94 ± 0.06, and the Saudi Model having an AUROC of 93% ± 6. This compared favorably to the cross-validation analysis of a Polish-trained model, which had an AUROC of 94% ± 5, suggesting that answers to the QCHAT-10 can be predictive of an official autism diagnosis, even across cultures. The New Zealand model with four features had an AUROC of 85% ± 13, and the Saudi model had a similar result of 87% ± 11. These results were somewhat lower than the Polish cross-validation AUROC of 91% ± 5. Adjusting probability thresholds improved sensitivity in some models, which is crucial for screening tools. However, this threshold adjustment often resulted in low levels of specificity during the final testing phase. Our findings suggest that these screening tools may generalize well across cultures; however, more research is needed regarding differences in feature importance for different populations.
more »
« less
Exploiting ChatGPT for Diagnosing Autism-Associated Language Disorders and Identifying Distinct Features
Abstract Diagnosing language disorders associated with autism is a complex and nuanced challenge, often hindered by the subjective nature and variability of traditional assessment methods. Traditional diagnostic methods not only require intensive human effort but also often result in delayed interventions due to their lack of speed and specificity. In this study, we explored the application of ChatGPT, a state-of-the-art large language model, to overcome these obstacles by enhancing diagnostic accuracy and profiling specific linguistic features indicative of autism. Leveraging ChatGPT's advanced natural language processing capabilities, this research aims to streamline and refine the diagnostic process. Specifically, we compared ChatGPT's performance with that of conventional supervised learning models, including BERT, a model acclaimed for its effectiveness in various natural language processing tasks. We showed that ChatGPT substantially outperformed these models, achieving over 13\% improvement in both accuracy and F1-score in a zero-shot learning configuration. This marked enhancement highlights the model’s potential as a superior tool for neurological diagnostics. Additionally, we identified ten distinct features of autism-associated language disorders that vary significantly across different experimental scenarios. These features, which included echolalia, pronoun reversal, and atypical language usage, were crucial for accurately diagnosing ASD and customizing treatment plans. Together, our findings advocate for adopting sophisticated AI tools like ChatGPT in clinical settings to assess and diagnose developmental disorders. Our approach not only promises greater diagnostic precision but also aligns with the goals of personalized medicine, potentially transforming the evaluation landscape for autism and similar neurological conditions.
more »
« less
- Award ID(s):
- 2401748
- PAR ID:
- 10612086
- Publisher / Repository:
- Research Square
- Date Published:
- Format(s):
- Medium: X
- Institution:
- Research Square
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Language in autism is heterogeneous, with a significant proportion of individuals having structural language difficulties and inclusion of language impairment as a specifier under Diagnostic and Statistical Manual of Mental Disorders (5th ed.) criteria for autism. This systematic review asked: What are the reporting patterns of variables pertaining to structural language in autism prior to and after publication of the Diagnostic and Statistical Manual of Mental Disorders (5th ed.)? What norm-referenced assessments does research use to characterize the language abilities of autistic individuals with respect to language impairment? This preregistered review (PROSPERO: CRD42021260394) followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Searches took place in September 2022 and included Linguistics and Language Behavior Abstracts, PsycINFO, PubMed, and the Directory of Open Access Journals. Search terms included three essential concepts: autism, language, and age. Two coders independently screened and evaluated articles. Searches yielded 57 qualifying studies, with mostly consistent reporting practices prior to and after the Diagnostic and Statistical Manual of Mental Disorders (5th ed.). Studies varied in how they defined language groups and in what norm-referenced measures they used. Interpreting research on structural language in autism requires attention to diagnostic and grouping criteria. Although inconsistency in reporting in original studies limited this review, better understanding the available information on structural language in autistic individuals aged 3–21 years may support identification of language needs. Lay abstractUnder the Diagnostic and Statistical Manual of Mental Disorders (5th ed.), language impairment can co-occur with autism. It is not yet clear how research defines, reports, and characterizes structural language abilities of autistic individuals eligible for school-based special education services (aged 3–21 years) in the United States. In the United States, students typically must be formally diagnosed to be eligible for services and supports. However, the quality of diagnosis is only as good as the research evidence on which diagnosis depends. To evaluate evidence quality, we examined how studies of school-aged autistic individuals report assessments of language ability. This systematic review included 57 studies using English language age-referenced assessments used to measure structural language. Findings showed many differences across studies in how language abilities were measured and reported. Also, none of the studies fully reported the variables relevant to characterizing language impairment. Outcomes were similar across versions of the Diagnostic and Statistical Manual of Mental Disorders. Findings indicate that researchers and clinicians should pay attention to reporting diagnostic and grouping criteria. Carefully interpreting research evidence is critical for ensuring that diagnostic criteria and supports are representative of and accessible to autistic individuals and relevant parties.more » « less
-
Tremendous recent literature show that associations between different brain regions, i.e., brain connectivity, provide early symptoms of neurological disorders. Despite significant efforts made for graph neural network (GNN) techniques, their focus on graph nodes makes the state-of-the-art GNN methods not suitable for classifying brain connectivity as graphs where the objective is to characterize disease-relevant network dysfunction patterns on graph links. To address this issue, we propose Multi-resolution Edge Network (MENET) to detect disease-specific connectomic benchmarks with high discrimination power across diagnostic categories. The core of MENET is a novel graph edge-wise transform that we propose, which allows us to capture multi-resolution “connectomic” features. Using a rich set of the connectomic features, we devise a graph learning framework to jointly select discriminative edges and assign diagnostic labels for graphs. Experiments on two real datasets show that MENET accurately predicts diagnostic labels and identify brain connectivities highly associated with neurological disorders such as Alzheimer’s Disease and Attention-Deficit/Hyperactivity Disorder.more » « less
-
Diagnosing Alzheimer's Disease (AD) early and cost-effectively is crucial. Recent advancements in Large Language Models (LLMs) like ChatGPT have made accurate, affordable AD detection feasible. Yet, HIPAA compliance and the challenge of integrating these models into hospital systems limit their use. Addressing these constraints, we introduce ADetectoLocum, an open-source LLM equipped model designed for AD risk detection within hospital environments. This model evaluates AD risk through spontaneous patient speech, enhancing diagnostic processes without external data exchange. Our approach secures local deployment and significantly surpasses previous models in predictive accuracy for AD detection, especially in early-stage identification. ADetectoLocum therefore offers a reliable solution for AD diagnostics in healthcare institutions.more » « less
-
null (Ed.)Detecting security vulnerabilities in software before they are exploited has been a challenging problem for decades. Traditional code analysis methods have been proposed, but are often ineffective and inefficient. In this work, we model software vulnerability detection as a natural language processing (NLP) problem with source code treated as texts, and address the auto-mated software venerability detection with recent advanced deep learning NLP models assisted by transfer learning on written English. For training and testing, we have preprocessed the NIST NVD/SARD databases and built a dataset of over 100,000 files in C programming language with 123 types of vulnerabilities. The extensive experiments generate the best performance of over 93% accuracy in detecting security vulnerabilities.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Posted Content:
- https://doi.org/10.21203/rs.3.rs-4359726/v1
