More Like this

1. Ensemble Machine Learning for Alzheimer’s disease Classification from Retinal Vasculature

   Lin, Hely; Fang, Ruogu (October 2021, Biomedical Engineering Society Annual Meeting)

   null (Ed.)

   Introduction: Alzheimer’s disease (AD) causes progressive irreversible cognitive decline and is the leading cause of dementia. Therefore, a timely diagnosis is imperative to maximize neurological preservation. However, current treatments are either too costly or limited in availability. In this project, we explored using retinal vasculature as a potential biomarker for early AD diagnosis. This project focuses on stage 3 of a three-stage modular machine learning pipeline which consisted of image quality selection, vessel map generation, and classification [1]. The previous model only used support vector machine (SVM) to classify AD labels which limited its accuracy to 82%. In this project, random forest and gradient boosting were added and, along with SVM, combined into an ensemble classifier, raising the classification accuracy to 89%. Materials and Methods: Subjects classified as AD were those who were diagnosed with dementia in “Dementia Outcome: Alzheimer’s disease” from the UK Biobank Electronic Health Records. Five control groups were chosen with a 5:1 ratio of control to AD patients where the control patients had the same age, gender, and eye side image as the AD patient. In total, 122 vessel images from each group (AD and control) were used. The vessel maps were then segmented from fundus images through U-net. A t-test feature selection was first done on the training folds and the selected features was fed into the classifiers with a p-value threshold of 0.01. Next, 20 repetitions of 5-fold cross validation were performed where the hyperparameters were solely tuned on the training data. An ensemble classifier consisting of SVM, gradient boosting tree, and random forests was built and the final prediction was made through majority voting and evaluated on the test set. Results and Discussion: Through ensemble classification, accuracy increased by 4-12% relative to the individual classifiers, precision by 9-15%, sensitivity by 2-9%, specificity by at least 9-16%, and F1 score by 712%. Conclusions: Overall, a relatively high classification accuracy was achieved using machine learning ensemble classification with SVM, random forest, and gradient boosting. Although the results are very promising, a limitation of this study is that the requirement of needing images of sufficient quality decreased the amount of control parameters that can be implemented. However, through retinal vasculature analysis, this project shows machine learning’s high potential to be an efficient, more cost-effective alternative to diagnosing Alzheimer’s disease. Clinical Application: Using machine learning for AD diagnosis through retinal images will make screening available for a broader population by being more accessible and cost-efficient. Mobile device based screening can also be enabled at primary screening in resource-deprived regions. It can provide a pathway for future understanding of the association between biomarkers in the eye and brain. 

   more » « less
2. Zero-shot Building Attribute Extraction from Large-Scale Vision and Language Models.

   Pan, Fei; Jeon, S.; Wang, B.; McKenna, F.; Yu, S.X. (January 2024, Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision)

   Existing building recognition methods, exemplified by BRAILS, utilize supervised learning to extract information from satellite and street-view images for classification and segmentation. However, each task module requires human-annotated data, hindering the scalability and robustness to regional variations and annotation imbalances. In response, we propose a new zero-shot workflow for building attribute extraction that utilizes large-scale vision and language models to mitigate reliance on external annotations. The proposed workflow contains two key components: image-level captioning and segment-level captioning for the building images based on the vocabularies pertinent to structural and civil engineering. These two components generate descriptive captions by computing feature representations of the image and the vocabularies, and facilitating a semantic match between the visual and textual representations. Consequently, our framework offers a promising avenue to enhance AI-driven captioning for building attribute extraction in the structural and civil engineering domains, ultimately reducing reliance on human annotations while bolstering performance and adaptability. 

   more » « less
3. INSIGHT: Explainable Weakly-Supervised Medical Image Analysis

   Zhang, W; Chen, J; Kanan, C (August 2025, Proc. Machine Learning for Healthcare Conference (MLHC))

   Due to their large sizes, volumetric scans and whole-slide pathology images (WSIs) are often processed by extracting embeddings from local regions and then an aggregator makes predictions from this set. However, current methods require post-hoc visualization techniques (e.g., Grad-CAM) and often fail to localize small yet clinically crucial details. To address these limitations, we introduce INSIGHT, a novel weakly-supervised aggregator that integrates heatmap generation as an inductive bias. Starting from pre-trained feature maps, INSIGHT employs a detection module with small convolutional kernels to capture fine details and a context module with a broader receptive field to suppress local false positives. The resulting internal heatmap highlights diagnostically relevant regions. On CT and WSI benchmarks, INSIGHT achieves state-of-the-art classification results and high weakly-labeled semantic segmentation performance. 

   more » « less
4. A Topological-Attention ConvLSTM Network and Its Application to EM Images

   https://doi.org/10.1007/978-3-030-87193-2_21  

   Yang, Jiaqi; Hu, Xiaoling; Chen, Chao; Tsai, Chialing (October 2021, Medical Image Computing and Computer Assisted Intervention (MICCAI))

   Structural accuracy of segmentation is important for fine-scale structures in biomedical images. We propose a novel Topological-Attention ConvLSTM Network (TACLNet) for 3D anisotropic image segmentation with high structural accuracy. We adopt ConvLSTM to leverage contextual information from adjacent slices while achieving high efficiency. We propose a Spatial Topological-Attention (STA) module to effectively transfer topologically critical information across slices. Furthermore, we propose an Iterative Topological-Attention (ITA) module that provides a more stable topologically critical map for segmentation. Quantitative and qualitative results show that our proposed method outperforms various baselines in terms of topology-aware evaluation metrics. 

   more » « less
5. Machine Learning for Parkinson’s Disease Diagnosis Using Fundus Eye Images

   Diaz, Maximillian; Tian, Jianqiao; Fang, Ruogu (December 2020, Annual Meeting of Radiology Society of North America (RSNA))

   Purpose: Parkinson’s Disease (PD) is the second most common form of neural degeneration and defined by the decay of dopaminergic cells in the substantia nigra. The current standard for diagnosing PD occurs once 80% of dopaminergic cells have decayed. The degradation of these cells has been shown to create thinning of the retina walls and retina microvasculature. This work serves to find machine learning techniques to provide PD diagnosis using non-invasive fundus eye images. Materials and Methods: Two age and gender matched datasets where constructed using data from the UK Biobank (UKB) and data collected at the University of Florida (UF). The first dataset consists of 476 fundus eye images, 238 CN and 238 PD, sourced entirely from the UKB database. The second dataset, UF-UKB, consist of 100 images, 28 CN and 72 PD, collected at UF and 44 CN images from UKB. A second set of datasets, UKB-Green and UF-UKB-Green, were created using the green color channels to improve vessel segmentation. Vessel segmentation was performed using U-Net segmentation network. The vessel maps served as inputs to SVM classifying networks. Saliency maps were created to assess areas of interest for the networks. Results: The top performing SVM network for the UKB and UKB-Green datasets were the sigmoid SVM networks which achieved accuracies of .698 and .719 respectively. Meanwhile the top performing networks for the UF-UKB and UF-UKB-Green datasets where the linear SVM networks which achieved accuracies of .821 and .857 respectively. The saliency maps indicate that the different networks focused on different vessel structures with the most successful networks focusing more on smaller vessels. Conclusion: The results indicate that the machine learning networks can classify PD based on retina vasculature, with the key features being smaller blood vessels. The proposed methods further support the idea that changes in brain physiology can be observed in the eye. Machine learning networks can be applied to clinically available data and still provide accurate predictions Clinical Relevance statement, not to exceed 200 characters: The work illustrates the feasibility of utilizing eye images as a potential method for diagnosing PD, opposed to the current method of using motor symptoms. 

   more » « less