Search for: All records

Training large language models (LLMs) increasingly relies on geographically distributed accelerators, causing prohibitive communication costs across regions and uneven utilization of heterogeneous hardware. We propose HALoS, a hierarchical asynchronous optimization framework that tackles these issues by introducing local parameter servers (LPSs) within each region and a global parameter server (GPS) that merges updates across regions. This hierarchical design minimizes expensive inter-region communication, reduces straggler effects, and leverages fast intra-region links. We provide a rigorous convergence analysis for HALoS under non-convex objectives, including theoretical guarantees on the role of hierarchical momentum in asynchronous training. Empirically, HALoS attains up to 7.5x faster convergence than synchronous baselines in geo-distributed LLM training and improves upon existing asynchronous methods by up to 2.1x. Crucially, HALoS preserves the model quality of fully synchronous SGD-matching or exceeding accuracy on standard language modeling and downstream benchmarks-while substantially lowering total training time. These results demonstrate that hierarchical, server-side update accumulation and global model merging are powerful tools for scalable, efficient training of new-era LLMs in heterogeneous, geo-distributed environments. 

Abstract BackgroundAs habitat fragmentation increases, ecological processes, including patterns of vector-borne pathogen prevalence, will likely be disrupted, but ongoing investigations are necessary to examine this relationship. Here, we report the differences in the prevalence of Lyme disease (Borrelia burgdorferisensu lato, s.l.) and haemoproteosis (Haemoproteusspp.) pathogens in avian populations of a fragmented habitat.B. burgdorferis.l. is a generalist pathogen that is transmitted byIxodes pacificusvectors in California, andHaemoproteusis an avian parasite transmitted byCulicoidesvectors. MethodsTo determine whether biotic (avian and mammalian abundance) or abiotic characteristics (patch size and water availability) correlated with infection prevalence change, we screened 176 birds sampled across seven sites in oak woodland habitat in northern California. ResultsWhile biotic factors correlated with an increase in both pathogens, infection prevalence ofHaemoproteusspp. was only associated with individual-level traits, specifically foraging substrate and diet, andB. burgdorferis.l. was associated with community-level characteristics, both total mammal and, specifically, rodent abundance. Proximity to water was the only abiotic factor found to be significant for both pathogens and reinforces the importance of water availability for transmission cycles. Larger patch sizes did not significantly affect infection prevalence ofHaemoproteus,but did increase the prevalence ofB. burgdorferi. ConclusionsThese results highlight that while environmental factors (specifically habitat fragmentation) have a limited role in vector-borne pathogen prevalence, the indirect impact to biotic factors (community composition) can have consequences for bothHaemoproteusandB. burgdorferiprevalence in birds. Given the pervasiveness of habitat fragmentation, our results are of broad significance. Graphical abstract 

The emerging field of smart healthcare has identified emotion detection as a key component in improving patient care, diagnostics, and therapeutic interventions. This paper introduces an innovative approach to emotion detection within the healthcare domain by integrating a Convolutional Neural Network (CNN) with a Maximum A Posterior (MAP) estimator prepared for Magnitude-Squared Spectrum (MSS) analysis. The effectiveness of CNN’s advanced feature extraction capabilities with the statistical strength of MAP estimation offers a promising avenue for interpreting complex physiological signals. The proposed methodology aims to accurately discern and quantify emotional states, thus contributing to the personalization and effectiveness of healthcare services. To validate the efficacy of this approach, the work conducted extensive experiments on a diverse data set composed of physiological signals, demonstrating that the proposed model outperforms existing limitations in emotion recognition tasks. The integration of MSS into CNN frameworks, added with MAP estimation, provides a significant improvement in the detection and analysis of emotions, resulting in more responsive and intelligent healthcare systems. This proposed paper not only presents a novel methodological contribution, but also demonstrates the groundwork for future research toward the intersection of emotional intelligence and healthcare technology.