Search for: All records

In-Context Learning (ICL) ability has been found efficient across a wide range of applications, where the Large Language Models (LLM) learn to complete the tasks from the examples in the prompt without tuning the parameters. In this work, we conduct a comprehensive study to understand ICL from a statistical perspective. First, we show that the perfectly pretrained LLMs perform Bayesian Model Averaging (BMA) for ICL under a dynamic model of examples in the prompt. The average error analysis for ICL is then built for the perfectly pretrained LLMs with the analysis of BMA. Second, we demonstrate how the attention structure boosts the BMA implementation. With sufficient examples in the prompt, attention is proven to perform BMA under the Gaussian linear ICL model, which also motivates the explicit construction of the hidden concepts from the attention heads' values. Finally, we analyze the pretraining behavior of LLMs. The pretraining error is decomposed as the generalization error and the approximation error. The generalization error is upper bounded via the PAC-Bayes framework. Then the ICL average error of the pretrained LLMs is shown to be the sum of O(T^{-1}) and the pretraining error. In addition, we analyze the ICL performance of the pretrained LLMs with misspecified examples. 

Synopsis In many species of birds, red carotenoid coloration serves as an honest signal of individual quality, but the mechanisms that link carotenoid coloration to animal performance remain poorly understood. Most birds that display red carotenoid coloration of feathers, bills, or legs ingest yellow carotenoids and metabolically convert the yellow pigments to red. Here, we review two lines of investigation that have rapidly advanced understanding of the production of red carotenoid coloration in birds, potentially providing an explanation for how red coloration serves as a signal of quality: the identification of the genes that enable birds to be red and the confirmation of links between production of red pigments and core cellular function. CYP2J19 and BDH1L were identified as key enzymes that catalyze the conversion of yellow carotenoids to red carotenoids both in the retinas of birds for enhanced color vision and in the feathers and bills of birds for ornamentation. This CYP2J19 and BDH1L pathway was shown to be the mechanism for production of red coloration in diverse species of birds and turtles. In other studies, it was shown that male House Finches (Haemorhous mexicanus) have high concentrations of red carotenoids within liver mitochondria and that redness is positively associated with mitochondrial function. These observations suggested that the CYP2J19 and BDH1L pathway might be tightly associated with mitochondrial function. However, it was subsequently discovered that male House Finches do not use the CYP2J19 and BDH1L pathway to produce red pigments and that both CYP2J19 and BDH1L localize in the endoplasmic reticulum, not the mitochondria. Thus, we have the most detailed understanding of links between cellular function and redness in a bird species for which the enzymes to convert yellow to red pigments remain unknown, while we have the best understanding of the enzymatic pathways to red in species for which links to cellular function are largely unstudied. Deducing whether and how signals of quality arise from these distinct mechanisms of ornamental coloration is a current challenge for scientists interested in the evolution of honest signaling. 

Carotenoid-based coloration is an essential feature of avian diversity and has important roles in communication and mate choice. The red feathers of birds from phylogenetically diverse orders and families are pigmented with C4-ketocarotenoids produced via the successive action of Cytochrome P450 2 J19 (CYP2J19) and 3-hydroxybutyrate dehydrogenase 1-like (BDH1L) on yellow dietary precursors. Yet, the biochemistry of these enzymes remains incompletely understood. Here we present a series of experiments characterizing the substrates, intermediates, and products of CYP2J19 and BDH1L expressed in heterologous cell culture. We confirm that CYP2J19 preferentially hydroxylates the 4 and 4′ positions of β-ring substrates, but can also hydroxylate the 3 and 3′ positions of C4-ketocarotenoids. We confirm that BDH1L catalyzes the conversion of zeaxanthin to canary xanthophyll B (ε,ε’-carotene-3,3′-dione) a major pigment in plumage of many yellow bird species. These results suggest that the actions of CYP2J19 and/or BDH1L can explain the presence of many metabolically transformed carotenoids in avian tissues. 

An animal's immune function is vital for survival and potentially metabolically expensive, but some pathogens could manipulate their hosts’ immune and metabolic responses. One example is Mycoplasma gallisepticum (MG), which infects both the respiratory system and conjunctiva of the eye in house finches (Haemorhous mexicanus). MG has been shown to exhibit immune- and metabolic-suppressive properties, but the physiological mechanisms are still unknown. Recent studies demonstrated that mitochondria could serve as powerhouses for both ATP production and immunity, notably inflammatory processes, through regulating complex II and its metabolites. Consequently, in this study, we investigate the short-term (3d post-inoculation) and long-term (34d post-inoculation) effects of MG infection on the hepatic mitochondrial respiration of house finches from two populations infected with two different MG isolates. After short-term infection, MG-infected birds had significantly lower state 2 and state 4 respiration, but only when using complex II substrates. After long-term infection, MG-infected birds exhibited lower state 3 respiration with both complex I and II substrates, resulting in lower respiratory control ratio compared to uninfected controls, which aligned with the hypothesized metabolic-suppressive properties of MG. Interestingly, there were limited differences in mitochondrial respiration regardless of house finch population of origin, MG isolate, and whether birds recovered from infection or not. We propose that MG may target mitochondrial complex II for its immune-suppressive properties during the early stages of infection and inhibit mitochondrial respiration for its metabolic-suppressive properties at later stage of infection, both of which should delay recovery of the host and extend infectious periods. 

Abstract Timely and accurate referral of end-stage heart failure patients for advanced therapies, including heart transplants and mechanical circulatory support, plays an important role in improving patient outcomes and saving costs. However, the decision-making process is complex, nuanced, and time-consuming, requiring cardiologists with specialized expertise and training in heart failure and transplantation. In this study, we propose two logistic tensor regression-based models to predict patients with heart failure warranting evaluation for advanced heart failure therapies using irregularly spaced sequential electronic health records at the population and individual levels. The clinical features were collected at the previous visit and the predictions were made at the very beginning of the subsequent visit. Patient-wise ten-fold cross-validation experiments were performed. Standard LTR achieved an average F1 score of 0.708, AUC of 0.903, and AUPRC of 0.836. Personalized LTR obtained an F1 score of 0.670, an AUC of 0.869 and an AUPRC of 0.839. The two models not only outperformed all other machine learning models to which they were compared but also improved the performance and robustness of the other models via weight transfer. The AUPRC scores of support vector machine, random forest, and Naive Bayes are improved by 8.87%, 7.24%, and 11.38%, respectively. The two models can evaluate the importance of clinical features associated with advanced therapy referral. The five most important medical codes, including chronic kidney disease, hypotension, pulmonary heart disease, mitral regurgitation, and atherosclerotic heart disease, were reviewed and validated with literature and by heart failure cardiologists. Our proposed models effectively utilize EHRs for potential advanced therapies necessity in heart failure patients while explaining the importance of comorbidities and other clinical events. The information learned from trained model training could offer further insight into risk factors contributing to the progression of heart failure at both the population and individual levels. 

In many species of animals, red carotenoid-based coloration is produced by metabolizing yellow dietary pigments, and this red ornamentation can be an honest signal of individual quality. However, the physiological basis for associations between organism function and the metabolism of red ornamental carotenoids from yellow dietary carotenoids remains uncertain. A recent hypothesis posits that carotenoid metabolism depends on mitochondrial performance, with diminished red coloration resulting from altered mitochondrial aerobic respiration. To test for an association between mitochondrial respiration and red carotenoids, we held wild-caught, molting male house finches in either small bird cages or large flight cages to create environmental challenges during the period when red ornamental coloration is produced. We predicted that small cages would present a less favorable environment than large flight cages and that captivity itself would decrease both mitochondrial performance and the abundance of red carotenoids compared to free-living birds. We found that captive-held birds circulated fewer red carotenoids, showed increased mitochondrial respiratory rates, and had lower complex II respiratory control ratios—a metric associated with mitochondrial efficiency—compared to free-living birds, though we did not detect a difference in the effects of small cages versus large cages. Among captive individuals, the birds that circulated the highest concentrations of red carotenoids had the highest mitochondrial respiratory control ratio for complex II substrate. These data support the hypothesis that the metabolism of red carotenoid pigments is linked to mitochondrial aerobic respiration in the house finch, but the mechanisms for this association remain to be established. 

ABSTRACT The carotenoid‐based colours of birds are a celebrated example of biological diversity and an important system for the study of evolution. Recently, a two‐step mechanism, with the enzymes cytochrome P450 2J19 (CYP2J19) and 3‐hydroxybutyrate dehydrogenase 1‐like (BDH1L), was described for the biosynthesis of red ketocarotenoids from yellow dietary carotenoids in the retina and plumage of birds. A common assumption has been that all birds with ketocarotenoid‐based plumage coloration used this CYP2J19/BDH1L mechanism to produce red feathers. We tested this assumption in house finches (Haemorhous mexicanus) by examining the catalytic function of the house finch homologues of these enzymes and tracking their expression in birds growing new feathers. We found that CYP2J19 and BDH1L did not catalyse the production of 3‐hydroxy‐echinenone (3‐OH‐echinenone), the primary red plumage pigment of house finches, when provided with common dietary carotenoid substrates. Moreover, gene expression analyses revealed little to no expression ofCYP2J19in liver tissue or growing feather follicles, the putative sites of pigment metabolism in moulting house finches. Finally, although the hepatic mitochondria of house finches have high concentrations of 3‐OH‐echinenone, observations using fluorescent markers suggest that both CYP2J19 and BDH1L localise to the endomembrane system rather than the mitochondria. We propose that house finches and other birds that deposit 3‐OH‐echinenone as their primary red plumage pigment use an alternative enzymatic pathway to produce their characteristic red ketocarotenoid‐based coloration. 

Missing data presents a challenge for machine learning applications specifically when utilizing electronic health records to develop clinical decision support systems. The lack of these values is due in part to the complex nature of clinical data in which the content is personalized to each patient. Several methods have been developed to handle this issue, such as imputation or complete case analysis, but their limitations restrict the solidity of findings. However, recent studies have explored how using some features as fully available privileged information can increase model performance including in SVM. Building on this insight, we propose a computationally efficient kernel SVM-based framework ( l 2 -SVMp+) that leverages partially available privileged information to guide model construction. Our experiments validated the superiority of l 2 -SVMp+ over common approaches for handling missingness and previous implementations of SVMp+ in both digit recognition, disease classification and patient readmission prediction tasks. The performance improves as the percentage of available privileged information increases. Our results showcase the capability of l 2 -SVMp+ to handle incomplete but important features in real-world medical applications, surpassing traditional SVMs that lack privileged information. Additionally, l 2 -SVMp+ achieves comparable or superior model performance compared to imputed privileged features.