Search for: All records

The ability to accurately interpret complex visual information is a crucial topic of multimodal large language models (MLLMs). Recent work indicates that enhanced visual perception significantly reduces hallucinations and improves performance on resolution-sensitive tasks, such as optical character recognition and document analysis. A number of recent MLLMs achieve this goal using a mixture of vision encoders. Despite their success, there is a lack of systematic comparisons and detailed ablation studies addressing critical aspects, such as expert selection and the integration of multiple vision experts. This study provides an extensive exploration of the design space for MLLMs using a mixture of vision encoders and resolutions. Our findings reveal several underlying principles common to various existing strategies, leading to a streamlined yet effective design approach. We discover that simply concatenating visual tokens from a set of complementary vision encoders is as effective as more complex mixing architectures or strategies. We additionally introduce Pre-Alignment to bridge the gap between vision-focused encoders and language tokens, enhancing model coherence. The resulting family of MLLMs, Eagle, surpasses other leading open-source models on major MLLM benchmarks. 

Bifunctional molecules such as targeted protein degraders induce proximity to promote gain‐of‐function pharmacology. These powerful approaches have gained broad traction across academia and the pharmaceutical industry, leading to an intensive focus on strategies that can accelerate their identification and optimization. We and others have previously used chemical proteomics to map degradable target space, and these datasets have been used to develop and train multiparameter models to extend degradability predictions across the proteome. In this study, we now turn our attention to develop generalizable chemistry strategies to accelerate the development of new bifunctional degraders. We implement lysine‐targeted reversible‐covalent chemistry to rationally tune the binding kinetics at the protein‐of‐interest across a set of 25 targets. We define an unbiased workflow consisting of global proteomics analysis, IP/MS of ternary complexes and the E‐STUB assay, to mechanistically characterize the effects of ligand residence time on targeted protein degradation and formulate hypotheses about the rate‐limiting step of degradation for each target. Our key finding is that target residence time is a major determinant of degrader activity, and this can be rapidly and rationally tuned through the synthesis of a minimal number of analogues to accelerate early degrader discovery and optimization.