Search for: All records

Synthetic data (SIM) drawn from simulators have emerged as a popular alternative for training models where acquiring annotated real-world images is difficult. However, transferring models trained on synthetic images to real-world applications can be challenging due to appearance disparities. A commonly employed solution to counter this SIM2REAL gap is unsupervised domain adaptation, where models are trained using labeled SIM data and unlabeled REAL data. Mispredictions made by such SIM2REAL adapted models are often associated with miscalibration - stemming from overconfident predictions on real data. In this paper, we introduce AUGCAL, a simple training-time patch for unsupervised adaptation that improves SIM2REAL adapted models by - (1) reducing overall miscalibration, (2) reducing overconfidence in incorrect predictions and (3) improving confidence score reliability by better guiding misclassification detection - all while retaining or improving SIM2REAL performance. Given a base SIM2REAL adaptation algorithm, at training time, AUGCAL involves replacing vanilla SIM images with strongly augmented views (AUG intervention) and additionally optimizing for a training time calibration loss on augmented SIM predictions (CAL intervention). We motivate AUGCAL using a brief analytical justification of how to reduce miscalibration on unlabeled REAL data. Through our experiments, we empirically show the efficacy of AUGCAL across multiple adaptation methods, backbones, tasks and shifts. 

We propose an automated algorithm to stress-test a trained visual model by generating language-guided counterfactual test images (LANCE). Our method leverages recent progress in large language modeling and text-based image editing to augment an IID test set with a suite of diverse, realistic, and challenging test images without altering model weights. We benchmark the performance of a diverse set of pretrained models on our generated data and observe significant and consistent performance drops. We further analyze model sensitivity across different types of edits, and demonstrate its applicability at surfacing previously unknown class-level model biases in ImageNet. 

Typical deep visual recognition models are capable of performing the one task they were trained on. In this paper, we tackle the extremely difficult problem of combining distinct models with different initializations, each solving a separate task, into one multi-task model without any additional training. Prior work in model merging permutes one model to the space of the other then averages them together. While this works for models trained on the same task, we find that this fails to account for the differences in models trained on disjoint tasks. Thus, we introduce "ZipIt!", a general method for merging two arbitrary models of the same architecture that incorporates two simple strategies. First, in order to account for features that aren't shared between models, we expand the model merging problem to allow for merging features within each model by defining a general "zip" operation. Second, we add support for partially zipping the models up until a specified layer, naturally creating a multi-head model. We find that these two changes combined account for 20-60% improvement over prior work, making it more feasible to merge models trained on disjoint tasks without retraining. 

Type IIb supernovae (SNe IIb) are core-collapse events whose optical spectra show strong hydrogen features that disappear over time, implying that their progenitors were nearly, but not completely, stripped of their hydrogen envelopes prior to core collapse. Thus, compared to hydrogen-rich SNe II, SNe IIb can provide a closer examination of the underlying structure of the progenitor system, particularly during early photospheric phases (less than +70 days relative to max. light). I will present early-time multi-epoch optical spectropolarimetry of several SNe IIb, obtained using the SPOL instrument at the University of Arizona. Using polarization diagnostics provides a way to track structural changes in the depleted hydrogen envelopes of these SNe as deeper layers of helium and other elements emerge and evolve. I find significant temporal polarization increases in the absorption wings of their H and He lines. Some of these line features make "loops" in Stokes Q-U diagrams, suggesting non-axisymmetic structure in the ejecta, perhaps arising from a transient absorbing clump. Furthermore, the majority of these SNe show polarimetric evidence for aspherical explosions along a preferred, or dominant, axis. I discuss the implications these findings have on the 3D geometry of the explosions by comparing the observed polarization to published synthetic spectropolarimetry that models axial symmetry and clump structures in stripped-envelope, core-collapse SNe. This comparative study naturally facilitates a broader discussion around the unresolved question as to what extent this SNe subclass shows common polarization characteristics. 

Neural network based computer vision systems are typically built on a backbone, a pretrained or randomly initialized feature extractor. Several years ago, the default option was an ImageNet-trained convolutional neural network. However, the recent past has seen the emergence of countless backbones pretrained using various algorithms and datasets. While this abundance of choice has led to performance increases for a range of systems, it is difficult for practitioners to make informed decisions about which backbone to choose. Battle of the Backbones (BoB) makes this choice easier by benchmarking a diverse suite of pretrained models, including vision-language models, those trained via self-supervised learning, and the Stable Diffusion backbone, across a diverse set of computer vision tasks ranging from classification to object detection to OOD generalization and more. Furthermore, BoB sheds light on promising directions for the research community to advance computer vision by illuminating strengths and weakness of existing approaches through a comprehensive analysis conducted on more than 1500 training runs. While vision transformers (ViTs) and self-supervised learning (SSL) are increasingly popular, we find that convolutional neural networks pretrained in a supervised fashion on large training sets still perform best on most tasks among the models we consider. Moreover, in apples-to-apples comparisons on the same architectures and similarly sized pretraining datasets, we find that SSL backbones are highly competitive, indicating that future works should perform SSL pretraining. 

Neural network based computer vision systems are typically built on a backbone, a pretrained or randomly initialized feature extractor. Several years ago, the default option was an ImageNet-trained convolutional neural network. However, the recent past has seen the emergence of countless backbones pretrained using various algorithms and datasets. While this abundance of choice has led to performance increases for a range of systems, it is difficult for practitioners to make informed decisions about which backbone to choose. Battle of the Backbones (BoB) makes this choice easier by benchmarking a diverse suite of pretrained models, including vision-language models, those trained via self-supervised learning, and the Stable Diffusion backbone, across a diverse set of computer vision tasks ranging from classification to object detection to OOD generalization and more. Furthermore, BoB sheds light on promising directions for the research community to advance computer vision by illuminating strengths and weakness of existing approaches through a comprehensive analysis conducted on more than 1500 training runs. While vision transformers (ViTs) and self-supervised learning (SSL) are increasingly popular, we find that convolutional neural networks pretrained in a supervised fashion on large training sets still perform best on most tasks among the models we consider. Moreover, in apples-to-apples comparisons on the same architectures and similarly sized pretraining datasets, we find that SSL backbones are highly competitive, indicating that future works should perform SSL pretraining. 

Summary In the hyperdiverse fungi, the process of speciation is virtually unknown, including for the > 20 000 species of ectomycorrhizal mutualists. To understand this process, we investigated patterns of genome‐wide differentiation in the ectomycorrhizal porcini mushroom, Boletus edulis , a globally distributed species complex with broad ecological amplitude. By whole‐genome sequencing 160 individuals from across the Northern Hemisphere, we genotyped 792 923 single nucleotide polymorphisms to characterize patterns of genome‐wide differentiation and to identify the adaptive processes shaping global population structure. We show that B. edulis exhibits contrasting patterns of genomic divergence between continents, with multiple lineages present across North America, while a single lineage dominates Europe. These geographical lineages are inferred to have diverged 1.62–2.66 million years ago, during a period of climatic upheaval and the onset of glaciation in the Pliocene–Pleistocene boundary. High levels of genomic differentiation were observed among lineages despite evidence of substantial and ongoing introgression. Genome scans, demographic inference, and ecological niche models suggest that genomic differentiation is maintained by environmental adaptation, not physical isolation. Our study uncovers striking patterns of genome‐wide differentiation on a global scale and emphasizes the importance of local adaptation and ecologically mediated divergence, rather than prezygotic barriers such as allopatry or genomic incompatibility, in fungal population differentiation.