Search for: All records

Referring expression comprehension aims to localize objects identified by natural language descriptions. This is a challenging task as it requires understanding of both visual and language domains. One nature is that each object can be described by synonymous sentences with paraphrases, and such varieties in languages have critical impact on learning a comprehension model. While prior work usually treats each sentence and attends it to an object separately, we focus on learning a referring expression comprehension model that considers the property in synonymous sentences. To this end, we develop an end-to-end trainable framework to learn contrastive features on the image and object instance levels, where features extracted from synonymous sentences to describe the same object should be closer to each other after mapping to the visual domain. We conduct extensive experiments to evaluate the proposed algorithm on several benchmark datasets, and demonstrate that our method performs favorably against the state-of-the-art approaches. Furthermore, since the varieties in expressions become larger across datasets when they describe objects in different ways, we present the cross-dataset and transfer learning settings to validate the ability of our learned transferable features.

Access to fast and reliable nucleic acid testing continues to play a key role in controlling the COVID-19 pandemic, especially in the context of increased vaccine break-through risks due to new variants. We report a rapid, low-cost (~ 2 USD), simple-to-use nucleic acid test kit for self-administered at-home testing without lab instrumentation. The entire sample-to-answer workflow takes < 60 min, including noninvasive sample collection, one-step RNA preparation, reverse-transcription loop-mediated isothermal amplification (RT-LAMP) in a thermos, and direct visual inspection of a colorimetric test result. To facilitate long-term storage without cold-chain, a fast one-pot lyophilization protocol was developed to preserve all required biochemical reagents of the colorimetric RT-LAMP test in a single microtube. Notably, the lyophilized RT-LAMP assay demonstrated reduced false positives as well as enhanced tolerance to a wider range of incubation temperatures compared to solution-based RT-LAMP reactions. We validated our RT-LAMP assay using simulated infected samples, and detected a panel of SARS-CoV-2 variants with successful detection of all variants that were available to us at the time. With a simple change of the primer set, our lyophilized RT-LAMP home test can be easily adapted as a low-cost surveillance platform for other pathogens and infectious diseases of global public health importance.

Few-shot classification aims to recognize novel categories with only few labeled images in each class. Existing metric-based few-shot classification algorithms predict categories by comparing the feature embeddings of query images with those from a few labeled images (support examples) using a learned metric function. While promising performance has been demonstrated, these methods often fail to generalize to unseen domains due to large discrepancy of the feature distribution across domains. In this work, we address the problem of few-shot classification under domain shifts for metric-based methods. Our core idea is to use feature-wise transformation layers for augmenting the image features using affine transforms to simulate various feature distributions under different domains in the training stage. To capture variations of the feature distributions under different domains, we further apply a learning-to-learn approach to search for the hyper-parameters of the feature-wise transformation layers. We conduct extensive experiments and ablation studies under the domain generalization setting using five few-shot classification datasets: mini-ImageNet, CUB, Cars, Places, and Plantae. Experimental results demonstrate that the proposed feature-wise transformation layer is applicable to various metric-based models, and provides consistent improvements on the few-shot classification performance under domain shift.