Search for: All records

Intrinsic image decomposition and inverse rendering are long-standing problems in computer vision. To evaluate albedo recovery, most algorithms report their quantitative performance with a mean Weighted Human Disagreement Rate (WHDR) metric on the IIW dataset. However, WHDR focuses only on relative albedo values and often fails to capture overall quality of the albedo. In order to comprehensively evaluate albedo, we collect a new dataset, Measured Albedo in the Wild (MAW), and propose three new metrics that complement WHDR: intensity, chromaticity and texture metrics. We show that existing algorithms often improve WHDR metric but perform poorly on other metrics. We then finetune different algorithms on our MAW dataset to significantly improve the quality of the reconstructed albedo both quantitatively and qualitatively. Since the proposed intensity, chromaticity, and texture metrics and the WHDR are all complementary we further introduce a relative performance measure that captures average performance. By analysing existing algorithms we show that there is significant room for improvement. Our dataset and evaluation metrics will enable researchers to develop algorithms that improve albedo reconstruction. Code and Data available at: https://measuredalbedo.github.io/ 

Although self-/un-supervised methods have led to rapid progress in visual representation learning, these methods generally treat objects and scenes using the same lens. In this paper, we focus on learning representations for objects and scenes that preserve the structure among them. Motivated by the observation that visually similar objects are close in the representation space, we argue that the scenes and objects should instead follow a hierarchical structure based on their compositionality. To exploit such a structure, we propose a contrastive learning framework where a Euclidean loss is used to learn object representations and a hyperbolic loss is used to encourage representations of scenes to lie close to representations of their constituent objects in a hyperbolic space. This novel hyperbolic objective encourages the scene-object hypernymy among the representations by optimizing the magnitude of their norms. We show that when pretraining on the COCO and OpenImages datasets, the hyperbolic loss improves downstream performance of several baselines across multiple datasets and tasks, including image classification, object detection, and semantic segmentation. We also show that the properties of the learned representations allow us to solve various vision tasks that involve the interaction between scenes and objects in a zero-shot fashion. 

Supervised learning of skeleton sequence encoders for action recognition has received significant attention in recent times. However, learning such encoders without labels continues to be a challenging problem. While prior works have shown promising results by applying contrastive learning to pose sequences, the quality of the learned representations is often observed to be closely tied to data augmentations that are used to craft the positives. However, augmenting pose sequences is a difficult task as the geometric constraints among the skeleton joints need to be enforced to make the augmentations realistic for that action. In this work, we propose a new contrastive learning approach to train models for skeleton-based action recognition without labels. Our key contribution is a simple module, HaLP – to Hallucinate Latent Positives for contrastive learning. Specifically, HaLP explores the latent space of poses in suitable directions to generate new positives. To this end, we present a novel optimization formulation to solve for the synthetic positives with an explicit control on their hardness. We propose approximations to the objective, making them solvable in closed form with minimal overhead. We show via experiments that using these generated positives within a standard contrastive learning framework leads to consistent improvements across benchmarks such as NTU-60, NTU- 120, and PKU-II on tasks like linear evaluation, transfer learning, and kNN evaluation. Our code can be found at https://github.com/anshulbshah/HaLP. 

In this paper, we present a technique for estimating the geometry and reflectance of objects using only a camera, flashlight, and optionally a tripod. We propose a simple data capture technique in which the user goes around the object, illuminating it with a flashlight and capturing only a few images. Our main technical contribution is the introduction of a recursive neural architecture, which can predict geometry and reflectance at 2 k ×2 k resolution given an input image at 2 k ×2 k and estimated geometry and reflectance from the previous step at 2 k−1 ×2 k−1 . This recursive architecture, termed RecNet, is trained with 256×256 resolution but can easily operate on 1024×1024 images during inference. We show that our method produces more accurate surface normal and albedo, especially in regions of specular highlights and cast shadows, compared to previous approaches, given three or fewer input images. 

Scleractinian corals typically form a robust calcium carbonate skeleton beneath their living tissue. This skeleton, through its trace element composition and isotope ratios, may record environmental conditions of water surrounding the coral animal. While bulk unrecrystallized aragonite coral skeletons can be used to reconstruct past ocean conditions, corals that have undergone significant diagenesis have altered geochemical signatures and are typically assumed to retain insufficient meaningful information for bulk or macrostructural analysis. However, partially recrystallized skeletons may retain organic molecular components of the skeletal organic matrix (SOM), which is secreted by the animal and directs aspects of the biomineralization process. Some SOM proteins can be retained in fossil corals and can potentially provide past oceanographic, ecological, and indirect genetic information. Here, we describe a dataset of scleractinian coral skeletons, aged from modern to Cretaceous plus a Carboniferous rugosan, characterized for their crystallography, trace element composition, and amino acid compositions. We show that some specimens that are partially recrystallized to calcite yield potentially useful biochemical information whereas complete recrystalization or silicification leads to significant alteration or loss of the SOM fraction. Our analysis is informative to biochemical-paleoceanographers as it suggests that previously discounted partially recrystallized coral skeletons may indeed still be useful at the microstructural level. 

Rhodolith distribution, morphology, and cryptofauna have been minimally studied on fringing reefs. We present the first study to examine both rhodolith distribution and associated cryptofauna in a tropical fringing reef, located along the microtidal, wave-dominated north shore of Moorea, French Polynesia. We find higher abundances of larger, rounder, and more branching rhodoliths in locations where longer waves impact the fringing reef. Among 1879 animals extracted and identified from 145 rhodoliths, ophiuroids, polychaetes, decapod crustaceans, and gastropods are most abundant, with a wide range of additional taxa contributing to diversity. Large and branching rhodoliths contain the greatest number and diversity of cryptofaunal organisms and are the preferred habitat of rigid-bodied, non-burrowing forms. Overall, exposure to waves entering the lagoon through passes appears to be a critical determinant of rhodolith abundance, morphotype, and in turn cryptofaunal composition in fringing reef habitats.

 

Abstract Here we report the first recovery, sequencing, and identification of fossil biomineral proteins from a Pleistocene fossil invertebrate, the stony coral Orbicella annularis . This fossil retains total hydrolysable amino acids of a roughly similar composition to extracts from modern O. annularis skeletons, with the amino acid data rich in Asx (Asp + Asn) and Glx (Glu + Gln) typical of invertebrate skeletal proteins. It also retains several proteins, including a highly acidic protein, also known from modern coral skeletal proteomes that we sequenced by LC–MS/MS over multiple trials in the best-preserved fossil coral specimen. A combination of degradation or amino acid racemization inhibition of trypsin digestion appears to limit greater recovery. Nevertheless, our workflow determines optimal samples for effective sequencing of fossil coral proteins, allowing comparison of modern and fossil invertebrate protein sequences, and will likely lead to further improvements of the methods. Sequencing of endogenous organic molecules in fossil invertebrate biominerals provides an ancient record of composition, potentially clarifying evolutionary changes and biotic responses to paleoenvironments. 

We propose a novel method for combining synthetic and real images when training networks to determine geomet- ric information from a single image. We suggest a method for mapping both image types into a single, shared domain. This is connected to a primary network for end-to-end train- ing. Ideally, this results in images from two domains that present shared information to the primary network. Our experiments demonstrate significant improvements over the state-of-the-art in two important domains, surface normal estimation of human faces and monocular depth estimation for outdoor scenes, both in an unsupervised setting. 

We propose a novel method for combining synthetic and real images when training networks to determine geomet- ric information from a single image. We suggest a method for mapping both image types into a single, shared domain. This is connected to a primary network for end-to-end train- ing. Ideally, this results in images from two domains that present shared information to the primary network. Our experiments demonstrate significant improvements over the state-of-the-art in two important domains, surface normal estimation of human faces and monocular depth estimation for outdoor scenes, both in an unsupervised setting.