skip to main content

This content will become publicly available on June 1, 2024

Title: DyLiN: Making Light Field Networks Dynamic
Light Field Networks, the re-formulations of radiance fields to oriented rays, are magnitudes faster than their coordinate network counterparts, and provide higher fidelity with respect to representing 3D structures from 2D observations. They would be well suited for generic scene representation and manipulation, but suffer from one problem: they are limited to holistic and static scenes. In this paper, we propose the Dynamic Light Field Network (DyLiN) method that can handle non-rigid deformations, including topological changes. We learn a deformation field from input rays to canonical rays, and lift them into a higher dimensional space to handle discontinuities. We further introduce CoDyLiN, which augments DyLiN with controllable attribute inputs. We train both models via knowledge distillation from pretrained dynamic radiance fields. We evaluated DyLiN using both synthetic and real world datasets that include various non-rigid deformations. DyLiN qualitatively outperformed and quantitatively matched state-of-the-art methods in terms of visual fidelity, while being 25 - 71x computationally faster. We also tested CoDyLiN on attribute annotated data and it surpassed its teacher model. Project page:  more » « less
Award ID(s):
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Page Range / eLocation ID:
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. New preservation technologies may allow for organ banking similar to blood and biomaterial banking approaches. Using cryoprotective agents (CPAs), aqueous solutions with organic components such as DMSO, propylene glycol, and added salts and sugars, organs can be used to vitrify and store organs at −140 °C. When needed, these organs can be rewarmed in a rapid and uniform manner if CPAs are supplemented with iron oxide nanoparticles (IONPs) in an applied radiofrequency field. Speed and uniformity of warming are both IONP concentration and CPA suspension dependent. Here we present a coating method of small molecule phosphonate linker (PLink) and biocompatible polymer ( i.e. polyethylene glycol PEG) that tunes stability and increases the maximum allowable concentration of IONPs in CPA suspension. PLink contains a phosphonate 'anchor' for high irreversible binding to iron oxide and a carboxylic acid 'handle' for ligand attachment. PLink-PEG removes and replaces the initial coating layer of commercially available IONPs (EMG1200 (hydrophobic) and EMG308 (hydrophilic) Ferrotec, Inc., increasing colloidal stability and decreasing aggregation in both water and CPAs, (verified with dynamic light scattering) from minutes (uncoated) to up to 6 days. Heating properties of EMG1200, specific absorption rate (SAR), measured using an applied field of 360 kHz and 20 kA m −1 , increased from 20 to 180 W per g Fe with increasing PLink-PEG5000. PEG replacing the initially hydrophobic coating decreased aggregation in water and CPA, consistent with earlier studies on heating performance. Furthermore, although the size is minimized at 0.20 mol PEG per g Fe, heating is not maximized until concentrations above 0.43 mol PEG per g Fe on EMG1200. SAR on hydrophilic EMG308 was preserved at 400 W per g Fe regardless of the amount of PLink added to the core. Herein concentrations of IONP in VS55 (common CPA) significantly above our previous capabilities, sIONP at 10 mg Fe per mL, was reached, 25 mg Fe per mL of 308-PEG5000 and 60 mg Fe per mL of 1200-PEG5000, approaching stock EMG308 in water, 60 mg Fe per mL. Furthermore, at these concentrations cryopreserved Human dermal fibroblast cells were successfully nanowarmed (at applied fields described above), with higher viability as compared to convective rewarming in a water bath and heating rate close to 200 °C min −1 , 2.5 times faster than our current system. Using PLink as the coating method allowed for higher concentrations of IONPs to be successfully suspended in CPA without affecting the heating ability. Additionally, the model ligand, PEG, allowed for increased stability over time in nanowarming experiments. 
    more » « less
  2. Larochelle, Hugo ; Kamath, Gautam ; Hadsell, Raia ; Cho, Kyunghyun (Ed.)
    Neural scene representations, both continuous and discrete, have recently emerged as a powerful new paradigm for 3D scene understanding. Recent efforts have tackled unsupervised discovery of object-centric neural scene representations. However, the high cost of ray-marching, exacerbated by the fact that each object representation has to be ray-marched separately, leads to insufficiently sampled radiance fields and thus, noisy renderings, poor framerates, and high memory and time complexity during training and rendering. Here, we propose to represent objects in an object-centric, compositional scene representation as light fields. We propose a novel light field compositor module that enables reconstructing the global light field from a set of object-centric light fields. Dubbed Compositional Object Light Fields (COLF), our method enables unsupervised learning of object-centric neural scene representations, state-of-the-art reconstruction and novel view synthesis performance on standard datasets, and rendering and training speeds at orders of magnitude faster than existing 3D approaches. 
    more » « less
  3. Simulating stiff materials in applications where deformations are either not significant or else can safely be ignored is a fundamental task across fields. Rigid body modeling has thus long remained a critical tool and is, by far, the most popular simulation strategy currently employed for modeling stiff solids. At the same time, rigid body methods continue to pose a number of well known challenges and trade-offs including intersections, instabilities, inaccuracies, and/or slow performances that grow with contact-problem complexity. In this paper we revisit the stiff body problem and present ABD, a simple and highly effective affine body dynamics framework, which significantly improves state-of-the-art for simulating stiff-body dynamics. We trace the challenges in rigid-body methods to the necessity of linearizing piecewise-rigid trajectories and subsequent constraints. ABD instead relaxes the unnecessary (and unrealistic) constraint that each body's motion be exactly rigid with a stiff orthogonality potential, while preserving the rigid body model's key feature of a small coordinate representation. In doing so ABD replaces piecewise linearization with piecewise linear trajectories. This, in turn, combines the best of both worlds: compact coordinates ensure small, sparse system solves, while piecewise-linear trajectories enable efficient and accurate constraint (contact and joint) evaluations. Beginning with this simple foundation, ABD preserves all guarantees of the underlying IPC model we build it upon, e.g., solution convergence, guaranteed non-intersection, and accurate frictional contact. Over a wide range and scale of simulation problems we demonstrate that ABD brings orders of magnitude performance gains (two- to three-orders on the CPU and an order more when utilizing the GPU, obtaining 10, 000× speedups) over prior IPC-based methods, while maintaining simulation quality and nonintersection of trajectories. At the same time ABD has comparable or faster timings when compared to state-of-the-art rigid body libraries optimized for performance without guarantees, and successfully and efficiently solves challenging simulation problems where both classes of prior rigid body simulation methods fail altogether. 
    more » « less
  4. The rising global trend to reduce dependence on fossil fuels has provided significant motivation toward the development of alternative energy conversion methods and new technologies to improve their efficiency. Recently, oscillating energy harvesters have shown promise as highly efficient and scalable turbines, which can be implemented in areas where traditional energy extraction and conversion are either unfeasible or cost prohibitive. Although such devices are quickly gaining popularity, there remain a number of hurdles in the understanding of their underlying fluid dynamics phenomena. The ability to achieve high efficiency power output from oscillating airfoil energy harvesters requires exploitation of the complexities of the event of dynamic stall. During dynamic stall, the oncoming flow separates at the leading edge of the airfoil to form leading ledge vortex (LEV) structures. While it is well known that LEVs play a significant role in aerodynamic force generation in unsteady animal flight (e.g. insects and birds), there is still a need to further understand their spatiotemporal evolution in order to design more effective energy harvesting enhancement mechanisms. In this work, we conduct extensive experimental investigations to shed-light on the flow physics of a heaving and pitching airfoil energy harvester operating at reduced frequencies of k = fc=U1 = 0.06-0.18, pitching amplitude of 0 = 75 and heaving amplitude of h0 = 0:6c. The experimental work involves the use of two-component particle image velocimetry (PIV) measurements conducted in a wind tunnel facility at Oregon State University. Velocity fields obtained from the PIV measurements are analyzed qualitatively and quantitatively to provide a description of the dynamics of LEVs and other flow structures that may be present during dynamic stall. Due to the difficulties of accurately measuring aerodynamic forces in highly unsteady flows in wind tunnels, a reduced-order model based on the vortex-impulse theory is proposed for estimating the aerodynamic loadings and power output using flow field data. The reduced-order model is shown to be dominated by two terms that have a clear physical interpretation: (i) the time rate of change of the impulse of vortical structures and (ii) the Kutta-Joukowski force which indirectly represents the history effect of vortex shedding in the far wake. Furthermore, the effects of a bio-inspired flow control mechanism based on deforming airfoil surfaces on the flow dynamics and energy harvesting performance are investigated. The results show that the aerodynamic loadings, and hence power output, are highly dependent on the formation, growth rate, trajectory and detachment of the LEV. It is shown that the energy harvesting efficiency increases with increasing reduced frequency, peaking at 25% when k = 0.14, agreeing very well with published numerical results. At this optimal reduced frequency, the time scales of the LEV evolution and airfoil kinematics are matched, resulting in highly correlated aerodynamic load generation and airfoil motion. When operating at k > 0:14, it is shown that the aerodynamic moment and airfoil pitching motion become negatively correlated and as a result, the energy harvesting performance is deteriorated. Furthermore, by using a deforming airfoil surface at the leading and trailing edges, the peak energy harvesting efficiency is shown to increase by approximately 17% and 25% relative to the rigid airfoil, respectively. The performance enhancement is associated with enhanced aerodynamic forces for both the deforming leading and trailing edges. In addition, The deforming trailing edge airfoil is shown to enhance the correlation between the aerodynamic moment and pitching motion at higher reduced frequencies, resulting in a peak efficiency at k = 0:18 as opposed to k = 0:14 for the rigid airfoil. 
    more » « less
  5. The conservation field is experiencing a rapid increase in the amount, variety, and quality of spatial data that can help us understand species movement and landscape connectivity patterns. As interest grows in more dynamic representations of movement potential, modelers are often limited by the capacity of their analytic tools to handle these datasets. Technology developments in software and high-performance computing are rapidly emerging in many fields, but uptake within conservation may lag, as our tools or our choice of computing language can constrain our ability to keep pace. We recently updated Circuitscape, a widely used connectivity analysis tool developed by Brad McRae and Viral Shah, by implementing it in Julia, a high-performance computing language. In this initial re-code (Circuitscape 5.0) and later updates, we improved computational efficiency and parallelism, achieving major speed improvements, and enabling assessments across larger extents or with higher resolution data. Here, we reflect on the benefits to conservation of strengthening collaborations with computer scientists, and extract examples from a collection of 572 Circuitscape applications to illustrate how through a decade of repeated investment in the software, applications have been many, varied, and increasingly dynamic. Beyond empowering continued innovations in dynamic connectivity, we expect that faster run times will play an important role in facilitating co-production of connectivity assessments with stakeholders, increasing the likelihood that connectivity science will be incorporated in land use decisions. 
    more » « less