More Like this

1. Synthesis and Completion of Facades from Satellite Imagery

   https://doi.org/10.1007/978-3-030-58536-5_34  

   Zhang, X.; May, C.; Aliaga, D. (August 2020, European Conference on Computer Vision)

   Automatic satellite-based reconstruction enables large and widespread creation of urban areas. However, satellite imagery is often noisy and incomplete, and is not suitable for reconstructing detailed building facades. We present a machine learning-based inverse procedural modeling method to automatically create synthetic facades from satellite imagery. Our key observation is that building facades exhibit regular, grid-like structures. Hence, we can overcome the low-resolution, noisy, and partial building data obtained from satellite imagery by synthesizing the underlying facade layout. Our method infers regular facade details from satellite-based image-fragments of a building, and applies them to occluded or under-sampled parts of the building, resulting in plausible, crisp facades. Using urban areas from six cities, we compare our approach to several state-of-the-art image completion/in-filling methods and our approach consistently creates better facade images. 

   more » « less
2. Automatic Deep Inference of Procedural Cities from Global-scale Spatial Data

   https://doi.org/10.1145/3423422  

   Zhang, Xiaowei; Shehata, Aly; Benes, Bedrich; Aliaga, Daniel (February 2021, ACM Transactions on Spatial Algorithms and Systems)

   null (Ed.)

   Recent advances in big spatial data acquisition and deep learning allow novel algorithms that were not possible several years ago. We introduce a novel inverse procedural modeling algorithm for urban areas that addresses the problem of spatial data quality and uncertainty. Our method is fully automatic and produces a 3D approximation of an urban area given satellite imagery and global-scale data, including road network, population, and elevation data. By analyzing the values and the distribution of urban data, e.g., parcels, buildings, population, and elevation, we construct a procedural approximation of a city at a large-scale. Our approach has three main components: (1) procedural model generation to create parcel and building geometries, (2) parcel area estimation that trains neural networks to provide initial parcel sizes for a segmented satellite image of a city block, and (3) an optional optimization that can use partial knowledge of overall average building footprint area and building counts to improve results. We demonstrate and evaluate our approach on cities around the globe with widely different structures and automatically yield procedural models with up to 91,000 buildings, and spanning up to 150 km 2 . We obtain both a spatial arrangement of parcels and buildings similar to ground truth and a distribution of building sizes similar to ground truth, hence yielding a statistically similar synthetic urban space. We produce procedural models at multiple scales, and with less than 1% error in parcel and building areas in the best case as compared to ground truth and 5.8% error on average for tested cities. 

   more » « less
3. An Inverse Procedural Modeling Pipeline for SVBRDF Maps

   https://doi.org/10.1145/3502431  

   Hu, Yiwei; He, Chengan; Deschaintre, Valentin; Dorsey, Julie; Rushmeier, Holly (April 2022, ACM Transactions on Graphics)

   Procedural modeling is now the de facto standard of material modeling in industry. Procedural models can be edited and are easily extended, unlike pixel-based representations of captured materials. In this article, we present a semi-automatic pipeline for general material proceduralization. Given Spatially Varying Bidirectional Reflectance Distribution Functions (SVBRDFs) represented as sets of pixel maps, our pipeline decomposes them into a tree of sub-materials whose spatial distributions are encoded by their associated mask maps. This semi-automatic decomposition of material maps progresses hierarchically, driven by our new spectrum-aware material matting and instance-based decomposition methods. Each decomposed sub-material is proceduralized by a novel multi-layer noise model to capture local variations at different scales. Spatial distributions of these sub-materials are modeled either by a by-example inverse synthesis method recovering Point Process Texture Basis Functions (PPTBF) [ 30 ] or via random sampling. To reconstruct procedural material maps, we propose a differentiable rendering-based optimization that recomposes all generated procedures together to maximize the similarity between our procedural models and the input material pixel maps. We evaluate our pipeline on a variety of synthetic and real materials. We demonstrate our method’s capacity to process a wide range of material types, eliminating the need for artist designed material graphs required in previous work [ 38 , 53 ]. As fully procedural models, our results expand to arbitrary resolution and enable high-level user control of appearance. 

   more » « less
4. Automatic Differentiable Procedural Modeling

   https://doi.org/10.1111/cgf.14475  

   Gaillard, Mathieu; Krs, Vojtěch; Gori, Giorgio; Měch, Radomír; Benes, Bedrich (May 2022, Computer Graphics Forum)

   Abstract Procedural modeling allows for an automatic generation of large amounts of similar assets, but there is limited control over the generated output. We address this problem by introducing Automatic Differentiable Procedural Modeling (ADPM). The forward procedural model generates a final editable model. The user modifies the output interactively, and the modifications are transferred back to the procedural model as its parameters by solving an inverse procedural modeling problem. We present an auto‐differentiable representation of the procedural model that significantly accelerates optimization. In ADPM the procedural model is always available, all changes are non‐destructive, and the user can interactively model the 3D object while keeping the procedural representation. ADPM provides the user with precise control over the resulting model comparable to non‐procedural interactive modeling. ADPM is node‐based, and it generates hierarchical 3D scene geometry converted to a differentiable computational graph. Our formulation focuses on the differentiability of high‐level primitives and bounding volumes of components of the procedural model rather than the detailed mesh geometry. Although this high‐level formulation limits the expressiveness of user edits, it allows for efficient derivative computation and enables interactivity. We designed a new optimizer to solve for inverse procedural modeling. It can detect that an edit is under‐determined and has degrees of freedom. Leveraging cheap derivative evaluation, it can explore the region of optimality of edits and suggest various configurations, all of which achieve the requested edit differently. We show our system's efficiency on several examples, and we validate it by a user study. 

   more » « less
5. Digitizing cities for urban weather: representing realistic cities for weather and climate simulations using computer graphics and artificial intelligence

   https://doi.org/10.1007/s43762-023-00111-z  

   Aliaga, Daniel; Niyogi, Dev (December 2024, Computational Urban Science)

   Abstract Due to their importance in weather and climate assessments, there is significant interest to represent cities in numerical prediction models. However, getting high resolution multi-faceted data about a city has been a challenge. Further, even when the data were available the integration into a model is even more of a challenge due to the parametric needs, and the data volumes. Further, even if this is achieved, the cities themselves continually evolve rendering the data obsolete, thus necessitating a fast and repeatable data capture mechanism. We have shown that by using AI/graphics community advances we can create a seamless opportunity for high resolution models. Instead of assuming every physical and behavioral detail is sensed, a generative and procedural approach seeks to computationally infer a fully detailed 3D fit-for-purpose model of an urban space. We present a perspective building on recent success results of this generative approach applied to urban design and planning at different scales, for different components of the urban landscape, and related applications. The opportunities now possible with such a generative model for urban modeling open a wide range of opportunities as this becomes mainstream. 

   more » « less