skip to main content


Title: Vegetation Coverage and Urban Amenity Mapping Using Computer Vision and Machine Learning [Vegetation Coverage and Urban Amenity Mapping Using Computer Vision and Machine Learning]
This paper proposes a computer vision-based workflow that analyses Google 360-degree street views to understand the quality of urban spaces regarding vegetation coverage and accessibility of urban amenities such as benches. Image segmentation methods were utilized to produce an annotated image with the amount of vegetation, sky and street coloration. Two deep learning models were used -- Monodepth2 for depth detection and YoloV5 for object detection -- to create a 360-degree diagram of vegetation and benches at a given location. The automated workflow allows non-expert users like planners, designers, and communities to analyze and evaluate urban environments with Google Street Views. The workflow consists of three components: (1) user interface for location selection; (2) vegetation analysis, bench detection and depth estimation; and (3) visualization of vegetation coverage and amenities. The analysis and visualization could inform better urban design outcomes.  more » « less
Award ID(s):
2131186 1827505
NSF-PAR ID:
10440685
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
Proceedings of the 3rd International Conference on Image Processing and Vision Engineering (IMPROVE 2023)
Page Range / eLocation ID:
67 to 75
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Urban areas are often warmer than rural areas due to the phenomenon known as the “urban heat island” (UHI) effect, which can cause discomfort for those engaging in outdoor activities and can have a disproportionate impact on low-income communities, people of color, and the elderly. The intensity of the UHI effect is influenced by a variety of factors, including urban morphology, which can vary from one area to another. To investigate the relationship between outdoor thermal comfort and urban morphology in different urban blocks with varying social vulnerability status, this study developed a geographic information system (GIS)-based workflow that combined the “local climate zone” (LCZ) classification system and an urban microclimate assessment tool called ENVI-met. To demonstrate the effectiveness of this methodology, the study selected two different urban blocks in Philadelphia, Pennsylvania–with high and low social vulnerability indices (SVI)–to compare their microclimate conditions in association with urban morphological characteristics such as green coverage area, sky view factor (SVF), albedo, and street height to width (H/W) ratio. The results of the study showed that there was a strong correlation between tree and grass coverage and outdoor air and mean radiant temperature during hot seasons and extremely hot days, which in turn affected simulated predicted mean vote (PMV). The effects of greenery were more significant in the block associated with a low SVI, where nearly 50% of the site was covered by trees and grass, compared to only 0.02% of the other block associated with a high SVI. Furthermore, the investigation discovered that reduced SVF, along with increased albedo and H/W ratio, had a beneficial impact on the microclimate at the pedestrian level within the two studied urban blocks. This study provided an effective and easy-to-implement method for tackling the inequity issue of outdoor thermal comfort and urban morphology at fine geographic scales. 
    more » « less
  2. Image segmentation is a fundamental task that has benefited from recent advances in machine learning. One type of segmentation, of particular interest to computer vision, is that of urban segmentation. Although recent solutions have leveraged on deep neural networks, approaches usually do not consider regularities appearing in facade structures (e.g., windows are often in groups of similar alignment, size, or spacing patterns) as well as additional urban structures such as building footprints and roofs. Moreover, both satellite and street-view images are often noisy and occluded, thus getting the complete structure segmentation from a partial observation is difficult. Our key observations are that facades and other urban structures exhibit regular structures, and additional views are often available. In this paper, we present a novel framework (RFCNet) that consists of three modules to achieve multiple goals. Specifically, we propose Regularization to improve the regularities given an initial segmentation, Fusion that fuses multiple views of the segmentation, and Completion that can infer the complete structure if necessary. Experimental results show that our method outperforms previous state-of-the-art methods quantitatively and qualitatively for multiple facade datasets. Furthermore, by applying our framework to other urban structures (e.g., building footprints and roofs), we demonstrate our approach can be generalized to various pattern types. 
    more » « less
  3. Abstract

    Due to climate change and rapid urbanization, Urban Heat Island (UHI), featuring significantly higher temperature in metropolitan areas than surrounding areas, has caused negative impacts on urban communities. Temporal granularity is often limited in UHI studies based on satellite remote sensing data that typically has multi-day frequency coverage of a particular urban area. This low temporal frequency has restricted the development of models for predicting UHI. To resolve this limitation, this study has developed a cyber-based geographic information science and systems (cyberGIS) framework encompassing multiple machine learning models for predicting UHI with high-frequency urban sensor network data combined with remote sensing data focused on Chicago, Illinois, from 2018 to 2020. Enabled by rapid advances in urban sensor network technologies and high-performance computing, this framework is designed to predict UHI in Chicago with fine spatiotemporal granularity based on environmental data collected with the Array of Things (AoT) urban sensor network and Landsat-8 remote sensing imagery. Our computational experiments revealed that a random forest regression (RFR) model outperforms other models with the prediction accuracy of 0.45 degree Celsius in 2020 and 0.8 degree Celsius in 2018 and 2019 with mean absolute error as the evaluation metric. Humidity, distance to geographic center, and PM2.5concentration are identified as important factors contributing to the model performance. Furthermore, we estimate UHI in Chicago with 10-min temporal frequency and 1-km spatial resolution on the hottest day in 2018. It is demonstrated that the RFR model can accurately predict UHI at fine spatiotemporal scales with high-frequency urban sensor network data integrated with satellite remote sensing data.

     
    more » « less
  4. Abstract

    After a disaster, teams of structural engineers collect vast amounts of images from damaged buildings to obtain new knowledge and extract lessons from the event. However, in many cases, the images collected are captured without sufficient spatial context. When damage is severe, it may be quite difficult to even recognize the building. Accessing images of the predisaster condition of those buildings is required to accurately identify the cause of the failure or the actual loss in the building. Here, to address this issue, we develop a method to automatically extract pre‐event building images from 360° panorama images (panoramas). By providing a geotagged image collected near the target building as the input, panoramas close to the input image location are automatically downloaded through street view services (e.g., Google or Bing in the United States). By computing the geometric relationship between the panoramas and the target building, the most suitable projection direction for each panorama is identified to generate high‐quality 2D images of the building. Region‐based convolutional neural networks are exploited to recognize the building within those 2D images. Several panoramas are used so that the detected building images provide various viewpoints of the building. To demonstrate the capability of the technique, we consider residential buildings in Holiday Beach in Rockport, Texas, United States, that experienced significant devastation in Hurricane Harvey in 2017. Using geotagged images gathered during actual postdisaster building reconnaissance missions, we verify the method by successfully extracting residential building images from Google Street View images, which were captured before the event.

     
    more » « less
  5. null (Ed.)
    Very high spatial resolution commercial satellite imagery can inform observation, mapping, and documentation of micro-topographic transitions across large tundra regions. The bridging of fine-scale field studies with pan-Arctic system assessments has until now been constrained by a lack of overlap in spatial resolution and geographical coverage. This likely introduced biases in climate impacts on, and feedback from the Arctic region to the global climate system. The central objective of this exploratory study is to develop an object-based image analysis workflow to automatically extract ice-wedge polygon troughs from very high spatial resolution commercial satellite imagery. We employed a systematic experiment to understand the degree of interoperability of knowledge-based workflows across distinct tundra vegetation units—sedge tundra and tussock tundra—focusing on the same semantic class. In our multi-scale trough modelling workflow, we coupled mathematical morphological filtering with a segmentation process to enhance the quality of image object candidates and classification accuracies. Employment of the master ruleset on sedge tundra reported classification accuracies of correctness of 0.99, completeness of 0.87, and F1 score of 0.92. When the master ruleset was applied to tussock tundra without any adaptations, classification accuracies remained promising while reporting correctness of 0.87, completeness of 0.77, and an F1 score of 0.81. Overall, results suggest that the object-based image analysis-based trough modelling workflow exhibits substantial interoperability across the terrain while producing promising classification accuracies. From an Arctic earth science perspective, the mapped troughs combined with the ArcticDEM can allow hydrological assessments of lateral connectivity of the rapidly changing Arctic tundra landscape, and repeated mapping can allow us to track fine-scale changes across large regions and that has potentially major implications on larger riverine systems. 
    more » « less