Extracting principal building variables from automatically collected urban scale façade images for energy conservation through deep transfer learning

Buildings account for 40% of the energy consumption and 13% of the greenhouse gas (GHG) emissions in the U.S. To improve building energy efficiency, cities around the U.S. issued energy policies, such as the energy performance disclosure requirement in New York City, to encourage building owners to make informed retrofitting decisions. However, complying with these policies is expensive and time-consuming for government agencies and building owners, especially for old buildings where detailed building information is not readily available. In this work, we propose an automatic, non-intrusive, and scalable framework to capture energy-essential building variables through reasoning building façade images - FaçadeReasoner. Specifically, we first build a comprehensive building information dataset and identify the most impactful (i.e., principal) variables in relation to building energy performance and GHG emissions using the state-of-the-art feature attribution model. Next, we propose a method to automatically collect an urban scale building image dataset with more than 10,000 façade images and extract principal-building-variables from these images using deep transfer learning. Results show that FacadeReasoner has the capability to predict principal building variables, namely “building type” (accuracy 0.77), “year built” (accuracy 0.62), “building height” (R² 0.80), and rough estimates of “building area” (R² 0.46) from façade images. This study is unique as it marks the first attempt to enable an automated end-to-end framework for urban scale principal-building-variables extraction, providing an efficient and economical alternative for large building portfolio owners and managers (e.g., municipalities) to comprehend urban scale energy-related building information for informed decision-making, directly contributing to Net-Zero 2050.