this was a small two-month project to establish and strengthen collaborations on residential retrofit activities undertaken by rise with the barnsley council. the main work focused on a proof-of-concept deployment of automated drive-by building assessment for the council’s social housing decarbonization program.
uk 2050 decarbonization target requires rapid deployment of retrofit measures across over 20m existing homes. our existing activities bring together the sheffield urban flows observatory’s multi-spectral imaging vehicle (marvel) to enable automated identification and characterization of buildings for retrofit purposes.
references
papers
-
Learning from Other Cities: Transfer Learning Based Multimodal Residential Energy Prediction for Cities with Limited Existing Data
Energy and Buildings, Jul 2025
Reliable prediction of residential energy consumption is essential for informing energy efficiency policies and retrofit strategies. However, traditional data-driven approaches are often constrained by the availability and quality of data. This study presents a novel approach combining multimodal neural networks with a transfer learning framework, leveraging both tabular and visual data to enhance prediction accuracy and enable knowledge transfer from data-rich to data-poor regions. Case studies conducted in Barnsley, Doncaster, and Merthyr Tydfil demonstrated that the proposed approach outperforms traditional mono-modal models. The multimodal model improved prediction accuracy significantly, achieving a MAPE reduction from 1.15 (with only visual data) and 0.86 (with only tabular data) to 0.43 (with both visual and tabular data), while the inclusion of transfer learning offers further performance improvements in data-scarce regions, with up to 63.6 % error reduction. Explainable AI is utilised to validate the model’s interpretability, confirming key features such as floor and wall insulation conditions as pivotal in energy consumption predictions. This integrated framework offers actionable insights for policymakers, facilitating data-driven decisions to enhance energy efficiency in diverse urban settings.
-
City-Scale Residential Energy Consumption Prediction with a Multimodal Approach
Scientific Reports, Feb 2025
The key role of buildings in tackling climate change has gained global recognition. To avoid unnecessary costs and time wasted, it is important to understand the conditions and energy usage for existing housing stock to identify the most important features affecting energy consumption and to guide the relevant retrofit measures. This paper investigated how the spatial, morphological and thermal characteristics of residential houses contribute to housing energy consumption. Additionally, it presents a rapid assessment tool using minimum data input to answer two main questions: 1) What type of properties may need retrofit? 2) What building elements/features may be prioritised to be retrofitted? A case study was performed with around 143,000 residential properties in Sheffield. An automated machine approach was applied which successfully estimated the energy consumption of target buildings with an \\R^2\\score of 0.828. Permutation feature importance and partial dependence of the features were examined against energy consumption. The results indicate that housing sizes and conditions of the external walls are found to be the most important features when estimating the energy consumption of residential buildings in Sheffield. Relatively larger and older detached houses in neighbourhoods with higher build density may benefit the most from home upgrading projects for energy consumption reduction.
confs
-
Deep Multimodal Learning for Residential Building Energy Prediction
In IOP Conference Series: Earth and Environmental Science, Feb 2022
The residential sector has become the second-largest energy consumer since 1987 in the UK. Approximately 24 million existing dwellings in England made up over 32% of the overall energy consumption in 2020. A robust understanding of existing buildings’ energy performance is therefore critical in guiding proper home retrofit measures to accelerate towards meeting the UK’s climate targets. A substantial number of predictions at a city scale rely on available data, e.g., Energy Performance Certificates (EPCs) and GIS products, to develop statistical and machine learning models to estimate energy consumption. However, issues with existing data are not negligible. This work adopted the idea of deep multimodal learning to study the potential for using Google Street View (GSV) images as an additional input for residential building energy prediction. 20,031 GSV images of 5,933 residential buildings in central Barnsley, UK, have been selected for a case study. All images were pre-processed using a state-of-the-art object detection algorithm to minimise the noise caused by other elements that may appear nearby. Building specifications that cannot be easily determined by the appearance are extracted from existing EPC information as text-based inputs for prediction. A multimodal model was designed to jointly take images and texts as inputs. These inputs are first propagated through a convolutional neural network and multi-layer perceptron, respectively, before being combined into a connected network for final energy prediction. The multi-input model was trained and tested on the case study area and predicted an annual energy consumption with a mean absolute difference of 0.01kWh/m2 per annum on average compared with what is recorded in the EPC. The difference between the predicted results and the EPC may also provide some hints on the bias the certificates potentially contain.