2024

a multimodal and transfer learning approaches to residential building operational energy estimation

decarbonizing the housing stock is one of the critical mechanisms for the uk to meet the net-zero target by 2050. to identify effective retrofit measures, a data-driven urban retrofit modelling approach is commonly used, involving operational energy estimation and retrofit scenario exploration, which are heavily dependent on the quantity and quality of data. however, accessibility to adequate housing data varies across regions, which poses challenges to establishing a robust understanding of the existing housing stock. in response, this thesis aims to investigate the potential of the application of multimodal and transfer learning approaches, to enhance the trustworthiness and adaptability of operational energy consumption estimations in residential buildings, especially in cases with limited data quality or quantity. three studies were conducted to answer the following research questions: 1) can incorporating multi-modalities improve the trustworthiness of operational energy estimation? and 2) can transfer learning techniques improve the adaptability of operational energy estimation?

to address these questions, this thesis started with an initial study using a machine learning model for operational energy estimation. with the help of an automated machine learning tool, the housings in sheffield were examined based on energy performance certificates (epc) and map data from the ordnance survey. the results were explained by permutation feature importance and partial dependence, which revealed the critical factors in estimation, and identified the key building elements to guide retrofit. applying the trained model to barnsley and merthyr tydfil revealed decreased performance in the latter, emphasizing the need for a more adaptable approach acknowledging the spatial heterogeneity.

the second study examined the capability of using street view images for energy estimation. the case study found that using images alone is not able to offer accurate estimation. therefore, to answer the first research question, the idea of multimodal learning was introduced, where both epc and google streetview (gsv) data were incorporated to reduce the bias caused by relying on a single source of data. the models trained using multiple modalities were compared with models built on a single modality, using statistical metrics and the shapley additive explanations (shap) to examine the effectiveness. statistic metrics r2 and mean absolute percentage error were employed to evaluate the multimodal network, which shown improvements in prediction accuracy. shap was used to examine the changes in feature importance and correlations with and without multiple modalities. the changes demonstrated the model is able to identify and connect key features from both modalities to perform the prediction.

the third study was designed for the second research question. the potential of transfer learning was investigated to address the challenge of adapting a trained model to changing local contexts when only limited data is available for training. three cities were examined, barnsley, doncaster and merthyr tydfil. they are either limited in data quantity or quality, to represent possible challenges in model implementation. developing upon the multimodal network trained on the second study, the layers in the network were adjusted with different train-ability and learning rates, so it is capable of leveraging knowledge from cities with sufficient data to assist in accurate estimation for cities with poor data. the predictions performed with and without transfer learning were evaluated, which demonstrated significant improvements among all the examined cities. each region was explained with their respective feature importance ranks, to present that, although the models were all developed upon one multimodal network, it is able to leverage key information from each region and adapt the model accordingly.

overall, this thesis examined the utilization of deep multimodal network and transfer learning methods, contributing to enhancing the trustworthiness and adaptability of energy estimation models, applied explainable ai to analyze the feature importance and partial dependence of the housing features to offer guidance on effective retrofit prioritizations and achieve the net zero targets.

Reliable prediction of residential energy consumption is essential for informing energy efficiency policies and retrofit strategies. However, traditional data-driven approaches are often constrained by the availability and quality of data. This study presents a novel approach combining multimodal neural networks with a transfer learning framework, leveraging both tabular and visual data to enhance prediction accuracy and enable knowledge transfer from data-rich to data-poor regions. Case studies conducted in Barnsley, Doncaster, and Merthyr Tydfil demonstrated that the proposed approach outperforms traditional mono-modal models. The multimodal model improved prediction accuracy significantly, achieving a MAPE reduction from 1.15 (with only visual data) and 0.86 (with only tabular data) to 0.43 (with both visual and tabular data), while the inclusion of transfer learning offers further performance improvements in data-scarce regions, with up to 63.6 % error reduction. Explainable AI is utilised to validate the model’s interpretability, confirming key features such as floor and wall insulation conditions as pivotal in energy consumption predictions. This integrated framework offers actionable insights for policymakers, facilitating data-driven decisions to enhance energy efficiency in diverse urban settings.

The key role of buildings in tackling climate change has gained global recognition. To avoid unnecessary costs and time wasted, it is important to understand the conditions and energy usage for existing housing stock to identify the most important features affecting energy consumption and to guide the relevant retrofit measures. This paper investigated how the spatial, morphological and thermal characteristics of residential houses contribute to housing energy consumption. Additionally, it presents a rapid assessment tool using minimum data input to answer two main questions: 1) What type of properties may need retrofit? 2) What building elements/features may be prioritised to be retrofitted? A case study was performed with around 143,000 residential properties in Sheffield. An automated machine approach was applied which successfully estimated the energy consumption of target buildings with an \\R^2\\score of 0.828. Permutation feature importance and partial dependence of the features were examined against energy consumption. The results indicate that housing sizes and conditions of the external walls are found to be the most important features when estimating the energy consumption of residential buildings in Sheffield. Relatively larger and older detached houses in neighbourhoods with higher build density may benefit the most from home upgrading projects for energy consumption reduction.

a multimodal and transfer learning approaches to residential building operational energy estimation

references

papers