We are entering a new age of AI applications where machine learning is the core technology but machine learning models are generally non-intuitive, opaque and usually complicated for people to understand. The current AI applications inability to explain is decisions and actions to end users have limited its effectiveness. The explainable AI will enable the users to understand, accordingly trust and effectively manage the decisions made by machine learning models. The heat recycler’s fault detection in Air Handling Unit (AHU) has been explained with explainable artificial intelligence since the fault detection is particularly burdensome because the reason for its failure is mostly unknown and unique. The key requirement of such systems is the early diagnosis of such faults for its economic and functional efficiency. The machine learning models, Support Vector Machine and Neural Networks have been used for the diagnosis of the fault and explainable artificial intelligence has been used to explain the models’ behaviour.