Improved performance electricity demand forecast can provide decentralized energy system operators, aggregators, managers, and other stakeholders with essential information for energy resource scheduling, demand response management, and energy market participation. Most previous methodologies have focused on predicting the aggregate amount of electricity demand at national or regional scale and disregarded the electricity demand for small-scale decentralized energy systems (buildings, energy communities, microgrids, local energy internets, etc.), which are emerging in the smart grid context. Furthermore, few research groups have performed attribute selection before training predictive models. This paper proposes a machine learning (ML)-based integrated feature selection approach to obtain the most relevant and nonredundant predictors for accurate short-term electricity demand forecasting in distributed energy systems. In the proposed approach, one of the ML tools-binary genetic algorithm (BGA) is applied for the feature selection process and Gaussian process regression (GPR) is used for measuring the fitness score of the features. In order to validate the effectiveness of the proposed approach, it is applied to various building energy systems located in the Otaniemi area of Espoo, Finland. The findings are compared with those achieved by other feature selection techniques. The proposed approach enhances the quality and efficiency of the predictor selection, with minimal chosen predictors to achieve improved prediction accuracy. It outperforms the other evaluated feature selection methods. Besides, a feedforward artificial neural network (FFANN) model is implemented to evaluate the forecast performance of the selected predictor subset. The model is trained using two-year hourly dataset and tested with another one-year hourly dataset. The obtained results verify that the FFANN forecast model based on the BGA-GPR FS selected training feature subset has achieved an annual MAPE of 1.96%, which is a very acceptable and promising value for electricity demand forecasting in small-scale decentralized energy systems.