The response and degradation of the hot mix asphalt (HMA) materials used in pavement structures are affected by their inherent heterogeneity. The objective of this work is to study the impact of two different sources of HMA heterogeneity in the uncertainty of the mechanical moisture degradation of HMA. The first source of heterogeneity is the spatial variability of the properties of the bulk fine aggregate matrix (FAM) of the mixture, and the second is the location and shape of the coarse aggregate particles. The heterogeneity of the bulk FAM phase was modelled using a random field technique, while that of the coarse aggregates was accounted for by randomly generating realistic probable sets of aggregate particles. Thus, ‘computational replicates’ of HMA microstructures were generated and subjected to moisture diffusion and mechanical loading using a finite element approach. In the mechanical simulations, a non-linear viscoelastic moisture damage constitutive relationship based on continuum damage mechanics theory was selected to characterise the response of the bulk FAM phase. The results show that conducting computational simulations with realistic HMA microstructures that properly capture the heterogeneity of the material is useful to quantify the mean values and dispersion (i.e. uncertainty) associated with the response and degradation of the mixture. This information, which cannot be easily obtained in the field or in the laboratory due to the difficulty of acquiring a sufficient amount of data, is useful to conduct structural reliability analysis and to predict the life cycle behaviour of the material.