We use atom-scale molecular dynamics simulations to clarify the role of glycosphingolipids in the dynamics of cholesterol-rich lipid rafts. To this end, we consider lipid membranes that contain varying amounts of galactosylceramide (GalCer), sphingomyelin, cholesterol, and phosphatidylcholine. The results indicate that increasing the portion of GalCer molecules greatly slows down the lateral diffusion. Only 5-10 mol % of GalCer causes a decrease of almost an order of magnitude compared to corresponding membranes without GalCer. The slowing down is not related to interdigitation, which becomes weaker with increasing GalCer concentration. Instead, the decrease in diffusion is found to correlate with the increasing number of hydrogen bonds formed between GalCer and the phospholipid molecules, which is also observed to have other effects, such as to increase the friction between the membrane leaflets.