This paper studies the effect of 3D magnetic perturbations (MPs) on fast-ion confinement, and its impact on the toroidal rotation velocity profile. Two low collisionality H-mode experiments carried out at the ASDEX Upgrade tokamak have been analysed. The two discharges feature different magnetic field helicity (q (95)), and differences in the velocity-space and level of fast-ion losses are observed. A new analysis technique has been developed that sheds light on the dependencies between fast-ion losses and toroidal rotation, providing for the first time correlation patterns resolved in radius and velocity space of the lost fast-ions. The correlation intensifies towards the plasma edge and is strongly dependent on the orbit topology of the lost fast-ions. The ASCOT orbit following code has been used to characterize the fast-ion resonant transport and beam driven torques, using the vacuum approach and including plasma response (PR). The change of the toroidal canonical momentum, which serves as figure of merit for resonant fast-ion transport, has been calculated with ASCOT. The beam geometry and q (95) are found to have a strong impact on the fast-ion transport and losses. The fast-ion transport induced by the MPs affects the beam driven torques. The effect of the changes of the j x B and collisional torques on plasma rotation is analysed using the torques simulated by ASCOT and simple momentum balance calculations. For the low q (95) = 3.8 discharge, which benefits from a resonant amplification, we find excellent agreement with the measured variation of the toroidal velocity. For the high q (95) = 5.5 discharge, the inclusion of the PR improves the comparison with experimental data with respect to the vacuum estimation, but still some differences with experiments are observed. This suggests that other non-resonant effects could play a role for the determination of the toroidal rotation profile.