Understanding the mechanisms of thermal conduction in graphene is a longstanding research topic due to its high thermal conductivity. Studies based on the Peierls-Boltzmann transport equation (PBTE) have revealed many unique phonon transport properties in graphene, but most previous works considered only three-phonon scatterings and relied on interatomic force constants (IFCs) extracted at 0 K. Recently, the importance of four-phonon scattering in graphene was revealed by Feng and Ruan [Phys. Rev. B 97, 045202 (2018)10.1103/PhysRevB.97.045202]. In this paper, we explore the temperature-dependent IFCs with regard to phonon transport in graphene through our PBTE calculations. We demonstrate that the strength of four-phonon scatterings was severely overestimated by previous work that used the IFCs extracted at 0 K. By using IFCs at finite temperatures, we find that four-phonon scattering is weakened but still significantly reduced the thermal conductivity of graphene by around 50%, even at room temperature. Furthermore, in order to reproduce the prediction from molecular dynamics simulations, phonon frequency broadening has to be taken into account when determining the phonon scattering rates. Our study elucidates the phonon transport properties of graphene at finite temperatures and could be extended to other crystalline materials.