Ice friction plays an important role in many engineering applications, e.g., tires on icy roads, ice breaker ship motion, or winter sports equipment. Although numerous experiments have already been performed to understand the effect of various conditions on ice friction, to reveal the fundamental frictional mechanisms is still a challenging task. This study uses in situ white light interferometry to analyze ice surface topography during linear friction testing with a rubber slider. The method helps to provide an understanding of the link between changes in the surface topography and the friction coefficient through direct visualization and quantitative measurement of the morphologies of the ice surface at different length scales. Besides surface polishing and scratching, it was found that ice melts locally even after one sweep showing the refrozen droplets. A multi-scale rubber friction theory was also applied to study the contribution of viscoelasticity to the total friction coefficient, which showed a significant level with respect to the smoothness of the ice; furthermore, the theory also confirmed the possibility of local ice melting.