Deterministic polarization reversal in ferroelectric and multiferroic films is critical for their exploitation in nanoelectronic devices. While ferroelectricity has been studied for nearly a century, major discrepancies in the reported values of coercive fields and saturation polarization persist in literature for many materials. This raises questions about the atomic-scale mechanisms behind polarization reversal. Unconventional ferroelectric switching in ε-Fe2O3 films, a material that combines ferrimagnetism and ferroelectricity at room temperature, is reported. High-resolution in situ scanning transmission electron microscopy experiments and first-principles calculations demonstrate that polarization reversal in ε-Fe2O3 occurs around pre-existing domain walls only, triggering local domain wall motion in moderate electric fields of 250–500 kV cm−1. Calculations indicate that the activation barrier for switching at domain walls is nearly a quarter of that corresponding to the most likely transition paths inside ε-Fe2O3 domains. Moreover, domain walls provide symmetry lowering of the polar structure near the domain boundary, which is shown to be necessary for ferroelectric switching in ε-Fe2O3. Local polarization reversal in ε-Fe2O3 limits the macroscopic ferroelectric response and offers important hints on how to tailor ferroelectric properties by domain structure design in other relevant ferroelectric materials.