We connect classical partial regularity theory for elliptic systems to Nonlinear Potential Theory of possibly degenerate equations. More precisely, we find a potential theoretic version of the classical ε-regularity criteria leading to regularity of solutions of elliptic systems. For non-homogenous systems of the type −div a(Du) = f, the new ε-regularity criteria involve both the classical excess functional of Du and optimal Riesz type and Wol potentials of the right hand side f. When applied to the homogenous case −div a(Du) = 0 such criteria recover the classical ones in partial regularity. As a corollary, we find that the classical and sharp regularity results for solutions to scalar equations in terms of function spaces for f extend verbatim to general systems in the framework of partial regularity, i.e. optimal regularity of solutions outside a negligible, closed singular set. Finally, the new ε-regularity criteria still allow to provide estimates on the Hausdor dimension of the singular sets.