In this paper, on the basis of the optimal sine transform based approximation we propose a new preconditioner for the interface system arising in the fast Helmholtz solver [G. Bao, W. Sun, A fast algorithm for the electromagnetic scattering from a large cavity, SIAM J. Sci. Comput. 27 (2005) 553574 (electronic)] for the electromagnetic scattering from a large cavity with layered media. We show that the spectrum of the preconditioned matrix is clustered around 1 if the preconditioner is not nearly singular. Numerical results show that the number of iterations of an preconditioned iterative method for the interface system is independent of the mesh size and the wavenumber. The computational cost of the fast method proposed in this paper for calculating the radar cross section, which is very important in electromagnetism, by means of fast Fourier transforms, is O( N2) on an N×N uniform partition of the unit square for the source free case.