The many-body system comprising a He nucleus, three electrons, and a positron has been studied using an explicitly correlated Gaussians basis and a stochastic variational method for the optimization of the basis. The purpose has been to clarify to which extent the system can be considered as a distinguishable positronium (Ps) atom interacting with a He atom and, thereby, to pave the way to a practical atomistic modeling of Ps states and annihilation in matter. The maximum value of the distance between the positron and the nucleus is constrained and the Ps atom at different distances from the nucleus is identified from the electron and positron densities, as well as from the electron-positron distance and center-of-mass distributions. The polarization of the Ps atom increases as its distance from the nucleus decreases. The contact density of the electrons of He with the positron is depleted, particularly when the overlap is small. The ortho-Ps pick-off annihilation rate calculated as the overlap of the positron and the free He electron densities has to be corrected for the observed depletion, especially at large pores or voids.