Direct sampling of interstellar neutral (ISN) atoms close to the Sun enables studies of the very local interstellar medium (VLISM) around the heliosphere. The primary population of ISN helium atoms has, until now, been assumed to reflect the pristine VLISM conditions at the heliopause. Consequently, the atoms observed at 1 au by the Interstellar Boundary Explorer (IBEX) were used to determine the VLISM temperature and velocity relative to the Sun, without accounting for elastic collisions with other species outside the heliopause. Here, we evaluate the effect of these collisions on the primary ISN helium population. We follow trajectories of helium atoms and track their collisions with slowed plasma and interstellar hydrogen atoms ahead of the heliopause. Atoms typically collide a few times in the outer heliosheath, and only ~1.5% of the atoms are not scattered at all. We use calculated differential cross sections to randomly choose scattering angles in these collisions. We estimate that the resulting primary ISN helium atoms at the heliopause are slowed down by ~0.45 km/s⁻¹ and heated by ~1100 K compared to the pristine VLISM. The resulting velocity distribution is asymmetric and shows an extended tail in the antisunward direction. Accounting for this change in the parameters derived from IBEX observations gives the Sun's relative speed of 25.85 km/s⁻¹ and temperature of 6400 K in the pristine VLISM. Finally, this paper serves as a source of the differential cross sections for elastic collisions with helium atoms.