To simulate the degradation mechanism of concrete piles underground exposed to sulfate attack, the effect of water pressure on sulfate erosion distribution in concrete piles was considered. Based on Darcy law, the relationship between flow velocity and water pressure and time was established. Furthermore, the penetration depth calculation model of water pressure was established. Considering the existing water pressure, a diffusion erosion equation of sulfate in concrete piles was established based on Fick’s second law. Via the method of separation of variables, variable substitution and integral transformation, the analytical solution of the convection erosion equation and diffusion erosion equation without the influence of water pressure were obtained. At the same time, the sensitivity of water pressure to penetration depth was analyzed. The effect of water pressure, water cement ratio, seepage depth, and crack width on the sulfate concentration distribution was analyzed. The results show that with the water pressure increasing the penetration depth increases significantly. Under water pressure, the concentration of sulfate in concrete is significantly higher than that of sulfate when there is no water pressure. The watercement ratio, penetration depth and the crack width have significant effects on the concentration distribution of sulfates in concrete.