In conventional design, buildings are typically assumed to be fixed to their foundations. This assumption holds mainly for structures on highly stiff soil; however, buildings on varying soil conditions may respond differently to ground excitations, potentially impacting surrounding structures. This paper investigates the seismic interaction between adjacent buildings with insufficient separation, constructed on different soil types. The study examines four adjacent single-story structures, modeled as lumped mass systems with diverse structural properties. The nonlinear viscoelastic pounding force model, "Hertzdamp," was used to calculate the pounding forces, while spring-dashpot elements represented the horizontal and rotational motions of the supporting soil. The soils studied range from very soft to soft rock. The numerical simulations present structural responses and time histories for each structure, with comparisons to fixed-base cases. Results include peak responses for different seismic gap sizes, highlighting the influence of soil-structure interaction (SSI) on outcomes. The findings demonstrate that SSI significantly affects building behavior during seismic-induced collisions. Notably, neglecting SSI in pounding scenarios can lead to underestimations of displacement, overestimations of acceleration, and excessive pounding forces. Specifically, peak absolute displacement increased by up to 118.03 % for building 4 on very soft soil, while SSI reduced acceleration, with a maximum decrease of 45.94 % for building 3 on very soft soil. Soil flexibility also altered pounding patterns compared to rigid base cases. Overall, these results underscore the importance of considering site-specific soil conditions, particularly for seismic behavior involving pounding.