The atmospheres of bodies in the Solar system display a great degree of diversity in their mass and composition. Impacts onto these bodies by smaller objects, such as asteroids, comets and planetesimals left over after terrestrial planet formation are evidenced by crater observations, and are an inevitable outcome of dynamical simulations of planet and moon formation. These impacts deliver mass and energy, and are capable of altering the atmosphere through erosion, volatile delivery and impact-triggered outgassing from the target body surface. In this thesis I investigate the effect of bombardment by these small bodies on the evolution of atmospheres on planets and moons. I first provide an introduction to the processes relevant to the formation and evolution of terrestrial planets and their atmospheres in Chapter 1, with a focus on the current state of research on the role impacts have played in shaping atmospheres. I develop in Chapter 2 an analytic method through which the characteristic stalling mass (at which impact induced erosion and accretion are balanced) can be predicted, and which can be used to predict the degree of stochastic variation expected for a given atmosphere and impactor combination. I also present a numerical model for stochastic atmosphere evolution due to bombardment, incorporating prescriptions for a range of impact outcomes (fragmentation and aerial bursts, cratering events and the non-local mass loss caused by giant impacts). These models are applied to the bombardment by asteroids, comets and left-over planetesimals on the Earth in Chapter 3 and a comparative study of the terrestrial Solar system planets (Venus and Mars) in Chapter 5, using distributions of impact velocities I calculate from the results of recent dynamical simulations. The sensitivity of these results to both the initial atmosphere conditions and the properties of the impacting populations are investigated. In Chapter 4, the numerical code and analytic predictions are also applied to cometary impacts of the atmospheres of the moons of the outer giant planets, incorporating a prescription for impact-triggered outgassing when applied to Titan. Finally in Chapter 6, I make general predictions about the influence of impacts on the atmospheres of exoplanets and hypothetical exomoons and present a simple model for the simultaneous evolution of a magma ocean and atmosphere on a terrestrial planet. The results of this dissertation are summarised in Chapter 7.