We discuss a numerical model for black hole growth and its associated feedback processes that for the first time allows cosmological simulations of structure formation to self-consistently follow the build up of the cosmic population of galaxies and active galactic nuclei (AGNs). Our model assumes that seed black holes are present at early cosmic epochs at the centres of forming haloes. We then track their growth from gas accretion and mergers with other black holes in the course of cosmic time. For black holes that are active, we distinguish between two distinct modes of feedback, depending on the black hole accretion rate itself. Black holes that accrete at high rates are assumed to be in a ‘quasar regime’, where we model their feedback by thermally coupling a small fraction of their bolometric luminosity to the surrounding gas. The quasar activity requires high densities of relatively cold gas around the black hole, as it is achieved through large-scale inflows triggered by galaxy mergers. For black holes with low accretion rates, we conjecture that most of their feedback occurs in mechanical form, where AGN-driven bubbles are injected into a gaseous environment. This regime of activity, which is subdominant in terms of total black hole mass growth, can be identified with radio galaxies in clusters of galaxies, and can suppress cluster cooling flows without the requirement of a triggering by mergers. Using our new model, we carry out treesph cosmological simulations on the scales of individual galaxies to those of massive galaxy clusters, both for isolated systems and for cosmological boxes. We demonstrate that our model produces results for the black hole and stellar mass densities in broad agreement with observational constraints. We find that the black holes significantly influence the evolution of their host galaxies, changing their star formation history, their amount of cold gas and their colours. Furthermore, the properties of intracluster gas are affected strongly by the presence of massive black holes in the cores of galaxy clusters, leading to shallower metallicity and entropy profiles, and to a suppression of strong cooling flows. Our results support the notion that AGNs are a key ingredient in cosmological structure formation. They lead to a self-regulated growth of black holes and bring the simulated properties of their host galaxies into much better agreement with observations.