We investigate the out-of-equilibrium dynamics of ultracold atoms trapped in an optical lattice and loaded into an optical resonator that is driven transversely. We derive an effective quantum master equation for weak atom-light coupling that can be brought into Lindblad form in both the bad- and good-cavity limits. In the so-called bad-cavity regime, we find that the steady state is always that of infinite temperature, but that the relaxation dynamics can be highly nontrivial. For small hopping, the interplay between dissipation and strong interactions generally leads to anomalous diffusion in the space of atomic configurations. However, for a fine-tuned ratio of cavity-mediated and on-site interactions, we discover a limit featuring normal diffusion. In contrast, for large hopping and vanishing on-site interactions, the system can be described by a linear rate equation leading to an exponential approach of the infinite-temperature steady state. Finally, in the good-cavity regime, we show that for vanishing on-site interactions, the system allows for optical pumping between momentum mode pairs enabling cavity cooling.