This thesis considers novel phenomena arising in the few and many-body physics of ultracold atomic gases. The specific models considered are motivated by recent experimental developments. A main focus will be the theoretical description of systems used in the simulation of artificial gauge fields via time-modulated tuning of system parameters and via the coupling of internal atomic states by Raman lasers. A second aspect will be the study of systems with synthetic dimensions and the effects of the unconventional infinite range interactions that arise in this description. It explores time-dependent effects in the single and few particle setting and the collective many-body phases arising in these models. Using the framework of Floquet theory it studies the interplay of time-dependence and particle-interactions both in continuum and lattice systems. In particular, it provides a general explanation for how heating effects can arise in interacting periodically time-dependent quantum systems, identifying an underlying mechanism for heating via two-particle collisions and the relevant scaling of the associated rates with system parameters. Furthermore, the general framework is applied to specific experimental set-ups. This yields heating rates and population dynamics in agreement with the experimental data. Finally, it also proposes improvements to the experiment that allow to limit the heating rates which is required to access the strongly interacting regime. In the case of synthetic dimensions the focus lies on the intriguing interplay of artificial gauge fields and strong particle interactions of atoms confined in optical lattices. In these systems a suitable coupling of internal spin-states of the atoms allows to simulate an additional finite dimension. In contrast to the typically encountered situation of short-range interactions the particle interactions are infinite-range in this synthetic dimension. In the strong coupling limit an effective description for this system is derived and its ground state phase diagram is studied using numerical tools.