We study the dynamics of a set of two-level systems coupled by dipolar interactions under a resonant external Rabi drive. The two-level systems are prepared initially in a coherent product state, and we ask how the nonequilibrium conditions caused by the drive affect this coherence. We study the full nonlinear dynamics of the coupled two-level systems within a classical approximation by analyzing numerically the equations of motion and determining the stability of the collective coherent state within classical Floquet theory. We establish the behavior analytically in the high Rabi coupling limit by employing a Magnus expansion and spin-wave analysis. Our results show that, typically, the dipole interactions between the two-level systems lead to instabilities that cause a breakdown of the collective Rabi oscillations. However, we identify parameter regimes for which the two-level systems undergo collective coherent Rabi oscillations even in the presence of the dipole interactions.