Optomechanics offers a natural way to implement synthetic dynamical gauge fields, leading to synthetic electric fields for phonons and, as a consequence, to unidirectional light transport. Here we investigate the quantum dynamics of synthetic gauge fields in the minimal setup of two optical modes coupled by phonon-assisted tunneling where the phonon mode is undergoing self-oscillations. We use the quantum van-der-Pol oscillator as the simplest dynamical model for a mechanical self-oscillator that allows us to perform quantum master equation simulations. We identify a single parameter, which controls the strength of quantum fluctuations, enabling us to investigate the classical-to-quantum crossover. We show that the generation of synthetic electric fields is robust against noise and that it leads to unidirectional transport of photons also in the quantum regime, albeit with a reduced isolation ratio. Our study opens the path for studying dynamical gauge fields in the quantum regime based on optomechanical arrays.