Viscoelasticity has been described since the time of Maxwell as an interpolation of purely viscous and purely elastic response, but its microscopic atomic-level mechanism in solids has remained elusive. We studied three model disordered solids: a random lattice, the bond-depleted fcc lattice, and the fcc lattice with vacancies. Within the harmonic approximation for central-force lattices, we applied sum rules for viscoelastic response derived on the basis of nonaffine atomic motions. The latter motions are a direct result of local structural disorder, and in particular, of the lack of inversion symmetry in disordered lattices. By defining a suitable quantitative and general atomic-level measure of nonaffinity and inversion symmetry, we show that the viscoelastic responses of all three systems collapse onto a master curve upon normalizing by the overall strength of inversion-symmetry breaking in each system. Close to the isostatic point for central-force lattices, power-law creep G(t) $\sim$ t$\mathrm{<mrow\; data-mjx-texclass="ORD">-1<mrow\; data-mjx-texclass="ORD">/</mrow>2</mrow>}$ emerges as a consequence of the interplay between soft vibrational modes and nonaffine dynamics, and various analytical scalings, supported by numerical calculations, are predicted by the theory.