One of the key challenges in spectroscopy is the inhomogeneous broadening that masks the homogeneous spectral lineshape and the underlying coherent dynamics. Techniques such as four-wave mixing and spectral hole-burning are used in optical spectroscopy, and spin-echo in nuclear magnetic resonance (NMR). However, the high-power pulses used in spin-echo and other sequences often create spurious dynamics obscuring the subtle spin correlations important for quantum technologies. Here we develop NMR techniques to probe the correlation times of the fluctuations in a nuclear spin bath of individual quantum dots, using frequency-comb excitation, allowing for the homogeneous NMR lineshapes to be measured without high-power pulses. We find nuclear spin correlation times exceeding one second in self-assembled InGaAs quantum dots - four orders of magnitude longer than in strain-free III-V semiconductors. This observed freezing of the nuclear spin fluctuations suggests ways of designing quantum dot spin qubits with a well-understood, highly stable nuclear spin bath.