© 2018 American Physical Society. A heavy hole confined to an InGaAs quantum dot promises the union of a stable spin and optical coherence to form a near perfect, high-bandwidth spin-photon interface. Despite theoretical predictions and encouraging preliminary measurements, the dynamic processes determining the coherence of the hole spin are yet to be understood. Here, we establish the regimes that allow for a highly coherent hole spin in these systems, recovering a crossover from hyperfine to electrical-noise dominated decoherence with a few-Tesla external magnetic field. Dynamic decoupling allows us to reach the longest ground-state coherence time, T2, of 4.0±0.2μs, observed in this system. The improvement of coherence we measure is quantitatively supported by an independent analysis of the local electrical environment.