Consider a vector bundle over a Kähler manifold which admits a Hermitian Yang-Mills connection. We show that the pullback bundle on the blowup of the Kähler manifold at a collection of points also admits a Hermitian Yang-Mills connection, for Kähler classes on the blowup which make the exceptional divisors small. Our proof uses gluing techniques, and is hence asymptotically explict. This recovers, through the Hitchin-Kobayashi correspondence, algebro-geometric results due to Buchdahl and Sibley.