Many works have concentrated on the observable signatures of the dark matter being an ultralight axion-like particle (ALP). We concentrate on a particularly dramatic signature in the late-time cosmological matter power spectrum that occurs if the symmetry breaking that establishes the ALP happens after inflation -- white-noise density fluctuations that dominate at small scales over the adiabatic fluctuations from inflation. These fluctuations alter the early history of nonlinear structure formation. We find that for symmetry breaking scales of $fA\sim 1013-1015$GeV, which requires a high effective maximum temperature after inflation, ALP dark matter with particle mass of $mA\sim 10-13-10-20$eV could significantly change the number of high-redshift dwarf galaxies, the reionization history, and the Ly$\alpha$ forest. We consider all three observables. We find that the Ly$\alpha$ forest is the most constraining of current observables, excluding $fA\gtrsim 1015$GeV ($mA\lesssim 10-17$eV) in the simplest model for the ALP and considerably lower values in models coupled to a hidden strongly interacting sector ($fA\gtrsim 1013$GeV and $mA\lesssim 10-13$eV). Observations that constrain the extremely high-redshift tail of reionization may disfavor similar levels of isocurvature fluctuations as the forest. Future $z\sim 20-30$ 21cm observations have the potential to improve these constraints further using that the supersonic motions of the isocurvature-enhanced abundance of $\sim 104M\odot$ halos would shock heat the baryons, sourcing large BAO features.