Deep observations are revealing a growing number of young galaxies in the first billion years of cosmic time. Compared to typical galaxies at later times, they show more extreme emission-line properties, higher star formation rates, lower masses, and smaller sizes. However, their faintness precludes studies of their chemical abundances and ionization conditions, strongly limiting our understanding of the physics driving early galaxy build-up and metal enrichment. Here we study a rare population of ultraviolet-selected, low-luminosity galaxies at redshift 2.4 < z < 3.5 that exhibit all the rest-frame properties expected from primeval galaxies. These low-mass, highly compact systems are rapidly forming galaxies able to double their stellar mass in only a few tens of millions of years. They are characterized by very blue ultraviolet spectra with weak absorption features and bright nebular emission lines, which imply hard radiation fields from young hot massive stars. Their highly ionized gas phase has strongly sub-solar carbon and oxygen abundances, with metallicities more than a factor of two lower than that found in typical galaxies of similar mass and star formation rate at z≤2.5. These young galaxies reveal an early and short stage in the assembly of their galactic structures and their chemical evolution, a vigorous phase that is likely to be dominated by the effects of gas-rich mergers, accretion of metal-poor gas and strong outflows.