Globally, access to higher education has increased, but inequalities by socio‐economic background remain. This article explores the relationship between early schooling opportunities (and learning) and progression into higher education in four low and middle‐income countries. We analyse data from the Young Lives longitudinal study, following cohorts of young people from age 5 to 22 in four country settings: Ethiopia, Peru, Vietnam and India. We reveal wide variability in higher education participation between the four countries, with a common pattern of a very strong association between early learning and later higher education participation, even after allowing for a range of demographic characteristics. Whilst early learning is important in predicting later higher education participation, we also find that significant barriers to higher education participation remain for low socio‐economic status groups, even if they initially show good levels of learning. We track the trajectories of children who have initial good levels of learning, and hence arguably the potential to progress to higher education, and assess the extent to which socio‐economic background plays a mediating role in these trajectories. Pupils with initially good levels of learning at primary school age, but who are from poor backgrounds, fall back in terms of their relative attainment during secondary schooling years. This implies that socio‐economic status continues to be a barrier to educational attainment throughout these children’s lives. We discuss the implications of these findings for policy initiatives aimed at narrowing inequalities in higher education access in poorer countries.