Although the presence of archaic hominin legacies in humans is taken for granted, little attention has been given as to how the data fit with how humans colonized the world. Here, I show that Neanderthal and Denisovan legacies are strongly correlated and that inferred legacy size, like heterozygosity, exhibits a strong correlation with distance from Africa. Simulations confirm that, once created, legacy size is extremely stable: it may reduce through admixture with lower legacy populations but cannot increase significantly through neutral drift. Consequently, populations carrying the highest legacies are likely to be those whose ancestors inter-bred most with archaics. However, the populations with the highest legacies are globally scattered and are unified, not by having origins within the known Neanderthal range, but instead by living in locations that lie furthest from Africa. Furthermore, the Simons Genome Diversity Project data reveal two distinct correlations between Neanderthal and Denisovan legacies, one that starts in North Africa and increases west to east across Eurasia and into some parts of Oceania, and a second, much steeper trend that starts in Africa, peaking with the San and Ju/’hoansi and which, if extrapolated, predicts the large inferred legacies of both archaics found in Oceania/Australia. Similar ‘double’ trends are observed for the introgression statistic f4 in a second large dataset published by Qin and Stoneking (Qin & Stoneking 2015 Mol. Biol. Evol. 32, 2665–2674 (doi:10.1093/molbev/msv141)). These trends appear at odds with simple models of how introgression occurred though more complicated patterns of introgression could potentially generate better fits. Moreover, substituting archaic genomes with those of great apes yields similar but biologically impossible signals of introgression, suggesting that the signals these metrics capture arise within humans and are largely independent of the test group. Interestingly, the data do appear to fit a speculative model in which the loss of diversity that occurred when humans moved further from Africa created a gradient in heterozygosity that in turn progressively reduced mutation rate such that populations furthest from Africa have diverged less from our common ancestor and hence from the archaics. In this light, the two distinct trends could be interpreted in terms of two ‘out of Africa’ events, an early one ending in Oceania and Australia and a later one that colonized Eurasia and the Americas.