Aging and Alzheimer's disease (AD) are associated with progressive brain disorganization. Although structural asymmetry is an organizing feature of the cerebral cortex it is unknown whether continuous age- and AD-related cortical degradation alters cortical asymmetry. Here, in multiple longitudinal adult lifespan cohorts we show that higher-order cortical regions exhibiting pronounced asymmetry at age ~20 also show progressive asymmetry-loss across the adult lifespan. Hence, accelerated thinning of the (previously) thicker homotopic hemisphere is a feature of aging. This organizational principle showed high consistency across cohorts in the Lifebrain consortium, and both the topological patterns and temporal dynamics of asymmetry-loss were markedly similar across replicating samples. Asymmetry-change was further accelerated in AD. Results suggest a system-wide dedifferentiation of the adaptive asymmetric organization of heteromodal cortex in aging and AD.