It is well established that brain structures and cognitive functions change across the life span. A long-standing hypothesis called "age differentiation" additionally posits that the relations between cognitive functions also change with age. To date, however, evidence for age-related differentiation is mixed, and no study has examined differentiation of the relationship between brain and cognition. Here we use multigroup structural equation models (SEMs) and SEM trees to study differences within and between brain and cognition across the adult life span (18-88 years) in a large (N > 646, closely matched across sexes), population-derived sample of healthy human adults from the Cambridge Centre for Ageing and Neuroscience (www.cam-can.org). After factor analyses of gray matter volume (from T1- and T2-weighted MRI) and white matter organization (fractional anisotropy from diffusion-weighted MRI), we found evidence for the differentiation of gray and white matter, such that the covariance between brain factors decreased with age. However, we found no evidence for age differentiation among fluid intelligence, language, and memory, suggesting a relatively stable covariance pattern among cognitive factors. Finally, we observed a specific pattern of age differentiation between brain and cognitive factors, such that a white matter factor, which loaded most strongly on the hippocampal cingulum, became less correlated with memory performance in later life. These patterns are compatible with the reorganization of cognitive functions in the face of neural decline, and/or with the emergence of specific subpopulations in old age.SIGNIFICANCE STATEMENT The theory of age differentiation posits age-related changes in the relationships among cognitive domains, either weakening (differentiation) or strengthening (dedifferentiation), but evidence for this hypothesis is mixed. Using age-varying covariance models in a large cross-sectional adult life span sample, we found age-related reductions in the covariance among both brain measures (neural differentiation), but no covariance change among cognitive factors of fluid intelligence, language, and memory. We also observed evidence of uncoupling (differentiation) between a white matter factor and cognitive factors in older age, most strongly for memory. Together, our findings support age-related differentiation as a complex, multifaceted pattern that differs for brain and cognition, and discuss several mechanisms that might explain the changing relationship between brain and cognition.