The adaption of movement to changes in the environment varies across life span. Recent evidence has linked motor adaptation and its reduction with age to differences in "explicit" learning processes. We examine differences in brain structure and cognition underlying motor adaptation in a population-based cohort (n = 322, aged 18-89 years) using a visuomotor learning task and structural magnetic resonance imaging. Reduced motor adaptation with age was associated with reduced volume in striatum, prefrontal, and sensorimotor cortical regions, but not cerebellum. Medial temporal lobe volume, including the hippocampus, became a stronger determinant of motor adaptation with age. Consistent with the role of the medial temporal lobes, declarative long-term memory showed a similar interaction, whereby memory was more positively correlated with motor adaptation with increasing age. By contrast, visual short-term memory was related to motor adaptation, independently of age. These results support the hypothesis that cerebellar learning is largely unaffected in old age, and the reduction in motor adaptation with age is driven by a decline in explicit memory systems.