Individuals with Down syndrome (DS) are more likely to experience earlier onset of multiple facets of physiological aging. This includes brain atrophy, beta amyloid deposition, cognitive decline, and Alzheimer's disease-factors indicative of brain aging. Here, we employed a machine learning approach, using structural neuroimaging data to predict age (i.e., brain-predicted age) in people with DS (N = 46) and typically developing controls (N = 30). Chronological age was then subtracted from brain-predicted age to generate a brain-predicted age difference (brain-PAD) score. DS participants also underwent [11C]-PiB positron emission tomography (PET) scans to index the levels of cerebral beta amyloid deposition, and cognitive assessment. Mean brain-PAD in DS participants' was +2.49 years, significantly greater than controls (p < 0.001). The variability in brain-PAD was associated with the presence and the magnitude of PiB-binding and levels of cognitive performance. Our study indicates that DS is associated with premature structural brain aging, and that age-related alterations in brain structure are associated with individual differences in the rate of beta amyloid deposition and cognitive impairment.