Down's syndrome results from trisomy of chromosome 21, a genetic change which also confers a probable 100% risk for the development of Alzheimer's disease neuropathology (amyloid plaque and neurofibrillary tangle formation) in later life. We aimed to assess the effectiveness of diffusion-weighted imaging and connectomic modelling for predicting brain amyloid plaque burden, baseline cognition and longitudinal cognitive change using support vector regression. Ninety-five participants with Down's syndrome successfully completed a full Pittsburgh Compound B (PiB) PET-MR protocol and memory assessment at two timepoints. Our findings indicate that graph theory metrics of node degree and strength based on the structural connectome are effective predictors of global amyloid deposition. We also show that connection density of the structural network at baseline is a promising predictor of current cognitive performance. Directionality of effects were mainly significant reductions in the white matter connectivity in relation to both PiB+ status and greater rate of cognitive decline. Taken together, these results demonstrate the integral role of the white matter during neuropathological progression and the utility of machine learning methodology for non-invasively evaluating Alzheimer's disease prognosis.