Abstract: Background: Nearly all persons with Down syndrome will show pathology of Alzheimer’s disease in their 40s. There is a critical need for studies to identify early biomarkers of these various pathological changes of Alzheimer’s disease in the Down syndrome population and understand the relationship of these biomarkers to cognitive symptoms in order to inform clinical trials. Although Alzheimer’s disease is often considered a disease of gray matter, white matter degeneration has been documented during the preclinical stage of Alzheimer’s disease. The current study examined the association between diffusion tensor imaging (DTI) measures of white matter microstructure and episodic memory performance in 52 adults with Down syndrome. Methods: Seventy (N = 70) participants (M = 40.13, SD = 7.77 years) received baseline scans as part of the Neurodegeneration in Aging Down Syndrome (NiAD) study at two imaging facilities (36 at the University of Wisconsin-Madison [UW-Madison] and 34 at the University of Pittsburgh Medical Center [UPMC]). All participants had genetically confirmed trisomy 21. Fifty-two (N = 52) participants remained after QC. The DTI measures, fractional anisotropy (FA) and mean diffusivity (MD), were calculated for each participant. A combined measure of episodic memory was generated by summing the z-scores of (1) Free and Cued Recall test and (2) Rivermead Behavioural Memory Test for Children Picture Recognition. The DTI data were projected onto a population-derived FA skeleton and tract-based spatial statistics analysis was conducted using the FSL tool PALM to calculate Pearson’s r values between FA and MD with episodic memory. Results: A positive correlation of episodic memory with FA and a negative correlation of episodic memory and MD in the major association white matter tracts were observed. Results were significant (p < 0.05) after correction for chronological age, imaging site, and premorbid cognitive ability. Conclusion: These findings suggest that white matter degeneration may be implicated in early episodic memory declines prior to the onset of dementia in adults with Down syndrome. Further, our findings suggest a coupling of episodic memory and white matter microstructure independent of chronological age.