Background: Alzheimer’s disease (AD) is a brain disorder characterized by atrophy of cerebral cortex and neurofibrillary tangles. Accurate identification of individuals at high risk of developing AD is key to early intervention. Combining neuroimaging markers derived from diffusion tensor images with machine learning techniques, unique anatomical patterns can be identified and further distinguished between AD and healthy control (HC). Methods: In this study, 37 AD patients (ADs) and 36 healthy controls (HCs) from the Alzheimer’s Disease Neuroimaging Initiative were applied to tract-based spatial statistics (TBSS) analysis and multi-metric classification research. Results: The TBSS results showed that the corona radiata, corpus callosum and superior longitudinal fasciculus were the white matter fiber tracts which mainly suffered the severe damage in ADs. Using support vector machine recursive feature elimination (SVM-RFE) method, the classification performance received a decent improvement. In addition, the integration of fractional anisotropy (FA) + mean diffusivity (MD) + radial diffusivity (RD) into multi-metric could effectively separate ADs from HCs. The rank of significance of diffusion metrics was FA > axial diffusivity (DA) > MD > RD in our research. Conclusions: Our findings suggested that the TBSS and machine learning method could play a guidance role on clinical diagnosis.