Objective Little is known about the mechanisms by which obesity influences brain structure. In this study, the obesity-related changes in brain white and gray matter integrity were examined. Design and Methods 23 morbidly obese subjects and 22 nonobese volunteers were studied using voxel-based analysis of diffusion tensor imaging and of T1-weighted MRI images. Full-volume statistical parametric mapping analysis was used to compare fractional anisotropy (FA) and mean diffusivity (MD) values as well as gray (GM) and white matter (WM) density between these groups. Results Obese subjects had lower FA and MD values and lower focal and global GM and WM volumes than control subjects did. The focal structural changes were observed in brain regions governing reward seeking, inhibitory control, and appetite. Regression analysis showed that FA and MD values as well as GM and WM density were negatively associated with body fat percentage. Moreover, the volume of abdominal subcutaneous fat was negatively associated with GM density in most regions. Conclusion These findings imply that changes in GM and WM in obesity may be due to metabolic factors. Atrophy in regions involved in reward processing and appetite control may further promote abnormal reward seeking and eating behavior.