Advanced aging negatively impacts memory performance. Brain aging has been associated with shrinkage in medial temporal lobe structures essential for memory-including hippocampus and entorhinal cortex-and with deficits in default-mode network connectivity. Yet, whether and how these imaging markers are relevant to age-related memory deficits remains a topic of debate. Using a sample of 182 older (age 74.6 ± 6.2 years) and 66 young (age 22.2 ± 3.6 years) participants, this study examined relationships among memory performance, hippocampus volume, entorhinal cortex thickness, and default-mode network connectivity across aging. All imaging markers and memory were significantly different between young and older groups. Each imaging marker significantly mediated the relationship between age and memory performance and collectively accounted for most of the variance in age-related memory performance. Within older participants, default-mode connectivity and hippocampus volume were independently associated with memory. Structural equation modeling of cross-sectional data within older participants suggest that entorhinal thinning may occur before reduced default-mode connectivity and hippocampal volume loss, which in turn lead to deficits in memory performance.