Most mathematical cognition research has focused on understanding normal adult function and child development as well as mildly and moderately impaired mathematical skill, often labeled developmental dyscalculia and/or mathematical learning disability. In contrast, much less research is available on cognitive and neural correlates of gifted/excellent mathematical knowledge in adults and children. In order to facilitate further inquiry into this area, here we review 40 available studies, which examine the cognitive and neural basis of gifted mathematics. Studies associated a large number of cognitive factors with gifted mathematics, with spatial processing and working memory being the most frequently identified contributors. However, the current literature suffers from low statistical power, which most probably contributes to variability across findings. Other major shortcomings include failing to establish domain and stimulus specificity of findings, suggesting causation without sufficient evidence and the frequent use of invalid backward inference in neuro-imaging studies. Future studies must increase statistical power and neuro-imaging studies must rely on supporting behavioral data when interpreting findings. Studies should investigate the factors shown to correlate with math giftedness in a more specific manner and determine exactly how individual factors may contribute to gifted math ability.