Strong evidence suggests that early-life exposures to suboptimal environmental factors, including those in utero, influence our long-term metabolic health. This has been termed developmental programming. Mounting evidence suggests that the growth and metabolism of male and female fetuses differ. Therefore, sexual dimorphism in response to pre-conception or early-life exposures could contribute to known sex differences in susceptibility to poor metabolic health in adulthood. However, until recently, many studies, especially those in animal models, focused on a single sex, or, often in the case of studies performed during intrauterine development, did not report the sex of the animal at all. In this review, we (a) summarize the evidence that male and females respond differently to a suboptimal pre-conceptional or in utero environment, (b) explore the potential biological mechanisms that underlie these differences and (c) review the consequences of these differences for long-term metabolic health, including that of subsequent generations.