A wealth of animal and human studies demonstrate that perinatal exposure to adverse metabolic conditions - be it maternal obesity, diabetes or under-nutrition - results in predisposition of offspring to develop obesity later in life. This mechanism is a contributing factor to the exponential rise in obesity rates. Increased weight gain in offspring exposed to maternal obesity is usually associated with hyperphagia, implicating altered central regulation of energy homeostasis as an underlying cause. Perinatal development of the hypothalamus (a brain region key to metabolic regulation) is plastic and sensitive to metabolic signals during this critical time window. Recent research in non-human primate and rodent models has demonstrated that exposure to adverse maternal environments impairs the development of hypothalamic structure and consequently function, potentially underpinning metabolic phenotypes in later life. This review summarizes our current knowledge of how adverse perinatal environments program hypothalamic development and explores the mechanisms that could mediate these effects.