Recent lineage tracing studies have revealed that mammary gland homeostasis relies on unipotent stem cells. However, whether and when lineage restriction occurs during embryonic mammary development, and which signals orchestrate cell fate specification, remain unknown. Using a combination of in vivo clonal analysis with whole mount immunofluorescence and mathematical modelling of clonal dynamics, we found that embryonic multipotent mammary cells become lineage-restricted surprisingly early in development, with evidence for unipotency as early as E12.5 and no statistically discernable bipotency after E15.5. To gain insights into the mechanisms governing the switch from multipotency to unipotency, we used gain-of-function Notch1 mice and demonstrated that Notch activation cell autonomously dictates luminal cell fate specification to both embryonic and basally committed mammary cells. These functional studies have important implications for understanding the signals underlying cell plasticity and serve to clarify how reactivation of embryonic programs in adult cells can lead to cancer.