Inhibitor of differentiation (ID) proteins dimerize with basic HLH (bHLH) transcription factors, repressing transcription of lineage-specification genes across diverse cellular lineages. ID4 is a key regulator of mammary stem cells; however, the mechanism by which it achieves this is unclear. Here, we show that ID4 has a cell autonomous role in preventing myoepithelial differentiation of basal cells in mammary organoids and in vivo. ID4 positively regulates proliferative genes and negatively regulates genes involved in myoepithelial function. Mass spectrometry reveals that ID4 interacts with the bHLH protein HEB, which binds to E-box motifs in regulatory elements of basal developmental genes involved in extracellular matrix and the contractile cytoskeleton. We conclude that high ID4 expression in mammary basal stem cells antagonizes HEB transcriptional activity, preventing myoepithelial differentiation and allowing for appropriate tissue morphogenesis. Downregulation of ID4 during pregnancy modulates gene regulated by HEB, promoting specialization of basal cells into myoepithelial cells.