Multiprotein chromatin remodelling complexes show remarkable conservation of function amongst metazoans, even though components present in invertebrates are often found as multiple paralogous proteins in vertebrate complexes. In some cases, these paralogues specify distinct biochemical and/or functional activities in vertebrate cells. Here, we set out to define the biochemical and functional diversity encoded by one such group of proteins within the mammalian Nucleosome Remodelling and Deacetylation (NuRD) complex: Mta1, Mta2 and Mta3. We find that, in contrast to what has been described in somatic cells, MTA proteins are not mutually exclusive within embryonic stem (ES) cell NuRD and, despite subtle differences in chromatin binding and biochemical interactions, serve largely redundant functions. ES cells lacking all three MTA proteins exhibit complete NuRD loss of function and are viable, allowing us to identify a previously unreported function for NuRD in reducing transcriptional noise, which is essential for maintaining a proper differentiation trajectory during early stages of lineage commitment.