Dioxygenases of the TET (Ten-Eleven Translocation) family produce oxidized methylcytosines, intermediates in DNA demethylation, as well as new epigenetic marks. Here we show data suggesting that TET proteins maintain the consistency of gene transcription. Embryos lacking Tet1 and Tet3 (Tet1/3 DKO) displayed a strong loss of 5-hydroxymethylcytosine (5hmC) and a concurrent increase in 5-methylcytosine (5mC) at the eight-cell stage. Single cells from eight-cell embryos and individual embryonic day 3.5 blastocysts showed unexpectedly variable gene expression compared with controls, and this variability correlated in blastocysts with variably increased 5mC/5hmC in gene bodies and repetitive elements. Despite the variability, genes encoding regulators of cholesterol biosynthesis were reproducibly down-regulated in Tet1/3 DKO blastocysts, resulting in a characteristic phenotype of holoprosencephaly in the few embryos that survived to later stages. Thus, TET enzymes and DNA cytosine modifications could directly or indirectly modulate transcriptional noise, resulting in the selective susceptibility of certain intracellular pathways to regulation by TET proteins.