The formation of trophectoderm (TE) and pluripotent inner cell mass (ICM) is one of the earliest events during mammalian embryogenesis. It is believed that the orientation of division of polarised blastomeres in the 8- and 16-cell stage embryo determines the fate of daughter cells, based on how asymmetrically distributed lineage determinants are segregated. To investigate the relationship between angle of division and subsequent fate in unperturbed embryos, we constructed cellular resolution digital representations of the development of mouse embryos from the morula to early blastocyst stage, based on 4D confocal image volumes. We find that at the 16-cell stage, very few inside cells are initially produced as a result of cell division, but that the number increases due to cell movement. Contrary to expectations, outside cells at the 16-cell stage represent a heterogeneous population, with some fated to contributing exclusively to the TE and others capable of contributing to both the TE and ICM. Our data support the view that factors other than the angle of division, such as the position of a blastomere, play a major role in the specification of TE and ICM.