The morphogenesis of branched organs remains a subject of abiding interest. Although much is known about the underlying signaling pathways, it remains unclear how macroscopic features of branched organs, including their size, network topology and spatial patterning, are encoded. Here we show that, in mouse mammary gland, kidney and human prostate, these features can be explained quantitatively within a single unifying framework of branching and annihilating random walks. Based on quantitative analyses of large-scale organ reconstructions and proliferation kinetics measurements, we propose that morphogenesis follows from the proliferative activity of equipotent tips that stochastically branch and randomly explore their environment, but compete neutrally for space, becoming proliferatively inactive when in proximity with neighboring ducts. These results show that complex branched epithelial structures in mammalian tissues develop as a self-organized process, reliant upon a strikingly simple, but generic, rule, without recourse to a rigid and deterministic sequence of genetically programmed events.