The segmented vertebral column comprises a repeat series of vertebrae, each consisting of two key components: the vertebral body (or centrum) and the vertebral arches. Despite being a defining feature of the vertebrates, much remains to be understood about vertebral development and evolution. Particular controversy surrounds whether vertebral component structures are homologous across vertebrates, how somite and vertebral patterning are connected, and the developmental origin of vertebral bone-mineralizing cells. Here, we assemble evidence from ichthyologists, palaeontologists and developmental biologists to consider these issues. Vertebral arch elements were present in early stem vertebrates, whereas centra arose later. We argue that centra are homologous among jawed vertebrates, and review evidence in teleosts that the notochord plays an instructive role in segmental patterning, alongside the somites, and contributes to mineralization. By clarifying the evolutionary relationship between centra and arches, and their varying modes of skeletal mineralization, we can better appreciate the detailed mechanisms that regulate and diversify vertebral patterning.