Fin whales Balaenoptera physalus were hunted unsustainably across the globe in the 19th and 20th centuries, leading to vast reductions in population size. Whaling catch records indicate the importance of the Southern Ocean for this species; approximately 730,000 fin whales were harvested during the 20th century in the Southern Hemisphere (SH) alone, 94% of which were at high latitudes. Genetic samples from contemporary whales can provide a window to past population size changes, but the challenges of sampling in remote Antarctic waters limit the availability of data. Here, we take advantage of historical samples in the form of bones and baleen available from ex-whaling stations and museums to assess the pre-whaling diversity of this once abundant species. We sequenced 27 historical mitogenomes and 50 historical mitochondrial control region sequences of fin whales to gain insight into the population structure and genetic diversity of Southern Hemisphere fin whales (SHFWs) before and after the whaling. Our data, both independently and when combined with mitogenomes from the literature, suggest SHFWs are highly diverse and may represent a single panmictic population that is genetically differentiated from Northern Hemisphere populations. These are the first historic mitogenomes available for SHFWs, providing a unique time series of genetic data for this species.