Cellular senescence has been implicated in tumor suppression, development, and aging and is accompanied by large-scale chromatin rearrangements, forming senescence-associated heterochromatic foci (SAHF). However, how the chromatin is reorganized during SAHF formation is poorly understood. Furthermore, heterochromatin formation in senescence appears to contrast with loss of heterochromatin in Hutchinson-Gilford progeria. We mapped architectural changes in genome organization in cellular senescence using Hi-C. Unexpectedly, we find a dramatic sequence- and lamin-dependent loss of local interactions in heterochromatin. This change in local connectivity resolves the paradox of opposing chromatin changes in senescence and progeria. In addition, we observe a senescence-specific spatial clustering of heterochromatic regions, suggesting a unique second step required for SAHF formation. Comparison of embryonic stem cells (ESCs), somatic cells, and senescent cells shows a unidirectional loss in local chromatin connectivity, suggesting that senescence is an endpoint of the continuous nuclear remodelling process during differentiation.