In development, wound healing, and cancer metastasis, vertebrate cells move through 3D interstitial matrix, responding to chemical and physical guidance cues. Protrusion at the cell front has been extensively studied, but the retraction phase of the migration cycle is not well understood. Here, we show that fast-moving cells guided by matrix cues establish positive feedback control of rear retraction by sensing membrane tension. We reveal a mechanism of rear retraction in 3D matrix and durotaxis controlled by caveolae, which form in response to low membrane tension at the cell rear. Caveolae activate RhoA-ROCK1/PKN2 signaling via the RhoA guanidine nucleotide exchange factor (GEF) Ect2 to control local F-actin organization and contractility in this subcellular region and promote translocation of the cell rear. A positive feedback loop between cytoskeletal signaling and membrane tension leads to rapid retraction to complete the migration cycle in fast-moving cells, providing directional memory to drive persistent cell migration in complex matrices.