A complex-valued transmission matrix describing a scattering medium can be constructed from a sequence of many interferometric measurements. A major challenge in such experiments is to correct for rapid phase drift of the optical system during the data acquisition process, especially when the phase drifts significantly between consecutive measurements. Therefore, a new method is presented whereby the exact phase drift between two measurements is characterised and corrected for using a single additional measurement. This approach removes the need to continuously track the phase and significantly relaxes the phase stability requirements of the interferometer, allowing transmission matrices to be constructed in the presence of fast and erratic phase drift.