We demonstrate how terahertz time-domain spectroscopy (THz-TDS) operating in reflection geometry can be used for quantitative conductivity mapping of large area chemical vapour deposited graphene films on sapphire, silicon dioxide/silicon and germanium. We validate the technique against measurements performed with previously established conventional transmission based THz-TDS and are able to resolve conductivity changes in response to induced back-gate voltages. Compared to the transmission geometry, measurement in reflection mode requires careful alignment and complex analysis, but circumvents the need of a terahertz transparent substrate, potentially enabling fast, contactless, in-line characterisation of graphene films on non-insulating substrates such as germanium.