The observed dipole anisotropy of the cosmic microwave background (CMB) temperature is much larger than the fluctuations observed on smaller scales and is dominated by the kinematic contribution from the Doppler shifting of the monopole due to our motion with respect to the CMB rest frame. In addition to this kinematic component, there is expected to be an intrinsic contribution with an amplitude about two orders of magnitude smaller. Here we explore a method whereby the intrinsic CMB dipole can be reconstructed through observation of temperature fluctuations on small scales which result from gravitational lensing. Though the experimental requirements pose practical challenges, we show that one can in principle achieve a cosmic variance limited measurement of the primary dipole using the reconstruction method we describe. Since the primary CMB dipole is sensitive to the largest observable scales, such a measurement would have a number of interesting applications for early universe physics, including testing large-scale anomalies, extending the lever-arm for measuring local non-Gaussianity, and constraining isocurvature fluctuations on super-horizon scales.