The exploitation of the concept of Digital Twins, i.e. virtual copies of physical assets, for existing rail infrastructure has the potential to revolutionise asset management in this sector. However, such exploitation is only possible if methods exist can cost effectively generate the Digital Twins of rail assets. The first step in this “twinning” process is the capture of the asset’s raw geometry and its conversion to high level geometry suitable for further enrichment with design, construction, operation and maintenance data. This paper investigates the state of the art in the first twinning step, i.e. generating geometrically accurate models of existing rail infrastructure, focusing on the track assets. The paper starts off by defining the digital twin, then explaining the benefits of real-virtual synchronisation and challenges to exploit the digital twin in its full potential. The subsequent sections provide a longitudinal literature indicate that current studies are sensitive to varying railway geometries, neighbourhood structures, scanning geometry and intensity of input data. These factors render methods designed for digital twinning ineffective for any track structure which contains varying horizontal and vertical elevations. Such variance is quite common; hence, we conclude that the problem of automatically generating geometric digital twins of track structure is yet to be solved.