Railhead roughness increases over time, leading to increased environmental noise and vibration. The use of axle-box acceleration (ABA) measurements on in-service railway vehicles to monitor rail roughness is potentially more cost-effective than other techniques. The measured acceleration requires signal processing to derive suitable metrics of railhead condition. A transfer function may be calibrated with direct roughness and ABA measurements made on a reference track, which may then be used to derive roughness spectra from subsequent ABA measurements. However, this approach is affected by variations in track dynamic behaviour, as well as variations in wheel roughness, which is inherently combined with rail roughness in the ABA measurement. This paper proposes an improved approach that (i) extracts the track’s dynamic stiffness parameters from the ABA measurements, enabling the derivation of the roughness-ABA transfer function for each section of track, and (ii) separates the wheel and rail roughness by synchronous averaging over several wheel revolutions. By accounting for variations in track properties and removing the influence of wheel roughness, initial modelling indicates that reliable measurements of rail roughness spectra can be obtained in practice.