In recent years, there has been increased interest in the scalp-recorded frequency following response (FFR), which is an electrical signal that reflects sustained phase locking to sound of large populations of neurons mainly in the upper brainstem in response to stimulus-related periodicities. It provides a non-invasive measure of neural processing in humans, which can be compared to behavioural responses concerning the listener’s perception. It has been argued that the FFR reflects processes important for the perception of pitch and that changes in the FFR with experience and/or training provide a measure of neural plasticity at the level of the brainstem. This paper reviews recent work aimed at elucidating the origin and the specifics of the information present in the FFR. It is argued that the neural responses measured by the FFR preserve temporal information important for pitch to a certain degree, but do not necessarily represent pitch-related processing over and above that present in the auditory periphery. In addition, multiple generators may affect the overall measure to various degrees, depending on the repetition rate of the stimulus.