A fundamental question in functional brain development is how the brain acquires specialised processing optimised for its individual environment. The current study is the first to demonstrate that distinct experience of eye gaze communication, due to the visual impairment of a parent, affects the specificity of brain responses to dynamic gaze shifts in infants. Event-related potentials (ERPs) from 6 to 10 months old sighted infants with blind parents (SIBP group) and control infants with sighted parents (CTRL group) were recorded while they observed a face with gaze shifting Toward or Away from them. Unlike the CTRL group, ERPs of the SIBP group did not differentiate between the two directions of gaze shift. Thus, selective brain responses to perceived gaze shifts in infants may depend on their eye gaze communication experience with the primary caregiver. This finding highlights the critical role of early communicative experience in the emerging functional specialisation of the human brain.