Music is an important part of life for many people. It can evoke a wide range of emotions, including sadness, happiness, anger, tension, relief, and excitement. People with hearing loss and people with cochlear implants have reduced abilities to discriminate some of the features of musical sounds that may be involved in evoking emotions. This paper reviews these changes in perceptual abilities and describes how they affect the perception of emotion in music. For people with acquired partial hearing loss, it appears that the perception of emotion in music is almost normal, whereas congenital partial hearing loss is associated with impaired perception of music emotion. For people with cochlear implants, the ability to discriminate changes in fundamental frequency (associated with perceived pitch) is much worse than normal and musical harmony is hardly perceived. As a result, people with cochlear implants appear to judge emotion in music primarily using tempo and rhythm cues, and this limits the range of emotions that can be judged.