Creating and sustaining closed-loop dynamic and social interactions with humans require robots to continually adapt towards their users’ behaviours, their affective states and moods while keeping them engaged in the task they are performing. Analysing, understanding and appropriately responding to human nonverbal behaviour and affective states are the central objectives of affective robotics research. Conventional machine learning approaches do not scale well to the dynamic nature of such real-world interactions as they require samples from stationary data distributions. The real-world is not stationary, it changes continuously. In such contexts, the training data and learning objectives may also change rapidly. Continual Learning (CL), by design, is able to address this very problem by learning incrementally. In this paper, we argue that CL is an essential paradigm for creating fully adaptive affective robots (why). To support this argument, we first provide an introduction to CL approaches and what they can offer for various dynamic (interactive) situations (what). We then formulate guidelines for the affective robotics community on how to utilise CL for perception and behaviour learning with adaptation (how). For each case, we reformulate the problem as a CL problem and outline a corresponding CL-based solution. We conclude the paper by highlighting the potential challenges to be faced and by providing specific recommendations on how to utilise CL for affective robotics.