Skillful object manipulation requires learning the dynamics of objects, linking applied force to motion 1 ,2 . This involves the formation of a motor memory 3 ,4 , which has been assumed to be associated with the object, independent of the point on the object that one chooses to control. Importantly, in manipulation tasks, different control points on an object, such as the rim of a cup when drinking or its base when setting it down, can be associated with distinct dynamics. Here we show that opposing dynamic perturbations, which interfere when controlling a single location on an object, can be learned when each is associated with a separate control point. This demonstrates that motor memory formation is linked to control points on the object, rather than the object per se . We also show that the motor system only generates separate memories for different control points if they are linked to different dynamics, allowing efficient use of motor memory. To account for these results, we develop a normative switching state-space model of motor learning, in which the association between cues (control points) and contexts (dynamics) is learned rather than fixed. Our findings uncover an important mechanism through which the motor system generates flexible and dexterous behavior.