Transcranial magnetic stimulation (TMS) is a non-invasive brain stimulation technique whose effects on neural activity can be uncertain. Within the visual cortex, phosphenes are a useful marker of TMS: They indicate the induction of neural activation that propagates and creates a conscious percept. However, we currently do not know how susceptible different areas of the visual cortex are to TMS-induced phosphenes. In this study, we systematically map out locations in the visual cortex where stimulation triggered phosphenes. We relate this to the retinotopic organization and the location of object- and motion-selective areas, identified by functional magnetic resonance imaging (fMRI) measurements. Our results show that TMS can reliably induce phosphenes in early (V1, V2d, and V2v) and dorsal (V3d and V3a) visual areas close to the interhemispheric cleft. However, phosphenes are less likely in more lateral locations (hMT+/V5 and LOC). This suggests that early and dorsal visual areas are particularly amenable to TMS and that TMS can be used to probe the functional role of these areas.