Domain wall motion driven by ultra-short laser pulses is a pre-requisite for envisaged low-power spintronics combining storage of information in magnetoelectronic devices with high speed and long distance transmission of information encoded in circularly polarized light. Here we demonstrate the conversion of the circular polarization of incident femtosecond laser pulses into inertial displacement of a domain wall in a ferromagnetic semiconductor. In our study, we combine electrical measurements and magneto-optical imaging of the domain wall displacement with micromagnetic simulations. The optical spin-transfer torque acts over a picosecond recombination time of the spin-polarized photo-carriers that only leads to a deformation of the initial domain wall structure. We show that subsequent depinning and micrometre-distance displacement without an applied magnetic field or any other external stimuli can only occur due to the inertia of the domain wall.