Seismic anisotropy throughout the oceanic lithosphere is often assumed to be generated by fossilized texture formed during deformation at asthenospheric temperatures close to the ridge. Here we investigate the effect of high-temperature and high-pressure static annealing on the texture of previously deformed olivine aggregates to simulate residence of deformed peridotite in the lithosphere. Our experiments indicate that the orientation and magnitude of crystallographic preferred orientation (CPO) will evolve due to the preferential growth of grains with low dislocation densities. These observations suggest that texture and stored elastic strain energy promote a style of grain growth that modifies the CPO of a deformed aggregate. We demonstrate that these microstructural changes alter the orientation distributions and magnitudes of seismic wave velocities and anisotropy. Therefore, static annealing may complicate the inference of past deformation kinematics from seismic anisotropy in the lithosphere.