Extended inverse Compton halos are generally anticipated around extragalactic sources of gamma rays with energies above 100 GeV. These result from inverse Compton scattered cosmic microwave background photons by a population of high-energy electron/positron pairs produced by the annihilation of the high-energy gamma rays on the infrared background. Despite the observed attenuation of the high-energy gamma rays, the halo emission has yet to be directly detected. Here, we demonstrate that in most cases these halos are expected to be highly anisotropic, distributing the up-scattered gamma rays along axes defined either by the radio jets of the sources or oriented perpendicular to a global magnetic field. We present a pedagogical derivation of the angular structure in the inverse Compton halo and provide an analytic formalism that facilitates the generation of mock images. We discuss exploiting this fact for the purpose of detecting gamma-ray halos in a set of companion papers.