Massive, quiescent galaxies at high redshift have been found to be considerably more compact than galaxies of similar mass in the local universe. How these compact galaxies formed has yet to be determined, though several progenitor populations have been proposed. Here we investigate the formation processes and quantify the assembly histories of such galaxies in Illustris, a suite of hydrodynamical cosmological simulations encompassing a sufficiently large volume to include rare objects, while simultaneously resolving the internal structure of galaxies. We select massive (~10^11 solar masses) and compact (stellar half-mass radius < 2 kpc) galaxies from the simulation at z=2. Within the Illustris suite, we find that these quantities are not perfectly converged, but are reasonably reliable for our purposes. The resulting population is composed primarily of quiescent galaxies, but we also find several star-forming compact galaxies. The simulated compact galaxies are similar to observed galaxies in star formation activity and appearance. We follow their evolution at high redshift in the simulation and find that there are multiple pathways to form these compact galaxies, dominated by two mechanisms: (i) intense, centrally concentrated starbursts generally triggered by gas-rich major mergers between z~2-4, reducing the galaxies' half-mass radii by a factor of a few to below 2 kpc, and (ii) assembly at very early times when the universe was much denser; the galaxies formed compact and remained so until z~2.