The small molecule ISRIB antagonizes the activation of the integrated stress response (ISR) by phosphorylated translation initiation factor 2, eIF2(αP). ISRIB and eIF2(αP) bind distinct sites in their common target, eIF2B, a guanine nucleotide exchange factor for eIF2. We have found that ISRIB-mediated acceleration of eIF2B's nucleotide exchange activity in vitro is observed preferentially in the presence of eIF2(αP) and is attenuated by mutations that desensitize eIF2B to the inhibitory effect of eIF2(αP). ISRIB's efficacy as an ISR inhibitor in cells also depends on presence of eIF2(αP). Cryoelectron microscopy (cryo-EM) showed that engagement of both eIF2B regulatory sites by two eIF2(αP) molecules remodels both the ISRIB-binding pocket and the pockets that would engage eIF2α during active nucleotide exchange, thereby discouraging both binding events. In vitro, eIF2(αP) and ISRIB reciprocally opposed each other's binding to eIF2B. These findings point to antagonistic allostery in ISRIB action on eIF2B, culminating in inhibition of the ISR.