Silica phases, SiO2, have attracted significant attention as important phases in the fields of condensed matter physics, materials science, and (in view of their abundance in Earth’s crust) geoscience. Here, we experimentally and theoretically demonstrate that coesite undergoes structural modulations under high pressure. Coesite transforms to a distorted modulated structure, coesite-II, at 22-25 GPa with modulation wave vector q = 0.5b*. Coesite-II displays further commensurate modulation along the y-axis at 36-40 GPa and the long-range ordered crystalline structure collapses beyond ~40 GPa and starts amorphizing. First-principles calculations illuminate the nature of the modulated phase transitions of coesite and elucidate the modulated structures of coesite caused by modulations along y-axis direction. The structural modulations are demonstrated to result from phonon instability, preceding pressured-induced amorphization. The recovered sample after decompression develops a rim of crystalline coesite structure, but its interior remains low crystalline or partially amorphous. Our results not only clarify that the pressure-induced reversible phase transitions and amorphization in coesite originate from structural modulations along y-axis direction, but also shed light on the densification mechanism of silica under high pressure.