Progress in the terahertz (THz) region of the electromagnetic spectrum is undergoing major advances, with advanced THz sources and detectors being developed at a rapid pace. Yet, ultrafast THz communication is still to be realized, owing to the lack of practical and effective THz modulators. Here, we present a novel ultrafast active THz polarization modulator based on GaAs semiconductor nanowires arranged in a wire-grid configuration. We utilize an optical pump-terahertz probe spectroscopy system and vary the polarization of the optical pump beam to demonstrate ultrafast THz modulation with a switching time of less than 5 ps and a modulation depth of -8 dB. We achieve an extinction of over 13% and a dynamic range of -9 dB, comparable to microsecond-switchable graphene- and metamaterial-based THz modulators, and surpassing the performance of optically switchable carbon nanotube THz polarizers. We show a broad bandwidth for THz modulation between 0.1 and 4 THz. Thus, this work presents the first THz modulator which combines not only a large modulation depth but also a broad bandwidth and picosecond time resolution for THz intensity and phase modulation, making it an ideal candidate for ultrafast THz communication.