High-performance photodetectors operating over a broad wavelength range from ultraviolet, visible, to infrared are of scientific and technological importance for a wide range of applications. Here, a photodetector based on van der Waals heterostructures of graphene and its fluorine-functionalized derivative is presented. It consistently shows broadband photoresponse from the ultraviolet (255 nm) to the mid-infrared (4.3 µm) wavelengths, with three orders of magnitude enhanced responsivity compared to pristine graphene photodetectors. The broadband photodetection is attributed to the synergistic effects of the spatial nonuniform collective quantum confinement of sp(2) domains, and the trapping of photoexcited charge carriers in the localized states in sp(3) domains. Tunable photoresponse is achieved by controlling the nature of sp(3) sites and the size and fraction of sp(3) /sp(2) domains. In addition, the photoresponse due to the different photoexcited-charge-carrier trapping times in sp(2) and sp(3) nanodomains is determined. The proposed scheme paves the way toward implementing high-performance broadband graphene-based photodetectors.