All-optical modulators are attracting significant attention due to their intrinsic perspective on high-speed, low-loss, and broadband performance, which are promising to replace their electrical counterparts for future information communication technology. However, high-power consumption and large footprint remain obstacles for the prevailing nonlinear optical methods due to the weak photon–photon interaction. Here, efficient all-optical mid-infrared plasmonic waveguide and free-space modulators in atomically thin graphene-MoS2 heterostructures based on the ultrafast and efficient doping of graphene with the photogenerated carrier in the monolayer MoS2 are reported. Plasmonic modulation of 44 cm−1 is demonstrated by an LED with light intensity down to 0.15 mW cm−2, which is four orders of magnitude smaller than the prevailing graphene nonlinear all-optical modulators (≈103 mW cm−2). The ultrafast carrier transfer and recombination time of photogenerated carriers in the heterostructure may achieve ultrafast modulation of the graphene plasmon. The demonstration of the efficient all-optical mid-infrared plasmonic modulators, with chip-scale integrability and deep-sub wavelength light field confinement derived from the van der Waals heterostructures, may be an important step toward on-chip all-optical devices.