Is this for a custom mod, or another application not related to vaping?
In any case, what you're looking for is a buck/boost DC/DC converter, because your battery will start at 4.2V when fully charged which will need to be lowered to your desired 4V output; however, as the battery gets drained its voltage will drop towards its nominal value of 3.7V, in which case you'll need to boost the output to 4V. For vaping, there are several pre-built modules that will do that such as the DNA30, SX350, ZMax... For non-vaping applications, search mouser or digikey for buck/boost converters matching your desired specs.