A fully charged battery will be between 4.1 - 4.2 volts. I tend to use the battery until either the vape becomes noticeably weak or when the delivery device no longer produces vapor. At that point the battery will be around 3.4 - 3.5 volts, not what would be considered a "dead battery" in the strictest sense, but needing a charge back up to full capacity.
With experience you'll learn when the vape becomes weak and know that the battery is low, and that would be the ideal time to switch out your battery. It's always a good idea to have a voltmeter in your vape setup to observe your battery's health by checking the voltage after removing the battery from both your PV and the charger.