I usually pull mine out around 3.6v-3.7v. But you can go a bit lower than that and still be on the healthy side for the batteries. Try not to go below 3.3v as a general rule though since the voltage sag (depending on your build and battery) may drop the battery below the recommended limit of 2.5-2.8v.
It takes some time to get the feel for it. A good rule of thumb is if you notice a drop in flavor or vapor production, it is likely time to swap. I run squonk mechs, so it just happens that my batteries are normally around 3.6v when the bottle is empty, so I swap batteries when I refill the bottle.