As the battery drains, your mod is asking for more current to maintain the power level of the vape. So long as your batteries' CDR is reasonably above the highest required current, you're not likely damaging them by using them until the mod cuts off.
Lots of folks say batteries last a lot longer by never letting their voltage drop below 3.5-3.6V. This may be true, but we're talking 200-500 charge cycles versus 400-500 for any decent battery used appropriately.
You likely get as much total use from your batteries letting the mod cut off versus if you pull them early, and you spend less time changing batteries.
If your batteries seem to last about as long as they did new, I'm pretty sure they're fine.
Helpful information to make more certain statements would be:
What wattage do you vape at?
What are your batteries?
Where did you purchase them?
People more helpful than me will be along soon, but if you can answer those questions they'll be better able to help you (or just reassure you, or start a fresh debate about when to pull batteries).