What batteries are you using? Most IMR's are safe to drain as low as 2.5 volts, but you want to check your specific battery about that. AW's fall in this category.
If you have a battery that is safe to 2.5 volts that eliminates some of the worry about safety and gives you some room here to understand your Reo. Depending on the resistance of your build you will hit a point where the voltage drops and the performance seems to suddenly poop out. Don't push it at that point - if you pull the battery when you notice the drop in performance chances are you are already at 3.2 volts or lower. With safe chemistry batteries you don't want to keep pushing it and running your voltage into an unsafe range below 2.5 volts. While you are figuring it all out a few drops below 2.5 volts won't cause damage, but you don't want to make that a chronic habit.
As an approach paranoia is a good thing here though. That means pulling the batteries and checking voltage if you think you have noticed a performance drop - and even if you aren't sure.
Since you are using a Reomizer screwing that on and off to use an in-line 510 voltage meter isn't practical.
Your best bet would be to pick up a cheap multimeter for checking battery voltage, a high end meter - like a Fluke - if you need a meter that can accurately measure ohms as well. (Cheap meters test DC voltage ok, but they are absolute crap for checking resistance in the low ranges that we use.)
Personally, I have an Xtar VP1 charger that has a digital display that will tell me the battery voltage when I drop the battery in to charge.
Like I said, you might get it wrong a few times when you start and allow your battery voltage to drop too low. A few times won't critically damage safe chemistry batteries. But you do need a meter of some sort to understand what you have going on.