I turn mine off every time I set it down, stick it in my pocket, or put it in my purse... for different reasons.

My PV is usually an arm's reach away from wherever I am. If I didn't turn it off, it would likely end up in between couch cushions or wedged underneath my pillow. That would not be good.
I think it wouldn't make much difference in battery life if you left it on... even turning it off shouldn't take too much extra power, the led isn't going to be a major drain on the battery, I would think.