As with what dormouse posted, there isn't much danger with a sealed manual battery unless enough juice leaks out to run down the body and possibly short the switch. CE2s in particular don't have a whole lot of issue with leakage as long as you're careful filling. I've found that you have to fill them relatively slowly, as if you fill too quickly and the liquid is anything more than pure PG, it doesn't slide down past the edge of the cup fast enough and some can back up at the lip and run inside, which means it's on its way down the air tube in the middle, thus leakage. The new CE3 smokeymizers are more of an issue because of the bottom coil design. They can leak when filling, and they can leak if you don't have that tiny stopper inserted properly.
Regardless, I always remove to fill. I just find it easy, and if it does leak there won't be any issue.