They eventually come off easier, but that is about the same time they start leaking!
I don't bother; I just picked up a bunch of needle tip caps for my juice bottles. If you also use plastic tipped juice bottles, you can actually just press the tip to the little hole in the bottom of the tank and fill it- just don't press too hard or it will leak out of the sides. Takes a little practice but easy once you get the hang of it, just try to keep from forming the little "juice seal" at the opening!
/r
Allen