My first choice would be a fingernail. Even short ones work fine. If I couldn't use a fingernail, I'd try a different fingernail.
Except for one of the smaller diameters, the glass is rather thick & very resilient. Even if you dropped it, it'd have a pretty decent chance of surviving.
I pull the caps on mine (I have 3 19's & 2 22's) for deep cleaning every so often. It's not that big of a deal. When you first get them, they can be a little stubborn to get off, but it's still not that difficult. The o-rings will break in & soften up a little after you remove them a couple times... not enough to make falling off or having a carto push them out an issue, but a touch easier to pry off.