Colossal owner and modifier here...
Both of the Colossal mods I own had this issue, until I did a little modification on the raised plastic, surrounding the positive post of the mod. And yes... the first time I pulled the battery out, my battery wrap was toast; ripped it, but good!
However, after I filed down the raised plastic surrounding the positive post of the mod, the battery slips out of the mod like butter.
I used a the rough file on my multi-tool... but I’m sure a rough nail file/emory board would work as well; perhaps not as fast, but it would eventually file that plastic down.
Good luck with modding your Colossal.