Hi bjannr. The problem would be caused by 3 things that I can think of. So let's start with the easiest one.
Make sure the device is full unlocked. Turn the notched rotating locking mechanism all the way to the left until you meet resistance. Is it fully open and unlocked and still not able to push it in? If so...
Continue to turn the locking mechanism to the left until the entire switch assembly starts to turn. Take the assembly out and make sure nothing is out of alignment. Is the screw tight? Is the spring seated correctly? You might have to take the screws/washers out and reassemble it. Sometimes, something can get out of alignment in there if it wasn't reassembled correctly the last time someone maintained it. Just pay attention to how you took it apart so you'll know how it goes back. It's a very simple device, but it never hurts to make a note of how it looked before you took it apart...
And finally, what kind of battery are you using? You mentioned 'battery' (singular), so that leads me to believe you're using a 17670 (3.7v) battery. But anyway, not all
batteries are created equal. PS says that the AW protected 17670
batteries will fit for 3.7v, as will two Tenergy or AW RCR123 (LiFePO4)
batteries for 6v. All of these should fit using the standard configuration. But if you tried the previous two things to no avail, take the assembly back out and remove a washer (start by removing the lock washer). Then try again. The washers are there to adjust for battery lengths. So you can even remove both of them if necessary.