You can use insulating tape to replace a battery label. Make sure it goes over both ends though, use a good battery to model the coverage on. It needs some cutting with a craft knife to get the ends to overlap smoothly.
It won't be as durable as the original so you'll have to keep an eye on it.
If the APV won't accept a regular battery of the correct diameter without wrecking the label then it is faulty and must be returned. Tight internal dimensions or internal sharp edges are a manufacturing fault. In fact all
devices should come with enough ID clearance to fit an internal mylar sheath, as this would be a useful safety measure.
In some circumstances a negative battery body to negative
device body can add to a chain of events that results in an incident, depending on the fuse location/s. However if someone inserts the battery in reverse by mistake, there is an immediate elevation of risk as there can then be a direct neg to pos short circuit through a label tear. Guard against that because it is a simple mistake that can stack the risk factor a whole lot higher. And don't ever use a
device that was 'designed' to have the battery inserted with reverse polarity, it's one of the worst possible design mistakes.