If a new battery is 3.8V or above I would use it before charging it. If it is above 3.8V you won't hurt it by charging it, though.
Despite industry-wide claims of 300 charge cycles before battery capacity is noticeably diminished, most of us are seeing Li-ions show some signs of capacity loss after 30-50 charge cycles. Still, standard cylindrical Li-ion batteries are a much more economical way to power an APV than battery/switch/connector combos.