Yes, your USB charger should work, although it will take a bit longer to charge them than a regular 510 batt takes. Your PCC and the USB both charge until the battery is full, so there should be no difference in the quality of charge between the two, although they may take different amounts of time to charge, since they may charge with a different current.
I like the XL 510 batts too! They were among my first
batteries I ever used, and I still use them even though I have other
mods. If you are as irritated with the obnoxious LED as I am, here is a trick: get a box of blank DSE901 carts. They have a black cap which fits exactly right on the 510 batt end, and make that big LED only glow
through a tiny hole...and it gives the batts a kind of techno look. Of course duct tape can also be used.