The output current rating on the USB power supply (the thing which has a plug on one side, a female USB on the other) should be greater than or equal to the input specified on the charger part (the thing with a USB male plug one one side, and a battery connector on the other).
If you have a PV charger which says input 5V 420 mA, it is not a problem to use it with a USB power supply that is rated, say, 5V 1000 mA output. A power supply just provides an available source of voltage at up to a given specified current -- it doesn't "push" the current at your charger, your charger "pulls" what it needs off the power supply.
What you don't want is for the power supply output number to be less than the charger input. In that case the charger wants more incoming power than the power supply can source; that is an over-stressing condition, mis-matched items.
Yes, you do want to make sure voltages match. The vast majority of standard proprietary PV chargers are designed for "normal" USB voltage, which is nominal 5VDC. If the vendor PV says to not plug their charger into a PC, or their "USB" power supply has an output voltage that is outside the range of around 4.8 - 5.2VDC, then I would not use it with a standard USB power supply, because then they may be using a USB connector but they are not operating within normal USB voltages.
On the PV charger side, if you use anything other than the manufacturer charger there are a couple thing to be careful about. 1) One item is to make sure that the charger output is not larger than the PV battery size. If you have a PV battery that is say 180 mAh (like an old fashioned small 510 battery), then you do not want to use it with a 420 mA output Riva charger -- the charger is putting out too much current for the 510 battery to take in properly. 2) Another is to know whether your PV or charger is one of the "oddball" ones with reverse polarity -- common practice is for the center post of the connector to be positive, while the other, threaded, part, is negative/return. Unfortunately there are several well-known cases where a vendor has chosen to flip their polarity, positive on the threaded part, negative on the center post. Even though something has a 510 connector, if the polarity is reversed it is NOT safe to mix-and-match a center-positive and a threaded-positive battery and charger. A cartomiser/atty doesn't know or care which part is positive, since it is just a resistive element, but the charger does care what the battery terminals are. 3) The third possible error in mixing chargers and PVs is a mismatch in charging voltages. It used to be that all of the "stock" PVs pretty much used the same battery chemistry, needed the same charger input voltage (4.2V or so). Unfortunately, nowadays it is possible to have a mismatch, as sometimes the battery is different. I've got a small, popular, new, all-in-one PV. It is a standard 510 connector device, standard center-positive, however the charger for it is marked output 5V 0.1 Amp. I like the device a lot, it performs well, but it is NOT appropriate to use its charger with another 510 connector device, since while the polarity is okay and the output mA is not high, the voltage is too high.
Ahh, the joys of product differentiation -- this used to be much simpler, not all that long ago, but with the increasing use of PVs there has been a parallel increase in product differences.