A USB port is just that. It can provide power to what is plugged into it. If the device draws 100 ma, then the USB port will give 100 ma. If the device wants 2 amps, it will do that. The charging circuits in your mods will try to draw from the USB port what it desires. If the USB port is being asked for 2 amps, but it can only provide 500 ma, then the device will only receive 500 ma. If charging a battery, it will simply take longer to charge; there is NO increased danger to the battery.
The only danger from trying to draw more from a USB port than it can provide is failure of the USB port. Kinda like your wall outlet. We plug things in them all day long that draw far less current than the outlet is capable of providing. My lights still work and the radio still plays. Even though I am giving them a bigger power source than they could ever need. But if I draw too much, the lights go out. Trips the breaker but does not cause failure of the light.
I use a USB port monitor between my wall adapter and mods. Some of my mods draw more current than others, as it shows on the amp display. I can watch the current draw from any mod while charging, from large to small. Even my stand alone charger can plug into the USB power source. Though it can provide more current than I need, there is no danger from charging a smaller device on it. The charging circuit inside the mod SHOULD regulate any current needed to charge the battery. It is NOT the responsibility or function of the USB power source to do this.
The only danger created by using a smaller USB power source than what is needed would be what would happen when the USB power source is overloaded. This may cause it to fail, and it may even cause it to overheat. What occurs during this failure is the real danger. If it fails and sends out more than 5 volts, it may cause the mod and its circuitry to fail. Then the problem occurs.
The best solution is to use the highest amp output USB source you can. That minimizes the chances of failure should you draw more current than it can supply.