Stacked on top of each other?
Assuming that you are speaking of 'in series', or ( charger - ) ( - cell 1 + ) ( - cell 2 + ) ( charger + ):
If you are using a charger designed to charge single cells independently, as most are designed, then absolutely not. You'd be, in effect, trying to charge a series 'battery pack' of 8.4v max using a 4.2v max charger, and the 'battery pack' would discharge to the charger while the charger is trying to charge the 'battery pack', because the charger is at a lower potential the the series cells are.
Since I've never had any experience with such a thing (simply because it is a bad idea), I can only use my electronics knowledge to theorize what would happen. A good charger should refuse to try and charge them, else I'd expect that they could severely overheat in the attempt, possibly damaging the charger circuit, and possibly physically damaging anything else in close proximity.
There are chargers designed to charge actual battery packs. They are used in many things. The manufacturers take precautions to make it as safe as is economically possible, such as adding a protection circuit for the entire pack. The charger is designed to charge the pack to the total series voltage of the cells, though.
I'd heartily recommend getting a multiple bay charger. One that is 'smart'. I'm pretty happy with the Nitecore I4 intellicharger, myself.