I had the exact same situation and concerns when I started vaping. Not just me, but family members also commented that I seemed to be hitting the vape more then the cigs. So I went to simple math for a comparison (NOTE: This does not take into consideration the variances of actual nicotine absorption because, frankly, I don't really care). And the results are:
When I smoked that up to a pack a day, I only smoked outside....about 1 per hour. I actually kept track for a bit and found that I would take +/- 15 pulls on each smoke, or +/- 15 pulls per hour. Eventually, I got a mod with a puff counter and started keeping track. Some days going over 300 and others around 225. Imagine that! Pretty much like smoking, a pack one day and another only 15. That compared to the +/- 270 puffs on the cigarettes per day pretty much makes it a wash.
The only difference for me? Smoking equates to +/- 15 puffs in about 6 minutes. Vaping equates to +/- 15 puffs in an hour, or every 4 minutes making it appear as though one is actually continuously hitting the vape. But that every 4 minutes is just an average as I may take 2 or 3 puffs in a few minutes and then none for the next 15 or 20, which still appears continuously to us and others. I still step outside on occasion with my vape, but instead of 15 puffs, I take 3 or 4 in the 10 minutes I am out there.
To me, it pretty much comes down to don't sweat the small stuff. I'm still not smoking!