Yeah, there will be left over heat when you vape.. so if you go to town on that thing, heat from the coil will work it's way down to the battery.
Too much heat will stress the components and shorten battery life. On a larger device, like a Vamo that was mentioned... there's more stuff there to absorb and dissipate heat.. on the ego, you don't have much there.